[codex] add evm-only executor load test harness#3658
Draft
codchen wants to merge 4 commits into
Draft
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## codex/evmonly-staking-dynamic-gas #3658 +/- ##
=====================================================================
+ Coverage 58.15% 58.30% +0.15%
=====================================================================
Files 2185 2179 -6
Lines 178564 177862 -702
=====================================================================
- Hits 103836 103701 -135
+ Misses 65413 65028 -385
+ Partials 9315 9133 -182
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
bd57e4c to
7c6c44c
Compare
8e28929 to
4ec8da5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a standalone
evmonly-loadtestcommand that continuously feeds generated EVM transfer blocks into the EVM-only executor with generated in-memory genesis state, discard state/receipt sinks, and Prometheus/stdout throughput metrics.The harness is intended for optimistic executor saturation testing with one external executor worker and internal OCC workers. It defaults recipients to unique addresses and keeps the workload structure isolated so future ERC20-style transactions can be added beside the transfer generator.
This also adds
--prebuild-blocks, which generates all bounded blocks before starting executor workers. That mode separates block-generation throughput from executor throughput for diagnosing whether builders are competing with execution.The current branch also freezes generated prebuilt genesis state before execution so the synthetic
StateReadercan serve immutable state without lock/copy overhead during the executor-only measurement.Adds OCC diagnostics for future ERC20-style load tests: block results now report OCC attempts, sequential fallbacks, fallback reasons, and aggregated conflict keys. The loadtest exports Prometheus counters for attempts/fallbacks/conflicts, fallback reasons, and conflict keys labeled by access/kind/address/slot; stdout totals include
occ_attempts,occ_fallbacks, andocc_conflicts.Validation
go test ./giga/evmonly/...go run ./giga/evmonly/cmd/evmonly-loadtest --metrics-addr= --report-interval=0 --prebuild-blocks --blocks=20 --txs-per-block=1000 --builders=16 --workers=1 --executor-workers=12 --gas-price-wei=0 --min-gas-price-wei=0 --queue-size=64EC2 benchmark
The 199k/202k EC2 benchmark numbers below were run from commit
4ec8da52con ac8i.48xlargeinus-east-1awith SMT disabled usingCoreCount=96,ThreadsPerCore=1, Go1.25.6, one external executor worker, prebuilt blocks, unique senders/recipients, and zero gas price/min gas price. The current head71eba7a0fadds metrics only on top of that benchmarked path.Reproduce the 199.3k TPS run
Launch the instance with CPU options equivalent to:
Build the loadtest binary on the instance:
curl -fsSL https://go.dev/dl/go1.25.6.linux-amd64.tar.gz -o /tmp/go1.25.6.linux-amd64.tar.gz sudo rm -rf /usr/local/go sudo tar -C /usr/local -xzf /tmp/go1.25.6.linux-amd64.tar.gz git clone --branch codex/evm-only-executor-loadtest https://github.com/sei-protocol/sei-chain.git cd sei-chain /usr/local/go/bin/go build -o /tmp/evmonly-loadtest ./giga/evmonly/cmd/evmonly-loadtestRun the benchmark:
GOMAXPROCS=96 GOGC=200 /tmp/evmonly-loadtest \ --metrics-addr "" \ --blocks 1000 \ --txs-per-block 5000 \ --prebuild-blocks \ --builders 96 \ --workers 1 \ --executor-workers 96 \ --gas-price-wei 0 \ --min-gas-price-wei 0 \ --report-interval 5sObserved output:
For comparison, the same shape and command with
GOGC=400crossed the 200k target:Other comparisons from the same testing round:
c8i.48xlargewith default SMT topped out around171kTPS on the 2M-tx sweep.c8i.48xlargewith SMT disabled and default GC reached about191kTPS over 5M tx.c7a.48xlargetopped out around161kTPS on the 2M-tx sweep.10k tx/blockblocks regressed to about184kTPS over 5M tx.