BAL cache lineage (PR #6 of the perf stack)#20893
Open
mh0lt wants to merge 126 commits into
Open
Conversation
mh0lt
requested review from
AskAlexSharov,
Giulio2002,
anacrolix,
canepat,
lupin012 and
taratorio
as code owners
mh0lt
added a commit
that referenced
this pull request
May 5, 2026
…atch bug (#20893) (#21003) Cherry-pick of the BAL diagnostic tooling (bal-scan / bal-test / debug_getRawBlockAccessList RPC) from #20893. Useful for the eth/71 BAL exchange testing on bal-devnet-3 (dump/delete/refetch/compare workflow). The server-dispatch wire-up half of #20893 is intentionally NOT cherry-picked — it's already on bal-devnet-3 via #20881.
yperbasis
requested changes
May 8, 2026
yperbasis
left a comment
yperbasis
left a comment
Member
There was a problem hiding this comment.
Title and description don't match the actual changes
Pure refactor — behavior preserved. Splits the encoded-branch→cells decode logic out of HexPatriciaHashed.unfoldBranchNode into a free function DecodeBranchInto so the same code is consumed by: - unfoldBranchNode (existing trie unfold path) - future cache populators (decoded-payload BranchCache) - future parallel pre-unfold orchestrator (Stage E) Today these would each have to re-derive the encoded-branch parsing logic. Centralising it ensures one decoder, one set of edge cases, one place to fix any bug in the on-disk format handling. DecodeBranchInto is intentionally PURE — it does not call deriveHashedKeys. Trie callers (which need hashed keys for the fold state machine) follow the decode with their own keccak loop. Cache callers can skip the derive step entirely until the cell is consumed by the trie. Tests: - TestDecodeBranchInto_RoundTrip: BranchEncoder.EncodeBranch produces bytes that DecodeBranchInto recovers cell-for-cell. Property test that keeps the canonical decoder consistent with the canonical encoder. - TestDecodeBranchInto_DeletedFlag: touchMap/afterMap convention with the deleted parameter. - TestDecodeBranchInto_TruncatedInput: clean errors on truncated input (no panic). Plus the existing commitment test suite (incl. trie-mismatch tests in TestBranchData_*) all pass without modification, confirming the refactor preserves unfoldBranchNode's behavior. This is the foundation for the next refactors in the representation-reduction track (see agentspecs/trie-data-pipeline-complexity-tax.md): subsequent PRs will introduce a decoded-payload cache that reads through this same decoder, and will lift unfoldKeyPath as a per-key traversal primitive that the warmer + future Stage E both consume. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces a new BranchCache type, distinct from WarmupCache and
designed for the longer-lived caching the cross-block persistence
work needs (step 7 of the representation-reduction sequence).
Distinguishing characteristics vs WarmupCache:
- Bounded LRU tail with configurable capacity (vs WarmupCache's
unbounded map). Suitable for caches that outlive a single Process
without unbounded memory growth.
- Single pinned slot for the root branch (compact prefix [0x00]).
Root never evicts. Atomic-pointer load/store on the hot read
path, no lock involved.
- dirty-flag + PutIfClean invariants — same semantics as the
invariants added to WarmupCache in the previous commit. Lets
cross-block writers race safely with fold updates.
- Lazy GetDecoded — same lazy-decode pattern as WarmupCache's
GetBranchDecoded; cells populated on first decoded-read and
cached for subsequent reads.
NOT yet wired into the trie's read or write paths. This commit just
adds the type, with tests. The trie integration (where this cache
plugs into branchFromCacheOrDB and the encoder's PutBranch) is the
discussion point at the step 6 boundary — see the conversation
captured at this point in the representation-reduction sequence.
Today the cache is intended to be ephemeral (per-Process,
constructed alongside the trie, dies with it). Step 7 lifts the
lifetime to the aggTx level for cross-block persistence; the cache
shape (bounded LRU + pinned root + dirty-flag) is in place ahead
of that.
Tests:
- TestBranchCache_RootPinning: root branch lands in pinned slot,
deep branches land in LRU tail; per-tier hit counters update
independently.
- TestBranchCache_RootSurvivesEvictionPressure: root persists when
tail is overfilled past capacity.
- TestBranchCache_DirtyFlag: PutIfClean refuses dirty entry,
unconditional Put replaces and clears dirty.
- TestBranchCache_GetDecoded: lazy-decode round-trip with
BranchEncoder; cells pointer reused across reads.
- TestBranchCache_Invalidate: removes from both tiers.
- TestBranchCache_Clear: empties both tiers, resets stats.
- TestBranchCache_Stats: deterministic format with per-tier counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Doc-only addition to BranchCache's package-level comment, capturing
the caller invariants the cache assumes and the conditions under
which the existing concurrent trie satisfies them.
Three caller invariants:
1. Single writer per prefix at any moment.
2. Mark-dirty-then-Put discipline for racing writers.
3. Decoded cells from GetDecoded are read-only (alias entry storage).
The current ConcurrentPatriciaHashed satisfies all three by
construction:
- Mounts partition by first nibble (disjoint prefix spaces, no
cross-mount writes to the same key).
- Root branch written by single sequential fold post-Wait.
- Mount→root grid roll-up is rootMu-protected (in-memory grid
only, separate from cache writes).
Doc explicitly flags that any future parallel fold redesign (Stage F
in agentspecs/stage-e-pre-unfold-design.md) MUST preserve these
invariants — particularly the single-writer-per-prefix one, which
breaks if parent branches are written incrementally as children
complete in parallel. The required coordination layer goes at the
orchestrator (per-parent atomic counter; only the last-decrementer
writes the parent), NOT inside the cache. The cache's existing
primitives (atomic dirty flag, thread-safe LRU, atomic root pointer)
are sufficient for that orchestrator to build on.
Motivation: Stage F is likely deferred because the bench data shows
fold isn't the bottleneck for the canonical SSTORE-bloat workload.
But adding the constraint to the cache later (after caching is in
production) is much harder than documenting it now — correctness
regressions from a missed coordination layer can hide for many
blocks. Documenting the contract on the cache itself ensures any
engineer touching parallel fold sees it.
No code change. Doc-only. All tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 7a of the representation-reduction sequence: per-Process
integration of the BranchCache type added in the previous commit.
Plumbing:
- Trie interface gains SetBranchCache(*BranchCache).
HexPatriciaHashed implementation propagates to its branchEncoder.
ConcurrentPatriciaHashed implementation propagates the SAME
instance to root + all 16 mounts (sharing one cache is correct
under the concurrency contract — mounts partition prefix space
by first nibble, so cross-mount writes target distinct keys).
- InitializeTrieAndUpdates constructs a new BranchCache(default)
per trie instance and attaches it. Lifetime today = trie
lifetime = per-Process. Future cross-block persistence work
(step 7b) lifts this to aggTx scope by constructing the cache
one layer up and passing it in.
Read path (HPH.branchFromCacheOrDB):
L1 WarmupCache (existing) → L2 BranchCache (new) → L3 ctx.Branch.
L3 hits with non-empty result populate L2 so subsequent reads hit
L2 within the cache's lifetime. L1 stays first because warmup
workers may have pre-fetched with prefix-walk-derived freshness.
Write path (BranchEncoder.CollectUpdate):
- MarkDirty(prefix) BEFORE encode work — protects against
concurrent warmup-style writers racing into PutIfClean during
the encode (race documented in the cache's Concurrency Contract).
- Put(prefixCopy, updateCopy) AFTER ctx.PutBranch succeeds —
replaces the dirty entry with fresh canonical bytes. Single
writer per prefix per fold step (current sequential fold +
first-nibble mount partitioning) means no race on this Put.
Lifecycle:
HPH.Reset clears the BranchCache when called from the root trie
(gated by !hph.mounted). Mounted subtries share the root's cache,
so a mount calling Clear would dump entries the root still wants.
Carries the invariant from PR #19954 commit 1612d56.
Today's expected performance impact: minimal. Per-Process lifetime
means cache is empty at Process start, so first reads always miss.
The cache helps only branches that are read multiple times within
ONE Process — uncommon in current code paths. Step 7b is where
the real perf swing comes from (cross-block persistence so block
N reads hit branches written by block N-1).
This commit is the safe stepping stone: it validates the wire-up
end-to-end (read path + write path + concurrency contract +
lifecycle) without changing perf characteristics. Bench should
match Run I baseline (7.16 mgas/s on canonical SSTORE-bloat block).
All commitment tests pass, lint clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the BranchCache was constructed inside InitializeTrieAndUpdates, giving it per-SharedDomains lifetime. SharedDomains is reconstructed for every batch / many tx boundaries — verified with logs in the prototype (921 fresh BranchCache constructions per bench run) — so the cache started cold every batch and never delivered the cross-block hits the design is about. Per-Process scope kept cache-cleanup correctness simple but defeated the whole point of caching. Place the cache on the commitment Domain struct (one cache per aggregator, matching the pattern on add_execution_context_with_caches where each domain owns its valueCache). ConfigureDomains attaches the cache once after domains are initialised — idempotent, lifetime = aggregator lifetime. AggregatorRoTx exposes BranchCache() returning the commitment domain's cache, so SharedDomains' construction path can fetch it without forcing db/state/execctx to import db/state (db/state already imports execctx via squeeze.go, so the reverse import would create a cycle). The placeholder commitment.BranchCacheProvider interface lets the SD construction path use a duck-typed type assertion on tx.AggTx() any. Plumb the cache through NewSharedDomainsCommitmentContext into InitializeTrieAndUpdates as an explicit parameter; nil falls back to a fresh per-init cache so test helpers without an aggregator still get a valid cache. Reset behaviour: HexPatriciaHashed.Reset no longer calls Clear on the cache. Aggregator-scope persistence requires the cache to survive Reset between commitment calculations. Callers that genuinely need to invalidate the cache (unwind, fork validation) now call ClearBranchCache explicitly. The bench is forward-only so this is a safe change for measurement; an explicit unwind clear path can land in a follow-up commit when needed.
Adds two debug-gated diagnostics for cross-block-cache-lifetime
investigations. Both off by default — meant to be flipped on when a
correctness regression surfaces and you need to localise *where in the
fold path* the cache started lying, instead of waiting for a downstream
trie-root-mismatch many blocks later.
1. BRANCH_CACHE_VERIFY: branchFromCacheOrDB cross-checks every L2
(BranchCache) hit against ctx.Branch and increments a divergence
counter when bytes disagree. Logs the prefix and both byte forms so
the first divergent read shows up directly in the erigon log.
BranchCache.VerifyDivergences() exposes the count for assertion in
tests.
2. BRANCH_CACHE_FINGERPRINT: SharedDomainsCommitmentContext.ComputeCommitment
emits a "[cache-fp]" log at end of every compute with (block, root,
cache fingerprint, divergence count). Two builds running the same
workload can be diffed offline ("first block at which their fp's
differ") to nail down the first block where lifecycle invariants
diverged. Fingerprint is an order-independent FNV-1a fold over
(key-hash, data-hash) pairs across both root and tail tiers.
Adds maphash.LRU.Range so the Fingerprint can iterate the LRU tail
without touching recency (Peek under the hood). The wrapper otherwise
discards original byte-keys on insert; mixing by hash instead of key is
correctness-equivalent up to hash collision, which is acceptable at the
working-set sizes here.
Motivation: today we just chased a wrong-root regression to commit
d673052 (aggregator-scope cache lifetime). Took two full bench cycles
(~12 min) to localise. With the divergence detector running, the first
divergent read would surface in the erigon log within seconds. With
fingerprint logging in two builds (one good, one regressed), the
diverging block boundary would be a single grep across two log files.
The deferred-encoding path (CollectDeferredUpdate + ApplyDeferredUpdates)
parallelises EncodeBranch + merge work at apply time. The cache
correctness consequence: between collect and apply, sd.mem holds the
old state while the cache is also unchanged. When apply finally fires
(end of Process or duplicate-prefix flush mid-Process), sd.mem
advances but the cache isn't touched — CollectDeferredUpdate doesn't
have a cache hook (and wiring one breaks
TestSharedDomain_RepeatedUnwindAcrossStepBoundary +
TestCustomTraceReceiptDomain because it violates an implicit
trie-during-Process cache stability invariant).
The result on the FV bench: cache stays at the very-first-Process's
read-side L3 fallback bytes for prefixes the trie writes, while
ctx.Branch advances to each block's actual state. Divergence on every
hot prefix (root, root-zone children) starting from block 2.
Use CollectUpdate (inline) instead. CollectUpdate writes
sd.mem + WarmupCache + BranchCache atomically at fold time via
PutBranch — cache mirrors sd.mem at every write, the trie sees its own
writes consistently, and the cross-Process cache state matches what
post-FCU MDBX commit produced. Loses the encoder's parallel-encoding
optimisation, but bench profile is I/O-bound, not encode-CPU-bound, so
the trade is favourable.
History (ETL) writes are still inline via DomainPut. Splitting that
("sd.mem inline, history queued for flush at FCU") is the proper
architectural answer to defer the slow disk work without touching the
sd.mem invariant — tracked as a follow-up.
Bisection helper: force branchFromCacheOrDB to skip the L2 BranchCache read path entirely so every read goes via ctx.Branch (sd.mem → MDBX). Cache writes still fire so verify-mode can keep comparing cache vs canonical. Flipping the env at runtime distinguishes "cache holds bad data" (bench passes further with cache reads off) from "deeper compute bug" (same failure regardless). Used in the 2026-05-06 investigation to confirm the cache was actively corrupting block 13's compute on the canonical SSTORE-bloat bench: with cache reads on, wrong-trie-root at block 13. With reads off, blocks 13-16 produce the correct roots and the bench advances to a separate failure at block 17 (unrelated pre-existing bug). Default off; gate via env DISABLE_BRANCH_CACHE_READS=true.
When verifyBranchCache=true and a cache hit disagrees with ctx.Branch, sample sd.mem, sd.parent.mem, and tx-direct (MDBX) for the same prefix and dump all layers in the divergence log line. Comparing those four byte sequences against the cached and canonical bytes pinpoints which state layer holds the bytes the cache disagrees with — the rewriter we need to identify before fixing the canonical-store-divergence bugs the cache currently exposes. Decision matrix (read off the log line): - cache != tx, sd.mem == cache → in-memory writer is fresh, MDBX is stale (commit-timing issue). - cache != tx, tx == ctx.Branch, sd.mem != cache, parent.mem != cache → MDBX has been rewritten by something outside the CollectUpdate write path (collation, file build, squeeze). - cache != ctx.Branch, parent.mem matches ctx.Branch but sd.mem doesn't → parent merge is the source. - cache != ctx.Branch, all of sd.mem / parent.mem / tx == ctx.Branch → cache itself was populated incorrectly (write-side bug). Three changes: 1. SharedDomains.ProbeReadLayers (db/state/execctx/domain_shared.go): public method that samples sd.mem, sd.parent.mem (private field accessed from the same package), and tx.GetLatest. Read-only; copies bytes so callers can hold them past tx lifetime. 2. TrieContext (execution/commitment/commitmentdb/commitment_context.go): add probeSd + probeTx fields populated at trieContext() construction; expose ProbeStateLayers method that delegates to sd.ProbeReadLayers. The local `sd` interface gets the ProbeReadLayers method too so the duck-typed reference can call it without an import cycle to execctx. 3. branchFromCacheOrDB log site (execution/commitment/hex_patricia_hashed.go): on divergence with verifyBranchCache, type-assert ctx for the probe interface and append sd_mem / parent_mem / mdbx fields to the log line. Existing field shape preserved for log parsers; new fields are additive. Pre-existing test failures (TestSharedDomain_RepeatedUnwindAcrossStepBoundary, TestValidateChainAndUpdateForkChoiceWithSideForksThatGoBackAndForwardInHeight) are unchanged — they were failing on the stack before this probe landed and are part of what the divergence work is meant to localise.
Per-write provenance for divergence-detection diagnostics. When a
divergence fires (cache hit disagrees with ctx.Branch), we now log
which write site produced the cached bytes and when, so we can
correlate the bad write against the FCU / build / step timeline.
Tag fields added to branchCacheEntry:
- origin short label of the write site (e.g. "CollectUpdate",
"L3-fallback-read")
- writeSeq monotonic counter per BranchCache instance
- writeTimeNanos unix nanos at write time
Put / PutIfClean signatures take an origin string. Two writers
updated:
- BranchEncoder.CollectUpdate → "CollectUpdate"
- branchFromCacheOrDB L3-fallback Put → "L3-fallback-read"
GetWithOrigin returns bytes plus the metadata; uses a non-counting
peek so it can be called alongside Get without double-counting hits.
The divergence-detection log site at branchFromCacheOrDB now appends
cache_origin / cache_seq / cache_t_ns fields. Combine with the
existing sd_mem / parent_mem / mdbx fields to localise both who wrote
the stale bytes and which layer the canonical value lives in.
Pre-existing test failures
(TestValidateChainAndUpdateForkChoiceWithSideForksThatGoBackAndForwardInHeight)
are unchanged from previous commits.
BranchCache previously sat in front of the sd.mem -> parent.mem -> MDBX
read chain (consulted in branchFromCacheOrDB before ctx.Branch). The
shared aggregator-scope cache was written from CollectUpdate by every
SD running commitment compute, including fork-validator SDs whose
writes never reach MDBX. Origin-tagged probe (run-step7b-probe-sdid)
showed five distinct SD pointers writing the same prefix to a single
cache entry, so any reader whose lineage didn't match the most-recent
writer saw bytes that disagreed with MDBX -> wrong trie root from
block 13 onward in the canonical SSTORE-bloat fork bench.
Layering after this change:
Read: sd.mem -> sd.parent.mem -> branchCache -> aggTx (MDBX)
Write: sd.mem only (DomainPut path)
Flush: sd.mem -> MDBX, then branchCache.Clear()
The cache now mirrors MDBX-flushed bytes only. Writers' in-flight bytes
live in sd.mem above; cache hits below sd.mem are always equivalent to
reading MDBX, so cross-SD pollution is impossible by construction.
Cache fills lazily on the MDBX-read path inside sd.GetLatest, and
clear-on-flush prevents pre-flush bytes from coexisting with new MDBX
state. Per-key invalidation is a follow-up (PR2).
BranchCache entries gain a step field so Get returns (data, step, ok)
matching the aggTx contract. Without this, sd.GetLatest's cache hit
returned step=0 and CheckDataAvailable rejected the boot SeekCommitment
with "commitment state out of date".
Removed:
- cache.Put from CollectUpdate (commitment.go)
- cache.Put + divergence detection from branchFromCacheOrDB
(hex_patricia_hashed.go); now just calls ctx.Branch
- L3-fallback Put (cache fills via sd.GetLatest now)
Validated on canonical cold bench (run-step8b): first FCU VALID, all
payloads through end VALID, 0 cache divergences, 0 wrong-root errors.
Prior probe bench had 23 divergences and INVALID payloads from the
fail block onward.
Probe scaffolding (SiteIdentity, ProbeStateLayers, divergence counters)
left in place for now; can be stripped in a cleanup follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ache state Update the BranchCache type comment to reflect the architectural state after the WarmupCache consolidation (steps 2a-2c, 3, 4): - BranchCache is the single branch cache (WarmupCache deleted) - Aggregator-scope lifetime, plumbed via BranchCacheProvider - Passive store: cache itself never reaches into underlying state - Branch warmer is branch-scoped; no leaf-data prefetch - Block-processing trie walker takes Updates from executor + memoization for siblings — no prefetch needed - Witness / proof generation walker drives its own state reads; if that path turns out to be cold-bound it indicates a need for separate account/storage caches (treat as separate concern with different scope/lifetime/invalidation; do not regrow the branch warmer to cover it) - disk_sto / disk_acc counters on cache-fp log surface any unexpected fall-through to ctx.Account / ctx.Storage as a signal of memoization gap or missing walker-side prefetch Doc-only; no behaviour change.
Adds a third cache tier between the root-pin slot and the LRU tail: a per-prefix pinned map fed by PinEntry. Pinned entries: - Never evict (no LRU pressure on this tier). - Are checked in lookup before the LRU tail, after the root slot. - Survive Put/SD.Flush updates: a Put for a pinned prefix updates the pinned entry in place rather than displacing it. Cross-block correctness via the existing dirty-flag invalidation discipline is preserved — the new bytes land in the same pinned slot. - Carry the same metadata as Put-tier entries (step, origin, writeSeq, writeTimeNanos) so divergence-detection and Stats treat them uniformly. Sized by the preload policy. Intended consumer is the storage trunk preload for big contracts (the 'storage root trunk cache for big accounts' direction): per-contract trunk branches at depth 65-70 get pinned at SD/cache creation, persist for the cache's lifetime. PinEntry is the public API; pinnedHits/pinnedMisses atomic counters are the new stats. PinnedCount() exposes current size for observability. Step 2 of the storage-trunk pin prototype.
Adds a function to pre-pin commitment branches for a given contract's storage subtree. Walks the trie depth-by-depth from depth 64 (storage subtree root) down to maxDepth: reads each branch via the supplied CommitmentReader, pins it via BranchCache.PinEntry, decodes the child bitmap, and recurses only into children that actually exist (no blind 16-way probing). contractHash is keccak256(address); the trunk lives at the prefix corresponding to the first 64 nibbles of the storage path. For a dense storage subtree (the SSTORE-bloat workload's bloat contract), expected pin count is ~16 + 256 + 4096 ≈ 4.4 K branches at depth 65/66/67, plus the root at depth 64. Sparse subtrees produce fewer. Loading strategy: per-prefix lookups via the reader. Simplest correct implementation. A bulk seg.Getter range-scan over sorted .kv would amortize disk seeks (per the parity-cluster observation in the consolidation memo) but requires building a prefix-range API on top of recsplit; defer until per-prefix lookup is shown to bottleneck. Step 3 of the storage-trunk pin prototype.
Adds a SharedDomains constructor hook that, when PIN_CONTRACT_TRUNKS is set, fires a one-shot background goroutine to preload the storage-subtree-trunk of each listed contract into BranchCache's pinned tier. Format: comma-separated list of 64-hex-char contract hashes (each is keccak256(addr)). Mechanism: - BranchCache.TryClaimPreload (atomic CAS) ensures the goroutine fires exactly once per cache lifetime, even though many SDs may be constructed (per-tx instances etc.). - Goroutine wraps sd.GetLatest as a CommitmentReader and calls commitment.PreloadContractTrunk for each contract hash, depth 64-70. - Logs progress per contract on completion. Closure-over-(sd, tx) is the prototype shape — works for the bench (both live for the whole process). Production deployment needs to revisit the lifetime — sd's tx may not outlive the goroutine. Step 4 of the storage-trunk pin prototype. Bench measurement is the next step (commit 5).
Previous async-goroutine shape (d204c1b) shared the SD's MDBX tx with the calling thread. Concurrent cursor use under the same tx tripped Go's cgo-pointer-pinning runtime check: panic: runtime error: cgo argument has Go pointer to unpinned Go pointer surfacing in an unrelated PruneBlocks goroutine during boot. Make the preload synchronous in the SD constructor for now: same TryClaimPreload guard (fires once per cache lifetime), but no goroutine. Boot pays the per-contract preload time as a one-off. Background-with-own-tx is the proper shape and remains a follow-up; owning the SD's tx exclusively for the preload duration is the safe shape until that lands.
… cap The previous bench (run-pin-trunk-instrumented-cold-cgroup-191347) hung at SD construction with no [trunk-preload] log lines for 5+ minutes. Erigon never reached "engine RPC ready" so all blocks came in as SYNCING. Two changes to localise + bound: 1. **Localisation**: add INFO logs at triggerTrunkPreload entry, per-contract starting/done with took, and a 500-prefix progress log inside PreloadContractTrunk. Whatever it does (or hangs on) is now observable. 2. **Bound**: cap PreloadContractTrunk at 10000 branches (vs ~4.4K expected for a saturated 4-level subtree at maxDepth=67). Drops maxDepth from 70 → 67 in the trigger (depth 64-67 = 16+256+4096 max branches) so we don't recurse into the per-slot tail where pinning has no value. Preload fails-fast on pathological subtrees rather than blocking SD construction indefinitely.
The previous shape (d204c1b) ran triggerTrunkPreload BEFORE sd.SeekCommitment in NewSharedDomains. Bench result: when the preload fired (PIN_CONTRACT_TRUNKS set), every subsequent block came back SYNCING — engine kept attempting backward-download which fails on this peerless setup, no block ever validated, no cache-fp ever fired. Without the preload firing, the same binary works fine (verify-bench PASS at 3.26s). Hypothesis (untested but matches the symptom): preload's sd.GetLatest reads ran before SeekCommitment had resolved the SD's view of the chain head. Pinned values were therefore inconsistent with the committed state, and the trie compute on the first block got wrong root → SYNCING → backward-download → no peers → death spiral with no Flush ever updating the (stale) pinned entries. Fix is mechanical: move the preload call to after SeekCommitment. The TryClaimPreload guard still ensures fire-once-per-cache lifetime. If subsequent bench shows pin_count > 0 + pin_hit > 0 + blocks validating normally, the hypothesis is confirmed; if SYNCING repeats, the bug is something else and we need to revert and debug differently.
…ParaTrieDB Previous prototype iterations both broke block validation: 1. Async sharing the SD's MDBX tx (d204c1b) → cgo "unpinned Go pointer" panic from concurrent cursor use. 2. Synchronous from NewSharedDomains (5a81976 / 4c9ead456d) → blocked the engine HTTP handler for ~3-4s during the preload window, causing the bench's first NewPayload to be dropped. Confirmed: the bench's height=24358001 is ABSENT from the erigon log; the next received block (24358002) then fails backward-download (no peers) → SYNCING forever. Restructure: - Move trigger from NewSharedDomains to EnableParaTrieDB. The latter is called from the staged-sync exec-stage init, NOT from request handlers, AND has access to a kv.TemporalRoDB. - triggerTrunkPreload now takes the DB (not a tx) and spawns a goroutine that opens its OWN tx via db.BeginTemporalRo. No shared cursors with the main pipeline; no blocking the engine. - Reader uses tx.GetLatest directly (not sd.GetLatest) — the SD layering would re-introduce shared-state risk and isn't needed (pinned bytes don't depend on sd.mem state). Same TryClaimPreload guard ensures the preload fires once per BranchCache lifetime regardless of how many SDs construct. If this works the bench should: - Show [trunk-preload] log lines firing once - Pin ~4369 branches - TEST block cache-fp shows pin_hit > 0 and files_comm < 1K - All blocks validate normally (no SYNCING failure)
Make the trunk-pin maxDepth configurable via env (default 67) so we can sweep depths to find the memory/perf sweet spot without rebuilding. Bump the per-contract maxBranches cap from 10K to 200K so deeper saturated subtrees don't get truncated mid-walk.
mh0lt
force-pushed
the
fix/eth71-multiclient-server-dispatch
branch
from
708a4a8 to
b2dfc0b
Compare
…nchCache + stateCache The optimization stack and the cache branch had grown two divergent FlushWithCallback methods — one single-domain for the BranchCache (commitment), one all-domains for the StateCache flush-only fix. Merge them into one. FlushWithCallback is now all-domains: it invokes cb for every (domain, key, latest-value, step) across all domains (sd.domains + sd.storage), then drains the mem-batch — callback, MDBX flush and drain in one latestStateLock window. SharedDomains.Flush passes a single callback that routes CommitmentDomain → BranchCache and Accounts/Storage/Code → StateCache. The per-write stateCache.Put/Delete in domainPut/DomainDel are removed: a write is in-flight, fork-specific state living in sd.mem; mirroring it into the process-wide cache let a sibling fork's re-execution read another fork's uncommitted bytes. The cache is now refreshed only on flush, so it mirrors committed, fork-agnostic state. drainLocked empties sd.mem as part of the flush so a child SD chained as parent reads through to the refreshed cache / DB instead of stale bytes. This folds the cache fix (was 527cb23077 on the cache branch) into the optimization stack so the local group-test exercises the final code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… eth/71
Introduces a 3-tier lookup for serving EIP-7928 Block Access Lists to
eth/71 GetBlockAccessLists peers:
1. In-memory LRU cache (recent blocks, freshly produced or recently
served). 100-entry window keyed on block hash.
2. Chaindata DB (BALs the eth/71 bal-downloader fetched from peers, or
legacy persisted BALs). DB hits are promoted to the cache so repeat
requests don't re-read MDBX.
3. Installed BALRegenerator backend that re-executes the block against
its parent state to derive the BAL. Result is cached before return.
The eth/71 handler (AnswerGetBlockAccessListsQuery) and the sentry
dispatch path now thread a context through BlockAccessListBytes, so
regeneration respects the peer's read deadline.
BALRegenerator implementation (execution/execmodule/balregen.go) uses a
simple IBS path: parent-state reader → IntraBlockState with VersionMap
enabled (read tracking) → InitializeBlockExecution → ApplyTransaction
loop → merge ibs.TxIO() into a per-block VersionedIO → AsBlockAccessList()
→ EncodeBlockAccessListBytes. No parallel-exec dependency, matching the
block-assembler pattern (execution/exec/block_assembler.go).
The Finalize step is intentionally omitted from the re-exec loop —
fork-specific engine.Finalize signatures vary and the finalize-time BAL
deltas (system contracts, withdrawals) are typically captured during
InitializeBlockExecution. If a downstream peer's hash verification flags
a mismatch on Finalize-touched accounts, that's the signal to wire
fork-aware finalize support here.
Cache + lookup tests:
- db/rawdb/balcache_test.go: 9 cases covering cache-hit short-circuit,
DB-hit-promotes-to-cache, regenerator-fallback, no-regenerator path,
nil-regeneration-not-cached, regenerator-error-not-cached, empty
data is no-op, defensive byte-copy, SetBALRegenerator replace+clear.
Handler tests:
- p2p/protocols/eth/handlers_test.go: existing tests updated to thread
ctx and reset the cache between runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…enerator fallback
The BAL was being written to MDBX on the NewPayload critical path — a
multi-hundred-KB Put per block that, on a churned DB, took tens of seconds.
This commit removes the persistent storage entirely:
- BlockAccessListBytes is now a 2-tier lookup: in-memory LRU cache →
installed BALRegenerator (re-executes the block to derive the BAL).
The chaindata-DB tier is gone.
- execmodule.InsertBlocks caches the sidecar BAL (was MDBX Put on the
blockOverlay) — same call-site, cache-only.
- bal-downloader caches fetched BALs (was MDBX rwDB.Update). The
backward-scan dedup check now consults the cache too.
Callers updated to use the cache-aware lookup (cache → regenerator):
- execution/stagedsync/exec3.go: BAL validation in parallel exec path
- execution/stagedsync/bal_create.go: ProcessBAL validator cross-check
- execution/execmodule/getters.go: GetPayloadBodiesByHash/Range RPC
- rpc/jsonrpc/eth_block_access_list.go: eth_getBlockAccessList RPC
The eth/71 server-side handler (AnswerGetBlockAccessListsQuery) stays
cache → regenerator only — explicitly NOT a peer-fetch relay, to avoid
amplification of remote BAL requests.
Tests:
- db/rawdb/balcache_test.go: dropped the now-dead DBHitPromotesToCache
test; the rest stay identical except for the dropped tx argument.
- p2p/protocols/eth/handlers_test.go: test fixtures use
rawdb.CacheBlockAccessList instead of WriteBlockAccessListBytes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…; wire at startup balregen.go now uses transactions.ComputeBlockContext to set up the IBS rooted at the parent state — same machinery the RPC tracing / receipts generation paths use, so the BAL replay sees the same read-tracking layout the BAL hash assumes. The previous NewParentStateReader factory abstraction is gone; deps shrink to (DB, ChainConfig, Engine, BlockReader, TxNumsReader, Logger). node/eth/backend.go installs the regenerator via rawdb.SetBALRegenerator right after the exec module is constructed. From this point on, BlockAccessListBytes lookup of a cache-miss block will re-execute it to produce the BAL — used by eth/71 GetBlockAccessLists serving, eth_getBlockAccessList RPC, and the engine GetPayloadBodiesByHash/Range RPCs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ng block sync
When the engine_block_downloader pulls blocks backwards from peers (catch-up
sync, segment recovery), each batch is now followed by a non-blocking
eth/71 GetBlockAccessLists request for those blocks. Hash-verified BALs
populate the in-memory cache so the subsequent exec stage avoids local
re-execution to derive them.
- db/rawdb/balcache.go: new BALSyncFetcher interface +
SetBALSyncFetcher / GetBALSyncFetcher. Mirrors the BALRegenerator
plumbing (atomic.Pointer slot, nil-safe).
- p2p/sentry/sentry_multi_client/bal_downloader.go: BALDownloader
implements BALSyncFetcher.FetchBALs — picks an eth/71 peer, batches
the request, caches every hash-validated response. Non-blocking by
intent; failures fall through to the BALRegenerator on later lookup.
- node/eth/backend.go: SetBALSyncFetcher(balDownloader) at startup
alongside the existing background scan.
- execution/engineapi/engine_block_downloader/block_downloader.go: after
each feed.Next batch, kick a goroutine to FetchBALs for the batch's
Amsterdam blocks (BlockAccessListHash != nil).
engine_newPayload is intentionally NOT updated: post-Amsterdam the
Engine API spec requires the CL to include the BAL bytes in the payload;
a missing BAL is rejected as InvalidParamsError. Catch-up sync (where
blocks arrive without BAL bodies) is covered by the
engine_block_downloader hook above.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cache is purely in-memory — no MDBX writes, no MDBX reads. Living under db/rawdb was a misnomer left over from the original implementation. New location is execution/balcache, neighbouring the types.BlockAccessList producers. No API renames; consumers swap rawdb.* → balcache.* prefixes. Drops the now-unused db/rawdb import from every consumer that only reached the package for the cache symbols. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… via balcache → cache.StateCache The BlockReadAheader's BAL prefetch was sourcing the access list from MDBX (kv.BlockAccessList table). On the perf-devnet-3 / bench-feeder path the BAL is never persisted to chaindata — it arrives via engine_newPayloadV5 and is held only in the payload. Diagnostic logging confirmed: AddHeaderAndBody fires, warmBody enters with stateCacheWired=true, txs=2 for the test block, but balLen=0 because the BAL was nowhere accessible to read-ahead. Result: no prefetch, EVM hot path paid the full file accessor stack on every cold address. Ports the balcache package from mh/perf-bal-cache as the canonical BAL store on the engine-API path: - execution/balcache/balcache.go — in-memory LRU (100 blocks), Cache/CachedBlockAccessList API, optional BALRegenerator / BALSyncFetcher hooks for fallback paths. - EngineServer.HandleNewPayload writes the BAL bytes into the cache on payload receipt, before any downstream processing. - BlockReadAheader.warmBody reads BAL via balcache.CachedBlockAccessList keyed by block hash, replacing the MDBX read entirely. Bench result (test_account_access[EXTCODESIZE-EXISTING_CONTRACT-30M], perf-devnet-3 v4.0.0 fixture, EXEC3_PARALLEL=true): binary mgas/s block time baseline 12.82 2.4 s L2b only 12.75 2.4 s L2b + codeSize 13.13 2.4 s L2b + codeSize + balcache 61.50 488 ms ← this commit ~4.7x speedup. Cross-client peer band on the same family (cycle-2 survey 2026-05-13): reth 50, geth 55, besu 55, nethermind 70. Erigon at 62 mgas/s now sits between geth and nethermind on a bench that was 2.2 mgas/s in the May 13 cross-client pull. Pprof signature confirms: seg.Getter.nextPos flat dropped from 52 % to 24 %; the dominant decompression cost is gone for BAL-listed addresses because EVM hits cache.StateCache (populated by read-ahead's cache-populating getter, commit cbe9044) instead of the file accessor stack. The kv.BlockAccessList table removal + rawdb.WriteBlockAccessListBytes deletion is intentionally NOT in this commit — that belongs to the bal-cache structural refactor on mh/perf-bal-cache, not on the optimisation branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…helpers BAL bytes are now cache-only (balcache, populated by ComputeBlockContext with on-demand regeneration). After the BAL writers were removed in 9ea90e4, the MDBX table and its Read/Write/Delete helpers have no producers — drop them outright instead of carrying dead surface. Removed: - kv.BlockAccessList table constant + ChaindataTables entry - rawdb.ReadBlockAccessListBytes / WriteBlockAccessListBytes - BlockAccessList delete in DeleteBody + 2 prune callbacks - PruneTable(kv.BlockAccessList) call in stage_execute - TestBlockAccessListStorage + the BlockAccessList seed rows in TestNoPruneSkipsAllPruneStages Readers (warmBody readAhead) already consult balcache.CachedBlockAccessList; the test helper in engine_server_test writes directly to balcache. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ing from the MDBX-era diagnostic) Adds debug_getRawBlockAccessList JSON-RPC method — returns the RLP-encoded BlockAccessList bytes this node has stored for a block (exactly what the eth/71 server-side handler returns to peers). Originally introduced as part of the dispatch-bug diagnostic tooling (b8bd26c) which also added cmd/bal-scan and cmd/bal-test. The cmd tools were tightly coupled to the kv.BlockAccessList MDBX table (scan / dump / delete / compare via cursor) — that table is removed in the prior commit of this PR, so the cmd tools have no surviving port. The RPC method does: it switches its read from rawdb.ReadBlockAccessListBytes to balcache.BlockAccessListBytes (cache hit, regenerator fallback). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mh0lt
force-pushed
the
fix/eth71-multiclient-server-dispatch
branch
from
b2dfc0b to
cd839e1
Compare
…emove the schedule-time ValidateAndPrepare purge The state cache carried a per-domain blockHash and was scrubbed by ValidateAndPrepare before every block. In the parallel executor that call sits in processRequest — a *schedule* step, not an apply step — copied from the serial path (#18868). With a 32-deep pipeline and heavy retry traffic the single blockHash almost never equals the next call's parentHash, so it took the wipe branch ~100% of the time: measured storage-cache purge_rate ~100%, hit ~35% during catch-up, the cache wiped every block. Make the cache what it should be — a SharedDomains implementation detail, populated only at flush (committed, fork-agnostic state) and invalidated only on unwind. Coherence is now txNum/epoch based, no block awareness and no diffset: - Each GenericCache entry carries (txNum, epoch). A read is valid iff it was written in the current epoch OR its txNum is at/below unwindFloor (predates every unwind). - Unwind(txNum) bumps the epoch and lowers the floor — O(1), no scan. Stale entries are dropped lazily on their next read. txNum slots are reused across forks, so the epoch (not the txNum) tells a dead fork's write from the live fork's at the same txNum. - CodeCache clears its small mutable addr layers on unwind; immutable content-addressed code is kept. Wiring: FlushWithCallback delivers txNum (cache stamps it; branchCache derives step); read-population and read-ahead stamp the step's txNum upper bound. The three exec-flow ValidateAndPrepare calls are removed; the unwind path calls stateCache.Unwind(txNum) unconditionally (diffset-free, matching the overlay's maxtx prune — diffsets aren't generated below the reorg window, so the old changeSet-gated cache revert left a stale gap). RevertWithDiffset/blockHash/ClearWithHash and the fork-validation cache scrub are removed. (DB-level diffset retirement is a follow-on.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mh0lt
force-pushed
the
mh/perf-statecache-lru-pr
branch
from
cf05caf to
2a41192
Compare
…-exec) SetHead (debug_setHead) built its unwind SharedDomains via NewSharedDomains but never called SetStateCache, so unwindExec3's doms.GetStateCache() returned nil and stateCache.Unwind(txNum) was skipped. The shared singleton cache kept pre-unwind values at the old epoch; the next UpdateForkChoice re-execution read them and computed a stale state root, returning ExecutionStatusBadBlock. Wire the shared cache into the SetHead SD, mirroring ValidateChain and the forkchoice path, so the epoch bump + floor lowering runs. TestSetHeadCanonicalCleanup covers this (failed cache-on, passes now). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ute tx ComputeCommitment clones a custom state reader with its own compute tx (commitment_context.go trieContext). When that reader is a LatestStateReader bound to a different database (e.g. recomputing commitment in an empty db while reading committed state from the source db, as TouchChangedKeysFromHistory does), Clone rebuilt sd.AsGetter against the foreign compute tx and read the wrong database — returning empty account/storage and thus a wrong (empty-leaf) root. This was masked until FlushWithCallback began draining sd.mem on flush: the in-memory batch previously still held the source values. Store the reader's source tx and rebind Clone to it. The default commitment path never routes through LatestStateReader.Clone (it constructs fresh readers), and the production custom readers build their own sub-readers, so this only affects readers explicitly bound to a foreign source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ranchCache) Remove the DomainMetrics write lock from the hot state-read path. Every GetLatest previously took DomainMetrics.Lock() — a write lock with no fast path — to bump read counters; under concurrent trie-warmup / parallel-exec workers that was a hard serialization point, so KV read metrics had to be left off. Replace it with a lock-free per-worker accumulator (changeset.WorkerMetrics) that each goroutine owns exclusively, combined into the shared DomainMetrics once per task via DomainMetrics.Merge — no lock, no atomics, no shared cache line on the hot path. The accumulator is carried via context (const-int ctx key, matching node/app) and read through getter.GetLatestContext / SharedDomains.GetLatestContext (mirroring the existing optional MeteredGetLatest). Commitment/warmup worker readers carry a per-worker ctx and merge at teardown; the main goroutine meters into sd.mainWM, flushed into the global on metrics read. Parallel-exec state readers use AsGetterNoMetrics for now (no metering); wiring their per-worker metrics is a follow-up on the state-read path. This also removes a data race those concurrent workers would otherwise hit on a shared accumulator once metrics are enabled. Also carries the tx-aware BranchCache (txNum/epoch unwind invalidation + maxStep-bounded reads) this sits on top of. Verified: build, make lint (0 issues), and -race with KV_READ_METRICS=true clean across commitmentdb / changeset / execmodule parallel block-assembly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The previous commit kept a single SharedDomains.mainWM that every AsGetter caller recorded into. But AsGetter is used by many concurrent goroutines on a live node (RPC, engine API, txpool, exec) — so a single lock-free accumulator was both raced and unbounded (its uniqueSeen map grew without limit and was written from every read). With KV_READ_METRICS=true this degraded a long-running node to a crawl (cancun 225/226 failing on insufficient-funds / timeouts); metrics-off was unaffected since the accumulator was never touched. Remove mainWM (and flushMainWM): GetLatest now collects no metrics. Read metrics are gathered only per-task / per-worker via GetLatestContext (the commitment + trie-warmup readers), each accumulator bounded to one task and merged into the shared DomainMetrics at teardown. AsGetterNoMetrics is kept as an explicit-intent alias. Wiring the exec/RPC read paths onto per-task contexts is a follow-up. Verified: KV_READ_METRICS=true cancun back to the 3 known failures (was 225); -race + metrics-on clean; build/gofmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The metrics-on safety fix left the main (root-fold) commitment reads unmetered — only trie-warmup workers collected, giving a partial read profile. Give the main fold its own per-ComputeCommitment lock-free accumulator (carried via the read context, merged into the shared metrics when commitment finishes). The fold is single-goroutine so it owns the accumulator exclusively; warmup / concurrent-mount workers keep their own. Now KV_READ_METRICS captures the full commitment read profile without any process-wide accumulator. Verified: -race + KV_READ_METRICS=true clean (3x) on execmodule parallel block-assembly; build/gofmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…stances Replaces changeset.WorkerMetrics (added in 44355fd) with plain per-worker *DomainMetrics instances — removing both the name collision with the unrelated execution/exec.WorkerMetrics and an extra type. The model is now: - Update* are lock-free and run on a single-owner per-worker instance (one per trie-warmup worker, one per ComputeCommitment for the main fold). - Merge folds a finished worker instance into the shared aggregate under the aggregate's lock — the only protected write. A separate goroutine reading the aggregate under RLock always sees a consistent snapshot. So: updates unprotected, aggregation/snapshot protected — no per-read lock, no atomics, no process-wide accumulator. Also drops the UniqueFileReadCount / UniqueLenBuckets read-amplification tracking: it had no consumer (only a comment referenced it) and cost a string-alloc + map-insert on every file read — a needless hot-path cost. The struct fields remain (unpopulated) to avoid churn. Update* gain nil receiver guards: a nil *DomainMetrics is the valid "collect no metrics" sentinel for reads with no per-worker context (e.g. SeekCommitment, plain AsGetter on RPC/exec paths). Verified: build, make lint (0 issues), gofmt, -race + KV_READ_METRICS=true clean (3x), and hive cancun with KV_READ_METRICS=true back to the 3 known fails (node healthy). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-task merge) Un-defers the exec-path read metrics that the metrics refactor had left collecting nothing (AsGetterNoMetrics), which made DomainMetrics undercount on a syncing node (exec reads dominate). Each exec Worker now owns a private *changeset.DomainMetrics; its state reader records into it via AsGetterMetered (lock-free, single-owner). The instance is merged into the shared SharedDomains metrics and reset once per task — right after the result is queued, off the hot path. That replaces a lock-per-read with a lock-per-task (a task has many reads), so contention is far lower; skipped entirely when KV_READ_METRICS is off. So exec3_metrics' per-domain Account/Storage/Code cache-layer profile is now populated, complementing exec.WorkerMetrics' per-worker read counts. Verified: build, make lint (0 issues), gofmt, -race + KV_READ_METRICS=true clean. Hive metrics-on validation pending (next). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The state-cache unwind set its invalidation floor to unwindToTxNum = Min(UnwindPoint+1) — the first txNum of the first rolled-back block — but the staleness check dropped entries only when txNum strictly exceeded the floor (txNum > floor). An entry stamped at exactly the floor therefore survived, even though that txNum belongs to a dead (unwound) block. This is the EIP-4788 beacon-root case: the beacon-root storage write runs in a block's begin-system-tx, stamped at the block's first txNum. When that block is the first one unwound by a reorg, its txNum equals the floor, so the dead-fork value was served stale during sidechain re-execution, yielding a wrong trie root (hive engine-cancun "Sidechain Reorg"). Drop entries at-or-above the floor (txNum >= floor). The surviving block's last txNum is floor-1, so this never evicts a live entry. Add a boundary regression test (TestUnwind_EvictsEntryAtFloor) covering txNum == floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- engineapitester: remove the now-orphaned EngineApiTester.ChainDB field (its only consumer — the EngineXTestRunner cache-clear — was removed in 4618add) plus its now-unused kv import - stagedsync/exec3, exec3_parallel: drop comments that describe code removed from main (the warmup-cache-disable workaround), per review - execmodule: add TestValidateForkPayloadOffNonTipCanonicalBlockWithCache, ExecModuleTester coverage for the unwind/fork-payload caching path — a fork off a NON-tip canonical block exercises the partial unwind + parent-link diffset masking of the BranchCache, round-tripped in both directions. The existing side-fork test only branches at genesis (full unwind). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconciles the perf-stack commitment caching with main's commitment refactor: - Adopt main's TrieConfig-based signatures: InitializeTrieAndUpdates(mode, tmpdir, cfg), NewHexPatriciaHashed(keyLen, ctx, cfg), NewSharedDomainsCommitmentContext(sd, mode, tmpdir, cfg, branchCache). - Keep the perf-stack BranchCache (cross-block, aggregator-scope, unwind-aware) + AdaptivePinController + BranchCacheProvider wiring; the trie's branchCache is attached via SetBranchCache by the commitment context. - Honor the perf stack's deliberate deletion of WarmupCache (39b758e: it duplicated the aggregator-scope BranchCache and was inactive on the block- application path). Main re-introduced it; dropped those re-additions (EnableWarmupCache/ClearWarmupCache/cache field) while keeping main's maxDeferredUpdates deferred-bound. - exec_module_test.go: keep both the fork-off-non-tip cache test and main's two FCU state-ahead-recovery tests. experiments.go: keep GOMAXPROCS(0) warmup default + main's PerfProfiles. Validated: go build ./...; commitment package tests; reorg/fork tests (fork-off-non-tip + side-fork + FCU recovery); make lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ripples the perf-caches-pr main merge (+ #21380 review fixes) up the stack. Resolved 5 conflicts by keeping statecache-lru's newer model where it has evolved past perf-caches-pr: - BranchCache: kept the txNum/epoch unwind model (Unwind + epoch invalidation) over perf-caches-pr's step/txN/UnwindTo model; ported the PutIfClean peek fix (avoid write-path miss-accounting). Removed the now-dead BranchCache.UnwindTo and converted the residual 5-arg PinEntry call (preload_parallel.go). - domain_shared.go: kept the lock-free wm metrics path + epoch-stamped branch Put + ClearBranchCache/DetachBranchCache; added the statecfg import for PickTrieVariant. generic_cache.go kept the freelru LRU. exec3_parallel.go kept AsGetterNoMetrics. - Restored statecache-lru's branch_cache_test.go / trunk_pin_test.go (they test the epoch model; the auto-merge had pulled perf-caches-pr's step/txN tests). Validated: go build ./...; commitment + execmodule reorg/fork gate + stateCache tests; make lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…patch (#20893) Ripples the statecache-lru main merge up (final fan-out branch). 1 conflict in bal_create.go: kept eth/71's cache-only BAL lookup (balcache.CachedBlockAccessList) over statecache-lru's DB-stored rawdb.ReadBlockAccessListBytes — BALs are cache-only on this branch (the BAL cache lineage). Aligned variable names (blockBalHash/blockBalBytes/blockBal) with the auto-merged cross-check body; reused the function-scoped err. gofmt fix in balregen.go. Validated: go build ./...; commitment + execmodule reorg tests; make lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restores the eth/71 server-side wiring that should have been part of #20794 (handler + sentry dispatch). The squash-merge of #20794 dropped the multi_client.go changes — only the eth/protocols handler (
AnswerGetBlockAccessListsQuery) and the sentry gRPC server bits made it in. As a result main today negotiates eth/71 capability with peers but silently drops every inboundGetBlockAccessListsmessage at the dispatcher, so peers requesting our BAL data time out and we never serve a single response.This PR restores those bits in a single, small change, as originally intended by the dev branch commit
8eed8fe7bf("Phase 5a"):MultiClient.RecvUploadMessageLoop— subscribe to inboundeth/71GetBlockAccessListsMsgrequests so the sentry actually pumps them into our handler.MultiClient.handleInboundMessage— add theGET_BLOCK_ACCESS_LISTS_71andBLOCK_ACCESS_LISTS_71cases to the dispatch switch.MultiClient.getBlockAccessLists71/blockAccessLists71— the in-process handler functions that read from rawdb and reply (server side) and route inbound responses to the in-flight fetcher (client side).This is Amsterdam-fork (EIP-7928 / EIP-8159) functionality — only blocks at or after the Amsterdam timestamp commit to a
BlockAccessListHash, and only those blocks have BAL bytes for peers to request.How it was found
While running PR #20795 (the BAL fetcher/downloader, which is the client-side counterpart) on bal-devnet-3, the BAL downloader's scan correctly identified missing BALs and fired
GetBlockAccessListsrequests at peers, but every request timed out — peers (Besu, Nimbus, our own erigon) were never delivering responses, and our own node was never replying when peers asked us for BALs. Adding the dispatch wiring in this PR resolves both directions: with this fix applied to both of two erigon nodes peering on bal-devnet-3, the BAL downloader successfully fetched ~328 BALs from a Besu peer and from our other erigon in a single run.The new tooling in the second commit was the means of finding and confirming the bug:
debug_getRawBlockAccessList(blockHash)JSON-RPC method (in thedebugnamespace, available with--http.api=debug): returns the raw RLP bytes this node has stored for the given block — exactly what the server-side eth/71 handler hands back to peers. Useful for byte-level cross-client comparison of BAL bytes.cmd/bal-test— dump / delete / compare BAL entries directly inchaindata(requires erigon stopped). Lets you snapshot a known-good set of BALs from one node, delete the same blocks on a peer, restart, watch the BAL downloader refetch from the wider devnet, and assert byte-equality against the snapshot.cmd/bal-scan— walk thekv.BlockAccessListtable and emit one(block, hash, len)line per entry within an optional range. Used to confirm what's actually persisted after prune (and how it intersects with what the downloader claims to have stored).Verification
make lint— cleanmake erigon— cleango test -short -count=1 ./p2p/sentry/sentry_multi_client/... ./p2p/protocols/eth/... ./rpc/jsonrpc/...— passeseth/71and our handler answers — without it, every fetch times out and[bal-downloader]only logsfetch failed err=\"bal: fetch timed out waiting for peer response\".Out of scope
This PR only restores the server-side wiring missing on
main. The client-side counterpart (BAL fetcher + downloader +BlockAccessListsMsgresponse subscription onRecvMessageLoop) is in #20795 and lands separately.