Backfill sync with beacon_blocks_by_head#82
Open
dapplion wants to merge 3 commits into
Open
Conversation
Replace the buffered epoch-batch backfill engine with a linear pipeline that walks the parent chain of the store anchor using beacon_blocks_by_head: fetch a run of ancestors, fetch their custody data columns by root, couple into RangeSyncBlock, process, and advance the anchor. - api_types: add SyncRequestId::BackfillBlocksByHead; make CustodyRequester an enum (SingleLookup / Backfill) with BackfillCustodyId. - network_context: backfill-origin blocks_by_head + custody-by-root requests. - manager/router: route backfill by_head and custody responses to backfill. - backfill_sync: linear Idle -> FetchingAncestors -> FetchingData -> Processing state machine; store AnchorInfo is the cursor. Gloas envelope backfill is deferred.
…alties - Reuse a single SyncRequestId::BlocksByHead carrying a BlocksByHeadRequester enum (Lookup/Backfill) and one blocks_by_head_requests map, instead of a separate backfill request id and map. Delete the now-unused RangeRequestId::BackfillSync. - Make backfill infallible: drop BackFillError/fail_sync and the Failed/restart machinery. On bad peers or data it penalizes and re-requests, or pauses; the store anchor lets it resume from scratch at any point. - Penalize exactly the peers that served a faulty segment (block peer plus the custody-column peers carried on the in-flight step) instead of an accumulated participating_peers set.
Gloas data lives in the execution payload envelope, which we do not backfill. Couple Gloas blocks via RangeSyncBlock::new_gloas(block, None) and skip the custody-by-root fetch for them, instead of pausing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworks backfill to walk the parent chain with
beacon_blocks_by_headinstead of epoch-rangedblocks_by_range.The store anchor (
oldest_block_parent) is the cursor: fetch a run of ancestors from it, fetch their data columns by root, verify and import, advance the anchor, repeat — one segment in flight at a time, no batch buffer or epoch bookkeeping.Everything is pinned to the trusted checkpoint anchor (block roots chain back to it; proposer signatures and KZG proofs are verified before import), so it's fully deterministic and infallible: a bad peer or bad data is penalized and refetched, and there's no terminal failed state.
Gloas envelope-by-root backfill is deferred. Stacked on #81.