feat(worker): per-block subscription stream#128
Open
prajwolrg wants to merge 3 commits into
Open
Conversation
8 tasks
prajwolrg
force-pushed
the
STR-3698-asm-worker-subscription
branch
from
49ac037 to
8a6d427
Compare
|
Commit: 02a0468
|
prajwolrg
force-pushed
the
STR-3698-asm-worker-subscription
branch
3 times, most recently
from
34e9c47 to
86a7de7
Compare
Consumers of committed blocks (Moho-state derivation, and later proof requests) were wired into the ASM worker's hot path. Expose AsmWorkerHandle::subscribe_blocks(): after each anchor write the service fans the new L1BlockCommitment out to subscribers over unbounded channels and prunes dropped receivers. send on an unbounded channel is non-blocking, so the worker never awaits a consumer and stays off the critical path. Subscribers fire only after a successful ASM commit, not on raw block arrival -- hence AsmSubscribers.
prajwolrg
force-pushed
the
STR-3698-asm-worker-subscription
branch
from
86a7de7 to
c19e042
Compare
* feat(moho): add subscription-driven Moho worker Materializes per-block MohoState from the ASM worker's commit stream as a deterministic forward-only fold: for each committed block it reads the anchor state and logs the ASM worker already persisted and derives the Moho state from them, persisting through a caller-supplied context. The context splits reads from writes — AsmStateProvider fetches the anchor state and its logs (committed atomically per block, hence the shared MissingAsmState miss), MohoStateStore loads and persists the derived state — mirroring how strata-asm-worker takes a WorkerContext. * fix(moho): fold onto the resolved parent to follow L1 reorgs The fold assumed each commit was the immediate height-successor of the last one processed and chained off an in-memory cur_moho. That breaks under an L1 reorg: a commit building on an earlier fork point isn't the height-successor of the previously folded block, so it was wrongly dropped as stale or rejected as a gap. Instead resolve the commit's actual parent (new L1ProviderContext) and fold from the parent's committed Moho state (MohoStateStore::get_moho_state), so the worker chains along real ancestry across reorgs. NonContiguousBlock gives way to MissingMohoState (the residual missing-parent gap) and MissingParentBlock. * refactor(moho): hold Moho state in memory and split orchestration into the service Mirror strata-asm-worker's shape: keep the current MohoState in the service state and move the fold orchestration into a process_block free function in the service layer, leaving the state as data plus a small update_moho_state mutation. The in-memory state is load-bearing again — the common in-order commit folds straight from it with no store read — while reorg-safety is kept by re-anchoring from the parent's committed state when the incoming commit doesn't build on the block held in memory. Surface the current MohoState in the worker status (via moho-types' serde feature), matching AsmWorkerStatus, and drop the launch-relative processed counter, which reset on restart and added nothing over cur_block. * feat(moho): add dedicated moho-state storage crate Per-block MohoState persistence lived in strata-asm-proof-db next to proof artifacts, conflating materialized state with proofs. Give it its own home so the Moho worker can own its store independently of the proof DB. Adds get_latest so the worker can resume from its latest committed state across restarts. * refactor(moho): run Moho as a standalone worker in the runner The ASM worker materialized MohoState inline on every anchor-state write (the AsmWorkerContext piggyback), tying Moho to the ASM worker's hot path. Spin the subscription-driven MohoWorker off onto its own service task instead: it folds each ASM commit onto its resolved parent and persists the derived state, following L1 reorgs without sitting in the ASM worker's write path. Persistence moves to the dedicated strata-asm-moho-storage store; the RPC server and proof orchestrator read from it unchanged. * refactor(proof-db): drop the moho-state store now owned by the moho worker MohoState persistence moved to the dedicated strata-asm-moho-storage crate, so strata-asm-proof-db no longer needs its own copy. Remove the MohoStateDb trait and SledMohoStateDb (the moho-proof code stays) along with the deps they alone pulled in. Both stores key the same "moho_states" tree with identical encoding, so existing databases carry over unchanged. * refactor(moho): fold the export-entries index into the moho worker The export-entries index mirrors the leaves of each MohoState's ExportState MMR: both derive from the same NewExportEntry logs and must advance from the same iteration, or the per-container indices drift from the MMR that commits to them and inclusion proofs break. MohoState derivation moved to the Moho worker, but the index was still written by AsmWorkerContext on the ASM worker — splitting one unit across two tasks. Move ExportEntriesDb from asm-storage into strata-asm-moho-storage and the indexing into the worker's per-block fold via a new ExportEntryStore concern, written before the Moho-state commit point so a reprocessed block re-appends idempotently. AsmWorkerContext::store_anchor_state now just writes the anchor. * refactor(moho): split export-entries store into trait and sled impl The export-entries store landed as a single sled-specific file, unlike the moho-state store which separates a backend-agnostic trait from its sled implementation. Mirror that layout: lift an async `ExportEntriesDb` trait to the top level and move the concrete store (renamed `SledExportEntriesDb`) under `sled/`. The sled type keeps its synchronous inherent methods for the worker/RPC call sites and implements the async trait for symmetry with `MohoStateDb`. Consumers in asm-runner are renamed to the concrete type. * docs(moho): flag reorg handling for export entries as TODO A block can emit multiple ExportEntry leaves, so the append path can't reuse the manifest MMR's reorg strategy. Record the follow-up (STR-3723) at the append site rather than leaving the gap undocumented. * fix(asm-runner): repair broken intra-doc link in moho_context `Self` does not resolve in a module-level doc comment, so the rustdoc intra-doc link failed under `-D warnings` and broke the Generate docs CI job. Reference the concrete `MohoWorkerContextImpl` type instead. * refactor(worker): extract the block-sync walk into a reusable helper The ASM worker's plan_block_processing hand-rolls a backward walk from a target tip to the most recent ancestor with stored state, collecting the blocks in between. The Moho worker needs the identical walk for its startup catch-up — only the forward step (run STF vs fold MohoState) differs — so the search belongs in one tested place rather than copied. Move it to a generic plan_sync over closures for the base lookup and parent resolution, exported from strata-asm-worker, and route plan_block_processing through it. The base-lookup closure also stops swallowing every storage error as "not a base": only MissingAsmState means keep walking, so a real DbError now propagates instead of being mistaken for an unprocessed block. * feat(moho): catch the Moho store up to the ASM tip on startup The ASM worker commits a block's anchor state before the Moho worker folds it, and the commit subscription has no replay. So a crash in that window — or any downtime while the ASM worker keeps committing — leaves the Moho store trailing the ASM store, and the next live commit folds onto a parent whose Moho state was never derived, wedging the worker on MissingMohoState. Add sync_to_tip, run once before the subscription is consumed: it walks real parents back from the ASM tip (get_latest_asm_block) to the last folded block and folds the gap forward, so the steady-state handler can keep assuming its parent is present. Routed through strata-asm-worker's plan_sync, with genesis as the floor. Walking real parents rather than heights keeps it correct across an L1 reorg during downtime.
…ubscription # Conflicts: # bin/asm-runner/src/worker_context.rs # crates/worker/src/state.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The ASM worker had derived-state consumers (Moho-state derivation today, proof requests later) wired into its hot path. This adds a per-block notification stream so consumers can react to each committed block on their own task instead of being baked into the worker.
AsmWorkerHandle::subscribe_blocks()hands out aSubscription<L1BlockCommitment>(aStream, with abacklog()depth signal). After each successful anchor-state commit, the service fans the new commitment out to all subscribers over unbounded channels and prunes dropped receivers viaretain.sendon an unbounded channel is non-blocking, so the worker never awaits a consumer and stays off the critical path. Subscribers fire only after a successful ASM commit — not on raw block arrival — henceAsmSubscribers. Mirrorsstrata-bridge'sbtc-trackersubscriber pattern.This is the foundation for STR-3698 (moho worker); the Moho consumer PR is stacked on top of this one.
Type of Change
Notes to Reviewers
Pure addition to
strata-asm-worker: no existing public API changed (subscribe_blocksis additive;submit_block/launchare unchanged).AsmWorkerHandle::new/AsmWorkerServiceState::neware nowpub(crate)since they take the privateAsmSubscribersregistry, but the builder is the only constructor and nothing outside the crate called them. Unit tests cover fan-out, ordering,backlog(), dead-subscriber pruning, and stream close.Checklist
Related Issues
Part of Jira STR-3698.