Skip to content

fix(worker): treat an already-processed block as a no-op, not a reorg#155

Merged
MdTeach merged 2 commits into
mainfrom
fix/asm-worker-phantom-reorg
Jun 19, 2026
Merged

fix(worker): treat an already-processed block as a no-op, not a reorg#155
MdTeach merged 2 commits into
mainfrom
fix/asm-worker-phantom-reorg

Conversation

@prajwolrg

@prajwolrg prajwolrg commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Description

The ASM worker misclassifies duplicate/lagging ZMQ hashblock notifications as L1 reorgs, and the resulting rollback corrupts derived state. This PR fixes it at two layers.

The bug

The block watcher submits the chain tip once on startup to catch up, then streams live hashblock notifications (drive_asm_from_bitcoin in bin/asm-runner/src/block_watcher.rs). When blocks are mined during that startup window, the one-shot tip-submit walks the chain forward past them, and the buffered ZMQ notifications for those same blocks drain afterwards — re-delivering blocks that are now ancestors of the in-memory tip. sync_to_block read each as a reorg: it logged a phantom ASM L1 reorg detected, rolled the in-memory tip back to the re-delivered ancestor, and re-committed that block's anchor.

The re-commit is the damaging part. get_anchor_state reconstructed the AsmState with empty logs (only the AnchorState is persisted; logs live in the manifest). store_anchor_state derives the block's MohoState and export-entry index from AsmState::logs(), so re-committing a logless state snapped the block's persisted MohoState back to its parent's — dropping that block's export entries and any predicate update. The next block's step proof then read the corrupted from state and the recursive-proof chain failed with InvalidMohoChain. This surfaced in the strata-bridge functional tests (which mine bursts of blocks carrying real deposit/withdrawal logs at runner startup); the ASM suite never hit it because it mines empty blocks one at a time after the runner is ready.

Fix 1 — worker: an already-processed block is a no-op, not a reorg

A genuine reorg always brings at least one new block on the new branch, i.e. a non-empty plan. An empty plan means the target already has a stored anchor and was processed before. sync_to_block now returns early in that case — no rollback, no re-commit, no reorg log — leaving the tip and every derived state where the forward pass left them. bitcoind only announces blocks on the active chain, so a re-delivered already-stored block is never a real fork.

Fix 2 — asm-runner: rejoin manifest logs when reconstructing AsmState

get_anchor_state/get_latest_asm_state returned empty logs on the assumption that reads only need AsmState::state(). But the log-derived artifacts (MohoState, the export-entry index) make that assumption false, so a reconstructed state was silently lossy — the root reason the re-commit corrupted anything. They now rejoin the block's logs from its manifest, so a reconstructed AsmState matches what the STF produced and store_anchor_state is genuinely idempotent as documented. Genesis has no manifest (seeded without the STF), so its logs come back empty. (Fix 1 already removes the re-commit, so this is defense-in-depth + an honest get_anchor_state contract.)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New or updated tests

Notes to Reviewers

The two sync_resync_* tests encoded the old "roll the tip back on resync" behaviour, which was only ever exercised by these stale notifications — there is no driver that deliberately rewinds the worker. They are rewritten to assert the new no-op semantics (tip and durable latest stay on the real tip across a restart). All reorg tests (plan_reorg_uses_fork_point, sync_reorg_overwrites_leaves, reorg_orphans_leaves_present_but_unprovable) are unchanged and still pass — genuine reorgs go through the non-empty-plan path untouched.

Fix 2 has no dedicated unit test: exercising AsmWorkerContext::get_anchor_state means standing up a full context (bitcoin client, runtime handle, a valid AnchorState), which the asm-runner tests deliberately avoid. It is covered end-to-end by the functional tests that read anchor state back (the restart test) and the Moho path (the bridge suite). Happy to add the scaffolding if reviewers prefer.

The phantom reorg was also reproduced independently in the ASM functional harness (restarting the runner into a background mining burst produces ASM L1 reorg detected with getchaintips showing a single linear chain).

Checklist

  • I have performed a self-review of my code.
  • I have commented my code where necessary.
  • I have updated the documentation if needed.
  • My changes do not introduce new warnings.
  • I have added tests that prove my changes are effective or that my feature works.
  • New and existing tests pass with my changes.

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown

Commit: e56f79c
SP1 Execution Results

program cycles gas
asm-stf 136,419,333 141,510,061
moho 5,223,535 5,525,300

@codecov

codecov Bot commented Jun 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
bin/asm-runner/src/worker_context.rs 73.18% <100.00%> (+1.14%) ⬆️
crates/worker/src/service.rs 98.00% <100.00%> (+<0.01%) ⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@prajwolrg

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c542a6492

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/worker/src/service.rs
The block watcher submits the chain tip on startup to catch up, then streams
live `hashblock` notifications. When blocks are mined during that startup
window, the tip-submit walks the chain forward past them and the buffered
notifications for those same blocks drain afterwards, re-delivering ancestors
of the current tip. `sync_to_block` misread each as an L1 reorg: it logged a
phantom "reorg detected", rolled the in-memory tip back to the re-delivered
ancestor, and re-committed that block's anchor.

The re-commit is the damaging part. `get_anchor_state` reconstructs the
`AsmState` with empty logs (only the `AnchorState` is persisted), so
re-deriving from it drops the block's STF logs. For the Moho extension that
silently rewrites the block's persisted MohoState — its export state and next
predicate are folded from those logs — desyncing it from what the step proof
attested and breaking the recursive-proof chain with `InvalidMohoChain` once a
rolled-back block carries real logs (a bridge deposit/withdrawal).

A genuine reorg always brings at least one new block on the new branch, i.e. a
non-empty plan; an empty plan means the target was already processed. Return
early in that case — no rollback, no re-commit, no reorg log — leaving the tip
and every derived state where the forward pass left them.
The anchor state DB persists only the `AnchorState`; the STF logs live in the
manifest store. `get_anchor_state`/`get_latest_asm_state` rebuilt the `AsmState`
with empty logs, on the assumption that reads only ever need
`AsmState::state()`. But `store_anchor_state` derives the MohoState and the
export-entry index from `AsmState::logs()`, so feeding it a reloaded, logless
state silently drops the block's export entries and predicate update — exactly
how a re-committed anchor desynced its persisted MohoState from the proven one.

Rejoin the logs from the block's manifest so a reconstructed `AsmState` matches
what the STF produced, keeping every log-derived artifact correct even when it
runs over reloaded state. Genesis has no manifest (it is seeded without running
the STF), so its logs come back empty.
@prajwolrg prajwolrg force-pushed the fix/asm-worker-phantom-reorg branch from 8c542a6 to b4ed6d5 Compare June 19, 2026 04:18
@prajwolrg prajwolrg marked this pull request as ready for review June 19, 2026 08:29
@prajwolrg prajwolrg requested a review from MdTeach June 19, 2026 08:39
@prajwolrg

Copy link
Copy Markdown
Collaborator Author

Validated end-to-end against strata-bridge#638

This fix unblocks alpenlabs/strata-bridge#638 (the asm version bump), whose functional tests were failing on the phantom reorg. The CI progression on that PR pins it down:

  • 6441e1a7 — bump asm to v0.1-alpha.14 (pre-fix): every functional-test group failed.
  • 7a4e1014 — feed asm-runner the hashblock zmq endpoint: most groups pass, but contest, fulfillment, liveness, and uncontested_payout still fail.
  • 703f0a93 — point asm at this branch's phantom-reorg fix (b4ed6d5): all functional tests green.

The remaining failures at 7a4e1014 are exactly the deposit/withdrawal-bearing groups that mine bursts of real-log blocks at runner startup — the window where the one-shot tip-submit races the buffered hashblock notifications and the re-delivered ancestors were misread as reorgs. Pointing at b4ed6d5 turns them green, confirming the fix on the path that originally surfaced the bug.

@MdTeach MdTeach enabled auto-merge June 19, 2026 09:22
@MdTeach MdTeach added this pull request to the merge queue Jun 19, 2026
Merged via the queue into main with commit 31d88a4 Jun 19, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants