Problem
Split out from #82 (bug 2). This is the underlying disease behind Beck's bridge-off prove failure, and it affects every bridge-off node, not just proving.
A bridge-off node can wedge its head ~10-20 epochs behind the real chain tip and stay there. Observed (Beck, v1.8.4-m, mainnet 2026-06-30):
[gossip-block] sub-rcv=125 installed=11 skipped=43 backfilled=54 lastEpoch=6149489
[sync] polls=21 advances=1 head=6149469
Gossip sees the tip at 6149489 but head is stuck at 6149469, and the polling Sync advances=1 over 22 polls.
Root cause
The #71 gossip-fresh skip in chain/header/store/sync.go::pollAndApply is not lag-aware:
if gossipFresh != nil && s.store.HeadEpoch() >= 0 && gossipFresh() {
s.stats.SkippedGossipFresh++
return nil // skips the entire catch-up poll
}GossipFresh() returns true if the gossip ingestor installed any block recently. But net/blockingest skips blocks arriving at head+N with N>1 (defers them to Sync). So gossip can be advancing the head a little (GossipFresh=true) while still being far behind the tip because it skips every far-ahead block. Sync then skips its catch-up poll on the assumption "gossip's got it" — but gossip can't close the gap and Sync won't run. Deadlock: head sticks ~20 behind.
The #71 optimization is correct in steady state (don't hammer Glif when gossip is keeping head at the tip). It's wrong when gossip is fresh-but-lagging.
Fix
Make the gossip-fresh skip lag-aware. Skip the catch-up poll only when gossip is fresh AND the store head is within a small tolerance of the chain tip. When the store is far behind, run the catch-up poll regardless of gossip freshness — that is exactly what Sync exists for.
To avoid paying an RPC HeadEpoch() just to measure lag (the thing #71 is avoiding), use the gossip-observed tip the ingestor already tracks (lastEpoch in net/blockingest stats): skip only if storeHead >= gossipObservedTip - tolerance. No extra RPC call.
- Add a
GossipObservedHead func() abi.ChainEpoch (or extend GossipFresh to a lag-reporting callback) wired from net/blockingest.
- Skip-poll condition becomes:
gossipFresh() && storeHead >= gossipObservedHead() - SkipTolerance (SkipTolerance small, e.g. 2-3 epochs).
- When lagging, fall through to the normal CatchUpChunk poll, which already paces contiguous catch-up.
Secondary (optional, separate): escalate gossip-skipped head+N blocks to immediate targeted backfill rather than waiting for the next Sync poll, so catch-up is faster under healthy gossip.
Acceptance
Reported via Beck v1.8.4-m. Parent: #82. Related: #71, #50, #53.
Problem
Split out from #82 (bug 2). This is the underlying disease behind Beck's bridge-off prove failure, and it affects every bridge-off node, not just proving.
A bridge-off node can wedge its head ~10-20 epochs behind the real chain tip and stay there. Observed (Beck, v1.8.4-m, mainnet 2026-06-30):
Gossip sees the tip at 6149489 but head is stuck at 6149469, and the polling Sync
advances=1over 22 polls.Root cause
The #71 gossip-fresh skip in
chain/header/store/sync.go::pollAndApplyis not lag-aware:GossipFresh()returns true if the gossip ingestor installed any block recently. Butnet/blockingestskips blocks arriving athead+Nwith N>1 (defers them to Sync). So gossip can be advancing the head a little (GossipFresh=true) while still being far behind the tip because it skips every far-ahead block. Sync then skips its catch-up poll on the assumption "gossip's got it" — but gossip can't close the gap and Sync won't run. Deadlock: head sticks ~20 behind.The #71 optimization is correct in steady state (don't hammer Glif when gossip is keeping head at the tip). It's wrong when gossip is fresh-but-lagging.
Fix
Make the gossip-fresh skip lag-aware. Skip the catch-up poll only when gossip is fresh AND the store head is within a small tolerance of the chain tip. When the store is far behind, run the catch-up poll regardless of gossip freshness — that is exactly what Sync exists for.
To avoid paying an RPC
HeadEpoch()just to measure lag (the thing #71 is avoiding), use the gossip-observed tip the ingestor already tracks (lastEpochinnet/blockingeststats): skip only ifstoreHead >= gossipObservedTip - tolerance. No extra RPC call.GossipObservedHead func() abi.ChainEpoch(or extend GossipFresh to a lag-reporting callback) wired fromnet/blockingest.gossipFresh() && storeHead >= gossipObservedHead() - SkipTolerance(SkipTolerance small, e.g. 2-3 epochs).Secondary (optional, separate): escalate gossip-
skippedhead+N blocks to immediate targeted backfill rather than waiting for the next Sync poll, so catch-up is faster under healthy gossip.Acceptance
Reported via Beck v1.8.4-m. Parent: #82. Related: #71, #50, #53.