v1.8.5: head catch-up + bridge-off proving + fork-choice & eclipse hardening (#50 #79 #80 #81 #82 #83)

CHANGELOG.md

@@ -2,6 +2,83 @@
 
 All notable changes to Lantern.
 
+## v1.8.5 (2026-06-30)
+
+**Head catch-up + bridge-off PDP proving robustness, and head fork-choice
+hardening.** A bridge-off node could wedge its head ~10-20 epochs behind the
+chain tip and then fail PDP proving with "cannot draw randomness from future
+epoch" (reported on v1.8.4-m). Root-caused to two stacked bugs plus a
+fork-choice gap on the running head.
+
+### Fixed
+
+- **Head catch-up no longer wedges behind the tip** ([#83](https://github.com/Reiers/lantern/issues/83)).
+  The #71 gossip-fresh skip suspended the polling Sync whenever gossip
+  installed any block recently — but a node whose gossip is fresh-but-lagging
+  (installing some blocks while skipping head+N>1 blocks it can't backfill)
+  would then have neither path reach the tip. The skip is now **lag-aware**:
+  it consults the gossip-observed tip (`blockingest.ObservedHead()`) and only
+  skips the catch-up poll when the store head is within a small tolerance of
+  it, so a lagging node resumes catch-up. Uses the gossip-observed tip, so
+  no extra upstream RPC — #71's 429-protection is preserved at the tip.
+
+- **Bridge-off PDP prove randomness** ([#82](https://github.com/Reiers/lantern/issues/82)).
+  `tipsetForRandomness` compared the requested epoch against the frozen boot
+  anchor (`Trusted.Epoch`, which never advances) instead of the live header
+  head, and the only fallback was bridge-only — so bridge-off nodes failed
+  outright and burned the prove task's retry budget. The ceiling is now the
+  live head, and when the requested epoch is within a small window above it
+  (normal sync catch-up) the call waits briefly (bounded) for the header sync
+  to reach it, then draws locally. No bridge required. Genuinely-future
+  epochs still error promptly.
+
+### Added
+
+- **Heaviest-ParentWeight fork choice on the running gossip head**
+  ([#79](https://github.com/Reiers/lantern/issues/79)). The head ingestor
+  advanced head on height-fence + parent-linkage only, so an eclipsed peer
+  table could feed parent-linked, height-advancing blocks on a
+  valid-but-lighter fork and walk a node onto it. Lantern now applies
+  Filecoin's fork-choice rule: adopt a candidate as head only when its
+  `ParentWeight` strictly exceeds the current head's. A competing lighter
+  fork is rejected (counted as `rejectedLighter`). Pure header arithmetic —
+  no proof verification, no ffi. Raises the eclipse cost from "spin up N
+  sybil peers" to "out-weight the real chain." Does not fully close the
+  un-finalized-tip split against an adversary with real power — that's F3.
+
+- **Un-evictable trusted peer floor** ([#80](https://github.com/Reiers/lantern/issues/80)).
+  The trusted bootstrap/beacon/direct-peer set is now `connmgr`-protected,
+  so the connection-manager trim path can never drop it. An attacker
+  flooding the node with dials can no longer fully replace the peer table
+  with peers it controls — the honest floor survives, keeping the heavier
+  canonical chain reachable. Complements #79 (which makes a lighter fork
+  unadoptable). Head-source diversity (≥N distinct peers) is a tracked
+  follow-up.
+
+- **Prefetch-on-send** ([#50](https://github.com/Reiers/lantern/issues/50)).
+  When `eth_sendRawTransaction` publishes a tx locally, its message/receipt
+  blocks are warmed into the Bitswap cache in the background so the
+  follow-up receipt poll resolves locally instead of racing a cold
+  cross-peer fetch. Closes the residual write-confirm availability gap.
+
+- **Explicit bridge-off head source** ([#50](https://github.com/Reiers/lantern/issues/50)).
+  New `FallbackRPC` (override the upstream RPC, e.g. your own Forest) and
+  `NoFallbackRPC` (remove the upstream RPC entirely — head from gossipsub,
+  cold blocks from gateway+Bitswap). Previously a bridge-off node silently
+  fell back to Glif when gossip stalled, a hidden third-party dependency;
+  now a stall surfaces as an observable stalled head. Refuses to start if
+  `NoFallbackRPC` is set without libp2p (no head source).
+
+### Changed
+
+- **`TRUST-MODEL.md` §2.1 corrected for accuracy**
+  ([#81](https://github.com/Reiers/lantern/issues/81)). The section claimed
+  election-proof and weight were re-verified on every header; it now states
+  precisely what is verified (CID integrity, signature shape, parent linkage,
+  heaviest-ParentWeight fork choice) vs what is not (election-proof VRF,
+  winning-PoSt, message re-execution) and why (no-ffi design), and points at
+  F3 + the boot quorum as what carries head trust.
+
 ## v1.8.4 (2026-06-29)
 
 **Fix: standalone `lantern daemon` could not fetch cold blocks over Bitswap.**

TRUST-MODEL.md

@@ -36,12 +36,47 @@ any of the following without being detected.
 
 ### 2.1 Header chain
 
-* Every header walked back to genesis (or to the trusted anchor checkpoint)
-  has its BLS signature, election proof, ticket, parent linkage, and weight
-  re-verified against the local copy of the previous header.
-* The header validator (`chain/header/header.go`) refuses any header that
-  fails any of these checks. Implemented logic lifted from
-  `lotus@v1.36 chain/sync.go::ValidateBlock`.
+Lantern is an **F3-anchored light client**, not a full re-executing
+validator. It deliberately does NOT do full Lotus-style block validation,
+because cryptographic election-proof / winning-PoSt verification requires
+`filecoin-ffi` (Rust), which Lantern's zero-CGo / one-binary design
+excludes by construction. Be precise about what that means per-field:
+
+**What every ingested header IS verified for:**
+
+* **CID integrity** — the header's CID is re-derived and checked against the
+  bytes (`header.VerifyBlockHeaderCID`). A peer cannot serve a header under
+  a CID it doesn't hash to.
+* **BLS / signature presence and shape** — structural validation
+  (`chain/header/header.go::ValidateHeader`, `ValidateTipsetShape`),
+  modelled on `lotus@v1.36 chain/sync.go`.
+* **Parent linkage** — a header is only adopted when its parents are present
+  in the local store (inline-backfilled if needed).
+* **Heaviest-ParentWeight fork choice** (#79) — the running gossip head is
+  advanced only when a candidate's `ParentWeight` strictly exceeds the
+  current head's. A competing valid-but-lighter fork is rejected. This is
+  Filecoin's fork-choice rule applied with pure header arithmetic.
+
+**What is NOT verified (by design, no-ffi):**
+
+* **Election-proof VRF validity** — Lantern checks that `ElectionProof` is
+  *present* for non-genesis heights, but does NOT cryptographically verify
+  the VRF (that needs the proofs/ffi path).
+* **Winning-PoSt** — not verified, same reason.
+* **Full message re-execution** — not performed.
+
+**What carries the trust instead:** the boot anchor (multi-source quorum on
+the F3-finalized tipset, §below) plus the F3 finality cert chain (§2.2,
+BLS-aggregate-verified). Past the finalized depth, the head is followed
+optimistically with the fork-choice + linkage checks above — the same
+probabilistic tip every node follows until F3 finalizes it. The residual
+eclipse/fork-selection exposure on the un-finalized tip is documented in
+§5 and tracked in #79 / #80; finality (F3) is what fully closes it.
+
+> Note: an earlier version of this section claimed election-proof and
+> weight were "re-verified" on every header back to genesis. That
+> overstated the code (weight fork-choice landed in #79; election-proof
+> VRF is presence-checked, not verified). Corrected for accuracy.
 
 ### 2.2 F3 finality
 

chain/header/store/sync.go

@@ -80,6 +80,28 @@ type SyncOptions struct {
 	// (the Sync polls every tick, the pre-#71 behavior).
 	GossipFresh func() bool
 
+	// GossipObservedHead, when set, reports the highest chain epoch the
+	// gossip ingestor has observed/installed - i.e. the gossip layer's
+	// view of the live tip (>= the canonical store head). It makes the
+	// #71 GossipFresh skip LAG-AWARE (#83): GossipFresh alone only tells
+	// us the head moved recently, not that it reached the tip. A node
+	// whose gossip is fresh-but-lagging (installing some blocks while
+	// skipping head+N>1 blocks it can't backfill) would otherwise have
+	// Sync skip its catch-up poll forever, wedging the head ~10-20 epochs
+	// behind the tip. With this set, Sync skips only when the store head
+	// is within SkipLagTolerance of the observed tip; when further behind
+	// it runs the catch-up poll regardless of GossipFresh. Returns a
+	// negative epoch when no observation is available (treated as "can't
+	// confirm we're at the tip" -> don't skip). Nil keeps the pre-#83
+	// behavior (skip purely on GossipFresh).
+	GossipObservedHead func() abi.ChainEpoch
+
+	// SkipLagTolerance is how many epochs behind the gossip-observed tip
+	// the store head may be and still skip the catch-up poll (#83). Only
+	// consulted when GossipObservedHead is set. Default 2 (allows the
+	// normal 0-1 epoch gossip latency without forcing an RPC poll).
+	SkipLagTolerance abi.ChainEpoch
+
 	// StaleResetThreshold is the lag (live head − store head, in epochs)
 	// beyond which a warm store is considered hopelessly stale and is
 	// re-bootstrapped near the live head instead of attempting a
@@ -177,6 +199,18 @@ func (s *Sync) Start(ctx context.Context) error {
 	return nil
 }
 
+// SetGossipObservedHead wires the gossip-observed-tip callback after
+// construction (#83). Pair with SetGossipFresh to make the #71 skip
+// lag-aware. tol is the SkipLagTolerance in epochs; <=0 uses the default.
+func (s *Sync) SetGossipObservedHead(fn func() abi.ChainEpoch, tol abi.ChainEpoch) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.opts.GossipObservedHead = fn
+	if tol > 0 {
+		s.opts.SkipLagTolerance = tol
+	}
+}
+
 // SetGossipFresh wires the gossip-freshness callback after construction.
 // The gossipsub ingestor is created after the Sync in both daemon paths,
 // so the callback can't be passed to NewSync; this lets the caller wire
@@ -238,11 +272,45 @@ func (s *Sync) pollAndApply(ctx context.Context) error {
 	// polls so the very first head can be bootstrapped from the RPC.
 	s.mu.Lock()
 	gossipFresh := s.opts.GossipFresh
+	gossipObservedHead := s.opts.GossipObservedHead
+	skipLagTol := s.opts.SkipLagTolerance
 	s.mu.Unlock()
+	if skipLagTol <= 0 {
+		skipLagTol = 2
+	}
 	if gossipFresh != nil && s.store.HeadEpoch() >= 0 && gossipFresh() {
-		s.mu.Lock()
-		s.stats.SkippedGossipFresh++
-		s.mu.Unlock()
+		// #83: the gossip-fresh skip must be lag-aware. GossipFresh only
+		// reports that the head moved recently, not that it reached the
+		// tip. A node whose gossip installs some blocks while skipping
+		// head+N>1 blocks it can't backfill stays fresh-but-lagging; if
+		// we skip the catch-up poll here the head wedges ~10-20 epochs
+		// behind the tip forever (Beck, v1.8.4-m). Only skip when we can
+		// confirm the store head is within SkipLagTolerance of the
+		// gossip-observed tip. When GossipObservedHead is unset or can't
+		// confirm (negative), fall back to the pre-#83 behavior of
+		// trusting GossipFresh, since without a tip estimate the only
+		// alternative is paying the RPC HeadEpoch poll #71 exists to
+		// avoid.
+		atTip := true
+		if gossipObservedHead != nil {
+			if tip := gossipObservedHead(); tip >= 0 {
+				atTip = s.store.HeadEpoch() >= tip-skipLagTol
+			}
+		}
+		if atTip {
+			s.mu.Lock()
+			s.stats.SkippedGossipFresh++
+			s.mu.Unlock()
+			return nil
+		}
+	}
+
+	// #50 part 3: a nil source means no upstream RPC was wired (bridge-off
+	// NoFallbackRPC). Gossipsub is the sole head driver in that mode, so
+	// the polling Sync has nothing to do - it stays a no-op rather than
+	// nil-panicking on HeadEpoch. The store head still advances via the
+	// gossip ingestor's SetHead calls.
+	if s.src == nil {
 		return nil
 	}
 

chain/header/store/sync_test.go

@@ -640,3 +640,107 @@ func TestSyncColdStartPollsDespiteGossipFresh(t *testing.T) {
 	require.NoError(t, sync.PollOnce(context.Background()))
 	require.GreaterOrEqual(t, src.calls(), 1, "cold store must poll HeadEpoch despite GossipFresh")
 }
+
+// TestSyncGossipFreshSkipIsLagAware covers #83: the #71 gossip-fresh skip
+// must not suspend the catch-up poll when gossip is fresh-but-lagging.
+// A node whose gossip installs some blocks while skipping head+N>1 blocks
+// it can't backfill would otherwise wedge its head behind the tip forever.
+func TestSyncGossipFreshSkipIsLagAware(t *testing.T) {
+	s, _ := newStore(t, false)
+	src := newFakeSource()
+
+	// Seed a warm chain 0..4 and bring the store to head 4.
+	g := mkBlock(t, 0, nil, 1000, "g")
+	src.put(g)
+	parents := []cid.Cid{g.Cid()}
+	for h := abi.ChainEpoch(1); h <= 4; h++ {
+		b := mkBlock(t, h, parents, 1000, "")
+		src.put(b)
+		parents = []cid.Cid{b.Cid()}
+	}
+	sync := hstore.NewSync(s, src, hstore.SyncOptions{MaxBacktrack: 50, CatchUpChunk: 200})
+	require.NoError(t, sync.PollOnce(context.Background()))
+	require.Equal(t, abi.ChainEpoch(4), s.HeadEpoch())
+
+	// Advance the live source far ahead (to 20) - simulating the real tip
+	// the lagging node hasn't reached.
+	parents = []cid.Cid{src.canon[4].Cid()}
+	for h := abi.ChainEpoch(5); h <= 20; h++ {
+		b := mkBlock(t, h, parents, 1000, "")
+		src.put(b)
+		parents = []cid.Cid{b.Cid()}
+	}
+
+	// Gossip is "fresh" (it installed *something* recently) but its
+	// observed tip is far above our store head (4) - the fresh-but-lagging
+	// state. The skip must NOT fire; catch-up must run and advance.
+	sync.SetGossipFresh(func() bool { return true })
+	sync.SetGossipObservedHead(func() abi.ChainEpoch { return 20 }, 0)
+
+	before := sync.Stats().SkippedGossipFresh
+	require.NoError(t, sync.PollOnce(context.Background()))
+	require.Equal(t, before, sync.Stats().SkippedGossipFresh,
+		"poll must not be skipped while gossip is fresh-but-lagging")
+	require.Greater(t, int64(s.HeadEpoch()), int64(4),
+		"head must advance toward the tip, not wedge at 4")
+}
+
+// TestSyncGossipFreshSkipAtTip covers the other half of #83: when gossip is
+// fresh AND the store head is at (within tolerance of) the observed tip, the
+// #71 skip must still fire so a healthy node doesn't hammer the upstream RPC.
+func TestSyncGossipFreshSkipAtTip(t *testing.T) {
+	s, _ := newStore(t, false)
+	src := newFakeSource()
+
+	g := mkBlock(t, 0, nil, 1000, "g")
+	src.put(g)
+	parents := []cid.Cid{g.Cid()}
+	for h := abi.ChainEpoch(1); h <= 10; h++ {
+		b := mkBlock(t, h, parents, 1000, "")
+		src.put(b)
+		parents = []cid.Cid{b.Cid()}
+	}
+	sync := hstore.NewSync(s, src, hstore.SyncOptions{MaxBacktrack: 50, CatchUpChunk: 200})
+	require.NoError(t, sync.PollOnce(context.Background()))
+	require.Equal(t, abi.ChainEpoch(10), s.HeadEpoch())
+
+	// Gossip fresh and observed tip == store head (at the tip): skip must
+	// fire (429-protection preserved).
+	sync.SetGossipFresh(func() bool { return true })
+	sync.SetGossipObservedHead(func() abi.ChainEpoch { return 10 }, 0)
+
+	before := sync.Stats().SkippedGossipFresh
+	require.NoError(t, sync.PollOnce(context.Background()))
+	require.Equal(t, before+1, sync.Stats().SkippedGossipFresh,
+		"poll must be skipped when at the tip (preserve #71 429-protection)")
+	require.Equal(t, abi.ChainEpoch(10), s.HeadEpoch())
+}
+
+// TestSyncNilSourceIsNoOp covers #50 part 3: a Sync constructed with no
+// upstream RPC source (bridge-off NoFallbackRPC, gossip is the sole head
+// driver) must not panic - PollOnce is a safe no-op and head is untouched.
+func TestSyncNilSourceIsNoOp(t *testing.T) {
+	s, _ := newStore(t, false)
+
+	// Seed a head via a normal source first, then drop the source.
+	src := newFakeSource()
+	g := mkBlock(t, 0, nil, 1000, "g")
+	src.put(g)
+	parents := []cid.Cid{g.Cid()}
+	for h := abi.ChainEpoch(1); h <= 3; h++ {
+		b := mkBlock(t, h, parents, 1000, "")
+		src.put(b)
+		parents = []cid.Cid{b.Cid()}
+	}
+	seed := hstore.NewSync(s, src, hstore.SyncOptions{MaxBacktrack: 10})
+	require.NoError(t, seed.PollOnce(context.Background()))
+	require.Equal(t, abi.ChainEpoch(3), s.HeadEpoch())
+
+	// Now a nil-source Sync: PollOnce must be a no-op, no panic, head intact.
+	nilSync := hstore.NewSync(s, nil, hstore.SyncOptions{MaxBacktrack: 10})
+	require.NotPanics(t, func() {
+		require.NoError(t, nilSync.PollOnce(context.Background()))
+	})
+	require.Equal(t, abi.ChainEpoch(3), s.HeadEpoch(),
+		"nil-source Sync must not alter head")
+}

cmd/lantern/main.go

@@ -1009,6 +1009,9 @@ func cmdDaemon(args []string) error {
 				// relaxed 30s cadence); when gossip goes quiet the Sync resumes.
 				if sync != nil {
 					sync.SetGossipFresh(func() bool { return ing.Fresh(60 * time.Second) })
+					// #83: lag-aware skip - resume catch-up when gossip is
+					// fresh-but-lagging instead of wedging behind the tip.
+					sync.SetGossipObservedHead(func() abi.ChainEpoch { return ing.ObservedHead() }, 0)
 				}
 			}
 		}