Skip to content

fix: keep the tip of a branch with no more work than ours, so that an announcement on it does not get the branch again (#88) - #93

Merged
lorenzolfm merged 13 commits into
mainfrom
issue-88
Nov 14, 2026
Merged

lorenzolfm merged 13 commits into
mainfrom
issue-88

Conversation

@lorenzolfm

@lorenzolfm lorenzolfm commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Part of #88. The second option of #88: remember the last branch that elo checked.

What

  • chain::BranchTip { hash, work_max, ours_height, ours_hash }: the last header of a branch that passed the checks of the chain, a work that is not less than the chain work of the branch at that header, and the height and hash of our tip when elo kept the branch.
  • Extended::KeptOver(BranchTip): our chain stays, and the batch ended a branch with no more work than ours. Three places give it, and all give Kept on main:
    • extend_above, for a batch that is not full, when the work of the branch is at most ours.
    • extend_above, for a batch below the margin that is not full. The work is a bound: not less than the work of the branch, and below ours less the margin.
    • extend_on_branch, for an empty batch after a full batch of the branch. The work at the fork comes from work_at.
  • Chain::branch_tip(work_at_fork, work_branch, hash) makes each tip, with the assertion that its work is at most ours.
  • Chain::branch_tip_current: a tip is current while our chain has the header at ours_height with ours_hash. A reorg below that height, or a restart, makes it not current.
  • Extended::Taken { full }: Taken is the only result that reads full, so chain_next has no full parameter, and its 3 assertions on full are gone.
  • Peer.branch_tip: Option<BranchTip>. chain_next sets it on KeptOver. The connection end drops it.
  • receive_announcement first drops a branch_tip that is not current. If the first header is on branch_tip, Chain::extend_branch_tip checks that each header is on the one before it and has its proof of work, and adds the work of the headers. It gives Announced::Ask if the branch has more work than ours or the announcement is full. Then a getheaders with our locator is due, as on main. If not, it gives Announced::Kept with the tip on the last header, and elo sends nothing. Other announcements leave branch_tip as it is.
  • extend_branch_tip takes Headers, so that the type bounds its loop to 2,000 headers.
  • The fuzz model of machines keeps the same tip: branch_tip: Option<(Hash, u128, usize)>, ExtendedModel::KeptOver, below_margin_work_max and receive_off_tip. A reorg in the model drops the tip when the fork is below the height of the tip. The driver compares the current tip of elo with the tip of the model after each message.
  • README: one paragraph.
  • Seed fork-below-margin-short-kept-then-announce-on-tip in the corpus and in SEED_ENDS.
  • Tests:
    • announcement_on_a_branch_that_lost_to_ours_asks_for_it_only_with_more_work_than_ours: after a low-work sync of a branch that loses, 2 announcements on the branch and 1 on our tip send no getheaders. The announcement that gives the branch more work than ours sends one.
    • branch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tip: a branch of equal work is kept, then gets one more header and takes our chain. The peer announces our tip again on the old branch tip, and elo asks with its locator. The next announcement on our tip extends it.
    • branch_that_lost_to_ours_stays_a_branch_tip_after_an_answer_that_extends_our_tip: a branch of equal work is kept. An answer extends our tip. Then an announcement on the branch tip with equal work sends no getheaders, and the next one with more work sends one.
    • extend_branch_tip_keeps_a_tip_with_no_more_work_and_asks_for_more_work_or_a_full_batch: Kept with the work of an empty, an equal and a 1,999-header batch, Ask for more work, and Ask for a full batch of 2,000 headers with less work than ours. Also NotContinuous at the first and at the second header, and ProofOfWork.
    • branch_tip_is_current_while_our_chain_has_our_tip_of_the_time_it_was_kept: current after we extend our tip and after a reorg above our old tip. Not current after a reorg below it, and after restart.
    • 4 chain tests now expect KeptOver with the exact tip. One of them has the bound below the margin.
    • L3 announcement_on_a_branch_that_lost_to_ours_after_a_low_work_sync_gets_no_getheaders: bitcoind reorgs to a branch of 2,951 against our 3,001. elo does a low-work sync of it, and keeps ours. bitcoind mines one block on the branch and announces it with headers. After 2 ping RPCs get their pong, bitcoind got no new getheaders, and elo did not end the connection.
  • Proof:
    • 223 tests pass in a netns, 16 of them L3. nix flake check passes.
    • The new L3 test fails on main: elo ended the connection, with a second low-work sync in one connection. With this PR it passed 10 of 10 alone and 9 of 9 in 3 loops at a time.
    • cargo mutants --in-diff: 31 mutants, 16 caught, 15 unviable, 0 missed. BranchTip::work_max is only in the fuzz build, so it has mutants::skip.
    • Hand mutants, each caught by a test: no branch_tip path, no move of branch_tip after an announcement, no branch_tip on KeptOver, >= in place of > for the work, and Kept in place of the bound below the margin (the fuzz comparison of the tips, on the seed fork-below-margin-short-kept). Two more after the review fixes: no drop of a tip that is not current (branch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tip), and the old clear of branch_tip on each Taken (branch_that_lost_to_ours_stays_a_branch_tip_after_an_answer_that_extends_our_tip).
    • The panic of the first version: a throwaway test with the 4 steps panicked at assert_ne!(tip.hash(), chain.tip().hash()). The test branch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tip is those steps.
    • machines fuzz from the committed corpus, 300 s: the first run found that the model cleared the tip for an announcement on our tip, and elo did not. The model is fixed, and the script is the seed fork-below-margin-short-kept-then-announce-on-tip. After the review fixes: 117,779 runs, no crash.

Why

Remember the branch, not the headers. A tip of 4 values is enough to know that an announcement continues the branch. A header tree (#87) also fixes #88, but #84 put it off until #67. The tip is about 110 bytes for each connection.

Only the proof of work of the announced headers. elo does not keep the headers of the branch, so it cannot check their bits or their time. It does not need to: the headers do not go on our chain. A peer that claims more work makes elo ask for the branch, and the full checks apply then. A peer that claims less work keeps only its own branch out. It could do that by not announcing the branch.

More work than ours: ask with our locator, as on main. With a fork deep below the margin, that answer starts a second low-work sync. That ends the connection with SecondLowWorkSync, and the next connection takes the branch with a new salt. This costs one reconnect, once, when the branch passes ours.

The chain says if a tip is current, not each caller. The first version kept the tip after a reorg onto its branch. The peer could then announce our tip again on the tip, and that moved the tip onto our tip. The next announcement on our tip panicked at assert_ne!(tip.hash(), chain.tip().hash()). The duplicate announcement needs a bad peer, but the next one can be an honest block. The second version cleared the tip on each Taken in chain_next. That held, but only because extend_above is the only path that reorgs. The header of a block that is not valid (#62 to #65) and a chain for more than one peer (#67) are two more paths, and neither goes through chain_next. Now the tip keeps our tip of that time. While our chain has that header, our chain only got more work, and no header of ours can be the tip. So assert!(tip.work_max <= self.work) and the assert_ne! hold for each path that changes the chain. A Taken that only extends our tip now keeps the tip, so the peer does not send the branch again.

A tip that extend_branch_tip keeps has no more work than ours. BranchTip::extend gave a tip with any work, and receive_announcement then compared it with ours. Now the chain makes the decision and gives Ask or Kept. So each BranchTip has a work that is at most ours. A unit test can now also reach the full batch with less work.

A bound below the margin, not the true work. The early stop of extend_above has the work of our chain above the fork only up to the stop. The true work at the fork needs the work of each header of ours above it. Our work less the work up to the stop is not less than the work at the fork, so the tip gets a bound for O(1). A bound that is too high can only make elo ask before the branch has more work, never after. The field is work_max for this reason.

A full announcement asks. An announcement on our tip that is full makes elo ask for the rest, and one on the branch tip now does the same. The answer gets the full checks.

Equal work keeps ours. The branch that elo had first stays, as in extend_above.

For each connection, not in Chain. The pending branch of #84 is also for each connection. A new connection gets the branch again with its first getheaders, but only once, so the loop of #88 ends.

restart does not need a check. restart runs in Handshake::new, before the Peer exists, and a tip lives only in a Peer. A tip after a restart is also not current: our chain has no header at its height.

The L3 test bans elo during the reorg. After invalidateblock, bitcoind connects the branch one block at a time and announces each one. One announcement can arrive before the answer to our getheaders. elo reads it as the answer and ends the connection with NotOnLocator. The Nix sandbox showed this on the first version of the test. The race is also on main, so I filed it as #92. The test now bans 127.0.0.1 while bitcoind reorgs, and elo reconnects with its chain after that.

Not confident:

  • The machines script writes headers only on our locator or on the last header of a download, never on the tip of a branch that lost. So it does not reach an announcement on branch_tip. The driver now compares the two tips after each message, so the model holds the tips that elo keeps and drops. A script base on branch_tip would reach the announcement path. I left that for a next PR, as Forks: more than one branch of headers #22 did with its model.
  • No peer test or L3 test reaches a full announcement on branch_tip with no more work. On regtest each header has the minimum work, and a branch tip has at least our work less the margin, so 2,000 headers always have more work. The unit test of extend_branch_tip reaches it with a tip that a peer cannot make on regtest.
  • Part of #88 or Closes #88? The note on Do not run the low-work sync of a branch again after each announcement #88 about the salt after a low-work sync from genesis is still open.

… announcement on it does not get the branch again
…o ours after a low-work sync gets no getheaders
…s never a header of our chain

After a reorg onto the branch of the tip, a peer could announce our tip
again on the branch tip. The branch tip then moved to our tip, and the
next announcement on our tip panicked at the assertion that the branch
tip is not our tip. Each `Taken` in `chain_next` now clears the branch
tip, in elo and in the `machines` model. A `Taken` that only extends our
tip also clears it, which costs one more batch of the branch when the
peer announces on it again.
…at the fork and the work of the branch, and extend it only with `Headers`

`Chain::branch_tip` took the work of the branch as `work_above_fork`,
the name that `extend_above` gives to the work of our chain above the
fork. It now takes `work_at_fork` and `work_branch`, and `extend_above`
makes its tip with it too, so that each tip has the assertion that its
work is at most ours. `BranchTip::extend` takes `Headers`, so that the
type bounds its loop to 2,000 headers. The test of `BranchTip::extend`
now has a batch that stops being continuous at its second header: the
caller already checks the first header.
… with a work that is not less than the work of the branch

A batch below the margin that was not full gave `Kept`, so each
announcement on its branch got the branch again. The early stop of
`extend_above` has the work of our chain above the fork only up to the
stop, so the work of our chain less that work is not less than the work
at the fork. With the work of the branch on it, that is a bound of the
work of the branch tip, and it is below ours less the margin. A bound
that is too high can only make elo ask before the branch has more work,
never after. The `machines` model makes the same bound, and its
`extend_above` now takes `full` and gives `KeptOver` itself.
… tip of the model after each message

The script reaches an announcement on a branch tip only in a corner, so
the tip of the model showed only through the messages that elo sends.
The driver now compares the hash and the work of the two tips after
each message that elo takes.
…and read the branch tip without taking it

`receive_announcement` asked for an announcement on our tip when it was
full, but not for one on the branch tip. A full announcement with no
more work than ours left the rest of the branch unknown until the next
announcement. It now asks, as for more work, and the full checks of the
answer apply to the headers. On regtest each header has the minimum
work, and a branch tip has at least our work less the margin, so a full
announcement always has more work there, and no test reaches this. On
mainnet, 2,000 headers at the minimum work do not.

`BranchTip` is `Copy`, so `receive_announcement` now reads it in place.
The arm that put back the tip for each other announcement is gone.
… it as elo does, so that the announcement keeps the branch tip

The model took an announcement on our tip through `extend_on_locator`
and `chain_next`, so its `Taken` cleared the branch tip. elo takes it
with `chain.extend` in `receive_announcement`, and keeps the branch tip.
The comparison of the two tips found it in 300 s from the corpus, with
a short batch below the margin and then 2 headers on our tip. That
script is now the seed `fork-below-margin-short-kept-then-announce-on-tip`.
… margin it is a bound and not the true work
…eps the tip or asks, so that each branch tip has no more work than ours
… it, so that `chain_next` has no `full` to assert against the other results
…lo kept the branch, so that the chain and not each caller drops a tip after a reorg
@lorenzolfm
lorenzolfm merged commit 1d66136 into main Nov 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Do not run the low-work sync of a branch again after each announcement

1 participant