Repository navigation
fix: keep the tip of a branch with no more work than ours, so that an announcement on it does not get the branch again (#88) - #93
Merged
Merged
Conversation
… announcement on it does not get the branch again
…o ours after a low-work sync gets no getheaders
…s never a header of our chain After a reorg onto the branch of the tip, a peer could announce our tip again on the branch tip. The branch tip then moved to our tip, and the next announcement on our tip panicked at the assertion that the branch tip is not our tip. Each `Taken` in `chain_next` now clears the branch tip, in elo and in the `machines` model. A `Taken` that only extends our tip also clears it, which costs one more batch of the branch when the peer announces on it again.
…at the fork and the work of the branch, and extend it only with `Headers` `Chain::branch_tip` took the work of the branch as `work_above_fork`, the name that `extend_above` gives to the work of our chain above the fork. It now takes `work_at_fork` and `work_branch`, and `extend_above` makes its tip with it too, so that each tip has the assertion that its work is at most ours. `BranchTip::extend` takes `Headers`, so that the type bounds its loop to 2,000 headers. The test of `BranchTip::extend` now has a batch that stops being continuous at its second header: the caller already checks the first header.
… with a work that is not less than the work of the branch A batch below the margin that was not full gave `Kept`, so each announcement on its branch got the branch again. The early stop of `extend_above` has the work of our chain above the fork only up to the stop, so the work of our chain less that work is not less than the work at the fork. With the work of the branch on it, that is a bound of the work of the branch tip, and it is below ours less the margin. A bound that is too high can only make elo ask before the branch has more work, never after. The `machines` model makes the same bound, and its `extend_above` now takes `full` and gives `KeptOver` itself.
… tip of the model after each message The script reaches an announcement on a branch tip only in a corner, so the tip of the model showed only through the messages that elo sends. The driver now compares the hash and the work of the two tips after each message that elo takes.
…and read the branch tip without taking it `receive_announcement` asked for an announcement on our tip when it was full, but not for one on the branch tip. A full announcement with no more work than ours left the rest of the branch unknown until the next announcement. It now asks, as for more work, and the full checks of the answer apply to the headers. On regtest each header has the minimum work, and a branch tip has at least our work less the margin, so a full announcement always has more work there, and no test reaches this. On mainnet, 2,000 headers at the minimum work do not. `BranchTip` is `Copy`, so `receive_announcement` now reads it in place. The arm that put back the tip for each other announcement is gone.
… it as elo does, so that the announcement keeps the branch tip The model took an announcement on our tip through `extend_on_locator` and `chain_next`, so its `Taken` cleared the branch tip. elo takes it with `chain.extend` in `receive_announcement`, and keeps the branch tip. The comparison of the two tips found it in 300 s from the corpus, with a short batch below the margin and then 2 headers on our tip. That script is now the seed `fork-below-margin-short-kept-then-announce-on-tip`.
… margin it is a bound and not the true work
…eps the tip or asks, so that each branch tip has no more work than ours
… it, so that `chain_next` has no `full` to assert against the other results
…lo kept the branch, so that the chain and not each caller drops a tip after a reorg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #88. The second option of #88: remember the last branch that elo checked.
What
chain::BranchTip { hash, work_max, ours_height, ours_hash }: the last header of a branch that passed the checks of the chain, a work that is not less than the chain work of the branch at that header, and the height and hash of our tip when elo kept the branch.Extended::KeptOver(BranchTip): our chain stays, and the batch ended a branch with no more work than ours. Three places give it, and all giveKeptonmain:extend_above, for a batch that is not full, when the work of the branch is at most ours.extend_above, for a batch below the margin that is not full. The work is a bound: not less than the work of the branch, and below ours less the margin.extend_on_branch, for an empty batch after a full batch of the branch. The work at the fork comes fromwork_at.Chain::branch_tip(work_at_fork, work_branch, hash)makes each tip, with the assertion that its work is at most ours.Chain::branch_tip_current: a tip is current while our chain has the header atours_heightwithours_hash. A reorg below that height, or arestart, makes it not current.Extended::Taken { full }:Takenis the only result that readsfull, sochain_nexthas nofullparameter, and its 3 assertions onfullare gone.Peer.branch_tip: Option<BranchTip>.chain_nextsets it onKeptOver. The connection end drops it.receive_announcementfirst drops abranch_tipthat is not current. If the first header is onbranch_tip,Chain::extend_branch_tipchecks that each header is on the one before it and has its proof of work, and adds the work of the headers. It givesAnnounced::Askif the branch has more work than ours or the announcement is full. Then agetheaderswith our locator is due, as onmain. If not, it givesAnnounced::Keptwith the tip on the last header, and elo sends nothing. Other announcements leavebranch_tipas it is.extend_branch_tiptakesHeaders, so that the type bounds its loop to 2,000 headers.machineskeeps the same tip:branch_tip: Option<(Hash, u128, usize)>,ExtendedModel::KeptOver,below_margin_work_maxandreceive_off_tip. A reorg in the model drops the tip when the fork is below the height of the tip. The driver compares the current tip of elo with the tip of the model after each message.fork-below-margin-short-kept-then-announce-on-tipin the corpus and inSEED_ENDS.announcement_on_a_branch_that_lost_to_ours_asks_for_it_only_with_more_work_than_ours: after a low-work sync of a branch that loses, 2 announcements on the branch and 1 on our tip send nogetheaders. The announcement that gives the branch more work than ours sends one.branch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tip: a branch of equal work is kept, then gets one more header and takes our chain. The peer announces our tip again on the old branch tip, and elo asks with its locator. The next announcement on our tip extends it.branch_that_lost_to_ours_stays_a_branch_tip_after_an_answer_that_extends_our_tip: a branch of equal work is kept. An answer extends our tip. Then an announcement on the branch tip with equal work sends nogetheaders, and the next one with more work sends one.extend_branch_tip_keeps_a_tip_with_no_more_work_and_asks_for_more_work_or_a_full_batch:Keptwith the work of an empty, an equal and a 1,999-header batch,Askfor more work, andAskfor a full batch of 2,000 headers with less work than ours. AlsoNotContinuousat the first and at the second header, andProofOfWork.branch_tip_is_current_while_our_chain_has_our_tip_of_the_time_it_was_kept: current after we extend our tip and after a reorg above our old tip. Not current after a reorg below it, and afterrestart.KeptOverwith the exact tip. One of them has the bound below the margin.announcement_on_a_branch_that_lost_to_ours_after_a_low_work_sync_gets_no_getheaders: bitcoind reorgs to a branch of 2,951 against our 3,001. elo does a low-work sync of it, and keeps ours. bitcoind mines one block on the branch and announces it withheaders. After 2pingRPCs get theirpong, bitcoind got no newgetheaders, and elo did not end the connection.nix flake checkpasses.main:elo ended the connection, witha second low-work sync in one connection. With this PR it passed 10 of 10 alone and 9 of 9 in 3 loops at a time.cargo mutants --in-diff: 31 mutants, 16 caught, 15 unviable, 0 missed.BranchTip::work_maxis only in the fuzz build, so it hasmutants::skip.branch_tippath, no move ofbranch_tipafter an announcement, nobranch_tiponKeptOver,>=in place of>for the work, andKeptin place of the bound below the margin (the fuzz comparison of the tips, on the seedfork-below-margin-short-kept). Two more after the review fixes: no drop of a tip that is not current (branch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tip), and the old clear ofbranch_tipon eachTaken(branch_that_lost_to_ours_stays_a_branch_tip_after_an_answer_that_extends_our_tip).assert_ne!(tip.hash(), chain.tip().hash()). The testbranch_that_lost_to_ours_and_then_took_our_chain_is_no_longer_a_branch_tipis those steps.machinesfuzz from the committed corpus, 300 s: the first run found that the model cleared the tip for an announcement on our tip, and elo did not. The model is fixed, and the script is the seedfork-below-margin-short-kept-then-announce-on-tip. After the review fixes: 117,779 runs, no crash.Why
Remember the branch, not the headers. A tip of 4 values is enough to know that an announcement continues the branch. A header tree (#87) also fixes #88, but #84 put it off until #67. The tip is about 110 bytes for each connection.
Only the proof of work of the announced headers. elo does not keep the headers of the branch, so it cannot check their bits or their time. It does not need to: the headers do not go on our chain. A peer that claims more work makes elo ask for the branch, and the full checks apply then. A peer that claims less work keeps only its own branch out. It could do that by not announcing the branch.
More work than ours: ask with our locator, as on
main. With a fork deep below the margin, that answer starts a second low-work sync. That ends the connection withSecondLowWorkSync, and the next connection takes the branch with a new salt. This costs one reconnect, once, when the branch passes ours.The chain says if a tip is current, not each caller. The first version kept the tip after a reorg onto its branch. The peer could then announce our tip again on the tip, and that moved the tip onto our tip. The next announcement on our tip panicked at
assert_ne!(tip.hash(), chain.tip().hash()). The duplicate announcement needs a bad peer, but the next one can be an honest block. The second version cleared the tip on eachTakeninchain_next. That held, but only becauseextend_aboveis the only path that reorgs. The header of a block that is not valid (#62 to #65) and a chain for more than one peer (#67) are two more paths, and neither goes throughchain_next. Now the tip keeps our tip of that time. While our chain has that header, our chain only got more work, and no header of ours can be the tip. Soassert!(tip.work_max <= self.work)and theassert_ne!hold for each path that changes the chain. ATakenthat only extends our tip now keeps the tip, so the peer does not send the branch again.A tip that
extend_branch_tipkeeps has no more work than ours.BranchTip::extendgave a tip with any work, andreceive_announcementthen compared it with ours. Now the chain makes the decision and givesAskorKept. So eachBranchTiphas a work that is at most ours. A unit test can now also reach the full batch with less work.A bound below the margin, not the true work. The early stop of
extend_abovehas the work of our chain above the fork only up to the stop. The true work at the fork needs the work of each header of ours above it. Our work less the work up to the stop is not less than the work at the fork, so the tip gets a bound for O(1). A bound that is too high can only make elo ask before the branch has more work, never after. The field iswork_maxfor this reason.A full announcement asks. An announcement on our tip that is full makes elo ask for the rest, and one on the branch tip now does the same. The answer gets the full checks.
Equal work keeps ours. The branch that elo had first stays, as in
extend_above.For each connection, not in
Chain. The pending branch of #84 is also for each connection. A new connection gets the branch again with its firstgetheaders, but only once, so the loop of #88 ends.restartdoes not need a check.restartruns inHandshake::new, before thePeerexists, and a tip lives only in aPeer. A tip after arestartis also not current: our chain has no header at its height.The L3 test bans elo during the reorg. After
invalidateblock, bitcoind connects the branch one block at a time and announces each one. One announcement can arrive before the answer to ourgetheaders. elo reads it as the answer and ends the connection withNotOnLocator. The Nix sandbox showed this on the first version of the test. The race is also onmain, so I filed it as #92. The test now bans127.0.0.1while bitcoind reorgs, and elo reconnects with its chain after that.Not confident:
machinesscript writes headers only on our locator or on the last header of a download, never on the tip of a branch that lost. So it does not reach an announcement onbranch_tip. The driver now compares the two tips after each message, so the model holds the tips that elo keeps and drops. A script base onbranch_tipwould reach the announcement path. I left that for a next PR, as Forks: more than one branch of headers #22 did with its model.branch_tipwith no more work. On regtest each header has the minimum work, and a branch tip has at least our work less the margin, so 2,000 headers always have more work. The unit test ofextend_branch_tipreaches it with a tip that a peer cannot make on regtest.Part of #88orCloses #88? The note on Do not run the low-work sync of a branch again after each announcement #88 about the salt after a low-work sync from genesis is still open.