Repository navigation
fix: skip a headers that does not start on the locator of our outstanding getheaders, so that an announcement before the answer keeps the connection (#92) - #94
Merged
Conversation
…anding `getheaders` (#92) An announcement can come before the answer to our `getheaders`. elo read it as the answer, and ended the connection with `NotOnLocator`. An answer starts on a hash of our locator, so elo now skips a `headers` that does not. The answer, or the next `getheaders`, gets those headers again. A skip does not restart the headers timeout.
… `getheaders` (#92) bitcoind drops its tip and mines 2 blocks, 10 times. Each time it announces the first block off our tip, and elo asks. Its announcement of the second block can come before the answer. Without the fix, elo ended the connection with `NotOnLocator` in 5 of 5 runs.
…o that an answer from genesis ends the connection at once (#92) A peer that does not have the one hash of such a locator on its chain answers from genesis. elo skipped that answer as an announcement, and the connection ended with `HeadersTimeout` after 2 min. With genesis in the locator, the answer starts on a hash of the locator. The sync then refuses it at once with `NotContinuous`, as before the skip. The model puts genesis below the last hash too, so the script can write an answer on it.
…state moves, so that a skip leaves the state as it is (#92) The skip took the outstanding `getheaders` out of the state and put it back. The check now reads the state in place, and only an answer moves it.
…tOnLocator` (#92) elo now skips a `headers` that does not start on a hash of our locator, so no peer can send such an answer. `Chain::extend_on_locator` and the model found the same headers again and returned `NotOnLocator`. That error can no longer happen, so both now crash on it, as a bug in elo. A Peer test takes the place of the Chain test: a branch on a header in a gap of our locator is skipped.
…nd drop the crash (#92) The `headers` check found the first hash in our locator, and gave only `true` or `false`. `Chain::extend_on_locator` then found the same hash again, and crashed when it did not find it. The two searches agreed only because both read `locator_heights`. Now `Chain::extend_on_locator` gives `None` for headers that do not start on our locator, and changes nothing. For `Download::Chain`, this search is the check. `receive_answer` gives `Answer::Skipped` with the download, and the `Peer` puts it back with the same `sent`. The check of the other downloads moves into `receive_answer`, before the state moves. `receive_on_genesis` uses `Chain::extend`: the check already found that the answer starts on genesis, which is our tip. Its match on `Extended` and its crash go. The model finds the height of the first hash in the check, and gives it to its `extend_on_locator`, so the crash of the model goes too. Removed: 3 crashes, the match on `Extended` in `receive_on_genesis`, the second search of our locator, and the `if let` before the match on the state. Added: `Answer`, with 2 variants.
…n it changed before the answer (#92) The check of a `headers` makes our locator again. It is the locator that we sent only while nothing writes the chain while our `getheaders` is out. Only a convention holds this: `receive_inv` and `receive_announcement` act only after the sync. A change that takes an announcement while our `getheaders` is out breaks it, and the check then reads the wrong locator with no error. `SyncState::Outstanding` now keeps the hash of our tip when we send the `getheaders`. `receive_headers` crashes when our tip is a different one. To keep the locator itself is not sufficient: `extend_on_locator` reads the chain, not the locator that we sent. The hash makes `Peer` 32 bytes larger, and `Step::Done` is then 216 bytes larger than `Step::Reply`, above the limit of `clippy::large_enum_variant`. `Step::Done` now holds a `Box<Peer>`: one allocation for each connection.
… the sync (#92) After the change to `Chain::extend_on_locator`, only the downloads that are not `Download::Chain` use `Download::is_answered_by`. No test that runs without `cfg(fuzzing)` sent such a download headers off its locator, so cargo-mutants did not kill `is_answered_by -> true`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #92. The first option of #92: skip an announcement while our
getheadersis out.What
Download::locator: the locator of thegetheadersfor each download.SyncState::ticknow uses it. The locator of a branch or a low-work sync is its last hash, then genesis.Download::is_answered_by: for each download that is notDownload::Chain, aheadersis the answer when it is empty, or when its first header is on a hash of that locator.receive_answerchecks the answer before the state moves. Aheadersthat is not the answer givesAnswer::Skippedwith the download.Peer::receive_headersputs it back with the samesentand givesNoReply. The state, the chain andbranch_tipdo not change, and the headers timeout does not restart.Chain::extend_on_locatorgivesNonefor a batch that is not on our locator, and changes nothing. ForDownload::Chain, this search is the check.ExtendError::NotOnLocatoris gone.receive_on_genesisusesChain::extend: the check found that the answer starts on genesis, which is our tip. Its match onExtendedand its crash are gone.SyncState::Outstandingkeeps the hash of our tip when we send thegetheaders.receive_headerscrashes when our tip changed before the answer.Step::Doneholds aBox<Peer>, forclippy::large_enum_variant.machinesskips in the same way, withPeerModel::locator. Its locator of a branch or a low-work sync has genesis below the last hash. The check finds the height of the first hash and gives it to itsextend_on_locator.headers-off-tip,headers-off-tip-after-full,headers-off-tip-after-sync-twice,sync-low-work-full-off-tip-signetandsync-low-work-regtest-second-off-last.headers-off-tip-after-sync-twice-then-answer: the steps of An announcement that arrives before the answer to ourgetheadersends the connection #92. An announcement off our tip, one more off our locator, then the answer ends the sync.sync-low-work-regtest-first-on-genesis: the second batch of a low-work sync starts on genesis. elo ends the connection at once withHeadersSync(NotContinuous).sync-low-work-regtest-second-break-second: the second batch of a low-work sync breaks at its second header.HeadersSync(NotContinuous)had a seed only through a first header off the last one, and that is now a skip.headers_off_our_locator_while_our_getheaders_is_outstanding_are_skippedreplacesheaders_off_our_tip_in_answer_to_our_getheaders_end_the_connection. It has the same 2 peers, one in the first sync and one after an announcement off our tip. Each skips theheaders, keeps its chain, and then takes the answer.headers_off_our_locator_do_not_restart_the_headers_timeout.headers_on_a_header_in_a_gap_of_our_locator_are_skippedreplaces the Chain testextend_on_locator_refuses_a_batch_on_a_header_that_is_not_in_the_locator. That Chain test is nowextend_on_locator_takes_no_batch_on_a_header_that_is_not_in_the_locator: its row in the gap givesNoneand keeps the tip.headers_off_the_locator_of_a_low_work_sync_are_skipped: a low-work sync skips a batch off its locator and then ends. It killsis_answered_by -> truewithoutcfg(fuzzing).low_work_sync_ends_the_connection_at_once_on_an_answer_from_genesisandbranch_download_ends_the_connection_at_once_on_an_answer_from_genesis.refused_answer_keeps_the_chainloses its row forNotOnLocator.low_work_sync_that_refuses_a_batch_ends_the_connectionnow breaks at the second header.announcement_before_the_answer_to_our_getheaders_keeps_the_connection: 10 times, bitcoind drops its tip withinvalidateblockand mines 2 blocks with one RPC. It announces the first block, which is off our tip, so elo asks. Its announcement of the second block can come before the answer. At the end, elo has the tip of bitcoind and the connection.announcement_on_a_branch_that_lost_to_ours_after_a_low_work_sync_gets_no_getheadershas no ban.settledoespingRPCs until bitcoind sends no moreheadersorinvand gets no moregetheaders. Then the test checks that elo logged no newtip.elo-test(229 tests, L3 included),elo-clippy,elo-fuzz,elo-fmtandelo-commentspass in Nix. A thermo-nuclear review of c06bb4a has no findings.--cfg fuzzing. clippy passes with and without--cfg fuzzing, and so docargo fmt --checkandelo-comments. I did not runnix flake check,cargo mutantsor the fuzz again.Chain::extend_on_locator.nix flake checkpasses. The 204 model and seed-table tests pass with--cfg fuzzing.headers do not start at a header of our locator. With the skip: 5 of 5 alone, 30 of 30 in 3 loops at a time, and in the Nix sandbox.invin its reorg, so no announcement came before an answer. So that test does not prove An announcement that arrives before the answer to ourgetheadersends the connection #92. The new L3 test does.cargo mutants --in-diff: 10 mutants, 4 caught, 6 unviable, 0 missed.machinesfuzz from a copy of the committed corpus, 300 s: 109,240 runs, no crash.Why
Skip, not read each
headersby its content. Option 2 of #92 needs elo to process an announcement in the middle of a download. A download can be a branch or a low-work sync, so an announcement on our tip would change the chain under it. Option 1 needs no new state. The answer, or the nextgetheaders, gets the skipped headers again.An answer starts on our locator. bitcoind sends the headers after the first hash of the locator on its chain (
net_processing.cpp:4441,validation.cpp:127). An announcement starts on a header that the peer knows we have (net_processing.cpp:5880). That is our tip or a header that it sent us. So the check uses only the hashes that elo sent, and no state about the peer.An announcement on the locator is read as the answer. For example, an announcement on our tip while our
getheaderswith our tip is out. This is the overlap of the two options. The answer of bitcoind starts on the same header or one below, so the announcement is a correct start of an answer. elo takes it, and the real answer then comes as an announcement. It is not on our new tip, so elo asks again. This costs one round trip and no error.Compute the locator again, do not keep it. The chain does not change while a
getheadersis out, soDownload::locatorgives the same hashes as thegetheaders. A copy inOutstandingwould be a second source for the same value.Outstandingkeeps only our tip, for the assert. The cost is one locator for eachheaders, at most about 40 hashes.A skipped branch-tip move is lost. An announcement on
branch_tipwhile ourgetheadersis out is skipped, andbranch_tipstays. The answer then has the branch, so it setsbranch_tipagain. The model skips in the same way, so the driver can compare the tips after each message.Genesis in a locator of one hash. If a peer left the branch of that hash, it answers from genesis (
validation.cpp:144). Without genesis in the locator, elo skipped that answer, and the connection ended withHeadersTimeoutafter 2 min. With genesis, the answer is on the locator, and the sync refuses it at once withNotContinuous, as onmain. Genesis is not added when the last hash is genesis. The other way was to always read aheaderson genesis as the answer. That keeps the oldgetheaderson the wire, but then the check accepts a hash that elo did not send.For
Download::Chain, the chain does the check. A first version checked the locator inis_answered_by, andextend_on_locatorthen found the same hash again and crashed when it did not find it. The two searches agreed only because both readlocator_heights. Nowextend_on_locatorgivesNoneoff the locator, so there is one search and no crash.Keep our tip in
Outstanding. The check makes our locator again. That is the locator that we sent only while nothing writes the chain while ourgetheadersis out. Only a convention holds this:receive_invandreceive_announcementact only after the sync. The hash of our tip and the assert make a break of that convention a crash, not a check against the wrong locator. To keep the locator itself is not sufficient:extend_on_locatorreads the chain, not the locator that we sent.No event for a skip, and no limit on the count. The events are for each connection, and a skip is one message. The timeout limits the skips to 2 min after each
getheaders, and a skip does no proof-of-work check. A limit on the count is a new rule for peers, so I did not add one.A skip does not restart the timeout. A peer that sends only headers off our locator still gets
HeadersTimeout2 min after ourgetheaders.Not confident:
getheadersof a branch or a low-work sync has a second hash now. The L3 low-work tests pass with it, but no L3 test makes bitcoind answer from genesis.