Replace VERSION instead of truncating it - #851
Merged
Merged
Conversation
`create_version_file` opened VERSION with `File::create`, so the previous contents were gone before the new ones were written, and nothing was flushed afterwards. The two writes that matter run once a migration chain has already rewritten the database, and the next start parses this file to decide whether to migrate again. A power loss around the write leaves it empty or partial, and the database then fails to open on an error naming nothing but the symptom, with no way back except editing a file in the data directory by hand. Staging through a temporary in the same directory and renaming makes the replacement atomic. Flushing the temporary and then the directory keeps the rename from reaching disk ahead of the bytes it publishes, which is what would otherwise leave a migrated database recorded as the version it held before the migration. The temporary asks for no particular mode, so VERSION keeps the umask-derived permissions it has always had. The temporary's name is fixed rather than unique because `migrate_data_dir` runs once at startup with exclusive access to the directory, so nothing can collide with it, and a single recognizable leftover is one `retrieve_or_create_version` can forgive when deciding whether a data directory is empty. Without that, an interrupted first write would make a fresh directory look populated and VERSION would never be created at all. Closes #849
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #851 +/- ##
=======================================
Coverage 82.69% 82.70%
=======================================
Files 88 88
Lines 33042 33083 +41
=======================================
+ Hits 27325 27361 +36
- Misses 5717 5722 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #849.
create_version_filetruncated VERSION in place and flushed nothing, against the atomic-write rule inAGENTS.md. It now writesVERSION.tmpin the same directory,sync_alls it, renames it over VERSION, andsync_alls the containing directory.What this is and is not worth
The exposure is narrow, and the change is sized to match. Only two of the three call sites can truncate anything —
retrieve_or_create_versionwrites into a directory it has just found empty — and those two run once per deployment per release that changes the database format. A process crash cannot produce the bad state either, since the truncate and the write both sit in the page cache; it takes a power loss or a kernel panic landing in a window of microseconds.What justifies the twenty lines is the cost when it does happen: an empty VERSION fails
Version::parseon the next start, and there is no automated way back from that — someone edits a file in a production data directory while the database will not open.Decisions worth reviewing
VERSION.tmp) rather than the timestamped nameClassifierFs::store_classifieruses.migrate_data_dirruns once at startup with exclusive access, so there is no concurrent writer to collide with, and a fixed name bounds what an interrupted write can leave behind to one file the next start recognizes.store_classifier's0o600. VERSION records a format version, not a secret. Sincerenameinstalls the temporary's inode,File::createkeeps exactly the umask-derived permissions VERSION has today — so nothing an operator can observe changes, and noCHANGELOGentry is due. (The entry Set the classifier temp file's mode and flush it before rename #844 added was for the permission change it made, not for the flush.)File::open(dir)followed bysync_allwas verified to work on macOS as well as Linux before being relied on.retrieve_or_create_versionforgives onlyVERSION.tmpwhen testing whether a data directory is empty. A leftover from an interrupted first write would otherwise make a fresh directory look populated, so VERSION would never be created and the start would fail on "cannot open VERSION" instead. The test was not replaced with a check for VERSION's existence: a populated directory without VERSION stays an error, because creating one there would stamp the current version onto an existing tree and skip its migration.populated_dir_without_version_is_rejectedpins that.Deliberately not addressed
Both are named in the issue; neither is created or worsened here.
backupanddataare two files in two directories. A crash between the two writes leaves them disagreeing and the next start stops on "mismatched database version". Per-file atomicity cannot close that.Test plan
cargo fmt -- --check --config group_imports=StdExternalCratecargo clippy --bins --tests --all-features -- -D warningscargo test --all-features— 510 passed, 9 ignored (pre-existing), 0 failedversion_file_is_replaced,stale_version_temp_does_not_block_creation,populated_dir_without_version_is_rejectedNo test here proves crash safety. The ordering the flushes buy cannot be observed without fault injection, which is out of proportion to this change; the tests cover the states that are reachable, and the doc comment on
version_file_is_replacedsays so rather than letting the name imply more.