Repository navigation
install-phone: the next run removes the verify directory a killed run left - #682
Conversation
… left Since derekross#648 the shipped ramdisk is read in a verify.XXXXXX directory of the run's own in --work, removed when the run exits. A run killed outright (SIGKILL, a closed terminal's kill, a crash) never exits that way, so its directory, with the decompressed ramdisk, stayed in --work for good, one more for every such run. A run can't tell a dead run's directory from a live one's by age (an install can wait at the codename prompt for hours), and a full install needs its directory until it exits. So each run holds a shared flock on --work itself (a descriptor on the directory, no file of its own there) until it exits, and marks its directory only once it holds the lock. A run that gets the lock exclusively, so no other run of this installer is live, first removes the marked directories: their runs have all ended. A directory not yet marked, or left by an installer from before this change (whose run may still be going), is never removed. flock comes from python3's fcntl, which the installer already needs; flock(1) is util-linux, not in its tool list, and the bundle ships install alone, so the helper is inline (L-497). Where --work can't be locked (some network file systems) nothing is removed and nothing is marked. Tested: tests/install-phone-initramfs gains the case (an install waiting at the codename, a second one killed with SIGKILL there, --verify-only beside the waiting one, the waiting one then flashing the key's append, and a later --verify-only alone), which fails with main's tool at "a run with no other live removes a killed install's directory"; mutants without the exclusive check, with the shared lock let go after marking, without the mark check, or with no mark written each fail it, and a blocking exclusive lock hangs it. Both install-phone suites pass, the initramfs one under umask 077, 022 and 002; tools/pr-check origin/main; tests/all. A bundle's ./install --unsigned, killed during --verify-only's read and at the codename prompt, then run again: the leftovers pile up on main and are gone on this branch, and an install waiting at the codename keeps its directory and flashes. Not tested: on a phone (laptop-side only); a network-mounted home; a run killed between creating its directory and marking it (its directory stays, as on main). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, @only21mil. That is the first of the two things I noted on #648, and you took the hard part seriously: age can't tell a dead run's directory from one waiting at the codename prompt, so a lock that dies with the run is the right witness. Marking the directory only once the shared lock is held, and sweeping only marked ones, is what makes the non-atomic lock change harmless. Thank you also for saying plainly what you didn't test. I checked it merged onto today's main: On the case you name as untested, a shared home whose Two small things for some later change, not this one:
Merging. |
|
Note 🤖 Opus 5.5 replying on behalf of Victor. #699 quiets the job notice. On its head the suite exited 0 with no "Killed" line, and the same run on main printed it. I closed the held job's stderr to #700 adds one sentence to the |
#699) Since #682 a passing run printed bash's job notice for the install the suite kills, which reads as a failure in a CI log. The notice comes from the held job's own shell, so that job's stderr now goes to `/dev/null` (its output is in its OUT file as before, which the suite prints on a failure) and the `wait` is plain. Keep the `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>` trailer.
…ory (#700) Since #682 a run that finds no other run of the installer live removes the `verify.*` directories killed runs left in `--work`, but `--help` described `--work` only as where the image copy goes. The entry now says so, and that directories left by installers without this cleanup are kept. Header comment only; the code is unchanged. Keep the `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>` trailer.
… left (derekross#682) A run killed outright never reaches its exit trap, so its verify.XXXXXX (the decompressed ramdisk) stayed in --work for good. Each run now holds a shared flock on --work until it exits and marks its directory once it holds it; a run that gets the lock exclusively, so no other run is live, first removes the marked directories. An unmarked directory, and anything while another run is live, is never removed; where --work can't be locked nothing is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What and why
This follows the maintainer's note on #648: a run killed outright leaves its
verify.XXXXXXbehind, and nothing sweeps old ones.Since #648 the shipped ramdisk is read in a
verify.XXXXXXdirectory of the run's own in--work, removed when the run exits. A run killed with SIGKILL, a crash or a power cut never gets there, so its directory (the decompressed ramdisk) stays for good, one more per such run.Age can't tell a dead run's directory from a live one's: an install can wait at the codename prompt for hours, and it needs its directory until it exits. So each run now takes a shared
flockon--workitself (a descriptor on the directory, so nothing new is left in--work) and keeps it until it exits. It marks its own directory only once it holds that lock. A run that gets the lock exclusively, so no other run of this installer is live, first removes the marked directories: their runs have all ended. These are never removed: a directory not yet marked, one left by an installer from before this change (its run may still be going), and anything while another run is live. The lock comes from python3'sfcntl, which the installer already needs.flock(1)is util-linux and not in its tool list, and the bundle shipsinstallalone (L-497). The run sweeps only after acquiring the exclusive lock and marks its directory only after acquiring the shared lock. All of this assumes aflockthat really excludes, on every host that shares the work directory.Children inherit the descriptor, so a directory stays while any process of its run lives. gpg, whose agent can outlive it, runs before the lock is taken. Leaving the unmarked directories of today's installer alone is my choice; sweeping them would mean a still-running install from before this change could lose its directory.
I first reproduced this on main e6bc8e3 before writing the fix. The first pair below repeats that original recipe: SIGKILL goes only to the installer process. At the codename prompt, its answer-reading child survives and retains the inherited lock descriptor, so the candidate leaves that directory while the child remains alive. Step 2 removes step 1's directory, but its surviving child prevents every later sweep, leaving both step 2's and step 4's directories at the end.
I then changed the fault to kill the installer and all its descendants. The supplemental control repeats that revised recipe on main and b4cb0f9 after the fix was written; it is a post-fix control.
Both recipes run the bundle’s real
./installin bwrap with the checkout mounted at /mnt. Fastboot is a stand-in. The script removes colour escapes and selects install output withgrep -E '==>|error'; parenthesized lines are the script’s notes.Before, main e6bc8e3, the original recipe (SIGKILL to the installer process only), taken before I wrote the fix:
After, this branch b4cb0f9, the original recipe (SIGKILL to the installer process only):
As a post-fix control, I used a recipe written after the fix, sending SIGKILL to the installer and all its descendants: the listings contain
1 2 2 3 3 | 4 4 3verify.*directories on main e6bc8e3 and1 1 0 1 0 | 1 1 0on b4cb0f9; in both runs, the waiting install still flashes its six partitions.Regression test (a stand-in, not the reproduction): in
tests/install-phone-initramfs, an install waits at the codename and a second one is killed there with SIGKILL.--verify-onlybeside the waiting one removes nothing. The waiting one then flashes the hook'svendor_boot.imgbyte for byte, which its key append reads from its directory. A later--verify-onlyon its own removes the killed one's directory and leaves an unmarked one. With main's tool the test fails at "a run with no other live removes a killed install's directory, and nothing unmarked". Each of these mutants also fails it: no exclusive check, the shared lock let go after marking, no mark check, no mark written, or a blocking exclusive lock. No test covers marking before locking (a race).The body is longer than usual because of the two primary transcripts.
Device and build
uname -r): not applicableChecks (no phone needed)
tools/pr-check origin/mainpassestests/allpasses (tests/install-phone-initramfs, with its byte comparisons, also under umask 077, 022 and 002)tests/packagespasses (or the packages I touched: ...): not run, no package changedversionbumped, and the new version is in the commit subject: no package changedCo-Authored-By:orAssisted-by:trailer, never aSigned-off-by:On the phone
userdata, weakens the storage guard, or reads registers from userspace (AGENTS.md §1)tools/phone-smokereportNot applicable: a laptop-side installer change; nothing on the phone changes.
Generated with Claude Opus 5.5 in T3 Code Nightly