Skip to content

fix(#5): exclusive takeover of stale build locks; status ignores dead owners - #7

Open
Packi1992 wants to merge 1 commit into
open-gsd:mainfrom
Packi1992:fix/5-stale-build-lock
Open

Packi1992 wants to merge 1 commit into
open-gsd:mainfrom
Packi1992:fix/5-stale-build-lock

Conversation

@Packi1992

Copy link
Copy Markdown

Problem

acquireBuildLock already takes over a .build.lock whose owner pid is gone, or which is older than 15 minutes. A plain sync after a SIGKILLed run therefore goes through on main. Around that path, four things were wrong:

  1. EPERM counted as dead. isPidAlive treated every kill(pid, 0) error as dead. A lock held by a live process of another user (EPERM) was taken over.
  2. Takeover was not exclusive. Taking over meant unlink, then create. Several processes that saw the same stale lock could each unlink the fresh lock another one had just created, and all proceed. With 8 processes racing for one dead-owner lock, main let 2 to 5 of them through in every one of 20 trials.
  3. No notice. Nothing said a lock had been taken over.
  4. status() / graph_status reported build_in_progress: true for any existing lock file, including one left by a process that no longer exists. That is what made the leftover lock look like a blocker.

Fix

  • Owner liveness. lockOwnerState(pid) treats ESRCH (or an invalid pid) as dead. EPERM, or any other error, counts as held.
  • Exclusive takeover.
    • A process that finds a stale lock must first create a claim file named after that exact lock content (.build.lock.takeover-<hash>-<gen>, openSync(…, 'wx')).
    • Only the claim holder re-checks that the lock still has that content, then replaces it. A contender acting on an outdated view finds different content and backs off.
    • A claim left by a claimant that died (ESRCH) is skipped by claiming the next generation, so a crash during takeover cannot wedge the lock again.
    • Claim files are removed afterwards.
  • Notice. After a takeover, one line goes to stderr, for example [gsd-graph] taking over stale build lock <path> (pid 1234 is not running; started … by cli). A new onNotice option overrides it.
  • Unreadable locks. These were stolen immediately before. They now use the same exclusive takeover, but only once they are older than 5 s. A fresh empty lock is a new lock whose payload is still being written, so it counts as held.
  • Age rule. The 15-minute rule is unchanged.
  • Status. status() reports build_in_progress: false when the recorded owner pid no longer exists. A live owner still reports true, even past 15 minutes, because the build may simply be long-running.
  • Docs. The build_locked troubleshooting row in docs/FULL-GUIDE.md is updated.

Tests

tests/lock.test.ts:

  • Lock of an exited process: the lock is written with the pid of a child that ran and was reaped. The takeover succeeds with exactly one one-line notice, and no claim files are left.
  • Lock of a running process: the pid of a live child is still refused, with no notice. After the child is killed, the lock is taken over.
  • EPERM: pid 1 counts as alive, so the lock is refused. Skipped on Windows.
  • Invalid recorded pids count as dead.
  • Unreadable lock: held while fresh, taken over once its mtime is old.
  • Outdated view: a contender acting on the outdated stale content backs off and leaves the new lock alone.
  • Claims: a live claim blocks other contenders, and a dead claimant's claim is skipped.
  • Concurrency: 8 processes race for one stale lock and exactly one wins. Against main's lock code, the same setup gave 2 to 5 winners in 20 of 20 trials; with this change, exactly 1 in 20 of 20.

tests/status.test.ts: build_in_progress is false for an exited owner pid and true for a live one.

npm run build and npm test (438/438) pass on Node 22 and 24; npm run test:coverage passes on Node 24.

Fixes #5

🤖 Generated with Claude Code

…res dead owners

acquireBuildLock already took over a .build.lock whose pid was gone, but:

- the owner probe counted every kill(pid, 0) error as "dead", so a lock
  held by a live process of another user (EPERM) was taken over;
- takeover was unlink-then-create, so several processes that saw the same
  stale lock could each remove the fresh lock another had just created and
  all proceed (8 concurrent contenders: 2-5 winners in every trial);
- nothing said a lock had been taken over;
- status() reported build_in_progress: true for any existing lock file,
  including one left by a process that no longer exists.

Now ESRCH means dead and EPERM (or any other error) means held. Takeover
goes through an exclusive claim file per stale lock instance; only the
claim holder re-checks the lock content and replaces it, and a claim left
by a dead claimant is skipped. A one-line notice goes to stderr
(onNotice overrides it). Unreadable locks use the same takeover, but only
once they are older than 5 s, since a fresh one is still being written.
The 15-minute age rule is unchanged. status() reports build_in_progress:
false when the recorded owner pid no longer exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale .gsd-graph/.build.lock with a dead pid blocks every later sync until removed by hand

1 participant