Skip to content

shared RPC host: pruning a pointer that names a dead generation leaves a live registered generation ownerless, so handoff refuses with unknown_owner #1911

Description

@code-yeongyu

Summary

pruneDeadGenerations (#1899) removes the daemon pointer (<layout>/host.pid) when it names a dead generation, but never re-points it at a LIVE registered generation that still serves the socket. Every subsequent readHostRegistration returns undefined, so handoffHost refuses with unknown_owner and host status cannot name the current generation - while a healthy host is serving the socket.

Reproduction (observed 2026-09-21 on a production daemon dir)

  1. Generation A dies (here: Bun SIGSEGV); the pointer still names A.
  2. A client of an OLDER build starts generation B (writeHostRegistration writes generations/B/host.pid; the pointer write is expected but the pointer ended up naming A - the older build's registration and B's start interleaved; exact ordering held locally).
  3. Any host status from a build carrying fix(rpc): drain a superseded host generation and prune dead daemon records #1899 runs pruneDeadGenerations: pointer names dead A -> pointer removed. generations/B/host.pid (pid alive, processStartTime matches) remains, nothing points at it.
  4. daemon handoff --json -> {"action":"refuse","reason":"unknown_owner"} although B is reachable (host status reachable: true, generations[] lists B alive: true).

Repair that worked: writing host.pid = { layout: 2, instance_id: B, generation_dir: "generations/B", writer: <B's writer> } by hand; the next daemon handoff produced generation 1 and B began draining.

Expected (ideal state)

  • When the pointer is dropped (or absent) and exactly one registered generation is alive AND its record's socket is the public path AND processMatchesPidFile proves the pid, pruneDeadGenerations (or readHostRegistration) elects that generation as the pointer target instead of leaving the directory ownerless.
  • host status reports pointer: missing, elected: <instanceId> when it had to elect.
  • Regression: dead-pointer + one live registered generation -> readHostRegistration returns the live one; handoffHost proceeds.

Actual

host-generations.ts pruneDeadGenerations: if (pointerRemoved) await rm(paths.pointerFile) with no election. host-daemon-registration.ts readHostRegistration: pointer-only.

Evidence

Held locally (sanitized): daemon handoff --json refusal payload, the daemon directory listing (no host.pid, two generations/*/host.pid), the repaired pointer, and the successful handoff payload (generation: 1).

Related

#1893, #1899 (drain + prune), #1905 / #1906 (this incident's host fixes), omo#8577.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions