Skip to content

Local: a stage stopped by a signal writes its spend line once more - #192

Merged
kirkmarple-clockgrove merged 3 commits into
mainfrom
fix/178-kill-spend-line
Nov 14, 2026
Merged

kirkmarple-clockgrove merged 3 commits into
mainfrom
fix/178-kill-spend-line

Conversation

@kirkmarple-clockgrove

@kirkmarple-clockgrove kirkmarple-clockgrove commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What was wrong: a local stage writes its spend so far only with its progress comment's writes (phase lines, and the status line at most every two minutes). A stage stopped by SIGTERM, SIGINT or SIGHUP (factory serve stopping, Ctrl-C) exited at once from proc.mjs stopGroups, so a pass that had spent since the last write was left at that write's figure: {"usd":0,"usdLowerBound":true} for a first pass killed seconds in.

What changed:

  • proc.mjs atStop(fn): stopGroups runs what is registered after it kills the harness groups and before it exits, synchronously, so no other stage code runs (the old guarantee stays).
  • progressReporter (session.mjs) registers one hook after its first write lands. If the stage has no final figure yet, the hook reads the stage's spend at once (harness.mjs stageSpendNow, the same sum as stageSpendSoFar with each running pass read from its stream through live-spend.mjs now()). If that is more than the comment's line shows, it PATCHes the comment it last wrote (by id, with no list read) with the same body and the new lower-bound line.
  • rest-sync.mjs restSync: that one rest() call runs in a worker thread while the stage thread waits on Atomics.wait, at most STOP_WRITE_MS (5 s). A slow or failed write never holds the exit longer. The loop's own 30 s wait and the stop sequence are unchanged.
    • Token: the caller's token may be TokenKeeper's getter, as in production (worker.mjs, review.mjs). restSync reads it on the stage thread and passes the string to the worker as data, never through env or argv.
    • Redaction: the body has the stage's held secrets (worker-secret values, the relay token) redacted on the stage thread. The worker's own github.mjs holds none, so it would not redact them.
    • One try: there is no retry. The answer carries sent, decided on structure: false when the request certainly never landed (the thread or token failed before sending, a connect-level failure that github.mjs marks unsent (ECONNREFUSED, ENOTFOUND, EAI_AGAIN, ENETUNREACH, …), or a 4xx refusal); true when it may have landed (a 5xx, no answer before the deadline, a connection lost after sending).
  • stageSpendSoFar and stageSpendNow share one sum (stageSpendOf), which keeps stageSpendSoFar's order: finished passes and session totals are taken before any meter is read.
  • docs/LOCAL.md: the spend-line and stopping paragraphs describe the stop write, including Codex's limit.

Why it cannot double-count: the stop write records the session totals it counted (session-totals.json) from the same reading as the figure, as every interim write does. It records them when the write landed or may have, so a resuming stage never counts that part again. If the write may have landed but did not, that part goes uncounted (a lower bound, never a second count). If it certainly did not land, nothing is recorded, and the resuming stage counts the part the line could not.

Codex: Codex reports usage only when a turn ends (turn.completed), so a Codex pass killed in its first turn still reads $0 at the stop. Claude Code (each model response) and Copilot (each usage checkpoint) do not have this limit. LOCAL.md says so; reading Codex's rollout at the stop is not part of this PR.

Proof: in plugin/test/stage-spend.test.mjs, the stage processes hold the token as a getter, as production does. A new term mode runs the pass in its own process group, as in local mode. The stage writes no line after the pass begins and is sent a real SIGTERM, then exits 143.

  • First stage, fresh session: the comment ends with {"usd": <half a pass>, "usdLowerBound": true}.
  • Three stages, end, X, end; each run checks the killed stage's line and the sum, with no pass counted twice:
    • term: the killed stage's line holds its half pass, and the sum is 2.5 passes.
    • term-unsent, where GitHub is unreachable for the stop write: nothing is recorded, and the resuming stage counts the half pass, so the sum is 2.5.
    • term-5xx, where the stop write is answered 502: the totals are recorded, and the sum is 2 (the lower-bound case).
  • restSync on its own, in a process of its own: a getter's token is sent; a held secret never reaches the request body; one try each; sent is false for a 422, an unreachable host and a token that fails, and true for a 502 and no answer.

Each new test fails without its fix. Without any of this (the test file on origin/main), both term tests show $0. Passing the getter through to the worker breaks every stop write, and the restSync test fails. With no redaction on the stage thread, the restSync test fails. With the earlier commit-on-any-failure rule, term-unsent fails. With sent always true, term-unsent and the restSync test fail.

Full suite (node --test plugin/test/*.test.mjs bench/test/*.test.mjs): 784 pass, 0 fail. The temp folder is left empty, and node bench/skills.mjs is within the cap.

Library check: the sync wait is about 30 lines of Node built-ins (worker_threads, Atomics). The HTTP call goes through Factory's own github.mjs rest. synckit does the same thing, but it would be a vendored dependency for no protocol or format code.

Fixes #178

🤖 Generated with Claude Code

https://claude.ai/code/session_01CS53Gu2D3bE8mKgFN92XP5

A stage killed before its next progress-comment write after a pass had
spent left that pass at the last line's figure ($0 for a first pass).
On SIGTERM, SIGINT or SIGHUP the stage now kills its harness's group,
reads the stage's spend at once (stageSpendNow, from each running
pass's stream) and, when that is more than the comment shows, PATCHes
the comment it last wrote, waiting at most 5 s in a worker thread so
no other stage code runs. The session totals that reading counted are
recorded as for any interim write, so a resuming stage never counts
them twice.

Fixes #178

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CS53Gu2D3bE8mKgFN92XP5
…e landed

- restSync reads a TokenKeeper getter on this thread (a function cannot
  go to the worker), and redacts this process's held secrets from the
  body here (the worker's github.mjs holds none).
- One try, and a `sent` field from structure: false when the request
  never left (the thread or token failed first, github.mjs `unsent`
  connect codes) or GitHub refused it (4xx); true for a 5xx, no answer
  in time or a connection lost after sending. The stop write records
  the session totals only when it landed or may have.
- Tests use the getter shape; new cases for a stop write never sent
  and one answered 502, and restSync's token, redaction and `sent`.
- LOCAL.md: Codex reports usage at a turn's end, so a Codex pass killed
  in its first turn still reads $0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CS53Gu2D3bE8mKgFN92XP5
…cides them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CS53Gu2D3bE8mKgFN92XP5
@kirkmarple-clockgrove
kirkmarple-clockgrove added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit 77bb8ac Nov 14, 2026
4 checks passed
@kirkmarple-clockgrove
kirkmarple-clockgrove deleted the fix/178-kill-spend-line branch November 14, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local: a stage killed before its first status write records its pass as $0

1 participant