Skip to content

orchestrate: a mid-turn reviewer kill loses the whole review, and no gate detects the scratch files it leaves undeclared in the worktree #385

Open

Description

@rodrigorjsf

Found during orchestrate run prd352-20260803-015333 (PRD #352, 12 slices, all passed).

Symptom

The reviewer-deep on slice #355 was terminated by a usage limit (idleReason: "failed", failureReason: "You've hit your session limit"). Two distinct losses followed.

1. The review itself was lost entirely

The reviewer had clearly done substantial adversarial work — it had authored a 208-line probe suite exercising recoverSliceProgress and resolveSliceProgressPath — but produced no envelope, so none of its conclusions survived. The envelope is emitted as a final flourish, which is precisely the moment a turn-budget or usage cutoff destroys.

2. It leaked a scratch file into the worktree

test/zz-review-probe.test.ts was left behind, self-labelled:

// TEMPORARY reviewer probe — deleted before the slice is handed back

The reviewer intended to delete it and died first. Because the file is untracked and undeclared, finalize_slice would not stage it (it stages exactly the declared union) — but it would be collected by vitest and change the orchestrator's pre-merge test count, and it survives into any later inspection of the worktree as apparent slice content.

Impact

Loss 1 is expensive but recoverable — the run-state subState: verified checkpoint is exactly the right resume anchor, so a re-spawned reviewer redoes only the review, not the investigation or implementation. That part of the design worked.

Loss 2 is the silent one: nothing in the pipeline detects an undeclared file appearing in the worktree between verify_changeset (step 4a, before the reviewer) and the pre-merge gate (step 5a, after it).

Workarounds used

Removed the probe file manually before re-running the gate; re-spawned the reviewer with two added instructions — write the envelope early rather than last, and delete every scratch file.

Suggested fixes

  • Make envelope emission incremental or checkpointed rather than terminal. The same argument ADR-0017 makes for the slice progress record applies to the envelope itself. At minimum, the bundled subagent definitions should instruct every role to write a provisional envelope as soon as it has a defensible verdict and revise it, rather than composing it once at the end.
  • Re-run verify_changeset (or an equivalent undeclared-file check) AFTER the reviewer returns, not only after the implementer. Today the reviewer is trusted to declare its own edits with no worktree cross-check — an asymmetry with the implementer that has no stated rationale.
  • Have the reviewer's prompt template mandate scratch-file cleanup. Currently nothing forbids leaving scratch files behind.

Relationship to existing issues (read before deduping)

Environment

orchestrate 1.6.0 (plugins/orchestrate/.claude-plugin/plugin.json:3 on development) plus the PRD #352 umbrella branch (slice-executor delegation layer). Run prd352-20260803-015333; slice #355, reviewer-deep.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions