fix(runs,cli): refuse to remove a run dir under a live session (#526) - #532
Conversation
`delete_run` / `archive_run` rmtree'd a run directory with no session check, and every caller's guard is keyed on engine pid liveness — so an orphan (engine dead, agent session still alive) passed all of them. For an untagged session the run dir is the only ownership proof `prunable_sessions` can read, so removing it leaked the session, and its server, for the life of the machine. Both removal paths now raise `LiveSessionError` while `bmad-loop-<run-id>` is live. `session_alive` degrades like every other observer here — an absent multiplexer, a dead server or a failed query reads as no session — so an unanswerable question never blocks a removal the operator is entitled to. `delete`/`archive` surface the refusal with the escape their own flag provides; `--force` kills the session first, satisfying the guard rather than bypassing it. `clean` classifies such a run as protected instead of removing it. Also adds #419's named missing test: an untagged session whose run id collides with a dead run dir in another project is claimed by that project, while its own project reads it as live. Pinned as the actual outcome — closing it needs an ownership proof that outlives the run dir, not a change to either path here.
WalkthroughRun removal now checks matching agent-session liveness. ChangesLive session protection
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant CLI
participant RunStore
participant Multiplexer
Operator->>CLI: delete, archive, or clean run
CLI->>RunStore: request removal or cleanup
RunStore->>Multiplexer: check bmad-loop-run-id session
Multiplexer-->>RunStore: live, foreign, or unavailable result
RunStore-->>CLI: protected result, LiveSessionError, or removal result
CLI-->>Operator: report outcome
Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@codex review |
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
💡 Codex Review
bmad-loop/src/bmad_loop/cli.py
Line 3071 in 0b0dbf1
If the first multiplexer observation returns no session because of a transient query failure, or a session starts after that sample, the second check inside delete_run/archive_run raises LiveSessionError here. cmd_clean does not catch it, so clean --json exits with only an stderr error instead of its required JSON object; catch the backstop refusal around both removal calls and classify the run as protected.
AGENTS.md reference: AGENTS.md:L31-L31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/bmad_loop/cli.py`:
- Around line 3054-3066: Move the runs.session_alive(run_dir.name) guard to the
beginning of the reclaimable loop, before runs.reconcile_orphan_worktrees() and
artifact trimming, so live sessions are added to protected and skipped
immediately. Add a regression test covering a live agent session with a
populated worktree and verify the worktree is preserved.
- Around line 2883-2889: The force-removal path around
`runs.kill_session(run_id)` must require durable proof that the session belongs
to the current project; do not terminate untagged or otherwise unowned sessions
by `run_id` alone. When ownership cannot be established, refuse the
delete/archive operation, and add CLI coverage for two projects claiming the
same untagged session ID to ensure the other project’s live session remains
intact.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a90708f4-cf47-4158-816f-b7d1a6793110
📒 Files selected for processing (9)
CHANGELOG.mddocs/FEATURES.mdsrc/bmad_loop/cli.pysrc/bmad_loop/documents.pysrc/bmad_loop/runs.pysrc/bmad_loop/tui/app.pytests/test_cleanup.pytests/test_cli.pytests/test_runs.py
…e every mutation Two P1s from the review gate, both raised independently by codex and CodeRabbit. `--force` killed the session by name before removing. A session name carries no project, so on a `--run-id` collision — reachable, the flag is caller-supplied and validated for shape only — `delete --force` in one project tore down another project's live run. This turned the guard's own false refusal into destructive cross-project teardown. `--force` now overrides the refusal and kills nothing; the scoped kill is `bmad-loop cleanup`, which proves ownership from the tag first, and the refusal message already names it. `cmd_clean` checked the session inside the past-retention branch, after `reconcile_orphan_worktrees` had already force-removed the worktree the live session may still be working in — and after the trim path, which a run inside retention reaches with no check at all. The run was then reported "protected" having already been half-reclaimed. The guard is now the first thing in the loop, ahead of every mutation. Its test grows a real worktree and moves past retention so the ordering is what fails: relocating the guard back below the reconcile (position only, nothing deleted) reddens it.
|
@codex review |
|
@coderabbitai full review |
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14f8c7c019
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex, round 2: the message told the operator to clear the session with `bmad-loop cleanup`, and the docstring called that the scoped kill because `prune_sessions` reads the tag. Both are wrong for the exact state the guard fires on. `prunable_sessions` proves ownership from the tag only when there IS one; for an untagged session it falls back to this same run dir, so on a shared run id project A's `cleanup` prunes project B's live session — the edge this PR's own collision test asserts. My rationale contradicted my own test. No sound automatic remedy exists here — that is #419's point, and minting an ownership proof that outlives the run dir is its direction (2), not this guard. So the message keeps `cleanup` but carries the confirmation it needs (`bmad-loop attach <id>`) and states plainly why. Pinned by the refusal test; ablating the confirmation out of the message reddens it. Same correction applied to the CHANGELOG entry and FEATURES.md, which repeated the claim.
|
@codex review |
|
@coderabbitai full review |
|
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@coderabbitai full review |
|
|
@coderabbitai full review |
|
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 191-200: Replace the verbose changelog narrative with one short,
scannable, imperative release-note item summarizing the run-directory removal
protection. Remove the collision rationale and multiplexer behavior from this
entry, leaving those details to the feature documentation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 207e13dd-4a2b-4b49-8252-daa4bc671933
📒 Files selected for processing (9)
CHANGELOG.mddocs/FEATURES.mdsrc/bmad_loop/cli.pysrc/bmad_loop/documents.pysrc/bmad_loop/runs.pysrc/bmad_loop/tui/app.pytests/test_cleanup.pytests/test_cli.pytests/test_runs.py
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25ad3794d5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex P2: the guard matched on session name alone, so project A's delete, archive, TUI actions and every `clean` reclamation were blocked whenever a caller-supplied run id collided with project B's *tagged* live session — until B's run exited, and `clean` has no override to escape with. A tag outside `accepted_tags` proves the session foreign, and a tagged session carries its own ownership proof: it does not need this project's run dir, so removing the dir strands nothing. The guard now reads ownership exactly as `prunable_sessions` does and stands down there. Untagged, or tagged as ours, still refuses — an unread tag is not proof of anything, and only the untagged case was ever load-bearing. `session_alive` becomes `live_session_may_be_ours(project, run_id)`, which `delete_run` needs the project for; it now takes one, matching `archive_run`. The session listing is read before the tag query, so the extra call only happens on a name collision. Both directions ablate: name-only matching reddens the two foreign-tag tests, and treating any tag as clearing the guard reddens the tagged-as-ours test.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a9284904a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… the clean Codex round 4. The loop-top guard is one sample, so a resume racing this clean can start a session after it and before the removal. The chokepoint refuses there — correctly — but nothing caught it, so one racing run took the whole invocation down through main's catch-all, after the trim had already run. The removal branch now catches LiveSessionError, records the run as protected, corrects the freed estimate down to what the trim actually reclaimed, and carries on with the remaining runs. Ablating the catch reddens the new test, which flips the ownership read between the two calls — a stateless fake would answer the same both times and pin nothing. The wider race the finding describes is left alone and filed as #533: every mutation in that loop runs against a `reclaimable` sample taken in the previous loop and never re-read, so a resumed run can lose its worktree — identically on main before this PR, where the loop has no session check at all. Closing it is a concurrency-model decision (a lock, or documenting clean as unsafe alongside resume), not a guard fix.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8fb69165a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ntended Codex round 5. A run the race caught after `reconcile_orphan_worktrees` and `trim_run_dir` had already taken its artifacts was still recorded as `protected` — the field `clean --json` documents as "left untouched", and the count text mode prints as untouched. A consumer would read a partially reclaimed run as a preserved one. It is now classified by the state it ends in: trimmed when anything above reached it, protected only when nothing did. Both branches ablate — forcing either one reddens one of the two race tests, which cover the two sides.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27a804e735
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…kend Codex round 6. `live_session_may_be_ours` claimed both its reads degrade to "no answer", but that is the bundled tmux backend's behavior, not the seam's contract: only `pipe_pane` and `kill_session` are contractually best-effort, so a conforming out-of-tree backend raises MultiplexerError from `list_sessions` and `session_options`. That exception escaped through `delete`, `archive`, `clean` and the TUI workers, turning a transient transport error into a failed removal — and `clean` has no override. Both reads are now caught, each degrading in its own direction. A listing that cannot answer reads as "no session", matching what tmux already returns for a dead server — a guard that fired differently per backend would be worse than none. A tag that cannot be read is not proof the session is foreign, and by then the listing has established a session is live, so it reads as untagged and the refusal stands. Each `except` ablates against its own test.
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@coderabbitai full review |
|
|
@coderabbitai full review |
|
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/bmad_loop/cli.py`:
- Around line 3026-3041: Qualify the cleanup comment near
runs.live_session_may_be_ours in src/bmad_loop/cli.py:3026-3041 to state that a
later clean reclaims the run only after the operator confirms ownership with
bmad-loop attach <id>. Update the prune_sessions docstring in
tests/test_cli.py:2142-2159 to state that untagged sessions fall back to the
local run directory, which is not sufficient ownership evidence across projects
with identical run IDs.
In `@src/bmad_loop/documents.py`:
- Around line 438-443: Update the documentation text around the protected-run
description in documents.py to state that protected runs may be inside or beyond
the retention window when they carry a live agent session, while preserving the
existing explanations for --keep-listed and non-terminal runs.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 053956ab-580d-48ad-8e3f-b75d3f9d069e
📒 Files selected for processing (9)
CHANGELOG.mddocs/FEATURES.mdsrc/bmad_loop/cli.pysrc/bmad_loop/documents.pysrc/bmad_loop/runs.pysrc/bmad_loop/tui/app.pytests/test_cleanup.pytests/test_cli.pytests/test_runs.py
CodeRabbit at head, two Minor findings, both accurate. Round 3 corrected the claim that `bmad-loop cleanup` is a scoped remedy — it is not, for the untagged case the guard fires on — in the refusal message, the `_refuse_live_session` docstring, FEATURES.md and the CHANGELOG. Two sites still carried the old wording: the comment on `cmd_clean`'s guard and the docstring of the --force test. Both now say what the message says: `cleanup` clears the session, but ownership is confirmed first. `clean_document`'s description of `protected` said "past retention but still carrying a live agent session". The guard sits at the top of the reclaim loop, so a run *inside* the window is protected the same way; a consumer modelling `protected` off that sentence would be wrong about which runs can appear.
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@coderabbitai full review |
|
Implements both residuals of #526 (which carried #419's direction (3) past the #523 merge).
Residual A — the backstop
runs.delete_run/runs.archive_runremoved a run directory with no session check. Theirdocstrings said "callers enforce the live guard" and the callers do — but every caller's guard
is keyed on engine pid liveness, not session liveness:
cli._stop_or_block_live_engine, the TUI'saction_delete_run/action_archive_run, andcmd_clean's retention path. An orphan — enginedead, agent session still alive — is precisely the state that passes all three, and it is the one
state where the run dir is load-bearing: for an untagged session it is the only ownership proof
prunable_sessionscan read, so removing it leaks the session (and its psmux server) for the lifeof the machine.
Both removal paths now raise
LiveSessionErrorwhile a matching session is live that this projectcannot prove belongs to another one. Per the observe-degrade / repair-raise doctrine:
runs.live_session_may_be_oursis an observation and degrades, each read in its owndirection. A listing that cannot answer reads as "no session" (what tmux already returns for a
dead server — a guard that fired differently per backend would be worse than none). A tag that
cannot be read is not proof the session is foreign, so it reads as untagged and the refusal
stands. Both
MultiplexerErrorpaths are caught explicitly: the seam only promisespipe_paneand
kill_sessionnever raise.prunable_sessionsreads it, so a session tagged to another project— which carries its own proof and does not need this run dir — never blocks a removal.
Callers adapted, not widened:
delete/archiveprint the refusal and exit 1, appending the escape their own flag provides(
main's catch-all would give the same exit code but a bareerror: …).--forceoverrides the refusal and kills nothing. A session name carries no project, sokilling
bmad-loop-<id>by name would tear down another project's live run on a--run-idcollision.
bmad-loop cleanupas the remedy together with the confirmation itneeds (
bmad-loop attach <id>), becausecleanupis not sound on its own here: for an untaggedsession
prunable_sessionsproves ownership by that same run dir, so a shared run id prunes theother project's session. No sound automatic remedy exists at this layer — that is Untagged sessions are weak ownership: they leak once their run dir is gone, and can be pruned by the wrong project on a run-id collision (the fallback behind #320) #419's thesis,
and minting a proof that outlives the run dir is its direction (2).
cleanleaves such a run untouched, with the guard as the first thing in the reclaim loop —ahead of
reconcile_orphan_worktreesandtrim_run_dir, not just ahead of the archive. If asession appears mid-loop anyway, the chokepoint refusal is caught: the run is recorded and the
clean continues, classified by what actually happened (
trimmedif the steps above alreadyreached it,
protectedonly if nothing did —protectedmeans "left untouched" in the--jsoncontract).
LiveSessionErroralongsideOSErrorso a refusal notifies instead ofkilling the worker thread.
Residual B — the missing characterization test
test_prunable_sessions_claims_an_untagged_session_on_a_run_id_collisionadds the case #419 namedand no fixture covered: untagged session, a dead run dir with the same id here, and the session
actually belonging to another project. Foreignness is constructed out of band — nothing in the
fixture marks it, because nothing can, which is the defect. Both projects' views are asserted, so
the test shows them disagreeing about one live session rather than echoing one side.
Pinned as the actual outcome, not the desired one: closing it needs an ownership proof that
outlives the run dir (#419 direction 2), not a change to either path in this PR.
Verification
Full suite 4971 passed / 39 skipped / 5 xfailed;
uv run pyright0 errors;trunk check --allclean. CI 10/10 including both Windows jobs.Every gate ablated singly, all bit (restored from file backups, never
git checkout— thatreverts to the commit and silently drops in-flight fixes):
_refuse_live_sessioncalls indelete_run/archive_runforce=args.forcepinned toFalse(override never reaches the leaf)--forcetestsforce=args.forcepinned toTrue(guard never fires)LiveSessionErrorcatch incmd_delete/cmd_archiveMultiplexerErrorcatch on the listing readMultiplexerErrorcatch on the tag readcmd_clean's live-session branchcmd_cleanguard tests (text +--json)reconcile_orphan_worktrees— position only, nothing deletedcmd_cleanworktree assertionLiveSessionErrorcatchprotected/ forced totrimmedprunable_sessions' untagged run-dir fallbackTwo ablations initially passed and the tests were strengthened rather than trusted: the CLI
refusal tests now assert
(or pass --force)(only the handler emits it —main's catch-allrepeats the run message and would have hidden its removal), and the mid-clean race test now
asserts the untouched count (the
else protectedbranch was unpinned).Review rounds
Seven codex rounds (the last clean), plus three CodeRabbit passes. Codex found six real defects
after the first draft, the first two also raised independently by CodeRabbit:
--forcekilled the session by name → cross-project teardown on a run-id collision. Removed.cmd_cleanchecked the session afterreconcile_orphan_worktreeshad already run. Guard movedto the top of the loop.
cleanupand the docstring called it "the scoped kill" — contradictedby this PR's own collision test. Message and docs corrected.
path,
cleanincluded, with no override. Scoped toaccepted_tags.cleanabort after partial mutation; and the run was then reported asprotected, which the--jsoncontract documents as "left untouched". Both fixed.MultiplexerErrorescaped the guard's observations on a conforming out-of-tree backend.CodeRabbit then caught two documentation-consistency defects at head: two sites still described
cleanupas a scoped remedy (residue of correction 3), andclean_document'sprotecteddescription still said "past retention", which stopped being true when the guard moved to the top
of the loop in fix 2. Its one style finding — rewrite the CHANGELOG headline in the imperative —
was declined with a re-run count over the
Unreleasedsection (54 entries open with "A", 22 with"The", 7 with "An", 0 with an imperative verb); the verbosity half of that finding was taken and
the entry trimmed from 10 lines to 6.
Not covered
exceptwidening has no test —app.py's lifecycle workers have no existingharness, and standing one up for a widened
exceptclause is out of proportion. The behavior itguards is tested at the layer that produces it (
runs.py) and at the CLI.cleanclassifies runs once and then mutates, so a concurrentresumeof a stopped run can lose its worktree. Identical onmainat814f0a2, where the loophas no session check at all — this PR narrows the window but cannot close it, and closing it is a
concurrency-model decision (a lock, or documenting
cleanas unsafe alongsideresume).runs.mux_sessions()'s own docstring still assumes the bundled backend's "returns[]onfailure", and
prunable_sessionsreads it. Pre-existing, on an observation path rather than arepair one, so the diff was not widened into it.
Closes #526. Refs #419, #523, #418, #320, #533.