fix(stop-hooks): circuit breaker to stop orchestrator LLM-turn-polling runaway (#29)

skills/bmad-story-automator/data/data-file-index.md

@@ -20,6 +20,7 @@
 |------|-----|
 | `orchestrator-rules.md` | Core rules for orchestrator behavior |
 | `execution-patterns.md` | FORBIDDEN patterns - must know before any execution |
+| `monitoring-pattern.md` | FORBIDDEN polling patterns + single-call spawn/monitor/verify cycle — MUST be in context before any tmux interaction (issue #29) |
 | `scripts-reference.md` | Script usage patterns |
 
 ### LOAD ON TRIGGER
@@ -60,7 +61,7 @@
 
 ```
 Starting execution?
-  → Load: orchestrator-rules.md, execution-patterns.md, scripts-reference.md
+  → Load: orchestrator-rules.md, execution-patterns.md, monitoring-pattern.md, scripts-reference.md
 
 Step failed?
   → Load: retry-fallback-strategy.md

skills/bmad-story-automator/data/monitoring-pattern.md

@@ -80,10 +80,14 @@ verified=$(echo "$validation" | jq -r '.verified')
 "$scripts" monitor-session <session_name> [options]
 
 # Options:
-#   --max-polls N     Maximum iterations (default: 30)
-#   --timeout MIN     Overall timeout in minutes (default: 60)
-#   --verbose         Print progress to stderr
-#   --json            Output as JSON instead of CSV
+#   --max-polls N            Maximum iterations (default: 30)
+#   --timeout MIN            Overall timeout in minutes (default: 60)
+#   --completion-rechecks N  Re-confirm a "completed-but-unverified" session
+#                            N times in-process before returning `incomplete`
+#                            (default: 3). Absorbs false-completes inside this
+#                            single call so you never hand-poll.
+#   --verbose                Print progress to stderr
+#   --json                   Output as JSON instead of CSV
 
 # Output (JSON):
 # {"final_state":"completed|crashed|stuck|timeout|incomplete|not_found","output_file":"/tmp/...","exit_reason":"..."}
@@ -123,7 +127,7 @@ After `$scripts monitor-session` returns:
 | final_state | Action |
 |-------------|--------|
 | `completed` | Run step verifier or parser for the active workflow |
-| `incomplete` | **(v2.2)** Session idle but workflow NOT verified → Escalate immediately |
+| `incomplete` | Session idle but workflow NOT verified, even after in-process rechecks → **re-spawn `monitor-session` once** (one blocking call). If a second `incomplete` comes back, escalate. **NEVER** `cat` the output file or `tmux capture-pane` in a fresh turn to "wait" — that is the forbidden polling loop (issue #29). |
 | `crashed` | Check retry count → retry or escalate |
 | `stuck` | Get output → investigate → may need restart |
 | `timeout` | Get output → escalate to user |

skills/bmad-story-automator/data/stop-hook-recovery.md

@@ -20,6 +20,34 @@
 | Unrecoverable error (all retries exhausted) | **STOP** → Follow stop procedure below | Cannot proceed without intervention |
 | External dependency down (API, service) | **RETRY** → Sleep with increasing delay (1m, 2m, 4m, 8m, 16m), max 5 attempts | Often recovers on its own |
 | User explicitly requested stop earlier | **STOP** → Follow stop procedure below | Honoring user intent |
+| **You are about to "check" an in-flight session in a new turn** | **STOP** → Do NOT poll. Re-spawn `monitor-session` (one blocking call) instead | Per-turn polling is the #1 cost runaway (issue #29) |
+| **This is the Nth resume in a row with no step progress** | **STOP** → Let the circuit breaker release; document why the step isn't progressing | A wait that isn't advancing is not "do whatever it takes" — it's a loop |
+
+---
+
+## 🚨 Cost Ceiling — "do whatever it takes" has a limit
+
+"Continue" does NOT mean "burn an entire quota window busy-waiting." The Stop hook now
+has a **circuit breaker**: after `STORY_AUTOMATOR_MAX_STOP_BLOCKS` (default **5**)
+consecutive blocks with **no step progress** (no story completed, no marker heartbeat
+bumped), it stops blocking and releases the session with a `systemMessage`. This is by
+design — a long-lived orchestrator stopping-and-resuming in fresh LLM turns replays its
+entire (growing) transcript every turn, so an idle busy-wait can cost thousands of dollars
+of cache-read with zero work done (issue #29: 1,599 turns / 478M cache-read tokens).
+
+**To stay productive — and keep the breaker from tripping on a healthy long run — you MUST:**
+
+1. **Wait with ONE blocking `monitor-session` call.** While that single Bash call runs, you
+   take no turns and pay nothing. NEVER re-read its output file, `tmux capture-pane`, or
+   `tmux-status-check` in a separate turn to see "is it done yet." That per-turn polling is
+   forbidden (see `monitoring-pattern.md`).
+2. **On a verified step, the orchestrator bumps the marker heartbeat** (`orchestrator-helper
+   marker heartbeat`). That heartbeat is what tells the breaker "real progress happened,
+   reset the counter." If you are doing real work, progress advances and the breaker never
+   fires. If the breaker fires, it means you were looping without progress — that is the bug
+   it exists to stop.
+3. **If `monitor-session` returns without a verified artifact, re-spawn it** (one more
+   blocking call) — do not hand-poll. See the Decision Flow in `monitoring-pattern.md`.
 
 ---
 
@@ -69,9 +97,13 @@ The stop hook fires when Claude pauses, not just when explicitly stopping.
 
 ## Core Principle
 
-**Your job is to finish every story in the correct order.**
+**Your job is to finish every story in the correct order — efficiently.**
 
-Do whatever it takes. Make autonomous decisions. Only stop when genuinely unrecoverable (all retries exhausted, user explicitly requested stop).
+Make autonomous decisions and keep moving. "Do whatever it takes" means *make progress*,
+not *spin in place*: a turn that neither advances a step nor waits inside a single blocking
+call is wasted, and enough of them trip the cost ceiling above. Stop when genuinely
+unrecoverable (all retries exhausted, user explicitly requested stop) **or** when the
+circuit breaker releases you because the active step has stalled without progress.
 
 ---
 
@@ -85,3 +117,5 @@ Do whatever it takes. Make autonomous decisions. Only stop when genuinely unreco
 | Waiting for user at decision points | Decide autonomously, keep moving |
 | Stopping on first API/service failure | Retry with exponential backoff (5 attempts) |
 | Asking user which path to take | Pick the path that finishes stories in order |
+| `cat`-ing a monitor output file / `tmux capture-pane` in a new turn to check progress | Wait inside ONE `monitor-session` call; re-spawn it if it returns early |
+| Treating the Stop hook firing as "poll again" | The hook is a guard, not a clock — make progress or wait in a single call, don't busy-loop |

skills/bmad-story-automator/src/story_automator/commands/basic.py

@@ -10,6 +10,7 @@
 from ..core.runtime_layout import active_marker_path, runtime_provider
 from ..core.stop_hooks import HookConfigError, ensure_stop_hook
 from ..core.utils import (
+    atomic_write,
     get_project_slug,
     run_cmd,
     write_json,
@@ -140,22 +141,93 @@ def cmd_ensure_stop_hook(args: list[str]) -> int:
     return 0
 
 
+DEFAULT_MAX_STOP_BLOCKS = 5
+
+
+def _max_stop_blocks() -> int:
+    """Consecutive no-progress Stop-hook blocks before the circuit breaker releases.
+
+    Kept below Claude Code's own ``CLAUDE_CODE_STOP_HOOK_BLOCK_CAP`` (defaults to
+    9) so the orchestrator releases gracefully with an explanation instead of the
+    harness force-ending the turn after a runaway busy-wait. See
+    ``data/stop-hook-recovery.md`` and issue #29.
+    """
+    raw = os.environ.get("STORY_AUTOMATOR_MAX_STOP_BLOCKS", "").strip()
+    if raw.isdigit() and int(raw) > 0:
+        return int(raw)
+    return DEFAULT_MAX_STOP_BLOCKS
+
+
 def cmd_stop_hook(_: list[str]) -> int:
-    sys.stdin.read()
+    raw_input = sys.stdin.read()
     if os.environ.get("STORY_AUTOMATOR_CHILD", "").lower() == "true":
         return 0
+    try:
+        hook_input = json.loads(raw_input) if raw_input.strip() else {}
+    except json.JSONDecodeError:
+        hook_input = {}
+    # Claude Code sets ``stop_hook_active`` when this stop only happened because a
+    # prior Stop hook blocked it (i.e. we are inside a continuation loop). The
+    # documented guard against infinite Stop-hook loops keys off this flag.
+    stop_hook_active = bool(hook_input.get("stop_hook_active"))
+
     marker = active_marker_path()
     if not marker.exists():
         return 0
     try:
         payload = json.loads(marker.read_text())
     except json.JSONDecodeError:
         return 0
+    if not isinstance(payload, dict):
+        return 0
     remaining = payload.get("storiesRemaining", 0)
     if isinstance(remaining, str) and remaining.isdigit():
         remaining = int(remaining)
-    if not remaining:
+    if not isinstance(remaining, int) or not remaining:
         return 0
+
+    # --- Circuit breaker -------------------------------------------------
+    # Count consecutive blocks that the orchestrator survived WITHOUT making
+    # real progress, so a long-lived session can't busy-wait an entire quota
+    # window away by stopping-and-resuming in fresh LLM turns (issue #29).
+    #
+    # "Progress" = a story completed (storiesRemaining decreased) OR the
+    # orchestrator bumped the marker heartbeat at a verified step. A healthy
+    # blocking ``monitor-session`` wait makes no stop attempts at all, so it
+    # never accrues blocks; only turn-by-turn polling does.
+    seen_heartbeat = payload.get("stopHookSeenHeartbeat")
+    seen_remaining = payload.get("stopHookSeenRemaining")
+    current_heartbeat = payload.get("heartbeat")
+    progressed = current_heartbeat != seen_heartbeat or (
+        isinstance(seen_remaining, int) and remaining < seen_remaining
+    )
+    if progressed or not stop_hook_active:
+        blocks = 0
+    else:
+        prev = payload.get("stopHookBlocks", 0)
+        blocks = (prev + 1) if isinstance(prev, int) else 1
+
+    payload["stopHookSeenHeartbeat"] = current_heartbeat
+    payload["stopHookSeenRemaining"] = remaining
+
+    if blocks >= _max_stop_blocks():
+        # Release: allow the stop so the session goes idle instead of burning
+        # turns. Reset the counter so a manual/background-triggered resume
+        # starts clean. The user sees why via systemMessage.
+        payload["stopHookBlocks"] = 0
+        _write_marker(marker, payload)
+        message = (
+            f"Story Automator auto-paused after {blocks} consecutive stop-hook blocks "
+            f"with no step progress (circuit breaker). {remaining} stories remain. "
+            "This guards against runaway LLM-turn polling — see "
+            f"{_workflow_doc_relative('stop-hook-recovery.md')}. "
+            "Resume the orchestrator to continue, or investigate why the active step is not progressing."
+        )
+        print(json.dumps({"systemMessage": message}, indent=2))
+        return 0
+
+    payload["stopHookBlocks"] = blocks
+    _write_marker(marker, payload)
     reason = (
         "Story Automator active "
         f"({remaining} stories remaining). Read "
@@ -165,6 +237,15 @@ def cmd_stop_hook(_: list[str]) -> int:
     return 0
 
 
+def _write_marker(path: Path, payload: dict[str, object]) -> None:
+    try:
+        atomic_write(path, json.dumps(payload, indent=2) + "\n")
+    except OSError:
+        # A marker we cannot persist must never crash the Stop hook; the
+        # breaker simply won't advance this cycle.
+        pass
+
+
 def cmd_commit_story(args: list[str]) -> int:
     repo = ""
     story = ""

skills/bmad-story-automator/src/story_automator/commands/tmux.py

@@ -29,6 +29,14 @@
     project_slug,
 )
 
+# When tmux reports a session "completed" but the workflow verifier still says
+# the artifact isn't there, the pane is often just idle mid-tool-call (a
+# false-complete). Rather than bounce an ``incomplete`` that the orchestrator
+# then hand-polls in fresh LLM turns (issue #29), re-confirm in-process a few
+# times inside the same blocking call before giving up.
+COMPLETION_RECHECKS = 3
+RECHECK_GRACE_SECONDS = 15
+
 
 def cmd_tmux_wrapper(args: list[str]) -> int:
     if not args:
@@ -285,12 +293,13 @@ def cmd_monitor_session(args: list[str]) -> int:
         return 1
     if args[0] in {"--help", "-h"}:
         print("Usage: monitor-session <session_name> [options]")
-        print("Options: --max-polls N --initial-wait N --project-root PATH --timeout MIN --verbose --json --agent TYPE --workflow TYPE --story-key KEY --state-file PATH")
+        print("Options: --max-polls N --initial-wait N --project-root PATH --timeout MIN --completion-rechecks N --verbose --json --agent TYPE --workflow TYPE --story-key KEY --state-file PATH")
         return 0
     session = args[0]
     max_polls = 30
     initial_wait = 5
     timeout_minutes = 60
+    completion_rechecks = COMPLETION_RECHECKS
     json_output = False
     workflow = "dev"
     story_key = ""
@@ -312,6 +321,10 @@ def cmd_monitor_session(args: list[str]) -> int:
             timeout_minutes = int(args[idx + 1])
             idx += 2
             continue
+        if arg == "--completion-rechecks" and idx + 1 < len(args):
+            completion_rechecks = max(0, int(args[idx + 1]))
+            idx += 2
+            continue
         if arg == "--json":
             json_output = True
         elif arg == "--agent" and idx + 1 < len(args):
@@ -346,6 +359,7 @@ def cmd_monitor_session(args: list[str]) -> int:
     start = time.time()
     last_done = 0
     last_total = 0
+    unverified_completions = 0
     for _ in range(1, max_polls + 1):
         if time.time() - start >= timeout_minutes * 60:
             return _emit_monitor(json_output, "timeout", last_done, last_total, "", f"exceeded_{timeout_minutes}m")
@@ -354,6 +368,10 @@ def cmd_monitor_session(args: list[str]) -> int:
             last_done = int(status["todos_done"])
             last_total = int(status["todos_total"])
         state = str(status["session_state"])
+        if state != "completed":
+            # Session is active again (or never idle): any earlier false-complete
+            # is void — reset the recheck counter.
+            unverified_completions = 0
         if state == "completed":
             output = session_status(session, full=True, codex=agent == "codex", project_root=project_root, mode=runtime_mode())["active_task"]
             verification = _verify_monitor_completion(
@@ -376,13 +394,27 @@ def cmd_monitor_session(args: list[str]) -> int:
                         reason,
                         output_verified=bool(verified.get("verified")),
                     )
+                reason = str(verified.get("reason") or "workflow_not_verified")
+                unverified_completions += 1
+                # A broken verifier CONTRACT won't fix itself by waiting — surface
+                # it immediately so the orchestrator escalates. Only re-poll the
+                # genuine false-complete case (pane idle mid-tool-call, artifact
+                # not written yet).
+                if reason != "verifier_contract_invalid" and unverified_completions < completion_rechecks:
+                    # Re-confirm in-process instead of returning an `incomplete`
+                    # the orchestrator would hand-poll. Stays inside this one call.
+                    remaining_time = timeout_minutes * 60 - (time.time() - start)
+                    if remaining_time <= 0:
+                        return _emit_monitor(json_output, "timeout", last_done, last_total, str(output), f"exceeded_{timeout_minutes}m")
+                    time.sleep(min(RECHECK_GRACE_SECONDS, max(1, int(remaining_time))))
+                    continue
                 return _emit_monitor(
                     json_output,
                     "incomplete",
                     last_done,
                     last_total,
                     str(output),
-                    str(verified.get("reason") or "workflow_not_verified"),
+                    reason,
                     output_verified=bool(verified.get("verified")),
                 )
             return _emit_monitor(json_output, "completed", last_done, last_total, str(output), "normal_completion")

skills/bmad-story-automator/steps-c/step-01b-continue.md

@@ -209,6 +209,12 @@ marker_entry=$(echo "$marker_info" | jq -r '.entry')
   --project-slug "$project_slug" --pid "$$" --heartbeat "{timestamp}"
 ```
 
+**🚨 BEFORE routing into any execution step, load `../data/monitoring-pattern.md` and
+`../data/execution-patterns.md` into context.** A Stop-hook resume re-enters mid-run with
+none of the execution guidance loaded; without the FORBIDDEN-polling table the orchestrator
+improvises per-turn `cat`/`tmux capture-pane` polling and burns a quota window (issue #29).
+This is mandatory on the resume path, not optional.
+
 **Then** route per Menu Handling Logic in section 5 above.
 
 ---

skills/bmad-story-automator/steps-c/step-03-execute.md

@@ -71,8 +71,19 @@ state_file="{outputFile}"
 - REQUIRED patterns (verify state after each step)
 - Monitoring failure fallback sequence
 
+**ALSO read `../data/monitoring-pattern.md` BEFORE any tmux interaction.** It carries
+the FORBIDDEN polling table and the single-API-call spawn→monitor→verify cycle. Skipping
+it is how a Stop-hook resume drifts into per-turn `cat /tmp/*.json` / `tmux capture-pane`
+polling that torches a quota window (issue #29). Do NOT spawn or poll any session until it
+is in context.
+
 **Key rule:** Each step (create/dev/auto/review) MUST be executed and monitored separately. NEVER chain steps in loops.
 
+**Waiting rule:** A session is waited on by ONE blocking `monitor-session` call. NEVER
+re-read a monitor output file, `tmux capture-pane`, or `tmux-status-check` in a fresh LLM
+turn to "check if it's done yet" — that is the forbidden polling pattern. If a wait returns
+without a verified artifact, re-spawn `monitor-session` (one call), don't poll by hand.
+
 ## Story Loop
 
 > **⚠️ SPAWN PATTERN - READ THIS:**
@@ -150,6 +161,9 @@ validation=$("$scripts" orchestrator-helper verify-step create {story_id} --stat
   # Update Story Progress: mark create-story done
   tmp_state=$(mktemp)
   sed "s/^| ${story_id} |.*$/| ${story_id} | done | - | - | - | - | in-progress |/" "$state_file" > "$tmp_state" && mv "$tmp_state" "$state_file"
+  # Heartbeat on real progress: resets the Stop-hook circuit breaker so long
+  # stories never trip it (see data/stop-hook-recovery.md). REQUIRED.
+  "$scripts" orchestrator-helper marker heartbeat >/dev/null
   ```
   → proceed to B
 - If `validation.verified == false` AND attempts < 5 → retry with next agent (see `{retryStrategy}`)
@@ -192,6 +206,8 @@ reasons=$(echo "$parsed" | jq -c '.reasons // []')
   # Update Story Progress: mark dev-story done
   tmp_state=$(mktemp)
   sed "s/^| ${story_id} |.*$/| ${story_id} | done | done | - | - | - | in-progress |/" "$state_file" > "$tmp_state" && mv "$tmp_state" "$state_file"
+  # Heartbeat on real progress: resets the Stop-hook circuit breaker. REQUIRED.
+  "$scripts" orchestrator-helper marker heartbeat >/dev/null
   ```
   → proceed to C (next step)
 - If `next_action == "retry"` OR `result.final_state == "crashed"`: