Repository navigation
feat(stop): a Stop hook that watches, and bouncer stops to score it (0.2.22) - #157
Merged
Merged
Conversation
… (0.2.22) When the agent stops, was it only waiting for a go-ahead? 315 of 2,345 typed prompts in thirty days of the owner's transcripts were short go-aheads (#156). `bouncer stop` judges the end of the closing message against a `stop` set of three questions and appends the answers to stops.jsonl. It prints nothing and exits 0 in every case: a Stop hook's stdout can keep an agent working after it chose to stop, so runStop returns nothing for the entrypoint to print, and the tests run the built bundle to hold that (ADR-015). It is off until a stop policy exists outside the plugin ($BOUNCER_STOP_POLICY, <repo>/.bouncer.stop.yaml, ~/.bouncer/stop.yaml), because it sends the end of that message to the backend. Nothing is sent while a background task runs. `bouncer stops` pairs each logged stop with the next prompt typed in the same session and prints what would have been saved and how often it would have run ahead. It sends nothing. The Stop payload was captured from Claude Code 2.1.201 and is a fixture. The set has not been run against a live backend: every judgment in the tests is the mock's. Bench against the base bundle: paired median +0.3 ms judged, +0.1 ms hard-rule. Refs #156. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposal B of #156: an observe-only
Stophook, and the command that scores it. ADR-015 is the decision.Why
In thirty days of the owner's transcripts, 315 of 2,345 typed prompts were short go-aheads ("proceed" 110, "merge it" 25, "go ahead" 15). Each is the agent stopping, a person deciding the next step was already covered, and two words typed. The next prompt after a stop is also a free label for whether the stop was needed.
What it does
bouncer stop, registered for theStopevent. It judges the end of the agent's closing message against astoppolicy set and appends the answers tostops.jsonlin the data directory.Stophook that prints ablockdecision or exits 2 keeps the agent working past where it chose to stop.runStopreturns nothing, so the entrypoint has nothing to print.bouncer stopspairs each logged stop with the next prompt typed in the same session and prints a two-by-two table: what the set would have done against whether a bare go-ahead followed. It reads the log and the transcripts and sends nothing.Three questions, combined by rules in
policy/stop.yaml: does the message ask for approval of a step it describes, does it need a choice or information from the user, is the proposed step hard to undo. A merge is "leave" by the third, however plainly it asks.It is off after this update
The stop set sends content: the last 6,000 characters of the closing message, after redaction. So the bundled
policy/stop.yamlis not read from where it ships. The hook looks for a stop policy at$BOUNCER_STOP_POLICY,<repo>/.bouncer.stop.yaml, then~/.bouncer/stop.yaml, and does nothing when none exists. A repository's file wins, which is how one project is put in or taken out.Nothing is sent when
background_tasksorsession_cronsis non-empty: the agent is waiting on a task, not on the user.New surface, flagged
stops.jsonlkeeps the redacted text locally.bouncer stopsonly, never the hook), from the path the payload named.Verified, and not
Stoppayload was captured from Claude Code 2.1.201 withscripts/capture-hook.mjsand committed astest/fixtures/payloads/stop-main.json. It carrieslast_assistant_message, so no transcript is read to judge.test/stop.test.tsruns the built bundle over empty input, non-JSON, wrong types, a 400,000-character message, a broken policy, a missing policy and a missing key, and asserts empty stdout and exit 0 each time.bouncer stopsis a fixed pattern over the user's words; the report says so.Cost
bench --againstthe base bundle: paired median +0.3 ms judged, +0.1 ms hard-rule, 30 pairs. The bundle is 11 KB larger.CACHE_VERSIONunchanged. 1765 tests pass.Not in this PR
Continuing anything. ADR-015 writes down what would have to be true first, and decides none of it.
Refs #156.
🤖 Generated with Claude Code