/compile handoff: give the implementation agent a move when the spec is silent #65

Activity

1. chrispezza

   added

   found-workStatus: found while doing other workStatus: found while doing other work

   on Oct 12, 2026
2. chrispezza

   closed this as completedin #69on Oct 12, 2026
3. chrispezza commented on Oct 12, 2026

   

   chrispezza

   on Oct 12, 2026

   Last edited by chrispezza

   CollaboratorAuthor

   More actions

   Trial result, 2026-10-12

   Twelve runs of a Sonnet subagent acting as the implementation agent on a fresh copy of the tea-tracker package. Eight had the new handoff, four the old one. Two requests: add a tasting_notes field while building the data layer, and add a search parameter while building list-teas.

   	New handoff (8)	Old handoff (4)
   Built the in-spec part	8	4
   Built the extra unannounced	0	0
   Refused everything	0	0
   Edited the spec files	0	0
   Named the spec files that would change	8	3
   Offered to add the extra without a spec change	0	1

   Reading. The halt this issue predicted did not occur with the old handoff: all four old-handoff runs built the in-spec part, withheld the extra and said the spec must change first. The difference is in how usable the reply was. Every new-handoff run listed the files that would change, with the specific edit, and asked a direct question. Of the old-handoff runs, three named the files more briefly, one was vague about the change, and one offered to add the field "as a documented exception" without a spec change.

   Limits.

   • Run as subagents of a contributor session, not as isolated headless agents, because no CLI login was available. The handoff was read on instruction, not auto-loaded, and the owner's global instructions were visible to the subagents (one run cited a convention from them).
   • Four old-handoff runs, two per request. A difference of one run is not an effect.
   • tasting_notes was not a spec-silent request: context/problem.md lists tasting notes as deferred, though no ADR does. Neither request tested a truly silent case.
   • Requests were bounded to one layer each; a full build was not attempted.
4. chrispezza commented on Oct 12, 2026

   

   chrispezza

   on Oct 12, 2026

   CollaboratorAuthor

   More actions

   Headless rerun with a spec-silent request, 2026-10-12

   This supersedes the subagent trial above, which was not isolated and used requests the package already mentioned.

   Sixteen isolated headless runs of claude-sonnet-5-5 (CLI 2.1.201, project settings only; a probe confirmed no user instructions, plugins or skills were visible). Fresh single-commit copy of the tea-tracker package per run. Request: build the data layer and include an is_favorite boolean on Tea. No file in the package mentions favourites.

   	Old handoff	New handoff
   Built the unspecified field	6 of 6	0 of 6
   Withheld it, proposed the spec change, asked	0 of 6	6 of 6
   Built the in-spec data layer	6 of 6	6 of 6
   Edited the spec files	0 of 6	0 of 6
   Control (data layer only): built, no halt	2 of 2	2 of 2

   • The failure this issue predicted, a halt, did not occur. The old handoff's actual failure was building the field and leaving the spec untouched, so code and package diverged in every run. Each run said so in its final message.
   • The new section changed that in every run.
   • Limits: one fixture, one model, one request, six runs per cell, one bounded layer, no follow-up turn. Whether the agent updates the spec before implementing once the builder agrees was not observed.

   Recorded in ADR 0010 (PR #70). Total cost of the batch as reported by the CLI: $3.49.
5. chrispezza commented on Oct 12, 2026

   

   chrispezza

   on Oct 12, 2026

   Last edited by chrispezza

   CollaboratorAuthor

   More actions

   Follow-up turn trial, 2026-10-12: after a bare approval, 6 of 6 runs edited the spec before any code. 3 of 6 left the package failing validation, because nothing in the handoff says to revalidate and the validator is not in the handed-off folder. Details and options are in #71.