Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions docs/adrs/0010-spec-package-compilation-target.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,42 @@ job; no runner check.
**Still not machine-checkable:** the handoff's prohibitions are not backed by
any check on the built code β€” `validate_spec.py` validates the package, not
the implementation.

---

## Amendment (2026-10-12): The handoff change, measured

The amendment above said nothing had measured the generated handoff. A trial
the same day did (record: issue #65).

**Setup.** Sixteen isolated headless runs of `claude-sonnet-5-5` (CLI 2.1.201,
project settings only, no user instructions, plugins, or MCP servers; file
tools only). Each ran in a fresh single-commit copy of the tea-tracker package
with either the handoff as it was before issue #65 or the current one. Request
in twelve runs: build the data layer and include an `is_favorite` boolean on
Tea β€” a feature no file in the package mentions. Four control runs asked for
the data layer only.

| | Old handoff | New handoff |
|---|---|---|
| Spec-silent request: built the field | 6 of 6 | 0 of 6 |
| Spec-silent request: withheld it, proposed the spec change, asked | 0 of 6 | 6 of 6 |
| Spec-silent request: built the in-spec data layer | 6 of 6 | 6 of 6 |
| Edited the spec files | 0 of 6 | 0 of 6 |
| Control: built the data layer, no halt | 2 of 2 | 2 of 2 |

**What it shows.** The prediction behind issue #65 was wrong about the failure
mode: the old handoff did not make the agent halt. It made the agent build the
unspecified field and leave the spec untouched, so code and package diverged
in every run. Each run said so in its final message. The new section changed
that in every run to: build what the spec covers, propose the edit to
`entities.yaml` and `endpoints.yaml`, ask. Neither handoff caused a halt on
in-spec work.

**What it costs.** Under the new handoff an explicit builder request for an
unspecified field takes one more exchange. That is the intended trade: this
ADR makes the package the source of truth.

**Limits.** One fixture, one model, one request, six runs per cell, one
bounded layer of work, and no follow-up turn β€” whether the agent then updates
the spec before implementing, as the section says, was not observed.
2 changes: 1 addition & 1 deletion docs/adrs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ listed under "Amended by" override its earlier text.
| [0007](0007-json-index-deprecation.md) | Deprecation of JSON Index Files and Stream Deck Document | Accepted | `no-legacy` (block) | β€” | β€” |
| [0008](0008-plugin-architecture.md) | Plugin Architecture | Accepted | `manifest` (block), `component-census` (block) | 0011 | 1, latest 2026-10-12 |
| [0009](0009-prompt-enhancement-pattern.md) | Prompt Enhancement Pattern v2 | Accepted | `frontmatter-v2` (block), `name-uniqueness` (block), `body-structure` (block), `placeholder-syntax` (block), `word-limit` (warn) | β€” | 1, latest 2026-07-25 |
| [0010](0010-spec-package-compilation-target.md) | Spec Package as Compilation Target | Accepted | `spec-fixture` (block) | β€” | 1, latest 2026-10-12 |
| [0010](0010-spec-package-compilation-target.md) | Spec Package as Compilation Target | Accepted | `spec-fixture` (block) | β€” | 2, latest 2026-10-12 |
| [0011](0011-plugin-self-containment.md) | Plugin Self-Containment via Bundled Assets | Accepted | `self-containment` (warn) | β€” | 1, latest 2026-07-25 |
| [0012](0012-adr-enforcement-architecture.md) | ADR Enforcement Architecture | Accepted | `adr-status` (block), `adr-index` (warn), `adr-amendment-link` (warn) | β€” | 4, latest 2026-10-12 |
| [0013](0013-ux-optimization-reverse-pass.md) | UX Optimization Reverse Pass (ux-optimization skill) | Accepted | `component-census` (block) | β€” | 1, latest 2026-10-12 |
Expand Down
Loading