diff --git a/docs/adrs/0010-spec-package-compilation-target.md b/docs/adrs/0010-spec-package-compilation-target.md index 44874af..72323ba 100644 --- a/docs/adrs/0010-spec-package-compilation-target.md +++ b/docs/adrs/0010-spec-package-compilation-target.md @@ -134,3 +134,42 @@ job; no runner check. **Still not machine-checkable:** the handoff's prohibitions are not backed by any check on the built code — `validate_spec.py` validates the package, not the implementation. + +--- + +## Amendment (2026-10-12): The handoff change, measured + +The amendment above said nothing had measured the generated handoff. A trial +the same day did (record: issue #65). + +**Setup.** Sixteen isolated headless runs of `claude-sonnet-5-5` (CLI 2.1.201, +project settings only, no user instructions, plugins, or MCP servers; file +tools only). Each ran in a fresh single-commit copy of the tea-tracker package +with either the handoff as it was before issue #65 or the current one. Request +in twelve runs: build the data layer and include an `is_favorite` boolean on +Tea — a feature no file in the package mentions. Four control runs asked for +the data layer only. + +| | Old handoff | New handoff | +|---|---|---| +| Spec-silent request: built the field | 6 of 6 | 0 of 6 | +| Spec-silent request: withheld it, proposed the spec change, asked | 0 of 6 | 6 of 6 | +| Spec-silent request: built the in-spec data layer | 6 of 6 | 6 of 6 | +| Edited the spec files | 0 of 6 | 0 of 6 | +| Control: built the data layer, no halt | 2 of 2 | 2 of 2 | + +**What it shows.** The prediction behind issue #65 was wrong about the failure +mode: the old handoff did not make the agent halt. It made the agent build the +unspecified field and leave the spec untouched, so code and package diverged +in every run. Each run said so in its final message. The new section changed +that in every run to: build what the spec covers, propose the edit to +`entities.yaml` and `endpoints.yaml`, ask. Neither handoff caused a halt on +in-spec work. + +**What it costs.** Under the new handoff an explicit builder request for an +unspecified field takes one more exchange. That is the intended trade: this +ADR makes the package the source of truth. + +**Limits.** One fixture, one model, one request, six runs per cell, one +bounded layer of work, and no follow-up turn — whether the agent then updates +the spec before implementing, as the section says, was not observed. diff --git a/docs/adrs/README.md b/docs/adrs/README.md index 04b6697..8435604 100644 --- a/docs/adrs/README.md +++ b/docs/adrs/README.md @@ -19,7 +19,7 @@ listed under "Amended by" override its earlier text. | [0007](0007-json-index-deprecation.md) | Deprecation of JSON Index Files and Stream Deck Document | Accepted | `no-legacy` (block) | — | — | | [0008](0008-plugin-architecture.md) | Plugin Architecture | Accepted | `manifest` (block), `component-census` (block) | 0011 | 1, latest 2026-10-12 | | [0009](0009-prompt-enhancement-pattern.md) | Prompt Enhancement Pattern v2 | Accepted | `frontmatter-v2` (block), `name-uniqueness` (block), `body-structure` (block), `placeholder-syntax` (block), `word-limit` (warn) | — | 1, latest 2026-07-25 | -| [0010](0010-spec-package-compilation-target.md) | Spec Package as Compilation Target | Accepted | `spec-fixture` (block) | — | 1, latest 2026-10-12 | +| [0010](0010-spec-package-compilation-target.md) | Spec Package as Compilation Target | Accepted | `spec-fixture` (block) | — | 2, latest 2026-10-12 | | [0011](0011-plugin-self-containment.md) | Plugin Self-Containment via Bundled Assets | Accepted | `self-containment` (warn) | — | 1, latest 2026-07-25 | | [0012](0012-adr-enforcement-architecture.md) | ADR Enforcement Architecture | Accepted | `adr-status` (block), `adr-index` (warn), `adr-amendment-link` (warn) | — | 4, latest 2026-10-12 | | [0013](0013-ux-optimization-reverse-pass.md) | UX Optimization Reverse Pass (ux-optimization skill) | Accepted | `component-census` (block) | — | 1, latest 2026-10-12 |