Skip to content

feat(layer5): v1.3 — reasoning-trace prototype at three choice points - #87

Merged
Joncik91 merged 1 commit into
mainfrom
v1.3-layer5-prototype
May 15, 2026
Merged

feat(layer5): v1.3 — reasoning-trace prototype at three choice points#87
Joncik91 merged 1 commit into
mainfrom
v1.3-layer5-prototype

Conversation

@Joncik91

Copy link
Copy Markdown
Owner

Why

Spectre v2.0 requires machine-parseable reasoning traces at agent choice points (Layer 5). This PR ships the v1.3 prototype: traces at exactly the three named commit sites, with a minimal six-field schema that can be extended for v2.0 without breaking existing consumers.

Without this, there is no audit trail for why the walker chose a particular exemplar, concern answer, or substitution — only the final locked spec, not the reasoning.

What changed

  • bin/_layer5.py (new) — build_trace_record() helper; project_substitution_trace() for substitution dict projection; SPECTRE_LAYER5=off kill switch.
  • bin/walker.pyWalkState.layer5_trace: list[dict] field; record_answer emits walker-concern or exemplar-binding traces (6 known exemplar concern IDs get the latter); persist/load/both JSON-mode CLI payloads updated.
  • bin/eval_metadata.pywrite_sidecar gains layer5_trace: list[dict] | None kwarg; writes layer5_trace key when provided; CLI write-sidecar subcommand passes it through.
  • docs/glossary.mdterm:layer5-trace and term:layer5-choice-point entries.
  • Three new test files — 20 tests covering all three choice points, the env-var kill switch, six-field completeness, and backwards-compat (None → key absent).

Schema (six fields + timestamp)

{
  "choice_point": "walker-concern" | "substitution" | "exemplar-binding",
  "step_or_concern_id": "<id>",
  "options_considered": ["<a>", "<b>"],
  "selected": "<a>",
  "rationale": "<why>",
  "validation_anchor": "<tier-or-check | null>",
  "source_anchor": "<spec-section | null>",
  "timestamp": "<ISO8601>"
}

Lives in .eval.json sidecar under layer5_trace: [...]. No new file type.

Test plan

  • tests/test_layer5_walker_trace.py — 6 tests: multichoice appends trace, six fields present, zero-prefab skips, single-prefab skips, accumulates multiple, SPECTRE_LAYER5=off disables.
  • tests/test_layer5_substitution_trace.py — 6 tests: None omits key, empty list written, round-trip, projection with options, projection without options returns None, coexists with substitutions.
  • tests/test_layer5_exemplar_trace.py — 8 tests: exemplar-binding emitted, six fields present, validation_anchor=tier2-cross-view-gate, non-exemplar concern gets walker-concern, three individual exemplar IDs each verified, env-var kill switch.
  • Full suite: pytest tests/ → 1987 passed, 0 failed.

🤖 Generated with Claude Code

Three commit sites instrument machine-parseable reasoning traces:

1. walker-concern — `bin/walker.py::record_answer` emits a trace before
   `state.answered[concern_id] = answer` when the concern has >=2 prefab
   options. Exemplar-binding concerns (6 known IDs) emit choice_point=
   "exemplar-binding" with validation_anchor="tier2-cross-view-gate" instead.

2. substitution — `bin/eval_metadata.py::write_sidecar` accepts a new
   `layer5_trace: list[dict] | None` kwarg and writes it under the top-level
   `layer5_trace` key. `bin/_layer5.project_substitution_trace` projects an
   extended substitution dict (carrying optional_considered + rationale) into a
   trace record. One event source, two payload keys.

3. exemplar-binding — handled through the same `record_answer` path as
   walker-concern (the concern carries catalog slugs as prefab_options).

Schema: six fields + timestamp — choice_point, step_or_concern_id,
options_considered, selected, rationale, validation_anchor, source_anchor.
Lives in `.eval.json` sidecar under `layer5_trace: [...]`. No new file type.

New `bin/_layer5.py` module owns `build_trace_record()` and
`project_substitution_trace()`. `SPECTRE_LAYER5=off` disables emission
entirely for token-cost A/B tests. Skips when options_considered <= 1.

WalkState gains `layer5_trace: list[dict]` field, persisted + loaded
(backwards-compat: defaults to [] on old state files). Both JSON-mode
CLI payloads (init-or-resume, get-state) include the field.

Glossary: two new term entries — `term:layer5-trace` and
`term:layer5-choice-point`.

Tests: 20 new tests across three files, all green. Full suite 1987 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@Joncik91
Joncik91 merged commit 8ce2721 into main May 15, 2026
2 checks passed
@Joncik91
Joncik91 deleted the v1.3-layer5-prototype branch May 15, 2026 21:51
Joncik91 added a commit that referenced this pull request May 15, 2026
…ype (#88)

Minor release composing three Wave 2 PRs (#85, #86, #87) plus a small
walker-side plumbing fixup surfaced by the Wave 3 Opus aggregate review.

- Plugin marketplace bump 1.2.1 → 1.3.0 (both metadata.version and
  plugins[0].version)
- README badges: test count 1967 → 2003, version 1.2.1 → 1.3.0
- README components line: v1.2 → v1.3
- CHANGELOG entry summarizing item 8 (catalog seed), item 9 (vocab
  fail-soft with block firewall), item 10 (layer5 prototype), and the
  walker fixup
- bin/walker.py: _VIEW_TO_CATALOG_VIEW_TYPE realigned so product-input
  routes to input-shape and product-output routes to output-shape, the
  exemplar-options call site queries input-shape instead of help-text,
  and the input-exemplar-pi concern prompt references the input-shape
  view type instead of the v1.1 placeholder language. The walker's
  binding-suggestion path and the cross_view_gate's validation path now
  query the same catalog view types.

No spec contract changes — v1.0/v1.1/v1.1.1/v1.2.0/v1.2.1 locked specs
continue to validate.

2003 tests passing (1967 v1.2.1 baseline + 36 across the three v1.3
PRs). Zero regressions.

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant