feat(layer5): v1.3 — reasoning-trace prototype at three choice points - #87
Merged
Conversation
Three commit sites instrument machine-parseable reasoning traces: 1. walker-concern — `bin/walker.py::record_answer` emits a trace before `state.answered[concern_id] = answer` when the concern has >=2 prefab options. Exemplar-binding concerns (6 known IDs) emit choice_point= "exemplar-binding" with validation_anchor="tier2-cross-view-gate" instead. 2. substitution — `bin/eval_metadata.py::write_sidecar` accepts a new `layer5_trace: list[dict] | None` kwarg and writes it under the top-level `layer5_trace` key. `bin/_layer5.project_substitution_trace` projects an extended substitution dict (carrying optional_considered + rationale) into a trace record. One event source, two payload keys. 3. exemplar-binding — handled through the same `record_answer` path as walker-concern (the concern carries catalog slugs as prefab_options). Schema: six fields + timestamp — choice_point, step_or_concern_id, options_considered, selected, rationale, validation_anchor, source_anchor. Lives in `.eval.json` sidecar under `layer5_trace: [...]`. No new file type. New `bin/_layer5.py` module owns `build_trace_record()` and `project_substitution_trace()`. `SPECTRE_LAYER5=off` disables emission entirely for token-cost A/B tests. Skips when options_considered <= 1. WalkState gains `layer5_trace: list[dict]` field, persisted + loaded (backwards-compat: defaults to [] on old state files). Both JSON-mode CLI payloads (init-or-resume, get-state) include the field. Glossary: two new term entries — `term:layer5-trace` and `term:layer5-choice-point`. Tests: 20 new tests across three files, all green. Full suite 1987 passed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
5 tasks
Joncik91
added a commit
that referenced
this pull request
May 15, 2026
…ype (#88) Minor release composing three Wave 2 PRs (#85, #86, #87) plus a small walker-side plumbing fixup surfaced by the Wave 3 Opus aggregate review. - Plugin marketplace bump 1.2.1 → 1.3.0 (both metadata.version and plugins[0].version) - README badges: test count 1967 → 2003, version 1.2.1 → 1.3.0 - README components line: v1.2 → v1.3 - CHANGELOG entry summarizing item 8 (catalog seed), item 9 (vocab fail-soft with block firewall), item 10 (layer5 prototype), and the walker fixup - bin/walker.py: _VIEW_TO_CATALOG_VIEW_TYPE realigned so product-input routes to input-shape and product-output routes to output-shape, the exemplar-options call site queries input-shape instead of help-text, and the input-exemplar-pi concern prompt references the input-shape view type instead of the v1.1 placeholder language. The walker's binding-suggestion path and the cross_view_gate's validation path now query the same catalog view types. No spec contract changes — v1.0/v1.1/v1.1.1/v1.2.0/v1.2.1 locked specs continue to validate. 2003 tests passing (1967 v1.2.1 baseline + 36 across the three v1.3 PRs). Zero regressions. Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Spectre v2.0 requires machine-parseable reasoning traces at agent choice points (Layer 5). This PR ships the v1.3 prototype: traces at exactly the three named commit sites, with a minimal six-field schema that can be extended for v2.0 without breaking existing consumers.
Without this, there is no audit trail for why the walker chose a particular exemplar, concern answer, or substitution — only the final locked spec, not the reasoning.
What changed
bin/_layer5.py(new) —build_trace_record()helper;project_substitution_trace()for substitution dict projection;SPECTRE_LAYER5=offkill switch.bin/walker.py—WalkState.layer5_trace: list[dict]field;record_answeremitswalker-concernorexemplar-bindingtraces (6 known exemplar concern IDs get the latter);persist/load/both JSON-mode CLI payloads updated.bin/eval_metadata.py—write_sidecargainslayer5_trace: list[dict] | Nonekwarg; writeslayer5_tracekey when provided; CLIwrite-sidecarsubcommand passes it through.docs/glossary.md—term:layer5-traceandterm:layer5-choice-pointentries.Schema (six fields + timestamp)
{ "choice_point": "walker-concern" | "substitution" | "exemplar-binding", "step_or_concern_id": "<id>", "options_considered": ["<a>", "<b>"], "selected": "<a>", "rationale": "<why>", "validation_anchor": "<tier-or-check | null>", "source_anchor": "<spec-section | null>", "timestamp": "<ISO8601>" }Lives in
.eval.jsonsidecar underlayer5_trace: [...]. No new file type.Test plan
tests/test_layer5_walker_trace.py— 6 tests: multichoice appends trace, six fields present, zero-prefab skips, single-prefab skips, accumulates multiple,SPECTRE_LAYER5=offdisables.tests/test_layer5_substitution_trace.py— 6 tests: None omits key, empty list written, round-trip, projection with options, projection without options returns None, coexists with substitutions.tests/test_layer5_exemplar_trace.py— 8 tests: exemplar-binding emitted, six fields present, validation_anchor=tier2-cross-view-gate, non-exemplar concern gets walker-concern, three individual exemplar IDs each verified, env-var kill switch.pytest tests/→ 1987 passed, 0 failed.🤖 Generated with Claude Code