fix(learn): canonical stateRoot resolution + scope-routed guideline synthesis - #33
Open
cmschuetz wants to merge 1 commit into
Open
fix(learn): canonical stateRoot resolution + scope-routed guideline synthesis#33cmschuetz wants to merge 1 commit into
cmschuetz wants to merge 1 commit into
Conversation
…ynthesis
The learn skill was the only strapped skill that never resolved stateRoot via
state.mjs — it left resolution to improvisation, so any mis-resolution glob'd to
zero and reported "no critiques, stop", indistinguishable from genuinely-none.
It also dumped every synthesized guideline into one CLAUDE.md, whose scope is
only "developing the plugin repo" — so harness-behavior rules learned from runs
never fired for runs against other repos.
- Add `state.mjs runroot`: slug-less { stateRoot, runRoot } resolution that
hard-fails on an unresolvable/relative anchor, so an empty cross-run glob is
unambiguously "no runs". Documented in conventions.md, +3 CLI tests.
- learn Step 1 now resolves via `runroot`, echoes the root + match count, and
treats a resolution error as a loud stop (never "no critiques").
- learn Steps 2-5 classify each cluster harness-behavior vs repo-development and
route it: harness rules to the most specific stage prompt / SKILL / conventions;
repo-dev rules to the *pertaining* repo's CLAUDE.md (resolved from the run's
repos map), not necessarily the plugin's.
- Relocate the 4 generalizable guidelines synthesized this pass into their
targeted prompts so they reach every run:
* verify plan claims vs origin/main (fetch first) -> plan.ts planner
* add a real test suite as the gate -> plan.ts planner
* review design decisions vs the ask's intent -> review-loop.ts lens a
* no grep-guard prose regression tests -> implement.ts implementer
* reconcile plan vs current main before implement -> implement/SKILL.md
CLAUDE.md keeps only genuinely repo-dev rules (new-repo/breaking-changes,
no-machinery-without-signal) + a pointer to where harness guidance lives.
- Bump plugin 0.6.0 -> 0.7.0.
5 tasks
cmschuetz
added a commit
that referenced
this pull request
Jul 28, 2026
Remove four genuine, meaning-preserving redundancies from the harness stage prompts, each verified via the D1-D3 eval suite's A/B mode with no correctness regression: - planner (stages/plan.ts, steps 3-5): define "meaningful diff" once; state the cross-repo base rule + ordering-only consequence once (drop step 4's restatement) - reviewer (review-loop.ts): collapse the AC-checklist paragraph's doubled equal-weight clause - soundness lens b (review-loop.ts PLAN_LENSES.b): define "meaningful diff" once instead of repeating the exclusion phrase twice - implementer (stages/implement.ts): fold the filler lead-in into the header and drop the doubled "commit" in the closing paragraph The eval cases (planner/reviewer/implementer) now carry baseline->candidate variants so `bun run eval --suite src/eval/suites/harness --ab` reproduces the verification (exit 0: correctness held at parity across all cases). End-to-end envelope cost/latency is below the harness's agentic execution-noise floor, so the attributable win is the deterministic prompt-text reduction (-302 chars); the eval's decisive role is certifying zero correctness regression. A written PR-#33 verdict and full A/B numbers live in the run dir's optimizations-report.md. Regenerates plugins/strapped/workflows/strapped-run.js; plugin 0.8.0 -> 0.8.1 (patch: prompt compaction, no behavior change). Implements D4 of the evaluation-system run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Investigating "is
/strapped:learnbroken? feedback isn't being collected" surfaced that feedback was being collected (10 unsynthesized critiques across 5 runs) — the failure was a false negative:learnwas the only strapped skill that never resolvedstateRootviastate.mjs, so any mis-resolution glob'd to zero and reported "no critiques, stop" — indistinguishable from genuinely none.A second, deeper issue:
learndumped every synthesized guideline into oneCLAUDE.md, whose load scope is only "working in the plugin repo." So harness-behavior rules learned from runs never fired for runs against other target repos — the majority of intended use.What
Resolver gap (the concrete bug)
state.mjs runroot— slug-less{ stateRoot, runRoot }resolution that hard-fails on an unresolvable/relative anchor, so an empty cross-run glob is unambiguously "no runs" and never a silently-wrong root. Documented inconventions.md; +3 CLI tests.learnStep 1 now resolves viarunroot, echoes the resolved root + match count, and treats a resolution error as a loud stop.Scope routing (the design fix)
learnSteps 2–5 now classify each cluster harness-behavior vs repo-development and route it:src/workflows/strapped-run/stages/*.ts,review-loop.ts) /SKILL.mdstep /conventions.mdCLAUDE.md(resolved from the run'sreposmap), not necessarily the plugin's.The 4 generalizable guidelines synthesized this pass were relocated out of
CLAUDE.mdinto their targeted prompts so they reach every run:origin/main(fetch first)plan.tsplannerplan.tsplannerreview-loop.tscompleteness lensimplement.tsimplementerimplement/SKILL.mdpre-flightCLAUDE.mdkeeps only genuinely repo-dev rules (new-repo/breaking-changes-fine, no-machinery-without-signal) + a pointer to where harness guidance lives.Plugin
0.6.0 → 0.7.0.Not addressed (flagged)
Critique writing remains a pure model-followed instruction with no hook/script enforcement — autonomous
/strapped:runflows skip it silently. Left as a follow-up.Test
bun run typecheck && bun run lint && bun test→ 109 pass (+3). Generatedstrapped-run.jsrebuilt and in sync; all 4 relocated snippets verified present.