feat(templates): reconcile authored scope against originating issues in propose, propose-batch, and decompose-phase - #113
Open
joctaTorres wants to merge 12 commits into
Open
joctaTorres wants to merge 12 commits into
joctaTorres wants to merge 12 commits into
Conversation
Harden the three change-authoring workflows so a scope reduction cannot escape review and a `Closes #N` cannot be claimed before it is earned: originating-issue reconciliation in propose/propose-batch, no premature close-claims in manifests, and prior-plan deferral carry-forward in decompose-phase. Planning artifacts only: features + plan. Targets #100. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`propose`, `propose-batch`, and `decompose-phase` must all carry the same originating-issue reconciliation procedure, close-claim rule, and stop-and-surface guardrail. Three hand-written copies of a rule are the drift mechanism issue #100 is about, so the rules get exactly one author: `scope-reconciliation.ts`, whose three exported constants each body interpolates. - `ISSUE_RECONCILIATION_STEP` — identify every originating issue, fetch each through the project's tracker, enumerate its material requirements from both its explicit fix items and the problems named in its narrative, map each to an authored artifact, and surface every uncovered one as an enumerated decision point. Hedged source wording ("consider", "optionally") does not lower the bar for a security-relevant requirement, and an omission may never be self-approved in plan prose. - `CLOSE_CLAIM_RULES` — a `Fixes #N` / `Closes #N` claim is permitted only when the issue's material requirements are actually implemented; partial work says "partially addresses #N"; a close-claim is an output of verification, never an input of planning. - `STOP_AND_SURFACE_GUARDRAIL` — a de-scope of security-, permission-, or integrity-relevant work halts the workflow and asks the user; an approved deferral requires a filed, linked tracking issue with a named owner. This operationalizes the `security-remediation` standard, which stated the policy but had no enforcement point in the authoring workflows. The prose is agent-neutral (`multi-agent-support`) and tracker-neutral (`generalizable-defaults`): `gh issue view <n>` is named only as a GitHub example, with a paste-the-issue-text fallback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`propose.ts` carried the same body twice — once as the skill's `instructions`, once as the command's `content` — differing in exactly two lines. Every edit had to be made twice or the two surfaces silently diverged, which is the same drift this change exists to eliminate; adding the reconciliation rules to only one copy would have reproduced it inside this very change. The body is now `buildProposeBody()`, parameterized by the only two deltas: the **Input** line and the closing **Prompt** line. Both were verified mechanically: each surface's rendered body is byte-identical to the text before this commit (`diff` of the extracted `instructions` / `content` strings against the previous revision reports no change). Pure refactor — no prose changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gainst scope drift Embeds the shared scope-reconciliation fragments into all three change-authoring workflow bodies, closing the three structural gaps issue #100 demonstrated end to end. propose — a new step 2 runs the originating-issue reconciliation BEFORE any artifact is authored, and the artifact loop completes the requirement-to-artifact map against what was actually written. Steps 2-5 renumber to 3-6; no other prose changed. propose-batch — a new step 4 reconciles the manifest against every originating issue before `ratchet new batch` runs, mapping each requirement onto the phase `goal` / `success` or change-level `done` that carries it. It adds the no-premature-close rule: no `Closes #N` / `Fixes #N` in a phase goal, a phase success criterion, or a change-level done for unscoped work; phase contracts say "targets #N" / "addresses #N"; partial coverage says "partially addresses #N"; the closing linkage is earned at PR-authoring time. The manifest-writing step repeats the phrasing rule at the point the fields are written. decompose-phase — the grounding step now requires reading each prior phase's shipped `plan.md` (the injected `done` criteria are a paraphrase in which plan-prose deferrals are invisible), extracting every out-of-scope / deferred / revisit item, and resolving each as exactly one of carried-forward, tracked against an open issue, or explicitly dropped by the user — silently ignoring one is not an outcome. It also requires earned-close verification before treating an issue as shipped: an unearned `Fixes/Closes #N` is surfaced and its remaining scope carried forward, never inherited as fact. All three bodies embed `CLOSE_CLAIM_RULES` and `STOP_AND_SURFACE_GUARDRAIL` verbatim, exactly once each, by interpolation — no body restates them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…three workflows - `scope-reconciliation.test.ts` asserts each shared constant carries its required rules, and that all three workflow bodies contain `CLOSE_CLAIM_RULES` and `STOP_AND_SURFACE_GUARDRAIL` verbatim — exactly once each, so a body that restated a rule by hand instead of interpolating it fails. - `propose.test.ts` pins the skill/command bodies to differing in exactly the two known lines, covers the reconciliation step and the no-self-approved-omission rule, asserts agent- and tracker-neutrality, and renders the command through every adapter in the registry. - `propose-batch.test.ts` gains the reconciliation step (criterion 1) and the no-premature-`Closes` / "targets #N" / "partially addresses #N" rules (criterion 2), plus an adapter-render assertion. - `decompose-phase.test.ts` covers the prior-plan sweep and the carried-forward / tracked / explicitly-dropped trichotomy (criterion 3), the earned-close verification (criterion 4), the shared guardrails (criterion 5), the shared skill/command body, and an adapter render. - `init-scope-reconciliation.test.ts` proves the acceptance criteria's "both trees" clause through the generator: it runs `init` with claude,opencode and asserts the hardened prose landed in `.claude/skills/…/SKILL.md` AND `.opencode/skills/…/SKILL.md` for all three skills. Those trees are gitignored and generated, so this is the only honest way to prove the clause — hand-editing a tree would be reverted on the next regeneration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Issue #100's acceptance criterion 6 asks for a checkable procedure that demonstrates the reconciliation behavior rather than asserting it. Deterministic coverage of criteria 1-5 stays in vitest so CI never depends on a live agent spawn; this binding is the demonstration. The fixture `.ratchet/evals/fixtures/issue-scope-reconciliation/` is a self-contained ratchet project holding the worked example as INPUT: issue #80's text verbatim (its Why names the permission-posture bypass; its fix proposal hedges the gating requirement as "Consider requiring…"), the phase-one manifest excerpt that hard-codes `Closes #80` and whose change `done` ends "Fixes #80.", and the shipped phase-one plan carrying the self-approved `## Out of scope` bullet — quoted from #100's worked example, ellipsis included, with no invented variants. The `llm-judge` binding does NOT restate the reconciliation procedure — that duplication is what this change eliminates. Its `setup` runs `ratchet init` inside the fixture working copy so the judge reads the procedure out of a real generated `SKILL.md`, which therefore cannot drift from what ships. The success criteria fail closed if the generated skill turns out not to carry the step, so a stale CLI surfaces as an explicit failure rather than a vacuous pass, and fail closed unless ALL THREE escapes are flagged as decision points requiring the user: the unimplemented gate, the unaddressed permission-posture bypass, and the premature close-claim. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spawned judge runs headless in a throwaway working copy, so it needs the same `.claude/settings.json` allowlist the `eval-self-run` fixture already ships (Bash/Read/Glob/Grep/LS). Without it the judge cannot read the fixture files it is asked to reconcile. Force-added because `.claude/` is gitignored at the repo root — exactly how `eval-self-run/.claude/settings.json` is tracked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… workflows Required by the `documentation` standard, which makes the documentation task mandatory and requires `README.md` to be updated alongside the `docs/` Reference entry whenever a change alters a surface the README describes. `docs/configuration/generated-artifacts.md` gains a "Scope reconciliation in the change-authoring workflows" section under the per-tool skills and commands chapter: an `## Overview` Mermaid flowchart (vertical, high-contrast, every `classDef` sets `color:`) followed by Reference entries for the originating-issue reconciliation step, the close-claim rules, the stop-and-surface guardrail, and the deferral carry-forward trichotomy. The diagram was verified to parse with mermaid 11. `README.md` gains a "Scope reconciliation" subsection, and its propose-batch and apply-batch descriptions now match the hardened prose rather than describing behavior this change made stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ation Full suite: 2969 tests across 207 files, all passing. Enforced coverage gate green at 95.54% line coverage against the 95% floor (threshold not lowered). `ratchet validate issue-scope-reconciliation` reports the change valid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s blocker Task 7.2 required README.md to match the hardened prose "wherever it describes the propose / propose-batch / decompose-phase workflows". The batch-workflows table row for propose-batch was updated, but the "Agent workflows" table rows for `propose` and `propose-batch` still described the pre-hardening behavior. Both now name the originating-issue reconciliation and link the new section. Task 6.3 is blocked by a pre-existing judge-harness defect, and its only evidence lived in `.ratchet/evals/runs/`, which is gitignored — the exact "deferral recorded where nothing durable can see it" failure this change exists to prevent. The run ids, the recorded verdicts, and the verified root cause are now recorded on the task itself. The task stays unchecked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8.1 is done — #112 tracks #80's remainder with a named owner, and its three material requirements were each verified against the phase-1 branch before filing. 8.2 was dropped by explicit decision of the repository owner, who will close PR #97 themselves rather than have its body amended. The `security-remediation` standard requires a deferral to carry recorded human sign-off inside the change, not merely a linked issue, so the decision and its reasoning are recorded on the task rather than left in session scrollback. No security scope goes untracked: #112 is linked from #80 independently of #97's body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Applies this change's own reconciliation procedure to the issue that commissioned it: all seven acceptance criteria enumerated, five fully met, two partial and surfaced rather than self-approved. Criterion 6's demonstration cannot be produced here — extractVerdictJson (judge.ts:211, from PR #40 on main) reads the last balanced top-level array in agent stdout, so under any stream-json adapter it parses the envelope's permission_denials instead of the judge's verdict. Both installed adapters (claude, opencode) set emitsStreamJson; the three that don't are not installed. Weakening the spec, `eval record`-ing a pass, or quietly checking the task off were each rejected as the exact patterns this change forbids. Consequently the PR says "partially addresses #100" rather than claiming Closes — the honest-close-claim rule applied to itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ratchet | 1f3900f | Commit Preview URL Branch Preview URL |
Aug 26 2026, 10:59 PM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hardens the three change-authoring workflows so the failure chain #100 documents — a security-relevant half of an issue silently dropped,
Closes #80claimed anyway, the deferral invisible at the next phase boundary — is structurally impossible.What changed
One shared fragment module, three embeds.
src/core/templates/workflows/scope-reconciliation.tsis now the single author of three rule blocks:ISSUE_RECONCILIATION_STEP— identify every originating issue, fetch it, enumerate its material requirements from both its explicit fix items and the problems named in its narrative, map each to an authored feature or task, and surface every uncovered one as an enumerated "issue asks X, this proposal does not include X" decision point. Hedged source wording ("consider", "optionally") explicitly does not lower the bar for a security-relevant requirement — engine: RATCHET_BATCH_AGENT_CMD / RATCHET_EVAL_AGENT_CMD silently replace the agent in production — no gating, no notice, no journal marker, permissions bypassed #80's own "Consider requiring…" is what handed the original agent its de-scope justification.CLOSE_CLAIM_RULES— a close-claim is an output of verification, never an input of planning; partial work says "partially addresses #N".STOP_AND_SURFACE_GUARDRAIL— any de-scope of security-, permission-, or integrity-relevant work halts the workflow and asks; an approved deferral requires a filed, linked tracking issue with a named owner. A prose bullet inplan.mdis not a deferral mechanism.Three copies of a rule are exactly the drift this issue is about, so the rules live in one module each body interpolates once — which also turns "verbatim in all three" into a mechanical assertion instead of a prose review.
Per workflow:
propose— reconciliation step before artifacts are authored; guardrails embedded. Its two verbatim-duplicated bodies (skillinstructions+ commandcontent) were first collapsed into one builder parameterized by the two lines that genuinely differ; adding the rules to only one of them would have reproduced the divergence this issue is about, inside the fix for it.propose-batch— reconciliation before the manifest is scaffolded, plus the no-premature-Closesrule: phase contracts say "targets #N" / "addresses #N", and adonecovering part of an issue says "partially addresses #N".decompose-phase— reads each prior phase'splan.md(not only the injecteddoneparaphrase, in which plan-prose deferrals are invisible), extracts every out-of-scope / deferred / revisit item, and resolves each as carried-forward | tracked | explicitly-dropped. Silently ignoring one is not an available outcome. It also verifies a prior phase'sFixes/Closes #Nwas actually earned before treating the issue as shipped.Both trees, through the generator.
.claude/and.opencode/are gitignored here — they are rendered into consuming projects from these templates.test/core/init-scope-reconciliation.test.tsruns the realInitCommandand asserts the hardened text in both trees for all three skills, so the "both trees" requirement is proven through the generator rather than by hand-editing a tree that regeneration would revert.Verification
Independent verify pass (fresh context, did not implement): PASS, no merge blockers. 20/20 feature scenarios backed by evidence; 207 files / 2969 tests green; coverage 95.54% ≥ 95% gate, threshold untouched;
ratchet validateclean; lint clean.Scope — partially addresses #100
Criteria 1–5 are fully met and mechanically tested. Two are partial, surfaced here rather than self-approved, per the rules this PR adds:
Criterion 6 — the worked-example demonstration. The fixture and
llm-judgebinding are authored, correctly bound (the case resolves tojudged, notunjudged), and runnable. What cannot be produced is a passing verdict:extractVerdictJson(src/core/eval/judge.ts:211, from PR #40, onmain) takes the last balanced top-level[...]in agent stdout, so under any stream-json adapter the real verdict array — which lives inside a JSON string value — is skipped in favour of the result envelope's ownpermission_denialsarray. Both installed adapters (claude,opencode) setemitsStreamJson; the three that don't are not installed, so switching adapters doesn't rescue it either. This branch touches neithersrc/core/eval/norsrc/core/batch/engine/—git diff main...HEADover both paths is empty.This is a repo-wide pre-existing defect: it silently fails-closed every
llm-judgebinding under a stream-json adapter, including the pre-existingeval-self.yaml. It needs its own change and is not fixed here.Weakening the spec so it passes,
ratchet eval record-ing the verdict to green, and quietly checking the task off were each considered and rejected — they are the exact patterns this PR exists to forbid. Task 6.3 is left unchecked with its evidence recorded durably inplan.md(the run records themselves live in gitignored.ratchet/evals/runs/, which is precisely the "deferral recorded where nothing durable can see it" failure this change is about).Criterion 7 — #80's remainder. The tracking half is done: #112 is open, assigned, and enumerates #80's unimplemented remainder as MUST-level close-gates — the ungated override seam, the permission posture voided under an override (
buildAgentSpawnRequest's override branch returns a barebash -c <override>and never callsbuildAdapterRequest(), so the adapter'sresolvePermissionFlags(...)block is skipped), andsrc/commands/batch/config.ts:118printing the posture as enforced with no override awareness. All three were verified againstorigin/erh/phase-1-spawn-env-plumbingbefore filing. The remaining half — amending PR #97'sCloses #80— was declined by the repository owner, who will close #97 directly instead; that decision and its sign-off are recorded on task 8.2 per thesecurity-remediationstandard.Because two criteria are partial, this PR partially addresses #100 and deliberately does not claim
Closes. That is this change's own honest-close-claim rule applied to itself.Standards
Follows
security-remediation(this PR is its enforcement point in the authoring workflows),documentation(docs/configuration/generated-artifacts.md+ README, with a Mermaid overview),testing,multi-agent-support(one shared definition rendered per agent through the adapter registry; bodies assert no agent-specific naming),generalizable-defaults(gh issue view <n>named only as a GitHub example, with a paste-the-issue-text fallback — no tracker baked in),instruction-fed-config(no config read added to any template body), anddelegated-lifecycle(templates remain the single author of lifecycle instruction text).🤖 Generated with Claude Code