Skip to content

feat(templates): reconcile authored scope against originating issues in propose, propose-batch, and decompose-phase - #113

Open
joctaTorres wants to merge 12 commits into
mainfrom
fix-issue-100
Open

joctaTorres wants to merge 12 commits into
mainfrom
fix-issue-100

Conversation

@joctaTorres

Copy link
Copy Markdown
Owner

Hardens the three change-authoring workflows so the failure chain #100 documents — a security-relevant half of an issue silently dropped, Closes #80 claimed anyway, the deferral invisible at the next phase boundary — is structurally impossible.

What changed

One shared fragment module, three embeds. src/core/templates/workflows/scope-reconciliation.ts is now the single author of three rule blocks:

  • ISSUE_RECONCILIATION_STEP — identify every originating issue, fetch it, enumerate its material requirements from both its explicit fix items and the problems named in its narrative, map each to an authored feature or task, and surface every uncovered one as an enumerated "issue asks X, this proposal does not include X" decision point. Hedged source wording ("consider", "optionally") explicitly does not lower the bar for a security-relevant requirement — engine: RATCHET_BATCH_AGENT_CMD / RATCHET_EVAL_AGENT_CMD silently replace the agent in production — no gating, no notice, no journal marker, permissions bypassed #80's own "Consider requiring…" is what handed the original agent its de-scope justification.
  • CLOSE_CLAIM_RULES — a close-claim is an output of verification, never an input of planning; partial work says "partially addresses #N".
  • STOP_AND_SURFACE_GUARDRAIL — any de-scope of security-, permission-, or integrity-relevant work halts the workflow and asks; an approved deferral requires a filed, linked tracking issue with a named owner. A prose bullet in plan.md is not a deferral mechanism.

Three copies of a rule are exactly the drift this issue is about, so the rules live in one module each body interpolates once — which also turns "verbatim in all three" into a mechanical assertion instead of a prose review.

Per workflow:

  • propose — reconciliation step before artifacts are authored; guardrails embedded. Its two verbatim-duplicated bodies (skill instructions + command content) were first collapsed into one builder parameterized by the two lines that genuinely differ; adding the rules to only one of them would have reproduced the divergence this issue is about, inside the fix for it.
  • propose-batch — reconciliation before the manifest is scaffolded, plus the no-premature-Closes rule: phase contracts say "targets #N" / "addresses #N", and a done covering part of an issue says "partially addresses #N".
  • decompose-phase — reads each prior phase's plan.md (not only the injected done paraphrase, in which plan-prose deferrals are invisible), extracts every out-of-scope / deferred / revisit item, and resolves each as carried-forward | tracked | explicitly-dropped. Silently ignoring one is not an available outcome. It also verifies a prior phase's Fixes/Closes #N was actually earned before treating the issue as shipped.

Both trees, through the generator. .claude/ and .opencode/ are gitignored here — they are rendered into consuming projects from these templates. test/core/init-scope-reconciliation.test.ts runs the real InitCommand and asserts the hardened text in both trees for all three skills, so the "both trees" requirement is proven through the generator rather than by hand-editing a tree that regeneration would revert.

Verification

Independent verify pass (fresh context, did not implement): PASS, no merge blockers. 20/20 feature scenarios backed by evidence; 207 files / 2969 tests green; coverage 95.54% ≥ 95% gate, threshold untouched; ratchet validate clean; lint clean.

Scope — partially addresses #100

Criteria 1–5 are fully met and mechanically tested. Two are partial, surfaced here rather than self-approved, per the rules this PR adds:

Criterion 6 — the worked-example demonstration. The fixture and llm-judge binding are authored, correctly bound (the case resolves to judged, not unjudged), and runnable. What cannot be produced is a passing verdict: extractVerdictJson (src/core/eval/judge.ts:211, from PR #40, on main) takes the last balanced top-level [...] in agent stdout, so under any stream-json adapter the real verdict array — which lives inside a JSON string value — is skipped in favour of the result envelope's own permission_denials array. Both installed adapters (claude, opencode) set emitsStreamJson; the three that don't are not installed, so switching adapters doesn't rescue it either. This branch touches neither src/core/eval/ nor src/core/batch/engine/git diff main...HEAD over both paths is empty.

This is a repo-wide pre-existing defect: it silently fails-closed every llm-judge binding under a stream-json adapter, including the pre-existing eval-self.yaml. It needs its own change and is not fixed here.

Weakening the spec so it passes, ratchet eval record-ing the verdict to green, and quietly checking the task off were each considered and rejected — they are the exact patterns this PR exists to forbid. Task 6.3 is left unchecked with its evidence recorded durably in plan.md (the run records themselves live in gitignored .ratchet/evals/runs/, which is precisely the "deferral recorded where nothing durable can see it" failure this change is about).

Criterion 7 — #80's remainder. The tracking half is done: #112 is open, assigned, and enumerates #80's unimplemented remainder as MUST-level close-gates — the ungated override seam, the permission posture voided under an override (buildAgentSpawnRequest's override branch returns a bare bash -c <override> and never calls buildAdapterRequest(), so the adapter's resolvePermissionFlags(...) block is skipped), and src/commands/batch/config.ts:118 printing the posture as enforced with no override awareness. All three were verified against origin/erh/phase-1-spawn-env-plumbing before filing. The remaining half — amending PR #97's Closes #80 — was declined by the repository owner, who will close #97 directly instead; that decision and its sign-off are recorded on task 8.2 per the security-remediation standard.

Because two criteria are partial, this PR partially addresses #100 and deliberately does not claim Closes. That is this change's own honest-close-claim rule applied to itself.

Standards

Follows security-remediation (this PR is its enforcement point in the authoring workflows), documentation (docs/configuration/generated-artifacts.md + README, with a Mermaid overview), testing, multi-agent-support (one shared definition rendered per agent through the adapter registry; bodies assert no agent-specific naming), generalizable-defaults (gh issue view <n> named only as a GitHub example, with a paste-the-issue-text fallback — no tracker baked in), instruction-fed-config (no config read added to any template body), and delegated-lifecycle (templates remain the single author of lifecycle instruction text).

🤖 Generated with Claude Code

joctaTorres and others added 12 commits August 26, 2026 18:55
Harden the three change-authoring workflows so a scope reduction cannot
escape review and a `Closes #N` cannot be claimed before it is earned:
originating-issue reconciliation in propose/propose-batch, no premature
close-claims in manifests, and prior-plan deferral carry-forward in
decompose-phase.

Planning artifacts only: features + plan. Targets #100.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`propose`, `propose-batch`, and `decompose-phase` must all carry the same
originating-issue reconciliation procedure, close-claim rule, and
stop-and-surface guardrail. Three hand-written copies of a rule are the drift
mechanism issue #100 is about, so the rules get exactly one author:
`scope-reconciliation.ts`, whose three exported constants each body
interpolates.

- `ISSUE_RECONCILIATION_STEP` — identify every originating issue, fetch each
  through the project's tracker, enumerate its material requirements from both
  its explicit fix items and the problems named in its narrative, map each to an
  authored artifact, and surface every uncovered one as an enumerated decision
  point. Hedged source wording ("consider", "optionally") does not lower the bar
  for a security-relevant requirement, and an omission may never be
  self-approved in plan prose.
- `CLOSE_CLAIM_RULES` — a `Fixes #N` / `Closes #N` claim is permitted only when
  the issue's material requirements are actually implemented; partial work says
  "partially addresses #N"; a close-claim is an output of verification, never an
  input of planning.
- `STOP_AND_SURFACE_GUARDRAIL` — a de-scope of security-, permission-, or
  integrity-relevant work halts the workflow and asks the user; an approved
  deferral requires a filed, linked tracking issue with a named owner.

This operationalizes the `security-remediation` standard, which stated the
policy but had no enforcement point in the authoring workflows. The prose is
agent-neutral (`multi-agent-support`) and tracker-neutral
(`generalizable-defaults`): `gh issue view <n>` is named only as a GitHub
example, with a paste-the-issue-text fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`propose.ts` carried the same body twice — once as the skill's `instructions`,
once as the command's `content` — differing in exactly two lines. Every edit
had to be made twice or the two surfaces silently diverged, which is the same
drift this change exists to eliminate; adding the reconciliation rules to only
one copy would have reproduced it inside this very change.

The body is now `buildProposeBody()`, parameterized by the only two deltas:
the **Input** line and the closing **Prompt** line. Both were verified
mechanically: each surface's rendered body is byte-identical to the text before
this commit (`diff` of the extracted `instructions` / `content` strings against
the previous revision reports no change). Pure refactor — no prose changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gainst scope drift

Embeds the shared scope-reconciliation fragments into all three
change-authoring workflow bodies, closing the three structural gaps issue #100
demonstrated end to end.

propose — a new step 2 runs the originating-issue reconciliation BEFORE any
artifact is authored, and the artifact loop completes the requirement-to-artifact
map against what was actually written. Steps 2-5 renumber to 3-6; no other prose
changed.

propose-batch — a new step 4 reconciles the manifest against every originating
issue before `ratchet new batch` runs, mapping each requirement onto the phase
`goal` / `success` or change-level `done` that carries it. It adds the
no-premature-close rule: no `Closes #N` / `Fixes #N` in a phase goal, a phase
success criterion, or a change-level done for unscoped work; phase contracts say
"targets #N" / "addresses #N"; partial coverage says "partially addresses #N";
the closing linkage is earned at PR-authoring time. The manifest-writing step
repeats the phrasing rule at the point the fields are written.

decompose-phase — the grounding step now requires reading each prior phase's
shipped `plan.md` (the injected `done` criteria are a paraphrase in which
plan-prose deferrals are invisible), extracting every out-of-scope / deferred /
revisit item, and resolving each as exactly one of carried-forward, tracked
against an open issue, or explicitly dropped by the user — silently ignoring one
is not an outcome. It also requires earned-close verification before treating an
issue as shipped: an unearned `Fixes/Closes #N` is surfaced and its remaining
scope carried forward, never inherited as fact.

All three bodies embed `CLOSE_CLAIM_RULES` and `STOP_AND_SURFACE_GUARDRAIL`
verbatim, exactly once each, by interpolation — no body restates them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…three workflows

- `scope-reconciliation.test.ts` asserts each shared constant carries its
  required rules, and that all three workflow bodies contain `CLOSE_CLAIM_RULES`
  and `STOP_AND_SURFACE_GUARDRAIL` verbatim — exactly once each, so a body that
  restated a rule by hand instead of interpolating it fails.
- `propose.test.ts` pins the skill/command bodies to differing in exactly the two
  known lines, covers the reconciliation step and the no-self-approved-omission
  rule, asserts agent- and tracker-neutrality, and renders the command through
  every adapter in the registry.
- `propose-batch.test.ts` gains the reconciliation step (criterion 1) and the
  no-premature-`Closes` / "targets #N" / "partially addresses #N" rules
  (criterion 2), plus an adapter-render assertion.
- `decompose-phase.test.ts` covers the prior-plan sweep and the
  carried-forward / tracked / explicitly-dropped trichotomy (criterion 3), the
  earned-close verification (criterion 4), the shared guardrails (criterion 5),
  the shared skill/command body, and an adapter render.
- `init-scope-reconciliation.test.ts` proves the acceptance criteria's "both
  trees" clause through the generator: it runs `init` with claude,opencode and
  asserts the hardened prose landed in `.claude/skills/…/SKILL.md` AND
  `.opencode/skills/…/SKILL.md` for all three skills. Those trees are gitignored
  and generated, so this is the only honest way to prove the clause — hand-editing
  a tree would be reverted on the next regeneration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Issue #100's acceptance criterion 6 asks for a checkable procedure that
demonstrates the reconciliation behavior rather than asserting it. Deterministic
coverage of criteria 1-5 stays in vitest so CI never depends on a live agent
spawn; this binding is the demonstration.

The fixture `.ratchet/evals/fixtures/issue-scope-reconciliation/` is a
self-contained ratchet project holding the worked example as INPUT: issue #80's
text verbatim (its Why names the permission-posture bypass; its fix proposal
hedges the gating requirement as "Consider requiring…"), the phase-one manifest
excerpt that hard-codes `Closes #80` and whose change `done` ends "Fixes #80.",
and the shipped phase-one plan carrying the self-approved `## Out of scope`
bullet — quoted from #100's worked example, ellipsis included, with no invented
variants.

The `llm-judge` binding does NOT restate the reconciliation procedure — that
duplication is what this change eliminates. Its `setup` runs `ratchet init`
inside the fixture working copy so the judge reads the procedure out of a real
generated `SKILL.md`, which therefore cannot drift from what ships. The success
criteria fail closed if the generated skill turns out not to carry the step, so
a stale CLI surfaces as an explicit failure rather than a vacuous pass, and
fail closed unless ALL THREE escapes are flagged as decision points requiring
the user: the unimplemented gate, the unaddressed permission-posture bypass, and
the premature close-claim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spawned judge runs headless in a throwaway working copy, so it needs the
same `.claude/settings.json` allowlist the `eval-self-run` fixture already ships
(Bash/Read/Glob/Grep/LS). Without it the judge cannot read the fixture files it
is asked to reconcile. Force-added because `.claude/` is gitignored at the repo
root — exactly how `eval-self-run/.claude/settings.json` is tracked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… workflows

Required by the `documentation` standard, which makes the documentation task
mandatory and requires `README.md` to be updated alongside the `docs/` Reference
entry whenever a change alters a surface the README describes.

`docs/configuration/generated-artifacts.md` gains a "Scope reconciliation in the
change-authoring workflows" section under the per-tool skills and commands
chapter: an `## Overview` Mermaid flowchart (vertical, high-contrast, every
`classDef` sets `color:`) followed by Reference entries for the
originating-issue reconciliation step, the close-claim rules, the
stop-and-surface guardrail, and the deferral carry-forward trichotomy. The
diagram was verified to parse with mermaid 11.

`README.md` gains a "Scope reconciliation" subsection, and its propose-batch and
apply-batch descriptions now match the hardened prose rather than describing
behavior this change made stale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ation

Full suite: 2969 tests across 207 files, all passing. Enforced coverage gate
green at 95.54% line coverage against the 95% floor (threshold not lowered).
`ratchet validate issue-scope-reconciliation` reports the change valid.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s blocker

Task 7.2 required README.md to match the hardened prose "wherever it describes
the propose / propose-batch / decompose-phase workflows". The batch-workflows
table row for propose-batch was updated, but the "Agent workflows" table rows
for `propose` and `propose-batch` still described the pre-hardening behavior.
Both now name the originating-issue reconciliation and link the new section.

Task 6.3 is blocked by a pre-existing judge-harness defect, and its only
evidence lived in `.ratchet/evals/runs/`, which is gitignored — the exact
"deferral recorded where nothing durable can see it" failure this change exists
to prevent. The run ids, the recorded verdicts, and the verified root cause are
now recorded on the task itself. The task stays unchecked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8.1 is done — #112 tracks #80's remainder with a named owner, and its three
material requirements were each verified against the phase-1 branch before
filing.

8.2 was dropped by explicit decision of the repository owner, who will close
PR #97 themselves rather than have its body amended. The `security-remediation`
standard requires a deferral to carry recorded human sign-off inside the change,
not merely a linked issue, so the decision and its reasoning are recorded on the
task rather than left in session scrollback. No security scope goes untracked:
#112 is linked from #80 independently of #97's body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Applies this change's own reconciliation procedure to the issue that
commissioned it: all seven acceptance criteria enumerated, five fully met,
two partial and surfaced rather than self-approved.

Criterion 6's demonstration cannot be produced here — extractVerdictJson
(judge.ts:211, from PR #40 on main) reads the last balanced top-level array
in agent stdout, so under any stream-json adapter it parses the envelope's
permission_denials instead of the judge's verdict. Both installed adapters
(claude, opencode) set emitsStreamJson; the three that don't are not
installed. Weakening the spec, `eval record`-ing a pass, or quietly checking
the task off were each rejected as the exact patterns this change forbids.

Consequently the PR says "partially addresses #100" rather than claiming
Closes — the honest-close-claim rule applied to itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ratchet 1f3900f Commit Preview URL

Branch Preview URL
Aug 26 2026, 10:59 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant