Skip to content

feat(batch): per-stage model selection and attributed model failures - #95

Merged
joctaTorres merged 10 commits into
mainfrom
batch/model
Jul 9, 2026
Merged

joctaTorres merged 10 commits into
mainfrom
batch/model

Conversation

@joctaTorres

Copy link
Copy Markdown
Owner

Summary

Driven end-to-end by the agent-model-selection ratchet batch (2 phases, 7 changes, both proof-of-work gates green).

Per-stage model selection

  • Every string position of the agent setting — scalar or per-stage map, project config and manifest — widens to an agent[:model] spec, split on the first colon so opencode provider/model ids pass through intact: propose: claude:fable, apply: opencode:zai/glm-5.2, verify: opencode:qwen/qwen-3.7.
  • The engine parses the resolved spec once per transition, resolves the adapter by the agent part (UnknownAgentError still fires before any spawn), and each adapter emits its own model flag (claude/opencode/cursor --model, codex/gemini -m) only when a model is named — a bare name emits no flag (agent's harness-default model) and keeps today's argv byte-for-byte.
  • Malformed specs (claude:, :fable) are rejected at config load naming the offending value; the model part is free-form pass-through (no model registry to rot). Nearest-wins per-stage merge operates on whole spec strings, so agent+model move atomically across scopes.

Attributed model failures

  • A transition that explicitly named a model and died with the argv-rejection signature (non-zero exit, no completion, zero session journal entries) surfaces a failure opening with an attribution hint — stage, agent, exact model string, and supplying scope (project config vs manifest), phrased as "if this model id is invalid…", never a diagnosis — above the verbatim stderr tail. Everything else surfaces byte-for-byte unchanged; no retry, no fallback model.
  • Resolution exposes per-stage supplying scopes beside the merge (merge results unchanged, proven by an agreement invariant).
  • ratchet doctor parses configured agent values through the same spec parser, probes the agent part's binary, and now fails the agent check when a configured agent's binary is missing; it never validates a model id.

Docs & tests

  • README + docs/configuration/config-yaml.md, docs/engine/overview.md, docs/commands/doctor.md document the spec syntax, flag emission, the failure hint, and doctor's spec-aware probing.
  • Full suite green: 203 files / 2780 tests, coverage 95.55% over the enforced 95% gate. Phase proofs: test/batch-engine/agent-model-selection.test.ts (config→argv end to end, real adapters + fake spawner) and test/batch-engine/model-failure-attribution.test.ts (attributed surface via runChangeStep).

🤖 Generated with Claude Code

https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b

joctaTorres and others added 2 commits July 5, 2026 12:58
Let a batch drive different coding agents per lifecycle stage and open
pull requests in configurable groupings, across three phases:

- per-stage-agent-routing: the `agent` setting accepts a scalar or a
  partial `{propose, apply, verify}` stage-map at project-config and
  manifest scope; resolution merges nearest-wins per stage (mapped ->
  scalar -> default) and the engine resolves the adapter per transition.
  Invalid stage keys, non-string values, and unknown agent names are
  rejected before any spawn.

- whole-batch-pr-opening: `prGrouping: whole-batch` spawns a dedicated
  instruction-following PR agent at batch completion that commits the
  accumulated work (repo git-log style, semantic by default) and opens
  one PR via the self-contained pr-open workflow. Adds `pr` as a fourth
  routable stage. No forge-specific code in ratchet; idempotent on rerun
  via per-group run-state; failures surface as reported step failures;
  doctor warns when grouping is active but no git remote is configured.

- stacked-pr-grouping-modes: `per-phase` / `per-change` open one stacked
  PR per unit (group 0 -> batch base, group N -> group N-1's branch).
  The engine spawns one PR agent per detected boundary with the resolved
  stacked base injected as instruction data, keyed per group so a resumed
  loop never double-opens; `batch apply` surfaces and routes the
  per-boundary PR steps.

Docs (README + docs/) document the settings, the stacked-base rule, and
carry vertical Mermaid overviews. Full suite green; line coverage 95.51%.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Let every agent value pick the model it runs with, and make model
mistakes cheap to diagnose, across two phases:

- per-stage-model-selection: every string position of the `agent`
  setting — scalar or per-stage map, project config and manifest —
  widens to an `agent[:model]` spec split on the first colon, so
  opencode provider/model ids pass through intact. The engine parses
  the resolved spec once per transition, resolves the adapter by the
  agent part (UnknownAgentError still fires before any spawn), and
  each adapter emits its own model flag (claude/opencode/cursor
  --model, codex/gemini -m) only when a model is named; a bare name
  emits no flag and keeps today's argv byte-for-byte. Malformed specs
  (empty agent or model part) are rejected at config load naming the
  offending value; the model part is free-form pass-through — no
  model registry to rot. Nearest-wins per-stage merge still operates
  on whole spec strings, so agent+model move atomically across scopes.

- model-failure-attribution: when a transition whose spec explicitly
  named a model dies with the argv-rejection signature (non-zero exit,
  no completion, zero session journal entries), the surfaced failure
  opens with an attribution hint — stage, agent, exact model string,
  and the scope (project config vs manifest) that supplied it, phrased
  as "if this model id is invalid…", never a diagnosis — above the
  verbatim stderr tail. Resolution now exposes per-stage supplying
  scopes beside the merge (results unchanged). Doctor parses configured
  agent values through the same spec parser, probes the agent part's
  binary, and fails the agent check when a configured agent's binary is
  missing; it never validates a model id.

Docs (README + docs/) document the spec syntax, the failure hint, and
doctor's spec-aware probing. Full suite green: 203 files / 2780 tests;
coverage 95.55% over the 95% gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ratchet 2697f4d Commit Preview URL

Branch Preview URL
Jul 09 2026, 01:05 AM

…#96)

* fix(batch): spec honesty at every boundary and attribution visibility

Close all 8 findings of the per-stage model selection review, in two
phases driven by the review-fixes batch:

- spec-honesty-at-every-boundary: parseAgentSpec additionally rejects
  whitespace-bearing agent/model parts and models starting with `-`,
  naming the offending value (findings 7, 8). `batch config set agent`
  validates through the shared AgentSettingSchema and refuses to persist
  what the loader rejects, with inline `{...}` stage maps parsed as real
  YAML maps (finding 3). readProjectConfig validates the batch section
  per key: warnings name `batch.<key>[.<stage>]` and the offending value
  (authToken never echoed), valid siblings are preserved instead of the
  whole section silently reverting to defaults (finding 2). A malformed
  standalone --agent fails before any spawn with an actionable error,
  and the residual skill-locus parse seam rethrows SkillLocusError so
  the engine maps it to a structured failed step (finding 4).

- attribution-visibility: the model-failure attribution hint threads
  into blocker/message in the mapper's single attribution branch, so
  the non-JSON blocked line, parked reason, journal entry, and the
  standalone renderer all carry it with zero renderer edits, asserted
  at the rendered surfaces (finding 1). agentStageScopes reaches the
  batch-driven pr and decompose spawns (uniformAgentScope for the
  scalar-covered decompose case), giving both the attributed surface
  (finding 5). The hint is gated on a real non-zero exit code and
  suppressed on signal kills, so a timeout SIGKILL under a valid model
  is never told its model id may be invalid (finding 6).

The phase proofs run through the real seams (readProjectConfig,
setProjectBatchSetting, rendered outputs), replacing the hand-rolled
YAML re-parse the review flagged. Full suite green: 203 files / 2879
tests; coverage 95.54% over the 95% gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b

* fix(config): ignore prototype-chain batch key names instead of nuking config

parseBatchPerKey used `key in shape`, which walks the prototype chain, so a
batch key named `constructor`/`toString`/etc. passed the known-key check and
then called `.safeParse` on the inherited prototype value — throwing a
TypeError that unwound to readProjectConfig's catch and reverted the ENTIRE
config to null (regression vs. base's bounded whole-section drop).

Switch to `Object.hasOwn(shape, key)` so prototype-chain names are ignored
like any other unknown key, matching the partial-schema contract. Adds a
regression test proving the whole config no longer collapses to null.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

@joctaTorres joctaTorres left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #95 review — "feat(batch): per-stage model selection and attributed model failures"

Base: 15d4625 · Head: f18740c · 16874+ / 104−

Scope note. Commit f18740c merges already-reviewed PR #96; every finding below was verified against the PR's original range (15d4625..acee4d1) and is attributable to this PR's own commits. Metrics were computed on the 24 in-scope src/**/*.ts files at base, at acee4d1 (original head), and at f18740c (current head, for line anchoring).

Quality snapshot

Metric Before After Δ Status
Cyclomatic (max changed fn) 22 (pickNextStep) 34 +12 🔴
CC density (max new fn) — 0.36 (resolveAgentSetting) new 🔴
ABC magnitude (max changed fn) 36.1 55.7 (pickNextStep, ⟨A=13,B=15,C=52⟩) +19.6 🟡
Coverage — line (changed files, agg) 93.9% 93.9% 0.0pp 🟢
Coverage — branch (changed files, agg) 90.3% 92.1% +1.8pp 🟢
Maintainability Index skipped (no TS MI tool installed; approximate AST-derived deltas: instructions.ts −5.4, outcome.ts −5.3, doctor/checks/agents.ts −8.6 — size-driven, informational only) 🟡
Halstead Difficulty skipped per-function (file-level approximation only; apply.ts D 125→146, informational) —
LCOM4 (max touched class) 1 1 (RatchetBatchEngine, manual estimate — new runPrStep/runDecompositionStep share the spawn/outcome seams) 0 🟢
New coupling types — none Content/Common; new edges are Data (typed PrStepContext/DecompositionStepContext) 🟢
DSQI / IPL / WEAVE DSQI manual rubric: layer direction clean, agent-setting.ts extraction deliberately avoids the project-config ↔ batch/config cycle · IPL skipped (not perf-critical) · WEAVE skipped (no integration) 🟢

Hard gates: 🔴 must-fix advisory — CC ceiling (3 functions), CC density (2 new functions). Per this review's playbook, the opinionated/style layer was not run while hard gates are red; findings below are metrics-layer only.

Coverage caveat: measured with test/cli-e2e/** excluded (requires the Python SWE-ReX sidecar). Aggregate holds, but src/commands/batch/apply.ts alone drops 95.8% → 91.2% line — the uncovered growth is the new PR-step routing exercised only by the excluded cli-e2e suites (see soft signals).

Must-fix advisories

  • 🔴 CC ceiling — pickNextStep CC 22 → 34 (AST-derived; lizard cannot parse this signature), above the 15 gate. Inline at src/commands/batch/apply.ts:356.
  • 🔴 CC ceiling — batchApplyCommand CC 15 → 19 (lizard), crosses the 15 gate. Inline at src/commands/batch/apply.ts:149.
  • 🔴 CC ceiling — mapSessionToOutcome CC 13 → 16 in this PR's range (17 at head after the #96 merge), crosses the 15 gate. Inline at src/core/batch/engine/outcome.ts:182.
  • 🔴 CC density — resolveAgentSetting CC 12 / NLOC 33 = 0.36, above the 0.30 gate. Inline at src/core/batch/config.ts:484.
  • 🔴 CC density — resolveAgentStageScopes CC 11 / NLOC 32 = 0.34, above the 0.30 gate. Inline at src/core/batch/config.ts:548.

No quality-gate: <metric> justified overrides were found at any of these function heads.

Reviewer attention (soft signals)

  • apply.ts per-file line coverage −4.7pp (95.8% → 91.2%, cli-e2e excluded). runPr/runStackedPr/runProofAtBoundary already take injectable seams (deps.branches, groupBranch, deps.proof) — a unit test driving batchApplyCommand with a fake engine through the pr target would restore vitest-visible coverage instead of relying solely on the sidecar-gated e2e suites.
  • ABC 55.7 on pickNextStep ⟨A=13, B=15, C=52⟩ — heavily C-dominated (pure branching), 92% of the way to the 60 gate; the CC advisory's split addresses this too.
  • CoN-Algorithm (same-module): resolveAgentStageScopes deliberately replays resolveAgentSetting's scalar-resets/map-merges fold; agreement is guarded only by tests. A single generic fold parameterized on what to record (value vs. scope) removes the drift risk and both density breaches at once. Inline at src/core/batch/config.ts:548.
  • CoN-Position: runStackedPr takes 11 positional parameters (runPr 8, runDecomposition 8; exported selectStackedBases 4). Inline at src/commands/batch/apply.ts:724.
  • resolveBatchSettings CC 10 → 15 — sits exactly at the gate; the next stage key or setting added tips it over. Inline at src/core/batch/config.ts:381.

Summary

Strong PR on the evidence side: every new module (agent-setting.ts, boundary.ts, stacked-base.ts, pr-remote.ts, pr-open.ts) lands at 100% line/branch coverage, aggregate branch coverage on changed files rises 1.8pp, coupling stays Data-typed, and layer direction is clean. The structural cost is concentrated in three ever-growing dispatchers — pickNextStep, batchApplyCommand, mapSessionToOutcome — which each absorbed another routing branch in this PR and now sit past the CC gate, plus two condensed new config folds. Address the five red advisories first (mostly mechanical extractions along the already-commented branch seams); the style layer of this review intentionally did not run while those gates are red.

@@ -246,7 +356,18 @@ export type ApplyTarget =
export function pickNextStep(

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC ceiling — pickNextStep CC 22 → 34, above the 15 gate (AST-derived; lizard cannot parse this signature). ABC 55.7 ⟨A=13,B=15,C=52⟩ — C-dominated, 92% of the ABC gate. The five selection branches are already comment-delimited (change scan / decompose boundary / terminal proof / whole-batch PR / stacked-PR tail): extract each into a named selector (selectChangeStep, selectDecomposeStep, selectBoundaryProof, selectCompletionPr, selectStackedPr) and compose them in order — same semantics, each piece independently under the gate.

const batch = resolveBatchName(projectRoot, name);
const manifest = loadBatchManifest(projectRoot, batch);
const { settings } = resolveBatchSettings(projectRoot, manifest);
const { settings, agentStageScopes } = resolveBatchSettings(projectRoot, manifest);

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC ceiling — batchApplyCommand CC 15 → 19 (lizard), crosses the 15 gate. ABC 33.9 → 45.1, B-dominated ⟨A=18,B=35,C=22⟩ — orchestration is accreting per target kind. Two mechanical cuts: (1) move the prContext assembly (L171–180) into a resolvePrContext(journal, settings) helper; (2) replace the target.kind if-chain (L211–253) with a dispatch to per-kind runner functions, mirroring how the runners themselves are already factored.

@@ -111,12 +180,40 @@ export function mapSessionToOutcome(input: MapOutcomeInput): EngineStepOutcome {
// Non-zero exit WITHOUT a completion report is a failed step. State stays
// consistent: the CLI parks it (failed -> blocked) and the batch is resumable.
if (nonZero && !completion) {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC ceiling — mapSessionToOutcome CC 13 → 16 in this PR's range (17 at head), crosses the 15 gate. The attribution-enriched failure added here is a self-contained sub-decision: extract this nonZero && !completion branch into a failedOutcome(input, spawn, stderrTail) helper that owns the attribution predicate — the parent maps session→state, the helper decides how a failure is worded.

Comment thread src/core/batch/config.ts
* the nearest scope that contributed any `agent` value (`undefined` when none did,
* so the caller keeps the `default` source).
*/
export function resolveAgentSetting(

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC density — resolveAgentSetting CC 12 / NLOC 33 = 0.36, above the 0.30 gate. Condensed branching: the scalar-reset / map-merge fold plus three materialization returns in 33 lines. See the companion comment on resolveAgentStageScopes — one shared fold fixes both density breaches.

Comment thread src/core/batch/config.ts
* or its results; the agreement invariant is asserted in tests over the same
* layers.
*/
export function resolveAgentStageScopes(

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC density — resolveAgentStageScopes CC 11 / NLOC 32 = 0.34, above the 0.30 gate — plus same-module CoN-Algorithm: this function deliberately replays resolveAgentSetting's fold (the docstring says agreement is asserted only in tests). A single generic foldAgentLayers(layers, record) parameterized on what to accumulate (spec string vs. supplying scope) eliminates the duplicated algorithm and brings both functions under the density gate at once.

Comment thread src/core/batch/config.ts
strategy: 'default',
proofOfWork: 'default',
locus: 'default',
prGrouping: 'default',

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 resolveBatchSettings CC 10 → 15 (lizard) — sits exactly at the gate; the next setting or stage key tips it over. suggestion: hoist the per-setting nearest-wins scans into a small resolveScalarSetting(layers, key) helper before this function grows again.

* Branch NAMING is delivered as data (`batchBaseBranch` + the `groupBranch`
* resolver); only git identities are involved and no forge CLI is invoked.
*/
async function runStackedPr(

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 CoN-Position — runStackedPr takes 11 positional parameters (runPr 8, runDecomposition 8; exported selectStackedBases 4 — callers must remember order). suggestion: pass one StackedPrRun params object (or share a StepRunEnv {projectRoot, batch, engine, settings, agentStageScopes, options} across the three runners); weakens the connascence to name.

joctaTorres and others added 4 commits July 6, 2026 21:02
…oOutcome

`mapSessionToOutcome` measured CC 16 (over the ≤15 gate), driven largely by the
attribution-enriched non-zero-exit failure branch: the outer `if (nonZero &&
!completion)` plus the nested three-condition guard `input.modelAttribution &&
sessionEntries.length === 0 && spawn.signal === null` and its stderr-tail
ternary/`||` fallbacks.

Extract that branch — the attribution-enriched failure return and its sibling
plain-failure return — into a named `buildNonZeroExitFailure(input)` helper. The
mapper now calls it in one line, dropping ~3 decision points (the nested `if`
and its two `&&`) out of `mapSessionToOutcome`, which brings it under the gate
(≈CC 13); the helper itself is a small two-return function.

Behavior-preserving: every returned field (`state`, `change`, `transition`,
`detail`, `blocker`, `journalRefs`, `message`) is byte-for-byte identical in
both the attributed and plain paths; the exported `mapSessionToOutcome` name,
signature, and return shape are unchanged. Affected suites green
(test/batch-engine, test/core/batch: 827 passed); lint/tsc/build clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… resolution

`resolveAgentSetting` (CC-density ≈0.36) and `resolveAgentStageScopes`
(≈0.34) both exceeded the ≤0.30 density gate, and each open-coded the SAME
scalar-resets / map-merges-per-stage fold over the same low→high `agent` layers —
one recording the resolved spec string, the other the supplying scope.

Extract that fold into one shared generic `foldAgentLayers<T>(layers, select)`
parameterized over what each layer records at a string position: the merge passes
`(_scope, value) => value` (record the spec), the scope sibling passes
`(scope) => scope` (record the supplying scope). Each function keeps only its own
final materialization — the merge's `stages[stage] ?? base` full-map (or partial
map / bare scalar), and the scope sibling's per-stage attribution map. This drops
the per-stage loop and scalar/map branching out of both functions, cutting each
one's decision density below the gate, and removes the duplicated fold.

Behavior-preserving: the fold performs byte-identical steps to the two inlined
folds (same `source`/`base`/`stages`/`mapContributed` accumulation, same
nearest-wins reset), and each function's returns are unchanged. Exported names,
signatures, and return shapes of both functions are identical. The agreement
invariant tests over the same layers stay green (test/core/batch/
agent-scope-resolution: 22 passed; full test/core/batch: 295 passed);
lint/tsc/build clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`pickNextStep` measured CC 34 (well over the ≤15 gate): one function held all
five sequential selection branches inline (runnable-change/boundary-proof →
decompose → terminal proof → whole-batch PR tail → stacked PR tail), each a
multi-condition block with its own loops and short-circuit returns.

Split each branch into a small named selector helper — `selectChangeOrBoundary
Proof`, `selectDecomposeStep`, `selectTerminalProofStep`, `selectWholeBatchPr
Step`, `selectStackedPrStep` — each returning `ApplyTarget | undefined`.
`pickNextStep` now just calls them in order and returns the first non-undefined
via `??`, so its own CC collapses to ~1 while each helper carries only its
branch's complexity (each well under the gate). A shared internal `PickPrContext`
type backs the two PR selectors; the exported `pickNextStep` signature (including
its inline `prContext` literal and doc) is unchanged.

Behavior-preserving: the selectors run in the exact previous order with identical
short-circuit semantics — the first branch to yield a target wins, and every
branch's guard conditions, phase/boundary lookups, and returned `ApplyTarget`
shapes are byte-for-byte the same as the prior inline body. The two PR selectors
keep the positive-condition block form so TypeScript's `prContext` narrowing is
identical. Affected suites green (test/commands, test/batch-engine: 764 passed);
lint/tsc/build clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… batchApplyCommand

`batchApplyCommand` measured CC 19 (over the ≤15 gate): after selecting a target
it inlined the PR-context assembly, the no-runnable-step rendering (a nested
ternary + json/text branch), and a four-way `if (target.kind === …)` dispatch
chain (decompose / proof-of-work / pr-with-boundary-sub-branch / change), each
with its own multi-argument engine call.

Extract those into named helpers:
  - `buildPrContext(journal, grouping)` — the PR gating-inputs object.
  - `renderNoTarget(status, phases, proofByPhase, options)` — the no-step output.
  - `dispatchNonChangeTarget(target, ctx, status)` — a `switch` over the
    non-`change` kinds routing each to its engine entry point, with the `pr`
    branch's boundary sub-routing in its own `runPrTarget`.
  - `runChangeTarget(target, ctx)` — the change-step body (park precheck, context
    build, runStep, persist, render).
A shared `ApplyDispatchContext` bundle threads the common inputs. The command now
resolves state, picks the target, and branches once (`kind !== 'change'`), so its
CC collapses to ~4.

Behavior-preserving: control flow is identical — same order of resolution, the
same single `pickNextStep` call, the same early returns, and each kind reaches
the same engine call with the same arguments as the prior inline chain (verified
by the `Exclude`/`Extract` narrowing matching the old `if` order). The exported
`batchApplyCommand` name and signature are unchanged. Affected suites green
(test/commands, test/core/batch, test/batch-engine: 1059 passed); lint/tsc/build
clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@joctaTorres joctaTorres left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #95 re-review — "feat(batch): per-stage model selection and attributed model failures"

Base: 15d4625 · Head: eb9de88 · 17154+ / 180−

Scope note. Follow-up to the review at f18740c, covering the four behavior-preserving refactor commits added since (1588e95, c2b209b, ae71bb1, eb9de88) that answer that review's five must-fix advisories. Merged commit f18740c (PR #96, reviewed separately) is not re-reviewed. Metrics are AST-derived with the same tooling and conventions as the prior review (lizard's TS tokenizer overcounts ?./?? and cannot parse pickNextStep's signature, so lizard is corroboration only). As in the prior review, the CC-density gate is evaluated on functions in the CC > 10 band — density distinguishes whether a CC-10+ function is genuinely condensed; applied below that band it would mechanically flag every small guard-style helper, including pre-existing ones the prior review deliberately did not flag.

Resolution of the prior review's must-fix advisories

Prior advisory Was Now Status
pickNextStep CC 34 (gate 15) 34 5 — split into 5 ordered selectors chained with ?? ✅ resolved
batchApplyCommand CC 19 (gate 15) 17–19 4 — prContext assembly, no-target rendering, and step-kind dispatch extracted ✅ resolved
mapSessionToOutcome CC 16–18 (gate 15) 18 12 — non-zero-exit failure branch extracted (buildNonZeroExitFailure, CC 7) ✅ resolved
resolveAgentSetting CC density 0.36 (CC 12 / NLOC 33) 0.36 CC 6 / NLOC 17 — fold delegated to foldAgentLayers ✅ resolved (out of the CC > 10 gate band; the raw ratio is 0.35 only because numerator and denominator shrank together — the residual 3 guard returns + materialization loop are not condensed branching)
resolveAgentStageScopes CC density 0.34 (CC 11 / NLOC 32) 0.34 CC 6 / NLOC 20 = 0.30, at (not above) the gate ✅ resolved

The prior review's CoN-Algorithm soft signal (the stage-scope resolver replaying the merge fold by hand) is also resolved: both consumers now call one generic foldAgentLayers parameterized on select, and the agreement invariant is asserted in test/core/batch/agent-scope-resolution.test.ts.

Behavior-preservation check on the four refactors

  • outcome.ts (1588e95) — verbatim lift: buildNonZeroExitFailure reproduces both return shapes byte-for-byte from the same destructured input. ✅
  • config.ts (c2b209b) — the fold is a faithful parameterization; the scope consumer destructures and ignores the fold's source field, which its hand-rolled predecessor never computed. ✅
  • pickNextStep (ae71bb1) — the ?? chain preserves the monolith's fall-through semantics exactly (a selector declining for any reason, e.g. !phase, proceeds to the next branch just as the old inline blocks fell through), in the same load-bearing order. ✅
  • batchApplyCommand (eb9de88) — dispatch bodies are identical, arguments match one-for-one, and the Exclude/Extract narrowing plus exhaustive switch slightly strengthen type safety. ✅
  • Backstop: the four commits touch zero test files and the full suite passes unchanged at head — 2804/2804 tests, 196 files (test/cli-e2e/** excluded; sidecar-gated).

Quality snapshot (base 15d4625 → head eb9de88)

Metric Before After Δ Status
Cyclomatic (max changed fn) 22 (pickNextStep) 14 (setProjectBatchSetting, validateSetting) −8 🟢
CC density (max new/changed fn, CC > 10 band) — 0.37 (selectChangeOrBoundaryProof, CC 11 / NLOC 30) new 🔴
ABC magnitude (max changed fn) 36 (pickNextStep) 29 (resolveBatchSettings) −7 🟢
Coverage — line (changed files, agg) 93.9% 94.2% +0.3pp 🟢
Coverage — branch (changed files, agg) 90.3% 91.2% +0.9pp 🟢
Maintainability Index skipped (no TS MI tool; AST approximation, size/doc-driven and informational only: outcome.ts −6.8, doctor/checks/agents.ts −8.6, instructions.ts −5.4 vs base) 🟡
Halstead Difficulty skipped per-function (file-level approximation only) —
LCOM4 (max touched class) 1 1 0 🟢
New coupling types — none Content/Common; the new ApplyDispatchContext/PickPrContext edges are Data (typed shapes) 🟢
DSQI / IPL / WEAVE DSQI: extraction preserved layer direction (selectors and dispatch helpers stay in the command layer, engine seams untouched) · IPL skipped (not perf-critical) · WEAVE skipped (no integration) 🟢

Pre-existing, not gated: readProjectConfig CC 27 → 25 (still above 15, but improved by this range) and hasToolProfileOrDeliveryDrift CC 24 → 24 (function body untouched). Coverage measured with test/cli-e2e/** excluded, same as the prior review.

Hard gates: 🔴 must-fix advisory — CC density (1 function). Per this review's playbook the opinionated/style layer was again not run while a gate is red; everything above is metrics-layer plus verification of the refactor claims.

Must-fix advisories

  • 🔴 CC density — selectChangeOrBoundaryProof CC 11 / NLOC 30 = 0.37, above the 0.30 gate. This is the relocated core of the old pickNextStep body — the split resolved the CC-34 ceiling breach but concentrated the branching of the first (largest) branch into one selector that lands in exactly the band the prior review red-flagged for resolveAgentSetting (CC 12 / 0.36). Two mechanical cuts clear it without gaming: a Set membership test for the three-way runnable-status check (−2 CC) and extracting the predecessor-boundary-proof guard (−3 CC) — which selectDecomposeStep near-duplicates, so the same helper removes a fresh two-site drift risk. Inline at src/commands/batch/apply.ts:505. No quality-gate: override is present at the function head.

Reviewer attention (soft signals)

  • selectDecomposeStep (CC 9 / NLOC 24 = 0.375) and selectStackedPrStep (CC 9 / NLOC 26 = 0.35) sit below the CC > 10 gate band but are the next-densest new selectors; the shared predecessor-proof-guard extraction above brings selectDecomposeStep down as a side effect. Inline at src/commands/batch/apply.ts:563.
  • setProjectBatchSetting CC 12 → 14 and validateSetting CC 13 → 14 — the write/validate pair on the config path each absorbed another key === … branch this PR and now sit one branch from the 15 gate; the next persisted key type tips them over. A key→codec table (parse/serialize per key) absorbs future keys at CC +0. Inline at src/core/batch/config.ts:1001.
  • src/commands/batch/apply.ts line coverage 95.8% → 92.3% (−3.5pp vs base, improved from 91.2% at the prior head). Unchanged finding: the uncovered growth is the PR-step routing exercised only by the sidecar-gated cli-e2e suites; a vitest-visible unit test driving batchApplyCommand through a pr target with a fake engine (the deps.branches/deps.proof seams already exist) would close it.
  • CoN-Position, still standing from the prior review: runStackedPr takes 11 positional parameters (src/commands/batch/apply.ts:893). The new ApplyDispatchContext is the natural carrier — threading it (plus a small per-target shape) through runPr/runStackedPr/runDecomposition would resolve the prior inline comment.
  • resolveBatchSettings CC 13 (AST; lizard counts 15 due to ??) — unchanged since the prior review's note: at/near the gate depending on counting convention; the next stage key or setting tips it over.

Summary

The four refactor commits do what they claim: all five previously red advisories are resolved (CC 34 → 5, 19 → 4, 18 → 12; both config-fold density breaches gone via one shared, agreement-tested fold), verified equivalent by line-level comparison and an unchanged 2804-test pass, with aggregate coverage on changed files up on both axes vs base. One residual red remains: the extraction relocated the old pickNextStep's densest branch into selectChangeOrBoundaryProof (CC 11, density 0.37), which needs the two small cuts described inline — one of which also removes a new duplicated boundary-proof guard. Address that first; the remaining signals are the same coverage and parameter-arity notes carried over from the prior review.

Comment on lines 505 to +541
@@ -278,45 +537,125 @@ export function pickNextStep(
}
}
}
return undefined;
}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CC density 0.37 (CC 11 / NLOC 30), above the 0.30 gate. The pickNextStep split resolved the CC-34 ceiling breach, but this selector inherited the monolith's densest branch and lands in the same band the prior review flagged for resolveAgentSetting (CC 12 / 0.36). Two mechanical cuts clear it without metric-gaming:

  1. Replace the three-way status check (L516–L523) with RUNNABLE_STATUSES.has(change.status) over a module-level ReadonlySet — −2 CC and the awaiting-verify comment moves to the constant.
  2. Extract the predecessor-boundary-proof guard (L527–L533) into a shared helper — −3 CC here, and it also deduplicates selectDecomposeStep's variant (L563–L575).

Result ≈ CC 6 / density ≈ 0.27. Alternatively, a quality-gate: cc-density justified — … comment at the function head opts out explicitly.

Comment on lines +563 to 575
const phaseIndex = status.phases.findIndex((p) => p.name === phase.name);
if (phaseIndex > 0) {
const predStatus = status.phases[phaseIndex - 1];
const predecessor = manifestPhases.find((p) => p.name === predStatus.name);
if (
predecessor &&
predStatus.status === 'done' &&
predecessor.proofOfWork &&
!recordedProofPhases.has(predecessor.name)
) {
return { kind: 'proof-of-work', phase: predecessor };
}
}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

§3.7 / CoN-Algorithm (soft) — this predecessor-boundary-proof guard is a near-duplicate of selectChangeOrBoundaryProof L527–L533 (this variant adds the predStatus.status === 'done' && predecessor.proofOfWork conditions). A single shared pendingBoundaryProof(...) helper keeps the two boundary-gate rules from drifting apart and drops this selector's density (CC 9 / NLOC 24 = 0.375) below the band as a side effect. Grounded in: two-site connascence of algorithm introduced by the split.

Comment thread src/core/batch/config.ts Outdated
batch[key] = Number(value.trim());
} else if (key === 'insecure') {
batch[key] = value.trim() === 'true';
} else if (key === 'agent' && validation.parsedValue !== undefined) {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: CC trend — setProjectBatchSetting CC 12 → 14 with this branch, and its sibling validateSetting is also at 14; each new persisted key type adds another key === … arm to both, and the next one crosses the 15 gate. A key→codec table ({ parse, serialize } per key, shared by validate and persist) absorbs future keys at CC +0. Non-blocking. Grounded in: CC 14, one branch under the 15 gate.

joctaTorres and others added 2 commits July 6, 2026 21:58
…oundary-proof helper

Reduce `selectChangeOrBoundaryProof` CC density from 0.37 (CC 11 / NLOC 30)
to ~0.27 with two behavior-preserving cuts, both byte-identical to before:

- Replace the three-way runnable-status check with a lookup against a new
  module-level `RUNNABLE_STATUSES: ReadonlySet<ChangeStatus>` (ready /
  in-progress / awaiting-verify); the awaiting-verify rationale moves onto the
  constant.
- Extract the predecessor-boundary-proof guard into a shared
  `pendingBoundaryProof(...)` used by BOTH selectors. The two call sites keep
  their DISTINCT gates via `requireDoneWithProof`: the change branch passes
  `false` (predecessor is done by the gate invariant); the decompose branch
  passes `true` (adds the `status === 'done' && proofOfWork` conditions). As a
  side effect `selectDecomposeStep` also drops below the density band.

No exported signature changes; `pickNextStep`'s public shape is unchanged.
Verified via pickNextStep / boundary-proof / decompose unit tests (532
batch-engine tests green) and the full affected suites (1059 green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the per-key `if (key === …)` chains in `validateSetting` (CC 14) and
`setProjectBatchSetting` (CC 14) into a single `SETTING_CODECS` table with
optional `validate`/`serialize` per key, read by BOTH functions. A future
persisted key now lands at CC +0 (one table entry) instead of a branch in each.

Behavior is byte-identical:
- Validation order preserved (unknown-key → ALLOWED_VALUES enum → per-key
  shape/agent-schema) and every error-message string is verbatim (image empty,
  host/authToken empty, port/agentTimeoutMs integer, agent via
  validateAgentSetting carrying parsedValue).
- Persist coercion preserved verbatim: port/agentTimeoutMs → Number(value.trim()),
  insecure → value.trim() === 'true', agent (parsedValue !== undefined) →
  parsedValue, everything else → value.

No exported signature changes. Config + command suites and the full 2880-test
suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@joctaTorres joctaTorres left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #95 re-review — "feat(batch): per-stage model selection and attributed model failures"

Base: 15d4625 · Head: 0c948f7 · 17286+ / 235−

Scope note. Third review, covering the two behavior-preserving refactor commits added since the review at eb9de88 (9756ca4, 0c948f7), which answer that review's one must-fix advisory and two of its signals. Merged commit f18740c (PR #96, reviewed separately) is not re-reviewed. Same tooling and conventions as both prior reviews: AST-derived CC with lizard as corroboration (lizard overcounts ?./??), the CC-density gate evaluated on functions in the CC > 10 band, coverage measured with test/cli-e2e/** excluded.

Resolution of the prior review's advisories

Prior advisory Was Now Status
🔴 CC density — selectChangeOrBoundaryProof 0.37 (CC 11 / NLOC 30) 0.37 CC 8 / NLOC 28 = 0.29, below the 0.30 gate — and below the CC > 10 gate band entirely ✅ resolved
🟡 §3.7 / CoN-Algorithm — selectDecomposeStep near-duplicates the boundary-proof guard (CC 9 / density 0.375) 0.375 CC 6 / NLOC 21 = 0.29 — both call sites now share one pendingBoundaryProof helper; the two-site drift risk is gone ✅ resolved
nit: CC trend — setProjectBatchSetting CC 14, validateSetting CC 14, one branch from the 15 gate 14 / 14 8 / 5 — both routed through one per-key SETTING_CODECS table (src/core/batch/config.ts:770); the next persisted key is one table entry, CC +0 in both functions ✅ resolved

Both commits implement exactly the cuts the prior review described (the Set membership test for the three-way runnable-status check, the shared predecessor-proof guard, the key→codec table), with no scope creep.

Behavior-preservation check on the two refactors

  • 9756ca4 (pendingBoundaryProof) — the two call sites' distinct gates are preserved via requireDoneWithProof: the change branch passes false and returns the predecessor whenever it exists with an unrecorded proof — no done/proofOfWork requirement, exactly as the old inline guard; the decompose branch passes true and still additionally requires predStatus.status === 'done' && predecessor.proofOfWork. The only delta is predicate order (the recorded-proof check now precedes the done/proof check in the decompose path) — a pure, side-effect-free conjunction, so the result is identical. RUNNABLE_STATUSES is exactly the old three statuses (ready, in-progress, awaiting-verify), comment included. ✅
  • 0c948f7 (SETTING_CODECS) — every validation error string is byte-identical: image keeps its bespoke hardcoded message; host/authToken/port/agentTimeoutMs render identical strings through the ${key} template; agent still short-circuits through the same validateAgentSetting. The codec lookup sits at the same position in validateSetting (after the key/enum gates, before the generic success), and per key at most one codec applies, so ordering among the old if arms is immaterial. Persisted YAML types are identical: Number(value.trim()) for port/agentTimeoutMs, value.trim() === 'true' for insecure, parsedValue !== undefined ? parsedValue : value for agent, and the raw string for serialize-less keys (image/host/authToken define validate only, so they fall through to the verbatim string, as before). ✅
  • Backstop: the two commits touch zero test files and the full suite passes unchanged at head — 2804/2804 tests, 900 suites (test/cli-e2e/** excluded; sidecar-gated).

Quality snapshot (base 15d4625 → head 0c948f7)

Metric Before After Δ Status
Cyclomatic (max new/changed fn) 22 (pickNextStep) 13 (resolveBatchSettings, AST; lizard 15) −9 🟢
CC density (max new/changed fn, CC > 10 band) 0.37 (selectChangeOrBoundaryProof) 0.21 (ensureCommandInSpawnLocus) −0.16 🟢
ABC magnitude (max changed fn) 36 (pickNextStep) 29 (resolveBatchSettings, untouched by these commits) −7 🟢
Coverage — line (changed files, agg) 93.9% 94.2% (3210/3406) +0.3pp 🟢
Coverage — branch (changed files, agg) 90.3% 91.1% (1016/1115) +0.8pp 🟢
Maintainability Index skipped (no TS MI tool; prior reviews' AST approximation was informational only) —
Halstead Difficulty skipped per-function (file-level approximation only) —
LCOM4 (max touched class) 1 1 (no class touched; SETTING_CODECS is a module-level table) 0 🟢
New coupling types — none Content/Common; one new intra-module boolean mode flag (requireDoneWithProof) — control-flag style, documented, nit below 🟢
DSQI / IPL / WEAVE DSQI: the codec table strengthens single-home discipline (validate and persist read one definition per key) · IPL skipped (not perf-critical) · WEAVE skipped (no integration) 🟢

Hard gates: all passing ✅ — first fully green snapshot in this PR's review cycle.

Skipped metrics: MI (no TS tool), per-function Halstead (approximation only), IPL (not perf-critical), WEAVE (no integration).

Pre-existing, not gated (unchanged from prior reviews): readProjectConfig CC 25 (improved from 27 by this PR) and hasToolProfileOrDeliveryDrift CC 24 (body untouched).

Opinionated findings (§3 — first run; both prior reviews stopped at red gates)

Applied §3.1–§3.9 to the PR's original in-scope files (portions belonging to f18740c/PR #96 skipped as separately reviewed). The structural picture is clean:

  • §3.1 / §3.2 — no findings. engine/boundary.ts and engine/stacked-base.ts are pure, single-home policy modules with typed surfaces and resolver-injected seams (no git/forge literals); engine/index.ts re-exports only. The selector split in apply.ts and the codec table in config.ts keep command-layer logic in the command layer and config-shape knowledge in the config module.
  • §3.8 — satisfied for this range. Behavior-preserving refactors with the untouched 2804-test suite as the regression harness; the earlier feature commits shipped their own test files.

src/commands/batch/apply.ts

  • nit: §3.3-adjacent control flag — pendingBoundaryProof's fourth positional requireDoneWithProof: boolean reads as a bare false/true at both call sites. Non-blocking (the docstring specifies both modes precisely). Grounded in: Control-coupling category (intra-module, weakest form). Inline at src/commands/batch/apply.ts:532.

Reviewer attention (soft signals — carried, still standing)

  • CoN-Position: runStackedPr still takes 11 positional parameters (src/commands/batch/apply.ts:928); the prior review's inline comment stands — ApplyDispatchContext plus a small per-target shape is the natural carrier.
  • src/commands/batch/apply.ts line coverage 92.2% — the uncovered lines remain the PR-step routing exercised only by the sidecar-gated cli-e2e suites; a vitest-visible unit test driving batchApplyCommand through a pr target with a fake engine (the deps.branches/deps.proof seams exist) would close it.
  • resolveBatchSettings CC 13 (AST; lizard 15) — at/near the gate depending on counting convention; the next stage key or setting tips it over.

Summary

The two commits close out this PR's review cycle: the last red gate (selectChangeOrBoundaryProof density 0.37) is cleared to 0.29 by exactly the two cuts the prior review asked for, and the codec table retires the setProjectBatchSetting/validateSetting CC-trend nit with a design that also removes a validate/persist divergence risk. Line-level comparison confirms both refactors are behavior-preserving — distinct boundary-proof gates intact, every error string and persisted YAML type identical — with an unchanged, fully green 2804-test suite as backstop, and aggregate coverage on changed files up on both axes vs base. Every hard gate is green; what remains is the small carried list above (parameter arity on runStackedPr, vitest-visible PR-step coverage, resolveBatchSettings near the gate) plus one non-blocking naming nit on the new helper's boolean flag.

predStatus: BatchStatusInfo['phases'][number] | undefined,
manifestPhases: Phase[],
recordedProofPhases: ReadonlySet<string>,
requireDoneWithProof: boolean

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: §3.3-adjacent control flag — requireDoneWithProof: boolean reads as a bare false/true at the two call sites (L567, L606), so the reader must return here to learn which gate each branch gets. Two named one-line wrappers (e.g. pendingProofBeforeChange / pendingProofBeforeDecompose) — or named constants at the call sites — would carry the intent without re-widening either function. Non-blocking: the docstring specifies both modes precisely, and this review verified the two gates match the pre-refactor call sites exactly. Grounded in: Control coupling (intra-module, weakest form; no gate).

Make `decompose` a first-class routable agent stage so its agent+model is
selectable from `.ratchet/config.yaml` (`batch.agent.decompose`) and the batch
manifest (`settings.agent.decompose`), merged nearest-wins per stage — exactly
as `pr` already is. `AGENT_STAGE_KEYS` gains `decompose`; the decomposition
spawn threads the `decompose` stage into `ensureCommandInSpawnLocus` and
`buildSpawnRequest`, and attribution reads `agentStageScopes.decompose`. The
`uniformAgentScope` workaround (helper + engine call + tests) is deleted — it
existed only because decompose was not a real stage. `Transition`/`StepKind`
are unchanged; unmapped/unset decompose still falls to the default agent with
no model flag (byte-for-byte prior behavior).

Rename the batch PR skill `pr-open` → `open-pr` via the single-source
`PR_OPEN_COMMAND_ID` value, the workflow module, and the `CORE_WORKFLOWS`/
`ALL_WORKFLOWS` profile lists; the forge-agnostic body and the `pr` stage key
are unchanged. Docs (README, config-yaml, agent-runtime, generated-artifacts,
batch, overview) updated for both; full suite + 95% coverage gate green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant