Repository navigation
feat(batch): per-stage model selection and attributed model failures - #95
Conversation
Let a batch drive different coding agents per lifecycle stage and open
pull requests in configurable groupings, across three phases:
- per-stage-agent-routing: the `agent` setting accepts a scalar or a
partial `{propose, apply, verify}` stage-map at project-config and
manifest scope; resolution merges nearest-wins per stage (mapped ->
scalar -> default) and the engine resolves the adapter per transition.
Invalid stage keys, non-string values, and unknown agent names are
rejected before any spawn.
- whole-batch-pr-opening: `prGrouping: whole-batch` spawns a dedicated
instruction-following PR agent at batch completion that commits the
accumulated work (repo git-log style, semantic by default) and opens
one PR via the self-contained pr-open workflow. Adds `pr` as a fourth
routable stage. No forge-specific code in ratchet; idempotent on rerun
via per-group run-state; failures surface as reported step failures;
doctor warns when grouping is active but no git remote is configured.
- stacked-pr-grouping-modes: `per-phase` / `per-change` open one stacked
PR per unit (group 0 -> batch base, group N -> group N-1's branch).
The engine spawns one PR agent per detected boundary with the resolved
stacked base injected as instruction data, keyed per group so a resumed
loop never double-opens; `batch apply` surfaces and routes the
per-boundary PR steps.
Docs (README + docs/) document the settings, the stacked-base rule, and
carry vertical Mermaid overviews. Full suite green; line coverage 95.51%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Let every agent value pick the model it runs with, and make model mistakes cheap to diagnose, across two phases: - per-stage-model-selection: every string position of the `agent` setting — scalar or per-stage map, project config and manifest — widens to an `agent[:model]` spec split on the first colon, so opencode provider/model ids pass through intact. The engine parses the resolved spec once per transition, resolves the adapter by the agent part (UnknownAgentError still fires before any spawn), and each adapter emits its own model flag (claude/opencode/cursor --model, codex/gemini -m) only when a model is named; a bare name emits no flag and keeps today's argv byte-for-byte. Malformed specs (empty agent or model part) are rejected at config load naming the offending value; the model part is free-form pass-through — no model registry to rot. Nearest-wins per-stage merge still operates on whole spec strings, so agent+model move atomically across scopes. - model-failure-attribution: when a transition whose spec explicitly named a model dies with the argv-rejection signature (non-zero exit, no completion, zero session journal entries), the surfaced failure opens with an attribution hint — stage, agent, exact model string, and the scope (project config vs manifest) that supplied it, phrased as "if this model id is invalid…", never a diagnosis — above the verbatim stderr tail. Resolution now exposes per-stage supplying scopes beside the merge (results unchanged). Doctor parses configured agent values through the same spec parser, probes the agent part's binary, and fails the agent check when a configured agent's binary is missing; it never validates a model id. Docs (README + docs/) document the spec syntax, the failure hint, and doctor's spec-aware probing. Full suite green: 203 files / 2780 tests; coverage 95.55% over the 95% gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ratchet | 2697f4d | Commit Preview URL Branch Preview URL |
Jul 09 2026, 01:05 AM |
…#96) * fix(batch): spec honesty at every boundary and attribution visibility Close all 8 findings of the per-stage model selection review, in two phases driven by the review-fixes batch: - spec-honesty-at-every-boundary: parseAgentSpec additionally rejects whitespace-bearing agent/model parts and models starting with `-`, naming the offending value (findings 7, 8). `batch config set agent` validates through the shared AgentSettingSchema and refuses to persist what the loader rejects, with inline `{...}` stage maps parsed as real YAML maps (finding 3). readProjectConfig validates the batch section per key: warnings name `batch.<key>[.<stage>]` and the offending value (authToken never echoed), valid siblings are preserved instead of the whole section silently reverting to defaults (finding 2). A malformed standalone --agent fails before any spawn with an actionable error, and the residual skill-locus parse seam rethrows SkillLocusError so the engine maps it to a structured failed step (finding 4). - attribution-visibility: the model-failure attribution hint threads into blocker/message in the mapper's single attribution branch, so the non-JSON blocked line, parked reason, journal entry, and the standalone renderer all carry it with zero renderer edits, asserted at the rendered surfaces (finding 1). agentStageScopes reaches the batch-driven pr and decompose spawns (uniformAgentScope for the scalar-covered decompose case), giving both the attributed surface (finding 5). The hint is gated on a real non-zero exit code and suppressed on signal kills, so a timeout SIGKILL under a valid model is never told its model id may be invalid (finding 6). The phase proofs run through the real seams (readProjectConfig, setProjectBatchSetting, rendered outputs), replacing the hand-rolled YAML re-parse the review flagged. Full suite green: 203 files / 2879 tests; coverage 95.54% over the 95% gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b * fix(config): ignore prototype-chain batch key names instead of nuking config parseBatchPerKey used `key in shape`, which walks the prototype chain, so a batch key named `constructor`/`toString`/etc. passed the known-key check and then called `.safeParse` on the inherited prototype value — throwing a TypeError that unwound to readProjectConfig's catch and reverted the ENTIRE config to null (regression vs. base's bounded whole-section drop). Switch to `Object.hasOwn(shape, key)` so prototype-chain names are ignored like any other unknown key, matching the partial-schema contract. Adds a regression test proving the whole config no longer collapses to null. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
joctaTorres
left a comment
There was a problem hiding this comment.
PR #95 review — "feat(batch): per-stage model selection and attributed model failures"
Base: 15d4625 · Head: f18740c · 16874+ / 104−
Scope note. Commit f18740c merges already-reviewed PR #96; every finding below was verified against the PR's original range (15d4625..acee4d1) and is attributable to this PR's own commits. Metrics were computed on the 24 in-scope src/**/*.ts files at base, at acee4d1 (original head), and at f18740c (current head, for line anchoring).
Quality snapshot
| Metric | Before | After | Δ | Status |
|---|---|---|---|---|
| Cyclomatic (max changed fn) | 22 (pickNextStep) |
34 | +12 | 🔴 |
| CC density (max new fn) | — | 0.36 (resolveAgentSetting) |
new | 🔴 |
| ABC magnitude (max changed fn) | 36.1 | 55.7 (pickNextStep, ⟨A=13,B=15,C=52⟩) |
+19.6 | 🟡 |
| Coverage — line (changed files, agg) | 93.9% | 93.9% | 0.0pp | 🟢 |
| Coverage — branch (changed files, agg) | 90.3% | 92.1% | +1.8pp | 🟢 |
| Maintainability Index | skipped (no TS MI tool installed; approximate AST-derived deltas: instructions.ts −5.4, outcome.ts −5.3, doctor/checks/agents.ts −8.6 — size-driven, informational only) |
🟡 | ||
| Halstead Difficulty | skipped per-function (file-level approximation only; apply.ts D 125→146, informational) |
— | ||
| LCOM4 (max touched class) | 1 | 1 (RatchetBatchEngine, manual estimate — new runPrStep/runDecompositionStep share the spawn/outcome seams) |
0 | 🟢 |
| New coupling types | — | none Content/Common; new edges are Data (typed PrStepContext/DecompositionStepContext) |
🟢 | |
| DSQI / IPL / WEAVE | DSQI manual rubric: layer direction clean, agent-setting.ts extraction deliberately avoids the project-config ↔ batch/config cycle · IPL skipped (not perf-critical) · WEAVE skipped (no integration) |
🟢 |
Hard gates: 🔴 must-fix advisory — CC ceiling (3 functions), CC density (2 new functions). Per this review's playbook, the opinionated/style layer was not run while hard gates are red; findings below are metrics-layer only.
Coverage caveat: measured with test/cli-e2e/** excluded (requires the Python SWE-ReX sidecar). Aggregate holds, but src/commands/batch/apply.ts alone drops 95.8% → 91.2% line — the uncovered growth is the new PR-step routing exercised only by the excluded cli-e2e suites (see soft signals).
Must-fix advisories
- 🔴 CC ceiling —
pickNextStepCC 22 → 34 (AST-derived; lizard cannot parse this signature), above the 15 gate. Inline atsrc/commands/batch/apply.ts:356. - 🔴 CC ceiling —
batchApplyCommandCC 15 → 19 (lizard), crosses the 15 gate. Inline atsrc/commands/batch/apply.ts:149. - 🔴 CC ceiling —
mapSessionToOutcomeCC 13 → 16 in this PR's range (17 at head after the #96 merge), crosses the 15 gate. Inline atsrc/core/batch/engine/outcome.ts:182. - 🔴 CC density —
resolveAgentSettingCC 12 / NLOC 33 = 0.36, above the 0.30 gate. Inline atsrc/core/batch/config.ts:484. - 🔴 CC density —
resolveAgentStageScopesCC 11 / NLOC 32 = 0.34, above the 0.30 gate. Inline atsrc/core/batch/config.ts:548.
No quality-gate: <metric> justified overrides were found at any of these function heads.
Reviewer attention (soft signals)
apply.tsper-file line coverage −4.7pp (95.8% → 91.2%, cli-e2e excluded).runPr/runStackedPr/runProofAtBoundaryalready take injectable seams (deps.branches,groupBranch,deps.proof) — a unit test drivingbatchApplyCommandwith a fake engine through theprtarget would restore vitest-visible coverage instead of relying solely on the sidecar-gated e2e suites.- ABC 55.7 on
pickNextStep⟨A=13, B=15, C=52⟩ — heavily C-dominated (pure branching), 92% of the way to the 60 gate; the CC advisory's split addresses this too. - CoN-Algorithm (same-module):
resolveAgentStageScopesdeliberately replaysresolveAgentSetting's scalar-resets/map-merges fold; agreement is guarded only by tests. A single generic fold parameterized on what to record (value vs. scope) removes the drift risk and both density breaches at once. Inline atsrc/core/batch/config.ts:548. - CoN-Position:
runStackedPrtakes 11 positional parameters (runPr8,runDecomposition8; exportedselectStackedBases4). Inline atsrc/commands/batch/apply.ts:724. resolveBatchSettingsCC 10 → 15 — sits exactly at the gate; the next stage key or setting added tips it over. Inline atsrc/core/batch/config.ts:381.
Summary
Strong PR on the evidence side: every new module (agent-setting.ts, boundary.ts, stacked-base.ts, pr-remote.ts, pr-open.ts) lands at 100% line/branch coverage, aggregate branch coverage on changed files rises 1.8pp, coupling stays Data-typed, and layer direction is clean. The structural cost is concentrated in three ever-growing dispatchers — pickNextStep, batchApplyCommand, mapSessionToOutcome — which each absorbed another routing branch in this PR and now sit past the CC gate, plus two condensed new config folds. Address the five red advisories first (mostly mechanical extractions along the already-commented branch seams); the style layer of this review intentionally did not run while those gates are red.
| @@ -246,7 +356,18 @@ export type ApplyTarget = | |||
| export function pickNextStep( | |||
There was a problem hiding this comment.
🔴 CC ceiling — pickNextStep CC 22 → 34, above the 15 gate (AST-derived; lizard cannot parse this signature). ABC 55.7 ⟨A=13,B=15,C=52⟩ — C-dominated, 92% of the ABC gate. The five selection branches are already comment-delimited (change scan / decompose boundary / terminal proof / whole-batch PR / stacked-PR tail): extract each into a named selector (selectChangeStep, selectDecomposeStep, selectBoundaryProof, selectCompletionPr, selectStackedPr) and compose them in order — same semantics, each piece independently under the gate.
| const batch = resolveBatchName(projectRoot, name); | ||
| const manifest = loadBatchManifest(projectRoot, batch); | ||
| const { settings } = resolveBatchSettings(projectRoot, manifest); | ||
| const { settings, agentStageScopes } = resolveBatchSettings(projectRoot, manifest); |
There was a problem hiding this comment.
🔴 CC ceiling — batchApplyCommand CC 15 → 19 (lizard), crosses the 15 gate. ABC 33.9 → 45.1, B-dominated ⟨A=18,B=35,C=22⟩ — orchestration is accreting per target kind. Two mechanical cuts: (1) move the prContext assembly (L171–180) into a resolvePrContext(journal, settings) helper; (2) replace the target.kind if-chain (L211–253) with a dispatch to per-kind runner functions, mirroring how the runners themselves are already factored.
| @@ -111,12 +180,40 @@ export function mapSessionToOutcome(input: MapOutcomeInput): EngineStepOutcome { | |||
| // Non-zero exit WITHOUT a completion report is a failed step. State stays | |||
| // consistent: the CLI parks it (failed -> blocked) and the batch is resumable. | |||
| if (nonZero && !completion) { | |||
There was a problem hiding this comment.
🔴 CC ceiling — mapSessionToOutcome CC 13 → 16 in this PR's range (17 at head), crosses the 15 gate. The attribution-enriched failure added here is a self-contained sub-decision: extract this nonZero && !completion branch into a failedOutcome(input, spawn, stderrTail) helper that owns the attribution predicate — the parent maps session→state, the helper decides how a failure is worded.
| * the nearest scope that contributed any `agent` value (`undefined` when none did, | ||
| * so the caller keeps the `default` source). | ||
| */ | ||
| export function resolveAgentSetting( |
There was a problem hiding this comment.
🔴 CC density — resolveAgentSetting CC 12 / NLOC 33 = 0.36, above the 0.30 gate. Condensed branching: the scalar-reset / map-merge fold plus three materialization returns in 33 lines. See the companion comment on resolveAgentStageScopes — one shared fold fixes both density breaches.
| * or its results; the agreement invariant is asserted in tests over the same | ||
| * layers. | ||
| */ | ||
| export function resolveAgentStageScopes( |
There was a problem hiding this comment.
🔴 CC density — resolveAgentStageScopes CC 11 / NLOC 32 = 0.34, above the 0.30 gate — plus same-module CoN-Algorithm: this function deliberately replays resolveAgentSetting's fold (the docstring says agreement is asserted only in tests). A single generic foldAgentLayers(layers, record) parameterized on what to accumulate (spec string vs. supplying scope) eliminates the duplicated algorithm and brings both functions under the density gate at once.
| strategy: 'default', | ||
| proofOfWork: 'default', | ||
| locus: 'default', | ||
| prGrouping: 'default', |
There was a problem hiding this comment.
🟡 resolveBatchSettings CC 10 → 15 (lizard) — sits exactly at the gate; the next setting or stage key tips it over. suggestion: hoist the per-setting nearest-wins scans into a small resolveScalarSetting(layers, key) helper before this function grows again.
| * Branch NAMING is delivered as data (`batchBaseBranch` + the `groupBranch` | ||
| * resolver); only git identities are involved and no forge CLI is invoked. | ||
| */ | ||
| async function runStackedPr( |
There was a problem hiding this comment.
🟡 CoN-Position — runStackedPr takes 11 positional parameters (runPr 8, runDecomposition 8; exported selectStackedBases 4 — callers must remember order). suggestion: pass one StackedPrRun params object (or share a StepRunEnv {projectRoot, batch, engine, settings, agentStageScopes, options} across the three runners); weakens the connascence to name.
…oOutcome `mapSessionToOutcome` measured CC 16 (over the ≤15 gate), driven largely by the attribution-enriched non-zero-exit failure branch: the outer `if (nonZero && !completion)` plus the nested three-condition guard `input.modelAttribution && sessionEntries.length === 0 && spawn.signal === null` and its stderr-tail ternary/`||` fallbacks. Extract that branch — the attribution-enriched failure return and its sibling plain-failure return — into a named `buildNonZeroExitFailure(input)` helper. The mapper now calls it in one line, dropping ~3 decision points (the nested `if` and its two `&&`) out of `mapSessionToOutcome`, which brings it under the gate (≈CC 13); the helper itself is a small two-return function. Behavior-preserving: every returned field (`state`, `change`, `transition`, `detail`, `blocker`, `journalRefs`, `message`) is byte-for-byte identical in both the attributed and plain paths; the exported `mapSessionToOutcome` name, signature, and return shape are unchanged. Affected suites green (test/batch-engine, test/core/batch: 827 passed); lint/tsc/build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… resolution `resolveAgentSetting` (CC-density ≈0.36) and `resolveAgentStageScopes` (≈0.34) both exceeded the ≤0.30 density gate, and each open-coded the SAME scalar-resets / map-merges-per-stage fold over the same low→high `agent` layers — one recording the resolved spec string, the other the supplying scope. Extract that fold into one shared generic `foldAgentLayers<T>(layers, select)` parameterized over what each layer records at a string position: the merge passes `(_scope, value) => value` (record the spec), the scope sibling passes `(scope) => scope` (record the supplying scope). Each function keeps only its own final materialization — the merge's `stages[stage] ?? base` full-map (or partial map / bare scalar), and the scope sibling's per-stage attribution map. This drops the per-stage loop and scalar/map branching out of both functions, cutting each one's decision density below the gate, and removes the duplicated fold. Behavior-preserving: the fold performs byte-identical steps to the two inlined folds (same `source`/`base`/`stages`/`mapContributed` accumulation, same nearest-wins reset), and each function's returns are unchanged. Exported names, signatures, and return shapes of both functions are identical. The agreement invariant tests over the same layers stay green (test/core/batch/ agent-scope-resolution: 22 passed; full test/core/batch: 295 passed); lint/tsc/build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`pickNextStep` measured CC 34 (well over the ≤15 gate): one function held all five sequential selection branches inline (runnable-change/boundary-proof → decompose → terminal proof → whole-batch PR tail → stacked PR tail), each a multi-condition block with its own loops and short-circuit returns. Split each branch into a small named selector helper — `selectChangeOrBoundary Proof`, `selectDecomposeStep`, `selectTerminalProofStep`, `selectWholeBatchPr Step`, `selectStackedPrStep` — each returning `ApplyTarget | undefined`. `pickNextStep` now just calls them in order and returns the first non-undefined via `??`, so its own CC collapses to ~1 while each helper carries only its branch's complexity (each well under the gate). A shared internal `PickPrContext` type backs the two PR selectors; the exported `pickNextStep` signature (including its inline `prContext` literal and doc) is unchanged. Behavior-preserving: the selectors run in the exact previous order with identical short-circuit semantics — the first branch to yield a target wins, and every branch's guard conditions, phase/boundary lookups, and returned `ApplyTarget` shapes are byte-for-byte the same as the prior inline body. The two PR selectors keep the positive-condition block form so TypeScript's `prContext` narrowing is identical. Affected suites green (test/commands, test/batch-engine: 764 passed); lint/tsc/build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… batchApplyCommand
`batchApplyCommand` measured CC 19 (over the ≤15 gate): after selecting a target
it inlined the PR-context assembly, the no-runnable-step rendering (a nested
ternary + json/text branch), and a four-way `if (target.kind === …)` dispatch
chain (decompose / proof-of-work / pr-with-boundary-sub-branch / change), each
with its own multi-argument engine call.
Extract those into named helpers:
- `buildPrContext(journal, grouping)` — the PR gating-inputs object.
- `renderNoTarget(status, phases, proofByPhase, options)` — the no-step output.
- `dispatchNonChangeTarget(target, ctx, status)` — a `switch` over the
non-`change` kinds routing each to its engine entry point, with the `pr`
branch's boundary sub-routing in its own `runPrTarget`.
- `runChangeTarget(target, ctx)` — the change-step body (park precheck, context
build, runStep, persist, render).
A shared `ApplyDispatchContext` bundle threads the common inputs. The command now
resolves state, picks the target, and branches once (`kind !== 'change'`), so its
CC collapses to ~4.
Behavior-preserving: control flow is identical — same order of resolution, the
same single `pickNextStep` call, the same early returns, and each kind reaches
the same engine call with the same arguments as the prior inline chain (verified
by the `Exclude`/`Extract` narrowing matching the old `if` order). The exported
`batchApplyCommand` name and signature are unchanged. Affected suites green
(test/commands, test/core/batch, test/batch-engine: 1059 passed); lint/tsc/build
clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joctaTorres
left a comment
There was a problem hiding this comment.
PR #95 re-review — "feat(batch): per-stage model selection and attributed model failures"
Base: 15d4625 · Head: eb9de88 · 17154+ / 180−
Scope note. Follow-up to the review at f18740c, covering the four behavior-preserving refactor commits added since (1588e95, c2b209b, ae71bb1, eb9de88) that answer that review's five must-fix advisories. Merged commit f18740c (PR #96, reviewed separately) is not re-reviewed. Metrics are AST-derived with the same tooling and conventions as the prior review (lizard's TS tokenizer overcounts ?./?? and cannot parse pickNextStep's signature, so lizard is corroboration only). As in the prior review, the CC-density gate is evaluated on functions in the CC > 10 band — density distinguishes whether a CC-10+ function is genuinely condensed; applied below that band it would mechanically flag every small guard-style helper, including pre-existing ones the prior review deliberately did not flag.
Resolution of the prior review's must-fix advisories
| Prior advisory | Was | Now | Status |
|---|---|---|---|
pickNextStep CC 34 (gate 15) |
34 | 5 — split into 5 ordered selectors chained with ?? |
✅ resolved |
batchApplyCommand CC 19 (gate 15) |
17–19 | 4 — prContext assembly, no-target rendering, and step-kind dispatch extracted | ✅ resolved |
mapSessionToOutcome CC 16–18 (gate 15) |
18 | 12 — non-zero-exit failure branch extracted (buildNonZeroExitFailure, CC 7) |
✅ resolved |
resolveAgentSetting CC density 0.36 (CC 12 / NLOC 33) |
0.36 | CC 6 / NLOC 17 — fold delegated to foldAgentLayers |
✅ resolved (out of the CC > 10 gate band; the raw ratio is 0.35 only because numerator and denominator shrank together — the residual 3 guard returns + materialization loop are not condensed branching) |
resolveAgentStageScopes CC density 0.34 (CC 11 / NLOC 32) |
0.34 | CC 6 / NLOC 20 = 0.30, at (not above) the gate | ✅ resolved |
The prior review's CoN-Algorithm soft signal (the stage-scope resolver replaying the merge fold by hand) is also resolved: both consumers now call one generic foldAgentLayers parameterized on select, and the agreement invariant is asserted in test/core/batch/agent-scope-resolution.test.ts.
Behavior-preservation check on the four refactors
outcome.ts(1588e95) — verbatim lift:buildNonZeroExitFailurereproduces both return shapes byte-for-byte from the same destructured input. ✅config.ts(c2b209b) — the fold is a faithful parameterization; the scope consumer destructures and ignores the fold'ssourcefield, which its hand-rolled predecessor never computed. ✅pickNextStep(ae71bb1) — the??chain preserves the monolith's fall-through semantics exactly (a selector declining for any reason, e.g.!phase, proceeds to the next branch just as the old inline blocks fell through), in the same load-bearing order. ✅batchApplyCommand(eb9de88) — dispatch bodies are identical, arguments match one-for-one, and theExclude/Extractnarrowing plus exhaustiveswitchslightly strengthen type safety. ✅- Backstop: the four commits touch zero test files and the full suite passes unchanged at head — 2804/2804 tests, 196 files (
test/cli-e2e/**excluded; sidecar-gated).
Quality snapshot (base 15d4625 → head eb9de88)
| Metric | Before | After | Δ | Status |
|---|---|---|---|---|
| Cyclomatic (max changed fn) | 22 (pickNextStep) |
14 (setProjectBatchSetting, validateSetting) |
−8 | 🟢 |
| CC density (max new/changed fn, CC > 10 band) | — | 0.37 (selectChangeOrBoundaryProof, CC 11 / NLOC 30) |
new | 🔴 |
| ABC magnitude (max changed fn) | 36 (pickNextStep) |
29 (resolveBatchSettings) |
−7 | 🟢 |
| Coverage — line (changed files, agg) | 93.9% | 94.2% | +0.3pp | 🟢 |
| Coverage — branch (changed files, agg) | 90.3% | 91.2% | +0.9pp | 🟢 |
| Maintainability Index | skipped (no TS MI tool; AST approximation, size/doc-driven and informational only: outcome.ts −6.8, doctor/checks/agents.ts −8.6, instructions.ts −5.4 vs base) |
🟡 | ||
| Halstead Difficulty | skipped per-function (file-level approximation only) | — | ||
| LCOM4 (max touched class) | 1 | 1 | 0 | 🟢 |
| New coupling types | — | none Content/Common; the new ApplyDispatchContext/PickPrContext edges are Data (typed shapes) |
🟢 | |
| DSQI / IPL / WEAVE | DSQI: extraction preserved layer direction (selectors and dispatch helpers stay in the command layer, engine seams untouched) · IPL skipped (not perf-critical) · WEAVE skipped (no integration) | 🟢 |
Pre-existing, not gated: readProjectConfig CC 27 → 25 (still above 15, but improved by this range) and hasToolProfileOrDeliveryDrift CC 24 → 24 (function body untouched). Coverage measured with test/cli-e2e/** excluded, same as the prior review.
Hard gates: 🔴 must-fix advisory — CC density (1 function). Per this review's playbook the opinionated/style layer was again not run while a gate is red; everything above is metrics-layer plus verification of the refactor claims.
Must-fix advisories
- 🔴 CC density —
selectChangeOrBoundaryProofCC 11 / NLOC 30 = 0.37, above the 0.30 gate. This is the relocated core of the oldpickNextStepbody — the split resolved the CC-34 ceiling breach but concentrated the branching of the first (largest) branch into one selector that lands in exactly the band the prior review red-flagged forresolveAgentSetting(CC 12 / 0.36). Two mechanical cuts clear it without gaming: aSetmembership test for the three-way runnable-status check (−2 CC) and extracting the predecessor-boundary-proof guard (−3 CC) — whichselectDecomposeStepnear-duplicates, so the same helper removes a fresh two-site drift risk. Inline atsrc/commands/batch/apply.ts:505. Noquality-gate:override is present at the function head.
Reviewer attention (soft signals)
selectDecomposeStep(CC 9 / NLOC 24 = 0.375) andselectStackedPrStep(CC 9 / NLOC 26 = 0.35) sit below the CC > 10 gate band but are the next-densest new selectors; the shared predecessor-proof-guard extraction above bringsselectDecomposeStepdown as a side effect. Inline atsrc/commands/batch/apply.ts:563.setProjectBatchSettingCC 12 → 14 andvalidateSettingCC 13 → 14 — the write/validate pair on the config path each absorbed anotherkey === …branch this PR and now sit one branch from the 15 gate; the next persisted key type tips them over. A key→codec table (parse/serialize per key) absorbs future keys at CC +0. Inline atsrc/core/batch/config.ts:1001.src/commands/batch/apply.tsline coverage 95.8% → 92.3% (−3.5pp vs base, improved from 91.2% at the prior head). Unchanged finding: the uncovered growth is the PR-step routing exercised only by the sidecar-gated cli-e2e suites; a vitest-visible unit test drivingbatchApplyCommandthrough aprtarget with a fake engine (thedeps.branches/deps.proofseams already exist) would close it.- CoN-Position, still standing from the prior review:
runStackedPrtakes 11 positional parameters (src/commands/batch/apply.ts:893). The newApplyDispatchContextis the natural carrier — threading it (plus a small per-target shape) throughrunPr/runStackedPr/runDecompositionwould resolve the prior inline comment. resolveBatchSettingsCC 13 (AST; lizard counts 15 due to??) — unchanged since the prior review's note: at/near the gate depending on counting convention; the next stage key or setting tips it over.
Summary
The four refactor commits do what they claim: all five previously red advisories are resolved (CC 34 → 5, 19 → 4, 18 → 12; both config-fold density breaches gone via one shared, agreement-tested fold), verified equivalent by line-level comparison and an unchanged 2804-test pass, with aggregate coverage on changed files up on both axes vs base. One residual red remains: the extraction relocated the old pickNextStep's densest branch into selectChangeOrBoundaryProof (CC 11, density 0.37), which needs the two small cuts described inline — one of which also removes a new duplicated boundary-proof guard. Address that first; the remaining signals are the same coverage and parameter-arity notes carried over from the prior review.
| @@ -278,45 +537,125 @@ export function pickNextStep( | |||
| } | |||
| } | |||
| } | |||
| return undefined; | |||
| } | |||
There was a problem hiding this comment.
🔴 CC density 0.37 (CC 11 / NLOC 30), above the 0.30 gate. The pickNextStep split resolved the CC-34 ceiling breach, but this selector inherited the monolith's densest branch and lands in the same band the prior review flagged for resolveAgentSetting (CC 12 / 0.36). Two mechanical cuts clear it without metric-gaming:
- Replace the three-way status check (L516–L523) with
RUNNABLE_STATUSES.has(change.status)over a module-levelReadonlySet— −2 CC and theawaiting-verifycomment moves to the constant. - Extract the predecessor-boundary-proof guard (L527–L533) into a shared helper — −3 CC here, and it also deduplicates
selectDecomposeStep's variant (L563–L575).
Result ≈ CC 6 / density ≈ 0.27. Alternatively, a quality-gate: cc-density justified — … comment at the function head opts out explicitly.
| const phaseIndex = status.phases.findIndex((p) => p.name === phase.name); | ||
| if (phaseIndex > 0) { | ||
| const predStatus = status.phases[phaseIndex - 1]; | ||
| const predecessor = manifestPhases.find((p) => p.name === predStatus.name); | ||
| if ( | ||
| predecessor && | ||
| predStatus.status === 'done' && | ||
| predecessor.proofOfWork && | ||
| !recordedProofPhases.has(predecessor.name) | ||
| ) { | ||
| return { kind: 'proof-of-work', phase: predecessor }; | ||
| } | ||
| } |
There was a problem hiding this comment.
§3.7 / CoN-Algorithm (soft) — this predecessor-boundary-proof guard is a near-duplicate of selectChangeOrBoundaryProof L527–L533 (this variant adds the predStatus.status === 'done' && predecessor.proofOfWork conditions). A single shared pendingBoundaryProof(...) helper keeps the two boundary-gate rules from drifting apart and drops this selector's density (CC 9 / NLOC 24 = 0.375) below the band as a side effect. Grounded in: two-site connascence of algorithm introduced by the split.
| batch[key] = Number(value.trim()); | ||
| } else if (key === 'insecure') { | ||
| batch[key] = value.trim() === 'true'; | ||
| } else if (key === 'agent' && validation.parsedValue !== undefined) { |
There was a problem hiding this comment.
nit: CC trend — setProjectBatchSetting CC 12 → 14 with this branch, and its sibling validateSetting is also at 14; each new persisted key type adds another key === … arm to both, and the next one crosses the 15 gate. A key→codec table ({ parse, serialize } per key, shared by validate and persist) absorbs future keys at CC +0. Non-blocking. Grounded in: CC 14, one branch under the 15 gate.
…oundary-proof helper Reduce `selectChangeOrBoundaryProof` CC density from 0.37 (CC 11 / NLOC 30) to ~0.27 with two behavior-preserving cuts, both byte-identical to before: - Replace the three-way runnable-status check with a lookup against a new module-level `RUNNABLE_STATUSES: ReadonlySet<ChangeStatus>` (ready / in-progress / awaiting-verify); the awaiting-verify rationale moves onto the constant. - Extract the predecessor-boundary-proof guard into a shared `pendingBoundaryProof(...)` used by BOTH selectors. The two call sites keep their DISTINCT gates via `requireDoneWithProof`: the change branch passes `false` (predecessor is done by the gate invariant); the decompose branch passes `true` (adds the `status === 'done' && proofOfWork` conditions). As a side effect `selectDecomposeStep` also drops below the density band. No exported signature changes; `pickNextStep`'s public shape is unchanged. Verified via pickNextStep / boundary-proof / decompose unit tests (532 batch-engine tests green) and the full affected suites (1059 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the per-key `if (key === …)` chains in `validateSetting` (CC 14) and `setProjectBatchSetting` (CC 14) into a single `SETTING_CODECS` table with optional `validate`/`serialize` per key, read by BOTH functions. A future persisted key now lands at CC +0 (one table entry) instead of a branch in each. Behavior is byte-identical: - Validation order preserved (unknown-key → ALLOWED_VALUES enum → per-key shape/agent-schema) and every error-message string is verbatim (image empty, host/authToken empty, port/agentTimeoutMs integer, agent via validateAgentSetting carrying parsedValue). - Persist coercion preserved verbatim: port/agentTimeoutMs → Number(value.trim()), insecure → value.trim() === 'true', agent (parsedValue !== undefined) → parsedValue, everything else → value. No exported signature changes. Config + command suites and the full 2880-test suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joctaTorres
left a comment
There was a problem hiding this comment.
PR #95 re-review — "feat(batch): per-stage model selection and attributed model failures"
Base: 15d4625 · Head: 0c948f7 · 17286+ / 235−
Scope note. Third review, covering the two behavior-preserving refactor commits added since the review at eb9de88 (9756ca4, 0c948f7), which answer that review's one must-fix advisory and two of its signals. Merged commit f18740c (PR #96, reviewed separately) is not re-reviewed. Same tooling and conventions as both prior reviews: AST-derived CC with lizard as corroboration (lizard overcounts ?./??), the CC-density gate evaluated on functions in the CC > 10 band, coverage measured with test/cli-e2e/** excluded.
Resolution of the prior review's advisories
| Prior advisory | Was | Now | Status |
|---|---|---|---|
🔴 CC density — selectChangeOrBoundaryProof 0.37 (CC 11 / NLOC 30) |
0.37 | CC 8 / NLOC 28 = 0.29, below the 0.30 gate — and below the CC > 10 gate band entirely | ✅ resolved |
🟡 §3.7 / CoN-Algorithm — selectDecomposeStep near-duplicates the boundary-proof guard (CC 9 / density 0.375) |
0.375 | CC 6 / NLOC 21 = 0.29 — both call sites now share one pendingBoundaryProof helper; the two-site drift risk is gone |
✅ resolved |
nit: CC trend — setProjectBatchSetting CC 14, validateSetting CC 14, one branch from the 15 gate |
14 / 14 | 8 / 5 — both routed through one per-key SETTING_CODECS table (src/core/batch/config.ts:770); the next persisted key is one table entry, CC +0 in both functions |
✅ resolved |
Both commits implement exactly the cuts the prior review described (the Set membership test for the three-way runnable-status check, the shared predecessor-proof guard, the key→codec table), with no scope creep.
Behavior-preservation check on the two refactors
9756ca4(pendingBoundaryProof) — the two call sites' distinct gates are preserved viarequireDoneWithProof: the change branch passesfalseand returns the predecessor whenever it exists with an unrecorded proof — nodone/proofOfWorkrequirement, exactly as the old inline guard; the decompose branch passestrueand still additionally requirespredStatus.status === 'done' && predecessor.proofOfWork. The only delta is predicate order (the recorded-proof check now precedes the done/proof check in the decompose path) — a pure, side-effect-free conjunction, so the result is identical.RUNNABLE_STATUSESis exactly the old three statuses (ready,in-progress,awaiting-verify), comment included. ✅0c948f7(SETTING_CODECS) — every validation error string is byte-identical:imagekeeps its bespoke hardcoded message;host/authToken/port/agentTimeoutMsrender identical strings through the${key}template;agentstill short-circuits through the samevalidateAgentSetting. The codec lookup sits at the same position invalidateSetting(after the key/enum gates, before the generic success), and per key at most one codec applies, so ordering among the oldifarms is immaterial. Persisted YAML types are identical:Number(value.trim())forport/agentTimeoutMs,value.trim() === 'true'forinsecure,parsedValue !== undefined ? parsedValue : valueforagent, and the raw string for serialize-less keys (image/host/authTokendefinevalidateonly, so they fall through to the verbatim string, as before). ✅- Backstop: the two commits touch zero test files and the full suite passes unchanged at head — 2804/2804 tests, 900 suites (
test/cli-e2e/**excluded; sidecar-gated).
Quality snapshot (base 15d4625 → head 0c948f7)
| Metric | Before | After | Δ | Status |
|---|---|---|---|---|
| Cyclomatic (max new/changed fn) | 22 (pickNextStep) |
13 (resolveBatchSettings, AST; lizard 15) |
−9 | 🟢 |
| CC density (max new/changed fn, CC > 10 band) | 0.37 (selectChangeOrBoundaryProof) |
0.21 (ensureCommandInSpawnLocus) |
−0.16 | 🟢 |
| ABC magnitude (max changed fn) | 36 (pickNextStep) |
29 (resolveBatchSettings, untouched by these commits) |
−7 | 🟢 |
| Coverage — line (changed files, agg) | 93.9% | 94.2% (3210/3406) | +0.3pp | 🟢 |
| Coverage — branch (changed files, agg) | 90.3% | 91.1% (1016/1115) | +0.8pp | 🟢 |
| Maintainability Index | skipped (no TS MI tool; prior reviews' AST approximation was informational only) | — | ||
| Halstead Difficulty | skipped per-function (file-level approximation only) | — | ||
| LCOM4 (max touched class) | 1 | 1 (no class touched; SETTING_CODECS is a module-level table) |
0 | 🟢 |
| New coupling types | — | none Content/Common; one new intra-module boolean mode flag (requireDoneWithProof) — control-flag style, documented, nit below |
🟢 | |
| DSQI / IPL / WEAVE | DSQI: the codec table strengthens single-home discipline (validate and persist read one definition per key) · IPL skipped (not perf-critical) · WEAVE skipped (no integration) | 🟢 |
Hard gates: all passing ✅ — first fully green snapshot in this PR's review cycle.
Skipped metrics: MI (no TS tool), per-function Halstead (approximation only), IPL (not perf-critical), WEAVE (no integration).
Pre-existing, not gated (unchanged from prior reviews): readProjectConfig CC 25 (improved from 27 by this PR) and hasToolProfileOrDeliveryDrift CC 24 (body untouched).
Opinionated findings (§3 — first run; both prior reviews stopped at red gates)
Applied §3.1–§3.9 to the PR's original in-scope files (portions belonging to f18740c/PR #96 skipped as separately reviewed). The structural picture is clean:
- §3.1 / §3.2 — no findings.
engine/boundary.tsandengine/stacked-base.tsare pure, single-home policy modules with typed surfaces and resolver-injected seams (no git/forge literals);engine/index.tsre-exports only. The selector split inapply.tsand the codec table inconfig.tskeep command-layer logic in the command layer and config-shape knowledge in the config module. - §3.8 — satisfied for this range. Behavior-preserving refactors with the untouched 2804-test suite as the regression harness; the earlier feature commits shipped their own test files.
src/commands/batch/apply.ts
- nit: §3.3-adjacent control flag —
pendingBoundaryProof's fourth positionalrequireDoneWithProof: booleanreads as a barefalse/trueat both call sites. Non-blocking (the docstring specifies both modes precisely). Grounded in: Control-coupling category (intra-module, weakest form). Inline atsrc/commands/batch/apply.ts:532.
Reviewer attention (soft signals — carried, still standing)
- CoN-Position:
runStackedPrstill takes 11 positional parameters (src/commands/batch/apply.ts:928); the prior review's inline comment stands —ApplyDispatchContextplus a small per-target shape is the natural carrier. src/commands/batch/apply.tsline coverage 92.2% — the uncovered lines remain the PR-step routing exercised only by the sidecar-gated cli-e2e suites; a vitest-visible unit test drivingbatchApplyCommandthrough aprtarget with a fake engine (thedeps.branches/deps.proofseams exist) would close it.resolveBatchSettingsCC 13 (AST; lizard 15) — at/near the gate depending on counting convention; the next stage key or setting tips it over.
Summary
The two commits close out this PR's review cycle: the last red gate (selectChangeOrBoundaryProof density 0.37) is cleared to 0.29 by exactly the two cuts the prior review asked for, and the codec table retires the setProjectBatchSetting/validateSetting CC-trend nit with a design that also removes a validate/persist divergence risk. Line-level comparison confirms both refactors are behavior-preserving — distinct boundary-proof gates intact, every error string and persisted YAML type identical — with an unchanged, fully green 2804-test suite as backstop, and aggregate coverage on changed files up on both axes vs base. Every hard gate is green; what remains is the small carried list above (parameter arity on runStackedPr, vitest-visible PR-step coverage, resolveBatchSettings near the gate) plus one non-blocking naming nit on the new helper's boolean flag.
| predStatus: BatchStatusInfo['phases'][number] | undefined, | ||
| manifestPhases: Phase[], | ||
| recordedProofPhases: ReadonlySet<string>, | ||
| requireDoneWithProof: boolean |
There was a problem hiding this comment.
nit: §3.3-adjacent control flag — requireDoneWithProof: boolean reads as a bare false/true at the two call sites (L567, L606), so the reader must return here to learn which gate each branch gets. Two named one-line wrappers (e.g. pendingProofBeforeChange / pendingProofBeforeDecompose) — or named constants at the call sites — would carry the intent without re-widening either function. Non-blocking: the docstring specifies both modes precisely, and this review verified the two gates match the pre-refactor call sites exactly. Grounded in: Control coupling (intra-module, weakest form; no gate).
Make `decompose` a first-class routable agent stage so its agent+model is selectable from `.ratchet/config.yaml` (`batch.agent.decompose`) and the batch manifest (`settings.agent.decompose`), merged nearest-wins per stage — exactly as `pr` already is. `AGENT_STAGE_KEYS` gains `decompose`; the decomposition spawn threads the `decompose` stage into `ensureCommandInSpawnLocus` and `buildSpawnRequest`, and attribution reads `agentStageScopes.decompose`. The `uniformAgentScope` workaround (helper + engine call + tests) is deleted — it existed only because decompose was not a real stage. `Transition`/`StepKind` are unchanged; unmapped/unset decompose still falls to the default agent with no model flag (byte-for-byte prior behavior). Rename the batch PR skill `pr-open` → `open-pr` via the single-source `PR_OPEN_COMMAND_ID` value, the workflow module, and the `CORE_WORKFLOWS`/ `ALL_WORKFLOWS` profile lists; the forge-agnostic body and the `pr` stage key are unchanged. Docs (README, config-yaml, agent-runtime, generated-artifacts, batch, overview) updated for both; full suite + 95% coverage gate green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Summary
Driven end-to-end by the
agent-model-selectionratchet batch (2 phases, 7 changes, both proof-of-work gates green).Per-stage model selection
agentsetting — scalar or per-stage map, project config and manifest — widens to anagent[:model]spec, split on the first colon so opencodeprovider/modelids pass through intact:propose: claude:fable,apply: opencode:zai/glm-5.2,verify: opencode:qwen/qwen-3.7.UnknownAgentErrorstill fires before any spawn), and each adapter emits its own model flag (claude/opencode/cursor--model, codex/gemini-m) only when a model is named — a bare name emits no flag (agent's harness-default model) and keeps today's argv byte-for-byte.claude:,:fable) are rejected at config load naming the offending value; the model part is free-form pass-through (no model registry to rot). Nearest-wins per-stage merge operates on whole spec strings, so agent+model move atomically across scopes.Attributed model failures
ratchet doctorparses configured agent values through the same spec parser, probes the agent part's binary, and now fails the agent check when a configured agent's binary is missing; it never validates a model id.Docs & tests
docs/configuration/config-yaml.md,docs/engine/overview.md,docs/commands/doctor.mddocument the spec syntax, flag emission, the failure hint, and doctor's spec-aware probing.test/batch-engine/agent-model-selection.test.ts(config→argv end to end, real adapters + fake spawner) andtest/batch-engine/model-failure-attribution.test.ts(attributed surface viarunChangeStep).🤖 Generated with Claude Code
https://claude.ai/code/session_019S4t1YHtiCS4H35XtyEC1b