Repository navigation
Add runtime model fallback to the agent loop (WI-041) - #122
Merged
Merged
Conversation
A role with a prioritized models list (ADR-039) already picked the best model a budget affords, statically, before any call. This closes the gap that left open: when the resolved primary model turns out unreachable at invocation time, the openai-http transport now falls back to the next model in the role's chain instead of failing the whole cycle, skipping any candidate the current budget cannot afford and never retrying a real (non-network) error from a reachable endpoint. The metric for a fallback run records the model that actually answered and which candidates were tried first, so the audit trail stays accurate.
nateshpp
added a commit
that referenced
this pull request
Jul 7, 2026
Resolves conflicts against WI-041 (runtime model fallback, #122), which merged first: both branches touched run-cycle.mjs and CHANGELOG.md's Unreleased section for complementary, non-overlapping features. Regenerates the snapshot from scratch.
nateshpp
added a commit
that referenced
this pull request
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
A role with a prioritized
modelslist (ADR-039) already picked the best model a budgetaffords, statically, before any call was made. This closes the gap that left open: when the
resolved primary model turns out unreachable at invocation time (a network-layer failure, or
a timeout),
scripts/agent/run-cycle.mjs's openai-http transport (invokeRoleOpenAI) nowfalls back to the next model in the role's chain instead of failing the whole cycle. A
candidate the current daily budget cannot afford is skipped without an attempt, so a fallback
can never silently spend past the budget. A real answer from a reachable endpoint, an auth
failure, a bad status, a malformed response, is never retried, only genuine unreachability is,
so a broken diff or a rejected request is never silently re-tried against a different model.
The metric recorded for a fallback run reflects the model that actually answered, plus a
model_fallback_fromfield listing every unreachable candidate tried first, so the audittrail never misattributes a diff to the originally planned model.
The change is fully backward compatible: a plan built without a runtime chain (any hand-built
plan, or a role with a single configured model) takes the exact original one-shot code path,
with no new checks and no behavior change. Closes WI-041.
Governance checklist
npm run verifypasses locally (drift guard + style check + tests + AgentProof)docs/specs/ratchet-spec.mdand added/updated a fixture (N/A, no ratchet logic touched)schema_versionbumped and CHANGELOG entry added (N/A, no schema change; CHANGELOG entry added regardless)npm run build:promptregenerated the bundle (N/A, no prompt change)Test evidence
New coverage:
tests/agent-model-fallback.test.mjs(10 tests) covers the unreachable-errorclassifier, the fallback chain builder (including CLI-override ordering and registry pruning),
planCycleattaching a resolved chain to maker/checker, andinvokeRoleOpenAIintegration:falls back on a genuinely unreachable primary, never falls back on a real non-network error,
skips an unaffordable paid candidate without attempting it, and is behavior-identical for a
plan with no chain.