diff --git a/docs/configuration-model.md b/docs/configuration-model.md index c46425ca..425939bb 100644 --- a/docs/configuration-model.md +++ b/docs/configuration-model.md @@ -362,7 +362,7 @@ Some providers report their quota only through a usage endpoint of their own, ne ## Virtual models and tiers -A `models` block in `~/.agent-shim/config.json` defines virtual models over the providers' models: `agent-shim model add fast --mode soonest-reset --member z/glm-5.3 --member m/MiniMax-M3` names a choice, and its `mode` is how the choice is taken (`ordered` tries the members in order and falls to the fallback whose allowance is worth the most when one is spent, `most-allowance` serves from the member holding the most remaining allowance, `soonest-reset` spends what is about to reset, `spread` walks the members turn by turn, `adaptive` rules out what the request rules out and any member whose window's own pace so far (the allowance spent over the time since it opened, once a tenth of the period has passed) would exhaust it before the reset, and stays on the current model while it stands unless another's gain, as the share more of the allowance a steady burn would leave unspent, beats the share of a full one-million-token context that re-sending the conversation costs, `lowest-latency` and `highest-success` serve from the observed best, and `cheapest` ranks by the `model price` entries weighted by the tokens the session has actually been sending). A member names a provider model (`z/glm-5.3`), an account model (`acct:work/opus`), another virtual model (cycles are refused), or a whole model family (`family:`, every provider and account offering it). `agent-shim model tier opus --ref fast` points a Claude tier at one of these refs: for a session that routes by name the tier's model setting (`ANTHROPIC_DEFAULT_OPUS_MODEL` and its siblings) becomes the ref, which the door resolves per request, overriding that provider's own tier table for the tier, and `--clear` returns the tier to the table. A session that does not route by name keeps its provider's table, because only the door's name resolution can serve a ref. Nothing reads the block until a launch selects it, so a defined-but-unselected model changes nothing. `doctor` checks the block's own consistency: members naming no defined provider, virtual model or identity, cycles, a `cheapest` member with no prices, and a ref both hidden and shown. The block cascades: the global block is the shallowest layer, and a configuration profile, a directory rule and a committed `.agent-shim.json` carry the same `models` key in a layer form where any definition may be `null` to remove what the shallower layers set (`hide` and `show` replace wholesale), and the deepest layer naming a definition wins. The result is what a launch lists in `/model`, checks credentials for and applies tier pointers from; when a deeper layer changed it, the launch records the block it resolved in its session's registry entry and the door resolves that session's names against it, and when none did the door follows the global block live. A session that routes by name lists the catalogue in `/model`: its virtual models first, then one row per model family offered by more than one target (its description names every target, and selecting it lets the family's mode choose among them), then every provider's models qualified (`z / glm-5.3`), honouring the block's `hide` and `show` lists. A launch turns the block on with `launch.routeByName` (`--[no-]route-by-name`, `AGENT_SHIM_ROUTE_BY_NAME`, a profile's `--launch-route-by-name`, a directory rule or a committed file): a provider session then sends a provider-qualified model name (`m/glm-5.3`) or a virtual model's name to the provider that serves it instead of its own, and a `router` provider (a kind with no endpoint or credential) does the same whatever it launches with. A provider-less (OAuth) session routes qualified names the same way while its bare names still go to Anthropic, and its `/model` rows follow Claude Code's own rather than replacing them. The front door takes a virtual model's choice per request, through its mode over the live signals: the quota windows the session's identity last recorded for each member's provider, the latency, success and token mix each target's recent requests observed (seeded from the usage log at door start), and the block's prices, so an edited block or a spent window changes the next request's answer with no restart. Whatever the request named, the target receives its own model name: a qualified or virtual name is rewritten in the request's one field, exactly as a provider's own `routes` entry that names a model is. A launch with routing on also resolves every provider the block names before it starts, a `family:` member included (through every provider offering a target of the family), so a provider whose credential yields no token fails the launch naming it instead of the first request that names it. The door also reads what a request needs (an image, tools, extended thinking, the size of the context it re-sends) against what each candidate's family carries, which `adaptive` filters on. The same or a similarly named model offered by several providers and accounts is one family: its canonical key is the model's family and version with any variant after a `+` (`glm-5-1`, `claude-opus-5-5+high`), which `agent-shim model match` lists beside the pairs that are only similar. `model same z/glm-5.1 o/zai-glm-51` asserts that names the normaliser cannot connect are one model, `model distinct` asserts that two alike names are two, and `model family glm-5-1 --mode cheapest --no-vision --context 200000 --name o/odd-name` sets one family's own settings: how a row selecting the whole family chooses among its targets (`ordered` when absent), the names pinned into it, and what every one of its targets can do, which `adaptive` reads instead of a hand-tagged target. A variant never merges into its family by name alone: a member names the variant it means (`family:claude-opus-5-5+high`) and a bare key means the targets without one. These assertions and settings cascade with the rest of the block, accumulating across layers rather than replacing. `doctor` warns when a virtual model lists targets that look like one model, when a `same` assertion contradicts what the names say, and when a `family:` member names a family no offered target belongs to. +A `models` block in `~/.agent-shim/config.json` defines virtual models over the providers' models: `agent-shim model add fast --mode soonest-reset --member z/glm-5.3 --member m/MiniMax-M3` names a choice, and its `mode` is how the choice is taken (`ordered` tries the members in order and falls to the fallback whose allowance is worth the most when one is spent, `most-allowance` serves from the member holding the most remaining allowance, `soonest-reset` spends what is about to reset, `spread` walks the members turn by turn, `adaptive` rules out what the request rules out and any member whose window's own pace so far (the allowance spent over the time since it opened, once a tenth of the period has passed) would exhaust it before the reset, and stays on the current model while it stands unless another's gain, as the share more of the allowance a steady burn would leave unspent, beats the share of a full one-million-token context that re-sending the conversation costs, `lowest-latency` and `highest-success` serve from the observed best, and `cheapest` ranks by the `model price` entries weighted by the tokens the session has actually been sending). A member names a provider model (`z/glm-5.3`), an account model (`acct:work/opus`), another virtual model (cycles are refused), or a whole model family (`family:`, every provider and account offering it). `agent-shim model tier opus --ref fast` points a Claude tier at one of these refs: for a session that routes by name the tier's model setting (`ANTHROPIC_DEFAULT_OPUS_MODEL` and its siblings) becomes the ref, which the door resolves per request, overriding that provider's own tier table for the tier, and `--clear` returns the tier to the table. A session that does not route by name keeps its provider's table, because only the door's name resolution can serve a ref. Nothing reads the block until a launch selects it, so a defined-but-unselected model changes nothing. `doctor` checks the block's own consistency: members naming no defined provider, virtual model or identity, cycles, a `cheapest` member with no prices, and a ref both hidden and shown. The block cascades: the global block is the shallowest layer, and a configuration profile, a directory rule and a committed `.agent-shim.json` carry the same `models` key in a layer form where any definition may be `null` to remove what the shallower layers set (`hide` and `show` replace wholesale), and the deepest layer naming a definition wins. The result is what a launch lists in `/model`, checks credentials for and applies tier pointers from; when a deeper layer changed it, the launch records the block it resolved in its session's registry entry and the door resolves that session's names against it, and when none did the door follows the global block live. A session that routes by name lists the catalogue in `/model`: its virtual models first, then one row per model family offered by more than one target (its description names every target, and selecting it lets the family's mode choose among them), then every provider's models qualified (`z / glm-5.3`), honouring the block's `hide` and `show` lists. A launch turns the block on with `launch.routeByName` (`--[no-]route-by-name`, `AGENT_SHIM_ROUTE_BY_NAME`, a profile's `--launch-route-by-name`, a directory rule or a committed file): a provider session then sends a provider-qualified model name (`m/glm-5.3`) or a virtual model's name to the provider that serves it instead of its own, and a `router` provider (a kind with no endpoint or credential) does the same whatever it launches with. A provider-less (OAuth) session routes qualified names the same way, and the names the block defines itself (a virtual model's, a `family:`), while every other bare name still goes to Anthropic, and its `/model` rows follow Claude Code's own rather than replacing them. The front door takes a virtual model's choice per request, through its mode over the live signals: the quota windows the session's identity last recorded for each member's provider, the latency, success and token mix each target's recent requests observed (seeded from the usage log at door start), and the block's prices, so an edited block or a spent window changes the next request's answer with no restart. Whatever the request named, the target receives its own model name: a qualified or virtual name is rewritten in the request's one field, exactly as a provider's own `routes` entry that names a model is. A launch with routing on also resolves every provider the block names before it starts, a `family:` member included (through every provider offering a target of the family), so a provider whose credential yields no token fails the launch naming it instead of the first request that names it. The door also reads what a request needs (an image, tools, extended thinking, the size of the context it re-sends) against what each candidate's family carries, which `adaptive` filters on. The same or a similarly named model offered by several providers and accounts is one family: its canonical key is the model's family and version with any variant after a `+` (`glm-5-1`, `claude-opus-5-5+high`), which `agent-shim model match` lists beside the pairs that are only similar. `model same z/glm-5.1 o/zai-glm-51` asserts that names the normaliser cannot connect are one model, `model distinct` asserts that two alike names are two, and `model family glm-5-1 --mode cheapest --no-vision --context 200000 --name o/odd-name` sets one family's own settings: how a row selecting the whole family chooses among its targets (`ordered` when absent), the names pinned into it, and what every one of its targets can do, which `adaptive` reads instead of a hand-tagged target. A variant never merges into its family by name alone: a member names the variant it means (`family:claude-opus-5-5+high`) and a bare key means the targets without one. These assertions and settings cascade with the rest of the block, accumulating across layers rather than replacing. `doctor` warns when a virtual model lists targets that look like one model, when a `same` assertion contradicts what the names say, and when a `family:` member names a family no offered target belongs to. An `acct:` member is served by relaunch, one login per process: the door leaves a move request for the session's supervisor (a pool launch under supervision) and answers the request saying it is being moved, which resumes under that identity with the picked model; a session nothing supervises is told the command to run instead (`claude @work --resume`). `agent-shim model explain ` shows which member a virtual model's mode picks over the same signals, and why each candidate stands where it does. diff --git a/src/frontdoor/providerRoute.test.ts b/src/frontdoor/providerRoute.test.ts index 5c0b144a..d9fa787b 100644 --- a/src/frontdoor/providerRoute.test.ts +++ b/src/frontdoor/providerRoute.test.ts @@ -144,7 +144,7 @@ describe("createProviderRouteResolver", () => { expect(qualified.request?.url).toContain("/providers/z/v1/messages"); } - // A bare name is Claude Code's own on an OAuth session: it still rides the pass-through to Anthropic, never resolved through the models block. + // A bare name the models block does not define is Claude Code's own on an OAuth session: it still rides the pass-through to Anthropic, never resolved through the models block. const bare = await resolve(byName("claude-sonnet-4-5")); expect(bare.ok).toBe(true); if (bare.ok) { @@ -159,6 +159,32 @@ describe("createProviderRouteResolver", () => { } }); + it("routes a name the models block defines, a virtual model's or a family's, from a provider-less (OAuth) session that asked to route by name", async () => { + const byName = (model: string): RoutedRequest => ({ + ...request("/v1/messages", JSON.stringify({ model, max_tokens: 16, messages: [{ role: "user", content: "hi" }] })), + session: { identity: undefined, sessionId: undefined, provider: undefined, headroom: false, routeByName: true, projectId: undefined }, + }); + const models: ModelsConfig = { virtual: { fast: { mode: "ordered", members: ["z/glm-5.3"] } } }; + const resolve = createProviderRouteResolver({ fs: fakeFs({ [`${PROVIDERS_DIR}/z.json`]: httpProvider }), providersDir: PROVIDERS_DIR, codexPorts: codexPorts(), directPort: () => OWN_PORT, env: { Z: "tok-from-z" }, credentials: fakeCredentials(), models: () => models, targets: () => ["z/glm-5.3"] }); + + const virtual = await resolve(byName("fast")); + expect(virtual.ok).toBe(true); + if (virtual.ok) { + expect(virtual.route.name).toBe("http:z"); + } + const family = await resolve(byName("family:glm-5-3")); + expect(family.ok).toBe(true); + if (family.ok) { + expect(family.route.name).toBe("http:z"); + } + // The same block leaves a bare name it does not define on the pass-through to Anthropic. + const bare = await resolve(byName("claude-sonnet-4-5")); + expect(bare.ok).toBe(true); + if (bare.ok) { + expect(bare.route.name).toBe("anthropic"); + } + }); + it("serves a provider session's bare /v1/ target through its provider's route, the request rewritten under the scoped path the route parses", async () => { // An antigravity provider whose credential never resolves: the route must get as far as its credential (a 401 naming the provider), which a bare unrewritten path never reaches because the route cannot parse a provider out of it. const resolve = resolver({ [`${PROVIDERS_DIR}/ag.json`]: { kind: "antigravity", displayName: "AG", credential: { sources: [{ env: "NOT_SET_ANYWHERE" }] }, antigravity: { mode: "cloud" } } }); diff --git a/src/frontdoor/providerRoute.ts b/src/frontdoor/providerRoute.ts index fa598fcb..54d32b8a 100644 --- a/src/frontdoor/providerRoute.ts +++ b/src/frontdoor/providerRoute.ts @@ -70,12 +70,11 @@ function needsOf(scan: Readonly<{ readonly hasImage?: boolean; readonly toolsPre return { hasImage: scan.hasImage === true, hasTools: scan.toolsPresent === true, thinking: scan.thinking === true, contextTokens: Math.ceil((scan.text?.length ?? 0) / CHARS_PER_TOKEN) }; } -/** Resolves a routing session's model name to its target: a qualified name names its provider directly, a virtual model or a family (`family:`) chooses among its members through its selection mode (quota windows from the session's snapshot, latency and success from the observed targets, prices from the models block, what the request needs against what each member's family carries), and a bare unknown name has no target. */ +/** Resolves a routing session's model name to its target: a qualified name names its provider directly, a virtual model or a family (`family:`) chooses among its members through its selection mode (quota windows from the session's snapshot, latency and success from the observed targets, prices from the models block, what the request needs against what each member's family carries), and a bare name the block does not define has no target. */ function resolveRoutedModel( model: string, providerName: string | undefined, session: Readonly<{ readonly identity?: string; readonly sessionId?: string }>, - includeUnqualified: boolean, deps: Readonly, memory: Readonly>, rotations: Readonly>, @@ -91,7 +90,8 @@ function resolveRoutedModel( } const isFamily = model.startsWith(FAMILY_PREFIX); const virtual = isFamily ? undefined : models?.virtual?.[model]; - if ((virtual === undefined && !isFamily) || !includeUnqualified) { + // A name the models block defines is the user's own, so it is routed on any session that asked to route by name, a provider-less (OAuth) one included; a bare name the block does not define stays Claude Code's own and is never resolved here. + if (virtual === undefined && !isFamily) { return undefined; } const declared = isFamily ? [model] : expandMembers(model, models); @@ -390,7 +390,7 @@ export function createProviderRouteResolver(deps: { : `model "${account.model}" runs under identity "${account.identity}", which needs a relaunch this session cannot make; start it there with: claude @${account.identity} --resume`, }; } - const routed = scan.model === undefined ? undefined : resolveRoutedModel(scan.model, providerName, current.session, isRouter || providerName !== undefined, deps, routingMemory, rotations, needsOf(scan)); + const routed = scan.model === undefined ? undefined : resolveRoutedModel(scan.model, providerName, current.session, deps, routingMemory, rotations, needsOf(scan)); const requested = scan.model; if (routed !== undefined) { routingMemory.set(current.session.sessionId ?? "", routed.ref);