Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ agent-shim run [@<identity>] [launch flags] [claude arguments]
# --claude-version <version> run exactly this installed Claude Code version
# --delay <duration> wait this long after the launcher's own output before starting claude; --[no-]pause waits for a key press instead
# --[no-]explain with @pool:<name>, list every member the pick ranked and why, not only the one picked
# --[no-]route-by-name route provider-qualified model names through the models block to the provider that serves them
# --[no-]route-by-name also route bare virtual-model names through the models block (qualified names route on every session)
# --native run the real claude with nothing from agent-shim applied; cannot be combined with the flags above
claude @<identity> ... # the same, once `agent-shim shim enable` has run
```
Expand Down
2 changes: 1 addition & 1 deletion docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ agent-shim run [@<identity>] [launch flags] [claude arguments]
# --claude-version <version> run exactly this installed Claude Code version
# --delay <duration> wait this long after the launcher's own output before starting claude; --[no-]pause waits for a key press instead
# --[no-]explain with @pool:<name>, list every member the pick ranked and why, not only the one picked
# --[no-]route-by-name route provider-qualified model names through the models block to the provider that serves them
# --[no-]route-by-name also route bare virtual-model names through the models block (qualified names route on every session)
# --native run the real claude with nothing from agent-shim applied; cannot be combined with the flags above
claude @<identity> ... # the same, once `agent-shim shim enable` has run
```
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ Some providers report their quota only through a usage endpoint of their own, ne

## Virtual models and tiers

A `models` block in `~/.agent-shim/config.json` defines virtual models over the providers' models: `agent-shim model add fast --mode soonest-reset --member z/glm-5.3 --member m/MiniMax-M3` names a choice, and its `mode` is how the choice is taken (`ordered` tries the members in order and falls to the fallback whose allowance is worth the most when one is spent, `most-allowance` serves from the member holding the most remaining allowance, `soonest-reset` spends what is about to reset, `spread` walks the members turn by turn, `adaptive` rules out what the request rules out and any member whose window's own pace so far (the allowance spent over the time since it opened, once a tenth of the period has passed) would exhaust it before the reset, and stays on the current model while it stands unless another's gain, as the share more of the allowance a steady burn would leave unspent, beats the share of a full one-million-token context that re-sending the conversation costs, `lowest-latency` and `highest-success` serve from the observed best, and `cheapest` ranks by the `model price` entries weighted by the tokens the session has actually been sending). A member names a provider model (`z/glm-5.3`), an account model (`acct:work/opus`), another virtual model (cycles are refused), or a whole model family (`family:<canonical>`, every provider and account offering it). `agent-shim model tier opus --ref fast` points a Claude tier at one of these refs: for a session that routes by name the tier's model setting (`ANTHROPIC_DEFAULT_OPUS_MODEL` and its siblings) becomes the ref, which the door resolves per request, overriding that provider's own tier table for the tier, and `--clear` returns the tier to the table. A session that does not route by name keeps its provider's table, because only the door's name resolution can serve a ref. Nothing reads the block until a launch selects it, so a defined-but-unselected model changes nothing. `doctor` checks the block's own consistency: members naming no defined provider, virtual model or identity, cycles, a `cheapest` member with no prices, and a ref both hidden and shown. The block cascades: the global block is the shallowest layer, and a configuration profile, a directory rule and a committed `.agent-shim.json` carry the same `models` key in a layer form where any definition may be `null` to remove what the shallower layers set (`hide` and `show` replace wholesale), and the deepest layer naming a definition wins. The result is what a launch lists in `/model`, checks credentials for and applies tier pointers from; when a deeper layer changed it, the launch records the block it resolved in its session's registry entry and the door resolves that session's names against it, and when none did the door follows the global block live. A session that routes by name lists the catalogue in `/model`: its virtual models first, then one row per model family offered by more than one target (its description names every target, and selecting it lets the family's mode choose among them), then every provider's models qualified (`z / glm-5.3`), honouring the block's `hide` and `show` lists. A launch turns the block on with `launch.routeByName` (`--[no-]route-by-name`, `AGENT_SHIM_ROUTE_BY_NAME`, a profile's `--launch-route-by-name`, a directory rule or a committed file): a provider session then sends a provider-qualified model name (`m/glm-5.3`) or a virtual model's name to the provider that serves it instead of its own, and a `router` provider (a kind with no endpoint or credential) does the same whatever it launches with. A provider-less (OAuth) session routes qualified names the same way, and the names the block defines itself (a virtual model's, a `family:<key>`), while every other bare name still goes to Anthropic, and its `/model` rows follow Claude Code's own rather than replacing them. The front door takes a virtual model's choice per request, through its mode over the live signals: the quota windows the session's identity last recorded for each member's provider, the latency, success and token mix each target's recent requests observed (seeded from the usage log at door start), and the block's prices, so an edited block or a spent window changes the next request's answer with no restart. Whatever the request named, the target receives its own model name: a qualified or virtual name is rewritten in the request's one field, exactly as a provider's own `routes` entry that names a model is. A launch with routing on also resolves every provider the block names before it starts, a `family:` member included (through every provider offering a target of the family), so a provider whose credential yields no token fails the launch naming it instead of the first request that names it. The door also reads what a request needs (an image, tools, extended thinking, the size of the context it re-sends) against what each candidate's family carries, which `adaptive` filters on. The same or a similarly named model offered by several providers and accounts is one family: its canonical key is the model's family and version with any variant after a `+` (`glm-5-1`, `claude-opus-5-5+high`), which `agent-shim model match` lists beside the pairs that are only similar. `model same z/glm-5.1 o/zai-glm-51` asserts that names the normaliser cannot connect are one model, `model distinct` asserts that two alike names are two, and `model family glm-5-1 --mode cheapest --no-vision --context 200000 --name o/odd-name` sets one family's own settings: how a row selecting the whole family chooses among its targets (`ordered` when absent), the names pinned into it, and what every one of its targets can do, which `adaptive` reads instead of a hand-tagged target. A variant never merges into its family by name alone: a member names the variant it means (`family:claude-opus-5-5+high`) and a bare key means the targets without one. These assertions and settings cascade with the rest of the block, accumulating across layers rather than replacing. `doctor` warns when a virtual model lists targets that look like one model, when a `same` assertion contradicts what the names say, and when a `family:` member names a family no offered target belongs to.
A `models` block in `~/.agent-shim/config.json` defines virtual models over the providers' models: `agent-shim model add fast --mode soonest-reset --member z/glm-5.3 --member m/MiniMax-M3` names a choice, and its `mode` is how the choice is taken (`ordered` tries the members in order and falls to the fallback whose allowance is worth the most when one is spent, `most-allowance` serves from the member holding the most remaining allowance, `soonest-reset` spends what is about to reset, `spread` walks the members turn by turn, `adaptive` rules out what the request rules out and any member whose window's own pace so far (the allowance spent over the time since it opened, once a tenth of the period has passed) would exhaust it before the reset, and stays on the current model while it stands unless another's gain, as the share more of the allowance a steady burn would leave unspent, beats the share of a full one-million-token context that re-sending the conversation costs, `lowest-latency` and `highest-success` serve from the observed best, and `cheapest` ranks by the `model price` entries weighted by the tokens the session has actually been sending). A member names a provider model (`z/glm-5.3`), an account model (`acct:work/opus`), another virtual model (cycles are refused), or a whole model family (`family:<canonical>`, every provider and account offering it). `agent-shim model tier opus --ref fast` points a Claude tier at one of these refs: for a session that routes by name the tier's model setting (`ANTHROPIC_DEFAULT_OPUS_MODEL` and its siblings) becomes the ref, which the door resolves per request, overriding that provider's own tier table for the tier, and `--clear` returns the tier to the table. A session that does not route by name keeps its provider's table, because only the door's name resolution can serve a ref. Nothing reads the block until a launch selects it, so a defined-but-unselected model changes nothing. `doctor` checks the block's own consistency: members naming no defined provider, virtual model or identity, cycles, a `cheapest` member with no prices, and a ref both hidden and shown. The block cascades: the global block is the shallowest layer, and a configuration profile, a directory rule and a committed `.agent-shim.json` carry the same `models` key in a layer form where any definition may be `null` to remove what the shallower layers set (`hide` and `show` replace wholesale), and the deepest layer naming a definition wins. The result is what a launch lists in `/model`, checks credentials for and applies tier pointers from; when a deeper layer changed it, the launch records the block it resolved in its session's registry entry and the door resolves that session's names against it, and when none did the door follows the global block live. Every session lists the catalogue's qualified rows in `/model`: every provider's models under its name (`z / glm-5.3`), honouring the block's `hide` and `show` lists; a session that routes by name lists its virtual models first, then one row per model family offered by more than one target (its description names every target, and selecting it lets the family's mode choose among them). A provider-qualified model name (`m/glm-5.3`) is routed on every session, whatever it launched on and with no setting asked for: a slash whose first segment names a defined provider sends the request to that provider, rewritten under its own model name, and a slash naming no provider (an aggregator's `vendor/model`) stays the session's own provider's business. A launch turns the bare names on with `launch.routeByName` (`--[no-]route-by-name`, `AGENT_SHIM_ROUTE_BY_NAME`, a profile's `--launch-route-by-name`, a directory rule or a committed file): a provider session then sends a virtual model's or a family's name to the provider its mode picks instead of its own, and a `router` provider (a kind with no endpoint or credential) does the same whatever it launches with. A provider-less (OAuth) session routes the names the block defines itself (a virtual model's, a `family:<key>`) the same way, while every other bare name still goes to Anthropic, and its `/model` rows follow Claude Code's own rather than replacing them. The front door takes a virtual model's choice per request, through its mode over the live signals: the quota windows the session's identity last recorded for each member's provider, the latency, success and token mix each target's recent requests observed (seeded from the usage log at door start), and the block's prices, so an edited block or a spent window changes the next request's answer with no restart. Whatever the request named, the target receives its own model name: a qualified or virtual name is rewritten in the request's one field, exactly as a provider's own `routes` entry that names a model is. A launch with routing on also resolves every provider the block names before it starts, a `family:` member included (through every provider offering a target of the family), so a provider whose credential yields no token fails the launch naming it instead of the first request that names it. The door also reads what a request needs (an image, tools, extended thinking, the size of the context it re-sends) against what each candidate's family carries, which `adaptive` filters on. The same or a similarly named model offered by several providers and accounts is one family: its canonical key is the model's family and version with any variant after a `+` (`glm-5-1`, `claude-opus-5-5+high`), which `agent-shim model match` lists beside the pairs that are only similar. `model same z/glm-5.1 o/zai-glm-51` asserts that names the normaliser cannot connect are one model, `model distinct` asserts that two alike names are two, and `model family glm-5-1 --mode cheapest --no-vision --context 200000 --name o/odd-name` sets one family's own settings: how a row selecting the whole family chooses among its targets (`ordered` when absent), the names pinned into it, and what every one of its targets can do, which `adaptive` reads instead of a hand-tagged target. A variant never merges into its family by name alone: a member names the variant it means (`family:claude-opus-5-5+high`) and a bare key means the targets without one. These assertions and settings cascade with the rest of the block, accumulating across layers rather than replacing. `doctor` warns when a virtual model lists targets that look like one model, when a `same` assertion contradicts what the names say, and when a `family:` member names a family no offered target belongs to.

An `acct:` member is served by relaunch, one login per process: the door leaves a move request for the session's supervisor (a pool launch under supervision) and answers the request saying it is being moved, which resumes under that identity with the picked model; a session nothing supervises is told the command to run instead (`claude @work --resume`). `agent-shim model explain <name>` shows which member a virtual model's mode picks over the same signals, and why each candidate stands where it does.

Expand Down
18 changes: 18 additions & 0 deletions src/frontdoor/providerRoute.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,24 @@ describe("createProviderRouteResolver", () => {
}
});

it("routes a provider-qualified model name on every session, whatever it launched on, with no flag asked or given", async () => {
const plain = (model: string, provider: string | undefined): RoutedRequest => ({ ...request("/v1/messages", JSON.stringify({ model, max_tokens: 16, messages: [{ role: "user", content: "hi" }] })), session: { identity: undefined, sessionId: undefined, provider, headroom: false, routeByName: false, projectId: undefined } });
const resolve = resolver({ [`${PROVIDERS_DIR}/z.json`]: httpProvider, [`${PROVIDERS_DIR}/s.json`]: { displayName: "S", baseUrl: "https://api.synthetic.new/anthropic", credential: { sources: [{ literal: "s-token" }] } } });
const withBlock = createProviderRouteResolver({ fs: fakeFs({ [`${PROVIDERS_DIR}/z.json`]: httpProvider }), providersDir: PROVIDERS_DIR, codexPorts: codexPorts(), directPort: () => OWN_PORT, env: { Z: "tok-from-z" }, credentials: fakeCredentials(), models: () => ({ virtual: { fast: { mode: "ordered", members: ["z/glm-5.3"] } } }), targets: () => ["z/glm-5.3"] });
// z naming s's model leaves z for s; a qualified name for the session's own provider is served right there under the bare name.
const cross = await resolve(plain("s/glm-5.2", "z"));
expect(cross.ok && cross.route.name).toBe("http:s");
const own = await resolve(plain("z/glm-5.3", "z"));
expect(own.ok && own.route.name).toBe("http:z");
// A slash alone is not a provider ref: a first segment no provider defines (an aggregator's `vendor/model`) stays the session's own provider's business.
const slashed = await resolve(plain("hf:zai-org/GLM-5.1", "z"));
expect(slashed.ok && slashed.route.name).toBe("http:z");
// A bare name is never resolved without the session's opt-in, even one the block defines.
const bare = await withBlock(plain("fast", "z"));
expect(bare.ok && bare.route.name).toBe("http:z");
if (bare.ok) expect(bare.modelChoice).toBeUndefined();
});

it("routes a provider-qualified model name from a provider-less (OAuth) session that asked to route by name onto that provider's route", async () => {
const byName = (model: string, url = "/v1/messages"): RoutedRequest => ({
...request(url, JSON.stringify({ model, max_tokens: 16, messages: [{ role: "user", content: "hi" }] })),
Expand Down
Loading
Loading