diff --git a/docs/architecture.md b/docs/architecture.md index a71f7300..3be76cda 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -205,7 +205,7 @@ When a launch resolves `headroom` on, the launcher (synchronous end to end, righ The front door, not the headroom supervisor, hosts the second server: the CONNECT surface OAuth launches point `HTTPS_PROXY` at. It terminates TLS for the intercept hosts, each with its own leaf signed by a local CA persisted under `/frontdoor/ca/` (the key mode 0600): `api.anthropic.com`, whose `/v1/` paths run through the same ordered pipeline the provider listener serves, and `platform.claude.com`, the claude.ai control plane, which is never HTTP-parsed and is pumped byte for byte; every path on the API host the pipeline does not serve, and every host outside the intercept set, is piped untouched. The transparent surface, a second listener for traffic that ignores proxies, terminates redirected connections the same way, keyed on the TLS SNI name the client presents. None of that happens for a client that has not authenticated: the CONNECT request must carry a live launch's capability as its `Proxy-Authorization` credential (the launcher puts it in the `HTTPS_PROXY` URL, from which clients derive that header), checked against the session registry with the same constant-time comparison the pipeline's admission uses, or it is answered 407 before the target is parsed or dialled. A connection ledger inside the surface enforces `ConnectLimits`: a head deadline, a pending-head cap that evicts the oldest waiting connection, a cap on authenticated connections, and a periodic revalidation that closes every connection whose launch's capability has since been pruned. Authenticated tunnels deliberately have no idle timeout, since Remote Control streams may sit silent indefinitely. That shape is what keeps Claude Code's Remote Control and connectors working under compression: they refuse any `ANTHROPIC_BASE_URL` the child can be given, but honour the proxy layer, so the base URL stays pointing at the real API. All of the surface's effects (TCP listening, TLS termination, HTTP parsing, forwarding) sit behind a `ConnectEffects` port in the same style as `SupervisorPorts`, and its routing decisions are pure functions over strings; the TLS round-trip tests run the real server against certificates this module issues with only the upstream target redirected. -The same CA signs the front door's provider listener, which serves HTTPS with a leaf for `127.0.0.1` and `localhost`. Provider sessions send their credential and launch capability on every request, so the child must be able to tell the door from anything else holding the port; a TLS leaf whose CA key only this user can read is that proof, and every routed child trusts the CA through `NODE_EXTRA_CA_CERTS` (combined with any bundle the parent named, since Node reads only one file). The launcher checks the same thing before trusting a door it finds: `ensureFrontDoor` probes the listener's health endpoint over TLS against the CA, from a worker thread that the synchronous launcher waits on with `Atomics.wait`, and a listener that keeps failing is distrusted and replaced, or the launch refused, before any capability is written. This tool's own supervisor that misses its probes is retired first, so the replacement keeps the sticky ports, but only when no live session is registered against it: a miss is what a busy door looks like, so a door with live sessions is waited on until the launch's own deadline and reported, never signalled, and `frontdoor restart` is the deliberate way to replace it: an explicit replace retires the door even when it is not answering, asking it to exit and killing it if it has not exited after a grace period, because a door whose event loop is blocked never runs its signal handler. A launch planned inside the serving door itself (the usage refresh, the window opener, a job) registers its session and returns the door's own addresses without probing, because the thread that would wait on the probe is the one event loop that has to answer it. The door enters its process role (`src/processRole.ts`) before it binds anything, and the primitives that block the calling thread (the synchronous sleep and the synchronous probe) refuse to run while a role is held, so a path that would stall the door throws `BlockingInEventLoopError` at its first wait instead of freezing every session. The door also carries a watchdog on a thread of its own (`frontdoor/watchdog.ts`): the supervisor beats a shared counter every tick and the thread, whose event loop a blocked main loop cannot stall, ends the process when the counter has not moved for twice the longest a launcher waits on a door before calling it unresponsive, after recording the stall in the state file's `lastError`. It is started once the listeners are bound and stopped before any retirement begins, and it restarts its clock when it was itself descheduled for longer than its cadence, so a suspended or starved machine is not taken for a hung door. A stall always leaves a replacement behind, started while the old pid is still dying so it can bind the same sticky ports the moment they free (the session registry is on disk either way); the chain stops after a bounded run of consecutive replacements, because a stall that has recurred through that many doors in a row is the machine's, and the next launch's ensure is what starts the door after that. `doctor` and `frontdoor status` ask the listeners themselves whether they accept (`frontdoor/listenerReadings.ts`): a TLS handshake against the CA on the provider listener and a plain TCP accept on the CONNECT surface, each timed, because a live supervisor pid cannot tell a door that serves from one whose event loop is blocked, and the two listeners carry different sessions (provider sessions on one, every plain OAuth session on the other). The direct listener headroom forwards back to stays plain HTTP, because headroom is what connects to it; what keeps that leg harmless is that no credential travels on it. The hop parks the provider's credential headers in an in-memory custody record and sends headroom placeholders and a hop id, and the direct listener's admission step redeems a live id (for the same provider, with the generation's hop secret) for the real headers before routing. Admission is a pipeline stage in its own right, returning the headers a request is routed with, so restoring credentials is admission's job and no route knows about it. The direct pipeline runs no response observers of its own (its responses are consumed by the hop, not the client, and observing both would record every hopped request twice), so a model choice a route notes while serving behind the hop rides home as an internal response header the hop reads, notes on the client-facing response and strips before any client sees it. +The same CA signs the front door's provider listener, which serves HTTPS with a leaf for `127.0.0.1` and `localhost`. Provider sessions send their credential and launch capability on every request, so the child must be able to tell the door from anything else holding the port; a TLS leaf whose CA key only this user can read is that proof, and every routed child trusts the CA through `NODE_EXTRA_CA_CERTS` (combined with any bundle the parent named, since Node reads only one file). The launcher checks the same thing before trusting a door it finds: `ensureFrontDoor` probes the listener's health endpoint over TLS against the CA, from a worker thread that the synchronous launcher waits on with `Atomics.wait`, and a listener that keeps failing is distrusted and replaced, or the launch refused, before any capability is written. This tool's own supervisor that misses its probes is retired first, so the replacement keeps the sticky ports, but only when no live session is registered against it: a miss is what a busy door looks like, so a door with live sessions is waited on until the launch's own deadline and reported, never signalled, and `frontdoor restart` is the deliberate way to replace it: an explicit replace retires the door even when it is not answering, asking it to exit and killing it if it has not exited after a grace period, because a door whose event loop is blocked never runs its signal handler. A launch planned inside the serving door itself (the usage refresh, the window opener, a job) registers its session and returns the door's own addresses without probing, because the thread that would wait on the probe is the one event loop that has to answer it. The door enters its process role (`src/processRole.ts`) before it binds anything, and the primitives that block the calling thread (the synchronous sleep and the synchronous probe) refuse to run while a role is held, so a path that would stall the door throws `BlockingInEventLoopError` at its first wait instead of freezing every session. The door also carries a watchdog on a thread of its own (`frontdoor/watchdog.ts`): the supervisor beats a shared counter every tick and the thread, whose event loop a blocked main loop cannot stall, ends the process when the counter has not moved for twice the longest a launcher waits on a door before calling it unresponsive, after recording the stall in the state file's `lastError`. It is started once the listeners are bound and stopped before any retirement begins, and it restarts its clock when it was itself descheduled for longer than its cadence, so a suspended or starved machine is not taken for a hung door. A stall always leaves a replacement behind, started while the old pid is still dying so it can bind the same sticky ports the moment they free (the session registry is on disk either way); the chain stops after a bounded run of consecutive replacements, because a stall that has recurred through that many doors in a row is the machine's, and the next launch's ensure is what starts the door after that. `doctor` and `frontdoor status` ask the listeners themselves whether they accept (`frontdoor/listenerReadings.ts`): a TLS handshake against the CA on the provider listener and a plain TCP accept on the CONNECT surface, each timed, because a live supervisor pid cannot tell a door that serves from one whose event loop is blocked, and the two listeners carry different sessions (provider sessions on one, every plain OAuth session on the other). The direct listener headroom forwards back to stays plain HTTP, because headroom is what connects to it; what keeps that leg harmless is that no credential travels on it. The hop parks the provider's credential headers in an in-memory custody record and sends headroom placeholders and a hop id, and the direct listener's admission step redeems a live id (for the same provider, with the generation's hop secret) for the real headers before routing. Admission is a pipeline stage in its own right, returning the headers a request is routed with, so restoring credentials is admission's job and no route knows about it. The direct pipeline runs no response observers of its own (its responses are consumed by the hop, not the client, and observing both would record every hopped request twice), so a choice a translator notes while serving behind the hop cannot reach the pipeline that records it through the daemon's relay; the record builder recognises it instead from two facts that pipeline sees itself, the model name the request's own body carried (which the hop reads out and hands back alongside the thread) and the model the response body was served under. The front door records the wire protocol it speaks (`FRONT_DOOR_PROTOCOL` in `src/frontdoor/state.ts`) in its state once it is serving. A launcher that finds a live door whose protocol is lower than its own (a door from before the field counts as 0) stops it, after its listener has authenticated as this tool's own, and spawns a replacement that rebinds the same sticky ports. That is safe because the capability registry is on disk and sessions already running keep reaching the addresses they froze at launch. A door of the same or a newer protocol is always joined, so an older launcher never downgrades a newer door. The rule for a bump: a door accepts every lower protocol's launches (protocol 1 accepts the former `x-claude-use-*` header names). diff --git a/src/frontdoor/assembly.ts b/src/frontdoor/assembly.ts index 0eea31d7..a12e3c95 100644 --- a/src/frontdoor/assembly.ts +++ b/src/frontdoor/assembly.ts @@ -74,8 +74,6 @@ export function createDoorPipelines(deps: DoorPipelineDeps): { readonly clientFa direct: { resolveRoute: deps.resolveRoute, responseObservers: [], - // The direct pipeline's responses are consumed by this door's own headroom hop, never by a client, so the noted model choice rides the head home as MODEL_CHOICE_HEADER (see `exposeModelChoiceOnHead`) instead of dying with the observers this pipeline deliberately runs none of. - exposeModelChoiceOnHead: true, admit: admitHop(deps.hopSecret, deps.custody), headroomDecision: deps.headroomDecision, now: deps.now, diff --git a/src/frontdoor/headroomHop.test.ts b/src/frontdoor/headroomHop.test.ts index d583d959..8c6d7d7b 100644 --- a/src/frontdoor/headroomHop.test.ts +++ b/src/frontdoor/headroomHop.test.ts @@ -21,7 +21,7 @@ import { createDoorPipelines } from "./assembly"; import { headroomSocketTarget } from "./status"; import { LOOPBACK_LEAF_NAMES, generateCa, mintLeaf, type CaMaterial, type LeafCert } from "./connect"; import { SEQUESTERED_CREDENTIAL, createCredentialCustody } from "./custody"; -import { AUTH_HEADER, HEADROOM_FLAG_HEADER, HOP_ID_HEADER, HOP_SECRET_HEADER, IDENTITY_HEADER, MODEL_CHOICE_HEADER, SESSION_HEADER } from "./route"; +import { AUTH_HEADER, HEADROOM_FLAG_HEADER, HOP_ID_HEADER, HOP_SECRET_HEADER, IDENTITY_HEADER, SESSION_HEADER } from "./route"; import type { ResponseObserver } from "./pipeline"; import { serveRouted } from "./pipeline"; import { createProviderRouteResolver } from "./providerRoute"; @@ -313,14 +313,6 @@ async function settle(): Promise { }); } -describe("the two pipelines the door assembles", () => { - it("exposes a noted model choice on the direct pipeline's head only, the private channel home through the hop", () => { - const pipelines = createDoorPipelines({ resolveRoute: async () => await Promise.resolve({ ok: false, status: 500, message: "unused" }), isLiveToken: () => false, headroomSocket: () => undefined, hopSecret: HOP_SECRET, custody: createCredentialCustody(() => randomUUID()), headroomDecision: followsLaunchFlag, responseObservers: [], now: () => 0, log: () => undefined }); - expect(pipelines.direct.exposeModelChoiceOnHead).toBe(true); - expect(pipelines.clientFacing.exposeModelChoiceOnHead).toBeUndefined(); - }); -}); - describe("the headroom hop", () => { it("sits before the translator: headroom receives the Anthropic-shaped request, and the translator serves what headroom forwards back through the direct listener", async () => { const headroom = await fakeHeadroom(); @@ -349,32 +341,13 @@ describe("the headroom hop", () => { } }); - it("notes a model choice the direct pipeline's head carries home, and strips the header before the client sees it", async () => { - const headroom = await fakeHeadroom({ responseHeaders: { [MODEL_CHOICE_HEADER]: JSON.stringify({ requested: "auto", providers: ["antigravity"] }) } }); - const choices: { requested: string; providers: readonly string[] }[] = []; - const door = await startDoor({ files: { [`${PROVIDERS_DIR}/codex.json`]: codexProvider }, responseObservers: [(event) => { if (event.modelChoice !== undefined) choices.push(event.modelChoice); return undefined; }] }); - try { - const response = await fetch(`${door.url}/providers/codex/v1/messages`, { - dispatcher: trusting, - method: "POST", - headers: { "content-type": "application/json", [IDENTITY_HEADER]: "work", [SESSION_HEADER]: "session-1", [HEADROOM_FLAG_HEADER]: "1", [AUTH_HEADER]: LAUNCH_TOKEN }, - body: MESSAGES_BODY, - }); - expect(response.status).toBe(HTTP_STATUS.ok); - // The choice the direct side exposed reached the client-facing pipeline's observer, which is where usage recording reads it. - expect(choices).toEqual([{ requested: "auto", providers: ["antigravity"] }]); - // And the internal header never reached the client. - expect(response.headers.get(MODEL_CHOICE_HEADER)).toBeNull(); - } finally { - await door.close(); - await headroom.close(); - } - }); - - it("drops a model-choice header that is not the shape this door writes, still stripping it from the client's head", async () => { - const headroom = await fakeHeadroom({ responseHeaders: { [MODEL_CHOICE_HEADER]: JSON.stringify({ requested: "auto", providers: "antigravity" }) } }); - const choices: unknown[] = []; - const door = await startDoor({ files: { [`${PROVIDERS_DIR}/codex.json`]: codexProvider }, responseObservers: [(event) => { if (event.modelChoice !== undefined) choices.push(event.modelChoice); return undefined; }] }); + it("carries the model name the request's own body carried to the client-facing observers, so a choice a translator made behind the hop can still be recognised", async () => { + const headroom = await fakeHeadroom(); + const requested: (string | undefined)[] = []; + const door = await startDoor({ files: { [`${PROVIDERS_DIR}/codex.json`]: codexProvider }, responseObservers: [(event) => { + requested.push(event.requestedModel); + return undefined; + }] }); try { const response = await fetch(`${door.url}/providers/codex/v1/messages`, { dispatcher: trusting, @@ -383,8 +356,7 @@ describe("the headroom hop", () => { body: MESSAGES_BODY, }); expect(response.status).toBe(HTTP_STATUS.ok); - expect(choices).toEqual([]); - expect(response.headers.get(MODEL_CHOICE_HEADER)).toBeNull(); + expect(requested).toEqual(["claude-sonnet-4-5"]); } finally { await door.close(); await headroom.close(); diff --git a/src/frontdoor/headroomHop.ts b/src/frontdoor/headroomHop.ts index e6596f55..009e6d46 100644 --- a/src/frontdoor/headroomHop.ts +++ b/src/frontdoor/headroomHop.ts @@ -7,7 +7,7 @@ import { HTTP_STATUS } from "../codex/http"; import type { HeadroomSocketTarget } from "../headroom/socket"; import { forwardableHeaders } from "./connect"; import type { CredentialCustody } from "./custody"; -import { HEADROOM_BASE_URL_HEADER, HOP_ID_HEADER, HOP_SECRET_HEADER, CLIENT_SESSION_HEADER, MODEL_CHOICE_HEADER, PROJECT_ID_HEADER, PROVIDER_HEADER, parseProviderPath, type ModelChoice, type RoutedRequest, type RoutedResponse } from "./route"; +import { HEADROOM_BASE_URL_HEADER, HOP_ID_HEADER, HOP_SECRET_HEADER, CLIENT_SESSION_HEADER, PROJECT_ID_HEADER, PROVIDER_HEADER, parseProviderPath, type RoutedRequest, type RoutedResponse } from "./route"; import { lineageOf, type Lineage } from "./lineage"; import { upstreamChunks } from "./server"; @@ -48,7 +48,13 @@ function responseHeaders(headers: Readonly): Record void): Promise { +/** What the hop learns from a request's own body as it reads it to forward: the conversation thread it belongs to, and the model name the request carried when it carried one. Both reach the client-facing pipeline before its response head is written, so its observers can use them. */ +export interface HopFacts { + readonly lineage: Lineage | undefined; + readonly requestedModel: string | undefined; +} + +export async function applyHeadroomHop(request: RoutedRequest, response: RoutedResponse, upstream: string | undefined, deps: HeadroomHopDeps, onFacts: (facts: HopFacts) => void): Promise { const target = deps.headroomSocket(); if (target === undefined) { await answerBadGateway(response, "agent-shim front door: the headroom daemon is restarting"); @@ -88,7 +94,7 @@ export async function applyHeadroomHop(request: RoutedRequest, response: RoutedR } headers[HOP_SECRET_HEADER] = deps.hopSecret; try { - await forwardThroughHeadroom(request, response, { socketPath, headers, log: deps.log, onThread }); + await forwardThroughHeadroom(request, response, { socketPath, headers, log: deps.log, onFacts }); } finally { // The hop is over (answered, failed, or abandoned by the client): its custody id stops redeeming, so a copy of it seen anywhere along the way is worthless from here on. if (hopId !== undefined) { @@ -200,31 +206,29 @@ export async function collectBody(body: Readable): Promise { }); } -/** - * Streams one request to headroom and its response back to the client, resolving once the response has ended, failed, or been abandoned. A hop the daemon resets before any response byte is replayed on a fresh connection (the request produced no output, so replaying it is what the client's own retry would do, minus the visible error); a reset after the response started cannot be replayed and ends the response. - */ -/** Reads the model choice the direct pipeline exposes on a response head, narrowed from whatever the relay actually carried: a value that is not the exact shape this door writes is not a choice. */ -function parseModelChoiceHeader(value: string | undefined): ModelChoice | undefined { - if (value === undefined) return undefined; - let parsed: unknown; +/** The model name a request body carries, when it carries a string one: the whole JSON is already parsed by the lineage reader beside this, and a body that is not JSON or names no model simply has no name to pair with the served one. */ +function requestedModelOf(body: Buffer): string | undefined { try { - parsed = JSON.parse(value); + const parsed: unknown = JSON.parse(body.toString("utf8")); + if (typeof parsed !== "object" || parsed === null || !("model" in parsed) || typeof parsed.model !== "string") return undefined; + return parsed.model; } catch { return undefined; } - if (typeof parsed !== "object" || parsed === null || !("requested" in parsed) || !("providers" in parsed) || typeof parsed.requested !== "string" || !Array.isArray(parsed.providers)) return undefined; - const providers: string[] = []; - for (const name of parsed.providers) { - if (typeof name !== "string") return undefined; - providers.push(name); - } - return { requested: parsed.requested, providers }; } +/** Reads a request body the hop is about to forward and returns what the client-facing pipeline learns from it: the thread it belongs to and the model name it asked for (the name a serving translator resolves to a concrete model, which is the fact the record builder pairs with the served model to recognise a choice). */ +function hopFactsOf(body: Buffer): HopFacts { + return { lineage: lineageOf(body), requestedModel: requestedModelOf(body) }; +} + +/** + * Streams one request to headroom and its response back to the client, resolving once the response has ended, failed, or been abandoned. A hop the daemon resets before any response byte is replayed on a fresh connection (the request produced no output, so replaying it is what the client's own retry would do, minus the visible error); a reset after the response started cannot be replayed and ends the response. + */ async function forwardThroughHeadroom( request: RoutedRequest, response: RoutedResponse, - target: { readonly socketPath: string; readonly headers: Readonly>; readonly log: (line: string) => void; readonly onThread: (lineage: Lineage | undefined) => void }, + target: { readonly socketPath: string; readonly headers: Readonly>; readonly log: (line: string) => void; readonly onFacts: (facts: HopFacts) => void }, ): Promise { const collected = await collectBody(request.body); if (collected.kind === "failed") { @@ -239,7 +243,7 @@ async function forwardThroughHeadroom( } const { body } = collected; // The thread this request belongs to, told from its own history, so a response can later be matched with the previous response of its own thread even while subagents and forks answer under the same session id. - target.onThread(lineageOf(body)); + target.onFacts(hopFactsOf(body)); for (let retriesLeft = HOP_RESET_RETRIES; ; retriesLeft -= 1) { const outcome = await hopOnce(request, response, target, body, retriesLeft > 0); if (outcome === "settled") { @@ -259,13 +263,7 @@ async function hopOnce( const { socketPath, headers } = target; return await new Promise<"settled" | "retry">((resolve) => { const onUpstreamResponse = (upstreamResponse: IncomingMessage): void => { - // The direct pipeline's model choice rides home on this header (see MODEL_CHOICE_HEADER): destructured out of the relayed head here, noted on this response before its head is written so the observers the head invokes record it, and left out of what the client receives. A value that does not parse as a choice is dropped rather than trusted: the daemon is the only writer this header has, but a corrupted or forged relay still cannot inject a choice. - const { [MODEL_CHOICE_HEADER]: choiceHeader, ...head } = responseHeaders(upstreamResponse.headers); - const choice = parseModelChoiceHeader(choiceHeader); - if (choice !== undefined) { - response.noteModelChoice?.(choice); - } - response.start(upstreamResponse.statusCode ?? HTTP_BAD_GATEWAY, head); + response.start(upstreamResponse.statusCode ?? HTTP_BAD_GATEWAY, responseHeaders(upstreamResponse.headers)); // Headers go out before the first body byte: a streaming (SSE) response must reach the client as its chunks arrive, not when it completes. response.flush(); const stream = async (): Promise => { diff --git a/src/frontdoor/pipeline.test.ts b/src/frontdoor/pipeline.test.ts index ecfc80a4..c18cb5c3 100644 --- a/src/frontdoor/pipeline.test.ts +++ b/src/frontdoor/pipeline.test.ts @@ -8,7 +8,7 @@ import { createCredentialCustody } from "./custody"; import { NO_ROUTING_SETTING } from "../headroom/routingSetting"; import { createHeadroomDecision } from "./headroomDecision"; import { createRoutedResponse, serveRouted, type PipelineDeps, type PipelineRequest, type ResponseBodyObserver, type ResponseObserver, type RouteResolution, type RoutedResponseEvent } from "./pipeline"; -import { HEADROOM_FLAG_HEADER, IDENTITY_HEADER, MODEL_CHOICE_HEADER, SESSION_HEADER, type FrontDoorRoute, type RoutedRequest, type RoutedResponse } from "./route"; +import { HEADROOM_FLAG_HEADER, IDENTITY_HEADER, SESSION_HEADER, type FrontDoorRoute, type RoutedRequest, type RoutedResponse } from "./route"; /** * A stand-in ServerResponse with the only behaviour `createRoutedResponse` reads: writeHead, write returning false on demand, drain and close events, and destruction. A new one per use keeps each test's socket semantics independent. @@ -100,7 +100,7 @@ function deps(resolution: RouteResolution, observers: readonly ResponseObserver[ describe("createRoutedResponse", () => { it("applies backpressure: a write that the socket refuses waits for its drain", async () => { const response = new FakeServerResponse(true); - const routed = createRoutedResponse(response as unknown as PipelineRequest["response"], { deps: deps({ ok: true, route: recordingRoute() }).deps, servedPath: "/v1/messages", session: { identity: "work", sessionId: "s", provider: undefined, headroom: false, routeByName: false, projectId: undefined }, route: "test", request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined } } }); + const routed = createRoutedResponse(response as unknown as PipelineRequest["response"], { deps: deps({ ok: true, route: recordingRoute() }).deps, servedPath: "/v1/messages", session: { identity: "work", sessionId: "s", provider: undefined, headroom: false, routeByName: false, projectId: undefined }, route: "test", request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined }, requestedModel: { value: undefined } } }); routed.start(HTTP_STATUS.ok, {}); let resolved = false; const write = routed.write("chunk").then(() => { @@ -116,7 +116,7 @@ describe("createRoutedResponse", () => { it("settles a backpressured write when the client goes away, so a route is never left waiting on a dead socket", async () => { const response = new FakeServerResponse(true); - const routed = createRoutedResponse(response as unknown as PipelineRequest["response"], { deps: deps({ ok: true, route: recordingRoute() }).deps, servedPath: "/v1/messages", session: { identity: "work", sessionId: "s", provider: undefined, headroom: false, routeByName: false, projectId: undefined }, route: "test", request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined } } }); + const routed = createRoutedResponse(response as unknown as PipelineRequest["response"], { deps: deps({ ok: true, route: recordingRoute() }).deps, servedPath: "/v1/messages", session: { identity: "work", sessionId: "s", provider: undefined, headroom: false, routeByName: false, projectId: undefined }, route: "test", request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined }, requestedModel: { value: undefined } } }); routed.start(HTTP_STATUS.ok, {}); let resolved = false; const write = routed.write("chunk").then(() => { @@ -338,7 +338,7 @@ function routedWith(response: FakeServerResponse, observers: readonly ResponseOb deps: d, session: { identity: "work", sessionId: "s", provider: undefined, headroom: false, routeByName: false, projectId: undefined }, route: "test", - request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined } }, + request: { method: "POST", path: "/v1/messages", receivedAt: 0, viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined }, requestedModel: { value: undefined } }, servedPath: "/v1/messages", }); return { routed, logs }; @@ -429,40 +429,6 @@ describe("a model choice reaching the response observers", () => { expect(choices).toEqual([{ requested: "auto", providers: ["z", "antigravity"] }]); }); - it("exposes a noted choice on the response head when the pipeline asks for it, as the direct pipeline does for the hop to carry home", async () => { - const notingRoute: FrontDoorRoute = { - name: "noting", - headroomEligible: false, - headroomUpstream: undefined, - serve: async (_request, response) => { - response.noteModelChoice?.({ requested: "auto", providers: ["antigravity"] }); - response.start(HTTP_STATUS.ok, {}); - await Promise.resolve(); - response.end(); - }, - }; - const { request, response } = pipelineRequest(); - await serveRouted(request, deps({ ok: true, route: notingRoute }, [], { exposeModelChoiceOnHead: true }).deps); - expect(response.writtenHead?.headers[MODEL_CHOICE_HEADER]).toBe(JSON.stringify({ requested: "auto", providers: ["antigravity"] })); - }); - - it("never puts the model-choice header on a client-facing head, which no client may see", async () => { - const notingRoute: FrontDoorRoute = { - name: "noting", - headroomEligible: false, - headroomUpstream: undefined, - serve: async (_request, response) => { - response.noteModelChoice?.({ requested: "auto", providers: ["antigravity"] }); - response.start(HTTP_STATUS.ok, {}); - await Promise.resolve(); - response.end(); - }, - }; - const { request, response } = pipelineRequest(); - await serveRouted(request, deps({ ok: true, route: notingRoute }).deps); - expect(response.writtenHead?.headers[MODEL_CHOICE_HEADER]).toBeUndefined(); - }); - it("carries a choice the route notes before its head, as the tier translator does", async () => { const choices: (RoutedResponseEvent["modelChoice"])[] = []; const notingRoute: FrontDoorRoute = { diff --git a/src/frontdoor/pipeline.ts b/src/frontdoor/pipeline.ts index 0fb4d456..8d325c91 100644 --- a/src/frontdoor/pipeline.ts +++ b/src/frontdoor/pipeline.ts @@ -4,7 +4,7 @@ import { HTTP_STATUS } from "../codex/http"; import type { HeadroomDecision } from "./headroomDecision"; import { applyHeadroomHop, headroomReady, type HeadroomHopDeps } from "./headroomHop"; import type { Lineage } from "./lineage"; -import { identifyRequest, MODEL_CHOICE_HEADER, type FrontDoorRoute, type ModelChoice, type RoutedRequest, type RoutedResponse, type SessionIdentity } from "./route"; +import { identifyRequest, type FrontDoorRoute, type ModelChoice, type RoutedRequest, type RoutedResponse, type SessionIdentity } from "./route"; /** What one routed response looked like as it started back to the client: the typed event response middleware observes. */ export interface RoutedResponseEvent { @@ -25,6 +25,8 @@ export interface RoutedResponseEvent { readonly lineage: Lineage | undefined; /** The model choice this response answers, when a routing layer chose among alternatives for the request's model name (a virtual model's or a tier alias's selection): the name the session asked for and the providers it could have landed on. Filled before the head is written, by the resolution or by the serving route, so middleware reading it at the head always sees it. */ readonly modelChoice: ModelChoice | undefined; + /** The model name the request's own body carried, when it crossed the headroom hop (whose reading of the body is the only client-facing look at it) and named a model. The record builder pairs it with the model the response body was served under to recognise a choice the serving translator made behind the hop, whose own noting cannot reach this pipeline through the daemon. */ + readonly requestedModel: string | undefined; readonly status: number; readonly headers: Readonly>; } @@ -71,10 +73,6 @@ export interface PipelineDeps { * The listener's admission step, run after identification and before anything is routed: it decides whether the request may be routed at all and with which headers. A client-facing listener admits a request carrying the per-launch capability of a live registered session, unchanged; the direct listener admits only what this process's own headroom hop sent back (the generation's hop secret plus a live custody id for the provider the path names) and swaps the hop's placeholder credentials for the real ones. A refused request is answered 401 without ever reaching a route, so a loopback process that never launched through agent-shim cannot spend a session's credentials or quota. */ readonly admit: (request: AdmissionRequest) => Admission; - /** - * Whether a noted model choice is exposed on the response head as `MODEL_CHOICE_HEADER`. Only the direct pipeline sets it: its responses are consumed by the door's own headroom hop rather than by a client, and the hop is the one place that can carry a choice a route noted while serving behind the hop back to the pipeline whose observers record it. A client-facing pipeline never sets it, so the header never reaches a client. - */ - readonly exposeModelChoiceOnHead?: boolean; } /** What the admission step sees of one request. */ @@ -136,6 +134,8 @@ interface ObservedRequest { readonly thread: { lineage: Lineage | undefined }; /** The model choice a routing layer made for this request, when it made one: filled by the resolution or by the serving route before the head is written, and read by the observers the head invokes. */ readonly modelChoice: { value: ModelChoice | undefined }; + /** The model name the request body carried, filled by the hop when it reads the body, and read by the observers the head invokes alongside the choice. */ + readonly requestedModel: { value: string | undefined }; } /** The request target's path without its query string: the query is the client's business, and middleware has no use for it. */ @@ -203,11 +203,9 @@ export function createRoutedResponse( return headersSent; }, start: (status, headers) => { - // The direct pipeline's private channel home (see `exposeModelChoiceOnHead`): the choice rides the response head because the direct pipeline has no observers of its own, and the hop strips it before any client sees the head. - const head = deps.exposeModelChoiceOnHead === true && request.modelChoice.value !== undefined ? { ...headers, [MODEL_CHOICE_HEADER]: JSON.stringify(request.modelChoice.value) } : headers; for (const observe of deps.responseObservers) { try { - const body = observe({ session, route, method: request.method, path: request.path, servedPath: context.servedPath, receivedAt: request.receivedAt, viaHeadroom: request.viaHeadroom, lineage: request.thread.lineage, modelChoice: request.modelChoice.value, status, headers }); + const body = observe({ session, route, method: request.method, path: request.path, servedPath: context.servedPath, receivedAt: request.receivedAt, viaHeadroom: request.viaHeadroom, lineage: request.thread.lineage, modelChoice: request.modelChoice.value, requestedModel: request.requestedModel.value, status, headers }); if (body !== undefined) { bodyObservers.push(body); } @@ -216,7 +214,7 @@ export function createRoutedResponse( deps.log(`front door: response observer threw for ${route}: ${describeThrown(error)}`); } } - response.writeHead(status, head); + response.writeHead(status, headers); headersSent = true; }, noteModelChoice: (choice) => { @@ -258,7 +256,7 @@ export function createRoutedResponse( * The ordered pipeline every routed session's request runs through: identify the session (splitting the launcher-injected headers out of what may leave the machine), then route, with the response middleware running at each response head. A route that cannot be resolved is answered as an Anthropic-shaped error, and the observers see that error like any other response, since quota and rate-limit state lives in exactly those. */ export async function serveRouted(request: PipelineRequest, deps: PipelineDeps): Promise { - const observed: ObservedRequest = { method: request.method, path: pathOf(request.url), receivedAt: deps.now(), viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined } }; + const observed: ObservedRequest = { method: request.method, path: pathOf(request.url), receivedAt: deps.now(), viaHeadroom: false, thread: { lineage: undefined }, modelChoice: { value: undefined }, requestedModel: { value: undefined } }; const identified = identifyRequest(request.headers); const admission = deps.admit({ url: request.url, headers: request.headers, forwardable: identified.forwardableHeaders }); if (!admission.ok) { @@ -298,8 +296,9 @@ export async function serveRouted(request: PipelineRequest, deps: PipelineDeps): try { if (viaHeadroom) { // The ordering the whole design turns on: headroom first, on the Anthropic-shaped request, then the route (a translator receives what headroom compressed). A route that opts out, or a listener that serves routes directly, falls through to serving. - await applyHeadroomHop(resolution.request ?? routed, response, resolution.route.headroomUpstream, deps.headroom, (lineage) => { - observed.thread.lineage = lineage; + await applyHeadroomHop(resolution.request ?? routed, response, resolution.route.headroomUpstream, deps.headroom, (facts) => { + observed.thread.lineage = facts.lineage; + observed.requestedModel.value = facts.requestedModel; }); } else { await resolution.route.serve(routed, response); diff --git a/src/frontdoor/route.ts b/src/frontdoor/route.ts index 2e7fda35..7f45a122 100644 --- a/src/frontdoor/route.ts +++ b/src/frontdoor/route.ts @@ -68,10 +68,6 @@ export const HOP_SECRET_HEADER = "x-agent-shim-hop"; */ export const HOP_ID_HEADER = "x-agent-shim-hop-id"; -/** - * Carries a routing layer's model choice from the direct listener's response back to the hop that forwarded the request there. The direct pipeline runs no response observers (its responses are consumed by the door's own hop, not by the client, and observing both would record every hopped request twice), so a choice a route notes there has no observer to read it; this header is the private channel it rides home on. The hop reads it, notes the choice on the client-facing response before that response's head is written, and strips it, so no client ever receives it. Only the direct pipeline writes it (see `exposeModelChoiceOnHead` on `PipelineDeps`). - */ -export const MODEL_CHOICE_HEADER = "x-agent-shim-model-choice"; /** What the identity step learned about one request from the launcher-injected headers. */ export interface SessionIdentity { diff --git a/src/usage/middleware.test.ts b/src/usage/middleware.test.ts index b09aeeed..016289a3 100644 --- a/src/usage/middleware.test.ts +++ b/src/usage/middleware.test.ts @@ -32,6 +32,7 @@ function event(overrides: Partial = {}): RoutedResponseEven viaHeadroom: false, lineage: undefined, modelChoice: undefined, + requestedModel: undefined, status: OK_STATUS, headers: SSE_HEADERS, ...overrides, @@ -533,4 +534,37 @@ describe("a model choice among alternatives", () => { expect(singleRecord(h).requestedModel).toBeUndefined(); }); + + it("reconstructs a choice behind the headroom hop from the request's own model name paired with the model the body was served under", () => { + const h = harness(); + const body = begin(h, event({ requestedModel: "auto" })); + body.chunk(bytes(sse("message_start", { type: "message_start", message: { id: "msg_1", model: "gemini-4-argon-high", content: [], usage: { input_tokens: 5, output_tokens: 1 } } }))); + finishBody(h, body); + h.flush(); + const record = singleRecord(h); + expect(record.requestedModel).toBe("auto"); + expect(record.model).toBe("gemini-4-argon-high"); + // One provider served it, so no candidate list. + expect(record).not.toHaveProperty("candidateProviders"); + }); + + it("records no reconstructed choice when the served model is the name the request carried, however the request travelled", () => { + const h = harness(); + const body = begin(h, event({ requestedModel: "claude-sonnet-4-5" })); + body.chunk(bytes(sse("message_start", { type: "message_start", message: { id: "msg_1", model: "claude-sonnet-4-5", content: [], usage: { input_tokens: 5, output_tokens: 1 } } }))); + finishBody(h, body); + h.flush(); + expect(singleRecord(h)).not.toHaveProperty("requestedModel"); + }); + + it("keeps an explicitly noted choice over a reconstructed one, and records its providers when there were several", () => { + const h = harness(); + const body = begin(h, event({ requestedModel: "auto", modelChoice: { requested: "auto", providers: ["z", "antigravity"] } })); + body.chunk(bytes(sse("message_start", { type: "message_start", message: { id: "msg_1", model: "glm-5.3", content: [], usage: { input_tokens: 5, output_tokens: 1 } } }))); + finishBody(h, body); + h.flush(); + const record = singleRecord(h); + expect(record.requestedModel).toBe("auto"); + expect(record.candidateProviders).toEqual(["z", "antigravity"]); + }); }); diff --git a/src/usage/middleware.ts b/src/usage/middleware.ts index 316797e2..319ab319 100644 --- a/src/usage/middleware.ts +++ b/src/usage/middleware.ts @@ -1,6 +1,7 @@ import * as zlib from "node:zlib"; import type { ResponseBodyObserver, ResponseObserver, RoutedResponseEvent } from "../frontdoor/pipeline"; +import type { ModelChoice } from "../frontdoor/route"; import { parseProviderPath } from "../frontdoor/route"; import { classifyLimit, rateLimitHeadersOf } from "./rateLimit"; import { createJsonScanner, createSseScanner, type BodyScanner, type ScanResult } from "./scan"; @@ -74,9 +75,18 @@ function destinationOf(path: string, sessionProvider: string | undefined): { rea } /** Builds one finished request's record from its head, what the body scan found, and how it ended. */ +/** The model choice a response answers, from the event's own noting when one exists and otherwise from the request's carried name paired with the model the response body was served under. */ +function modelChoiceOf(event: RoutedResponseEvent, servedModel: string | undefined, provider: string): ModelChoice | undefined { + if (event.modelChoice !== undefined) return event.modelChoice; + if (event.requestedModel === undefined || servedModel === undefined || event.requestedModel === servedModel) return undefined; + return { requested: event.requestedModel, providers: [provider] }; +} + function buildRecord(event: RoutedResponseEvent, context: { readonly headAt: number; readonly endedAt: number; readonly outcome: "completed" | "aborted"; readonly scan: ScanResult | undefined }): UsageRecord { const { scan } = context; const { provider, endpoint } = destinationOf(event.servedPath, event.session.provider); + // The choice this response answers: the one a routing layer noted outright, or the one reconstructed here when the request crossed the headroom hop and the serving translator behind it resolved the name the body carried into a different model (the hop carries the request's own name home because the daemon's relay never brings the translator's noting back, and the body scan names what served it). A single-provider reconstruction names the provider the record itself already does. + const choice = modelChoiceOf(event, scan?.model, provider); const rateLimitHeaders = rateLimitHeadersOf(event.headers); const limit = classifyLimit({ status: event.status, headers: rateLimitHeaders ?? {}, error: scan?.error, nowMs: context.endedAt }); const requestId = headerValue(event.headers, "request-id"); @@ -93,8 +103,8 @@ function buildRecord(event: RoutedResponseEvent, context: { readonly headAt: num method: event.method, endpoint, ...(scan?.model === undefined ? {} : { model: scan.model }), - ...(event.modelChoice === undefined ? {} : { requestedModel: event.modelChoice.requested }), - ...(event.modelChoice !== undefined && event.modelChoice.providers.length > 1 ? { candidateProviders: [...event.modelChoice.providers] } : {}), + ...(choice === undefined ? {} : { requestedModel: choice.requested }), + ...(choice !== undefined && choice.providers.length > 1 ? { candidateProviders: [...choice.providers] } : {}), status: event.status, latencyMs: Math.max(0, Math.round(context.headAt - event.receivedAt)), durationMs: Math.max(0, Math.round(context.endedAt - event.receivedAt)),