feat(ai): reuse the prompt cache on loopback Responses relays by default

[Heyaaz]

Problem

Custom openai-responses relays on loopback (an opencodex-style proxy on :10100, a codex router on :9091, a local gateway) never send session_id / x-client-request-id, because shouldSendOpenAIResponsesSessionHeaders() requires an explicit compat.supportsResponsesSessionAffinity opt-in. The ChatGPT codex backend only reuses a warmed prompt prefix for requests that carry a session identity, so those relays report cached_tokens: 0 on every append-only turn and replay the entire context.

Measured through a local relay's request log, same account and same model:

request shape	prefix reuse
no session header	0–50%, most turns fully cold
session_id sent	4/4 pairs reused; 114,944 / 115,112 tokens cached at ~110k

Live traffic before the fix on that relay path: main-thread prompts over 100k reused their prefix 4–8% of the time, and each turn re-uploaded 100k–300k tokens, draining a weekly subscription quota in hours (subagent prompts on the same relay reused 94–98% because they already carried the identity).

Change

Loopback hostnames (127.0.0.0/8, localhost, ::1) now default the affinity headers on for openai-responses, in both the custom-relay branch and the official openai provider pointed at a local gateway. The check is a URL-hostname match, never DNS resolution, so a DNS alias pointed at loopback stays opt-in.

Unchanged elsewhere: remote relays stay opt-in, known non-OpenAI provider IDs stay excluded, cacheRetention: "none" still suppresses the headers, and an explicit compat.supportsResponsesSessionAffinity: false opts a loopback relay back out.

Verification

• bun test packages/ai/test/openai-responses-cache-affinity.test.ts — 42 pass, including new cases for loopback spellings, non-loopback opt-in, explicit opt-out, retention gate, known-provider exclusion, and the official provider on a loopback gateway.
• bun test packages/ai/test/openai-completions-session-headers.test.ts packages/ai/test/openai-completions-compat.test.ts packages/coding-agent/test/models-config-send-session-headers.test.ts packages/coding-agent/test/model-registry.test.ts — pass (422 across the affected files).
• bun --cwd=packages/ai run check.
• End-to-end against a live local relay: turn 2/3 went from cacheRead: 0 to 12,800 / 12,928 cached tokens with only 118–667 fresh input tokens.

Note: bun run check:ts currently trips a check:sdk-closure flake in packages/coding-agent/test/sdk-host-wiring.test.ts under parallel load; it passes standalone on this branch and reproduces independently of these changes.

[probepark]

Review (head c759a63, gajae-reviewer on behalf of probepark)

Blocking: None found in the code review.
ocr: blocking 0 / nit 1

CI: Green on this exact head. The approval gate returned ALLOW (exit 0); no pending, failed, or missing local checks.

• bun test packages/ai/test/openai-responses-cache-affinity.test.ts: 42 pass, 0 fail. CI finished at 2026-10-14T06:12:39Z: https://github.com/Yeachan-Heo/gajae-code/actions/runs/38519463894/job/115709713645
• check:@gajae-code/ai: success at 2026-10-14T06:12:55Z: https://github.com/Yeachan-Heo/gajae-code/actions/runs/38519463894/job/115709713627
• Observed at 2026-10-14T06:19:34Z. The PR remained OPEN at the reviewed head. No local builds or tests were run in this pod.

Scope: +126 / -9, 5 files — Responses provider, affinity regression tests, model documentation, AI README, and changelog fragment. OCR selected one source file (+28 / -7).

Conventions: Release fragment present; released changelog sections unchanged. No generated files or labels in this diff.

Notable:

• packages/ai/src/providers/openai-responses.ts:180-190 — HTTP(S) URL parsing delegates normalized host validation to isLoopbackHostname. IPv4 loopback literals, localhost, and bracketed IPv6 are handled without accepting hostname-prefix impostors.
• packages/ai/src/providers/openai-responses.ts:201-223 — both provider branches preserve retention suppression and explicit opt-out for relays. Known non-target providers remain excluded. Canonical OpenAI behavior is unchanged.
• packages/ai/src/providers/openai-responses.ts:683-687 — session identity reaches the default headers before request transforms. Caller-set defaults and transform precedence remain unchanged.

Note: docs/models.md:1037 says the base URL “resolves to loopback.” This implementation checks the URL hostname; it does not resolve DNS aliases. Use “has a loopback hostname” to make that boundary explicit.

Spec: The PR's Change section matches the loopback default-on and remote opt-in branches. The new tests check externally visible request headers, retention suppression, and provider exclusion; existing expectations were not rewritten.
Checked and clean: Origin validation, provider allowlisting, cache-retention gates, opt-out, header precedence, release-note placement, and exact-head CI coverage.
Not independently assessed: The live relay cache-hit measurements in the PR body. CI proves request shaping, not backend cache reuse.

Verdict: gajae.pr-review-verdict.v1 merge-approved sha256:c02a24705cd4904d935637c41edb2f194e747ea7d4f0ae2b2a1235d3dae22885 reviewer:human reviewer-id:probepark evidence:exact-head-ci-green;affinity-boundaries-checked;42-regressions-pass

PR body verdict line count=0, not updated. Body verdict line is owned by author; not edited. Suggested verdict line: gajae.pr-review-verdict.v1 merge-approved sha256:c02a24705cd4904d935637c41edb2f194e747ea7d4f0ae2b2a1235d3dae22885 reviewer:human reviewer-id:probepark evidence:exact-head-ci-green;affinity-boundaries-checked;42-regressions-pass

[probepark]

Review (head 0ca8c03, gajae-reviewer on behalf of probepark)

Blocking: No blocking findings in the code review.
ocr: blocking 0 / nit 0

CI: Green on this exact head. gajae-approve-gate.py 6744 --head 0ca8c03e53b3d83d7a63077f11ade4db36487073 returned ALLOW, exit 0. No pending, failed, or missing local checks.

• bun test packages/ai/test/openai-responses-cache-affinity.test.ts: CI success, completed 2026-10-14T06:45:47Z. https://github.com/Yeachan-Heo/gajae-code/actions/runs/38522419809/job/115719979122
• check:@gajae-code/ai: CI success, completed 2026-10-14T06:46:02Z. https://github.com/Yeachan-Heo/gajae-code/actions/runs/38522419809/job/115719979140
• Both jobs report this head SHA. Evidence observed after completion; no local builds or tests ran in this pod.

Scope: +128 / -9, 5 files — Responses provider, affinity regression tests, documentation, and release fragment. OCR selected one source file (+30 / -7).

Conventions: Release fragment present. No released changelog edits, generated files, or labels.

Notable:

• packages/ai/src/providers/openai-responses.ts:181-189 — HTTP(S) URL parsing delegates hostname validation to isLoopbackHostname. Its strict IPv4 validation rejects prefix impostors; localhost and bracketed IPv6 loopback are supported. No DNS alias resolution is performed.
• packages/ai/src/providers/openai-responses.ts:204-216 — loopback defaults preserve explicit relay opt-out, retention suppression, and known-provider exclusion. Canonical OpenAI behavior remains unchanged.
• packages/ai/src/providers/openai-responses.ts:648-672 — the resolved endpoint controls affinity selection. Session identity reaches default headers before request transforms; caller-set headers retain precedence.

Spec: The PR's Change section matches the implemented loopback default and remote opt-in branches. Added request-boundary regressions cover host variants, retention, opt-out, and provider exclusion. Existing expectations are not rewritten. Documentation now says “has a loopback hostname,” matching the implementation.
Checked and clean: Origin validation, provider filtering, header precedence, retention gates, error fallback, regression coverage, release-note placement, and exact-head CI.
Not independently assessed: Live backend cache-hit measurements in the PR body. Request-shaping regressions do not prove backend cache reuse.

Verdict: gajae.pr-review-verdict.v1 merge-approved sha256:a8d658e76875674f6e2eedef4a36228017d25fb328102b0b134a50047b4d3f57 reviewer:human reviewer-id:probepark evidence:exact-head-ci-green;affinity-boundaries-checked;request-regressions-pass

PR body verdict line count=0, not updated. Body verdict line is owned by author; not edited. Suggested verdict line: gajae.pr-review-verdict.v1 merge-approved sha256:a8d658e76875674f6e2eedef4a36228017d25fb328102b0b134a50047b4d3f57 reviewer:human reviewer-id:probepark evidence:exact-head-ci-green;affinity-boundaries-checked;request-regressions-pass

[snowykr]

Verdict

APPROVED

Summary

This PR enables Responses session-affinity headers by default for eligible loopback relays while preserving provider, retention, and opt-out safeguards. No merge-blocking defect was found. One conditional P3 request-ID contract concern is non-blocking.

Findings / Required Changes

1. [P3] Keep X-Client-Request-Id request-scoped — packages/ai/src/providers/openai-responses.ts:215 (non-blocking)
   • Compared with base, this branch now enables affinity for unrecognized-provider loopback URLs without an explicit opt-in. Existing request construction assigns the same normalized, stable session ID to both session_id and x-client-request-id (packages/ai/src/providers/openai-responses.ts:668-670). If a loopback relay forwards that request-ID header upstream, successive turns reuse the ID; OpenAI documents request IDs as unique per request, and non-ASCII values can be rejected with HTTP 400. This is conditional: the repository auth gateway does not forward x-client-request-id, and other relays may consume or rewrite it, so it is not merge-blocking.
   • Keep stable affinity in session_id/prompt_cache_key and use a per-request ASCII x-client-request-id, or limit stable request-ID use to an explicit relay contract.

CI / Verification

The exact reviewed head 0ca8c03e53b3d83d7a63077f11ade4db36487073 passed CI run 38522419809, including the changed affinity test, @gajae-code/ai check, and affected-path and GJC state-gate aggregates. Public surfaces validation also passed. A later run for metadata-only PR edits was skipped by workflow conditions, not due to a product failure. No local tests were run.

Axis Coverage

Axis	Verdict	Coverage
A1 — Intent / Policy / Contract	APPROVED	Loopback, remote opt-in, provider, retention, and opt-out behavior match the documented intent; Finding 1 is conditional and non-blocking.
A2 — Architecture / Correctness / Failure	APPROVED	Request construction, retry/session behavior, caller overrides, and downstream consumers were traced; no actionable defect found.
A3 — Security / Privacy / Trust	APPROVED	The change does not redirect credentials or grant new authority; the loopback boundary and existing exclusions remain in place.
A4 — Verification / Tests / CI	APPROVED	New tests cover loopback defaults and key exclusions; the changed test and relevant exact-head CI aggregates passed.
A5 — Context / Compatibility / Platform	APPROVED	Callers, config validation, docs, packaging, and shared loopback classification were checked; no material reuse issue found.

Limitations

External branch-protection required-status configuration could not be independently verified because the GitHub API returned 401. The exact-head CI checks and aggregates listed above completed successfully.

[probepark]

Merged to dev: approved at 0ca8c03, checks green, no open concerns.