[bman654]

When clodex opens a new ChatGPT connection — for a subagent, a Claude Code background-agent summary, or to resume a conversation whose connection was closed — it sends the conversation again, and some ChatGPT models mostly failed to reuse the copy of that text ChatGPT had just processed. This change makes that reuse much more reliable, so repeated conversation history is far more often served from ChatGPT's cache instead of being processed again.

Fixes #324. Thanks to @0xdhx for the report, the root cause and the fix sketch.

What was wrong

ChatGPT's Codex backend derives prompt-cache affinity from the session-id header on the Responses WebSocket handshake. Native Codex sends it (openai/codex: codex-rs/codex-api/src/requests/headers.rs build_session_headers; codex-rs/core/src/client.rs responses_session_id, commented "ChatGPT derives cache affinity from the Responses session-id header"); a root Codex session sends its prompt cache key there. clodex sent prompt_cache_key in the body but never the header, and the body field alone did not keep fresh sockets on a warm cache: on the Responses-Lite models we sampled, requests that open a new socket and send full context — a subagent's first turn, an agent-summary fork, a conversation resent after its socket expired or was replaced — mostly missed the cache; same-socket continuation (previous_response_id) was unaffected.

Reachability: every ChatGPT/Codex OAuth Responses request goes through createResponsesWebSocketFetch (there is no OAuth HTTP path and no WebSocket-to-HTTP fallback), in proxy mode, endpoint mode and the server alike.

The change

createResponsesWebSocketFetch now sets session-id to the request's prompt_cache_key on the handshake, unless the caller already supplied a session-id. One value per socket is always correct: prompt_cache_key is part of the socket partition key, replacement sockets reuse the same headers, and a request with no key gets no header and never reuses a socket. The value is the existing session-derived key, already sent in the body, so nothing new is disclosed.

Left out deliberately: Codex's thread-id and x-codex-routing-hint (not measured; session-id alone reproduced the measured cache improvement), and per-agent session-id values (see below).

Tests

tests/responses-websocket-session-header.test.ts — only the ws socket is faked:

• the header equals the prompt_cache_key that translateRequest derives from the Claude session, driven through createLanguageModel and the real WebSocket fetch;
• distinct sessions get distinct values; a subagent's fresh socket carries the parent session's value;
• a transport-retry replacement socket carries it;
• no header when prompt_cache_key is missing, empty or not a string; a caller-supplied Session-Id is kept and not duplicated.

The fake socket copies its headers at construction, as ws does when it builds the upgrade request, so a header added after the socket exists would not count.

Mutations, full-file runs (8 tests): deleting the call reds 4; moving it after socket creation reds 3; sending a constant value reds 3; dropping the caller guard, the string check, or the empty check each reds 1.

pnpm typecheck && pnpm test && pnpm build on Node v24.14.1 with an isolated CLODEX_HOME and a dead HTTPS_PROXY: 3391 passed / 20 skipped, against 3383 / 20 on main.

Live evidence (our ChatGPT account, 2026-10-13)

Built product, main vs this branch. Each build ran as its own endpoint server; the same script sent a ~28k-token request through alias sol (gpt-6.1-sol), then resent it 5 times, each under a new Claude agent id so it opened a new socket (all 12 head decisions per server were new_partition_head); two rounds per build.

build	fresh-socket resends served from cache	cached tokens per hit
main 2065b12	0 / 10	—
this branch	10 / 10	28,160 of ~28,305 (99.5%)

Header-only A/B through main's real transport (header injected via request headers; one warm-up then repeats per arm, arms interleaved, each repeat on a new socket):

arm	Responses-Lite models (gpt-6.1-sol, gpt-daybreak-blue-latest)
no header	3 / 22 hits
session-id = prompt_cache_key	21 / 22 hits, ~99% cached

On gpt-6.1-sol, a stable value different from the key also hit 16/16, while a new random value per request hit 1/8 — stability is what matters, and the key is the natural stable value. The non-Lite gpt-5.5 already hit 5/6 without the header (and 3/6 with a random value), so the effect is concentrated on Responses-Lite models. An 8-way concurrent burst sharing one session-id hit 15/16 warm with no overload errors (1 generic server_error in 33 attempts vs 0 in 32 without the header).

Smoke test from this build: clodex claude -- --model luna -p … and --model haiku -p … both answered.

How much it helps depends on how you use subagents

The reporter measured 34% of prompt tokens uncached on their traffic, mostly agent-summary forks. On our own 30-day ledger it was 5.7%, with forks a negligible share; requests that opened a new socket held 18.1% of the uncached tokens. After the change, measured by @0xdhx on a background-agent-heavy session running clodex 2.18.10 with the equivalent one-line patch (same version before and after; all traffic on gpt-6.1-sol over the ChatGPT route), 24 hours either side of the patch: uncached share of prompt tokens fell from 44.1% (1,133 requests, 139.9M prompt tokens) to 13.3% (1,207 requests, 158.8M). See their comment below for method and a possible error-rate signal still being watched.

On our own traffic the fix had not been deployed, so how much it recovers there was not measured. The more a workload resends full context on new connections — heavy background-agent use is the reporter's case — the more it stands to gain; for ours the recoverable share is around a point at most. Our tests and A/B establish the mechanism; the before/after above is the size measurement for a fork-heavy workload.

What we could not verify

• Whether one session-id shared by a large fan-out (sessions here reach 200+ agents) concentrates load in a way that raises overload errors. An 8-way burst showed nothing. In @0xdhx's session, failed translations rose from 2/1,139 to 20/1,235 in the day after patching. At least part of that coincides with an upstream spike: our header-less account failed 14/414 (3.4%) in the 15:00Z hour of the same day. The rest is too small to attribute, and they are comparing per-agent against session-wide values. Native Codex uses the agent's own session id for non-root agents. We kept the session-wide key because it is the existing cache key, siblings share the session's prompt prefix, and nothing measured favors per-agent values — but no one has compared shared and per-agent header values under heavy fan-out either. Overload and server-error rates are visible in --ws-diagnostics if this needs revisiting.
• How ChatGPT uses session-id beyond cache routing — no public contract exists; the evidence is Codex's source comment and the measurements above. Affinity raises the chance of a cache hit; it does not guarantee one.

Rollback is reverting the one call in createResponsesWebSocketFetch; nothing is persisted.

[0xdhx]

Thanks for picking this up, and for the much more thorough measurement than mine. I read the diff and the guards look right to me. Adding production data in case it helps with the open question in "What we could not verify".

Since the issue was filed I have run an installed 2.18.10 with the equivalent one-line patch: session-id set to the request's prompt_cache_key, the same session-wide value this PR uses. I patched it at 2026-10-13 07:35Z. Below are the 24 hours before and after, read from ~/.clodex/logs/inference-requests.jsonl and limited to the one Claude session that spans both windows. That session is heavy on background agents, and all of its traffic goes through gpt-6.1-sol on the ChatGPT route.

Window (same session)	Requests with usage	Prompt tokens	Uncached share
24 h before the patch	1,133	139.9M	44.1%
24 h after the patch	1,207	158.8M	13.3%

So the size you couldn't measure after the change is large on a fork-heavy workload, as the PR predicts.

On fan-out sharing one session-id, one signal worth reporting but not yet a finding. In the same session, translation_failed went from 2 of 1,139 started translations before the patch to 20 of 1,235 after. Over 10-10 to 10-12, the daily ChatGPT-route failure rate was 0.2 to 0.4%. The 20 break down as:

• 12 websocket_transport_error, each a retryable 500 on attempt 1. The output idle before the failure varies widely, from 0 to 197 s, so they are not one consistent stall shape.
• 7 client-side aborts (lastPartType: abort). At least 4 of them came after exactly ~120 s of output idle, which looks like an idle timeout firing rather than an upstream error.
• 1 UpstreamError with no code.

Reasons I can't attribute it to the header:

• It's one day of data from one session, so a bad upstream day is an equally good explanation.
• The failures don't follow load. The busiest hour after the patch (400 requests) had 2 failures, while 14:00 to 15:59Z (126 requests) had 7.

I'll keep the patch in place and post again if the failure rate stays above baseline for a few more days. If you would rather I compare a per-agent value against the session-wide one, I can run that here.

[bman654]

Thank you. The before/after on a fork-heavy session is exactly the measurement we were missing, and I've added it to the description with credit.

On the failure signal, here's a control from our side. Our server doesn't send session-id (it runs a release from before this fix), on a different ChatGPT account, over the same day:

2026-10-13, after 07:35Z	Your session (header)	Ours (no header)
Failed / started translations	20 / 1,235 (1.6%)	44 / 11,904 (0.37%)
15:00Z hour	(your 14:00–15:59Z: 7 / 126)	14 / 414 (3.4%)

So at least the 14–16Z cluster looks upstream: we spiked in the same hour without the header. Outside that window you're still about 1.2% against a 0.2–0.4% baseline, which is roughly ten more failures than expected. That's too few to attribute in either direction, as you said.

We'd like to take you up on the per-agent comparison. Codex itself sends the agent's own session id for non-root agents. A per-agent value would still cover the agent-summary forks, since they reuse their parent's agent id in our logs, but it would give up some cache sharing between sibling subagents' first turns. If a few days show a failure-rate difference between the two, we'll switch to per-agent in a follow-up.

We're merging this now, since the cache win is large and the guards are unchanged by that question.

[0xdhx]

Following up on the failure-rate question, as promised. Short version: the transport-error excess went back to baseline, while client aborts stayed elevated. Those aborts are concentrated in two very long sessions, so I can't attribute them to the header.

Same source as before (~/.clodex/logs/inference-requests.jsonl, provider == openai-oauth), but this time every Claude session on this install rather than the one session from my earlier comment. This install still runs 2.18.10 with the one-line patch (session-id = prompt_cache_key, applied 2026-10-13 07:35Z).

Day (UTC)	Started	Failed	Rate	Error	Abort	Uncached share
10-10	1,870	7	0.37%	7	0	38.9%
10-11	3,329	9	0.27%	6	3	40.7%
10-12	934	2	0.21%	2	0	38.1%
10-13 (header from 07:35; 0 / 403 before)	4,087	38	0.92%	25	13	13.2%
10-14	7,507	49	0.65%	30	19	3.1%
10-15	7,550	44	0.58%	17	27	2.7%
10-16 (to 05:39Z)	1,216	6	0.49%	5	1	2.9%

"Error" means lastPartType: error (UpstreamError, including websocket_transport_error, plus a few AI_APICallError). "Abort" means lastPartType: abort. The uncached share is inputTokens / (inputTokens + cacheReadInputTokens) over message_delta usage.

Errors returned to baseline. Error-type failures per day were 0.37 / 0.18 / 0.21% before the header, then 0.61 / 0.39 / 0.22 / 0.41% after. websocket_transport_error went 22 → 26 → 8 → 2. That fits your control: 10-13 and 10-14 look like a bad upstream stretch, not the header.

Aborts stayed elevated. The baseline had 3 aborts in three days, all at about 600 s. Since the header there have been 60:

• 22 at about 120 s with translatedBytes: 0. That is clodex's own upstream idle timeout (DEFAULT_UPSTREAM_IDLE_TIMEOUT_MS = 12e4) firing on a request where upstream sent nothing for two minutes.
• 26 at about 600 s, the same shape as the baseline aborts.
• 12 in between.

51 of the 60 come from two long-running sessions: 25 aborts over 7,706 requests in one, and 26 over 6,115 in the other. Every other session combined had 9 aborts over 6,136 requests (0.15%). The worst hour, 10-15 21Z, was 11 failures in a single session, 7 of them zero-output 120 s aborts.

What I can't separate. Volume roughly tripled after 10-13 (3–14 active sessions per day, against 3–5 before). There is no no-header control on this install for the same period, and the abort counts are small and clustered by session. So whether a single session-wide affinity key across thousands of requests contributes to those first-byte stalls is still open.

Running the per-agent comparison now. As of 2026-10-16 06:29Z this install sends session-id = prompt_cache_key + -<x-claude-code-agent-id>. Every GPT request here is a subagent request with an agent id, so every request gets a per-agent key. The two abort-heavy sessions above ran 11 and 26 distinct GPT agents under one shared key. To keep the comparison on one variable, I'm holding off on 2.21.x until it's done. I'll post the same table for the per-agent days, against 10-13..16 as the session-wide period, by 2026-10-21.

The cache win held across all sessions: the uncached share went from about 39–41% to about 3%.

[bman654]

Thanks for following up as promised, and for running the per-agent comparison.

Here's the same view from our install, since you had no no-header control on yours. Our server moved from a release without the header to 2.21.0 (session-wide session-id = prompt_cache_key) at 2026-10-14 03:01Z, so we have a before and after on one install and one ChatGPT account. Same source and filter as yours (~/.clodex/logs/inference-requests.jsonl, provider == openai-oauth), with data through 2026-10-16 15:00Z.

	Before (10-01 → 10-14 03:01Z)	After (→ 10-16 15:00Z)
Started translations	106,858	13,427
120 s idle aborts ("no data received from provider for 120s")	178 (1.7 per 1k)	22 (1.6 per 1k)
websocket_transport_error failures	107 (1.0 per 1k)	12 (0.9 per 1k)
~600 s aborts	7	0
Uncached share, all traffic	5.9%	5.9%

• The 120 s idle aborts aren't new here. We had them at about the same rate with no header, including 44 on 10-09 and 70 on 10-12. Both periods also include stretches where our own network was failing. On 10-16, 12 of the 21 idle aborts sit right next to TLS certificate has expired errors on the connection to chatgpt.com, which is a problem local to us. So our rates mix upstream and local causes on both sides of the cutover.
• Our heaviest shared-key session didn't stall. One session ran on both sides of the cutover. After it, 145 agents sent 3,846 requests under one session-id with no idle aborts. 5 failed (0.13%), all mid-stream websocket_transport_error, and four of those came within six minutes on 10-14. Before the cutover the same session failed 37 of 19,597 (0.19%). That's a wider fan-out per key than your two abort-heavy sessions (11 and 26 agents), though over fewer requests.
• Fresh connections cache much better, but the overall number doesn't move for us. From --ws-diagnostics, requests that opened a new connection and sent full context (new_partition_head) on the sol and daybreak models went from 29% to 54% getting any cache hit. Their uncached share went from 95.7% to 55.4%. In the session above, hits went from 31% to 87%. Those requests are only about 1% of our input tokens, so our overall share stays at 5.9%, which is what we expected for our workload.

Caveats: our after window is 2.5 days at lower volume (about 5.4k requests a day, against 8.2k before), on a different account. It can't rule out an effect specific to your traffic, so your per-agent run is still the real test.

We look forward to your next update.

[0xdhx]

@bman654, here's the per-agent comparison I promised. Short version: after the switch, aborts and errors fell back to the no-header baseline, and the overall cache share didn't move. Fresh connections cached noticeably worse. I can't credit the drop to the per-agent key, though, because over the same days volume fell about sixfold and the two sessions behind most of the earlier aborts ended or shrank.

This is the same install, the same clodex 2.18.10 build and the same definitions as my 10-16 comment (~/.clodex/logs/inference-requests.jsonl, provider == openai-oauth, failures are translation_failed split by lastPartType, uncached share over message_delta usage). Per-agent mode (session-id = prompt_cache_key + -<x-claude-code-agent-id>) went live at 2026-10-16 06:29:41Z. Data runs to 2026-10-21 00:42Z.

Period	Started	Failed	Error	Abort	Uncached share
No header (10-10 → 10-13 07:35Z)	6,536	18 (0.28%)	15 (0.23%)	3 (0.05%)	40.8%
Session-wide (→ 10-16 06:29Z)	20,143	137 (0.68%)	77 (0.38%)	60 (0.30%)	4.0%
Per-agent (→ 10-21 00:42Z)	5,307	7 (0.13%)	5 (0.09%)	2 (0.04%)	3.5%

Day (UTC)	Started	Failed	Error	Abort	Uncached share
10-16 (from 06:29Z)	2,182	3	2	1	3.4%
10-17	1,113	0	0	0	4.3%
10-18	618	3	2	1	7.4%
10-19	191	1	1	0	6.4%
10-20	548	0	0	0	2.9%
10-21 (to 00:42Z)	655	0	0	0	1.1%

• Aborts. At the session-wide rate, 5,307 requests would have produced about 16 aborts. There were 2, both clodex's 120 s idle timeout on gpt-6.1-sol. One had zero output (10-18 00:18Z), the other came after 18.6 KB of output (10-16 13:11Z). There were no ~600 s aborts.
• Errors. 3 websocket_transport_error and 2 UpstreamError, against 58 and 12 in the session-wide period.
• Cache. The overall share stayed flat (4.0% → 3.5%). The cost shows up on fresh connections. From --ws-diagnostics, requests that opened a new connection and sent full context (new_partition_head) went from 78% to 30% getting any cache hit, and their uncached share went from 14.5% to 35.3%. That is what you'd expect: a new agent no longer lands where its siblings' shared prefix is already cached. It's the mirror of your 29% → 54% on the way in. Those requests are about 1% of input tokens here (5.9M of 510M), so the net cost is roughly 0.2 points of uncached share.
• Per-agent stayed live the whole window. Across five proxy processes, every logged start reported the per-agent patch in place, and every one of the 5,327 ChatGPT requests in the diagnostics carried an agent id. None fell back to the session-wide key.

What I can't separate.

• Volume dropped from about 6,800 started translations a day to about 1,100.
• The two sessions that produced 51 of the 60 session-wide aborts changed. One ended before the switch. The other kept going, with 1 abort over 1,130 requests across 9 GPT agents, against 26 over 6,286 requests across 27 agents before. At its old rate it would have had about 4.7, so 1 is suggestive (p ≈ 0.05), but it was also running a narrower fan-out.
• 10-13 to 10-15 looks like a bad upstream stretch (your control and our error mix both point that way), and it fell entirely inside the session-wide period.

So the per-agent key didn't hurt anything measurable except fresh-connection caching, but my data can't show it helped. Your install ran session-wide 2.21.0 over the same 10-16 → 10-20 window, which makes it the control. If your 120 s idle aborts held near 1.6 per 1k there, the drop here points at the key. If yours fell too, it was upstream. Could you share those numbers?

On this evidence I wouldn't make per-agent the default for reliability alone. I'm moving this install to 2.21.x with its built-in session-wide header and dropping my local patch.

[bman654]

Thanks for running this. Our side points to the same conclusion as yours.

Our raw idle-abort rate fell too, but that drop came from our own network. Since 10-14 03:01Z our server has run 2.21.0, with the session-wide session-id, as a single process. Over your per-agent window, its 120 s idle aborts went from 1.52 to 0.57 per 1k. Most aborts on both sides of that change are local. When our WAN drops, our router breaks DNS and TLS, which shows up as certificate has expired and ENOTFOUND on every provider. I dropped every request that fell within 10 minutes of one of those errors, which removes 4–8% of requests. That leaves:

Our install	session-id	Started	120 s idle aborts	Without local outages	websocket_transport_error without local outages
10-10 → 10-14 03:01Z	none	56,395	85 (1.51/1k)	12 (0.22/1k)	69 (1.27/1k)
10-14 03:01Z → 10-16 06:29Z	session-wide	11,838	18 (1.52/1k)	2 (0.17/1k)	12 (1.03/1k)
10-16 06:29Z → 10-21 00:42Z (your per-agent window)	session-wide	10,552	6 (0.57/1k)	2 (0.20/1k)	0

So the 1.6 per 1k I gave you on 10-16 was mostly our network, not an upstream baseline. I flagged that it included local failures but didn't remove them, and I should have. With them removed, our idle-abort rate held at about 0.2 per 1k with no header, with the session-wide header, and through your per-agent days.

Reading the two installs together:

• Transport errors fell on both installs over the same days, whether or not the key changed. Ours went from 12 to 0 (1.03 to 0 per 1k) with the session-wide key unchanged. Yours went from 58 to 3 (2.9 to 0.6 per 1k). That looks like upstream recovering, not an effect of the key.
• The session-wide key didn't raise idle stalls here, even with wider fan-out than yours. Our busiest session sent 3,846 requests from 145 agents under one key, with up to 13 in flight at once, and had no idle aborts. Your per-agent rate (2 in 5,307, or 0.38 per 1k) is within noise of our session-wide rate over the same days (2 in 9,758, or 0.20 per 1k). Both of ours stalled during reasoning, before any text or tool call. Your excess was concentrated in two sessions that later ended or shrank, so the cause looks like something in those sessions or that stretch, not the key.
• Fresh-connection caching also moves with workload alone. Across the same two windows, with no change to the key, the share of our new_partition_head requests getting any cache hit went from 57% to 46%, and their uncached share went from 54% to 83%. So part of your drop from 78% to 30% is probably workload. The direction still matches what a per-agent key should cost, and those requests are about 1% of input tokens on both installs.

We agree with your conclusion: session-wide stays the default, and we won't open a per-agent follow-up. If idle stalls come back on 2.21.x in long, wide sessions, please open an issue with the session's agent count and how many requests it had in flight at once, and we'll look again.

Thanks for running the comparison so carefully, and for reporting the results that went against the hypothesis as plainly as the ones that supported it.