Check that headroom sessions keep their prompt cache as warm as direct ones #412

Usage records carry viaHeadroom from the release that shipped it, so the assumption behind the per-session routing (headroom's compressed prefix is identical from turn to turn, which keeps the prompt cache warm) can be checked from the records: compare the cache-read share of input tokens for sessions whose requests crossed headroom against ones that went direct, over sessions long enough to have a warm cache.

If the headroom sessions read markedly less from the cache, the compressed prefix is not stable and the cache-boundary rules in the epic need revisiting.

First measurement, from the usage log before the field shipped, using the project value that only headroom launches record to tell the two kinds apart, over sessions of at least ten requests and leaving out each session's cold first request: headroom sessions read 98.64% of their input from the cache (about 2878 tokens written per request over 108 sessions), and direct sessions read 98.95% (about 2950 written per request, over only 2 sessions). Headroom is not rewriting more than direct, which is consistent with a stable prefix, but the direct side is too small to settle it.

Revisit trigger: the first day a door running a release with viaHeadroom has served both kinds of session. Repeat the comparison with the field and close this then.

Second measurement, from Claude Code's own transcripts (assistant usage per turn, sessions of at least ten turns, each session's first turn left out). Sessions that began between the start of September and the start of October, before headroom was in use, read 99.01% of their input from the cache and wrote about 3300 tokens per request (480 sessions). Sessions that began from the fifth of October, with headroom on, read 98.05% and wrote about 4700 tokens per request (109 sessions).

That is the opposite of the first reading: the headroom era writes noticeably more per request. The two eras differ in workload (model, subagent use, session length) as well as routing, so it does not show headroom is the cause, but it does not clear it either. A like-for-like comparison needs direct and headroom sessions running the same work in the same period, which viaHeadroom records make possible once a door with the field has served both.

Controlled run (the same ten scripted turns in a directory holding one large file, on one identity, with the haiku model, once direct and once through headroom; per-turn usage from the print-mode result). Direct: after the first turn it read 99.3% and 99.0% of its input from the cache in two runs, writing a few hundred to a thousand tokens a turn. Through headroom: in the first run the cache missed on turns one to four (about 13k tokens read, 47k sent uncached and 25k rewritten per turn) and recovered only after a 76k rewrite; in the second, with the arm order swapped and different content, it missed on every turn (a 15.5% read share, about 45k uncached and 28k written per turn). So for a conversation that reads a large file, headroom can change the prefix from turn to turn and cost the whole context each time, which contradicts the stable-prefix assumption and the log-based reading above. The earlier readings measured real work where tool results are small.

A session on headroom whose responses twice in a row read back less than the response before wrote now goes direct at its next cache boundary. What this does not explain is why headroom changes the prefix; that is a question for the headroom fork and is the next thing to look at, with this scripted conversation as the repro.

Measurement from records that carry viaHeadroom (a door on the current release, restarted for this; the scripted conversation repeated twice more, and the fleet's own sessions as the sample). The 92 /v1/messages records written through headroom since the restart read 97.35% of their input from the cache, but sent about 5100 tokens per request uncached (where a direct session sends a handful) and wrote about 4500. The scripted conversation again showed the failure: through headroom 16% read share with about 45k uncached and 25k rewritten tokens per turn, against 99.4% with a few hundred tokens written direct. No record has viaHeadroom: false yet, because the direct arm of the script bypasses the door; a door that serves every launch by default will write those as ordinary direct sessions run.

Conclusion: the stable-prefix assumption does not hold for headroom as deployed. It does not break the cache wholesale on ordinary work, but it sends a large uncached tail on every request and fails outright for a conversation that reads a large file. The decision now takes a session off headroom when its own thread keeps missing its cache, and the cause (what headroom changes in the prefix, or whether it moves the cache breakpoints) belongs to the headroom fork, with the scripted conversation in claude-use-worktrees/ab-cache as the repro.

Both sides from records that carry the field (the scripted direct run repeated through the door, so its requests are written with viaHeadroom: false): 232 requests through headroom read 98.41% of their input from the cache, wrote about 3900 tokens and sent about 2240 uncached per request; 11 direct requests read 95.26% (its first turn is a cold write), wrote about 4000 and sent 2 uncached per request. The share of input read from the cache is close, so the cache is not lost wholesale on ordinary work; the difference is the uncached tail, about 2200 tokens a request more through headroom, billed at the full input price instead of the cache read price. The direct sample is small (one scripted conversation), and a larger one accumulates as direct launches now go through the door by default.