Place explicit prompt-cache breakpoints on the OpenAI-family legs

[rezaho]

What was wrong

On the Responses legs the prompt cache never covered the conversation body. The provider's default mode places its breakpoint on the latest message, and this stack's latest message is a per-request status row whose clock changes on every call, so the entry each request wrote ended inside bytes the next request no longer sent.

Measured on gpt-5.6-terra, three requests of an agent turn's shape growing by one tool round each: every step read about 1,642 tokens — the system text and the tool list and none of the body — and rewrote nine and then ten thousand tokens of byte-identical conversation at the 1.25x cache-write premium. Sending the third request again byte for byte read 13,714 of 13,717, so the cache itself was reachable and within its lifetime the whole time.

What this does

OpenAIAdapter.format_request_payload now asks for explicit cache mode and puts a prompt_cache_breakpoint on every durable input item — system, developer and user messages, and function-call outputs.

Marking every item rather than the tail is the whole rule, and the tail rule is the trap. A breakpoint is matched only while it is still present in the request being sent, so a single marker moved forward one row per request leaves nothing behind for the next request to match: measured at 0 tokens read on every request, costing more than sending nothing at all. Marking every item makes the marked set a function of each row's own position from the start of the list, so the row that happened to be last on the previous request is marked by construction and the rule needs no memory of what an earlier request sent. The same three requests then read 10,620 and 12,109 of a 13,717-token prompt with a 1,489-token write per step.

The provider's own limits are satisfied by construction rather than by arithmetic: at most four new cache writes per request (in explicit mode the latest four breakpoints, and a breakpoint covers everything before it, so the newest one writes the whole new prefix), and reads consider the latest fifty breakpoints (the newest marker is always within a handful of the end).

Two gates, both load-bearing

• Model. Anything before the GPT-5.6 family answers either field with a 400. A gpt-<major>[.<minor>] name is parsed as a number pair and compared, not sorted as a string — gpt-5 must not read as later than gpt-5.6.
• Provider. The factory routes every unrecognized provider to this adapter, so an OpenAI-compatible third-party endpoint behind a 5.6-shaped model name would otherwise receive fields only OpenAI's and Azure's surfaces have been measured to accept. For that check ProviderAdapterFactory now stamps the requested provider onto each adapter it builds — it is the only layer that knows which provider was asked for, since several providers share one adapter class and the fallback hands unknown ones to this class outright. BaseAPIModel was doing the same assignment one line later for the sync adapter; that duplicate is removed.

prompt_cache_options is set by the branch that placed a marker, and only then. Explicit mode with no breakpoint is the documented way to turn caching off and measures as exactly that — nothing written, nothing read, the whole prompt at plain input price — so mode and placement are one decision in one place.

What does not change

A caller's contribution to caching stays one flag. CACHE_EXEMPT_KEY moves to base now that both adapter families read it, re-exported from anthropic where it was defined, so every existing import keeps working. A row carrying it is skipped wherever it sits, keeps the shape it arrived in, and its flag never reaches the wire.

The Anthropic family keeps its tail rule unchanged. The OpenAI OAuth twin is untouched: it subclasses the base adapter rather than this one and speaks to a backend neither provider page documents.

Payloads for every pre-5.6 model, both OAuth legs, both Anthropic legs and Bedrock are byte-identical to the previous commit — checked by rebuilding the same fourteen payloads from that commit's code and diffing them key by key, not by inspection.

The one wire-shape change: a tool result's output stops being a string and becomes a one-part input_text list, because a breakpoint can only ride a content block. The text the model reads is unchanged, and a test pins that by flattening every item to its text and comparing against the pre-generation build.

Tests

tests/models/test_prompt_cache_key.py becomes test_openai_prompt_cache.py: the file now covers the caller's routing key and the markers, which are one feature from a caller's side, and its six-adapter fixture gained a model-name parameter so the existing key assertions run on both sides of the generation gate. 361 tests there, plus the Anthropic-family no-leak assertions in test_prompt_cache_breakpoint.py and the generation gate replacing the old "no cache field is ever sent" test on the Azure leg.