Skip to content

Mark the request's end on its last block, so a 1h block TTL does not clash with a top-level 5m mark - #187

Merged
jonaslsaa merged 1 commit into
jonaslsaa:mainfrom
0xTomDaniel:fix/explicit-end-cache-mark
Oct 17, 2026
Merged

jonaslsaa merged 1 commit into
jonaslsaa:mainfrom
0xTomDaniel:fix/explicit-end-cache-mark

Conversation

@0xTomDaniel

Copy link
Copy Markdown
Contributor

With a proxy that gives Anthropic block cache marks a 1h TTL (ours is a self-hosted CLIProxyAPI in front of Anthropic), every request without a status line fails, which is every compactor call:

400 Top-level cache_control has ttl='5m' but the target block already has cache_control with ttl='1h'. When both are specified on the same block, they must have matching TTLs.

cachePayload ends such a request with top-level automatic caching (payload.cache_control), whose TTL defaults to 5m and which lands on the last block. Once the proxy has turned the block marks into 1h marks, Anthropic refuses the pair. Requests with a status line already avoid this, because their turn mark is a block mark.

Fix: in src/cache.ts, the end-of-request mark goes on the last block itself, the same way the turn mark does: the last block of the last non-system message, skipping thinking blocks. Nothing is written at the top level any more. It is still four marks (three on the view, one at the end), Anthropic's limit, and a cached prefix ends in the same place, so caching works as before. When a proxy rewrites TTLs, all four now carry the same one.

Tests (test/memory.test.ts):

  • a compactor-shaped request (view, instruction, an assistant reply with thinking, a string follow-up, Pi's empty settings message) gets no top-level mark and an end mark on the last user block; with every mark rewritten to 1h, nothing is left at a default 5m
  • a request ending in a thinking block marks the text block before it
  • the existing view-marks test now expects the end mark on the last block instead of payload.cache_control

Checked live through the proxy with claude-sonnet-5-5 and max_tokens: 1: the old shape (top-level plus a block mark) returns the 400 above; the payload cachePayload now builds returns 200, with cache_creation.ephemeral_1h_input_tokens showing the proxy's 1h TTL applied to the block marks.

npm run check passes; npm test: 289 pass, 1 skipped (the Windows-only lock test).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused change has regression coverage for the affected request shapes, with no unresolved issues identified.

0 open findings

What changed in this PR

Moves end-of-request caching from the top level to an eligible message block, avoiding TTL conflicts when a proxy applies 1-hour block caching.

Changes:

  • Preserves turn-mark behavior while adding block-level end marks.
  • Adds regression coverage for compactor-shaped requests and trailing thinking blocks.
File Description
test/​memory.test.ts Verifies block-level marks and absence of top-level caching.
src/​cache.ts Places the end mark on the last eligible non-system message block.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jonaslsaa
jonaslsaa merged commit 25ba0b8 into jonaslsaa:main Oct 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants