Repository navigation
Mark the request's end on its last block, so a 1h block TTL does not clash with a top-level 5m mark - #187
Merged
jonaslsaa merged 1 commit intoOct 17, 2026
Conversation
…clash with a top-level 5m mark
There was a problem hiding this comment.
🟢 Approval recommended
The focused change has regression coverage for the affected request shapes, with no unresolved issues identified.
0 open findings
What changed in this PR
Moves end-of-request caching from the top level to an eligible message block, avoiding TTL conflicts when a proxy applies 1-hour block caching.
Changes:
- Preserves turn-mark behavior while adding block-level end marks.
- Adds regression coverage for compactor-shaped requests and trailing thinking blocks.
| File | Description |
|---|---|
| test/memory.test.ts | Verifies block-level marks and absence of top-level caching. |
| src/cache.ts | Places the end mark on the last eligible non-system message block. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This was referenced Oct 17, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With a proxy that gives Anthropic block cache marks a 1h TTL (ours is a self-hosted CLIProxyAPI in front of Anthropic), every request without a status line fails, which is every compactor call:
cachePayloadends such a request with top-level automatic caching (payload.cache_control), whose TTL defaults to 5m and which lands on the last block. Once the proxy has turned the block marks into 1h marks, Anthropic refuses the pair. Requests with a status line already avoid this, because their turn mark is a block mark.Fix: in
src/cache.ts, the end-of-request mark goes on the last block itself, the same way the turn mark does: the last block of the last non-system message, skipping thinking blocks. Nothing is written at the top level any more. It is still four marks (three on the view, one at the end), Anthropic's limit, and a cached prefix ends in the same place, so caching works as before. When a proxy rewrites TTLs, all four now carry the same one.Tests (
test/memory.test.ts):payload.cache_controlChecked live through the proxy with
claude-sonnet-5-5andmax_tokens: 1: the old shape (top-level plus a block mark) returns the 400 above; the payloadcachePayloadnow builds returns 200, withcache_creation.ephemeral_1h_input_tokensshowing the proxy's 1h TTL applied to the block marks.npm run checkpasses;npm test: 289 pass, 1 skipped (the Windows-only lock test).