Repository navigation
Conversation
…anxianglei#6861) Claude Code stamps a cache_control breakpoint on its last message and moves it as the conversation grows. anthropicToCore harvests those marks into a per-request map that coreToAnthropic re-emits verbatim, so the message that carried the breakpoint last turn loses it and its bytes rewrite mid-history on every request, collapsing the prefix cache. Pin client-managed breakpoints cumulatively, the same discipline as bili's own marks: a marked message stays marked until it folds away, within the 4-breakpoint budget. Proxy mode only; in plugin mode the host owns the rolling marks.
(ai: vllm-qwen/qwen3.8-27b) Dedup: not a duplicate. This is the only open PR that pins client-managed message breakpoints. Related open PRs #4139/#4391 fix seam-detector classification (reading marker migration as append rather than mid-history) and #4234/#4232 are kernel system-message mark fixes — different mechanism and surface from this byte-stability fix. PR #6869 review — pin client-managed cache marks (fixes #6861) I pulled the head ( Root cause (verified, independent of the issue's framing): real and correctly located. Regression review (existing behavior on the touched path):
Code + tests: clean, well-named, comments explain the why. 4 unit tests (pinning, fold retirement, budget eviction order, knob) + 1 real two-turn rig test asserting turn-1 messages serialize byte-identically on turn 2 and mark indices Local verification: typecheck clean, build OK, new suite 5/5, neighbors Known boundary (non-blocking): sidecar messages (image blocks, 结构化八问(中文):
合并判定: 方向正确(修根因非堵症状)、无回归、CI 全绿(54 checks)、本地 typecheck/build/新测试 5/5/邻接套件 20/20 通过、diff 干净(3 文件 +354 −7,每行相关)。属 wire 消息形状变更(出站 cache_control 落点),按 §7.4 必须人工合并、不可自动合并。建议:可合并(需人工)。 侧车消息断点被丢弃的边界建议作为后续 kernel 跟进(需 owner 批准 kernel 变更),不阻塞本 PR。 |
Claude Code stamps a cache_control breakpoint on its last message and moves it as the conversation grows. bili harvested those marks per request and re-emitted only the current ones, so the message that carried the breakpoint last turn lost it and its bytes rewrote mid-history on every request. That is the seam the issue reports: prefix cache hit down to 8.7% on one request, 119K tokens re-billed.
What changed: client-managed breakpoints are now pinned cumulatively, the same discipline as bili's own marks. A marked message stays marked until it folds away, inside the 4-breakpoint budget. Proxy mode only; in plugin mode the host owns the rolling marks, so pinning there is skipped.
How verified: new regression test tests/issue6861-client-cache-marks-pinned.test.ts, 5/5 green (unit tests for pinning, fold retirement, budget eviction, and the no-cache-control knob, plus a two-turn rig test proving turn-1 messages serialize byte-identically on turn 2). Neighbor suites anthropic-cache-control and opencode-rolling-cache-marks pass 20/20. Typecheck clean in the changed files. Full suite verdict is remote CI.
Fixes #6861