Skip to content

[PR] fix: re-carry the anthropic cache_creation breakdown on the synthetic terminal - #4074

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-21_anthropic-cache-creation-breakdown
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-21_anthropic-cache-creation-breakdown

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

Closes #4072

Since #1315, bili forwards the anthropic message_start usage-neutral ({input_tokens:0, output_tokens:0}) so the synthetic terminal is the sole usage authority. But Anthropic puts the usage.cache_creation breakdown object (ephemeral_5m_input_tokens / ephemeral_1h_input_tokens) in exactly that frame, and the terminal re-emitted only the four numeric fields — so the breakdown was lost on every turn through bili. Claude Code VS Code derives its prompt-cache TTL countdown icon solely from that breakdown; it disappeared for every user routed through bili (caching itself kept working).

Fix

Behavior change disclosure

Additive wire change on the anthropic lane: the synthetic terminal's usage object now carries a cache_creation object (byte-identical to what the upstream start frame had) when the upstream provided one. Old: key absent. New: present when provided. Clients reading the breakdown (Claude Code VS Code countdown) recover; clients ignoring unknown usage keys are unaffected. Totals/cache_read semantics and start-frame neutrality unchanged.

Config surface

N/A — no config field added, renamed, or re-semantics'd.

Tests

  • Extended the [bug] claude native: stitched-stream usage double count after in-stream compress locks session ('Prompt is too long') — #300 variant #1310 non-stitched companion test: the terminal carries the same cache_creation object as the upstream start frame; the forwarded start stays neutral.
  • New stitched test: the final round's breakdown wins (round 1's pre-fold value must not leak).
  • Negatives: no breakdown upstream → no cache_creation key in the terminal; an all-zero schema-shaped breakdown echoed by an incomplete delta does not clobber the start-frame values.
  • Local targeted runs green (rebased head): loop-adapters 38/38, host-usage-postfold 27/27, issue1766-terminal-extra + proxy-stream-error + plugin-terminal-usage 23/23; typecheck clean. Full suite + real-client e2e lanes left to CI.

@ranxianglei ranxianglei changed the title fix: re-carry the anthropic cache_creation breakdown on the synthetic terminal [PR] fix: re-carry the anthropic cache_creation breakdown on the synthetic terminal Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_anthropic-cache-creation-breakdown (fffa539)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4911-pr.4074.46abc96

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4074.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 本 PR 是 #4072(原始缺陷)的修复;open+closed 中无其他覆盖同一失败签名(synthetic terminal 丢失 cache_creation breakdown)的在途条目。相关条目仅 #1315(引入该缺陷的中性化改动)与 #1310(双计保护),均与本修复不重叠。

标签: bug, claude-code, scope/wiring, p2(缓存命中本身正常,丢的是 breakdown 对象与 CCVS 倒计时图标——功能降级有绕行,非缓存泄露类)

进入 review:这是已完成的工作项,不重做——按 PR 审核流程独立重审。初步发现两条,正在处理:

  1. 分支现 behind master 两个提交([PR] remove the #726 dead-end cooldown; expire #3846 local-refusal records after 10m (#4079) #4105、[PR] test: pin storage mode in compress-from-store (CI red — same class as #4076) #4113 已入 master)→ rebase 到最新 master 后重新验证再下结论;
  2. 当前 head(cf8f653bb)的 CI test 分片 6 红 5(ubuntu 22/2、22/3、24/1 与 windows 22/1、22/2 红,仅 ubuntu 22/1 绿)——需判定失败是本 PR 引入还是当时 master 状态所致:拉失败分片日志定位测试名 + 对照当前 master head 的 CI。

审完给结构化八问结论。

… terminal (#4072)

The cache_creation breakdown object (ephemeral_5m_input_tokens /
ephemeral_1h_input_tokens) lives only in the upstream message_start frame,
which #1315 forwards usage-neutral to keep the synthetic terminal the sole
usage authority. The breakdown had no other carrier, so hosts deriving
their prompt-cache TTL from it (Claude Code VS Code countdown icon) lost
it on every turn through bili.

Capture usage.cache_creation verbatim per round (start frame unconditional,
incomplete message_delta frames only when they carry tokens — same relay
zero-echo guard as the numerics) and emit it from buildTerminal next to
cache_creation_input_tokens. Stitched turns take the final round's
breakdown, matching how the totals are chosen. Never synthesized: absent
upstream means absent downstream. The start frame stays neutral, so the
#1310/#1315 double-count protection is untouched.
@ranxianglei
ranxianglei force-pushed the 2026-10-21_anthropic-cache-creation-breakdown branch from cf8f653 to fffa539 Compare October 21, 2026 06:24
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR Review 结论:#4074(fixes #4072)— 代码与测试通过审查;CI 红已定位为陈旧基线产物,rebase 后新 head 待 CI 确认。

① 方向审核(原 issue):根因成立。#1315 起 bili 把 anthropic message_start 的 usage 中成({input_tokens:0,output_tokens:0}),synthetic terminal 成为唯一 usage 权威;而 Anthropic 恰好把 usage.cache_creation breakdown 对象(ephemeral_5m_input_tokens/ephemeral_1h_input_tokens)放在那个帧里,buildTerminal 只重发四个数字字段 → breakdown 每轮丢失。Claude Code VS Code 的 prompt-cache TTL 倒计时图标只从该对象推导,于是所有走 bili 的用户图标消失(缓存本身仍工作)。修复打在源头(adapter 逐字捕获 + terminal 重带),不是症状层过滤。方向正确。

② 回归审查(diff 触碰路径的既有行为逐项):

③ 代码审查:捕获为逐字透传,不改对象形状;src/loop/core.ts 是纯类型管道;无死代码、无魔法值、风格一致。

④ 测试有效性:非 stitched 伴生测试断言 terminal 携带与上游 start frame 相同的对象(回归时抓得住);stitched 测试钉「末轮胜出」;两个负例(上游缺席 → 键缺席;全零 delta 回显不覆盖)。断言有牙。

⑤ diff 整洁度:src/loop/adapter-anthropic.ts +40、src/loop/core.ts +9/-3、tests/loop-adapters.test.ts +92/-2 —— 每行都与本 PR 目的相关。

CI 红根因(逐 job 日志定位,全部是陈旧基线产物,与本 PR 改动无关):旧 head cf8f653bb 上 5 个 shard 红——

已执行:分支 rebase 到当前 master(5f3b1e601,含 #4105/#4113/#4114),零冲突,force-push → 新 head fffa53973。新 head 本地定向验证:目标套件 88/88(loop-adapters / host-usage-postfold / issue1766-terminal-extra / proxy-stream-error / plugin-terminal-usage)+ 先前红的两个 verif 文件 5/5 + typecheck 干净。全量回归交给 CI(已在 fffa53973 上运行)。

结构化八问:

  1. 是什么问题:anthropic 车道的合成终帧丢了 usage.cache_creation breakdown 对象,依赖它的客户端(CCVS 的 TTL 倒计时图标)功能消失。
  2. 什么原因导致的:fix: anthropic stitched terminal as the sole usage authority (#1310) #1315 把 start 帧 usage 中立化、权威移交 terminal,但 buildTerminal 只重发四个数值字段,没带同一帧里的 breakdown 对象。
  3. 影响范围与严重程度:所有经 bili 的 anthropic 车道会话(proxy 模式);后果是客户端一个展示性能力丢失(倒计时图标),缓存命中本身不受影响——p2 体验缺陷,非不可用。
  4. 怎么修复的:adapter 每轮逐字捕获 cache_creation(start 帧无条件取;不完整 delta 帧仅在实际带 tokens 时取),terminal 在 cache_creation_input_tokens 旁重带;stitched 取末轮。
  5. 是否有回归问题:无(见 ② 逐项:start 帧中性化、totals/cache_read、缺席即缺席、零值守卫、stitched 规则均未动)。
  6. 是否破坏缓存:不涉及——请求侧前缀字节不变,只是响应 usage 字段增量加键。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。

判定:可合并(待新 head CI 绿);合并操作归你:#4074

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 确认(新 head fffa53973):核心矩阵六条腿全部绿——

  • test (ubuntu-latest, 22, 1/2/3) ✅
  • test (ubuntu-latest, 24, 1) ✅
  • test (windows-latest, 22, 1/2) ✅
  • 外加 static-gates(ubuntu 24 / windows 22:typecheck + kernel + pi-subagents 套件 + build + thin-install 冒烟)、mysql-contract、全部 guard 均 ✅。

注:期间有一条「success」的 CI run 是标签同步触发的 skip-scoped run(plan job 判 scope=skip、测试腿全跳过),不是有效结论——上面这份是真实跑完的矩阵。旧 head 上红的 verif 两条 + #3769 flake 在新基线上不再出现,印证了根因定位(陈旧基线,master 侧 #4084/#4087 已修)。

最终判定不变:可合并。 合并操作归你:#4074

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4525(master 6d7c587,合并方式 merge commit,提交保留原作者署名)。本 PR 关单收尾;若列车中对应提交出问题,回滚点为 #4525 单提交 revert。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][claude] #1315 drops usage.cache_creation (ephemeral_5m/1h breakdown) → Claude Code VS Code prompt-cache countdown icon disappears

1 participant