Skip to content

fix: count Anthropic cache tokens in plugin + loop usage accounting (#790) - #791

Merged
ranxianglei merged 1 commit into
masterfrom
wt-790-ework-daemon
Sep 15, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
wt-790-ework-daemon

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #790

Problem

Anthropic reports its input context as three segments: fresh (input_tokens) + cache-read (cache_read_input_tokens) + cache-write (cache_creation_input_tokens). The context size is the sum of all three; the cache hit is the read segment only.

The plugin passthrough and the compress-loop paths never accounted for two of those segments:

  • usageFromSseEvent (plugin SSE): message_start parsed only input_tokens + cache_read_input_tokens; message_delta ignored both cache fields entirely — so an upstream that delivers full usage in message_delta (after an all-zero message_start) was recorded as input=344 cached=0 instead of input=56040 cached=53696, collapsing lastInputTokens and breaking compression triggering.
  • The total computation (promptInputTotal) had no notion of the write segment at all, so even when a field was parsed it never reached the reported context size.
  • Same root cause in proxy mode: src/loop/adapter-anthropic.ts never parsed cache_creation_input_tokens and recordUsage computed the total without it. Fixed in this PR because it is the identical defect on the other compression mode (AGENTS.md §6: wire changes must be reasoned about in both modes).
  • Non-streaming plugin JSON path: added cache_creation_input_tokens to the sample. (Proxy-mode non-streaming JSON already summed all three via usageTotals — no change needed there.)

Changes

  • src/util.ts — promptInputTotal(protocol, input, cached, creation?): under split semantics (Anthropic / OpenAI 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 violation) the write segment is additive; under total-includes-cached wires (openai/responses) it is deliberately NOT added back.
  • src/plugin.ts — UsageSample gains creationTokens; usageFromSseEvent parses all three fields in both anthropic branches (message_start: record whatever is present — zero is a real value for a fully cache-hit turn; message_delta: adopt only > 0, extending the existing relay zero-echo guard so a zeroed echo cannot clobber start values); mergeUsageSample carries the field (per-field overwrite ⇒ omitted keeps prior, repeated snapshots don't accumulate); applyUsageSample folds it into the total; pipePluginJson parses the field. Cache writes count toward context size, never toward hits (cachedTokens stays = read only).
  • src/loop/core.ts — usage event/recordUsage carry creationTokens into promptInputTotal.
  • src/loop/adapter-anthropic.ts — parse the field on message_start; on message_delta adopt it in the complete-authoritative branch (#299 atomic adoption, keep-prior-if-absent) or > 0 in the incomplete branch.
  • tests/host-usage-postfold.test.ts — 8 regression tests: the issue's exact minimal repro (zero start + full delta → 56040/53696), real-shape stream, repeated-snapshot non-accumulation, zero-echo non-clobbering, fully cache-hit turn (start input_tokens: 0), plugin JSON (anthropic + openai total-semantics unchanged), and both loop-path variants.

Verification

  • Issue repro replayed against the built code before/after: [acp-usage] input=344 cached=0 (cache hit 0%) → [acp-usage] input=56040 cached=53696 (cache hit 96%) — matches the expected stats from the issue.
  • Forwarded wire bytes are byte-identical (accounting is sniff-only); asserted in tests.
  • npm run typecheck ✓ · npm test ✓ (1520 pass / 0 fail, E2E-gated ones skipped) · npm run build ✓
  • Full tests/e2e/e2e-codex.test.ts not run here: the sandbox network perimeter forbids probing the local upstream this suite targets. The change touches no forwarded bytes (unit tests assert verbatim forwarding), only internal accounting.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: wt-790-ework-daemon (b1061ec)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-791

Each push to this PR publishes a new version under the pr-791 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr791.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR review 完成,已直接在分支上修复冲突并重新验证。

查重:无重复。相关历史项(#779/#783 DeepSeek prompt_cache_hit_tokens、#300/#303 stitched-stream usage、#170 per-protocol cache accounting)均已关闭且与本 PR 正交;唯一同主题开放项是本 PR 所修的 #790。

diff 整洁度:干净。5 个文件(src/util.ts、src/plugin.ts、src/loop/core.ts、src/loop/adapter-anthropic.ts、tests/host-usage-postfold.test.ts),+246/-18,每一行都服务于本 PR 目的;无锁文件连带变更、无 version bump、无无关改动。

发现并已修复的问题(直接补了 commit):PR 基于 bfb0b68 开出后 master 前进了(#748/#775/#784 等合入),GitHub 报 mergeable_state: dirty。已在本地 rebase 到 origin/master(6fff170),唯一冲突在 src/plugin.ts:1268(pipePluginJson 的 cachedTokens 取值链)——master 侧 #779 加了 DeepSeek 顶层字段回退 ?? num(usage["prompt_cache_hit_tokens"]),本 PR 加了 creationTokens 字段,两者独立,合并保留双方。已 force-push 到 PR 分支(head 现为 b1061ec),rebase 后的 commit 仍只含原 5 文件,无夹带。

逻辑审核要点:

验证(rebase 后在 b1061ec 上重跑):typecheck ✓ · build ✓ · 全量测试 1585 pass / 0 fail / 2 skipped(E2E-gated)✓(连跑两次均干净;首次运行出现过 1 例未复现的失败,疑似负载下的计时敏感用例,后续两轮稳定)。新增 8 组回归测试覆盖了 issue 最小复现(zero start + full delta → 56040/53696)、重复快照不累加、零回显不覆盖、全命中轮次、plugin/loop 双路径及 openai 语义不变量,质量合格。

遗留:tests/e2e/e2e-codex.test.ts 在本沙箱仍无法运行(其默认本地上游不可达,与 PR body 声明一致)。本改动只影响内部记账、不动转发字节(有单测断言),风险低;建议合并前在有可用上游的环境或 CI manual dispatch 上补跑一次 e2e 四阶段核心。

中文摘要:修复了 Anthropic 缓存写入/读取 token 在 plugin 与 loop 两条记账路径漏计导致的上下文大小塌缩(#790);review 中顺手把 PR rebase 到最新 master 解决了与 #779 的一处冲突(两边改动都保留),重跑 typecheck/全量测试/build 全部通过,可以合并(建议合并前补跑一次 e2e)。

@ranxianglei
ranxianglei merged commit bc1b2c4 into master Sep 15, 2026
7 checks passed
@ranxianglei
ranxianglei deleted the wt-790-ework-daemon branch October 15, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Anthropic plugin passthrough 漏计最终 usage 中的缓存 token,导致上下文大小异常下降

1 participant