Skip to content

fix: meter signature-only thinking mass (Claude Code) — #1320 - #1326

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-25_thinking-mass-projection
Sep 25, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-25_thinking-mass-projection

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #1320

Root cause

Claude Code round-trips extended-thinking blocks as signature-only entries ({type:"thinking", signature} with no visible text). The provider restores and bills the hidden thinking tokens server-side, so its usage.input_tokens is correct — but locally anthropicToCore converts them to empty-text reasoning messages, and every per-message meter (nudge compressible mass, block compressedTokens receipts, context breakdown/gauge) sees 0 tokens for them. Verified against master 0cef2d47 + acp-kernel 0.0.89: the wire round-trip is faithful (signature re-emitted by coreToAnthropic); only local metering is blind.

Fix (decision-point projection, not a payload mask)

New projectThinkingMass() in src/server/budget.ts, called in prepareAnthropic right after anthropicToCore:

  • U = max(0, provider-measured previous-turn input total − local estimate of visible message text − system/tools overhead − image tokens)
  • Attributed to signature-only reasoning messages weighted by signature length, via the kernel's existing host-projected CoreMessage.thinkingTokens seam — countMessageTokens counts it at every metering site already (nudge ranges, applyCompression receipts, render tags, context breakdown), so all surfaces pick it up with no further changes
  • No-op unless a real usage report AND ≥1 signed thinking block exist → sessions without hidden thinking are byte-for-byte unaffected
  • Metering-only: outbound wire bytes unchanged (deep-equal asserted in tests)
  • Plugin-panel unprunedTokens switched to the same countMessageTokens caliber

No kernel change needed (the seam already exists in pinned acp-kernel 0.0.89) — no cross-repo release ordering.

⚠️ Behavior change (old → new)

For Anthropic sessions carrying signature-only thinking blocks (i.e. Claude Code):

  • Nudge compressible mass / pendingT1: previously excluded the entire thinking share → now includes the provider-measured residual. Growth nudges can now arm where they were idle (the reported symptom).
  • Compress receipts (~N tokens saved, tokensCompressed): previously understated by the thinking share of folded ranges → now include it (issue numbers "~17k saved" vs ~41k real are corrected).
  • Web/plugin context gauge + tag token values: rise by the projected residual, toward the billed total.
  • Cadence constants (flat 50K), config schema, persistence format, wire bytes: unchanged.

Non-Anthropic wires and sessions without signed thinking: unchanged (projection no-ops).

The projection lags one turn (it uses the previous turn's provider total — the same contract as the nudge's own "previous turn input_tokens" input); steady-state Claude Code sessions converge from their 3rd turn on.

Known residuals (documented, out of scope)

  • Per-turn output_tokens apportionment not done — gap-based global consistency instead (self-bounding).
  • Zero-baseline anonymous forks still estimate from chars only.
  • prepareCountTokens (read-only /messages/count_tokens preview path) not touched.

Folded-in observation (#1310 obs 4 / #684 sentinel)

warnAnthropicThinkingPairs fires when some assistant tool_use turns lack any thinking block while others carry one — this over-fires on legitimate Claude Code histories (turns that simply had no thinking; plus bili's own #651 deliberate reasoning drops). Left untouched here (a logging-behavior change deserves its own scope); filed as a separate issue.

Pre-flight

  • typecheck ✅ · full suite 2567 pass / 0 fail ✅ · build ✅
  • New regression suite tests/issue1320-thinking-mass.test.ts (11 tests): projection arithmetic (exact-residual weighting, clamps, exclusions); kernel-level bug repro asserting the unprojected run's compressible mass sits below the 50K threshold with an idle nudge while the projected run arms tier-1 on identical growth; wire-fidelity deep-equal; real-pipeline gauge E2E through the proxy.
  • e2e-codex: local upstream unreachable from this environment → CI will run it (diff touches src/server.ts).

Requires human review: changes nudge/metering semantics on a hot file — outside the auto-merge gate (§7.4).

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-25_thinking-mass-projection (0987a31)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1326

Each push to this PR publishes a new version under the pr-1326 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1326.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 开始 review 本 PR(#1320 的修复)。已核对:分支落后当前 master 7 个 commit,其中 master 刚把 acp-kernel 从 0.0.89 升到 0.0.90(#1316 refold-wiring)——PR 的投影逻辑依赖 kernel 的 thinkingTokens seam,需要验证 0.0.90 下该 seam 语义未变。正在做:代码审查 → rebase 到最新 master → 全量 typecheck/test/build。结论稍后更新。

@ranxianglei
ranxianglei force-pushed the 2026-09-25_thinking-mass-projection branch from 9e66503 to 2faa53e Compare September 25, 2026 11:55
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — 可以合并(merge 操作留给人工)。

What I did to the PR

  • Rebased onto current master (d4f630b7, was 7 commits behind): 9e66503f → 2faa53e0, clean single-commit history, no textual conflicts (master-ahead commits touch launcher/decompress/kernel-pin only).
  • Verified the kernel seam across the pin bump: master's feat: pin acp-kernel 0.0.90 + wire refold-in-place into proxy decompress #1316 moved acp-kernel 0.0.89 → 0.0.90 while this PR was built against 0.0.89. Diffed the installed kernels: tokenize.d.ts and wire/bili-message.d.ts identical; types.d.ts diff is purely additive refold fields (restoredInline, hiddenOrphanRefs, notes). countMessageTokens / CoreMessage.thinkingTokens / BiliMessage.thinkingSignature semantics unchanged.
  • Full pre-flight on the rebased head: typecheck ✅ · 2583 tests, 2573 pass / 0 fail / 10 skipped ✅ · build ✅ (dist-import-annotations: OK).
  • CI on 2faa53e0: all 11 checks green, including real codex vs deterministic fake upstream (the e2e-codex suite that couldn't run locally) and hermetic-registry e2e.

① Original issue (#1320) — direction

Confirmed: root cause is a metering blind spot, not a wire defect. Signature-only thinking blocks round-trip faithfully (coreToAnthropic re-emits the signature; provider bills restored tokens server-side), so patching payload bytes would be wrong. Fixing at the projection decision point via the kernel's existing host-projected thinkingTokens seam is the right layer — no kernel change needed, and the litmus holds: if hidden mass later surfaced in another shape (e.g. visible-but-uncounted text), the gap-based projection still absorbs it.

② Regression review — existing behaviors on touched paths

Path Verdict
prepareAnthropic flow ordering (classifier passthrough, panel/marker strip, pre-turn ref snapshot, effectiveTokenCount) Preserved — projection inserted between anthropicToCore and effectiveTokenCount; nudge tokenCount input unchanged (still provider-measured lastInputTokens directly)
Sessions without signed thinking OR without a usage report Byte-for-byte unaffected — double early-return guard (measured && providerInputTokens > 0, then targets.length > 0); covered by unit tests
Outbound wire bytes Unchanged — coreToAnthropic ignores thinkingTokens; deep-equal asserted in-test
Nudge cadence constants, config schema, persistence format Unchanged (flat 50K intact; no schema/persist diff)
Plugin-panel unprunedTokens (src/plugin.ts:789) Intended change: defaultCountTokens(m.text) sum → countMessageTokens(m) sum. Identical where thinkingTokens absent; rises by the projected residual where present. Same caliber as breakdown rows — correct
Overflow-arm provenance edge Noted, acceptable: after an overflow error, lastInputTokens = declared window with source "usage" (#857 arms value), so the turn right after overflow can over-project up to (window − visible) before the next real usage report self-corrects. Directionally conservative (toward billed reality) and consistent with how the arm is treated elsewhere
Post-fold steady state Noted, self-bounding: once proxy-mode folds accumulate, gap ≈ H_unfolded + |block| − V_folded − summary; in sessions whose folded visible mass dominates, projection degrades toward 0 (under-meter, never over-meter due to the clamp). Thinking-heavy sessions (the target case) keep a positive gap. Matches the PR's "self-bounding" claim
First request after restart Trivial: systemPromptTokens isn't persisted, so one turn falls back to recounting inbound system+tools (misses ACP-injected tools) → slight over-projection, self-corrects next turn

③ Code review

projectThinkingMass (src/server/budget.ts:82-108) is clean: deterministic exact-sum apportionment (floors early, remainder last), guards against non-finite stored overhead, visible-text reasoning excluded from targets but still counted in the local estimate (no double count), image term excluded from the gap. Call site passes the previous-turn outbound overhead measurement (captured at src/server.ts:2649, i.e. after projection runs — timing is exactly right: both terms refer to the same prior request). No as any, no console.error, comments follow house style.

④ Test validity

The 11 tests have teeth: exact share arithmetic (not tautologies), a kernel-level repro that asserts the pre-fix symptom (compressible mass below 50K → idle nudge) vs post-fix arming on identical growth, wire-fidelity deep-equal, and a real-pipeline gauge E2E through an actual HTTP server with dynamic ports (listen(0) — no environment luck). This bug regressing would trip the repro test.

⑤ Diff cleanliness

4 files, +431/−6, every line on-purpose, single linear commit, no lockfile churn (kernel pin comes from master's own #1316, not this PR).

Coordination note

Two open PRs also touch src/server.ts in different regions: #1328 (overflow rescue, +177/−66 around the forward/error path) and #1325 (registry window caps). Whichever merges second will need a rebase — no semantic overlap seen, but sequence them deliberately.

中文摘要:修复了 Claude Code signature-only thinking block 导致本地计量全盲的问题(nudge 一直 idle、压缩回执虚报省量),方案是在计量决策点把 provider 实测残差投影到签名块上而非改线包字节;已 rebase 到最新 master(含 kernel 0.0.90)并验证 seam 未变,本地与 CI 全套检查(含 e2e-codex)全绿,行为变更仅影响带签名的 Anthropic 会话的计量数值、线包字节不变,可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Coexistence verification on current master head: both this and the sibling claude-native estimation PR (#1325/#1326) cherry-picked cleanly onto today's master and pass the full gate together — typecheck ✓ / 2608 tests 0 fail (10 e2e-gated skips) / build ✓. Shared-file regions are disjoint (window resolution vs prepareAnthropic thinking projection), any merge order works.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,coexistence 结论与我 review 时的 coordination note 一致。补充:master 又前进了(#1328 overflow-rescue、#1298 fingerprint receipts、#1329 sentinel、#1331 kernel 0.0.90→0.0.91 均已合入),而本分支还停在 d4f630b7。我现在把分支再 rebase 到当前 master tip(6bdc6b97),复核 0.0.91 的 thinkingTokens seam,重跑全套 gate + CI 后回报——让 PR head 直接坐在最新 master 上,省得合并时再动一次。

@ranxianglei
ranxianglei force-pushed the 2026-09-25_thinking-mass-projection branch from 2faa53e to 0987a31 Compare September 25, 2026 13:40
@ranxianglei
ranxianglei merged commit 98c5aaf into master Sep 25, 2026
11 checks passed
@ranxianglei
ranxianglei deleted the 2026-09-25_thinking-mass-projection branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] anthropic wire: signature-only thinking blocks missing from compressible estimate — savings receipts overstate, nudge stays idle (claude native)

1 participant