Repository navigation
fix: make effectiveTokenCount's stale-high bound image-aware - #1120
Conversation
The min-cap against the current request's inbound upper bound was text-only (estimateCoreMessagesUpper sums m.text; wire codecs move images out of CoreMessage.text into sidecars), so whenever the previous turn's image contribution exceeded the current turn's text growth the image term was capped away and the silent-backend nudge numerator went blind to byte-billed multimodal sessions. Carry the current request's image term (resolved billing) into the bound at all four call sites; a client-side shrink shrinks the bound in both terms, preserving the #728C stale-high invariant. Fixes #1119
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1120Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1120.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[PR] verification report — I verified the existing work rather than redoing it (the fix landed from the #1119 session). Everything checks out: Branch & base
Code review
Evidence (ran it myself, not just trusting the pre-flight)
Gate assessment (§7.4) PR: https://github.com/raxianglei/billion-context/pull/1120 (link above is the canonical one; open from the issue header) 一句话中文摘要:核对了这个 PR(图片项进入 effectiveTokenCount 的 stale-high 上限、四处调用点按各自计费模式解析)——代码与描述一致、diff 干净、基线是最新 master、CI 全绿;我本地重跑了全套预检并做了负向对照(revert 修复后新测试必挂、其余用例不受影响),回归测试真实有效,符合 §7.4 自动合并门槛,可以合并。 |
The anonymous prefix-affinity early return in effectiveTokenCount still returned estimateCoreMessagesUpper(msgs) — text only — even though ranxianglei#1120 made the explicit-identity min-cap branch image-aware and all four prepare sites pass inboundImageTokens. Wire codecs move images out of CoreMessage.text into sidecars, so an image-heavy fork replaying its full raw history read a text-only char count: usage % far below reality, decideNudge structurally unfireable, over-window payload forwarded raw. Hoist raw above the early return: a fork replays the FULL raw history in one request, so this request's image term IS the whole history's image mass. Conservative direction unchanged (upper bound → compress earlier, never later); the ranxianglei#728 stale-high invariant is preserved untouched. Regression pin in tests/fork-nudge-trigger.test.ts: an anonymous image-heavy fork whose text-only bound (~58% of an 80k window) sits below every nudge band but whose image-aware bound (~83%) crosses the 75% pressure band — fails pre-fix (no nudge), passes post-fix; preflight stays silent and nothing folds. Fixes ranxianglei#1137
What
effectiveTokenCount(src/server.ts) feeds the kernel nudge decision from the previous turn's recorded outbound upper bound (session.stats.localInputEstimate= text + system/tools + image tokens) — but only while capping it against the current request's inbound upper bound, and that bound was text-only:estimateCoreMessagesUppersumsm.text, and the wire codecs move images out ofCoreMessage.textinto sidecars. Whenever the previous turn's image contribution exceeded the current turn's text growth,min()picked the text-only bound and the nudge numerator went blind to images — exactly in non-reporting upstream sessions (#728 regime), i.e. the classimageBilling: "bytes"exists for.Fix
Make the bound image-aware at all four call sites (anthropic/openai/google/responses):
effectiveTokenCountgains an optionalinboundImageTokensparam (stays pure; the term is computed per-wire with that wire's protocol + resolved billing).prepareGooglegains anupstreamOriginparam so its site can resolve billing like the others.Both compression modes
No wire change in either mode: this only alters the fallback numerator used when
lastInputTokens == 0(upstreams that never report usage). Reporting upstreams are untouched. In plugin mode the proxy suppresses nudge injection (the agent owns compression), so there is no client-visible behavior change there either — the corrected figure only improves internal decisions/logs.Tests
New regression test in tests/silent-backend-nudge.test.ts (#728 harness pattern): silent backend, 80k window, ~50k chars of code history + two 10k-token inline images. Text-only bound ≈ 63% — below every band (45% first-sight / 70% host emergency escalation / 75% pressure); image-aware numerator ≈ 88% → nudge fires on turn 2; shrunk image-less turn 3 stays silent. Verified the test FAILS with the fix temporarily reverted (turn-2 nudge missing = the bug) and passes with it.
Window note: 80k instead of the shared 60k because the kernel's absolute-token zones (preserveRecentTokens 5k + minPressureBenefit 5k ⇒ ≥ ~10k tokens of compressible mass) squeeze a 60k window so thin that pre-fix/post-fix states collide.
Pre-flight (rebased onto master @ 14d06f1)
npm run typecheck✅npm test: 2195 pass / 0 fail / 2 skipped ✅npm run build✅Scope: single-module pure
fix:(src/server.ts + one test file). No config schema, persistence, wire-shape, or cross-repo changes. Orthogonal to the parked #1072 hunk / open #1118 billing-semantics discussion (bytes routes are unchanged by both).Fixes #1119