Skip to content

fix: make effectiveTokenCount's stale-high bound image-aware - #1120

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-22_image-term-nudge-bound
Sep 22, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-22_image-term-nudge-bound

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

effectiveTokenCount (src/server.ts) feeds the kernel nudge decision from the previous turn's recorded outbound upper bound (session.stats.localInputEstimate = text + system/tools + image tokens) — but only while capping it against the current request's inbound upper bound, and that bound was text-only: estimateCoreMessagesUpper sums m.text, and the wire codecs move images out of CoreMessage.text into sidecars. Whenever the previous turn's image contribution exceeded the current turn's text growth, min() picked the text-only bound and the nudge numerator went blind to images — exactly in non-reporting upstream sessions (#728 regime), i.e. the class imageBilling: "bytes" exists for.

Fix

Make the bound image-aware at all four call sites (anthropic/openai/google/responses):

const raw = estimateCoreMessagesUpper(msgs)
    + imageTokensInParsedBody(protocol, parsedBody, imageBillingFor(opts, upstreamUrl));
  • effectiveTokenCount gains an optional inboundImageTokens param (stays pure; the term is computed per-wire with that wire's protocol + resolved billing).
  • prepareGoogle gains an upstreamOrigin param so its site can resolve billing like the others.
  • A client-side shrink (native-compaction echo, history edit) shrinks the bound in BOTH terms — fewer messages AND fewer images — preserving the #728C stale-high invariant. A request that genuinely carries images stops being undercounted.

Both compression modes

No wire change in either mode: this only alters the fallback numerator used when lastInputTokens == 0 (upstreams that never report usage). Reporting upstreams are untouched. In plugin mode the proxy suppresses nudge injection (the agent owns compression), so there is no client-visible behavior change there either — the corrected figure only improves internal decisions/logs.

Tests

New regression test in tests/silent-backend-nudge.test.ts (#728 harness pattern): silent backend, 80k window, ~50k chars of code history + two 10k-token inline images. Text-only bound ≈ 63% — below every band (45% first-sight / 70% host emergency escalation / 75% pressure); image-aware numerator ≈ 88% → nudge fires on turn 2; shrunk image-less turn 3 stays silent. Verified the test FAILS with the fix temporarily reverted (turn-2 nudge missing = the bug) and passes with it.

Window note: 80k instead of the shared 60k because the kernel's absolute-token zones (preserveRecentTokens 5k + minPressureBenefit 5k ⇒ ≥ ~10k tokens of compressible mass) squeeze a 60k window so thin that pre-fix/post-fix states collide.

Pre-flight (rebased onto master @ 14d06f1)

  • npm run typecheck ✅
  • npm test: 2195 pass / 0 fail / 2 skipped ✅
  • npm run build ✅

Scope: single-module pure fix: (src/server.ts + one test file). No config schema, persistence, wire-shape, or cross-repo changes. Orthogonal to the parked #1072 hunk / open #1118 billing-semantics discussion (bytes routes are unchanged by both).

Fixes #1119

The min-cap against the current request's inbound upper bound was
text-only (estimateCoreMessagesUpper sums m.text; wire codecs move
images out of CoreMessage.text into sidecars), so whenever the previous
turn's image contribution exceeded the current turn's text growth the
image term was capped away and the silent-backend nudge numerator went
blind to byte-billed multimodal sessions. Carry the current request's
image term (resolved billing) into the bound at all four call sites;
a client-side shrink shrinks the bound in both terms, preserving the
#728C stale-high invariant.

Fixes #1119
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-22_image-term-nudge-bound (f2541b1)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1120

Each push to this PR publishes a new version under the pr-1120 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1120.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[PR] verification report — I verified the existing work rather than redoing it (the fix landed from the #1119 session). Everything checks out:

Branch & base

  • Single commit f2541b1 on 2026-09-22_image-term-nudge-bound, rebased onto current master @ 14d06f1 (no stale-base drift; mergeable_state: clean).
  • Diff is tight: src/server.ts (+15/-5 lines of logic) + one test file (+109/-23). Every line relates to the fix; no whitespace churn, no lockfile/generated-file noise.

Code review

  • All four effectiveTokenCount call sites now pass the image term computed per-wire: anthropic/google use imageBillingFor(opts, upstreamOrigin); openai/responses use billingUpstream ?? upstreamOrigin — identical to the billing resolution already used by their clampOutgoingOutput sites in the same functions, so the bound and the outbound size decision can't disagree on billing mode.
  • effectiveTokenCount stays pure (optional inboundImageTokens = 0 param); prepareGoogle gained the upstreamOrigin param purely to resolve billing like its siblings. Confirmed there are exactly these four call sites — none missed.
  • Change surface is exactly the fallback numerator used while lastInputTokens == 0; reporting upstreams and both compression modes' wire shape are untouched, as claimed.
  • Stale-high invariant holds structurally: the cap is min(est, textBound + imageBound) where both terms come from the current request, so a client-side shrink shrinks both terms together (#728C preserved — the existing test C still passes).

Evidence (ran it myself, not just trusting the pre-flight)

  • Local pre-flight on the PR head: typecheck ✅ · npm test 2195 pass / 0 fail / 2 skipped ✅ · build ✅.
  • Negative control: restored master's src/server.ts while keeping the new test → exactly the #1119 case fails on its turn-2 assertion ("turn 2 must carry the trailing nudge"), the other three cases in the file still pass. Genuine regression test, not a tautology.
  • CI: all 8 checks green (test ubuntu/windows × node 22/24, version-guard, codex-overlay, real-codex zero-token e2e, build-artifact).

Gate assessment (§7.4)
Passes all eight auto-merge criteria: single-module pure fix:, reproducing regression test, green on the rebased head, no config/persistence/wire/cross-repo changes, clean diff, Fixes #1119, no load-bearing infra touched. Per repo policy this is auto-merge eligible; the merge action itself remains yours.

PR: https://github.com/raxianglei/billion-context/pull/1120 (link above is the canonical one; open from the issue header)

一句话中文摘要:核对了这个 PR(图片项进入 effectiveTokenCount 的 stale-high 上限、四处调用点按各自计费模式解析)——代码与描述一致、diff 干净、基线是最新 master、CI 全绿;我本地重跑了全套预检并做了负向对照(revert 修复后新测试必挂、其余用例不受影响),回归测试真实有效,符合 §7.4 自动合并门槛,可以合并。

@ranxianglei
ranxianglei merged commit 66a36f2 into master Sep 22, 2026
8 checks passed
@ranxianglei
ranxianglei deleted the 2026-09-22_image-term-nudge-bound branch October 15, 2026 05:32
cskwork pushed a commit to cskwork/billion-context that referenced this pull request Oct 21, 2026
The anonymous prefix-affinity early return in effectiveTokenCount still returned estimateCoreMessagesUpper(msgs) — text only — even though ranxianglei#1120 made the explicit-identity min-cap branch image-aware and all four prepare sites pass inboundImageTokens. Wire codecs move images out of CoreMessage.text into sidecars, so an image-heavy fork replaying its full raw history read a text-only char count: usage % far below reality, decideNudge structurally unfireable, over-window payload forwarded raw.

Hoist raw above the early return: a fork replays the FULL raw history in one request, so this request's image term IS the whole history's image mass. Conservative direction unchanged (upper bound → compress earlier, never later); the ranxianglei#728 stale-high invariant is preserved untouched.

Regression pin in tests/fork-nudge-trigger.test.ts: an anonymous image-heavy fork whose text-only bound (~58% of an 80k window) sits below every nudge band but whose image-aware bound (~83%) crosses the 75% pressure band — fails pre-fix (no nudge), passes post-fix; preflight stays silent and nothing folds.

Fixes ranxianglei#1137
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

effectiveTokenCount's text-only raw bound caps away the image term from the nudge numerator (silent-backend sessions)

1 participant