Repository navigation
fix: keep estimate-grade values out of context sizing denominators (#1839) - #1846
Conversation
…1839) A turn without an upstream usage report (client abort / 5xx) armed a local character-count estimate as the session baseline; the next turn's effectiveTokenCount() then re-derived that bound from the FULL INBOUND history and fed it to the nudge denominator, preflight floors and the UI — inflating context by 12-65x until the next real usage report landed, triggering false EMERGENCY folds (destroying healthy cache prefixes), self-contradictory fail-fast 502/503 after the fold already fit, and impossible [acp-compress-obs] ratios. - effectiveTokenCount returns {tokens, source}; the anchor branch now returns the last usage-grade measured input instead of a calibrated estimate of the unfolded inbound history - armFailureShrink no longer raises lastInputTokens (log-only); preflight measures the real outbound payload every turn anyway - overflow arming is tagged 'overflow-arm' (its own provenance grade), accepted by the effectiveTokenCount fast path and the #496 evidence gate only; billing-grade consumers stay usage-only - preflight fit loop breaks early when a stale floor alone holds currentTokens >= target while the payload's own measure fits (#1492-B) - post-preflight divergence guard: preflight view fits but the rebuilt payload measures over -> forward once for upstream arbitration instead of a self-contradictory hard 502 - [acp-compress-obs] marks untrustworthy baselines instead of printing impossible shrink ratios / zero denominators - stats.contextTokensSource persisted + exposed in /__bili/stats and the web top bar (estimate-grade values marked) Rebased onto 1c87a6b; semantic merge with #1800 image-arbitration in preflightCompressIfNeeded (overflow-arm-aware noOverflowEvidence). Fixes #1839
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1844-pr.1846.805b7e0This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1846.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Reviewing PR #1846 now (fix for #1839, ghost context denominators). CI status so far: e2e / hermetic-registry / lane jobs all green; the 4 test-matrix jobs still in_progress. I will post a structured seven-question conclusion once code + regression review is done. |
…1839) CI on the first revision broke 11 tests in the arming/rescue family: armFailureShrink's baseline write is load-bearing for the #604 emergency-band relay-5xx rescue, and the preflight floors must accept the new overflow-arm provenance or the #1195 in-request refold / #987 next-turn fold lose their trigger. - restore armFailureShrink's raise-only write (estimate-grade); the #1839 ghost is killed by effectiveTokenCount's anchor precedence instead of deleting the arm (a usage-grade anchor always wins over any estimate-grade value) - preflight trigger + fit-loop floors now accept "overflow-arm" alongside "usage" (upstream rejection evidence at that size); estimate-sourced failure arms still floor only while the payload is unmeasured (#1492) - drop the out-of-scope stale-floor early break (it broke #847 honest exhaustion reporting and #301's fitting-payload-forward contract) - tests: G1 re-pins the corrected contract (arm written, anchor sizes turn 3); #987 T2 expects the "overflow-arm" provenance grade; #1492 B back to its original pinned behavior
[bot] 🏷 owner 指示(ework-aio#3):上轮 review 报告的 armFailureShrink doc 注释因果误述,未按「小问题直接修」policy 处置(仅报告、未立项、PR 已合并)。请开一个小 follow-up PR 修正注释措辞:按 review 分析,新读取端闸门下 band 实际经由 min(localInputEstimate, raw) fallback 生效,恢复的 armed 写入其当前活跃消费者是未测量 payload 的 preflight floor(+持久化),而非「armed 值喂给 kernel emergency band」。标题如 |
What & why
A turn that ends without an upstream usage report (client abort/disconnect, upstream 5xx) armed a local character-count estimate as the session baseline (
armFailureShrink→lastInputTokens,source = "estimate"). The next turn'seffectiveTokenCount()found the baseline was not usage-grade and fell through to a bound re-derived from the full INBOUND history (estimateCoreMessagesUpperover all resent messages — chars, not tokens), which then fed the nudge denominator,[acp-compress-obs]and the UI top bar. Observed in #1839: denominators of 2.9M–3.4M against real inputs of 52K–143K (12–65×), frozen for minutes until the next real usage report landed, with:usage=1158%,1323%,1325%) on windows never approached;preflight fail-fast 502: context ~44231 tokens … exceeds the model window 253725after the fold already brought the payload under the window;[acp-compress-obs]lines (shrink 10527%, zero denominators).The fix kills the ghost at the reader: while any usage-grade anchor exists (
lastUsageGradeTokens), every sizing decision sizes on the MEASURED input — estimate-grade values (failure arms, char bounds) can no longer reach the nudge denominator, the display or the floors. The failure arm itself stays functional (#604's emergency-band rescue is load-bearing — CI proved it), and overflow arming gets an honest provenance grade (overflow-arm) instead of masquerading as billing-gradeusage. Preflight — which measures the actual outbound payload every round — remains the hard backstop.Changes
effectiveTokenCount()now returns{ tokens, source }. The anchor branch returns the last usage-grade measured input (lastUsageGradeTokens) instead of a calibrated estimate of the unfolded inbound history; PFA/raw/min branches reportsource: "estimate". The fast path acceptsusageand the newoverflow-armgrade.armFailureShrinkkeeps its raise-only write (lastInputTokens = est,source = "estimate"— the arm emergency shrink on upstream failure to break relay-5xx deadlock #604 relay-5xx emergency-truncate rescue depends on it, and [Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens #1493 tuned its outbound-payload sizing). What changes is that this estimate-grade value can no longer drive a sizing decision while a usage-grade anchor exists (item 1); anchor-less sessions behave exactly as before.armOverflowShrink, both branches) is tagged with its own provenance grade"overflow-arm"instead of billing-grade"usage". Accepted by theeffectiveTokenCountfast path, the PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496 image-arbitration evidence gate, and the preflight trigger/fit-loop floors (upstream rejection evidence at that size — without the floor the 使用pi 压缩后就提示prompt is too long #1195 in-request refold and context window problem continues #987 next-turn fold lose their trigger whenever the local estimate undershoots); deliberately excluded from the other usage-gated consumers (output-budget clamp, thinking-mass measured flag,usageGradeInputBaseline). Estimate-sourced failure arms still floor only while the payload is unmeasured (emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492).[acp-compress-obs]: untrustworthy baselines (≤0, or folded > baseline) log abaseline untrustworthyline instead of impossible ratios /≥0%ceilings.stats.contextTokensSource(usage|estimate) persisted, exposed in/__bili/statsand the web top bar (estimate-grade values marked 估算/estimated).Behavior changes (old → new)
armFailureShrink)lastInputTokens(estimate-grade) AND that value could reach nudge/display/floors on the next turnsource = "usage"(billing-grade, unconditionally trusted)source = "overflow-arm"; trusted by the fast path, #496 gate and preflight floors onlyusage-sourced baselines onlyusageoroverflow-arm(restores the #1195 refold / #987 fold trigger under the new provenance tag)[acp-compress-obs]invalid baselinesshrink 10527%,~N/0,ceiling ≥0%baseline untrustworthyline, ratio/ceiling suppressedcontextTokensSourcefield)Never-reporting backends are unchanged (existing bounded-estimate paths intact); both compression modes covered (shared prepare/nudge path). The pinned contracts of #847 (honest exhaustion reporting), #301 (fitting payload forwards when the summary call 429s) and #1492-B (a usage-grade floor still drives preflight rounds) are untouched and green.
Notes
1c87a6be(master moved during development). One semantic merge with 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 image-arbitration inpreflightCompressIfNeeded: kept 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800's two-phaseimageArbitrationstructure verbatim, replaced its usage-onlynoOverflowEvidencewith the overflow-arm-aware definition. Composition: an overflow-arm ≥ limit makes the flag false → arbitrate off → fold/fail-fast (correct — we hold real overflow evidence); the divergence guard returns before 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800's post-block, consistent forward-once semantics. 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800's suite passes against this branch.32072287(after first-revision CI): the initial revision madearmFailureShrinklog-only and added a stale-floor early break — CI broke 11 tests in the arming/rescue family (arm emergency shrink on upstream failure to break relay-5xx deadlock #604 emergency-band rescue, 使用pi 压缩后就提示prompt is too long #1195/context window problem continues #987 refold trigger, [Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens #1493/fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498 arm provenance, 出现持续压缩失败,无法自动恢复 #847 honest exhaustion, fix: honor anthropic-beta context-1m to size the context window #301 fitting-forward). Both changes reverted/replaced as described above; the ghost stays dead via anchor precedence.npm run typecheckclean; targeted suites 57/57 green across 14 files —issue1839-ghost-denominator(new: G1 failure arm written but anchor sizes turn 3 e2e, G2 anchor read-path pins denominator === last usage report, G3 overflow-arm provenance + rescue chain),issue1569(B1 tightened: anchored sizing === last usage report exactly),issue1595,silent-backend-nudge,fork-nudge-trigger,issue857-image-baseline,issue1800-image-arbitration-compress, plus the six CI-failing filese2e-overflow-selfheal,e2e-relay-5xx-selfheal,issue1195-overflow-refold,issue1493-arm-outbound-payload,issue847-compress-failure-guards,preflight-fail-fast;npm run buildgreen. Global regression left to CI.< 25_000(calibrated inbound estimate) → now exactly the last usage report; context window problem continues #987 T2 assertedsource === "usage"for an arm → now its honestoverflow-armgrade (issue ask 3). emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492 B keeps its original pinned behavior (restored from master after the first revision repinned it wrongly).Fixes #1839