Skip to content

fix: keep estimate-grade values out of context sizing denominators (#1839) - #1846

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-01_ghost-context-denominator
Oct 1, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-01_ghost-context-denominator

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

What & why

A turn that ends without an upstream usage report (client abort/disconnect, upstream 5xx) armed a local character-count estimate as the session baseline (armFailureShrink → lastInputTokens, source = "estimate"). The next turn's effectiveTokenCount() found the baseline was not usage-grade and fell through to a bound re-derived from the full INBOUND history (estimateCoreMessagesUpper over all resent messages — chars, not tokens), which then fed the nudge denominator, [acp-compress-obs] and the UI top bar. Observed in #1839: denominators of 2.9M–3.4M against real inputs of 52K–143K (12–65×), frozen for minutes until the next real usage report landed, with:

  • false EMERGENCY nudges (usage=1158%, 1323%, 1325%) on windows never approached;
  • unnecessary irreversible folds destroying healthy cache prefixes (97% → 11% hit, ~129K tokens re-billed);
  • self-contradictory preflight fail-fast 502: context ~44231 tokens … exceeds the model window 253725 after the fold already brought the payload under the window;
  • impossible [acp-compress-obs] lines (shrink 10527%, zero denominators).

The fix kills the ghost at the reader: while any usage-grade anchor exists (lastUsageGradeTokens), every sizing decision sizes on the MEASURED input — estimate-grade values (failure arms, char bounds) can no longer reach the nudge denominator, the display or the floors. The failure arm itself stays functional (#604's emergency-band rescue is load-bearing — CI proved it), and overflow arming gets an honest provenance grade (overflow-arm) instead of masquerading as billing-grade usage. Preflight — which measures the actual outbound payload every round — remains the hard backstop.

Changes

  1. effectiveTokenCount() now returns { tokens, source }. The anchor branch returns the last usage-grade measured input (lastUsageGradeTokens) instead of a calibrated estimate of the unfolded inbound history; PFA/raw/min branches report source: "estimate". The fast path accepts usage and the new overflow-arm grade.
  2. armFailureShrink keeps its raise-only write (lastInputTokens = est, source = "estimate" — the arm emergency shrink on upstream failure to break relay-5xx deadlock #604 relay-5xx emergency-truncate rescue depends on it, and [Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens #1493 tuned its outbound-payload sizing). What changes is that this estimate-grade value can no longer drive a sizing decision while a usage-grade anchor exists (item 1); anchor-less sessions behave exactly as before.
  3. Overflow arming (armOverflowShrink, both branches) is tagged with its own provenance grade "overflow-arm" instead of billing-grade "usage". Accepted by the effectiveTokenCount fast path, the PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496 image-arbitration evidence gate, and the preflight trigger/fit-loop floors (upstream rejection evidence at that size — without the floor the 使用pi 压缩后就提示prompt is too long #1195 in-request refold and context window problem continues #987 next-turn fold lose their trigger whenever the local estimate undershoots); deliberately excluded from the other usage-gated consumers (output-budget clamp, thinking-mass measured flag, usageGradeInputBaseline). Estimate-sourced failure arms still floor only while the payload is unmeasured (emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492).
  4. Post-preflight divergence guard: when preflight actually folded, its own final view fits, but the fresh rebuild measurement disagrees → forward once for upstream arbitration (house pattern PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496) instead of a hard 502 whose message contradicts itself.
  5. [acp-compress-obs]: untrustworthy baselines (≤0, or folded > baseline) log a baseline untrustworthy line instead of impossible ratios / ≥0% ceilings.
  6. Additive: stats.contextTokensSource (usage | estimate) persisted, exposed in /__bili/stats and the web top bar (estimate-grade values marked 估算/estimated).

Behavior changes (old → new)

Area Old New
Nudge/display sizing on non-usage-grade turns calibrated estimate of the FULL INBOUND history (could be 12–65× the real input) last usage-grade measured input; growth backstopped by preflight's real-payload measurement + #1595 reference re-anchor
Failure arming (armFailureShrink) raised lastInputTokens (estimate-grade) AND that value could reach nudge/display/floors on the next turn same raise-only write (rescue intact), but estimate-grade values are excluded from sizing decisions while a usage-grade anchor exists; anchor-less sessions unchanged
Overflow arming provenance source = "usage" (billing-grade, unconditionally trusted) source = "overflow-arm"; trusted by the fast path, #496 gate and preflight floors only
Preflight trigger/fit-loop floor usage-sourced baselines only usage or overflow-arm (restores the #1195 refold / #987 fold trigger under the new provenance tag)
Fail-fast on post-fold divergence hard 502/503 even when the folded payload fit (self-contradictory message) forward once for upstream arbitration when preflight's own view fits
[acp-compress-obs] invalid baselines printed shrink 10527%, ~N/0, ceiling ≥0% baseline untrustworthy line, ratio/ceiling suppressed
Display single unmarked number top bar marks estimate-grade values (additive contextTokensSource field)

Never-reporting backends are unchanged (existing bounded-estimate paths intact); both compression modes covered (shared prepare/nudge path). The pinned contracts of #847 (honest exhaustion reporting), #301 (fitting payload forwards when the summary call 429s) and #1492-B (a usage-grade floor still drives preflight rounds) are untouched and green.

Notes

Fixes #1839

…1839)

A turn without an upstream usage report (client abort / 5xx) armed a local
character-count estimate as the session baseline; the next turn's
effectiveTokenCount() then re-derived that bound from the FULL INBOUND
history and fed it to the nudge denominator, preflight floors and the UI —
inflating context by 12-65x until the next real usage report landed,
triggering false EMERGENCY folds (destroying healthy cache prefixes),
self-contradictory fail-fast 502/503 after the fold already fit, and
impossible [acp-compress-obs] ratios.

- effectiveTokenCount returns {tokens, source}; the anchor branch now
  returns the last usage-grade measured input instead of a calibrated
  estimate of the unfolded inbound history
- armFailureShrink no longer raises lastInputTokens (log-only); preflight
  measures the real outbound payload every turn anyway
- overflow arming is tagged 'overflow-arm' (its own provenance grade),
  accepted by the effectiveTokenCount fast path and the #496 evidence gate
  only; billing-grade consumers stay usage-only
- preflight fit loop breaks early when a stale floor alone holds
  currentTokens >= target while the payload's own measure fits (#1492-B)
- post-preflight divergence guard: preflight view fits but the rebuilt
  payload measures over -> forward once for upstream arbitration instead
  of a self-contradictory hard 502
- [acp-compress-obs] marks untrustworthy baselines instead of printing
  impossible shrink ratios / zero denominators
- stats.contextTokensSource persisted + exposed in /__bili/stats and the
  web top bar (estimate-grade values marked)

Rebased onto 1c87a6b; semantic merge with #1800 image-arbitration in
preflightCompressIfNeeded (overflow-arm-aware noOverflowEvidence).

Fixes #1839
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-01_ghost-context-denominator (cab341b)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1844-pr.1846.805b7e0

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1846.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewing PR #1846 now (fix for #1839, ghost context denominators). CI status so far: e2e / hermetic-registry / lane jobs all green; the 4 test-matrix jobs still in_progress. I will post a structured seven-question conclusion once code + regression review is done.

…1839)

CI on the first revision broke 11 tests in the arming/rescue family:
armFailureShrink's baseline write is load-bearing for the #604 emergency-band
relay-5xx rescue, and the preflight floors must accept the new overflow-arm
provenance or the #1195 in-request refold / #987 next-turn fold lose their
trigger.

- restore armFailureShrink's raise-only write (estimate-grade); the #1839 ghost
  is killed by effectiveTokenCount's anchor precedence instead of deleting the
  arm (a usage-grade anchor always wins over any estimate-grade value)
- preflight trigger + fit-loop floors now accept "overflow-arm" alongside
  "usage" (upstream rejection evidence at that size); estimate-sourced
  failure arms still floor only while the payload is unmeasured (#1492)
- drop the out-of-scope stale-floor early break (it broke #847 honest
  exhaustion reporting and #301's fitting-payload-forward contract)
- tests: G1 re-pins the corrected contract (arm written, anchor sizes turn 3);
  #987 T2 expects the "overflow-arm" provenance grade; #1492 B back to its
  original pinned behavior
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 完成:PR #1846 可合并(远程 CI 全绿 + 独立本地验证通过)。合并操作按仓库规则由人工执行。

审核过程(两版都逐行审过):

结构化七问:

  1. 是什么问题:任何一轮拿不到上游 usage 报告(客户端 abort / 上游 5xx / 网络失败)后,本地字符数估算值会进入上下文 sizing 的各处分母(nudge、preflight floor、obs、UI 顶栏),把真实 52K–143K 的上下文放大成 2.9M–3.4M(12–65×)并冻结到下一次真实 usage 落地;期间触发假 EMERGENCY 折叠(摧毁健康缓存前缀、约 129K 重计费)、自我矛盾的 fail-fast 502、以及物理不可能的观测值(shrink 10527%、分母为 0)。
  2. 什么原因导致的:provenance 等级混用——只有 billing 级(usage)数值才该占据的槽位被 estimate 级 / 拒绝证据级数值写入(armOverflowShrink else 分支把估算标成 "usage");且基线非 usage 级时 effectiveTokenCount() 的 fallback 从客户端完整入站历史(含未折叠原始内容)重新推导字符上界,把已膨胀的值在同一次决策内再放大约 4×(719521→2939167)。emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492/[OpenCode v2] Configured model context limit never reaches the proxy (x-bili-plugin-context-window not stamped) - silent registry-peek fallback suppresses proactive nudge and misfires emergency shrink #1569 的防御只存在于注释和局部 gate,没堵住这条 fallback。
  3. 怎么修复的:读取端杀幽灵 + 诚实 provenance + 救援功能保全:① 存在 usage-grade 锚点(lastUsageGradeTokens)时,nudge 分母 / 展示 / floor 一律取实测值,estimate 级值到不了任何 sizing 决策;② overflow 武装用独立 "overflow-arm" 等级,preflight trigger+fit floor 显式接受它,context window problem continues #987/使用pi 压缩后就提示prompt is too long #1195 救援链保持完整;③ 失败武装(arm emergency shrink on upstream failure to break relay-5xx deadlock #604)保留 raise-only 写入、标 "estimate",仅在 payload 未测量时可当 floor(emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492 不变);④ 折叠后分歧守卫:preflight 自身视图达标而重建测量不达标时转发一次让上游仲裁,不再发自我矛盾的 502;⑤ obs 对不可信基线打印 baseline untrustworthy,不再输出不可能比值。第一版的两处越界(删除失败武装写入、stale-floor 早停)破坏了 arm emergency shrink on upstream failure to break relay-5xx deadlock #604/使用pi 压缩后就提示prompt is too long #1195/context window problem continues #987 救援链与 出现持续压缩失败,无法自动恢复 #847/fix: honor anthropic-beta context-1m to size the context window #301 契约,已在第二版回退。
  4. 是否有回归问题:未发现回归。逐项审过的既有行为:arm emergency shrink on upstream failure to break relay-5xx deadlock #604 relay-5xx 救援(武装恢复,e2e 端到端钉住截断+恢复)✓;使用pi 压缩后就提示prompt is too long #1195/context window problem continues #987 overflow 救援(同请求 refold / 次轮折叠、retry body 语义)✓;emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492/[Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens #1493/fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498 武装语义 ✓;fix: honor anthropic-beta context-1m to size the context window #301 fail-fast 家族(含 fits+429 放行契约,早停移除后回到 master 原有行为)✓;出现持续压缩失败,无法自动恢复 #847 honest exhaustion 措辞 ✓;[OpenCode v2] Configured model context limit never reaches the proxy (x-bili-plugin-context-window not stamped) - silent registry-peek fallback suppresses proactive nudge and misfires emergency shrink #1569/nudge: no-usage recovery leaves stale reference after network or 5xx failure #1595 nudge 锚定/重锚 ✓;[Question] 非白名单上游(如 DeepSeek)图片按 b64/4 估算,实测高估约 100×;且该估算被写回 lastInputTokens 并自愈成 confirmed 窗口,使 #496 逃生门永久失效 #857 image hatch、疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 image arbitration ✓;never-reporting 后端与 PFA 匿名路径不变 ✓;persist 向后兼容(附加字段、白名单校验)✓。两点残留说明(均非回归):陈旧 usage floor 的轮次仍会尝试一次摘要调用后才放行 fitting payload(可能 429)——与 master 行为相同,既有取舍;持久 opaque-5xx 死锁形态依赖运维声明真实窗口——有意设计,已由测试钉住。
  5. 是否破坏缓存:否。本修复消除的正是「幽灵分母触发不必要折叠 → 前缀作废」这条计费损失路径。持久化格式只新增字段 contextTokensSource(旧文件缺省 = 未知,向后兼容);消息引用 / 游标 / 去重键均未触碰。
  6. 是否改了设计方向:没有转向。分级信任(usage > overflow-arm > estimate)成为随值流动的显式数据,让代码与仓库注释的原意(估算只可作压缩上界,不可用于展示/决策)一致;最终形态比第一版更贴近 ask fix: address all code review findings (condense, routing, session-id, dead code) #2 原文。
  7. 是否引入了新的配置:无新 env/config/CLI。仅附加 API/UI 字段 contextTokensSource(usage|estimate|overflow-arm),顶栏对估算级数值标注「估算/estimated」。

小问题发现(仅报告,不在本 PR 处理):

  • armFailureShrink 的 doc 注释把 arm emergency shrink on upstream failure to break relay-5xx deadlock #604 救援因果归给「armed 值喂给 kernel emergency band」;但在新读取端闸门下(fast path 排除 estimate 级),band 实际经由 min(localInputEstimate, raw) fallback 生效,恢复的 armed 写入当前活跃消费者是未测量 payload 的 preflight floor(+ 持久化)。可观察行为不变且有 e2e 钉住,建议后续清理时修正注释措辞。
  • 流程建议:第一版本地 pre-flight 未覆盖受影响的 6 个套件(由 CI 兜住)。凡涉及 source-tag 语义变更,建议在 PR 正文中例行列出 lastInputTokensSource 的全部消费者与受影响套件清单。

结论:方向正确、根因修掉、两条救援链保全、既有契约全部钉住、CI 全绿——可合并。请人工执行合并:#1846

@ranxianglei
ranxianglei merged commit 815027f into master Oct 1, 2026
13 checks passed
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 owner 指示(ework-aio#3):上轮 review 报告的 armFailureShrink doc 注释因果误述,未按「小问题直接修」policy 处置(仅报告、未立项、PR 已合并)。请开一个小 follow-up PR 修正注释措辞:按 review 分析,新读取端闸门下 band 实际经由 min(localInputEstimate, raw) fallback 生效,恢复的 armed 写入其当前活跃消费者是未测量 payload 的 preflight floor(+持久化),而非「armed 值喂给 kernel emergency band」。标题如 docs: correct armFailureShrink rescue causality comment (#1846 follow-up),正文带 来源 标记。可观察行为不变,勿动逻辑。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant