Skip to content

fix(preflight): an estimate may not block the forward — probe once for upstream evidence (#2313) - #2361

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-07_preflight-probe-forward
Oct 8, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-07_preflight-probe-forward

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

The livelock (#2313)

A 10,341-message/61.5MiB session behind a local OpenAI-compatible shim (~200 B/token billing) lost its usage baseline on a route change (random loopback port ⇒ new origin ⇒ #1933 F2 demotion) and fell to lastInputTokensSource="estimate". The trigger then read the chars/4 estimate at face value:

  • preflight estimated 7.3M–10.3M vs the real 305K input (~24–34× over);
  • calibration cannot bridge it: k̂ is one-way clamped 0.25–1, consistent samples below CALIBRATION_SAMPLE_MIN are discarded — 1/24 is unrepresentable;
  • so preflight blocked every forward until folding finished, folding cannot finish inside the client's ~300s stream-idle abort, and the loop never lets a forward through: 0 successful forwards in 12h, while every failed turn ratcheted the estimate meter higher.

The rule (implements option (i) of the #2313 audit as a standing invariant)

An estimate may not block the forward. When the session carries a nonzero baseline NONE of which is upstream evidence for the current route (estimate-sourced, or usage/overflow-arm demoted by #1933 F2 — in both cases baselineFloor === 0 after the existing #1492/#1933 gating), forward once and let the upstream arbitrate size instead of folding first:

  • success ⇒ settleUsageReport writes a usage baseline for this route;
  • 4xx overflow ⇒ armOverflowShrink arms one;
  • either way honest metering re-arms and fail-fast semantics resume from the next request.

Cost bound: one wasted round trip in the worst case (a 4xx returns in seconds, well within stream patience). This generalizes the #496/#1800 forward-once image arbitration to the whole estimate channel. Fresh sessions (lastInputTokens <= 0, #553 anonymous forks) keep the fold-first judgment — they hold no known-wrong meter to replace, and their replays mint fresh session ids so probe evidence would not accumulate.

Companion fix: stamp the arm origin

armOverflowShrink wrote lastInputTokensSource="overflow-arm" but never lastInputTokensOrigin — the only writer was settleUsageReport. Without the stamp the arm inherits a stale settle origin, #1933 F2 demotes it on the next prepare, the #1195 in-request refold re-probes the raw body, takes a second 400, and the session locks again. Both arm branches now stamp the origin from the rejecting upstream (normalizeUpstreamOrigin(upstreamUrl)).

What this does NOT fix

Tests

tests/issue2313-probe-forward.test.ts (e2e, chat lane against a divergent-caliber mock billing chars/40):

  • a — estimate-only trigger forwards the raw body (1 forward, 0 summary calls) and the response settles a usage baseline at real scale (~3K, not 7.3M);
  • b — probe 400 with a stated window arms current-route evidence; the 使用pi 压缩后就提示prompt is too long #1195 refold folds within the request and the folded retry succeeds (client 200). Pins the origin stamp: mutated away, this test fails with a passthrough 400;
  • c — with a current-route usage baseline the fold-first behavior is unchanged.

Mutation-checked: disabling the probe gate fails a+b; faking the arm origin fails b. Full-suite A/B against master (this machine, 40 pre-existing environmental failures in the opencode/omp/pi/dsh lanes): identical failure sets before/after; the 3 new tests pass.

Fixes the livelock half of #2313.

…r upstream evidence (#2313)

A 10,341-message session behind a local OpenAI-compatible shim (~200
B/token billing) lost its usage baseline on a route change and fell to
lastInputTokensSource=estimate; the trigger then read the chars/4
estimate at face value (7.3M-10.3M against a real 305K input, ~24-34x
over — k̂ cannot express 1/24: one-way clamp 0.25-1, samples below
CALIBRATION_SAMPLE_MIN discarded), blocked every forward until folding
finished, folding cannot finish inside the client's ~300s stream-idle
abort, and the loop never lets a forward through: 0 successful forwards
in 12h while failed turns ratcheted the estimate meter higher.

Rule: when the session carries a nonzero baseline NONE of which is
upstream evidence for THIS route (estimate-sourced, or usage/overflow-arm
demoted by #1933 F2 — baselineFloor === 0), forward once and let the
upstream arbitrate size instead of folding first. Success settles a
usage baseline; a 4xx overflow arms one. Either outcome re-arms honest
metering and fail-fast semantics resume from the next request.
Generalizes the #496/#1800 forward-once image arbitration to the
estimate channel. Fresh sessions (lastInputTokens <= 0 / #553) keep the
fold-first judgment — no known-wrong meter to replace.

Companion: armOverflowShrink now stamps lastInputTokensOrigin from the
rejecting upstream — without it the arm keeps a stale settle origin,
#1933 F2 demotes it on the next prepare, the #1195 in-request refold
re-probes the raw body, takes a second 400 and the session locks again.

Fixes the livelock half of #2313 (evidence acquisition). The metering
ratchet audit stays tracked in #2274.
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-07_preflight-probe-forward (c37b346)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2531-pr.2361.e299f48

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2361.tgz
npm install -g package

Install instructions are refreshed on each push.

The PR stamps lastInputTokensOrigin in both armOverflowShrink branches,
but only the stated-window branch was pinned: test b's rejection carries
a parseable window, so removing the fallback-branch stamp (the
min(declared, estimate) arm) left every existing test green. Test d
rejects with 'context length exceeded' (isOverflow, no window number),
arms the fallback branch and asserts the #1195 refold still recovers —
mutation-checked: dropping that stamp fails d alone.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #2361 review 结论

先说一件已直接做的事:向 PR 分支补了一个 test-only commit c37b34662。PR 声称 "mutation-checked: faking the arm origin fails b",但 b 只钉住了window 分支的 stamp——本地验证删掉 fallback 分支(无可解析 window、min(declared, est) arm)的 stamp 后三个测试全绿,即该 stamp 无回归兜底。新增 test d(context length exceeded 拒绝 → fallback arm → #1195 refold 恢复),mutation 复核:删该行 d 单独失败,a/b/c 不受影响。

结构化七问

  1. 是什么问题:livelock——baseline 降为 estimate-only(路由变更 → DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 F2 demotion)后,preflight 把 chars/4 estimate(高估 24–34×)当事实,每次 forward 前都先折;折叠在 client ~300s stream patience 内完不成 → 12h 0 次成功 forward,且每次失败把 meter 推得更高。
  2. 什么原因导致的:两层。主因:baseline 无当前路由证据时(baselineFloor === 0)preflight 仍按 estimate fail-fast,而 calibration 桥不过 24–34× 的缺口(k̂ 单向钳制 0.25–1、低于 CALIBRATION_SAMPLE_MIN 的一致样本被丢弃)→ 死循环。次因:armOverflowShrink 写 source="overflow-arm" 却不写 lastInputTokensOrigin,拿到的 overflow 证据在下次 prepare 被 F2 demote,使用pi 压缩后就提示prompt is too long #1195 in-request refold 重发 raw body 吃第二个 400,再次锁死。
  3. 怎么修复的:把 billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 audit 的 option (i) 落成 standing invariant——"estimate 不得阻塞 forward":非零 baseline + 无当前路由证据 ⇒ forward 一次让上游仲裁大小(成功 → settleUsageReport 写 usage baseline;overflow 4xx → armOverflowShrink 装证据);两个 arm 分支都从拒绝方 upstream 盖 origin 戳,使证据在本路由存活。
  4. 是否有回归问题:逐项清点过——(a) 不触发/limit≤0/fresh session(lastInputTokens≤0、unknownBaseline)/kernel-transform-failure(processedMessages 空)全部走旧路径,fold-first 保留;(b) 有当前路由证据的会话行为字节级不变(test c 钉住);(c) fail-fast 未弱化为吞错:probe forward 走正常 forward() 路径,overflow 仍触发 arm+refold,且 origin 戳使其下次正确恢复;(d) 阈值/时序/输出格式/缓存均无变化。一处精确化备注:对不匹配 overflow pattern 的尺寸类拒绝,armFailureShrink(src/server.ts:6918)装的是 source="estimate"、floor 保持 0,probe 会每轮重复一次——PR 描述里 "worst case one wasted round trip" 只对 recognized-overflow/success 路径成立;该场景旧行为是多轮折叠尝试(可能超出 stream patience),新行为仍严格更优,不算回归。
  5. 是否破坏缓存:不涉及。session identity、ref space、head system message、prefix cache 均不受影响(probe forward 发的就是任何一次普通 forward 的同款 body)。
  6. 是否改了设计方向:没有转向。这是 billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 audit option (i) 的落地(owner 拍板);metering-ratchet 半属于已开的 fix(preflight): trust only upstream-evidenced baselines in the entry meter (#2313) #2316(entry-meter provenance gate)——两者互补,同函数内不同机制,合并顺序无关,建议一并合并。
  7. 是否引入了新的配置:无。

其余检查:rebase 至最新 master(2495ad2)无冲突(master 侧 #2302 只动 nudge 注入区,与本 PR hunk 相距甚远);双压缩模式推理完成(gate 在共享 pipeline 内、模式对称,settle/arm 在响应侧);diff 纯增量(+33/+255),无无关改动;测试 hermetic + listen(0)、确定性、ungated 进 CI。本地定向验证:新套件 4/4 通过,相邻 9 套(issue1933/1195/496/1800/1839/context-overflow/e2e-overflow-selfheal/2122/2189)58/58 通过。CI 状态沙箱读不到,以 CI 为最终门禁。

判定:可合并。 按 §7.4 本 PR 触碰 preflight forward/fail-fast 决策语义,需 human merge;建议与 #2316 一起合(同一 issue 的互补两半)。

@ranxianglei
ranxianglei merged commit 149d10e into master Oct 8, 2026
27 checks passed
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.122-master.149d10e (rolling master tag).

npm install -g billion-context@0.0.122-master.149d10e  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant