Repository navigation
fix(preflight): an estimate may not block the forward — probe once for upstream evidence (#2313) - #2361
Conversation
…r upstream evidence (#2313) A 10,341-message session behind a local OpenAI-compatible shim (~200 B/token billing) lost its usage baseline on a route change and fell to lastInputTokensSource=estimate; the trigger then read the chars/4 estimate at face value (7.3M-10.3M against a real 305K input, ~24-34x over — k̂ cannot express 1/24: one-way clamp 0.25-1, samples below CALIBRATION_SAMPLE_MIN discarded), blocked every forward until folding finished, folding cannot finish inside the client's ~300s stream-idle abort, and the loop never lets a forward through: 0 successful forwards in 12h while failed turns ratcheted the estimate meter higher. Rule: when the session carries a nonzero baseline NONE of which is upstream evidence for THIS route (estimate-sourced, or usage/overflow-arm demoted by #1933 F2 — baselineFloor === 0), forward once and let the upstream arbitrate size instead of folding first. Success settles a usage baseline; a 4xx overflow arms one. Either outcome re-arms honest metering and fail-fast semantics resume from the next request. Generalizes the #496/#1800 forward-once image arbitration to the estimate channel. Fresh sessions (lastInputTokens <= 0 / #553) keep the fold-first judgment — no known-wrong meter to replace. Companion: armOverflowShrink now stamps lastInputTokensOrigin from the rejecting upstream — without it the arm keeps a stale settle origin, #1933 F2 demotes it on the next prepare, the #1195 in-request refold re-probes the raw body, takes a second 400 and the session locks again. Fixes the livelock half of #2313 (evidence acquisition). The metering ratchet audit stays tracked in #2274.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2531-pr.2361.e299f48This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2361.tgz
npm install -g packageInstall instructions are refreshed on each push. |
The PR stamps lastInputTokensOrigin in both armOverflowShrink branches, but only the stated-window branch was pinned: test b's rejection carries a parseable window, so removing the fallback-branch stamp (the min(declared, estimate) arm) left every existing test green. Test d rejects with 'context length exceeded' (isOverflow, no window number), arms the fallback branch and asserts the #1195 refold still recovers — mutation-checked: dropping that stamp fails d alone.
[bot] 🏷 PR #2361 review 结论 先说一件已直接做的事:向 PR 分支补了一个 test-only commit 结构化七问
其余检查:rebase 至最新 master(2495ad2)无冲突(master 侧 #2302 只动 nudge 注入区,与本 PR hunk 相距甚远);双压缩模式推理完成(gate 在共享 pipeline 内、模式对称,settle/arm 在响应侧);diff 纯增量(+33/+255),无无关改动;测试 hermetic + 判定:可合并。 按 §7.4 本 PR 触碰 preflight forward/fail-fast 决策语义,需 human merge;建议与 #2316 一起合(同一 issue 的互补两半)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.122-master.149d10e # this exact build
npm install -g billion-context@master # always the newest merged build
|
The livelock (#2313)
A 10,341-message/61.5MiB session behind a local OpenAI-compatible shim (~200 B/token billing) lost its usage baseline on a route change (random loopback port ⇒ new origin ⇒ #1933 F2 demotion) and fell to
lastInputTokensSource="estimate". The trigger then read the chars/4 estimate at face value:CALIBRATION_SAMPLE_MINare discarded — 1/24 is unrepresentable;The rule (implements option (i) of the #2313 audit as a standing invariant)
An estimate may not block the forward. When the session carries a nonzero baseline NONE of which is upstream evidence for the current route (estimate-sourced, or usage/overflow-arm demoted by #1933 F2 — in both cases
baselineFloor === 0after the existing #1492/#1933 gating), forward once and let the upstream arbitrate size instead of folding first:settleUsageReportwrites a usage baseline for this route;armOverflowShrinkarms one;Cost bound: one wasted round trip in the worst case (a 4xx returns in seconds, well within stream patience). This generalizes the #496/#1800 forward-once image arbitration to the whole estimate channel. Fresh sessions (
lastInputTokens <= 0, #553 anonymous forks) keep the fold-first judgment — they hold no known-wrong meter to replace, and their replays mint fresh session ids so probe evidence would not accumulate.Companion fix: stamp the arm origin
armOverflowShrinkwrotelastInputTokensSource="overflow-arm"but neverlastInputTokensOrigin— the only writer wassettleUsageReport. Without the stamp the arm inherits a stale settle origin, #1933 F2 demotes it on the next prepare, the #1195 in-request refold re-probes the raw body, takes a second 400, and the session locks again. Both arm branches now stamp the origin from the rejecting upstream (normalizeUpstreamOrigin(upstreamUrl)).What this does NOT fix
lastInputTokenswithout emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492 source gating) — that is fix(preflight): trust only upstream-evidenced baselines in the entry meter (#2313) #2316's entry-meter provenance gate + [调查] 窗口/计量口径审计:会话中途 model/window 探测翻转、上游回账 vs 本地估算偏差、回滚累计 token 量(#2110/#1921) #2274's metering audit.Tests
tests/issue2313-probe-forward.test.ts(e2e, chat lane against a divergent-caliber mock billing chars/40):Mutation-checked: disabling the probe gate fails a+b; faking the arm origin fails b. Full-suite A/B against master (this machine, 40 pre-existing environmental failures in the opencode/omp/pi/dsh lanes): identical failure sets before/after; the 3 new tests pass.
Fixes the livelock half of #2313.