Skip to content

[PR] fix(preflight): use calibrated estimate in the no-compression fits check (#6395) - #6460

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-12_preflight-fits-calibrated-estimate
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-12_preflight-fits-calibrated-estimate

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What breaks

When preflight compression triggers (calibrated payload >= limit) but finds 0 viable ranges to compress, the fits check in the no-compression branch used the uncalibrated estimateCoreMessages + overhead + imageTokens, while the trigger decision used the calibrated payload. When the calibration factor k̂ > 1, calibrated > uncalibrated, so the trigger fires (calibrated >= limit) but the fits check says the payload is under the limit → bili forwards a request that is guaranteed to be rejected with a 400 CONTEXT_WINDOW_EXCEEDED.

This is defect (C) in the same failure domain as PR #5899 (fixes #5880): anthropic-wire + wrapper provider + near-wall → 400. PR #5899 fixes (A) overflow-arm evidence demotion and (B) missing anthropic egress clamp. This PR fixes (C) the preflight fits-check caliber inconsistency.

Root cause

src/server.ts:4400-4403 (the else-branch, no compression happened):

// OLD
fits = unknownBaseline ? result.fitsWindow
  : estimateCoreMessages(prepared.processedMessages) + overheadEstimate + imageTokens < limit;

The if-branch (line 4381, compression happened) uses rebuiltCalibrated < limit (calibrated). The else-branch used the uncalibrated estimate. The trigger at line 4093 uses calibratedPayload < limit. When k̂ > 1, calibrated > uncalibrated, so trigger fires but fits says under → forwards guaranteed-400.

Fix

Use the same calibrated estimate in the fits check as the trigger uses:

// NEW
fits = unknownBaseline ? result.fitsWindow : calibratedPayload < limit;

Behavior change (disclosed)

  • Old: when preflight triggers but finds 0 compressible ranges, the fits check compared the uncalibrated estimate against the limit. With k̂ > 1, this could say "fits" even when the calibrated payload is over the limit → forward → 400.
  • New: the fits check compares the calibrated payload against the limit (same as the trigger). If the calibrated payload is over the limit and no compression is possible, the request fails fast with 413 instead of forwarding a guaranteed-400.
  • Impact: sessions with k̂ > 1 (calibration factor > 1, i.e. the local estimator undercounts) that hit the preflight trigger with 0 viable ranges will now get a 413 fail-fast instead of a 400 from upstream. Sessions with k̂ <= 1 are unaffected (calibrated <= uncalibrated, so the old check was already conservative).

Dup check

Fixes #6395

@ranxianglei ranxianglei added bug Something isn't working dsh p1 scope/wiring labels Nov 12, 2026
@ranxianglei ranxianglei changed the title fix(preflight): use calibrated estimate in the no-compression fits check (#6395) [PR] fix(preflight): use calibrated estimate in the no-compression fits check (#6395) Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-12_preflight-fits-calibrated-estimate (6855f65)

Latest build — install this exact version

npm install -g billion-context@0.0.9642-pr.6460.3f5f91b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
6855f65 billion-context@0.0.9642-pr.6460.3f5f91b build
b84a52f billion-context@0.0.9353-pr.6460.6264c9a build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6460.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(bot) 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — 本线程对应的在途 [PR] 只有 #6460 自己(已核对镜像 open [PR] 清单);同失败域的相关项是 PR #5899(fixes #5880),修的是 (A) overflow-arm evidence demotion + (B) anthropic egress clamp,与本 PR 的 (C) preflight fits-check 口径不一致是不同机制,不重复。

Live status (verified just now):

Review in progress: diff is 2 files (src/server.ts +8/−1, new tests/issue6395-preflight-fits-calibrated.test.ts +184). Next: read the diff, chase the windows shard-4 failure, then rebase/merge master and push. Full 8-question conclusion to follow in this thread.

…6395)

The no-compression branch of the preflight fits check judged fit on the
uncalibrated estimateCoreMessages while the trigger used the calibrated
estimate (k̂-scaled). When k̂ > 1 the trigger fires (calibrated ≥ limit)
but the fits check says the payload fits (uncalibrated < limit) → forwards
a guaranteed-400. The if-branch (compression happened) already uses
rebuiltCalibrated; the else-branch now uses calibratedPayload for the
same reason.

Fixes #6395
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review: PR #6460 is superseded by master — recommend closing rather than merging.

I reviewed the PR against current master (99c22d283f) and found all three of its parts already ship on master:

1. The fix is already on master (identical).

  • The PR changes the zero-fold fits check (src/server.ts else-branch) from the uncalibrated estimateCoreMessages(...) + overheadEstimate + imageTokens < limit to calibratedPayload < limit.
  • Master commit f75e45f287 ("fix(preflight): preserve calibrated fit after zero folds", cherry-picked Oct 27, the [PR] fix(preflight): preserve calibrated fit after zero folds #4933 change) makes the same fix as fits = unknownBaseline ? result.fitsWindow : payloadFitsWindow, where payloadFitsWindow (src/server.ts:4093) is defined as exactly calibratedPayload < limit. Semantically identical — master's form just reuses the precomputed const.
  • The PR branch is 178 commits behind master and does not contain f75e45f287; merging master conflicts in exactly this else-branch (both rewrote it, same semantics).

2. Master's regression test is a strict superset of the PR's.

  • The PR adds tests/issue6395-preflight-fits-calibrated.test.ts — one positive-direction case (k̂=10, calibrated over → 413).
  • Master already has tests/preflight-zero-fold-calibration.test.ts (182 lines, 10 scenarios): both directions (fitting→forward at k̂=0.25; inflated→413 at k̂=2), route/model provenance (otherOrigin/otherModel), invalid k̂ (NaN), raw-body fallback, anonymous forwarding, scoped-refusal guard, and the Codex health target. It covers the PR's case and more.

3. The PR's CI failure is already fixed on master.

  • All 3 red shards (windows-22-4, ubuntu-22-1, ubuntu-24-1) fail the same test: issue5631-restart-survival → pi lane (anthropic shape) → 413 "context ~12533 tokens exceeds the model window 12288 … 0 compressible range(s)".
  • Cause: the calibrated zero-fold verdict (this fix) meets an unrealistic fake upstream — the PR head's UpstreamRecorder bills whole-body Math.ceil(bytes / 4) (tests/issue5631-restart-survival.test.ts:235), and the JSON wire envelope is ~2× the core text, so it teaches the per-route calibration factor a false k̂≈1.9. The post-restart baseline (raw ≈6.4K, calibrated ≈12.3K) sits ~120 tokens over the 12288 wall → deterministic 413 on the pi lane.
  • Master commit a223959a50 ("test(5631): bill the fake upstream at token scale with a disjoint anthropic cache partition", landed today) fixes exactly this (token-scale billedTokens + disjoint anthropic cache partition; "suite green 3x locally; calibration ring settles ~1.006"). The PR head lacks this commit — that's why its CI is red.

Recommendation: close PR #6460 as superseded. The identical fix (f75e45f287), a more comprehensive test (preflight-zero-fold-calibration.test.ts), and the CI-failure fix (a223959a50) all ship on master. Merging would only add a redundant test file on top of master's more complete one. Closing is your call (I can't close PRs). If you'd rather keep a #6395-named regression test, I can strip the redundant src hunk and keep just the test — but master's test already covers the case.


八问

  1. 是什么问题:PR 要修 preflight 零折叠(无可折叠区间)分支的 fits 检查口径不一致——触发用校准估计,fits 检查却用未校准估计。
  2. 什么原因导致的:else-branch 用 estimateCoreMessages + overhead + imageTokens(未校准),而触发条件用 calibratedPayload(校准)。k̂>1(本地估计器低估)时校准值 > 未校准值 → 触发命中但 fits 判"能放下" → 转发必然被上游 400 的请求。
  3. 影响范围与严重程度:k̂>1 且触发 preflight 但 0 可折叠区间的会话,收到上游 400 CONTEXT_WINDOW_EXCEEDED 而非本地 413 fail-fast。属特定条件下请求失败(失败模式 400→413 改善,但请求仍拿不到响应);PR 标 p1,我评估更接近 p2(特定条件降级,非普遍不可用),由 owner 定。
  4. 怎么修复的:fits 检查改用与触发条件同一口径的校准估计 calibratedPayload < limit。关键点:此修复已在 master 落地(f75e45f287,语义完全相同),PR 属重复。
  5. 是否有回归问题:PR 改动本身无回归(k̂≤1 或未知时 applyEstimateCalibration 返回原值,与旧行为一致)。但 PR 分支落后 master 178 个提交,其 CI 红是因为缺 master 的 a223959a50(假上游计费口径),不是 PR 代码问题。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及(沿用 [Bug] recommend/apply 对同一区间的 turn-integrity 判定永久不一致:acp_status 反复推荐的区间被 apply 侧整段拒绝(0.1.196,Range would split N turn(s) at the protected-zone boundary) #3236 单一投影判定的既有方向)。
  8. 是否引入了新的配置:不涉及。

判定:不建议合并 — 修复、更全面的回归测试、CI 修复三者均已在 master;合并只会叠加一个冗余测试文件。建议以"已被 master 取代"关闭本 PR。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Status: the review is complete and already posted (the full 8-question review is the comment above) — nothing is pending or blocked.

The session had actually finished: the review landed at 09:24Z and the run was marked completed at 09:26Z. The "stuck" alarm appears to have fired after completion (the process idled during final cleanup once the reply was already posted). No work was lost and no reply is missing.

Conclusion (unchanged): PR #6460 is superseded by master — recommend closing.

  • Identical fix already on master (f75e45f287, payloadFitsWindow ≡ calibratedPayload < limit).
  • More comprehensive regression test already on master (tests/preflight-zero-fold-calibration.test.ts, 10 scenarios vs the PR's 1).
  • The CI red is already fixed on master (a223959a50); the PR head is simply 178 commits behind and lacks that commit.

Next step is yours: close the PR as superseded, or tell me to strip the redundant src hunk and keep only the #6395-named test. No blockers on my side.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] auto-fix stalled: test (windows-latest, 22, 4), test (ubuntu-latest, 24, 1), test (ubuntu-latest, 22, 1) — stopping for human review

1 similar comment
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] auto-fix stalled: test (windows-latest, 22, 4), test (ubuntu-latest, 24, 1), test (ubuntu-latest, 22, 1) — stopping for human review

@ranxianglei
ranxianglei force-pushed the 2026-11-12_preflight-fits-calibrated-estimate branch from b84a52f to 6855f65 Compare November 13, 2026 18:03
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Resolved the merge conflict on PR #6460 and pushed the rebased head.

What happened

  • The PR branched from an older master (merge base 33104b1f95); master has since moved to 99c22d283f. In that window, master already landed the identical fix in the same region of src/server.ts (the no-compression fits check) — using a named variable payloadFitsWindow (defined as calibratedPayload < limit at src/server.ts:4093, used in the else-branch at src/server.ts:4413). The PR's inline calibratedPayload < limit is byte-for-byte equivalent, so the two edits collided on rebase.
  • Resolution (mechanical, no design decision): rebased the PR onto current master and kept master's src/server.ts (dropping the PR's redundant hunk), keeping the PR's new regression test tests/issue6395-preflight-fits-calibrated.test.ts. The PR is now test-only — a regression test for [Root Cause] advisory-update (auto-maintained, do not close manually) | [根因] advisory-update(自动维护,勿手动关闭) #6395, which master already fixes.
  • Verified the test passes locally (targeted run, not the full suite): ok 1 - #6395: calibrated estimate over window + uncalibrated under + 0 compressible ranges → fail-fast 413, not forward (pass 1 / fail 0).
  • Force-pushed the rebased head (b84a52fe7c → 6855f652b6) to 2026-11-12_preflight-fits-calibrated-estimate. The remote head was unchanged before the push (no concurrent agent); used --force-with-lease.

Result: PR merge state is now mergeable: true (state behind — base has newer commits, no conflict). Ready for CI + human merge.


八问

  1. 是什么问题: PR [PR] fix(preflight): use calibrated estimate in the no-compression fits check (#6395) #6460 与 master 冲突(merge state = dirty),阻塞合并,需要解决冲突。
  2. 什么原因导致的: PR 从旧 master(merge base 33104b1f95)分出,期间 master 演进到 99c22d283f,并在同一区域(preflight 无压缩 fits 检查)用等价的命名变量实现(payloadFitsWindow,即 calibratedPayload < limit)先修掉了 [Root Cause] advisory-update (auto-maintained, do not close manually) | [根因] advisory-update(自动维护,勿手动关闭) #6395;PR 的 inline 写法与之语义相同,故 rebase 时冲突。
  3. 影响范围与严重程度: 仅影响该 PR 的合并流程,无任何用户/运行时行为影响。PR 的 src 改动与 master 已合入的修复语义完全等价,行为零差异。
  4. 怎么修复的: 把 PR rebase 到当前 master;冲突处保留 master 的 src/server.ts(丢弃 PR 冗余 hunk),保留 PR 新增的回归测试文件。最终 PR diff 只剩测试文件(184 行),成为对 [Root Cause] advisory-update (auto-maintained, do not close manually) | [根因] advisory-update(自动维护,勿手动关闭) #6395 已修复行为的回归测试。
  5. 是否有回归问题: 无。master 的 src/server.ts 原封未动(保留 HEAD 版本),仅新增测试文件;测试针对行为(k̂=10 校准超窗 + 未校准未超 + 0 可压缩区间 → 413 fail-fast 而非转发),本地定向跑通(pass 1 / fail 0)。
  6. 是否破坏缓存: 不涉及。无缓存/重试/游标/去重键改动。
  7. 是否改了设计方向: 不涉及。纯冲突解决,无架构/边界/数据流变化。
  8. 是否引入了新的配置: 不涉及。无新 env/config/默认值。

判定: 冲突已解决,PR 可合并(mergeable: true,state behind),等待 CI 全量把关 + 人工合并。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant