Repository navigation
[PR] fix(preflight): fail fast locally when a post-drift transform failure leaves an over-window raw body (#5223) - #5275
ranxianglei wants to merge 5 commits into
Conversation
…ansform failed (#5223) After a cross-provider re-serialization drifts the whole history, repair mode re-emits the unfolded originals AND a kernel-transform failure leaves prepared.processedMessages empty while the outbound body IS the raw client history. preflightCompressIfNeeded judged the zero-fold-result fit on the empty array ("0 + overhead < window") and forwarded a ~50MB / ~5.5M-token body upstream — guaranteed 413 plus a wasted upload per turn. - zero-range fits judgment now mirrors outboundPayloadBreakdown's #1498-F2 fallback and the compressedRanges>0 branch: measure the shipping view (processedMessages, falling back to originalMessages when empty) - the #1839 arbitration escape now carries the same doomed-bytes bar (max(2MB, window*7.5B/tok)) as its sibling escapes (#2313/#4056): a rebuilt body past the bar fails fast instead of "letting the upstream arbitrate" - the fail-fast verdict names the #3641 fold pause explicitly when folding is epoch-paused, so hosts see why no fold could be applied this turn Behavior change (old -> new): a session whose transform failed and whose raw body exceeds the effective window used to ship that body upstream (413 after a full upload); it now fails fast locally (502 retryable=false while folding is paused, 413 on deterministic exhaustion). Fitting raw bodies and all non-empty-view paths are unchanged.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9519-pr.5275.9dc5246This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5275.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…ocal # Conflicts: # src/server.ts
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) The full run summary for this conflict-fix lane is already posted above (comment by ci-engineer-r1, ~12:37 UTC): master merged into the branch, both Fresh status since that reply (verified live from the mirror cache at 12:45 UTC): CI on head |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: qwen/qwen3.8-27b)
APPROVE — clean, well-scoped preflight fail-fast fix. No kernel / config / wire-shape changes; both holes are pinned by regression tests with both-direction controls; every behavior change is disclosed old→new. Reviewed independently against the diff (not the issue's own conclusions).
Independently verified:
- Symbols resolve:
DOOMED_PROBE_BYTES_PER_TOKEN = 7.5/DOOMED_PROBE_ABSOLUTE_BYTES = 2_000_000(src/server.ts:3312-3313),estimateCoreMessagesUpperBytes(src/preflight.ts:544),epochConfirmPending(src/fold-reconcile.ts:212); imports present at src/server.ts:89-90. - Data flow is correct:
outboundisrebuiltwhencompressedRanges > 0, else staysprepared; each new fallback reads the object that actually ships. The transform-failure catch (src/server/prepare-openai.ts:561-564) resetsprocessedMessages = []whileoriginalMessageskeeps the full parsed history — so measuringoriginalMessagesproxies the shipped raw body. Same idiom already used at src/server.ts:4335/4553/4598 and prepare-openai.ts:619/624. - Change 1 activates ONLY when
processedMessagesis empty (the transform-failure shape); non-empty paths are byte-identical to before → no regression surface. - Change 2 reuses the established doomed-bytes bar (#2490/#2313): a core-text body past
max(2MB, window×7.5 B/tok)cannot fit on any tokenizer → no false refusal. - Change 3 is additive message text only (zero behavior change) and reads through the sanctioned
epochConfirmPendinghelper, never the raw metadata key. - Tests match the 17-arg
preflightCompressIfNeededsignature exactly. Test 1 fails without change 1 (it would forward the raw history); test 4 fails without change 2 (it would ride the #1839 arbitration escape); tests 2–3 are controls pinning the measured over-window path and the fitting-body passthrough.
八问:
- 是什么问题:跨提供方重序列化触发漂移后,repair 模式把未折叠的原始历史重放到线上;若此时 kernel transform 又失败(
processedMessages被清空),preflight 用空数组判 fit、读成"fits",把 ~50MB / 5.5M token 的原始请求体整包转发上游——必然 413,且每次重试、每一轮都白传一次。 - 什么原因导致的:零折叠分支的 fits 判定只量了
prepared.processedMessages(失败时为空),没有像兄弟分支那样回退到真正出站的视图(originalMessages);#1839 仲裁逃生门也缺一道 doomed-bytes 闸。 - 影响范围与严重程度:仅当「transform 失败 + 原始体超窗 + 该轮零折叠」三者叠加(典型是漂移 repair 轮)才触发——特定条件下的严重浪费(30–42s 上传 × N 次重试 × 每轮),非全局不可用;正常非空路径完全不受影响。
- 怎么修复的:让 fits 判定与 #1839 逃生门都改判"实际出站的视图"(空则回退
originalMessages),并给 #1839 逃生门加上与兄弟逃生门一致的 doomed-bytes 闸(超max(2MB, window×7.5B/tok)直接本地 fail-fast 而非转发仲裁);漂移暂停时在 verdict 里点名 #3641 epoch pause,给出可操作指引。 - 是否有回归问题:无。改动只在
processedMessages为空时生效(transform 失败形态),非空路径逐字节不变;doomed-bytes 闸不可能误拒(超该字节数在任意 tokenizer 下都装不进窗口);第 3 处纯加消息文本、零行为变更。 - 是否破坏缓存:不涉及(未改任何缓存键 / 游标 / 去重逻辑)。
- 是否改了设计方向:不涉及(沿用既有 #1498-F2 回退惯用法与 #2490/#2313 字节闸,无新架构 / 边界)。
- 是否引入了新的配置:无新 env / config / 默认值。
行为变更(已按交付规范披露 old→new):
| 状态 | 旧 | 新 |
|---|---|---|
| transform 失败 + 原始体超窗 + 零折叠 | 整包转发上游(上传后 413) | 本地 fail-fast:漂移暂停时 502 retryable=false / 确定性耗尽时 413 |
| 重建记账 fit 但出站体超 doomed 闸(#1839 逃生门) | 转发一次"交上游仲裁" | 本地 fail-fast |
| transform 失败 + 原始体 FITS | 原样转发 | 不变 |
| 所有非空视图路径 | — | 不变 |
判定:可合(merge 仍由人工执行)。CI 绿(head ed35b7e)。
Posting as a COMMENT review: GitHub blocks self-APPROVE (the posting identity is the PR author), so this records the verdict without flipping the approval state — merge stays human-only regardless. The preceding one-word "probe" entry on this PR was an accidental endpoint diagnostic made while preparing this review; disregard it.
(ai: qwen/qwen3.8-27b) review-worker-r1 完成:PR #5275(本线程)已端到端审完,独立核对(未照抄 issue 结论)三处改动的符号解析、数据流与测试有效性后判定 APPROVE。已在 GitHub 提交 COMMENT review(review_id 5843661017)。 说明两点:① 发布身份即 PR 作者(ranxianglei),GitHub 禁止自我 结论要点:改动仅 host 侧 |
# Conflicts: # src/server.ts
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Conflict re-resolved (second pass) — PR is mergeable again. Note: my earlier resolution ( What conflicted and how it was resolved: Only ONE textual conflict this time — the The rest auto-merged cleanly, so I verified the semantics rather than trusting the text: master's #5974 corrective second pass runs strictly UPSTREAM of the #1839 arbitration arm (it improves the fold before the arm's decision point and updates Verification (targeted suites only, per local-test discipline): typecheck clean; 21/21 green across 结构化结论(七问)
收工判定:冲突已解, |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: qwen3.8-27b) Re-reviewed on the new head after the branch was rebased onto current master.
Head moved ed35b7ef9 → 7e9a66d86. The new commit is a pure Merge remote-tracking branch 'origin/master' into pr5275-local (parents ed35b7ef9 + 58a8796224) — 77 commits of upstream master, no new work in this PR. Base is now master@58a8796224.
What I verified on the new head:
- The PR's
src/server.tsadded lines (all 55) and removed lines are byte-for-byte identical between4dea18eabf..ed35b7ef9(old base..head) and58a8796224..7e9a66d86(new base..head); same for the new test file. The visible server.ts delta across heads is entirely master's own changes in the window, not this PR's. - Every referenced symbol still exists at the new head —
DOOMED_PROBE_BYTES_PER_TOKEN/DOOMED_PROBE_ABSOLUTE_BYTES(src/server.ts:3330-3331),estimateCoreMessagesUpperBytes(src/preflight.ts:546),epochConfirmPending(src/fold-reconcile.ts:212); imports intact (src/server.ts:89-90). - Re-read all three hunks against the newer master context: the else-branch
shipMsgsfallback (src/server.ts:4434), the #1839-arm doomed-byte gate (src/server.ts:4542-4575), and the additive fail-fast drift-pause text (src/server.ts:4731). Logic and ordering unchanged; no semantic drift introduced by the re-merge.
Verdict: APPROVE stands. CI green on 7e9a66d86, mergeable:true; the only remaining gate is human merge. No code changes needed — nothing mechanical to fix, no open ends.
八问(新头 7e9a66d 复审,结论与上一轮一致):
- 是什么问题:跨 provider 重序列化漂移后,若 kernel 变换也失败(processedMessages 被清空),preflight 用空数组判 fit,把 ~50MB / 550万 token 的原始历史原样转发上游,必然 413。
- 什么原因导致的:fit 判定度量的是空数组而非实际要发的视图;#1839 逃生分支对「重建失败、原始体兜底」这一形状没有字节兜底门。
- 影响范围与严重程度:仅当「漂移修复重放原始史 + kernel 变换失败」叠加时触发;触发即整轮不可用且每轮重试都白传几十 MB(p1 可用性缺陷)。非该形状的路径零变化。
- 怎么修复的:三处——① else 分支改用实际要发的视图判 fit(processedMessages 为空时回退 originalMessages);② #1839 逃生前加 doomed-bytes 门,超阈值直接本地 fail-fast 而非转发;③ fail-fast 文案在折叠暂停时补一句可操作说明(纯文案追加)。
- 是否有回归问题:无。非空视图路径字节级不变;doomed 门只在「重建成功但核心文本超窗」时收紧转发;fail-fast 文案纯追加。已逐项过既有行为,无高危模式命中。
- 是否破坏缓存:不涉及(不改缓存 / 游标 / 去重键)。
- 是否改了设计方向:不涉及(host 侧 preflight 局部修正,无架构转向)。
- 是否引入新配置:无新 env / config / 默认值。
判定:可合(仅剩人工合并)。
# Conflicts: # src/server.ts
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Conflict re-resolved (third pass) — PR is mergeable again. Same pattern as the previous two passes: my last push ( What conflicted — and this one was semantic, not mechanical: One conflict, in the zero-fold Resolution: took master's side — the PR's hunk 1 is superseded, not lost. The PR's other two hunks are intact and still needed (no conflict there): the #1839 arbitration arm's doomed-bytes bar (withhold past the bar → fall through to the generic fail-fast) and the fold-drift epoch-pause explanation in the fail-fast verdict (#3641). I left a short comment at the resolution site recording that #5223's zero-fold case is covered by the entry breakdown's fallback, so the supersession is visible in the diff. Verification (targeted suites only, per local-test discipline): typecheck clean; 37/37 green across the PR's own 结构化结论(七问)
收工判定:冲突已解, |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: qwen3.8-27b) Re-reviewed on the new head 25f15c1aa after the branch was rebased onto current master. This re-merge changed the PR's content — unlike the prior head (where the fix was byte-identical), I re-verified the actual contribution rather than assuming carry-over.
Head moved 7e9a66d86 → 25f15c1aa (pure merge origin/master, 184 commits; base now 99c22d283).
What the re-merge did to the fix
Master's window independently refactored the preflight image-token path and rewrote the zero-fold else-branch to fits = unknownBaseline ? result.fitsWindow : payloadFitsWindow. That superseded the PR's original hunk 1 (the explicit shipMsgs + estimateCoreMessages(shipMsgs) + overheadEstimate + imageTokens < limit). The PR's hunk 1 is now only a 3-line comment noting that master's payloadFitsWindow covers the #5223 zero-fold case.
Verified: master's payloadFitsWindow genuinely covers the #5223 zero-fold transform-fail case (no lost fix)
textEstimate(src/server.ts:3937) is destructured fromoutboundPayloadBreakdown(src/server.ts:3885), which uses the #1498-F2 raw-body fallback (src/server.ts:3541:msgs = processedMessages.length > 0 ? processedMessages : originalMessages).- So
calibratedPayload(src/server.ts:3938) →payloadFitsWindow(src/server.ts:4094) measures the raw body when the transform emptiesprocessedMessages; an over-window raw body fails the gate and falls to the local fail-fast. Master's version is a calibrated superset of the PR's raw estimate and is consistent with thecompressedRanges>0branch — no regression.
The PR's remaining code contributions (intact, correct, coherent against master's new image refactor)
epochConfirmPendingimport (src/server.ts:88).- #1839-arm doomed-bytes gate (src/server.ts:4515-4556): wraps master's measured tier inside
if (shipDoomBytes <= max(DOOMED_PROBE_ABSOLUTE_BYTES, limit*DOOMED_PROBE_BYTES_PER_TOKEN)); a rebuilt body past the bar is withheld and fails fast locally instead of riding the #1839 forward-once escape. Master does not have this gate (its #1839 arm goes straight to the measured tier), so this is a genuine, non-redundant fix. - Fail-fast drift-pause text (src/server.ts:4710-4715): additive; makes the verdict actionable when folding is drift-paused.
Tests — all 4 in tests/issue5223-drift-unmeasured-forward.test.ts still pass (CI green; T1/T2/T3 exercise the else-branch via payloadFitsWindow, T4 exercises the #1839 gate). File unchanged between heads.
Scope — 2 files, +207/-25; no kernel/, no config surface, no wire-shape, no version change. mergeable:true, mergeable_state:blocked (human merge only).
Behavior deltas (old→new)
- (a) zero-fold transform-fail fit judgment: PR's raw
estimateCoreMessages(shipMsgs)+overhead+image→ master's calibratedpayloadFitsWindow; identical outcome for the #5223 shape (5.5M ≫ 1M at any k̂∈[0.25,4]), more consistent otherwise. - (b) #1839 escape withheld past the doomed bar (local fail-fast) — unchanged from prior heads.
- (c) fail-fast message gains an actionable drift-pause note — additive.
Verdict: APPROVE stands (self-approve blocked — reviewer identity = PR author — so this is posted as a review comment). No code changes needed; nothing mechanical to fix, no open ends.
八问(新头 25f15c1 复审;本次 re-merge 改变了 PR 内容,已重新独立核验,非沿用上轮结论):
- 是什么问题:跨 provider 重序列化漂移后,kernel 变换失败(processedMessages 清空)时,preflight 若按空数组判 fit,会把 ~50MB / ~550万 token 原始历史原样转发上游,必然 413 且每轮白传;#1839 逃生分支对「重建后核心文本超窗」的 doomed 体也会白转一次。
- 什么原因导致的:fit 判定必须度量「实际要发的视图」(#1498-F2 原始体回退)而非空数组;#1839 逃生缺字节兜底门;fail-fast 文案在折叠暂停时缺可操作说明。
- 影响范围与严重程度:仅「漂移修复重放原始史 + kernel 变换失败」或「#1839 逃生撞上 doomed 体」时触发;触发即整轮不可用且每轮白传几十 MB(p1 可用性缺陷);其余路径零变化。
- 怎么修复的:① 零折叠 transform-fail 的 fit 判定由 master 既有的
payloadFitsWindow(复用 #1498-F2 原始体回退)覆盖,PR 在此仅补一句文档注释;② #1839 逃生前加 doomed-bytes 门(shipDoomBytes <= max(2MB, window×7.5B/tok)),超阈值本地 fail-fast 而非转发;③ fail-fast 文案在epochConfirmPending时追加可操作说明。 - 是否有回归问题:无。非空视图 / 非 doomed 体路径行为不变;doomed 门只在数学上必然 400 的体上收紧;文案纯追加。已逐项过既有行为,无高危模式命中。
- 是否破坏缓存:不涉及。
- 是否改了设计方向:不涉及(host 侧 preflight 局部修正;零折叠修复由 master 既有
payloadFitsWindow承接)。 - 是否引入新配置:无新 env / config / 默认值。
判定:可合(仅剩人工合并)。
[bot] (ai: qwen3.8-27b) 已对 PR #5275 新头 关键发现(与上轮不同:本次 re-merge 改变了 PR 内容,已重新独立核验,非沿用上轮「字节一致」结论)
八问
判定:可合(仅剩人工合并)。无机械小修可补,无遗留。 |
What
#5223: after a cross-provider re-serialization drifts the whole history (repair mode honestly re-emits the unfolded originals), a kernel-transform failure on that drifted history made bili forward the entire raw client history upstream — ~50MB / ~5.5M tokens against a 1M window in the field: guaranteed 413 after a 30–42s upload, repeated per retry and per turn until the next turn re-folded.
Root cause
The prepare catch keeps the inbound JSON as the outbound body while leaving
prepared.processedMessagesempty (src/server/prepare-openai.ts:441). The trigger still fires correctly (outboundPayloadBreakdownalready falls back tooriginalMessagesfor sizing, #1498-F2), and preflight returns zero foldable ranges (#3641 epoch pause or deterministic exhaustion). But the zero-range fits judgment measuredprocessedMessages— the empty array — so0 + overhead < windowread "fits" and the raw body shipped ("preflight made no progress but the payload fits; forwarding as-is"). The sibling branch (compressedRanges>0) already carried the right fallback; this one missed it.Changes (all
src/server.ts, host-side)processedMessages, falling back tooriginalMessageswhen empty) — mirroringoutboundPayloadBreakdown's fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498-F2 fallback and the compressedRanges>0 branch.max(2MB, window×7.5B/tok)fails fast instead of being forwarded "for upstream arbitration".Behavior change (old → new)
No new config surface, no kernel changes, no wire-shape changes.
Tests
New
tests/issue5223-drift-unmeasured-forward.test.ts(4 tests): pins both holes (verified failing on unpatched master via stash-run) plus two controls (an over-window MEASURED view still fails fast; a fitting raw body still forwards). Adjacent preflight / fail-fast / epoch suites green locally (25 tests); full-suite verdict comes from CI on this head.Closes #5223