Repository navigation
[PR] fix: book upstream-stated overflow counts into the fold target (#3450) - #3453
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4627-pr.3453.ea75d9dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3453.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) PR #3453 审查结论(独立复审:逐文件通读全部 diff、对触碰路径的既有行为逐项清点、在 PR head 本地实跑直接受影响的 3 个测试文件 37/37 通过。CI:30 通过 / 15 skipped(门控 lane)/ 1 个分片 八问
行为变更清单(旧 → 新,均有原因):
测试有效性:真测试,断言有牙。新 hermetic e2e T1/T2 我推演过旧代码必挂(T1:旧代码 arm 记 1048576,落在断言区间外;T2:旧代码底线单调增、收敛不到 fit → 透传 400 而非 200);parser 单测含负回归守卫;issue1195-B 断言随新语义同步更新(byte-identical 契约本身未变)。本地在 PR head 实跑受影响的 3 个文件:37/37 通过。 合并顺序提示:#3328(Closes #3017,同样动 src/server.ts 与 src/server/handle.ts 的墙/分母逻辑,open)与本 PR 同域——后合者需要 rebase 并核对语义交互(两者都在调"判断用哪面墙")。PR 描述里已注明此协调事项。 判定:可合。代码层面无拦阻项;待最后一个 CI 分片 |
|
解决冲突 |
12e0bf4 to
b9e5ad7
Compare
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决:分支已 rebase 到当前 master( 八问
行为变更披露(相对本 PR 原始描述,旧值→新值+原因)
收工判定:冲突闭环,可合并;等待 CI 全量裁决后由人工合并。 — 备注:组合回归作为本次冲突收尾的一部分以独立标注 commit( |
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决完毕(这条可能赶在我上一条回复之前):分支已 rebase 到当前 master( 两处文本冲突(cache-ledger.ts、relay.ts)按意图取并集解决;rebase 验证中还修掉一个两个修复组合出的双重扣减回归(独立标注 commit 等 CI 全量裁决后即可由你人工合并。 |
|
ci失败 |
An upstream overflow 400 that states the rejected payload's own counts
('you requested N tokens (M in the messages, K in the completion)') fed
only its WINDOW into the one-shot arm — and discarded M and K entirely.
The arm booked a ceiling as if it were a measurement, so preflight saw
'payload ~window, deficit 0' on an underestimating local meter and
forwarded a payload the upstream then billed over the wall (messages AND
completion against the same window), killing the turn.
- inspectContextOverflow now extracts requestedTotal/requestedMessages/
requestedCompletion from provider-specific phrasings (OpenAI/DeepSeek
messages+completion split; Anthropic 'prompt is too long'; Gemini
'input token count') — a wrong guess is worse than no guess, so each
pattern stays provider-specific.
- The arm books the measured MESSAGE count (a ceiling is not a size); the
side-request guard keeps the CEILING via overflowArmTokens.
- The proven completion share is reserved off the wall for every
payload-vs-wall judgment while the arm lives (the foldable quantity is
MESSAGES); it retires with the arm on the next real usage report.
- One wall per request on the refold path: prepare/nudge and preflight
judge on min(declared, upstream-stated) instead of two different walls.
- commitFold nets each fold out of the input baseline like the live path
does, so an armed floor at-or-above the target descends and the loop
can exit on fit instead of burning budget or stalling.
- One-time warning when the upstream-stated window differs from the
window in use, naming both values (operator hint: modelContextLimit).
Behavior changes (disclosed): armed baseline is the upstream measurement
when stated (was: the stated window); preflight judges on window minus
the proven completion reservation while armed; lastInputTokens nets down
during preflight folds (was monotonic-increase within a run); new
persisted stat fields overflowArmCompletionReserve /
overflowArmMismatchWarned (legacy files: absent = legacy behavior).
Tests: parser cases incl. the verbatim incident body + regression guards;
hermetic e2e for honest-fail (uncloseable gap surfaces as 400, never a
masked over-forward) and in-request rescue (fold under the message
budget, reservation retires with the arm). issue1195 B updated to the
new arm value (measured 130000 vs ceiling 128000).
… gross fold (#3450 trace correction)
…3450 × #3738 merge regression) Rebasing this branch over #3738 exposed a joint defect neither PR's own tests cover: handle.ts rewrites config.modelContextLimit IN PLACE with the output-headroom reservation before preflight runs (its [headroom] line says 'nudge/preflight judge input against reserved'), so subtracting the demonstrated completion share from that already-reserved value counts the completion share TWICE. On any session with an output-budget reservation whose overflow 400 states a completion count, the preflight wall collapsed toward zero (converge fixture: 40,000 - 60,000 -> 0) and the rescue folded nothing. Verified by the new case-3 convergence run failing on the naive merge and passing after this fix. The reservation now applies MIN-wise to the upstream-stated wall: limit = min(reservedLimit, statedWindow - completionReserve) while armed, falling back to the untouched limit otherwise. The incident arithmetic (stated 1,048,576 / messages 841,304 / completion 256,000, reserved limit 792,576) yields exactly 792,576; the no-reservation path is byte-identical to before. Test reconciliations in tests/fix-arm-headroom-clamp.test.ts (each with an inline rationale): case 1's guard now pins the stated CEILING (the side-request guard reserves each request's own output from that value itself, so feeding it the pre-reserved limit would double-subtract); case 2 pins lastInputTokens at the stated message count 841,304 (since #3450 the measurement is booked when present; the no-reservation clamp stays a no-op above it); case 3's guard pin likewise moves to the ceiling.
… case Rebasing onto master surfaced an interaction this PR's own base predated: #3739's case 3 (merged 2026-10-19, postdating this PR's base) pins the PRE-#3450 refold behavior — fold against the bare stated window, retry rejected again by the completion share, 400 surfaced until the next turn. Since #3450 the same-request refold folds against window − proven completion share, so the rejected request itself is rescued (client sees the retry's 200) and the next turn converges on the stored folds. The stronger behavior is the point of #3450; update the case to pin it: turn 1 asserts the in-flight rescue + paired arm/reservation retirement on the settled usage report, turn 2 keeps the no-loop convergence.
b3e38d0 to
5b91fb5
Compare
(ai: vllm-qwen/qwen3.8-27b) CI 两个红门禁已定位并修复,分支已 rebase 到最新 master( 1. 是什么问题:CI 红在两个静态门禁——unused-exports 与 docs-sync;都不是本 PR 的代码缺陷,是「基线滞后」产物。另有一个 rebase 后才暴露的测试交互(见下)。 2. 什么原因导致的:① unused-exports 报 3. 影响范围与严重程度:门禁两项无运行时影响。行为交互只影响「400 正文给出 M in the messages / K in the completion 计数、且折到 W−K 能装下」这一种场景:旧行为客户看到 400、下一轮再收敛;新行为请求内救回、客户拿到重试的 200。这是 #3450 的设计目标(可用性改善),不是退化;无计数的溢出正文行为不变(issue1195 B/C 用例钉住并通过)。 4. 怎么修复的:重新 rebase 到当前 master(零冲突,两处门禁修复随基线自动带入),本地镜像跑通两个门禁脚本(unused-exports clean / docs-sync clean);case 3 改写为钉新行为——turn 1 断言请求内救援(折叠摘要上线路径、尾部保留)+ usage 报告一次结算同时退役 arm 与预留(#3450 成对退役契约),turn 2 断言靠存储的折叠继续收敛不循环。独立标注 commit 5. 是否有回归问题:审过的既有行为:无计数溢出正文的 verbatim 透传/重试判决契约(issue1195 B/C)— 保持;#3738 clamp 不变量(case 1-2:基线 ≤ judging limit、guard 持天花板)— 保持且继续通过;arm 退役契约(#1110/#1129)— 扩展到预留字段,方向一致。typecheck 干净,定向 5 文件套件 46/46(单文件复跑 5 次稳定)。全量裁决交给 CI。 6. 是否破坏缓存:不涉及——未触碰缓存/重试/增量游标/去重键策略。 7. 是否改了设计方向:不涉及——冲突解决收尾 + 测试期望对齐,无转向。 8. 是否引入新配置:不涉及。 行为变更披露(old→new):含计数溢出 400 后的同请求 refold 目标:裸 stated window → min(声明窗口, stated window) − 完成预留;后果是部分原先会透给客户的 400 现在会在请求内救回为 200(#3450 意图;此前因 #3739 测试晚于本 PR 基线而未被任何测试覆盖)。此前已披露的两项(armed 基线额外钳制 ≤ judging limit;preflight armed wall 改 min 形式)不变。 判定:等本轮 CI 全量;绿了即可由你人工合并。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.638-master.aba7062 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #3450
What was wrong
An upstream overflow 400 that states the rejected payload's own counts ("you requested N tokens (M in the messages, K in the completion)") fed only its WINDOW into the one-shot arm — M and K were parsed away and discarded. The arm booked a ceiling as if it were a measurement, so preflight saw "payload ~window, deficit 0" on an underestimating local meter (incident: local ~773k vs upstream-counted 904k) and forwarded a payload the upstream then billed over the wall — messages AND completion against the same 1048576 — killing the turn.
Fix
inspectContextOverflownow extractsrequestedTotal/requestedMessages/requestedCompletionfrom provider-specific phrasings (OpenAI/DeepSeek messages+completion split, Anthropic "prompt is too long", Gemini "input token count"). Provider-specific on purpose: a wrong guess is worse than no guess.overflowArmTokens).Per-fold accounting is UNCHANGED by this PR:
commitFoldcalls the shared apply machinery (stream.ts applyRanges), which already credits the fold and nets the GROSS mass out oflastInputTokens; the existing+= defaultCountTokens(summary)add-back makes the baseline track the post-fold payload size exactly once. That pairing previously had no comment explaining it — added insrc/preflight.ts(see S1 below for why it matters).Why the incident's loop stopped after exactly one fold (S1) — root-caused from the reporter's raw log
The refold round 2 DID start but exited silently at its fit-check (
if (decisionTokens < textTarget) break, positioned before the per-round log line), because after one small fold the armed floor had already crossed below the wall: arm booked the ceiling 1048576 as payload size → fold netted −(4253 gross − 2845 summary) = −1408 → floor 1047168 < wall 1048576 → "converged", one fold short of the true measured gap (~112k). With this PR's measured booking (904518) plus the reserved completion share (wall judged at 744000 under the incident config), a single small fold cannot flip the judgment — the floor must descend ~160k before the fit-exit may fire, so the marathon actually runs. No loop-code change needed for S1; it falls out of booking the right numbers.Behavior changes (old → new)
overflowArmCompletionReserve, +overflowArmMismatchWarned(absent on legacy files = legacy behavior)No config schema change. No kernel change. Wire shape unchanged (folded bodies are the existing preflight artifact). Per-fold
lastInputTokensaccounting unchanged (see note above).Commit
12e0bf4d9reverts an earlier netting edit incommitFold: tracing the reporter's exact log values (1048576 → 1047168) showed the shared apply path already nets the gross fold, so that edit would have double-netted and made the S1 premature-convergence class worse. Revert restores master's accounting byte-for-byte; the kept comment documents the cross-file invariant.Tests
tests/context-overflow.test.ts: verbatim incident body, comma grouping, Anthropic/Gemini counts, plus regression guards (no-count bodies keep legacy behavior; legacy OpenAI "your request used …" phrasing must NOT match).tests/e2e-issue3450-stated-counts.test.ts(new, hermetic): T1 uncloseable gap → visible 400, never a masked over-forward (arm facts asserted: measured baseline, reservation, ceiling guard, mismatch flag); T2 small stated window → in-request rescue under the message budget, reservation retires with the arm.tests/issue1195-overflow-refold.test.tsB: assertion updated to the new arm value (measured 130000 vs ceiling 128000) — the byte-identical passthrough contract it guards is unchanged.查重 / coordination
查重: 不重复 — same family as #3017 (denominator drift) but different symptom (dead turn vs unpredictable thresholds) and different mechanism (wall-selection inconsistency + discarded upstream counts vs per-request sampling drift); full comparison in the issue reply. Adjacent open work: PR #3328 (Closes #3017) touches the same
modelContextLimitplumbing — if it lands first, rebase this onto it.