Repository navigation
fix(preflight): fold deep enough for a payload far over the window (#868) - #869
Conversation
A preflight folds one range per round and can spend a bounded number of summarization calls, so a payload that is well over the window and split into many ranges needs several rounds. Eight of each was not enough for the cases that matter: a live Gemini session of 1.39M tokens against a 1.05M window had every fold it attempted succeed and still came out ~8k short, then dropped the turn with "the compress budget was exhausted after 8 rounds" — the session was recoverable and the budget refused to recover it. Both budgets move to 16, which the new test pins: a fixture whose fold needs more rounds than the old budget allows is refused at 8 and forwarded at 16 (needs a fold deeper than one round, and stays bounded). The preflight is the last resort before an outright failure, so a bounded doubling of its work is cheaper than the failure it prevents.
|
I would like both, in this order, and you are welcome to add them here.
Neither becomes an issue: the filing rule for this project is pull requests only, because an issue in either repository sets the review agent implementing in parallel — which is what happened on the kernel's #319. If you would rather do 2 as a design change, say so and I will file it as its own PR instead. Thanks for confirming the root cause and for deriving the test from the exported constant instead of pinning |
… fail-fast 502 The exhaustion path quoted the original tokenCount captured before folding, so a 502 after 16 successful folds read as if nothing happened (the ranxianglei#868 incident: 1.39M -> 1.06M, still over the 1M window). The fail-fast now reports the post-fold size ("~N tokens (down from ~M before preflight)") and the compressible ranges still visible, via PreflightResult.rangesRemaining tracked per round from the kernel's viable ranges; original phrasing is kept verbatim when no fold ran. MAX_PREFLIGHT_ROUNDS is exported next to MAX_SUMMARY_CALLS_PER_PREFLIGHT so tests derive budgets from constants. New e2e case scales the round-budget fixture past the bound and pins the honest 502 (round-budget wording, post-fold size, remaining-range count, exactly one summary call per round, nothing forwarded).
Document next to MAX_PREFLIGHT_ROUNDS / MAX_SUMMARY_CALLS_PER_PREFLIGHT how far over the window 16 folds can reach (one range per round x at most CHUNK_FRACTION of the window per fold ~= 9.6x-window overshoot, i.e. a payload up to ~10.6x the window, best case; lower in practice) and that beyond the bound the loop still exits cleanly with the post-fold size and remaining compressible-range count reported. Note that 16 is tuned to the ranxianglei#868 incident class — an overshoot-scaled budget is deliberately left as a separate design question.
[bot] 🏷 Both items are done and pushed to your branch 1. Honest failure detail —
2. Coverage bound documented, not redesigned — Verification: 中文摘要:按你的顺序完成了两项——502 报错现在报告折叠后 token 数(含折叠前原值对比)与剩余可压缩区间数,覆盖边界文档写在常量旁而未改动预算机制;两个提交已推送到你的分支,typecheck、全量测试(1651 通过 0 失败)和构建均已验证。 |
[bot] 🏷 Correction to my previous comment: I garbled the quoted message format. The exact shape of the new 502 body (for the filed incident's numbers) is:
Everything else in the report stands unchanged. |
|
解决冲突 |
…glei#869) # Conflicts: # src/preflight.ts
|
继续 |
[bot] 🏷 Conflict resolved and pushed. The branch What conflicted. A single hunk in Resolution. Kept master's #857 block verbatim and appended this branch's Verification on the merged head (
No other commits were touched; the five branch commits keep their hashes, so everything in my earlier report still applies. Ready for you to merge when you are. 中文摘要:解决了分支与 master(至 v0.1.117)在 src/preflight.ts 末尾的唯一冲突——保留 master #857 的基线逻辑、追加本分支新增的 rangesRemaining 赋值,合并提交 eb6c475 已推送;typecheck/全量测试(1670 通过)/构建全部通过,PR 现显示可干净合并。 |
Problem
The preflight folds one range per round, with both per-invocation budgets at 8. A payload far beyond the window therefore fails even when every fold succeeds, and the failure reads as a total one:
That conversation was folded to within 8k tokens of a 1,048,576 window and then declared exhausted. The message also quotes the original context, so a reader sees 1.39M tokens and no folding, and the session takes the preflight dead-end cooldown on top.
Fix
MAX_PREFLIGHT_ROUNDSandMAX_SUMMARY_CALLS_PER_PREFLIGHTinsrc/preflight.tsare raised to 16, and the latter is exported so the budget-cap tests derive it instead of pinning a literal. Nothing else changes: a round still folds one range, and a payload already over 16 ranges wide still stops and reports, but at a depth that matches a window-sized payload rather than a marginally oversized one.Tests
tests/preflight-round-budget.test.tsdrives the fold loop over an openai-wire fixture of roughly 83k tokens against a 30k window. It refuses the request at 8 rounds (preflight_compress_failed) and forwards it at 16, using more than 8 summary calls on the way, which pins the behaviour rather than the constant. The #574 cap test intests/preflight-multi-range.test.tsnow takes its expected number from the exported constant instead of re-pinning 8, so the two cannot drift apart again.Verification
Full proxy suite on the branch: 1669 tests, 1661 passing, the same six environmental failures upstream shows.
npx tsc --noEmit -p tsconfig.build.jsonclean. The live case that produced the report now folds further and forwards instead of failing; the numbers above are from that session, and the test fixture reproduces the same shape at a smaller scale so the behaviour is pinned without needing a 1.39M-token fixture in the suite.