Repository navigation
[PR] fix: root-cause batch for preflight compression (persist folds, wall budget, terminal frame, truncate estimate-hold, shared summary pot) - #6926
Conversation
A hard process death mid-preflight (crash / OOM / restart on multi-million-token sessions) skipped every post-loop persistence path, so rounds already compressed lived only in memory; a client retry under a new session id found no parent state worth inheriting and re-compressed from zero — the permanent unavailability loop reported in #3978. commitFold now marks the session dirty as each fold lands; the persist debounce coalesces the calls into one write per quiet window, so durable progress accumulates regardless of how the request ends. A clean client abort already persists via the post-loop rebuild (src/server.ts:3554 runs before the :3607 fail-fast return); this closes the death-mid-request gap that path cannot cover. tests/issue3978-preflight-abort-persistence.test.ts pins it end-to-end: T1 guards the clean-abort ordering, T2 is the negative control (no fold committed => nothing fold-like persisted), T3 SIGKILLs a real child proxy mid-marathon and asserts a fresh instance's new-id retry inherits the persisted blocks instead of starting fresh. Without the fix the red run fails T3 exactly at the adopted-block count. (cherry picked from commit 775d622)
…3978) The 60s READY deadline lost the race on ubu22-1/win22-1 at head 9be2ec6: the child cold-transforms the whole src tree via tsx, which exceeds 60s under parallel-shard load. Raise the deadline to 180s, fail fast when the child exits before READY (diedP), and capture its stderr in-memory so a future crash surfaces in the assertion message instead of getting lost in interleaved shard logs. (cherry picked from commit dbcc6c8)
…ts resolve (#3978) On CI the per-test tmp dir sits outside the repo tree, so Node finds no package.json walking up from the generated child script and defaults it to CJS. Its bare import of acp-kernel then resolves under the require condition, which acp-kernel's exports map does not define, killing the child at boot with ERR_PACKAGE_PATH_NOT_EXPORTED on every leg. Locally the tmp dir lives inside the repo (inheriting its type:module), which is why this only failed on CI. Writing a minimal {"type":"module"} package.json beside the child script pins the module format explicitly. Verified with an out-of-repo TMPDIR reproduction (red before, green after). (cherry picked from commit 5722ac7)
… T3 child The blind sleep(1000) settle raced the store's trailing-debounce write under loaded CI runners: windows 22-1 at head 5722ac7 found NO parent record at all when the retry booted (T3 red; signature distinct from #4049's two #3769 flakes). Replace the fixed sleep with a wait on '[durable] N' — the child's own codec-aware store.loadSync disk read (envelopes are BILIENC1- framed at rest, so the test process cannot scan the file for the summary mark) — so the SIGKILL can never race an unwritten save. Failure messages now carry the child stderr tail and the store dir listing. (cherry picked from commit 12de55b)
…78 T3 child win22-1 @ a46fd0d: [durable] was observed yet the retry still started fresh (resume lines empty) — findResumeParent (#1486) matches on the parent's affinity-chain entry, which a fresh instance hydrates from the machine-shared state-dir snapshot (prefix-affinity.json), a SEPARATE artifact from the session record with its own 500ms-debounced single-shot persist whose write can lose a rename race or starve under event-loop pressure and never re-arms. The child now proves the FILE carries the t3978-p entry (re-arming the persist while it does not) and the test waits for that proof before SIGKILL; the final assertion also surfaces the child stderr tail. (cherry picked from commit 683d90b)
… EBUSY observation Minor fix found while fixing this lane's CI (labeled per the small-fix rule): the #2771 corrupt-index proof spent the full ~9.6s retry ladder on EBUSY unlinking index.db (transient Windows AV/indexer lock, #2799 family) — extend the flat tail by two 3200ms rungs (~16s total) and align the budget unit test. (cherry picked from commit 2b8c26d)
…ence wall clock (#3573) A summarization round costs 25-44s against a real upstream regardless of range size (#2682), while the common client timeout class (dsh ~60s) only fits ONE such round. An over-window preflight kept admitting rounds until the client disconnected mid-round: the attempt died, the in-flight summary call was wasted, and the user saw an opaque 60s hang-retry loop (#3573). Rounds that make upstream calls are now admitted only while the wall clock leaves room for the next one (estimate: 2x the last round's measured time, first round always admitted, digest regime exempt — CPU-only by design). A stopped attempt keeps its completed folds (they are persisted), so the client's retry resumes from the smaller view and the marathon converges across attempts; the fail-fast stays a plain 502 with no dead-end cooldown, and the tier-recovery funnel is suppressed for the stopped attempt so it cannot sneak extra upstream calls past the budget. Fixes #3573 (companion to #3586 — same issue, independent half). (cherry picked from commit 2f00ba6)
…rdict (#3573 review) When the client-patience wall budget stops the marathon while the payload still overflows the window, the post-loop verdict fell through to the generic branches: 'the compress budget was exhausted after N rounds' (citing a round cap that was never reached), or — in the relaxed-regime subcase — a deterministic exhaustion that server.ts maps to 413, a terminal verdict headless hosts honor by giving up. That kills exactly the retry-convergence story this fix exists for: the folds ARE persisted and the next request resumes from a smaller view. Give the wall-budget stop its own first-in-cascade verdict: kind exhausted with retryable true and an honest detail (rounds run, folds persisted, retry resumes). No deterministic flag -> stays 502; the dead-end cooldown gate (compressedRanges === 0) is untouched because a wall-budget stop always has at least one landed fold. The issue3573 test now pins the honest wording in every stopped attempt's body. (cherry picked from commit 7e40615)
(cherry picked from commit 7d915c5)
…ame (#3978) Once #568 has committed early headers (200 + SSE/JSON keep-alive), every no-forward exit left the client holding a 200 and an open stream that never completes: respond:false writes nothing, and a later status code cannot be written after the 200 — so clients could not distinguish 'bili gave up' from a dead network and waited out their own watchdog (#3978 floor 17/18). All four preflight no-forward exits now emit one terminal frame before returning when headers are already committed: SSE gets an 'event: error' frame, non-streaming JSON gets the error object appended after the whitespace padding (still one valid JSON document), then end(). Body reuses the native fail-fast shape {ok:false,error:{message,retryable}}. Uncommitted exits keep their normal status-code path byte-for-byte, and the abort branch logs its outcome (terminal frame sent / socket gone / not committed). (cherry picked from commit 4bcbcf9)
Review alignment for the terminal-frame fix: the first commit introduced a
new {ok:false,error:{message,retryable}} frame for committed no-forward
exits, which would have silently replaced the in-band contract master
already emits on this path (protocol-native preflight_compress_failed
frames per #568/#1455 - client SDKs parse those natively; the {ok:false}
shape exists only on /__bili/* admin endpoints).
- commitPreflightTerminalFrame now delegates to emitPreflightError (SSE)
and appends preflightFailFastBody (non-streaming): byte-identical to
what the consumer's committed branch already emits, so no wire-shape
change anywhere.
- preflightFailFastBody extracted into src/stream-error.ts as the single
source of truth, shared with handle.ts's status-code bodies.
- The genuinely new terminations are unchanged: the abort exit (respond:
false used to skip all writes) and the overflow-refold re-run
(respondFailFast=false suppressed the consumer) now terminate with the
same established frames instead of dangling / hitting
ERR_HTTP_HEADERS_SENT.
- Tests assert the native shapes; uncommitted negative control unchanged.
Targeted local: issue3978 suite 4/4, adjacent preflight suites 24/24,
tsc clean. Full-suite verdict: CI.
(cherry picked from commit 34020b6)
…allowing them The executor discarded every candidate rejection, so a context-budget guard or a post-fetch parse failure left only a bare tN:error with no warn line, and the compress failure text blamed maxSummaryLength for every failure. Capture the safe cause for our own fixed error strings (unknown provider errors stay opaque), log it once, and branch the failure advice by the batch's own stop reason: budget on deadline, maxSummaryLength only for over-cap replies. Fixes #6885 (cherry picked from commit f0867d7)
… readings (#6882) An ESTIMATE-grade reading (no billing sample landed this cycle — stream interruption, cache-TTL expiry) structurally overstates what will be billed (observed 1.75-1.87x), so a truncate-watermark crossing on such a reading latched clips the bill never saw: content destroyed plus a burned prefix cache (16.3% hit vs 100% control). The pressure band already uses the estimate-hold guard (nudge.requireUsageSampleForPressure + the nudge.maxEstimateHoldRounds backstop); the truncate watermark now honors the same guard: while the backstop allows, the crossing is held one round for the usage-grade reading the next request carries. Already-latched clips still replay byte-for-byte; the round is unjudged, so terminalStreak rides through untouched. Billed readings (usage, incl. the overflow arm), legacy callers (source omitted), and requireUsageSampleForPressure=false keep byte-identical behavior. kernel-change: #6882 option A — the truncate watermark is the same phantom-pressure surface as the pressure band, so it reuses the guard the band already uses rather than a new knob. No new config field; the two existing nudge fields' semantics now cover the truncate watermark too (docs updated in kernel/src/types.ts and the host docs in a follow-up commit).
…dates (#6885) The per-target window was a fresh targetTimeoutMs for every candidate, so early candidates ate the whole pot: with the 25s/50s defaults, t0 ran 25s, t1 was killed at the 50s batch deadline, and t2 never got a request out — the chain failed 100% on long payloads no matter how fast the later candidates were (558 target-timeouts / 258 batch-deadlines / 5 successes in the incident log). The batch deadline is now a shared pot: each candidate gets min(targetTimeoutMs, remaining-pot / remaining-candidates), so a slow early candidate times out at its fair share and the next candidate gets the remaining budget (fast failures return their share to the pot). The configured targetTimeoutMs stays the per-candidate cap; the 600s/180s workaround keeps its full 180s windows (600s/3 > 180s). The timeout detail now reports the ACTUAL window the attempt had (the shrunk target window, or the range's real remaining total) instead of the nominal configured value — the attempts list is model-visible in the compress tool result, so the number the model reads now matches what the attempt actually had. Fixes #6885 (issue 1: default budget + serial attempts fail long-payload chains 100%).
…d config reload (#6882, #6885) - compress.requireUsageSampleForPressure / maxEstimateHoldRounds now cover the emergency-truncate watermark as well as the pressure band (#6882): table rows, the detail section (the stale 'truncate stays armed regardless' line is replaced), and the host config.ts doc comments. - external-summary deadline semantics: the batch deadline is a shared pot — each target gets min(targetTimeoutMs, remaining pot / remaining targets) (#6885); the stale 'each target gets a fresh window' sentence and the sizing advice are updated accordingly. - external-summary: a direct config-file edit is not watched — it applies on POST /__bili/config/reload or restart (Web-UI saves hot-apply); reasoning targets need a large outputTokens or they end in finish_reason=length with an empty body (#6885). - en + zh kept in parity.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11009-pr.6926.5852c92This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6926.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…le text The PR body's docs commit edited the GENERATED table rows in docs/configuration.md / .zh-CN.md directly, without updating the website/config-reference/compress.yaml seed they are generated from (#2107 drift guard A). docs-sync went red and, via the test job's !failure() needs gate, skipped the whole test matrix. This commit moves the PR's exact table text (en+zh) into the seed's desc_en/desc_zh for compress.requireUsageSampleForPressure and compress.maxEstimateHoldRounds; regenerating now reproduces the committed docs byte-for-byte (verified: gen-config-docs generate + git diff --exit-code clean). desc_ja and the site_* website fields are untouched, matching the PR's en+zh-only docs scope.
The kernel-change commit bumped kernel/package.json 0.0.198 -> 0.0.199 but left both lockfiles at 0.0.198, so the #3782 lock-sync drift guard went red (and, like docs-sync, skipped the test matrix via the !failure() needs gate). Regenerated with npm install --package-lock-only (root + kernel); the diff is exactly the version line in each file.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR #6926 审核结论(preflight 根因批)独立审完:逐段读了全 diff(22 文件 +1555/−106)、核对了三个源 issue 的方向、在本地跑过定向测试、live 查了 CI 并修掉了两条 PR 自身漂移(见下)。 ① 方向关(原 issue):三个 issue 的根因判定都成立,PR 修的是生产者不是症状——
② 回归关(既有行为清点):逐路径过过——
③ 代码关:无 ④ 测试关:全是真测试——kernel 10 例逐字节/ latch / streak / 兜底/ guard-off / cap-0 / legacy 各有断言;共享池测试把旧断言反向钉住(backup 候选从"饿死"变"成功");issue3573 真 server e2e(慢轮:wall stop→502→无 cooldown 污染→fold 持久→重试收敛 200;快轮:单 attempt 完成、零 stop 行);terminal-frame e2e 用 deadline race 把修复前的静默悬挂变红;abort-persistence e2e 的 T3 是真 SIGKILL 子进程中途死亡,新 id 重试经 affinity 链继承持久 fold——这是唯一能覆盖「进程硬死跳过 post-loop 重建」的形状。 CI 状态(live 核实 + 已修):原 head 963ed72 上 docs-sync + lock-sync 红(PR 自身漂移:直接改生成物未更种子;kernel bump 漏 lockfile),test 矩阵被 八问
收工判定:代码与测试层面 ready;kernel 变更留人审档(§7.4 不可 auto-merge)。等 head 3319d46 的 CI 转绿后由 owner 合并;合并后四个被取代的在途 PR(#4046 / #3590 / #5057 / #6890)请 owner 关闭(按政策我不代关)。 |
Resolve kernel version collision: PR 0.0.199 vs master 0.0.200 -> 0.0.201 (kernel/package.json + kernel/package-lock.json + root package-lock.json). Precedent 889493b: master's version + this PR's own kernel step (#6882 truncate estimate-hold). All 9 both-sides host files auto-merged; each verified byte-identical to an independent git merge-file 3-way. Lockfiles stable under npm install; typecheck + build + targeted tests green.
(ai: vllm-qwen/qwen3.8-27b) PR #6926 merge conflict resolved and pushed. What was done this run:
八问(结构化结论):
判定: 冲突已解决并以最终头 |
compress.maxSummaryLength; named error → "the attempts below name the failing cause"; otherwise → "see the attempt outcomes") instead of always pointing atmaxSummaryLength(cherry-picked from [PR] fix(compress): name non-fetch external-summary failures instead of swallowing them #6890).What this PR does
One root-cause PR for the
preflight-compressbatch (rcagg #6896): preflight compression on large contexts either ran for hours, or failed 100% on long payloads, or latched truncation off meter readings the bill never saw.Fixes #3978
Fixes #6882
Fixes #6885
The four prior PRs on these issues (#4046, #3590, #5057, #6890) are all direction-aligned with the root cause; their commits are cherry-picked below with
-x(authorship preserved) instead of shipping four overlapping PRs.#3978 — the preflight marathon (three halves)
commitFoldmarks the session dirty as each fold lands, so a client disconnect/abort mid-marathon no longer loses the attempt's progress — the retry resumes from the smaller view instead of re-compressing from zero.#6882 — the truncate watermark acts on estimate-grade reads (kernel, option A)
kernel-change: #6882 option A — the emergency-truncate watermark now honors the SAME estimate-hold guard the pressure band already uses: a watermark crossing on an estimate-grade reading (1.75–1.87× overstated after a stream interruption) is held for up to
compress.maxEstimateHoldRoundsestimate rounds (default 2) instead of latching clips the bill never saw. Already-latched clips still replay byte-for-byte; the held round is unjudged (the terminal streak rides through untouched). Billed readings (incl. the overflow arm), hosts omitting the source marker, and the guard-off / backstop-elapsed cases keep byte-identical behavior. New kernel suitekernel/tests/issue6882-truncate-estimate-hold.test.ts(10 cases); kernel 0.0.198 → 0.0.199. Per the Kernel Boundary this stays in the human-only review tier; it implements the option requested in #6882 via the rcagg pipeline.#6885 — the external-summary serial budget starves later candidates
min(targetTimeoutMs, remaining pot / remaining targets). Before, every target got a freshtargetTimeoutMswindow, so t0 (25s) + t1 (25s) ate the 50s total and t2 never got a request out — the chain failed 100% on long payloads (558 target-timeouts / 258 batch-deadlines / 5 successes in the incident log). The configuredtargetTimeoutMsstays the per-candidate cap; the 600s/180s workaround keeps its full 180s windows (600s/3 > 180s).POST /__bili/config/reloador restart applies it; Web-UI saves hot-apply), and the reasoning-targetoutputTokenspitfall (finish_reason=length+ empty body).Behavior changes (old → new)
compress.requireUsageSampleForPressure: false.targetTimeoutMswindow drawn from the shared total (early candidates could eat the whole pot). New — each candidate getsmin(targetTimeoutMs, remaining pot / remaining targets); the total deadline still bounds the chain.Config surface
No new config fields. Two EXISTING fields gain coverage (semantics extend, no migration):
compress.requireUsageSampleForPressure(defaulttrue): was "the pressure band waits for a usage sample on estimate readings"; now also covers the emergency-truncate watermark (Kernel authorization request: emergency-truncate — does not check the reading level; after a stream interruption, it wil | Kernel authorization request: emergency-truncate 不看读数等级,流中断后按估计值也会截断并锁住;建议沿用 requireUsageSampleForPressure 的守卫和 maxEstimateHoldRoun… #6882).falserestores the legacy estimate-driven pressure injection AND estimate-driven truncation.compress.maxEstimateHoldRounds(default2): was the backstop for the pressure band; now also bounds how many estimate rounds the truncate watermark may wait.0disables the hold for both.Both keep their existing per-field merge (global → provider → model) and kernel mapping (
nudge.*). The existingcompress.externalSummary.budgetfields are unchanged — only howtotalTimeoutMsis divided across targets changed (behavior change 5).Tests
kernel/tests/issue6882-truncate-estimate-hold.test.ts— 10 new cases: held round (no clip, no latch,truncationSkippednames the estimate grade), held round byte-identical to a below-threshold round, latch replays byte-identically on the held round, backstop elapse (streak > cap → truncates), usage reading truncates, no source marker truncates,requireUsageSampleForPressure: falsetruncates,maxEstimateHoldRounds: 0truncates, terminalStreak rides through, held-after-stuck keeps the streak. Kernel suite 1650/1650;verif-subset-guard+ the compress.ts verif bug/generic/induction suites green.tests/issue3978-preflight-abort-persistence.test.ts(from [PR] fix: persist each committed preflight fold so a dead marathon leaves durable progress (#3978) #4046) — clean-abort durability, negative control, SIGKILLed-child retry inherits via the affinity chain.tests/issue3573-preflight-client-patience.test.ts(from [PR] fix: stop preflight marathon rounds that would blow the client's patience wall clock (#3573) #3590) — slow-summary retry converges to 200; fast marathon is a single attempt.tests/issue3978-preflight-terminal-frame.test.ts(from [PR] fix: terminate committed preflight responses with a protocol error frame (#3978) #5057) — SSE frame / JSON frame / uncommitted negative control / abort log outcome.tests/external-summary.test.ts— 3 new shared-pot tests (fair share, actual-budget reporting, total still bounds the chain); 96/96 with the http suite.Local verdict (rebased head)
typecheck clean, build clean, targeted suites green (listed above). Full-suite verdict = remote CI on this head.