Repository navigation
fix: fold several preflight ranges in ONE external-summary call (#2678, #2667) - #2723
ranxianglei wants to merge 4 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2971-pr.2723.5668846This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2723.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…2667) executeBatch awaited each work item end-to-end, so the pool bounded cross-session calls but never a single batch: k ranges in one summarize() call still cost k sequential round trips, and the shared per-pass deadline cut multi-range passes off with whole ranges never dispatched. Workers now pull indices from a shared counter so up to `concurrency` items are in flight; results stay index-aligned as the dense prefix of settled ranges, and an early stop (caller abort / any range hitting the shared deadline) latches the workers so unstarted ranges are never dispatched.
The preflight fold pass submitted one external-summary request per range, sequentially, inside the session lock: k viable ranges cost k x ~20-40s of user-visible stall before the shared 180s pass deadline cut the rest off (v0.1.191 evidence: fold passes median 59s, max 180365ms, ~1.85 ranges/pass). A round now nominates up to MAX_PREFLIGHT_SUMMARY_BATCH=4 disjoint ranges (oldest first, largest span each, same preview/content/chunking gates as the walk) into a SINGLE plan.summarize() call that the concurrent executor runs in parallel; successful summaries commit through the same apply path as the walk (commitFold), failed ranges fall through to the serial walk unchanged and get their halving/transient retries there. Log line reports real batch sizes (was hardcoded '1/1 range(s)'). Gated on the external lane and >=2 viable ranges; single-range, legacy-streaming, and emergency-digest paths stay byte-identical.
…nk ranges (#2678) Review findings on the #2678/#2667 batch-fold change, corrected in place: - Span selection: nomination took spans[0] of the REVERSED worklist — the newest tail chunk — while the mirrored walk pops the oldest full-budget chunk first. Now identical (spans.pop()). - SkipSet poisoning: a rejected preview or empty render during nomination added the WHOLE range to skipSet, where the walk only continues to the next span. A sub-minimum newest tail therefore killed a range whose older chunks were foldable for the rest of the pass. Removed both adds. - Call accounting: nominations reserved summaryCalls before knowing whether the batch would fire (≥2 nominations); a sub-batch set falls through to the walk, which reserves its own calls — double booking could trip budgetHit early. Reservations now commit only when the batch fires. - rangesTried counted a range at both sites (nomination AND walk); now deduped per range key so the failure detail reports distinct ranges.
…2678) - New E2E: a multi-chunk kernel range (same-role run with an over-budget message and a sub-minimum tail) must carry its OLDEST full-budget chunk in the first batch wave and keep folding chunk by chunk (≥5 blocks). Under the previous tail-first + poison behavior this fails: the range never enters any batch and only the tail range folds. - New E2E: a batch-failed range is retried across later rounds (≥3 attempts) while siblings keep folding. - Corrected the header/test comments claiming every batch-failed range 'falls through to the serial walk': while siblings fold, retries are re-nominations into later batches; the walk takes over only once the batch applies nothing in a round.
26e88f1 to
fe8d498
Compare
[bot] 🏷 ## PR #2723 审查结论(结构化八问) 审查方式:通读全量 diff; 1. 是什么问题PR 本身是 #2678 option B + #2667 executor 并发化(方向正确、已按 issue 方案实现)。审查在其新 nomination 路径里发现 3 个真实缺陷 + 描述/测试注释过度声称。 2. 什么原因导致的
3. 影响范围与严重程度F1+F2 组合:触发需「多 chunk 可行范围 + 最新 chunk 低于 minCompressRange」(生产大 range 数据见 #2678/#726;代码层面证明可达,合成探针中 kernel 分段规则会把 user 连发 ≥3 条切开,需 assistant-only 连发才成单一大段)。命中时:该 pass 对该 range 零进展、每轮空转烧预算,用户侧表现为折叠停滞 + payload 超窗直至内容变化。高严重度、窄触发带。F3:中等偏低(预算边缘)。F4:低(诚实性/测试质量)。 4. 怎么修复的以 walk 为基准对齐 nomination: 5. 是否回归问题无。逐项核查:typecheck 干净;定向套件 18/18(executor parallel 4 + batch contract 10 + e2e 4);模块邻域全套(preflight-* + external-summary* + issue1372/1615/1841/1933/1987/1993/2189)= 306 tests,305 pass / 0 fail / 1 skipped(live-gated);rebase 后定向复跑 18/18。单 range 旧路径字节不变(gate 要求 ≥2 viable ranges);legacy-streaming 与 emergency-digest 通道未触碰;kernel 未触碰;walk 本体除遥测去重外未改。相对 PR 原始 head 的行为差已在 PR 正文 "Review fixes" 节披露:span 选择改为 oldest-first(与 walk 一致)、preview-reject/empty-render 不再投毒(严格更多可折叠量)、<2-nomination 边缘 6. 是否破坏缓存不涉及。apply 路径复用原 7. 是否改了设计方向不变。是把 PR 自身机制修正到真正的 walk parity;option A(拼接请求+响应切分)仍在 #2678 待议,本 PR 范围不变。 8. 是否引入了新的配置无。 披露:① PR 分支已 rebase 到当前 master 并 push(force-with-lease 仅限该 bot-authored 分支——原两提交作者均为 ework-agent,authorship 保留,master 未动,§7.2 要求 rebase-to-current-master);② PR 正文已 patch:修正 F4 相关不实表述、更新 behavior-change 表与 Tests 节、新增 "Review fixes" 节(含 vs 原始 head 的行为差清单);③ CI 已在新 head 判定:可合 —— 待 CI 在新 head 上转绿后由人合并;#2667 随合并关闭,#2678(option A)保持打开。 |
[bot] 🏷 补报 CI:新 head PR 当前状态:open / mergeable / base master,正文已含修正后的行为变更表与 "Review fixes" 节。结论维持:可合,请人合并(合并后 #2667 自动关闭,#2678 option A 保持打开)。 |
[bot] 🏷 已按指示处理,关闭前先验证了取代关系(不凭描述直接关): #2806 确实完整包含本 PR:其分支( #2806 状态:open / mergeable / head 已执行:PR #2723 已关闭(closed, not merged),避免与 #2806 在同一批三个文件上 double-land。本线程工作到此收工;后续审查/合并以 #2806 为准。 |
What
Two facets of one mechanism: external-summary requests serialize on the user-request critical path.
Executor half (#2667).
ExternalSummaryExecutor.executeBatchawaited each work item end-to-end before starting the next, so the pool bounded cross-session calls but never a single batch: k ranges in onesummarize()call still cost k sequential round trips. v0.1.191 evidence from the issue: batch wall time scaled linearly with range count (4→62s, 6→85s, 7→88s, 9→180s ceiling) while per-range completions were spaced 10–17s apart.Driver half (#2678). The preflight fold pass submitted ONE external-summary request per range, sequentially, awaited inside the session lock on the user-request path:
23s/request → a k-range pass cost k×20–40s of user-visible stall, capped by the shared per-pass deadline of 180s that then cut the rest off (batch=deadline, whole ranges never dispatched). v0.1.191 bili.log: 390 ranges / 211 passes ≈ 1.85/pass; folded requests waited 30–180s vs 77–97ms normal; the log line was hardcodedcompleted (preflight) 1/1 range(s).This is the filer's option B for #2678 (several requests, submitted together so the pool runs them concurrently) combined with the #2667 fix it depends on — without concurrent
executeBatch, a single N-itemsummarize()call would have run serially inside the executor and gained nothing. Option A (one concatenated request + response splitting) is deliberately NOT in this PR: it changes the summary prompt/response contract against arbitrary external models and needs its own design discussion; #2678 stays open for it.How
src/external-summary.ts—executeBatchnow dispatches up toconcurrencyworkers pulling indices from a shared counter. Results stay index-aligned withwork: the reported list is the dense prefix of settled ranges (downstream zips entries with their work slot); an early stop (caller abort / any item hitting the shared deadline) latches the workers so unstarted ranges are never dispatched and are reported as absent, not fabricated.executeUntilitself is untouched — per-item target-timeout fallback, credential rotation, invalid-summary classification all unchanged.src/preflight.ts— each round now nominates up toMAX_PREFLIGHT_SUMMARY_BATCH = 4disjoint viable ranges (oldest first, OLDEST FULL-BUDGET CHUNK per range — the same first choice the serial walk makes, byte-identical preview/content/chunking gates) into a SINGLEplan.summarize(workItems, signal)call. Successful summaries commit through the same apply path as the walk (the old inline success tail was hoisted intocommitFold, used by both). Ranges the batch cannot summarize stay OUT of the skipSet: later rounds re-nominate them while siblings keep folding, and once the batch applies nothing in a round the serial walk takes them over with its halving / transient retries. A rejected preview or an empty render never poisons the rest of a multi-chunk range (walk parity). Gated on the external lane and ≥2 viable ranges.completed (preflight) N/M range(s); calls=C; …;describeExternalSummaryFailurereports the first failing slot instead of slot 0 (single-range output identical).Behavior changes (old → new)
executeBatchintra-batch executionsummaryCallsaccounting for nominated rangescompleted (preflight) 1/1 range(s)hardcodedN/M range(s); calls=CNo change to: wire/message shape, config schema, persistence, kernel, summary prompt wording, per-call retry/backoff semantics (transient-empty retries, replay backoffs, target-timeout fallbacks), or the shared per-pass deadline contract (still one deadline per plan; #2646's fresh-budget-per-call composes with this — a single N-item call gets one fresh budget, which is exactly what batching needs).
Config surface
None. No new fields, env vars, or CLI flags.
MAX_PREFLIGHT_SUMMARY_BATCH = 4is a private constant sized at the executor pool's default concurrency (extra work items beyond it would only queue inside the same pass — they still run in this PR's concurrent executor, just after the first wave).Tests
tests/external-summary-executor-parallel.test.ts(4 tests): bounded concurrency (third item starts only after a slot frees), index alignment under out-of-order completion, caller-cancellation dense prefix + stop latch (unstarted range never dispatched), shared-deadline dense prefix under pool 1.tests/external-summary-batch.test.ts: the 'ordered fallback chain' assertion pinned cross-range call serialization (it pinned exactly the bug [Bug][external-summary] ranges in one batch are summarized sequentially — concurrency only applies across batches (6 ranges = 85 s on the request path) #2667 removes) — rewritten parallel-safe (per-target subsequence order preserved, cross-range interleaving allowed); all other contract assertions untouched and green.tests/issue2678-preflight-batch-fold.test.ts(e2e, hermetic mock upstreams, 4 tests): (1) ≥2 ranges fold in one pass with measured summary-lane overlap (peak concurrency ≥2 via interval sweep-line) and ≥2 active blocks; (2) a batch-failed range (500 on its content) is retried across later rounds instead of dropped silently (≥2 attempts); (3) a multi-chunk kernel range (same-role run with an over-budget message plus a sub-minimum tail) carries its OLDEST full-budget chunk in the first batch wave and keeps folding chunk by chunk (≥5 blocks) — red under the previous tail-first + poison-once nomination, green as-is; (4) a batch-failed range accumulates ≥3 retries across rounds while siblings keep folding.tests/preflight-*+tests/external-summary*+ issue1372/1615/1841/1933/1987/1993/2189 neighbor set = 306 tests, 305 pass / 0 fail / 1 skipped (live-gated). Rebased onto current master (zero file overlap with the intervening commits); global suite left to CI per local testing discipline.Review fixes (landed as follow-up commits on this branch)
A code-level review of this PR found three defects in the new nomination path; all fixed here (
fix(preflight)+test(preflight)commits):spans[0]of the REVERSED worklist — the newest tail chunk. It was neither the "largest span" the original description claimed nor the walk's oldest-full-chunk-first choice. For single-chunk ranges (the common case) the two agree; for multi-chunk ranges (>0.6×window) the batch then summarized the smallest newest piece instead of the biggest oldest chunk. Nowspans.pop(), byte-identical to the walk.skipSeton preview-reject / empty-render, where the walk only continues to the next span. Combined with (1), a sub-minimum newest tail (token regime,minUnits = 0) killed a range whose older chunks were perfectly foldable for the rest of the pass — zero-progress passes where the payload stays over window until content changes. Both adds removed (walk parity); pinned by e2e test (3).summaryCallsbefore knowing whether the batch would fire; a sub-batch set (<2 nominations) falls through to the walk, which reserves its own calls — double booking could tripbudgetHitearly (effective budget roughly halved in that edge). Reservations now commit only when the batch fires. Also dedupedrangesTried(was incremented at both sites; feeds only the failure-detail string).Deltas introduced by these fixes versus this PR's ORIGINAL head: span selection now matches the walk exactly (older-first); preview-reject / empty-render no longer poison a range (strictly more folds available per pass);
summaryCallsin the <2-nomination edge is now what the walk alone accounts (lower than the original head's double booking). Everything else in this PR is unchanged.Hot-file contention notes
src/external-summary-runtime.ts, which this PR does NOT touch; semantics compose (batching benefits from per-call fresh budgets rather than conflicting).Closes #2667. Part of #2678 (option B interim; option A remains there).