Repository navigation
feat(#2657): expose external-summary pool concurrency as config - #2661
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2875-pr.2661.fbb297eThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2661.tgz
npm install -g packageInstall instructions are refreshed on each push. |
compress.externalSummary.concurrency (integer 1..32, default 4) sizes the process-wide summary pool: every session's preflight auto-fold and model-initiated compress calls share one ExternalSummaryExecutor, whose size was hard-coded at 4. The pool now grows to the largest value demanded by in-flight batches (refcounted scopes) and returns to base when they settle, so chains with different values can coexist; absent key keeps today's behavior exactly.
…gate SUMMARY_CONCURRENCY_MIN/MAX and SummaryPoolOptions are referenced from no other module — the gate requires them unexported.
90882b9 to
66ee17c
Compare
[bot] 🏷 PR #2661 review(feat(#2657):expose external-summary pool concurrency as config) 先说处置:原 head 的 CI 里 修了什么(commit
本地验证(rebase 后的新 head)
结构化八问
测试有效性:6 个新测试断言有牙——「持作用域期间池放行并发工作」「结算后回到 base(用 50 tick 轮询钉住 B 必须排队)」「更大在途需求胜出」「非法作用域拒绝且不派发」「chain 形式与展开 rail 形式都携带/都校验」,能抓住这类 bug 回归,不是恒真断言。 判定:方向正确、实现干净、既有行为无回归;待新 head 的 CI 全绿即可合(目前全量矩阵在跑,本地门禁已过)。 |
[bot] 🏷 已关闭本 PR(未合并),按指示以 #2806 为准。 关闭前做了核验,#2806 的 stack 确实完整包含本 PR 的两个提交(cherry-pick 到当前 master
加上 #2646/#2723/#2761 的其余 4 个 cherry-pick 与两个 fusion fix( |
Closes #2657
What
Every session's external-summary work — preflight auto-folds AND model-initiated
compresscalls — shares ONE process-wide pool whose size was hard-coded at 4 (new ExternalSummaryExecutor(4)). On a host running many sessions against a summary endpoint that is fast in isolation but slow under aggregate load, queue time eats the per-batch deadline before any attempt dispatches: the issue reports 32/402 batch success withbatch=deadline+ emptyattempts, plus the resulting 10-minute auto-fold backoff oscillation.This adds
compress.externalSummary.concurrency(integer,1..32, default4) — a sibling ofbudget/autoFoldin the chain.Pool semantics (design decision)
The value rides each batch as a refcounted scope: while batches carrying it are in flight, the shared pool grows to the largest value demanded; when those batches settle it returns to base size. Consequences:
The executor primitive itself accepts any positive integer (same contract as its constructor); the
1..32product range is owned by the settings layer, which rejects out-of-range at save time (Web API → HTTP 400) and on the request path (warn + legacy preflight fallback), exactly like every other chain field.config surface
Per the owner-gated configuration-surface discipline:
compress.externalSummary.concurrency— integer1..32, optional, default4(= today's hard-coded value). Documented in CONFIGURATION.md en+zh including the interaction note from the issue: fold rate of a batch is bounded byconcurrency × targetTimeoutMs, and a saturated pool surfaces asbatch=deadlinewithattempts=[]— if direct calls to the target succeed quickly while you see those lines, the pool, not the timeout, is the binding constraint.budget,providers, or the three-levelcompresshierarchy carries pool sizing; the value lives inside the executor, not in any config object.budget.*andautoFold*are config-only; this follows them.min=1 max=32), disabled while the form has parse errors; i18n zh/en added.Behavior changes (flagged per policy)
Tests
tests/external-summary-settings.test.ts: the knob rides both the chain form and the expanded rail form (absent →undefined→ base size); boundaries 1/32 accepted;0/33/-4/2.5/"12"rejected by BOTH parsers; disabled-chain early-return stays inert.tests/external-summary-batch.test.ts: executor pool scoping — a scoped batch admits concurrent work under the grown pool; the pool returns to base once the scoped work settles (a later batch queues behind a sole base-size permit again); an oversized scope yields to a larger in-flight demand; non-positive scope rejects without dispatch.tests/external-summary*.test.tsgreen (settings 40, batch 19, runtime 17, autofold/http/badge/live 64 total incl. 1 design-skip), build green. Full regression is CI's job.Not included here (separate mechanism, filed separately): not arming the 10-minute auto-fold backoff when a batch died in the queue without any attempt (
attempts=[]).