Skip to content

feat(#2657): expose external-summary pool concurrency as config - #2661

Closed
ranxianglei wants to merge 2 commits into
masterfrom
2026-10-10_external-summary-concurrency
Closed

ranxianglei wants to merge 2 commits into
masterfrom
2026-10-10_external-summary-concurrency

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2657

What

Every session's external-summary work — preflight auto-folds AND model-initiated compress calls — shares ONE process-wide pool whose size was hard-coded at 4 (new ExternalSummaryExecutor(4)). On a host running many sessions against a summary endpoint that is fast in isolation but slow under aggregate load, queue time eats the per-batch deadline before any attempt dispatches: the issue reports 32/402 batch success with batch=deadline + empty attempts, plus the resulting 10-minute auto-fold backoff oscillation.

This adds compress.externalSummary.concurrency (integer, 1..32, default 4) — a sibling of budget / autoFold in the chain.

Pool semantics (design decision)

The value rides each batch as a refcounted scope: while batches carrying it are in flight, the shared pool grows to the largest value demanded; when those batches settle it returns to base size. Consequences:

  • one session's chain can never permanently resize the pool for everyone else;
  • chains with different values (possible: whole-chain replace across the three-level compress ladder) coexist — the max-in-flight wins, not last-writer-wins;
  • a lower-than-base value is inert (the pool never shrinks below base for others' benefit).

The executor primitive itself accepts any positive integer (same contract as its constructor); the 1..32 product range is owned by the settings layer, which rejects out-of-range at save time (Web API → HTTP 400) and on the request path (warn + legacy preflight fallback), exactly like every other chain field.

config surface

Per the owner-gated configuration-surface discipline:

  • Added: compress.externalSummary.concurrency — integer 1..32, optional, default 4 (= today's hard-coded value). Documented in CONFIGURATION.md en+zh including the interaction note from the issue: fold rate of a batch is bounded by concurrency × targetTimeoutMs, and a saturated pool surfaces as batch=deadline with attempts=[] — if direct calls to the target succeed quickly while you see those lines, the pool, not the timeout, is the binding constraint.
  • Why the existing surface cannot express it: nothing in budget, providers, or the three-level compress hierarchy carries pool sizing; the value lives inside the executor, not in any config object.
  • No new env var (deliberate): nested compress fields ride the three-level cascade by design; the env tier is reserved for top-level keys (彻底整治环境变了滥用,除非必要都走配置文件 #2030). Siblings budget.* and autoFold* are config-only; this follows them.
  • Compat/migration: key absent → behavior byte-identical to before (base pool 4, no scopes registered). No migration, no persistence-format change.
  • Web UI: the summary-config editor gains a "Pool size" number input (min=1 max=32), disabled while the form has parse errors; i18n zh/en added.

Behavior changes (flagged per policy)

  • Unset configs: none.
  • Set configs: previously impossible (the knob did not exist); new users see the widened pool only while their batches are in flight.
  • No change to wire format, message refs, session persistence, retry/backoff timing, or kernel code.

Tests

  • tests/external-summary-settings.test.ts: the knob rides both the chain form and the expanded rail form (absent → undefined → base size); boundaries 1/32 accepted; 0/33/-4/2.5/"12" rejected by BOTH parsers; disabled-chain early-return stays inert.
  • tests/external-summary-batch.test.ts: executor pool scoping — a scoped batch admits concurrent work under the grown pool; the pool returns to base once the scoped work settles (a later batch queues behind a sole base-size permit again); an oversized scope yields to a larger in-flight demand; non-positive scope rejects without dispatch.
  • Local verification: typecheck clean, all tests/external-summary*.test.ts green (settings 40, batch 19, runtime 17, autofold/http/badge/live 64 total incl. 1 design-skip), build green. Full regression is CI's job.

Not included here (separate mechanism, filed separately): not arming the 10-minute auto-fold backoff when a batch died in the queue without any attempt (attempts=[]).

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-10_external-summary-concurrency (66ee17c)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2875-pr.2661.fbb297e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2661.tgz
npm install -g package

Install instructions are refreshed on each push.

ework-agent and others added 2 commits October 10, 2026 19:42
compress.externalSummary.concurrency (integer 1..32, default 4) sizes the process-wide summary pool: every session's preflight auto-fold and model-initiated compress calls share one ExternalSummaryExecutor, whose size was hard-coded at 4. The pool now grows to the largest value demanded by in-flight batches (refcounted scopes) and returns to base when they settle, so chains with different values can coexist; absent key keeps today's behavior exactly.
…gate

SUMMARY_CONCURRENCY_MIN/MAX and SummaryPoolOptions are referenced from no
other module — the gate requires them unexported.
@ranxianglei
ranxianglei force-pushed the 2026-10-10_external-summary-concurrency branch from 90882b9 to 66ee17c Compare October 10, 2026 11:44
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #2661 review(feat(#2657):expose external-summary pool concurrency as config)

先说处置:原 head 的 CI 里 unused-exports 门禁红了(3 个 INTERNAL_ONLY export),已按「小问题直接修」在 PR 分支上补了一个标注 commit 修掉,并把分支 rebase 到当前 master(原来落后 48 个提交)。新 head:66ee17c9b,CI 正在重跑。

修了什么(commit 66ee17c9b)

  • src/external-summary-settings.ts:SUMMARY_CONCURRENCY_MIN/MAX 去掉 export(仅本文件内引用;SUMMARY_CONCURRENCY_DEFAULT 被 runtime 引用,保留)。
  • src/external-summary.ts:SummaryPoolOptions 去掉 export(仅本文件内引用,runtime 传对象字面量不需要类型名)。
  • rebase 说明:master 侧对触碰文件的漂移只有 CONFIGURATION.md/zh 的 diagnostics dump 上限与 agent-providers skip 规则两段,与本 PR 加文档的位置不重叠、无语义交互;代码文件在 master 侧零改动。

本地验证(rebase 后的新 head)

  • node scripts/check-unused-exports.mjs → clean(2 waived,均为既有豁免)。
  • 定向测试 92/92 全绿:tests/external-summary-batch.test.ts(含新增的 4 个池范围测试)、external-summary-settings.test.ts(含 2 个新校验测试)、external-summary-runtime.test.ts、external-summary.test.ts。
  • npm run typecheck 干净。全量矩阵交给 CI。

结构化八问

  1. 是什么问题:进程级外部摘要队列的在途上限被硬编码为 4(src/external-summary-runtime.ts 的 new ExternalSummaryExecutor(4)),本机所有会话的 auto-fold 和模型发起的 compress 共享这一个池,有余量的主机无法调大([Feature] expose external-summary concurrency as config — hard-coded ExternalSummaryExecutor(4) throttles every session on the host #2657 实测 batch 成功率 ~7%、20 次 10 分钟退避,直连目标 1.6s 即回 200——瓶颈是池不是超时)。
  2. 什么原因导致的:执行器并发度是构造参数但从未暴露到配置面;属于功能缺口而非缺陷。
  3. 影响范围与严重程度:特定条件下有问题——多会话/大窗口主机上外部摘要吞吐受限,表现为 batch=deadline + attempts=[] + 退避风暴;默认单链小流量用户无感。体验/吞吐级,非数据损坏。
  4. 怎么修复的:新增 compress.externalSummary.concurrency(整数 1..32,默认 4),随整链替换语义走三级 compress 层级;每个批次把它作为引用计数作用域注册到共享执行器——该批工作期间池子涨到在途需求最大值,批次结束(含取消/超时/抛错,finally 释放)回到基础值;低于基础值的取值惰性无效。保存时校验(越界 → HTTP 400),Web UI 加 Pool size 输入框,en/zh 文档各加一段(含「concurrency × targetTimeoutMs 共同决定折叠速率」「池饱和表现为 batch=deadline/attempts=[] 而非可见错误」的排障指引)。刻意不新增 env var(彻底整治环境变了滥用,除非必要都走配置文件 #2030 纪律)。
  5. 是否有回归问题:逐项过完触碰路径的既有行为,未发现回归:
    • key 缺省时 beginScope 返回 no-op、effectiveConcurrency() 在无作用域时等于 base —— 调度行为与旧版逐字节等价(代码验证 + 4 个测试文件共 86 个既有测试全绿);
    • 默认值 4 = 原硬编码值,构造函数校验不变;src 内唯一调用点(runtime:59)追加的是可选尾参,其余调用方零改动;
    • knownKeys 白名单只多一个键;disabled 链早退在读 concurrency 之前(有测试钉住);
    • 一个已核实的边界:deadline 退出时孤儿 HTTP 调用仍可能持有 permit(permit 由 .finally(release) 持有至真实调用落定),此时 endScope 先跑、池子缩到 base——新准入受限于 base 直到孤儿落定。保守方向,无死锁(孤儿必然落定放行)、无超发,可接受;
    • wire 消息形状、持久化格式、kernel、update.ts、session 身份逻辑:全部未触碰。
  6. 是否破坏缓存:不涉及——纯内部调度器参数,不影响 prefix cache、重试策略、去重键、增量游标。
  7. 是否改了设计方向:没有转向。扩展现有三级 compress 层级与整链替换语义;作用域机制是对现有共享执行器的纯增量扩展。per-target 并发未做(池是进程级的,per-target 表达不了,PR 正文有说明);attempts=[] 不挂 10 分钟退避按 PR 正文留待单独 issue,未混入。
  8. 是否引入了新的配置:是——compress.externalSummary.concurrency,整数,合法域 [1, 32],默认 4(=旧硬编码值),缺省 = 旧行为,无需迁移。CONFIGURATION.md 与 zh 版均已同步;生成式索引种子不跟踪 externalSummary 子字段(autoFold/budget 同样不在其中),无漂移。

测试有效性:6 个新测试断言有牙——「持作用域期间池放行并发工作」「结算后回到 base(用 50 tick 轮询钉住 B 必须排队)」「更大在途需求胜出」「非法作用域拒绝且不派发」「chain 形式与展开 rail 形式都携带/都校验」,能抓住这类 bug 回归,不是恒真断言。

判定:方向正确、实现干净、既有行为无回归;待新 head 的 CI 全绿即可合(目前全量矩阵在跑,本地门禁已过)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI 跟进:新 head 66ee17c9b(rebase + unused-exports 修复)的 CI 已全部通过——24/24 check 绿,含全量测试矩阵(ubuntu-latest 22/24、windows-latest 22/24)、unused-exports、hermetic registry e2e(ubuntu/windows)。

最终判定:可合。 合并请人工操作(系统规则禁止 agent merge):#2661

@ranxianglei

Copy link
Copy Markdown
Owner Author

Superseded by the consolidated stack in #2806 (this PR's commits cherry-picked + conflict-resolved on current master, plus the cross-PR fusion fixes — see its description). Prefer reviewing/merging #2806 and closing this one to avoid double-landing the same three files.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 已关闭本 PR(未合并),按指示以 #2806 为准。

关闭前做了核验,#2806 的 stack 确实完整包含本 PR 的两个提交(cherry-pick 到当前 master d0cebb90 之上):

加上 #2646/#2723/#2761 的其余 4 个 cherry-pick 与两个 fusion fix(f3c55ff2、a47b574b),共 8+2 个提交。本 PR 分支(2026-10-10_external-summary-concurrency)不再维护,避免三个文件双落;后续 review/merge 请走 #2806。

ranxianglei added a commit that referenced this pull request Oct 11, 2026
…-consolidated

external-summary: consolidate #2646 + #2661 + #2723 + #2761 (fresh budgets, pool knob, concurrent batches, Go session header)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] expose external-summary concurrency as config — hard-coded ExternalSummaryExecutor(4) throttles every session on the host

1 participant