Skip to content

[PR] External summary failures: name the real abort source and expose request size (#4833) - #4839

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-26_external-summary-abort-provenance
Oct 26, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-26_external-summary-abort-provenance

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4833

What this changes

When an external-summary dispatch dies from one of bili's OWN deadlines, every log surface said client-abort:

[warn] [external-summary] candidate <model> attempt 1/3 failed before response (client-abort): This operation was aborted

Root cause: the HTTP candidate classifies any pre-response fetch failure whose incoming signal is aborted as client-abort (src/upstream-fail.ts, externalAborted short-circuit) without asking WHY the signal aborted. But the signal the executor hands out is its own per-attempt controller, and three different sources can abort it: the per-target deadline timer, the total/batch deadline, or a genuine downstream cancellation. All three rendered identically, so an operator could not tell a budget miss from a user disconnect, and no request-size context accompanied the failure.

The fix makes the abort carry its provenance and lets each surface say the truth:

  • Typed aborts (src/external-summary.ts): the executor now raises SummaryAbortError with source ∈ target-timeout | batch-deadline | cancelled riding signal.reason. Per-attempt signals propagate the overall reason, so the HTTP layer reads ground truth instead of assuming.
  • Honest labels (src/external-summary-http.ts): the warn line reports the real source — target-timeout / batch-deadline / client-abort (the latter only for genuine downstream cancellation or foreign unmarked aborts, preserving legacy attribution). Transport analysis (classifyUpstreamFailure) still runs whenever no typed reason exists, so provider/network errors keep their existing kinds and retry gating is unchanged.
  • Safe diagnostics on the same line: elapsed ms, configured target=/total= budgets, protocol, stream mode, payload byte size, input token estimate, output char cap. No content, no URLs beyond the already-masked error text, no secrets.
  • Attempt detail flows into the existing completed N/M range(s) log line (and the preflight variant of it) via SummaryAttempt.detail: e.g. attempts=[t0:timeout (target-timeout 25000ms (elapsed 25002ms))].
  • Tool result (src/external-summary-compress.ts): when compression fails wholesale, the bounded secret-free diagnostic string is appended to the tool text so the agent sees the same provenance the operator sees.
  • Docs (CONFIGURATION.md en+zh): single-target chains are still capped by targetTimeoutMs per dispatch even when totalTimeoutMs is larger; deadline aborts are NOT retried (the retry loop only re-dispatches transient provider verdicts and fail-fast network deaths); sizing guidance for large-context non-streaming summaries (non-streaming pays full generation latency before the first byte — raise targetTimeoutMs accordingly and keep totalTimeoutMs ≥ fall-through depth × targetTimeoutMs).

Behavior changes (old → new)

Surface Old New Why
Warn line label for a per-target deadline abort (client-abort) (target-timeout; elapsed=…, budget target=… total=…, protocol=…, stream=…, payload≈…B, input≈…tok[, outCap=…ch]) internal deadline is not a client disconnect
Warn line label for a total/batch deadline abort (client-abort) (batch-deadline; …) same
Warn line label for genuine caller cancellation (client-abort) (client-abort; …) unchanged meaning, gains diagnostics
Warn line label for unmarked foreign abort (client-abort) (client-abort; …) legacy attribution preserved
Provider/network error labels & retry gating transport kind unchanged no semantic drift
completed N/M range(s) log attempts without detail for timeouts timeout attempts carry source budget (elapsed) detail ask #2 of #4833
Tool failure text on wholesale failure generic sentence generic sentence + [batch=… result=… attempts=[…]] (first failing range only, per the existing #2678 bound) ask #1/#2 of #4833

No retry behavior changed: a deadline-aborted signal still never retries (the gate requires a classified transient kind AND an un-aborted signal); only the LABEL and the accompanying diagnostics changed. No wire format, persistence, or config-schema change.

Config surface

No new config keys, env vars, or CLI flags. Existing externalSummary.budget.* semantics are untouched; CONFIGURATION.md en+zh gain explanatory paragraphs only (deadline cap on single-target chains, no-retry-after-deadline rule, sizing guidance).

Kernel boundary

No changes under kernel/ — all seams are host-side (src/external-summary*.ts). No kernel-change declaration needed.

Coordination

Related open request #3638 / PR #3659 asks for broader per-candidate diagnostics in the tool result. This PR deliberately stays minimal and complementary: it fixes the misclassification at the source (executor → signal.reason) and appends the existing bounded diagnostic string to the tool text; it does not implement #3659's fuller per-candidate report, so the two should merge cleanly (only the tool-text append touches shared territory).

Verification

Rebased onto current master (341e76c). Local on the pushed head: npm run typecheck clean; targeted suites green — new tests/issue4833-summary-abort-provenance.test.ts (executor-level: target-timeout / batch-deadline / caller-cancel each produce the right outcome, detail, and signal.reason; HTTP-level: hanging upstream logs target-timeout not client-abort; caller cancel over HTTP logs client-abort; foreign unmarked abort keeps legacy attribution; leak-free assertions) plus the existing external-summary suites (175 tests pass). npm run build clean (dist external-dependency gate OK, no new deps). Full-suite verdict: remote CI on this head.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-26_external-summary-abort-provenance (e130c81)

Latest build — install this exact version

npm install -g billion-context@0.0.6250-pr.4839.7236ff6

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
e130c81 billion-context@0.0.6250-pr.4839.7236ff6 build
c2d3f67 billion-context@0.0.6247-pr.4839.9e89619 build
16bf882 billion-context@0.0.6246-pr.4839.c1be26b build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4839.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title External summary failures: name the real abort source and expose request size (#4833) [PR] External summary failures: name the real abort source and expose request size (#4833) Oct 26, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 检索 open+closed 的 issue 与 PR(external summary / client-abort / abort provenance / targetTimeoutMs 等签名):无同机制在途 PR。相关但不同:#3638 / PR #3659 要的是「工具结果里更完整的逐候选诊断」(面更广、展示层),本 PR 修的是「内部 deadline 中止被误标为 client-abort」的分类根因(executor→signal.reason 来源传递),两者互补、按 PR 描述只有一处工具文案追加会碰同一块地——合入顺序上本 PR 先行不冲突。

标签: pr + bug + scope/wiring + p3(诊断面缺陷:日志误标 + 缺请求规模上下文,压缩行为本身不变,已有绕过手段)。

开始独立审核:先读原 issue #4833 与分支 diff(head 16bf88274,单 commit,+268/-29,9 文件),再逐项过回归面与测试有效性,结论按八问回复。

@ranxianglei ranxianglei added bug Something isn't working p3 scope/wiring labels Oct 26, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

先说结论:无需我推送任何改动——被标记的 unused-exports 红在我处理期间已被并发提交收敛。

我拉取分支时,头已从平台通知里的 16bf8827 前进到 c2d3f67d(11:24Z 推送),新提交正是修这个红:c2d3f67d fix(gate): un-export SummaryAbortSource。我原本准备的是「加 WAIVER 豁免」方案,确认已被抢先修复后已丢弃,没有再推竞争性改动(避免重复/冲突)。

结构化八问:

  1. 是什么问题:unused-exports CI 门禁在本 PR 变红——新增的类型别名 src/external-summary.ts:25 SummaryAbortSource 被判 INTERNAL_ONLY(导出了,但所有引用都在本文件内)。
  2. 什么原因导致的:该 PR 一次新增三个导出;SummaryAbortError(类)、summaryAbortSource()(函数)有跨文件引用故不触发,但类型别名 SummaryAbortSource 本身只在 external-summary.ts 内被文本引用,门禁按字面词频统计即判它「仅内部使用」。
  3. 影响范围与严重程度:仅 unused-exports 门禁拦截合并,无运行时/用户面影响;属构建门禁阻塞,非功能缺陷。
  4. 怎么修复的:已由并发提交 c2d3f67d 完成——去掉该类型的 export(降为模块内部)。此做法在本仓库安全:tsup 不产 .d.ts、tsc --noEmit 不发声明文件,故两个在签名里引用它的导出符号(SummaryAbortError.source 字段、summaryAbortSource() 返回型)不会触发 TS4023 类错误。我在当前头本地复跑门禁确认 ✓ unused-export gate: clean。
  5. 是否有回归问题:无。单行去导出,类型仍在模块内部及两个导出签名中被正常引用;无行为/wire/配置/持久化变更。当前头 c2d3f67d 上各 e2e 套件已全绿。
  6. 是否破坏缓存:不涉及(未触碰任何缓存/重试/游标逻辑)。
  7. 是否改了设计方向:不涉及(无架构/数据流转向)。
  8. 是否引入新配置:不涉及(无新 env/config/flag)。

收工判定:不推送、不立项。被标记失败已由 c2d3f67d 收敛,本地门禁验证通过;当前头主 CI(in-progress)含该门禁,预期跑完转绿。

…re the retry loop (review fix #4839)

diagnostics was joined into a string before the attempt loop, freezing
elapsed near zero for every failure line — a 25s target timeout logged
elapsed=0ms, defeating the diagnostic this PR adds. Elapsed is now
computed per failure site; static bits stay pre-computed. The stalled-
upstream regression test now also asserts the elapsed value reflects
the real ~200ms hang (>=150ms), pinning the fix.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核完成(独立重读了 diff 全文、issue #4833 全线程与既有代码路径,未照抄 issue 结论)。结论:可以合并——但审核中发现了两个问题,已直接修掉并推到 PR 分支,新 head e130c81d8 CI 全绿(10/10 check-suites success)。

审核发现并修复的两个问题:

  1. CI 红:unused-exports 门禁(PR 原 head 16bf88274)。新增导出 SummaryAbortSource 只在本文件内被引用,触发 unused-exports 门禁;且主 CI 的 test lanes 都 needs: 该门禁,门禁一红全部测试 lane 被跳过,等于「红了但没跑任何测试」。修法:去掉 export 关键字(一行)。说明:我独立发现并实现了这个修复,但作者的会话几乎同时推了一个逐字节相同的修复(c2d3f67d1),我的重复提交在 rebase 时丢弃,以先到者为准。
  2. elapsed 诊断冻结(我修,commit e130c81d8)。warn 行里的 elapsed= 是在重试循环之前拼好的静态字符串——一次真实的 25s 超时打出来会是 elapsed=0ms,恰好砸在这个 PR 要交付的核心承诺上。旧测试只断言 elapsed=\d+ms 存在所以没抓住。修法:elapsed 改为每次失败现场求值,并把测试从正则存在性升级为数值钉住(≥150ms @ 200ms 预算)。本地实测确认:60ms 取消现在打出 elapsed=61ms(修复前约 0ms)。

结构化八问:

  1. 是什么问题:外部摘要(external summary)调用因为 bili 自己的超时预算而死掉时,所有日志面都报 client-abort(客户端取消)——把内部超时误标成用户侧取消;而且日志里没有任何请求尺寸信息,大上下文场景下无法判断该调哪个预算参数。
  2. 什么原因导致的:执行器(executor)有三类超时会中止同一个 per-attempt AbortController——单目标超时、整批总超时、真正的下游取消——但 JS 的 signal 只携带「被中止」这一个布尔量,不携带原因;HTTP 候选层拿到中止信号后短路判成 client-abort,从不追问是谁中止的。原因在执行器和 HTTP 候选之间的传递链上就丢了。
  3. 影响范围与严重程度:纯诊断/可观测性缺陷,不影响压缩结果本身(失败判定、降级回退行为都不变),但影响所有 externalSummary 配置用户的排障体验——内部超时被系统性误读为客户端取消,可能误导用户去查客户端而不是调 bili 预算。p3。
  4. 怎么修复的:给中止加类型——执行器用携带 source ∈ {target-timeout, batch-deadline, cancelled} 的 SummaryAbortError 作为 signal.reason 传播到 HTTP 层,fetch 点按真实来源打标;同时在同一行追加安全数字诊断(elapsed、预算值、协议、流式模式、payload 字节数、输入 token 估算、输出上限),不含内容/URL/密钥。文档 en+zh 同步补充了「单目标仍受 targetTimeoutMs 封顶」「deadline 中止不重试」「非流式大上下文调参建议」。
  5. 是否有回归问题:逐项审过既有行为——①重试门控不变:deadline 中止前后都不重试,瞬态网络类(proxy-reset/upstream-reset/connect-refused/connect-timeout/dns)的重试条件一字未动;②真正的下游取消和外来无标记 abort 保留 client-abort 标签(grep/告警连续性);③attempts 结果的 outcome 枚举不变,detail 为纯增量字段,渲染器是 External summaries over compress.maxSummaryLength are silently discarded after paying full latency — and the cap is not configurable #3635 既有的;④createSummaryHttpCandidate 新增第 3 参可选,唯一调用方 runtime + 测试;⑤每轮 summarize 多两次 defaultCountTokens(字符扫描级,可忽略);⑥kernel/ 零改动,wire/持久化/配置 schema 零改动,两种压缩模式均不受影响(host 侧日志通道)。未发现回归。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及——只在既有 executor→HTTP→日志的工具通道上加类型化原因与安全诊断,不改任何决策逻辑。
  8. 是否引入了新的配置:不涉及。无新 env/配置键/CLI flag。

行为变更披露(旧→新):

场景 旧 新
单目标 deadline warn (client-abort) (target-timeout; elapsed=…, budget target=… total=…, protocol=…, stream=…, payload≈…B, input≈…tok[, outCap=…ch])
整批 deadline (client-abort) (batch-deadline; …同上)
真·下游取消 / 外来无标记 abort (client-abort) (client-abort; …+诊断)(语义不变,增益诊断)
provider/network 类失败 各自标签 不变
completed N/M range(s) 行 超时 attempts 无细节 带 tN:timeout (source 预算ms (elapsed Xms))
工具失败文本 无 追加 [batch=… result=… attempts=[…]](首个失败 range,受既有 #2678 长度边界约束)

CI:e130c81d8 上 10/10 check-suites success(含此前被跳过的完整 test matrix)。#4599 已留一行记录。等 owner review 后由人合并(agent 不合并 PR)。

@ranxianglei
ranxianglei merged commit d562d76 into master Oct 26, 2026
47 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.863-master.d562d76 (rolling master tag).

npm install -g billion-context@0.0.863-master.d562d76  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p3 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

External summary deadline aborts are logged as client-abort; expose timeout source and request size

1 participant