Skip to content

[PR] fix(compress): name non-fetch external-summary failures instead of swallowing them - #6890

Open
sx4im wants to merge 1 commit into
ranxianglei:masterfrom
sx4im:fix-6885-external-summary-failures
Open

sx4im wants to merge 1 commit into
ranxianglei:masterfrom
sx4im:fix-6885-external-summary-failures

Conversation

@sx4im

@sx4im sx4im commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

What broke

compress with external summaries failed every range on long payloads, and the failure gave nothing to work with: every candidate showed up as a bare tN:error with no warn line, and the result always blamed compress.maxSummaryLength, a knob that cannot help a budget or parse failure.

What changed

  • The executor now keeps the safe cause of a non-fetch rejection on the attempt (t0:error (External summary input exceeds the target context budget)) and logs it once. Only our own fixed error strings are named; unknown provider errors stay opaque, so no provider text leaks into results.
  • The compress failure text now picks its advice from the batch itself: budget knobs on deadline/timeout, maxSummaryLength only for over-cap replies, and the named cause otherwise.

Not changed on purpose: the 25s/50s default budgets. Those are the other half of #6885's problem 1, but changing defaults is a product decision for the owner, so this PR only makes the failure diagnosable instead of misattributed.

How verified

  • New regression tests in tests/external-summary.test.ts (4): named cause plus a single warn for the context-budget guard, named parse failure, unknown errors stay opaque, advice branches.
  • Updated the three existing expectations that asserted bare error attempts (HTTP 503/400 rejection, incomplete responses).
  • Affected suites (external-summary, external-summary-http, external-summary-batch, external-summary-executor-parallel, wire-contract-go-session, external-summary-runtime, issue2484-overflow-and-summary): 148 tests, all pass. Typecheck clean on the touched files. Full-suite verdict is remote CI.

Fixes #6885

…allowing them

The executor discarded every candidate rejection, so a context-budget guard or a post-fetch parse failure left only a bare tN:error with no warn line, and the compress failure text blamed maxSummaryLength for every failure. Capture the safe cause for our own fixed error strings (unknown provider errors stay opaque), log it once, and branch the failure advice by the batch's own stop reason: budget on deadline, maxSummaryLength only for over-cap replies.

Fixes ranxianglei#6885
@ranxianglei ranxianglei changed the title fix(compress): name non-fetch external-summary failures instead of swallowing them [PR] fix(compress): name non-fetch external-summary failures instead of swallowing them Nov 16, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: not a duplicate. #6890 is the only open PR fixing #6885. Searched open PRs + issues: no other PR targets the same issue. The related external-summary PRs (#5564 / #3659 / #3660 / #3735 / #5105 / #5545) are different issues (different symptom/mechanism) — #3659/#4833 added the outcome classification (timeout/error/invalid_summary); #6890 adds the named cause (detail) on error outcomes. Complementary, not overlapping.

Base: base_sha == master HEAD (4ea79ac), no conflicts (mergeable=true; mergeable_state=blocked only because the test shards are still running + awaiting review).

I read the full diff against the surrounding code (src/external-summary.ts, src/external-summary-compress.ts, src/external-summary-http.ts, src/logger.ts). Findings:

  • No-leak rule holds. All 8 named causes are our own fixed error strings thrown in src/external-summary-http.ts:148,157,173,181,183,186,234,304 and src/external-summary.ts:276. The only dynamic part is the HTTP status code (a number) at line 304. summaryErrorDetail matches by startsWith on these prefixes, so an unknown provider/network error stays opaque (pinned by the new "unknown error stays opaque" test). No provider text can leak.
  • No double-warn. warnedByCandidate is set true only for the upstream-rejection prefix — the one case the HTTP candidate already warns (external-summary-http.ts:299). The executor's new warn is suppressed there; for the sync guard / parse failures (which never warn today) it logs exactly once.
  • Advice branches correctly. externalSummaryFailureAdvice picks budget knobs on deadline/timeout, maxSummaryLength only on invalid_summary, the named cause on a detailed error, a neutral pointer otherwise — matching both of [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885's scenarios (budget exhaustion → budget advice; context-budget guard → named cause, not maxSummaryLength).
  • Tests are real. 4 new + 3 updated; they assert specific outcomes (not tautologies) and would catch the original bare-error regression.

代码审查八问

  1. 是什么问题: [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 里 compress 走 external summary 时,非 fetch 失败(同步守卫 / 解析失败)在 attempts 里只剩裸 tN:error、零 warn 行;且失败文案一律把用户引向 compress.maxSummaryLength——一个对预算/解析失败毫无作用的旋钮。
  2. 什么原因导致的: executeUntil 的 .then 错误处理器把 rejection 原因整个丢弃(() => ({ kind: "error" })),既不落 warn 也不带 detail;失败文案写死 maxSummaryLength 建议,不看 batch 的真实 stop reason。
  3. 影响范围与严重程度: 所有配置了 external summary 的长载荷 compress 失败现场——日志/工具结果完全无法定位真实原因(只能进 bundle 反查)。属"可诊断性被毁"级(p2):压缩本身仍会失败(问题 1 的预算耗尽本 PR 未修),但失败原因从此可读。
  4. 怎么修复的: ① 执行器保留非 fetch rejection 的"安全原因"在 attempt 上(summaryErrorDetail 只认我们自己的固定错误串,未知 provider 错误保持 opaque 不泄密),并落一次 warn(上游 rejection 候选已 warn 过,用 warnedByCandidate 抑制二次 warn);② 失败文案按 batch 自身 stop reason 分支:deadline/timeout→预算旋钮,invalid_summary→maxSummaryLength,error 带 detail→指向具名原因,其余→指向 attempt outcomes。
  5. 是否有回归问题: 审过的既有行为——
    • 失败文案(模型可见):写死 maxSummaryLength → 按原因分支,是改进非回归(旧文案对预算/解析失败是误导)。但这是 model-facing text change,PR 正文缺 ⚠️ MODEL-FACING TEXT CHANGE 顶栏(§7.3 要求)——流程瑕疵,非正确性问题。
    • describeExternalSummaryFailure 输出:error attempt 现在带 (detail),模型可见文本变长——改进;该函数本就渲染 a.detail,只是以前 error 没填。
    • 新增 warn 行:纯增量诊断,不改既有行为。
    • AttemptResult 加可选字段 detail? / warnedByCandidate?:内部类型,可选,既有 { kind: "error" } 构造不受影响。
    • 高危模式逐项:无 fail-fast 变吞错、无阈值/时序漂移、无输出格式破坏(增量加 detail,非改格式)。
  6. 是否破坏缓存: 不涉及——纯失败路径诊断/文案,不碰缓存/重试/游标/去重键。
  7. 是否改了设计方向: 不涉及——不加新能力面,只在既有失败路径补诊断;预算默认值(问题 1)刻意不动,留给 owner 决策。
  8. 是否引入新配置: 不涉及——无新 env / config / 默认值。

Key caveat for the owner

This PR does NOT fix #6885 problem 1 (the actual 100% failure). Problem 1 is the default 25s/50s budget + serial attempts: t0 eats 25s → t1 is killed by the 50s batch deadline → t2 never even sends a request, so the chain fails 100% on 30–365KB payloads. The PR only makes the failure diagnosable (you can now see it's a budget problem); it does not change the default budget behavior. Changing defaults is a product decision the author explicitly deferred. So after merging, long-payload compress still fails 100% — only the cause is now readable. Root-fixing problem 1 (budget defaults / targetTimeout = min(target, total − elapsed)) needs a separate owner decision.

Minor nits (non-blocking)

  • Missing model-facing text banner: the failure text is a model-visible compress receipt; §7.3 requires a ⚠️ MODEL-FACING TEXT CHANGE top banner + regression inference in the PR body. Absent. Suggest the author add it (regression inference covered above).
  • Error-string prefixes duplicated in two places: throw side (external-summary-http.ts 148/157/173/181/183/186/234/304) and match side (summaryErrorDetail in external-summary.ts). If a throw-side string changes, the match side silently stops matching (degrades to opaque — safe, but a maintenance hazard). Could be named constants; out of scope here.
  • find((e) => e.status !== "success") ?? results[0] appears in both externalSummaryFailureAdvice and describeExternalSummaryFailure (2 places — §7.3 says "≥2 consider extracting"). Could extract; non-blocking.

CI status

Static gates all green (static-gates/ubuntu, mysql-contract, config-surface-guard, lifecycle-ledger-guard, lock-sync, plan, docs-coverage); test shards still running (ubuntu/windows × node 22/24). Final mergeable verdict pending the test shards.

Verdict

Code review is clean; direction is correct. This is a solid "stabilize" PR: it fixes #6885 problem 2 (silent failures) and problem 3 (misdirected advice), making the failure diagnosable; problems 1 (100% failure) and 4 (docs) are deliberately deferred. Merge once CI is green — but the owner should note: after merge, long-payload compress still fails 100% (problem 1 unfixed); the budget-default decision is separate.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI progress update: the first two ubuntu test shards are green — test (ubuntu-latest, 24, 1) and test (ubuntu-latest, 22, 1) passed. 9 shards still running (windows × node 22, remaining ubuntu 22/24). No failures so far; all static gates + e2e + lock-sync/config-surface/plan/docs-coverage green. I'll confirm the final mergeable verdict once the remaining test shards land.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants