Repository navigation
Description
中文摘要:externalSummary 启用时,手动
compress工具对模型提名的每个 range 一对一组装摘要任务,没有任何尺寸上限、折半或分片;一个超大 range 会让全部候选在 fetch 前被 context-budget 守卫拒绝(最大窗口候选则超时),整批exhausted且不回退到经典的当前模型摘要路径。自动 preflight 车道已有完整的分片纪律(CHUNK_FRACTION=.6、pushSpanHalves、6 万字符 T2 预算等),两车道防护不对称。实测案例:39 个嵌套块、重建 work.content ≈ 521,295 token,五链窗口 131072/131072/262144/1048576/272000,1.4–1.7 秒全灭、四链零出站;同会话 18 块小 range 成功形成对照。建议:手动车道复用 preflight 分片原语、全失败时提供 opt-in legacy 回退、拒绝信息给出实测 token 与建议分片数。
Summary
When compress.externalSummary is enabled, the manual compress tool lane (applyConfiguredCompression) builds one summary work item 1:1 per model-nominated range with no size cap, no span-halving and no sharding. A single oversized range therefore fails against every configured target — most via the pre-fetch context-budget guard, the largest-window target via timeout — and the whole batch is refused with reason=exhausted, with no fallback to the legacy in-model summary path. The automatic preflight lane already has a complete size-discipline machinery; the two lanes are asymmetric.
Environment
- billion-context
0.1.213(commit133f3525c), DSH plugin mode, Windows 11, bundledist/index.js - 5 external-summary targets (windows): 131072 / 131072 / 262144 / 1048576 / 272000 tokens
- A manually nominated batch of 39 nested blocks; faithful offline rebuild of the assembled
work.content(direct messages + nested block summaries, samedefaultCountTokenslogic) measured ≈ 521,295 input tokens
Symptom
The batch finished in ~1.4–1.7 s with:
reason=exhausted attempts=[t0:error,t1:error,t2:error,t3:error,t4:error]
- Zero outbound HTTP requests in the proxy log for four of the five targets — they threw before fetch inside
summarize():
if (plan.contextWindow !== void 0 &&
defaultCountTokens(system) + defaultCountTokens(content) + 256 + plan.outputTokens > plan.contextWindow)
throw new Error("External summary input exceeds the target context budget");- The 1,048,576-window target passed the guard but then hit
target-timeoutat the old 120 s budget (a ~521K-token summary cannot finish in 120 s). - A smaller 18-block range from the same session succeeded on the 1M-window target in the same time window — control evidence that credentials, network and endpoints were healthy; size alone explains the failure.
Root cause: protection asymmetry between the two lanes
Preflight (automatic) lane has full size discipline (bundle offsets in 0.1.213):
CHUNK_FRACTION = 0.6— one chunk ≤ 60 % of the main model windowpushSpanHalves— HTTP 400/413 or empty summary triggers half-span retry (floor 4K tokens)TIER_DISTILL_SOURCE_CHAR_BUDGET = 60000— 60K-char cap per T2 distillation sourceMAX_SUMMARY_CALLS_PER_PREFLIGHT = 16,MIN_CHUNK_TOKENS = 2000- A ready-made
splitSummaryContentprimitive also exists in the bundle
Manual lane (applyConfiguredCompression, ~@3740918):
- For each nominated range it assembles
work.content = collectBlockContent(full:false).join(...)1:1 - A bundle-wide search for
shard/splitRange/sliceRangereturns zero matches — no size cap, no halving, no sharding - When all targets fail, the call returns
refused("do not retry unchanged configuration") and never falls back to the legacyapplyRangespath (the current-model summary that works with external summary disabled)
The documented "100K context is enough" guarantee holds for the nudge-driven normal flow (ranges stay small), but the manual tool gives the model no guardrail against nominating a huge multi-block range, and the failure mode is an opaque whole-batch refusal.
Reproduction
- Enable external summary with ≥1 target whose
contextWindowis smaller than the nominated range. - In a long session, call
compresswith one range spanning enough nested blocks to assemble >contextWindowtokens (our case: 39 blocks, ~521K tokens). - Observe sub-2-second
exhaustedwith zero outbound requests for the small-window targets and no partial success stored.
Proposals
Any one of these would close the failure class; 1+2 is the robust combination:
- Reuse the preflight primitives in the manual lane: when a range would fail the smallest target's budget guard, pre-split it (or half-span retry on guard rejection) using the existing chunking code instead of refusing the batch.
- Legacy fallback opt-in: when the whole external chain fails, allow falling back to
applyRanges(current-model summary) behind a config flag — the current "never fall back" semantics (documented) can stay the default. - Minimum viable diagnostics: pre-validate each nominated range against the smallest configured
contextWindowand name the offending range, its measured tokens and a suggested split count in the refusal, instead of a genericexhausted(see also [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 Problem 3 on misleading failure text, and fix: all external summaries failing bypasses consecutive failure protection, repeated compress continues dispatching req | fix: 外部摘要全失败绕过连续失败保护,重复 compress 继续派发请求 #6903 on failure-streak bypass in this same function).
Workarounds applied locally
- Raised budgets to
totalTimeoutMs=600000 / targetTimeoutMs=300000(helps the large-window target only). - Usage discipline: small ranges across multiple batches; disabled external summary for now and use the legacy lane.
- Local observability patch that logs the guard's actual numbers and carries each rejection message into
attempts(happy to share if useful; overlaps the fix requested in [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885).
Related
- [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 — serial budget + non-fetch failures swallowed (same guard, same
executeUntil) - fix: all external summaries failing bypasses consecutive failure protection, repeated compress continues dispatching req | fix: 外部摘要全失败绕过连续失败保护,重复 compress 继续派发请求 #6903 — all-failed external batches bypass consecutive-failure protection in
applyConfiguredCompression - [Bug][preflight] the chunked retry after a summary failure splits on raw message indices — halves land on protected messages, every sub-chunk is rejected as "BLOCKED..mNNNNN", and the whole range is abandoned #2672 — preflight half-span chunks can land on protected messages (preflight-side sharding caveat)
- [Bug][compress] Tail-truncated tool-call
contentis misdiagnosed askind=content-not-array; no salvage path recognizes truncated JSON-array input | 尾部截断的 compress 入参被误诊为 content-not-array,现有 salvage 均不识别截断的 JSON 数组 #6946 — companion issue: truncated content misdiagnosed as content-not-array (same investigation).
Activity
ranxianglei commentedon Nov 16, 2026
🔶 需要决策 — kernel-change 批准:修复截断 compress 入参的误诊(kernel/src/parse-compress-input.ts)
content(字符串编码的 JSON 数组)尾部被截断时,被误诊为「格式错误」(content-not-array / no-valid-ranges),模型被指引去「修格式」而不是「重发」;且 salvage 不识别字符串元素,完整条目被丢。修复点全在 kernel 解析模块——host 侧不复制解析逻辑就解决不了(pi/omp 宿主共用此模块,acp-kernel#108 收敛的单一实现)。[开头 + JSON.parse 报 EOF 类错误」分类为kind=truncated(复用现有 kind,不新增);③ 字符串 content 设置diag.length,让既有 host receipt(src/stream.ts:1248,已含正确的「重发 string content + 行格式头」指引)自动生效。+ 回归测试。代价:一个 kernel PR,人工评审档。查重:不重复。 最近邻:#6019(truncated compression-draft turns——输出侧 prose envelope,不同层)、#6927(truncated summary outputs——输出校验侧)、#5080(注入参数混合格式——格式混淆根因,不同机制)。无 open PR 指向本线程。
验证 — 当前 master 复现(bili 0.1.215 / kernel 0.0.198),gap 仍然存在:
{content: "<对象数组,第一条目内截断>"}content-not-array(reason: "no mNNNNN/bN refs in header")truncated{content: "<对象数组,第二条目内截断>"}truncated,第一条目已抢救 ✓{content: "<字符串元素数组,第二元素内截断>"}no-valid-ranges("missing summary"),第一个完整字符串条目丢失truncated+ 抢救第一元素{content: "<字符串元素数组,第一元素内截断>"}no-valid-rangestruncatedcontent-not-arraytruncated要点:报告者的 0.1.213 已捆绑 kernel 0.0.186,其中对象条目 salvage 已存在(case B 可用)——所以其 transcript 里三次失败必然落在今天 master 仍坏的形状(第一条目内截断,或字符串元素数组)。issue 核心论断在 master 成立。
根因(两机制,同一失败域——入参解析分类):
salvageContentEntries(kernel/src/parse-compress-input.ts:984)只恢复 depth-1 的{...}对象条目;字符串元素(行格式字符串——模块注释 :609 自述「arrays of strings are the primary line-form transport」)从不被恢复 → case C 丢完整条目。方案评估: issue 三条提议均成立。细化设计(选项 A):
truncatedkind 而非新增truncated-input——finishSalvage(parse-compress-input.ts:426)已确立「截断输入 → 零条目也是 kind=truncated」语义;新增 kind 需要新 receipt 路径,无语义收益。[开头 且 JSON.parse 报 EOF 类错误("Unterminated string in JSON" / "Unexpected end of JSON input")。完整但转义损坏的文档(Unexpected token)与不以[开头的行格式文本保持现状——无回归面。diag.length修复:argCorruption receipt 仅在diagnostics.length有值时触发,而它目前只对裸字符串输入设置——{content: "..."}形状下模型拿到的是泛化 "no valid ranges" receipt + 误导性的数组包裹提示。设上 length 后,既有 receipt 已给出 issue 要求的指引,无需改 host 文案。八问(决策请求):
{content: "<JSON 数组字符串>"}发送且调用被上游截断(传输切断/输出上限)的车道;dsh/opencode 等插件车道均可能。严重程度 p3:边缘场景(需上游截断),有绕行(行格式重发,issue 已验证端到端可用),无数据丢失(截断调用什么都没压)。[开头 + EOF 错误」触发,转义损坏文档(Unexpected token)与非[行格式文本路径不变;salvage 扩展只作用于 tryParseLenient 已失败的分支,正常解析路径不触碰;kind 集合不变;host receipt 路径不变。B——同 A 减 salvage 扩展。C——无。选项对比:
标签:bug, dsh, scope/kernel, p3(边缘场景 + 有绕行 + 无数据丢失;报告车道为 DSH 插件车道,修复域在 kernel)。本环境镜像 shim 未实现 label API,标签行在此记录。
ranxianglei commentedon Nov 16, 2026
[bot] 🏷 更正:上一条评论(floor 3,「🔶 需要决策 — kernel-change 批准:修复截断 compress 入参的误诊」)是误发到本线程的——它属于姊妹 issue「Tail-truncated tool-call content is misdiagnosed as kind=content-not-array」(GitHub #6946,mirror #6947),不是本 issue(手动 compress 车道无分片/回退)。该决策请求已重新发布到正确线程(mirror #6947),那边为准。本条与其中内容均与本 issue 无关,请忽略/删除。
ranxianglei commentedon Nov 16, 2026
(ai: vllm-qwen/qwen3.8-27b)
🔶 需要决策
Background (one line): The manual compress lane has no size sharding and no legacy fallback — one oversized range exhausts every external-summary target and the refusal is a generic "exhausted" with no hint of which range to split. The no-decision diagnostics part (proposal 3) is already delivered in PR #6951; the two remaining proposals need your call.
Decision needed: Which of the two remaining proposals from this issue should be implemented?
Options:
pushSpanHalves,splitSummaryContent) in the manual lane — pre-split oversized ranges or retry with half-spans. Cost: a significant feature (changes the "one range = one summary" contract, adds recursive splitting or map-reduce logic, needs new tests). Benefit: fully closes the failure class — the user never sees the "exhausted" refusal for an oversized range.applyRanges(current-model summary). Cost: a new config surface field (owner-gated per Configuration Surface Discipline), a new code path, and a behavior change (the "never fall back" default stays, but the opt-in changes the contract). Benefit: a safety net — the user's compression still succeeds (with the main model) when the external chain fails.Recommendation: Option A (sharding) is the robust fix — it closes the failure class at the source. Option B (fallback) is a reasonable safety net but changes the contract. Option C (diagnostics only) is the minimum viable fix and is already delivered.
Default if no decision: PR #6951 ships as-is. Proposals 1 and 2 remain open in this issue.
Duplicate screening
查重: 不重复。 Verified against GitHub: #6885 (serial budget handling — distinct mechanism), #6903 (failure-streak protection — distinct mechanism), #2672 (preflight lane chunked retry — different lane, closed), #6946 (truncated content misdiagnosis — different symptom). No existing PR targets this issue.
Triage 八问
applyConfiguredCompression)把每个 range 1:1 映射为一个SummaryWork,无大小检查、无分片、无回退。HTTP 候选的 context-budget guard 在 fetch 前抛出 "input exceeds the target context budget",但错误消息被 executor 吞掉([Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 的领域),拒绝消息只说 "exhausted" 不说哪个 range 太大。externalSummary配置 + 手动 compress 工具的用户;触发条件是单个 range 的 input tokens 超过最小 target 的 input budget。严重程度:能用但有明显问题(需手动拆 range 或调大 budget)→ p2。tightestInputBudgetgetter,在全候选失败的拒绝消息中列出超大 range 的 ref、实测 tokens、建议拆分数。Proposals 1(分片)和 2(回退)需 owner 决策。① Delivered: diagnostics PR
PR #6951 (labels: bug, p2, external-summary) — the no-decision diagnostics part (proposal 3). The all-candidates-failed refusal now names the oversized range, its measured input tokens, and a suggested split count. The HTTP guard's behavior is unchanged (byte-identical strings, same budget formula). Regression test added.
ranxianglei commentedon Nov 16, 2026
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)
Status update — the full triage + decision request is in floor 4 above. Summary of what's done:
needs-decisionlabel applied to the upstream issue.contentis misdiagnosed askind=content-not-array; no salvage path recognizes truncated JSON-array input | 尾部截断的 compress 入参被误诊为 content-not-array,现有 salvage 均不识别截断的 JSON 数组 #6946).Waiting on: (a) CI green on PR #6951, (b) your decision on proposals 1/2.
fix: name the oversized range in the manual compress lane's refusal (#…