Skip to content

[Bug][compress] Manual compress lane has no size sharding/halving and no legacy fallback — one oversized range exhausts every external-summary target | 手动 compress 车道无分片/折半、无 legacy 回退:单个超大 range 耗尽全部外部摘要候选 #6945

Open

Description

@passingby000

中文摘要:externalSummary 启用时,手动 compress 工具对模型提名的每个 range 一对一组装摘要任务,没有任何尺寸上限、折半或分片;一个超大 range 会让全部候选在 fetch 前被 context-budget 守卫拒绝(最大窗口候选则超时),整批 exhausted 且不回退到经典的当前模型摘要路径。自动 preflight 车道已有完整的分片纪律(CHUNK_FRACTION=.6、pushSpanHalves、6 万字符 T2 预算等),两车道防护不对称。实测案例:39 个嵌套块、重建 work.content ≈ 521,295 token,五链窗口 131072/131072/262144/1048576/272000,1.4–1.7 秒全灭、四链零出站;同会话 18 块小 range 成功形成对照。建议:手动车道复用 preflight 分片原语、全失败时提供 opt-in legacy 回退、拒绝信息给出实测 token 与建议分片数。

Summary

When compress.externalSummary is enabled, the manual compress tool lane (applyConfiguredCompression) builds one summary work item 1:1 per model-nominated range with no size cap, no span-halving and no sharding. A single oversized range therefore fails against every configured target — most via the pre-fetch context-budget guard, the largest-window target via timeout — and the whole batch is refused with reason=exhausted, with no fallback to the legacy in-model summary path. The automatic preflight lane already has a complete size-discipline machinery; the two lanes are asymmetric.

Environment

  • billion-context 0.1.213 (commit 133f3525c), DSH plugin mode, Windows 11, bundle dist/index.js
  • 5 external-summary targets (windows): 131072 / 131072 / 262144 / 1048576 / 272000 tokens
  • A manually nominated batch of 39 nested blocks; faithful offline rebuild of the assembled work.content (direct messages + nested block summaries, same defaultCountTokens logic) measured ≈ 521,295 input tokens

Symptom

The batch finished in ~1.4–1.7 s with:

reason=exhausted attempts=[t0:error,t1:error,t2:error,t3:error,t4:error]
  • Zero outbound HTTP requests in the proxy log for four of the five targets — they threw before fetch inside summarize():
if (plan.contextWindow !== void 0 &&
    defaultCountTokens(system) + defaultCountTokens(content) + 256 + plan.outputTokens > plan.contextWindow)
  throw new Error("External summary input exceeds the target context budget");
  • The 1,048,576-window target passed the guard but then hit target-timeout at the old 120 s budget (a ~521K-token summary cannot finish in 120 s).
  • A smaller 18-block range from the same session succeeded on the 1M-window target in the same time window — control evidence that credentials, network and endpoints were healthy; size alone explains the failure.

Root cause: protection asymmetry between the two lanes

Preflight (automatic) lane has full size discipline (bundle offsets in 0.1.213):

  • CHUNK_FRACTION = 0.6 — one chunk ≤ 60 % of the main model window
  • pushSpanHalves — HTTP 400/413 or empty summary triggers half-span retry (floor 4K tokens)
  • TIER_DISTILL_SOURCE_CHAR_BUDGET = 60000 — 60K-char cap per T2 distillation source
  • MAX_SUMMARY_CALLS_PER_PREFLIGHT = 16, MIN_CHUNK_TOKENS = 2000
  • A ready-made splitSummaryContent primitive also exists in the bundle

Manual lane (applyConfiguredCompression, ~@3740918):

  • For each nominated range it assembles work.content = collectBlockContent(full:false).join(...) 1:1
  • A bundle-wide search for shard / splitRange / sliceRange returns zero matches — no size cap, no halving, no sharding
  • When all targets fail, the call returns refused ("do not retry unchanged configuration") and never falls back to the legacy applyRanges path (the current-model summary that works with external summary disabled)

The documented "100K context is enough" guarantee holds for the nudge-driven normal flow (ranges stay small), but the manual tool gives the model no guardrail against nominating a huge multi-block range, and the failure mode is an opaque whole-batch refusal.

Reproduction

  1. Enable external summary with ≥1 target whose contextWindow is smaller than the nominated range.
  2. In a long session, call compress with one range spanning enough nested blocks to assemble > contextWindow tokens (our case: 39 blocks, ~521K tokens).
  3. Observe sub-2-second exhausted with zero outbound requests for the small-window targets and no partial success stored.

Proposals

Any one of these would close the failure class; 1+2 is the robust combination:

  1. Reuse the preflight primitives in the manual lane: when a range would fail the smallest target's budget guard, pre-split it (or half-span retry on guard rejection) using the existing chunking code instead of refusing the batch.
  2. Legacy fallback opt-in: when the whole external chain fails, allow falling back to applyRanges (current-model summary) behind a config flag — the current "never fall back" semantics (documented) can stay the default.
  3. Minimum viable diagnostics: pre-validate each nominated range against the smallest configured contextWindow and name the offending range, its measured tokens and a suggested split count in the refusal, instead of a generic exhausted (see also [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 Problem 3 on misleading failure text, and fix: all external summaries failing bypasses consecutive failure protection, repeated compress continues dispatching req | fix: 外部摘要全失败绕过连续失败保护,重复 compress 继续派发请求 #6903 on failure-streak bypass in this same function).

Workarounds applied locally

Related

Activity

  1. ranxianglei commented on Nov 16, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    🔶 需要决策 — kernel-change 批准:修复截断 compress 入参的误诊(kernel/src/parse-compress-input.ts)

    • 背景一句话:compress 入参 content(字符串编码的 JSON 数组)尾部被截断时,被误诊为「格式错误」(content-not-array / no-valid-ranges),模型被指引去「修格式」而不是「重发」;且 salvage 不识别字符串元素,完整条目被丢。修复点全在 kernel 解析模块——host 侧不复制解析逻辑就解决不了(pi/omp 宿主共用此模块,acp-kernel#108 收敛的单一实现)。
    • 待决问题:Kernel Boundary 规则 1——kernel 变更需 owner 批准后才可实施。本项不属于 mechanical-bug 豁免(新分类语义 + salvage 状态机扩展,非错字/假守卫)。
    • 选项(代价见后):
      • A(推荐):完整三段 kernel PR——① salvage 状态机扩展恢复 depth-0 完整字符串元素;②「content 以 [ 开头 + JSON.parse 报 EOF 类错误」分类为 kind=truncated(复用现有 kind,不新增);③ 字符串 content 设置 diag.length,让既有 host receipt(src/stream.ts:1248,已含正确的「重发 string content + 行格式头」指引)自动生效。+ 回归测试。代价:一个 kernel PR,人工评审档。
      • B:仅诊断最小(②+③,不做字符串 salvage)——更便宜,但 case C(字符串元素)的完整条目仍丢失。
      • C:hold——不实施,issue 保留为已验证的 triage 记录。
    • 你的推荐:A。
    • 不决定的默认行为:不实施(kernel 门禁),issue 保持 open。

    查重:不重复。 最近邻:#6019(truncated compression-draft turns——输出侧 prose envelope,不同层)、#6927(truncated summary outputs——输出校验侧)、#5080(注入参数混合格式——格式混淆根因,不同机制)。无 open PR 指向本线程。

    验证 — 当前 master 复现(bili 0.1.215 / kernel 0.0.198),gap 仍然存在:

    case 输入形状 当前 kind 应为
    A {content: "<对象数组,第一条目内截断>"} content-not-array(reason: "no mNNNNN/bN refs in header") truncated
    B(对照) {content: "<对象数组,第二条目内截断>"} truncated,第一条目已抢救 ✓ —
    C {content: "<字符串元素数组,第二元素内截断>"} no-valid-ranges("missing summary"),第一个完整字符串条目丢失 truncated + 抢救第一元素
    D {content: "<字符串元素数组,第一元素内截断>"} no-valid-ranges truncated
    E issue 签名:Unterminated string at position N == content 长度 content-not-array truncated

    要点:报告者的 0.1.213 已捆绑 kernel 0.0.186,其中对象条目 salvage 已存在(case B 可用)——所以其 transcript 里三次失败必然落在今天 master 仍坏的形状(第一条目内截断,或字符串元素数组)。issue 核心论断在 master 成立。

    根因(两机制,同一失败域——入参解析分类):

    1. salvageContentEntries(kernel/src/parse-compress-input.ts:984)只恢复 depth-1 的 {...} 对象条目;字符串元素(行格式字符串——模块注释 :609 自述「arrays of strings are the primary line-form transport」)从不被恢复 → case C 丢完整条目。
    2. 字符串 content 回退分支(parse-compress-input.ts:197–253)把一切不可解析的 content 字符串当行格式文本读;截断的 JSON 数组前缀(干净前缀、EOF 错误)被误分类为 content-not-array(对象条目)/ no-valid-ranges(字符串条目),而非截断。

    方案评估: issue 三条提议均成立。细化设计(选项 A):

    • 复用现有 truncated kind 而非新增 truncated-input——finishSalvage(parse-compress-input.ts:426)已确立「截断输入 → 零条目也是 kind=truncated」语义;新增 kind 需要新 receipt 路径,无语义收益。
    • salvage 扩展:depth-0 完整字符串元素(尾部半截字符串丢弃,永不猜测)。
    • 门控必须精确:content(去 fence 后)以 [ 开头 且 JSON.parse 报 EOF 类错误("Unterminated string in JSON" / "Unexpected end of JSON input")。完整但转义损坏的文档(Unexpected token)与不以 [ 开头的行格式文本保持现状——无回归面。
    • diag.length 修复:argCorruption receipt 仅在 diagnostics.length 有值时触发,而它目前只对裸字符串输入设置——{content: "..."} 形状下模型拿到的是泛化 "no valid ranges" receipt + 误导性的数组包裹提示。设上 length 后,既有 receipt 已给出 issue 要求的指引,无需改 host 文案。

    八问(决策请求):

    1. 是什么问题:截断的 compress 入参被误诊为格式错误,模型按错误指引重试(修格式而非重发),每次浪费 1–3 个 round-trip;字符串元素形状还丢完整条目。
    2. 什么原因导致的:入参解析分类的两个机制缺陷——salvage 状态机看不见字符串元素;字符串 content 回退分支不识别「截断的 JSON 数组前缀」这一形状,一律当行格式文本误分类。
    3. 影响范围与严重程度(按选项):A/B/C 受影响面相同——任何以 {content: "<JSON 数组字符串>"} 发送且调用被上游截断(传输切断/输出上限)的车道;dsh/opencode 等插件车道均可能。严重程度 p3:边缘场景(需上游截断),有绕行(行格式重发,issue 已验证端到端可用),无数据丢失(截断调用什么都没压)。
    4. 怎么修复的(推荐 A):见上「细化设计」——复用 truncated kind + salvage 扩展 + EOF 门控 + diag.length,host 零改动。
    5. 回归风险(按选项):A——门控只在「[ 开头 + EOF 错误」触发,转义损坏文档(Unexpected token)与非 [ 行格式文本路径不变;salvage 扩展只作用于 tryParseLenient 已失败的分支,正常解析路径不触碰;kind 集合不变;host receipt 路径不变。B——同 A 减 salvage 扩展。C——无。
    6. 是否破坏缓存:不涉及(纯解析分类,不触缓存/去重/游标)。
    7. 是否改了设计方向:不涉及(kernel 解析模块职责边界不变,补的是 salvage 语义的缺口)。
    8. 是否引入了新的配置:无。

    选项对比:

    改动面 修复 case 残留 代价
    A(推荐) kernel 3 处 + 测试 A/B/C/D/E 全 无 1 个 kernel PR(人工评审档)
    B kernel 2 处 + 测试 A/D/E(诊断正确) C 仍丢完整字符串条目 略省
    C 无 无 全部 0

    标签:bug, dsh, scope/kernel, p3(边缘场景 + 有绕行 + 无数据丢失;报告车道为 DSH 插件车道,修复域在 kernel)。本环境镜像 shim 未实现 label API,标签行在此记录。

  2. ranxianglei commented on Nov 16, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 更正:上一条评论(floor 3,「🔶 需要决策 — kernel-change 批准:修复截断 compress 入参的误诊」)是误发到本线程的——它属于姊妹 issue「Tail-truncated tool-call content is misdiagnosed as kind=content-not-array」(GitHub #6946,mirror #6947),不是本 issue(手动 compress 车道无分片/回退)。该决策请求已重新发布到正确线程(mirror #6947),那边为准。本条与其中内容均与本 issue 无关,请忽略/删除。

  3. ranxianglei commented on Nov 16, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    🔶 需要决策

    Background (one line): The manual compress lane has no size sharding and no legacy fallback — one oversized range exhausts every external-summary target and the refusal is a generic "exhausted" with no hint of which range to split. The no-decision diagnostics part (proposal 3) is already delivered in PR #6951; the two remaining proposals need your call.

    Decision needed: Which of the two remaining proposals from this issue should be implemented?

    Options:

    • A. Size sharding / half-span retry (proposal 1): Reuse the preflight sharding primitives (pushSpanHalves, splitSummaryContent) in the manual lane — pre-split oversized ranges or retry with half-spans. Cost: a significant feature (changes the "one range = one summary" contract, adds recursive splitting or map-reduce logic, needs new tests). Benefit: fully closes the failure class — the user never sees the "exhausted" refusal for an oversized range.
    • B. Legacy fallback opt-in config flag (proposal 2): Add a config flag that, when the whole external chain fails, falls back to applyRanges (current-model summary). Cost: a new config surface field (owner-gated per Configuration Surface Discipline), a new code path, and a behavior change (the "never fall back" default stays, but the opt-in changes the contract). Benefit: a safety net — the user's compression still succeeds (with the main model) when the external chain fails.
    • C. Neither (diagnostics only): PR [PR] fix: name the oversized range in the manual compress lane's refusal (#6945) #6951 ships as-is. The user splits the range manually based on the diagnostic. Cost: the user still has to do the splitting manually. Benefit: no new feature surface, no config change.

    Recommendation: Option A (sharding) is the robust fix — it closes the failure class at the source. Option B (fallback) is a reasonable safety net but changes the contract. Option C (diagnostics only) is the minimum viable fix and is already delivered.

    Default if no decision: PR #6951 ships as-is. Proposals 1 and 2 remain open in this issue.


    Duplicate screening

    查重: 不重复。 Verified against GitHub: #6885 (serial budget handling — distinct mechanism), #6903 (failure-streak protection — distinct mechanism), #2672 (preflight lane chunked retry — different lane, closed), #6946 (truncated content misdiagnosis — different symptom). No existing PR targets this issue.

    Triage 八问

    1. 是什么问题: 手动 compress 车道无分片/折半、无 legacy 回退——单个超大 range 耗尽全部外部摘要候选,拒绝消息是泛化的 "exhausted",用户不知道哪个 range 太大、该怎么拆。
    2. 什么原因导致的: 手动车道(applyConfiguredCompression)把每个 range 1:1 映射为一个 SummaryWork,无大小检查、无分片、无回退。HTTP 候选的 context-budget guard 在 fetch 前抛出 "input exceeds the target context budget",但错误消息被 executor 吞掉([Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885 的领域),拒绝消息只说 "exhausted" 不说哪个 range 太大。
    3. 影响范围与严重程度: 影响所有使用 externalSummary 配置 + 手动 compress 工具的用户;触发条件是单个 range 的 input tokens 超过最小 target 的 input budget。严重程度:能用但有明显问题(需手动拆 range 或调大 budget)→ p2。
    4. 怎么修复的: 本 PR([PR] fix: name the oversized range in the manual compress lane's refusal (#6945) #6951)实现 proposal 3(诊断):提取共享的 system prompt / content payload / token estimate helper,添加 tightestInputBudget getter,在全候选失败的拒绝消息中列出超大 range 的 ref、实测 tokens、建议拆分数。Proposals 1(分片)和 2(回退)需 owner 决策。
    5. 是否有回归问题: HTTP guard 行为不变(byte-identical 字符串、相同 budget 公式);诊断是附加的,只在全候选失败且 range 超预算时出现。无回归。
    6. 是否破坏缓存: 不涉及缓存。
    7. 是否改了设计方向: 不涉及架构变更;诊断是纯附加。
    8. 是否引入了新的配置: 本 PR 无新配置。Proposal 2(回退 flag)若实施需 owner 批准新配置字段。

    ① Delivered: diagnostics PR

    PR #6951 (labels: bug, p2, external-summary) — the no-decision diagnostics part (proposal 3). The all-candidates-failed refusal now names the oversized range, its measured input tokens, and a suggested split count. The HTTP guard's behavior is unchanged (byte-identical strings, same budget formula). Regression test added.

  4. ranxianglei commented on Nov 16, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

    Status update — the full triage + decision request is in floor 4 above. Summary of what's done:

    1. Diagnostics PR delivered: PR [PR] fix: name the oversized range in the manual compress lane's refusal (#6945) #6951 (labels: bug, p2, external-summary) — the no-decision part (proposal 3). The all-candidates-failed refusal now names the oversized range, its measured input tokens, and a suggested split count. HTTP guard behavior unchanged. Regression test added. Awaiting CI.
    2. Decision request posted (floor 4): proposals 1 (sharding) and 2 (fallback) need your call. needs-decision label applied to the upstream issue.
    3. Duplicate screening: 查重: 不重复 (verified against [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures a | [Bug][compress][externalSummary] Serial 25s/50s default budget makes long-payload chains fail 100%; non-fetch failures are swallowe… #6885, fix: all external summaries failing bypasses consecutive failure protection, repeated compress continues dispatching req | fix: 外部摘要全失败绕过连续失败保护,重复 compress 继续派发请求 #6903, [Bug][preflight] the chunked retry after a summary failure splits on raw message indices — halves land on protected messages, every sub-chunk is rejected as "BLOCKED..mNNNNN", and the whole range is abandoned #2672, [Bug][compress] Tail-truncated tool-call content is misdiagnosed as kind=content-not-array; no salvage path recognizes truncated JSON-array input | 尾部截断的 compress 入参被误诊为 content-not-array,现有 salvage 均不识别截断的 JSON 数组 #6946).

    Waiting on: (a) CI green on PR #6951, (b) your decision on proposals 1/2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions