Skip to content

feat(compress): reject truncated summaries; optional proportional summary floor - #521

Open
miaomiaozii wants to merge 4 commits into
ranxianglei:masterfrom
miaomiaozii:2026-11-15_summary-integrity-guards
Open

miaomiaozii wants to merge 4 commits into
ranxianglei:masterfrom
miaomiaozii:2026-11-15_summary-integrity-guards

Conversation

@miaomiaozii

Copy link
Copy Markdown

Supersedes #520, reopened only to satisfy the YYYY-MM-DD_short-title branch-name check (GitHub can't change a PR's head branch).

Same commits as #520 (including the maintainer's prettier fix fef29fe), plus one new commit:

  • 79f9491 fix(compress): run the truncation check on the refold path too. Addresses review finding F1: applyRefolds only called validateSummaryLength, so a truncated summary could still land via decompress-inline → refold. Extracted validateSummaryNotTruncated and call it from both paths; the proportional floor stays fresh-compression-only. Two new tests in tests/refold.test.ts (reject … / unclosed fence on refold and leave the block untouched; opt-out still works). The reject test fails without the fix.

Local: tsc --noEmit clean; refold + summary-integrity tests 29/29; prettier clean.


Fixes ranxianglei/billion-context#6927.

Change

  • rejectTruncatedSummary (default true): detectTruncatedSummary() rejects a summary ending in … / ... or with an odd number of ``` fences. The error tells the model to finish the sentence/fence and resubmit.
  • minSummaryRatio (default 0, off): min summary chars per token of the range's direct messages. Checked after compressedTokens is computed and before consumed blocks are added, so tier-2/3 distillation is not penalized. Floor capped at maxSummaryLength / 2 (or summaryMaxChars / 2).

Both live in CompressValidationConfig as optional fields, so existing configs and adapters compile unchanged.

Tests

tests/summary-integrity.test.ts (7 tests): detector true/false cases, default rejection, opt-out, ratio reject/pass, ceiling cap, ratio off by default.

  • node --import tsx --test tests/summary-integrity.test.ts → 7 pass
  • npm test → 1238 pass / 2 fail. The 2 failures (applyRetrieve exports large originals to a file, default builders are byte-stable against checked-in fixtures) fail identically on unmodified master on this Windows machine (verified with git stash), so they are environment-related, not from this change.
  • tsc --noEmit clean.

miaomiaozii and others added 3 commits November 15, 2026 16:52
…mary floor

- rejectTruncatedSummary (default true): reject summaries ending in an
  ellipsis or with an unclosed ``` fence. A cut-off summary becomes the
  only record of the range and is silently lossy.
- minSummaryRatio (default 0 = off): min summary chars per token of the
  range's direct messages. Consumed blocks are excluded so tier-2/3
  distillation is unaffected; floor is capped at maxSummaryLength/2.
The PR was committed with two lines over the 80-col print width, so
format:check was red. Rewrap only; no logic change. Verified: 7/7
summary-integrity tests pass, tsc --noEmit clean, format:check green.
applyRefolds only called validateSummaryLength, so a truncated summary
could still replace a block's summary via decompress-inline -> refold.
Extract validateSummaryNotTruncated and call it from both fresh
compression and refold. The proportional floor stays fresh-only: a
refold has no new direct messages.

Addresses review finding F1 on ranxianglei#520.
- Add a regression test: the minSummaryRatio floor must be computed from
  the range's direct messages only. A mixed range (5 consumed-block
  messages + 5 new direct messages) with a 300-char summary passes the
  direct-only floor (250) and would be rejected if the consumed block's
  summary tokens were counted (450). Verified the test fails when the
  validateSummaryProportion call is moved after the consumed-block loop.
- Drop the dead 'merge' key from the test config helper: Config has no
  merge field (hidden by the 'as Config' cast, read by nothing).
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Duplicate check: no duplicate. Related: closed predecessor PR #520 (same author, superseded by this PR for the branch-name check). No other open PR or issue targets the same mechanism (summary truncation / undersized validation).

Review verdict: mergeable, with two test-quality fixes I pushed to the PR branch as commit 17a9c19 (maintainer edit, maintainer_can_modify was on).

What I verified

  • All summary-entry paths are covered. The only block-creation site is allocateBlockId at src/compress.ts:1434 (inside applySingleRange), and every refold goes through applyRefolds (src/compress.ts:317, called from the main loop at :611 and from inside applySingleRange at :1389). The F1 fix correctly shares validateSummaryNotTruncated via applyRefolds, so both refold entry points and the fresh-compression path (validateSummaryProportion at :1422) are covered.
  • No state corruption on reject. Validation throws before any mutation; the refold test pins that a rejected refold leaves the block's summary and restoredInline flag untouched.
  • Proportional floor is direct-only by construction. The call sits before consumed-block tokens are added (:1427–:1432), so tier-2/3 distillation is not penalized. The ceiling/2 cap makes a floor/ceiling conflict impossible, and summaryMaxChars ?? maxSummaryLength matches the existing nullish semantics in validateSummaryLength (explicit 0 = unlimited).
  • Additive config. Both fields are optional on CompressValidationConfig; defaultConfig/validateConfig need no change and existing configs/adapters compile unchanged.
  • The 2 reported test failures don't reproduce here. applyRetrieve exports large originals to a file and default builders are byte-stable against checked-in fixtures both pass on this Linux machine (71/71 in those files) and in CI — they are Windows-environment-specific, not caused by this change.

Fixes I pushed (17a9c19, test-only)

  1. Added a regression test pinning the direct-only floor — the PR's core design claim ("consumed blocks are excluded") had no test: if the validateSummaryProportion call were moved after the consumed-block loop, all existing tests still pass (they use direct-only ranges) while tier distillation would start getting rejected. New test: mixed range (5 consumed-block messages + 5 direct × 100 chars, ratio 0.5, 300-char summary) passes the direct-only floor (250) and would be rejected if the consumed block's summary tokens were counted (450). I verified the test fails against the mutated call order — it has teeth.
  2. Dropped a dead merge key from the test config helper in tests/summary-integrity.test.ts — Config has no merge field (hidden by the as Config cast, read by nothing in src/).

CI on 17a9c19: green — test (22), test (24), pr-validation, build-artifact all success. Locally: tsc --noEmit clean, 61/61 across summary-integrity + refold + compress, prettier stable.

结构化八问

  1. 是什么问题:内核的 summary 校验只查非空/≥50 字符/≤上限,截断的 summary(句子中途被切断)和对范围来说过短的 summary 都能静默通过;而 summary 是该范围唯一记录,细节就此丢失。
  2. 什么原因导致的:校验层缺少语义完整性检查——完全没有截断检测,且下限是绝对值(50 字符),不随范围大小缩放。
  3. 影响范围与严重程度:波及所有压缩路径(新压缩、refold、tier 蒸馏);触发条件是模型产出截断/过短 summary(issue 实际案例:200 字符 summary 替代了三次代码评审的细节;~8K token 范围只给了 117 字符)。属静默数据丢失,严重度高,但并非每个会话都触发。
  4. 怎么修复的:加两道校验——截断启发式(尾部 …/... 或奇数个 ``` fence,默认开、可关)+ 比例下限(按范围直接消息 token 数 × 比率,默认关,上限封顶于 maxSummaryLength/2 保证与上限永不冲突)。截断检查新压缩与 refold 两条路径共用;比例下限只作用于新压缩(refold 无新直接消息)。
  5. 是否有回归问题:审过触碰路径的既有行为——合法的新压缩与 refold 不受影响;拒绝发生在任何状态变更之前(block 保持原样,有测试钉住);无 fail-fast 变吞错(新检查与既有校验一样 throw);无 wire 格式变化(仅 result.errors 新增错误文案);缓存/重试/增量游标不受影响。唯一行为变化即下述新默认拒绝。
  6. 是否破坏缓存:不涉及——纯校验逻辑,不触碰缓存/重试/游标/去重键。
  7. 是否改了设计方向:不涉及——无架构/边界/数据流变化,仅在校验配置上加两个可选字段(纯增量)。
  8. 是否引入了新的配置:两个新可选字段——compress.rejectTruncatedSummary(默认 true,设 false 恢复旧行为)与 compress.minSummaryRatio(默认 0 = 关;如 0.02 → 20K token 范围需 ≥400 字符,且下限封顶于 maxSummaryLength 的一半)。既有配置编译不变。

Behavior change disclosure (old → new)

Merge judgment

Mergeable. Direction hits the root cause (kernel-level validation, not a host-side band-aid), code is clean, tests are real (including the two I added), CI green. Note: this is a feat: with a new default-on behavior — a product decision — so per the repo's auto-merge gate (§7.4.5) it stays human-merge. The issue's third item (billion-context-pi forwarding these settings from acp.json) is a separate repo/PR and does not block this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Summary validation lets truncated or undersized summaries through (silent data loss)

2 participants