Skip to content

feat(compress): reject truncated summaries; optional proportional summary floor - #520

Closed
miaomiaozii wants to merge 2 commits into
ranxianglei:masterfrom
miaomiaozii:summary-integrity-guards
Closed

miaomiaozii wants to merge 2 commits into
ranxianglei:masterfrom
miaomiaozii:summary-integrity-guards

Conversation

@miaomiaozii

Copy link
Copy Markdown

Fixes ranxianglei/billion-context#6927.

Change

  • rejectTruncatedSummary (default true): detectTruncatedSummary() rejects a summary ending in … / ... or with an odd number of ``` fences. The error tells the model to finish the sentence/fence and resubmit.
  • minSummaryRatio (default 0, off): min summary chars per token of the range's direct messages. Checked after compressedTokens is computed and before consumed blocks are added, so tier-2/3 distillation is not penalized. Floor capped at maxSummaryLength / 2 (or summaryMaxChars / 2).

Both live in CompressValidationConfig as optional fields, so existing configs and adapters compile unchanged.

Tests

tests/summary-integrity.test.ts (7 tests): detector true/false cases, default rejection, opt-out, ratio reject/pass, ceiling cap, ratio off by default.

  • node --import tsx --test tests/summary-integrity.test.ts → 7 pass
  • npm test → 1238 pass / 2 fail. The 2 failures (applyRetrieve exports large originals to a file, default builders are byte-stable against checked-in fixtures) fail identically on unmodified master on this Windows machine (verified with git stash), so they are environment-related, not from this change.
  • tsc --noEmit clean.

miaomiaozii and others added 2 commits November 15, 2026 16:52
…mary floor

- rejectTruncatedSummary (default true): reject summaries ending in an
  ellipsis or with an unclosed ``` fence. A cut-off summary becomes the
  only record of the range and is silently lossy.
- minSummaryRatio (default 0 = off): min summary chars per token of the
  range's direct messages. Consumed blocks are excluded so tier-2/3
  distillation is unaffected; floor is capped at maxSummaryLength/2.
The PR was committed with two lines over the 80-col print width, so
format:check was red. Rewrap only; no logic change. Verified: 7/7
summary-integrity tests pass, tsc --noEmit clean, format:check green.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

PR review — #520 (reject truncated summaries; optional proportional summary floor)

Duplicate screening: 不重复. This PR addresses ranxianglei/billion-context#6927 (summary validation lets truncated/undersized summaries through → silent data loss). No other open PR does this work; #520 is the acp-kernel half of ranxianglei/billion-context#6927's two-repo proposal (the billion-context-pi config-forwarding half is separate).

What I verified live (on the PR head):

  • tests/summary-integrity.test.ts: 7/7 pass.
  • tsc --noEmit: clean.
  • The 2 failures you reported (applyRetrieve exports large originals to a file, default builders are byte-stable against checked-in fixtures) both pass on Linux — confirming they're Windows-environment artifacts, not caused by this change.
  • format:check was red (two lines over the 80-col print width). I fixed it and pushed commit fef29fe to the PR branch; it's green now. CI is re-running (4 checks queued) — I'll confirm once it lands.

Findings:

F1 — refold-in-place path bypasses the truncation guard (recommend closing before merge).
applyRefolds (src/compress.ts:315) writes a new summary onto existing blocks but only calls validateSummaryLength — not the new truncation check. So a truncated summary submitted via the refold path (#398/#400 in-place update) is still accepted, which defeats rejectTruncatedSummary. Fresh compression (applySingleRange) is guarded; refold is not. Suggested fix: pull the truncation check into a small shared helper and call it from both applySingleRange (via validateSummaryProportion) and applyRefolds; keep the minSummaryRatio floor fresh-compression-only (refold/distillation is meant to be short). I have push access and can apply this directly if you want.

F2 — behavior change (intended, flagged per policy). rejectTruncatedSummary defaults to true. Old: truncated summaries accepted. New: rejected by default. This is exactly ranxianglei/billion-context#6927's proposal, so it's intended — but it does change default behavior for every host (opt-out via rejectTruncatedSummary: false).

F3 — minSummaryRatio default-off (intended, informational). Per ranxianglei/billion-context#6927 it defaults to 0 (off), so the undersized-summary case (e.g. 117 chars for ~8K tokens) is only protected when a host opts in. Not a defect — just noting the 2nd half of ranxianglei/billion-context#6927 is opt-in by design.

F4 — companion work in another repo (informational). ranxianglei/billion-context#6927 also proposes forwarding these fields from acp.json in billion-context-pi (and notes minSummaryLength currently has no effect there). That's a separate repo/PR, not part of this one.

8-question checklist (中文):

  1. 是什么问题: Summary validation lets truncated or undersized summaries through (silent data loss) billion-context#6927 报告摘要校验只查非空/最小/最大长度,放过了两类静默数据丢失——结尾被截断的摘要、以及相对范围过小的摘要。本 PR 加两道校验。
  2. 什么原因导致的: 内核对摘要只有长度上下界,没有"是否截断"与"是否相对范围足够"的语义校验;截断/过小的摘要被静默接受后成为该范围唯一记录,细节丢失。
  3. 影响范围与严重程度: 所有走 applyCompression 的宿主。截断摘要=该范围细节永久丢失(数据丢失类,严重);过小摘要=信息密度不足(中)。默认只开截断校验,过小校验 opt-in。
  4. 怎么修复的: ① detectTruncatedSummary 启发式(结尾 …/... 或未闭合 ``` 围栏),默认拒绝、可 opt-out;② minSummaryRatio 按"直接消息 token 数"设字符下限,排除被消费块(不惩罚 tier-2/3 蒸馏),上限压到 `maxSummaryLength/2` 避免与上限冲突,默认 0(关)。
  5. 是否有回归问题: 默认行为变更——原接受的截断摘要现默认被拒(F2,属 Summary validation lets truncated or undersized summaries through (silent data loss) billion-context#6927 预期设计)。完整性缺口:refold-in-place 路径未接截断校验,截断摘要仍可经该路径写入(F1)。其余长度校验顺序不变,无意外回归。
  6. 是否破坏缓存: 不涉及。纯校验逻辑,不改 ref/块/游标/去重键。
  7. 是否改了设计方向: 不涉及架构转向;在既有 CompressValidationConfig 上加两个可选字段,适配层编译不变。
  8. 是否引入了新的配置: 是。CompressValidationConfig 新增可选字段 rejectTruncatedSummary?(默认 true)、minSummaryRatio?(默认 0),均可选,现有配置无需迁移。

Verdict: Faithful, well-tested implementation of ranxianglei/billion-context#6927's kernel proposal. Formatting fixed and pushed (CI re-running). One narrow completeness gap (F1: refold path) recommended to close before merge; F2/F3 are intended design. Happy to apply the F1 fix directly if you'd like.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Cross-thread finding from the ranxianglei/billion-context#6927 triage (not a full review — the #520 thread owns that): the refold path bypasses the new truncation check.

applyRefolds (src/compress.ts:316-336) calls only validateSummaryLength (empty/min/max) — it does NOT run the new rejectTruncatedSummary check. The refold early-return at src/compress.ts:1386-1398 fires before validateCompressionRange (line 1409) and before the new validateSummaryProportion call, so an in-place refold skips both.

Repro (confirmed locally on this PR head):

  1. compress m1..m1 with a good summary → block b1
  2. markBlockRestoredInline(b1) (src/decompress.ts:252 — full-log host precondition)
  3. re-compress m1..m1 with a summary ending in … → refold path → accepted (errors: [], blocksCreated: 1, truncated summary stored)

That is the exact silent data loss ranxianglei/billion-context#6927 fixes, reachable through refold. Suggested fix: run the truncation check in applyRefolds too (the ratio check doesn't apply there — a refold has no new direct tokens). The applyRefolds doc comment ("passes the same length checks as a fresh compression") should be updated to match once the check is added.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策 — pr-validation is red because the PR head branch name doesn't match the YYYY-MM-DD_short-title convention, and fixing it requires a branch rename on the author's fork, which I cannot do from here (pull-only access).

Verified live this run (head fef29fe):

  • Only red check is pr-validation: Branch name must match YYYY-MM-DD_short-title (got: summary-integrity-guards) — the check at scripts/ci/check-pr.sh:15.
  • test (22), test (24), build-artifact all pass on this head. Nothing in the code is broken; the only blocker is the branch name.
  • My account has pull-only permission on miaomiaozii/acp-kernel, so I cannot create/rename branches there (push → permission denied).
  • GitHub's API also silently refuses to repoint a PR head between the base repo and a fork (PATCH /pulls/520 with head → HTTP 200, head unchanged), so I can't repoint the PR to a branch in the main repo either.

What I did:

  • Created 2026-11-15_summary-integrity-guards at fef29fe in ranxianglei/acp-kernel — a correctly-named fallback branch pointing at the exact PR head commit (no code change). It's there if option C is chosen.

Options:

  • A (recommended): author renames the branch on their fork. @miaomiaozii — from your fork:
    1. git push origin HEAD:refs/heads/2026-11-15_summary-integrity-guards
    2. On the PR page, change the head branch to 2026-11-15_summary-integrity-guards (head-ref dropdown → "Change head branch"; the API equivalent PATCH /pulls/520 {"head":"2026-11-15_summary-integrity-guards"} works because it stays inside the same fork repo)
    3. Optionally git push origin --delete summary-integrity-guards
      CI re-runs on the new head ref and pr-validation goes green.
  • B: owner grants ranxianglei write access to miaomiaozii/acp-kernel (fork → Settings → Collaborators → Add). The CI lane then performs steps 1–3 itself on its next wake.
  • C: move the PR into the main repo — close feat(compress): reject truncated summaries; optional proportional summary floor #520 and open a fresh PR from 2026-11-15_summary-integrity-guards (already created in ranxianglei/acp-kernel at fef29fe). Heaviest option: loses this PR number/thread and its review comments.

Default if undecided: A — it's the author's fork, it's a two-step git + UI fix, and no design decision is involved; only the rename is missing.

8问 (CI 修复轮):

  1. 是什么问题: pr-validation 检查 PR 头分支名是否符合 YYYY-MM-DD_short-title 规范;本 PR 分支名 summary-integrity-guards 缺日期前缀,校验失败。
  2. 什么原因导致的: 分支在作者 fork(miaomiaozii/acp-kernel)里创建时未带日期前缀;CI 脚本 scripts/ci/check-pr.sh:15 按规范拒绝。
  3. 影响范围与严重程度: 仅阻塞本 PR 合并(其余 3 项检查全绿);不影响任何用户或运行时行为。
  4. 怎么修复的: 修法本身是机械的——把分支改名为 2026-11-15_summary-integrity-guards(同一 commit,零代码改动)。我已无法代做:对作者 fork 只有 pull 权限,且 GitHub API 不允许把 PR 头从 fork 挪到主仓。已在主仓预建同名分支 2026-11-15_summary-integrity-guards @ fef29fe 作为现成后备。
  5. 是否有回归问题: 无。本轮未改动任何代码;只新增了一个指向 PR 头 commit 的普通分支引用。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。

判定: 本轮无法自行收工——分支改名需要作者或 owner 侧动作(选项 A/B/C,推荐 A)。已打 needs-decision 标签;选 A 或 B 后,下一次 CI 唤醒我会自动复核并收尾。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Re-verified live this wake — no change since my previous comment:

  • pr-validation still red on the same run (branch name summary-integrity-guards fails scripts/ci/check-pr.sh:15); test (22), test (24), build-artifact still green on head fef29fe.
  • Fork miaomiaozii/acp-kernel still pull-only for my account → I still cannot rename the branch, and the API still won't repoint the PR head across fork/base.

Still waiting on A/B/C from the previous comment (recommended A: author renames the branch on their fork; B: owner grants ranxianglei write access to the fork — the moment B lands I complete the rename on the next wake). needs-decision label already applied.

8问 (本轮复核): 本轮零代码改动——①问题/②原因/③影响/④修法 同上条「🔶 需要决策」评论(分支名不合规,需作者 fork 侧改名);⑤回归:无;⑥缓存:不涉及;⑦设计方向:不涉及;⑧新配置:不涉及。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] auto-fix stalled: pr-validation — stopping for human review

@miaomiaozii

Copy link
Copy Markdown
Author

Superseded by #521: same commits on a correctly named branch (2026-11-15_summary-integrity-guards) plus a fix for review finding F1 (truncation check now also runs on the refold path, with tests). Thanks for the review and the prettier fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Summary validation lets truncated or undersized summaries through (silent data loss)

2 participants