Skip to content

fix: surface parse-dropped compress entries; normalize single-object content (#1494) - #1497

Merged
ranxianglei merged 2 commits into
masterfrom
2026-09-27_compress-param-drift-fix
Sep 27, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-09-27_compress-param-drift-fix

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Root cause

Issue #1494 reported two things in the same session:

  1. Silent partial failure. A batched compress call with 3 entries applied only 2 — "Compressed m02815–m03212, m04083–m04092 → 2 block(s)" — with no note about the dropped entry. The kernel parser already reports per-entry rejections in diagnostics.invalidReasons, but applyRanges only surfaced them when all entries failed (ranges.length === 0). When ≥1 range survived, diagnostics.ok === true, so neither the model-facing receipt nor the server log mentioned the drop — the receipt's detail lists only surviving ranges, so a 3-entry call read as a clean 2-block success.
  2. Asymmetric shape tolerance on retry. Retrying the lost range with content given as the single entry object hit kind=content-not-array, dropped=0 — a hard failure. The kernel's salvage ladder already accepts a bare {startId,…} at the TOP level as one range, but hard-rejects the same shape INSIDE content. Bare top-level arrays (kind=not-object) and string-encoded variants fall into the same wall.

The summary-length hypothesis from the issue does not hold: the per-summary cap is 20000 chars (#888) and a cap breach fails the WHOLE call atomically with "Summary too long" — it cannot drop one entry of three. The drop happened at argument parse time (e.g. an unescaped quote/newline inside the long summary corrupting the array, salvaged entries minus the broken fragment).

Changes

  • src/compress-tool.ts: new normalizeCompressInput() runs before the kernel parse — wraps single-object content in a one-element array, bare top-level entry arrays into {content:[…]}, and recovers the same shapes from string-encoded args (fenced / double-stringified). Canonical inputs pass through byte-identical (asserted).
  • src/stream.ts applyRanges: appends a model-facing REJECTED note carrying the kernel's index-prefixed reasons to BOTH the success receipt and the 0-blocks failure message; logs [acp-proxy: compress PARTIAL …] server-side.
  • src/compress-tool.ts parseCompressInput: warn-logs partial drops (previously invisible because ok was true whenever ≥1 range survived).
  • tests/issue1494-compress-params.test.ts: 8 regression tests — both defects reproduced pre-fix (silent partial success; content-not-array on single-object content), plus pass-through guarantees for canonical/empty/line-form inputs.

Behavior changes (disclosed)

old new
content = single entry object / bare top-level array / string-encoded variants full call failure (content-not-array / not-object) normalized and folded
partial batch (some entries invalid) silent — receipt lists survivors only, no log explicit REJECTED note with reasons + re-issue guidance, server log line
garbage object content ({foo:1}) opaque kind=content-not-array kind=no-valid-ranges with per-entry reason (still a failure)

Proxy mode only: plugin-mode agents own their folds and never route through applyRanges, so plugin mode is untouched. No wire/upstream protocol change — tool-result text gains an optional extra line.

Pre-flight

  • npm run typecheck ✅
  • npm test ✅ 2879 pass / 0 fail / 12 skipped (pre-existing skips)
  • npm run build ✅
  • e2e: not runnable locally here (no reachable upstream); this diff touches src/stream.ts (request-pipeline hot file), so ci-e2e.yml auto-runs on this PR.

Note: #1495 reports the exact same transcript/incident (lean-mode framing) — this fix is mode-independent (proxy-layer param handling); one PR covers both.

Closes #1494

Fixes #1494
Fixes #1495

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-27_compress-param-drift-fix (11c3a16)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1438-pr.1497.3fffc4a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1497.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

收到,开始审核 PR #1497(fix #1494)。流程:核对原 issue 方向 → 回归审查 → 代码审查 → 测试有效性 → diff 整洁度,本地跑 typecheck/test/build,并确认 CI(含 e2e)状态。结论稍后在此汇报。

…content (#1494)

Two param-shape defects in the proxy compress path:

1. Silent partial failure: entries a batched compress call contained that
   the kernel parser rejected (bad bounds, missing summary, broken JSON
   fragment) vanished from the receipt — 'Compressed A, B → 2 block(s)'
   with no note about the dropped entry and no server log line (the warn
   only fired when ALL entries failed). Models believed the fold landed
   and never re-issued the lost range.

2. Asymmetric shape tolerance: retrying the lost range with content given
   as the single entry object (or a bare top-level array / string-encoded
   variants) hit kernel kind=content-not-array/not-object — a hard failure
   on shapes the kernel's salvage ladder already accepts at the top level.

Fixes:
- normalizeCompressInput (src/compress-tool.ts): wrap single-object content
  in a one-element array, bare top-level arrays into {content:[…]}, and
  recover the same shapes from string-encoded args (fenced / double-
  stringified). Canonical inputs pass through byte-identical.
- applyRanges (src/stream.ts): append a model-facing REJECTED note with the
  kernel per-entry reasons to BOTH the success receipt and the 0-blocks
  failure message; log server-side ('compress PARTIAL').
- parseCompressInput: warn-log partial drops (previously invisible because
  diagnostics.ok was true whenever ≥1 range survived).

Behavior changes (disclosed):
- Previously hard-failing param shapes are now accepted (single-object
  content, bare array, their string-encoded variants): old = full call
  failure, new = normalized fold.
- Tool receipts gain an optional extra line on partial drops: old = silent
  under-reporting, new = explicit rejection note + re-issue guidance.
- Garbage object content now reports kind=no-valid-ranges with per-entry
  reasons instead of opaque content-not-array (still a failure).
@ranxianglei
ranxianglei force-pushed the 2026-09-27_compress-param-drift-fix branch from 3d6d0c4 to ea53130 Compare September 27, 2026 12:50
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Review complete — #1497 is correct, green, and mergeable once coordinated with parallel PR #1499 (the ⚠️ section is the important one).

① Direction check (vs #1494) ✅

  • Root cause verified at code level: the kernel parser reports per-entry rejections in diagnostics.invalidItems/invalidReasons, but applyRanges (src/stream.ts) only surfaced them when ranges.length === 0. With ≥1 surviving entry ok === true, so neither the receipt nor the server log mentioned the drop — exactly the "3 entries in, 2 blocks out, no note" transcript.
  • The fix sits at the decision point, not a mask: normalizeCompressInput() normalizes shape before the kernel parse; droppedEntriesNote() surfaces kernel diagnostics where the receipt is synthesized. Litmus test passes: future corruption arriving in a new shape still gets normalized (or rejected with a reason), and any drop still surfaces through the same diagnostics channel.
  • The PR body's refutation of the summary-length hypothesis holds: the 20000-char cap fails the whole call atomically and cannot drop one entry of three.

② Regression audit (existing behaviors on the touched paths, read from pre-change code)

Existing behavior Verdict
Success-receipt line 1 [Compressed … → N block(s)…] preserved — note appends as an optional line 2; multi-line receipts already existed (per-block fingerprint lines, src/stream.ts:424)
0-blocks FAILED line stays single-line (#1029 visibility marker) preserved — note concatenated inside the existing brackets
Canonical inputs byte-identical preserved — asserted in tests (object / empty / string / line-form passthrough)
#1366 empty-call detection (missing-content / not-object) preserved — asserted in tests
Plugin mode (agent-owned folds) untouched — plugin mode never routes through applyRanges
Wire/upstream protocol, config surface, persistence, kernel pin untouched
Server logging new warn line only in cases that were previously 100% silent

③ Code review — sound. Minor, non-blocking observations:

  • looksLikeRangeEntry (src/compress-tool.ts): hasEnd also accepts messageId (the same field as hasStart) — sloppy but harmless: the kernel validates bounds anyway, worst case is a rejection with a reason.
  • Receipt output is properly bounded: ≤3 reasons × 160 chars each — no unbounded payload growth.
  • One untested combo: the 0-blocks FAILED path with a dropped note (simple concat; low risk).

④ Test validity ✅ — 8 tests with teeth: both defects reproduce pre-fix (silent partial success; content-not-array on single-object content); canonical passthrough is byte-pinned; the partial-drop e2e through applyRanges asserts receipt text + server log + exactly-one-active-block state. A regression of either defect would fail this suite.

⑤ Diff cleanliness ✅ — 3 files, +217/−3, every line on-purpose. No lockfile/generated/whitespace churn.

Evidence (rebased head)

⚠️ Critical: overlap with parallel PR #1499 — coordinate before merging either
PR #1499 (from sibling issue #1495 — the same incident/transcript) fixes the same root cause and modifies the exact same lines: success-receipt construction, the 0-blocks FAILED return, and the parseCompressInput else-if. Merging both as-is → textual conflict and doubled notes on one receipt ([…REJECTED…] + [PARTIAL …]).

Division of unique value (verified against both diffs + kernel source):

Neither alone covers the full incident class. Kernel evidence (pinned acp-kernel 0.0.95, dist/index.js:7434-7437 and 7469-7475): the truncated-salvage paths set diag.invalidItems = invalid, where invalid counts only salvaged-but-invalid entries — fragments lost to truncation are discarded by the salvage process itself and never counted. So a gateway-truncated batch (the exact mechanism of this transcript per #1495's analysis) can arrive as invalidItems === 0 && kind === "truncated" with ranges silently lost — this PR's note would not fire; only #1499's truncated branch names it.

Recommendation: merge this PR first (fully green above), then rebase #1499 to strip its overlapping invalidItems note/log and keep its unique parts (truncated branch + applyErrorNote). Or reverse the order if you prefer — the end state must carry both normalization and truncated handling. Once the fix lands, #1494/#1495 should be closed as the same incident (keep one, dup-close the other).

Verdict: mergeable once coordinated with #1499 — this PR alone is complete, correct, and green; the sole risk is receipt-text duplication from the sibling PR landing without dedupe. Merge stays human-only.

一句话中文摘要:修复了批量压缩部分失败时静默丢条目(回执只报成功、无日志)与单对象 content 重试被硬拒两个缺陷,修在参数归一化+回执合成的决策点而非过滤症状;已 rebase 到当前 master,本地与 CI 全绿(含 codex e2e);注意与 #1499 改同一批回执行需去重协调——本 PR 独有形状归一化、#1499 独有 truncated-salvage 可见性,二者缺一都覆盖不全本次事故,建议先合本 PR、再让 #1499 rebase 去重后合并,合并由人来执行。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review: verified, two merge-prep follow-ups applied ✅

Independent reviewer pass (mutation-verified) confirms both fixes are real, correctly implemented, and pinned:

  • m1 (revert success-branch receipt note) → exactly the partial-success test reddens (1 red / 7 green)
  • m2 (bypass normalizeCompressInput) → 6 red / 2 green (all normalization tests + end-to-end)
  • Full suite under the PR's own kernel pin: 2918 pass / 0 fail / 12 skipped; typecheck clean
  • No double-normalization conflict (bare top-level object still takes the kernel's own path; strict bili parse fails on single-quoted JSON → kernel quote-salvage untouched); #1366 empty-call detection intact; advertised tool schema unchanged

Applied on top (head 11c3a16)

  1. F1 test gap — the FAILED-branch (blocksCreated === 0) dropped-entry note had no regression test: reverting that single line passed the suite. Added #1494 A: a parse-dropped entry is surfaced in the FAILED receipt too (0-blocks apply failure) — mutation-verified (revert → 1 red / 8 green, restore → 9/9).
  2. PR body now carries Fixes #1494 + Fixes #1495 so the merge closes both.

Filed for the root-cause home (kernel)

The asymmetric salvage wall (single-object content rejected while top-level bare object is tolerated) is kernel-owned: ranxianglei/acp-kernel#470. The bili wrapper is a defensible interim for the proxy lane, but plugin-mode agents bundle their own kernel and keep the wall — the kernel fix closes the divergence.

Non-blocking follow-ups (noted, not fixed here)

Verdict: human-merge ready (receipt text = model-visible message shape, §7.4 criterion 4 — stays human by design).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lean模式批量压缩丢失 compress failed may be caused by params

1 participant