Repository navigation
feat: deterministic pre-crush channel in front of model absorb (#1094) - #1098
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1098Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1098.tgz
npm install -g packageInstall instructions are refreshed on each push. |
核对完成 — 可以合并(合并留给人工操作)。我没有重做工作,只做了拉取核对 + 门禁 + CI 确认,外加一处 review 中发现的小问题已直接修掉。 VerifiedBase & diff hygiene
Code review (both compression modes considered)
Gates
Found & fixed directly during review (docs overclaim)The shipped docs said "ERROR lines are never dropped" (CHANGELOG.md, CONFIGURATION.md/.zh-CN.md, README.md/.zh-CN.md, and the PR body), but the code — and its own test, which asserts 中文摘要:核对了 PR #1098 的代码、测试与 CI(两级门槛、fail-open、确定性/前缀缓存稳定均验证通过,本地 typecheck/test/build 与全部 8 项 CI 检查全绿),发现文档把日志错误行的选择预算(每级固定 10 行、首末必留)夸大为「ERROR 行永不丢弃」,已在 5 个文档文件和 PR 描述里直接修正措辞并补了一个 docs-only commit( |
ranxianglei
left a comment
There was a problem hiding this comment.
审查结论:HOLD 维持(block 通知在且正确);两个发现请带进 #1097 后的重设计
确认:PR body 的 { ...msg, text: crushed.text } 共享 msg.id,src/pre-crush.ts:1260——不破坏 #1097 的原件可检索前提,冲突只在 wire 槽位归属,与封存理由一致)、parse 失败逐字透传有测试、默认关双重门(absorb.enabled && preCrush.enabled)。
重设计时请带上:
- [Medium]
[ACP absorb]标记截断丢 payload 尾部:src/pre-crush.ts:1246-1247payload = text.slice(0, markerAt),之后视图消息只剩 crushed 前缀——工具输出里天然出现该字面标记(例如开发者把 bili transcript cat 进工具)时,标记之后的全部内容从模型视图中静默消失。 - [Medium] JSON 数字保真漏洞:
crushJson(src/pre-crush.ts:299)经JSON.parse/JSON.stringify重序列化,实测1e999(合法 JSON 文法)round-trip 变null——折叠发生时模型视图里的静默数值损坏。fail-open 只接异常不接值漂移。 - [Low]
__pre_crush键碰撞只文档未守卫(pre-crush.ts:80):用户 payload 合法携带该保留键时与 crushed 信封不可区分;"输入含该键即 bail" 一行可闭。
|
@ework — please re-review the current head 6f7cccb in full. Additional context since your last pass: an independent deep review is at #1098 (review) (hold maintained per the #1094 freeze; three findings — marker-truncation payload loss, JSON numeric fidelity via 1e999, |
对 head Re-verification on 6f7cccb
The three findings — all confirmed with runtime evidence1. Marker truncation (Medium) — real, and the consequence is worse than a "missing tail". 2. JSON numeric fidelity (Medium) — real. 3. DispositionPer the #1094 freeze I pushed no fixes to this branch — all three findings land as design inputs on #1097 (comment posted there with mechanism, repro numbers, and fix directions), where the digest layer and the eventual pre-crush unfreeze will consume them. The rest of the implementation stands as verified in my first pass; nothing new found in this re-review. Merge stays with you — and given HOLD, no merge action is expected until #1097 lands. 中文摘要:对 6f7cccb 做了全量复审(增量仅文档、本地门禁与 CI 全绿),独立评审的三条发现全部经运行时复现确认——标记截断会丢合法 payload 尾部(连 ERROR 行都从模型视图消失)、 |
|
Superseded by #1176 — reworked on top of acp-kernel 0.0.84 (#1175), which now ships the crush strategies natively (kernel PR ranxianglei/acp-kernel#359, pipeline node between absorb-hide and absorb-prompt). #1176 keeps only the config surface ( |
src/pre-crush.ts+tests/pre-crush.test.ts) and will be reworked after the kernel API lands — see the latest comments on #1094.Closes #1094
What
Opt-in
compress.absorb.preCrush.enabledruns a deterministic, model-free denoise over each eligible oversized tool result before the[ACP absorb]decision. Newsrc/pre-crush.ts(pure functions, zero new runtime deps); wired intoapplyAbsorbView(src/absorb.ts) so every prepare/preflight path gets it; config parsed/merged/resolved insrc/config.ts+src/compress-settings.tsfollowing the existing three-level deepest-wins convention.Three conservative strategies:
__pre_crushenvelopes that decode back byte-for-byte.[… N frames collapsed], honest[N lines omitted: …]footer instead of CCR hashes.The two-tier gate falls out of the existing threshold: results crushed below
absorb.minToolTokensskip the model round-trip entirely; results still over it are forwarded denoised with the normal absorb prompt. Crushed bytes are a pure function of the payload, so re-sent history stays byte-stable for prefix caching.Safety
absorb.enabledandpreCrush.enabledare true;minReduction(default0.1) rejects marginal savings.Deliberate deviations from headroom (flagged for review)
headroom_retrievechannel; dangling hash refs would be noise, so the footer states honest counts instead.minTokensfrom the issue folded into the existingabsorb.minToolTokens— a separate size knob would have duplicated the eligibility gate.minReductiondefault0.1rather than the triage sketch's ≥50% — below-threshold results skip the model regardless of percentage; above-threshold ones benefit from even modest denoise before distillation. User-tunable either way.Pre-flight
npm run typecheck— cleannpm test— 2191 pass / 0 fail / 2 skipped (e2e gates)npm run build— successtests/pre-crush.test.ts(29 tests: envelope losslessness, scanner bail-outs, log budgets, dispatch guards, two-tier integration throughapplyAbsorbView, config parse/merge/resolve); extendedtests/absorb.test.tsmerge expectations.