Repository navigation
fix: classify tail-append body pairs by prefix+suffix coverage (#2866) - #2872
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3338-pr.2872.675a7ffThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2872.tgz
npm install -g packageInstall instructions are refreshed on each push. |
a5a73e6 to
36b2dad
Compare
[bot] 🏷 Review 完成 — 独立复核(非转述作者自述),结论:可合,待新 head CI 绿。 先说两件我在 review 过程中实际动手的事:
结构化八问
跨 PR 依赖(需 owner 排程决策):#2870(byte-lcp-utf8)把 本地验证清单(定向,全量交给 CI):17 个 cache-ledger 消费方测试文件 135/135 绿;golden-storage-render 5/5 绿(regen 后);旧分类器 stash 反证如上。typecheck/build 由 CI 把关。 判定:可合——待 |
[bot] 🏷 CI 全绿确认: 最终判定维持:可合。合并顺序提醒仍有效——#2870(byte-lcp-utf8)与本 PR 同区硬依赖,建议先合本 PR 再 rebase #2870(适配 |
ed23d0b to
94b7220
Compare
Real wire bodies serialize fields AFTER the message array (tools, stream_options), so a pure tail-append diverges at the array's close, far short of the payload end. The ledger's append test compared LCP reach against payload length minus 64B, which misclassified every growth pair of an append-only session as mid-history divergence and made /acp-cache report phantom 'real prefix rewrites'. The append class now also accepts prefix+suffix coverage under the same tolerance: byteLcs proves the trailing bytes (the fields after the message array) survived verbatim. Head pairs are never suffix-rescued so model switches on small bodies stay head. The slow path stores the suffix reach in a new sparse bls field (also exported as bodyLcs). Genuine mid-history rewrites (length-changing edits anywhere in history) still classify mid; identical bodies still be=1; capped bodies still unknown. Regression tests pin the realistic wire shape in both directions plus the head asymmetry.
The new sparse bls field (and the mid->append reclassifications) change the persisted session metadata section; only stateDigest/metadata sectionDigest drift per lane, no wire or block-content changes. Regenerated with scripts/update-storage-render-goldens.ts.
94b7220 to
f8d3dfe
Compare
|
Closing: superseded by #2983 (already merged to master). While this PR was in flight, the same root cause — trailing top-level fields ( Verified empirically on master My prefix+suffix-coverage approach is subsumed by the structural check (which also catches trailing-param jitter beyond any byte budget). The only remaining unique delta here was a |
Closes #2866
Root cause (verified empirically, not by assumption)
The issue hypothesized that token-estimate churn in ACP tag attributes polluted the BODY STABILITY mid-history bucket. A minimal repro driving 4 append-only turns through the real in-process proxy (chat wire, default config) shows the misclassification is universal and independent of tags:
Real wire bodies serialize fields AFTER the message array on every lane (
tools~6KB withinjectTool, plusstream_options). The ledger's append test compared LCP reach againstminLen - 64— anchored at the payload END — so a pure tail-append diverges at the array's close, far short of the end, and every growth pair of an append-only session landed inmid→/acp-cacheprinted "N mid-history divergence(s) — real prefix rewrites" for sessions nobody rewrote. This explains #2863's "267 pairs: 0 byte-identical · 266 mid" on an append-only session.The tag hypothesis is refuted as the dominant cause on two counts: snapshot rendering freezes per-ref token values (kernel
renderWithSnapshot), and the repro misclassifies even when message content carries no rendered tags at all.Fix
src/cache-ledger.ts: the append class now also accepts prefix+suffix coverage under the same ±64B tolerance — newbyteLcsproves the trailing bytes (the fields after the message array) survived verbatim, which is what "the whole shorter payload survives" actually means when the payload doesn't end where the history ends. Order preserved: fast path unchanged; head pairs (lcp < 1KiB) are never suffix-rescued, so model switches on small bodies stayhead. The slow path stores the suffix reach in a new sparseblsfield (exported asbodyLcsin the report JSON) — additive, ledger staysv:1.Genuine mid-history rewrites (length-changing edits anywhere in history) still classify
mid; identical bodies stillbe=1; capped bodies stillunknown.Behavior change disclosure (old → new)
mid-historytotail-append. The warning line wording is unchanged; it now fires only for genuine rewrites. This is the intended fix — the old counts were false positives.blson ledger lines /bodyLcsin the exported report object: omitted unless the slow path ran. No persistence version bump.chainEgressStampusers still seemidfor their declared volatile carrier — correct, because those bytes really do change between calls and really truncate the provider prefix cache there.Verification
tail-append 3 · mid-history 0(wasmid-history 3+ phantom warning); chainstamp scenario unchanged (mid, justified above).No config surface changes. No kernel/ or pi-subagents/ changes.