Repository navigation
[PR] fix: classify body stability on a marker-blind view (migrating cache_control stops reading as mid) - #4391
ranxianglei wants to merge 6 commits into
Conversation
…e_control no longer reads as mid-history Client-managed Anthropic caching migrates cache_control breakpoints onto the newest message each turn, so a pure-append turn's raw bytes diverge exactly where the marker moved (strictly before the messages-array close). The raw-LCP arms could only ever reach "mid", producing hundreds of spurious "mid-history divergence" samples on long cached dsh sessions (#4137: 247 samples / 287 requests) while isolated experiments showed a 99.7% upstream prefix hit. Strip every cache_control field from BOTH bodies before the LCP/array-close comparison (stripCacheControlFields), applied together only when both sides normalize, else fall back to raw bytes. The forensic offset (bl, #2868) keeps the raw wire offset; only the class moves. Every rung (including #3476's in-unit-insertion detection) now judges the same marker-blind view. Regression tests: tests/issue4137-cache-control-migration.test.ts. Fixes #4137
Review finding on #4391: the new scan runs on every settled pair of every lane, and its nothing-to-strip fast path still pays a full JS byte-loop over the whole body (~0.9 ms at 512 KiB, ~2.5 ms at 2 MiB measured locally). A native Buffer.includes precheck returns the original buffer after ~0.1 ms when the key bytes are absent anywhere — strictly equivalent (a field can only exist if its quoted key bytes do), ~10x cheaper on mark-free wires.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6186-pr.4391.ca6b4b1This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4391.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) Review 结论(head
|
master window commits 293ae65/de5e03659 bumped kernel/package.json to 0.0.165 without regenerating the lockfiles; lock-sync re-resolution then rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and fails. Pure recorded-version sync via npm install --package-lock-only; no dependency or semantic change.
What & why
The per-call body-stability classifier (
src/cache-ledger.ts) compared RAW outbound bytes. On the Anthropic wire, client-managed prompt caching migratescache_controlbreakpoints onto the newest message each turn (src/loop/cache-control.tsforwards pre-existing client marks verbatim instead of bili's own cumulative stamps). A pure-append turn therefore diverges in raw bytes exactly where the marker moved — strictly before the messages-array close — and the raw-LCP arms could only ever reach"mid".On one long dsh session that produced 247 spurious "mid-history divergence" samples across 287 requests, while isolated experiments (pure-append / markers-stripped / markers-migrated) each showed a 99.7% upstream prefix hit. Found while analyzing #4128.
Fix
stripCacheControlFields()strips every"cache_control":<object>field (plus one separating comma) from BOTH bodies before the LCP / array-close / in-unit-insertion comparison. It is applied together only when both sides normalize (a mixed normalized/raw pair would compare different coordinate spaces); otherwise it falls back to raw bytes. Every rung — including #3476's in-unit-insertion detection — now judges the same marker-blind view.Behavior change (old → new): for a turn whose only byte-level difference is a migrating
cache_controlbreakpoint, the reported stability class changesmid→append/tail(the true content growth). This is a diagnostic/attribution surface only — no billing or compression decision reads this class. The forensic offsetbl(#2868) is unchanged: it still reports the raw wire divergence offset; only the class moves. Genuine content rewrites still readmid(pinned by negative tests).Scope
kernel/change.src/cache-ledger.ts(+121/−4), newtests/issue4137-cache-control-migration.test.ts.Tests
Targeted local runs: 134 pass across the body-stability, seam-detector, and #3476 in-unit-insertion suites;
npm run typecheckclean. Full-suite verdict = remote CI on this head.Closes #4137