Skip to content

[PR] fix: classify body stability on a marker-blind view (migrating cache_control stops reading as mid) - #4391

Open
ranxianglei wants to merge 6 commits into
masterfrom
2026-10-23_cache-control-seam-class
Open

ranxianglei wants to merge 6 commits into
masterfrom
2026-10-23_cache-control-seam-class

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

The per-call body-stability classifier (src/cache-ledger.ts) compared RAW outbound bytes. On the Anthropic wire, client-managed prompt caching migrates cache_control breakpoints onto the newest message each turn (src/loop/cache-control.ts forwards pre-existing client marks verbatim instead of bili's own cumulative stamps). A pure-append turn therefore diverges in raw bytes exactly where the marker moved — strictly before the messages-array close — and the raw-LCP arms could only ever reach "mid".

On one long dsh session that produced 247 spurious "mid-history divergence" samples across 287 requests, while isolated experiments (pure-append / markers-stripped / markers-migrated) each showed a 99.7% upstream prefix hit. Found while analyzing #4128.

Fix

stripCacheControlFields() strips every "cache_control":<object> field (plus one separating comma) from BOTH bodies before the LCP / array-close / in-unit-insertion comparison. It is applied together only when both sides normalize (a mixed normalized/raw pair would compare different coordinate spaces); otherwise it falls back to raw bytes. Every rung — including #3476's in-unit-insertion detection — now judges the same marker-blind view.

Behavior change (old → new): for a turn whose only byte-level difference is a migrating cache_control breakpoint, the reported stability class changes mid → append/tail (the true content growth). This is a diagnostic/attribution surface only — no billing or compression decision reads this class. The forensic offset bl (#2868) is unchanged: it still reports the raw wire divergence offset; only the class moves. Genuine content rewrites still read mid (pinned by negative tests).

Scope

  • No config surface change (no new env/config/CLI flag).
  • No kernel/ change.
  • No version bump (content branch).
  • Files touched: src/cache-ledger.ts (+121/−4), new tests/issue4137-cache-control-migration.test.ts.

Tests

Targeted local runs: 134 pass across the body-stability, seam-detector, and #3476 in-unit-insertion suites; npm run typecheck clean. Full-suite verdict = remote CI on this head.

Closes #4137

…e_control no longer reads as mid-history

Client-managed Anthropic caching migrates cache_control breakpoints onto the
newest message each turn, so a pure-append turn's raw bytes diverge exactly
where the marker moved (strictly before the messages-array close). The raw-LCP
arms could only ever reach "mid", producing hundreds of spurious "mid-history
divergence" samples on long cached dsh sessions (#4137: 247 samples / 287
requests) while isolated experiments showed a 99.7% upstream prefix hit.

Strip every cache_control field from BOTH bodies before the LCP/array-close
comparison (stripCacheControlFields), applied together only when both sides
normalize, else fall back to raw bytes. The forensic offset (bl, #2868) keeps
the raw wire offset; only the class moves. Every rung (including #3476's
in-unit-insertion detection) now judges the same marker-blind view.

Regression tests: tests/issue4137-cache-control-migration.test.ts.

Fixes #4137
@ranxianglei ranxianglei changed the title fix: classify body stability on a marker-blind view (migrating cache_control stops reading as mid) [PR] fix: classify body stability on a marker-blind view (migrating cache_control stops reading as mid) Oct 23, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 同家族、不同诊断面——与先到的 #4133 / PR #4139 在 body-stability ladder 上覆盖重叠。 按失败签名(bodyClass="mid" / cache_control 迁移 / pure-append 误判 / seam suspect)检索 open+closed issue 与 PR:

标签: bug / dsh / scope/ui / p2(诊断分类缺陷,dsh anthropic-lane 复现,纯观察面无功能影响,与 #4133 定级一致)

正在做独立审核:分支已拉取(head 196d946fd8,单 commit,与 master 同步、0 behind),diff 与测试已通读,CI 22 个 check 排队中。定向本地验证 + 回归清点完成后,结论以八问形式跟进。

@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/ui labels Oct 23, 2026
Review finding on #4391: the new scan runs on every settled pair of every
lane, and its nothing-to-strip fast path still pays a full JS byte-loop over
the whole body (~0.9 ms at 512 KiB, ~2.5 ms at 2 MiB measured locally). A
native Buffer.includes precheck returns the original buffer after ~0.1 ms
when the key bytes are absent anywhere — strictly equivalent (a field can
only exist if its quoted key bytes do), ~10x cheaper on mark-free wires.
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_cache-control-seam-class (5df7d30)

Latest build — install this exact version

npm install -g billion-context@0.0.6186-pr.4391.ca6b4b1

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
5df7d30 billion-context@0.0.6186-pr.4391.ca6b4b1 build
565f174 billion-context@0.0.6099-pr.4391.de6e473 build
fadb1e7 billion-context@0.0.6003-pr.4391.b140a33 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4391.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 结论(head 06d092fcf = 原 196d946fd8 + 本审核补的 1 个性能小修 commit;独立审核,不复述 PR 描述与 issue 侧八问)

⓪/① 方向关:独立读了 diff 与 master 侧 rung 实现后确认——根因在判定口径(ladder 各 rung 以原始字节做「前缀存活」判据,把合法的客户端 cc 断点迁移读成内容改写),修复打在判定层(统一归一化视图),不在传输路径过滤 payload:litmus test 通过(同一根因换个标记位置再现——system/tools 位——cc-blind 比较依然成立)。不是治标。

八问:

  1. 是什么问题:客户端自管滚动 cache_control 断点每轮迁移到最新消息,纯尾追加的两次出站 body 原始字节恰好在标记移走处分叉(严格位于 messages 数组闭合之前),body-stability ladder 的原始字节 LCP 停在那里,把稳定追加误标为 mid-history divergence([Bug] anthropic lane: cache-hit % halved when the upstream relays an OpenAI-semantics backend (billing_usage.semantic=openai, input_tokens inclusive) #4128 会话 247/287 假样本)。纯诊断误分类。
  2. 什么原因导致的:判定缺 modulo-cache_control 维度。bili 自家累加标记([Bug] Anthropic lane never emits cache_control breakpoints — prompt cache can only ever cover the system head (~11K), the whole history re-bills at full price every request #1637)不产生此问题,只有客户端自管滚动断点(dsh profile web / Claude Code 式)触发;cache-ledger bs 分歧分类缺 tail 类:尾部参数抖动/临时尾注被误标为「历史中段重写」 #2982/[Bug] Two stable false positives in BODY STABILITY / acp-cache diff: dump request-header preamble folded into the byte comparison, and the auto-review giant single user message makes a pure append look like a mid-history insert | [Bug] BODY STABILITY /… #3476 各 rung 注释证明 arrClose/midInsert 臂是为尾部参数与单元内生长设计的,未考虑标记迁移——是盲区,不是已知取舍。
  3. 影响范围与严重程度:anthropic wire 且客户端自管滚动断点的会话的诊断读数;wire 字节、压缩、计费、真实命中率零影响。严重度:p2 系统性假告警误导排障,无可用性影响。
  4. 怎么修复的:一次归一化、全 rung 同视图。stripCacheControlFields(单遍线性扫描,string-aware,剥每个 "cache_control":<object> 字段+恰好一个分隔逗号,使带/不带标记的同成员归一后逐字节一致;非对象体/畸形值对象返回 null)对两侧 body 归一,仅当两侧都归一成功才一起用,否则回退原始字节(混合坐标空间不可比);±64 append / <1KiB head / arrClose tail / [Bug] Two stable false positives in BODY STABILITY / acp-cache diff: dump request-header preamble folded into the byte comparison, and the auto-review giant single user message makes a pure append look like a mid-history insert | [Bug] BODY STABILITY /… #3476 in-unit-insertion 全部改判归一视图(nlcp+归一长度);bl([Bug] BODY STABILITY 的 byteLcp 实际统计 UTF-16,导致非 ASCII 字节偏移和 head/mid 分类错误 #2868 取证偏移)保留原始 wire 偏移,只动类不动偏移。
  5. 是否有回归问题(逐项清点既有行为):
    • 无标记 body(OpenAI wire 等):strip 走 fast path 返回原 buffer(同一对象引用)→ nlcp===lcp、长度不变 → 所有 rung 与修复前逐字节一致 ✓(t6 钉住)
    • 一侧归一失败的 mixed 对:双双回退 raw → 保守路径 = 修复前行为 ✓
    • head 臂:只有「首个分叉本身就是迁移标记」的对才会换类;真实的头部内容改写在归一视图中照样分叉 → 仍 head ✓
    • tail 臂:arrClose 与 nlcp 同在归一坐标计算,坐标空间自洽;marker 对的真·数组闭合后分叉现在能正确落 tail(修复前落 mid/midInsert)✓
    • [Bug] Two stable false positives in BODY STABILITY / acp-cache diff: dump request-header preamble folded into the byte comparison, and the auto-review giant single user message makes a pure append look like a mid-history insert | [Bug] BODY STABILITY /… #3476 臂:改判归一视图;t5 钉住「末消息内部真替换(长度变化)仍 mid」——empty-residual 洗白守卫未被削弱 ✓
    • bl/be:diff 未触碰(bl 在归一前赋值,be 完全未动)✓
    • bs 消费方清点(grep 全 src):仅 src/cache-ledger.ts:2055-2058 报告计数与 :2123 bodyClass 行项——纯展示面,无控制流依赖 ✓
    • 高危模式逐项:无语义漂移(fast path 恒等)、无 fail-fast 变吞错(畸形→null→raw 保守)、阈值 ±64/1KiB/arrClose 逐字未动、输出格式仅已披露的分类结果变化、无缓存/重试策略失效 ✓
    • 一处披露不完整(非回归):PR 描述只披露 mid → append/tail,但我的负控制实测:system/tools 位标记(位于 body 前 1 KiB 内)的 marker 迁移对,修复前读的是 head,修复后读 append——同一机制同一方向的假阳性被一并修正,但披露文本没列这个分支。建议作者在 PR 描述补一句。
  6. 是否破坏缓存:否。纯观察层:wire 字节、压缩/fold/reconcile、provider prefix cache 零接触;效果是诊断信号更准(更少假 mid)。
  7. 是否改了设计方向:否。架构/边界/数据流不变,给既有观察面补一个判定维度。治理提示(首条回复已述,重申):与先到的 [PR] fix: classify marker-migrated tail appends as append, not mid-history rewrite (#4133) #4139/cache-seam: pure tail-append misclassified as mid-history rewrite (cache_control marker migration breaks byte-LCP) #4133 在 ladder 上覆盖重叠但互不完全包含([PR] fix: classify marker-migrated tail appends as append, not mid-history rewrite (#4133) #4139 独占 detectSeam/msgIndexCc 与归一失败兜底 rescue;本 PR 独占 arrClose/nlcp 坐标与 [Bug] Two stable false positives in BODY STABILITY / acp-cache diff: dump request-header preamble folded into the byte comparison, and the auto-review giant single user message makes a pure append look like a mid-history insert | [Bug] BODY STABILITY /… #3476 臂统一口径)。建议先合 [PR] fix: classify marker-migrated tail appends as append, not mid-history rewrite (#4133) #4139 再 rebase 本 PR(预计 bs 表达式行有小冲突,语义按并集解);是否并成单 PR 由 owner 拍板。
  8. 是否引入新配置:否。无新 env/config/CLI 标志。

测试有效性(④):新文件 7 用例走真实入口(noteForwardedBody+settleUsageReport+buildSessionCacheReport),断言有牙(具体类值+报告计数)。我独立跑了负控制(pre-fix master 上执行该文件):t2/t3 红(expected append / actual head)——bug 复现成立,回归必被抓;t1 在 pre-fix 也绿——其形状恰好被 #3476 empty-residual 臂偶然接住(这正说明各 rung 需要统一视图,也解释了为何线上样本按标记位置呈现 mid/head 两种假阳性);t4-t7 为守卫用例,两侧皆绿符合预期。

diff 整洁度与代码质量(⑤/⑤b):原 commit 单文件+测试(+121/−4),每行服务于目的;命名达意、无死代码、错误处理完整(null 回退有 why 注释)、无魔法数字(CACHE_CONTROL_KEY 具名常量)、无新依赖、风格与文件一致。

验证证据:

  • 本地定向(PR head):13 个 body-stability/seam 家族文件 72 pass / 0 fail;typecheck clean
  • 负控制:pre-fix master 上 t2/t3 红(见上)
  • 本审核顺手小修(独立 commit 06d092fcf,已在 PR 分支):strip 的无标记 fast path 原本仍要跑完整 JS 字节扫描(实测 512 KiB ≈0.9 ms、2 MiB ≈2.5 ms,每次 settle 都付);加一行原生 Buffer.includes 前置检查(键字节不存在 ⇒ 直接返回原 buffer,实测 ≈0.1 ms,~10×),行为严格等价(字段存在则其引号键字节必在)。已重跑定向 31 pass + typecheck clean 后推送
  • CI(06d092fcf):进行中——8 success / 10 skipped(opt-in e2e 门,正常)/ 14 queued(test shards 排队中);全量判定以该 head 的 CI 为准,绿了我会跟一条确认

判定:可合并(待 06d092fcf 的 CI test shards 全绿 + 人工合并;与 #4139 的合并顺序由 owner 定,建议 #4139 先行)。请自行合并:#4391

ci-check added 4 commits October 26, 2026 10:18
master window commits 293ae65/de5e03659 bumped kernel/package.json to
0.0.165 without regenerating the lockfiles; lock-sync re-resolution then
rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and
fails. Pure recorded-version sync via npm install --package-lock-only;
no dependency or semantic change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/ui

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cache report: append-only turns misclassified as mid-history divergences because cache_control markers migrate each turn

1 participant