Skip to content

[PR] acp-cache report: per-line switch sections + disjoint seam sections + fold-attribution X logging (#3307) - #3333

Closed
ranxianglei wants to merge 7 commits into
masterfrom
2026-10-16_acp-cache-wire-upstream-restart-per-line
Closed

ranxianglei wants to merge 7 commits into
masterfrom
2026-10-16_acp-cache-wire-upstream-restart-per-line

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

What

Two fixes plus three reporting/observability improvements, all on the same #3307 surface (acp-cache attribution + report rendering). No wire/message-shape change, no config key, no kernel change, no update.ts touch.

Root cause (shared by both fixes)

A stateless proxy client (e.g. codex_webui over responses) resends its FULL history every turn, and bili rebuilds the outbound by collapsing folded ranges into summaries. Two quantities were being measured over the client's un-shrunken inbound view instead of the compressed/billed view:

  • V (the fold-shrink denominator) → inflated ~50× → a degenerate shrink% and effectively-dead coverage gates.
  • X = firstFoldStartTokens (prefixTokensBeforeRef(start_ref) over the in-memory tokenSnapshot, which is built on the same un-shrunk inbound) → lands ≥ billed input → decomposeSample zeroes structuralExcess → the fold's re-pay becomes unattributed → it falls into the ⚠ CACHE SEAM bucket as a false "suspected mid-history prefix break".

FIX 1 (P1) — measure fold-shrink V over the live view (e31661f74)

src/stream.ts: the fold-shrink denominator V now counts ctx.messages (the live/processed view = the billed basis) instead of ctx.compressMessages ?? ctx.messages (the raw resend). applyCompression still runs on the raw history (fold geometry must map to raw message boundaries, #2627) — only the observability denominator changed. This aligns with #1911's own stated intent ("V is the pre-fold LIVE view size"); the bug was that production's ?? preferred the bloated raw over the correct live view.

FIX 2 (P2-A) — attribute fold re-pay via the wire-measured prefix (da6834ee9)

src/cache-ledger.ts recordCacheSample: when a genuine bili-side mid-history break is present (substantive residual, both outbound bodies captured & uncapped, not a client-side inbound rewrite #3023, mid-list shape), the stable prefix is measured off the wire — byte-LCP of this outbound vs the previous, scaled to billed input — and min()'d into X before decomposeSample, so the fold is credited again instead of crying a false seam. Host-side only; decomposeSample is called as-is with the corrected firstFoldStartTokens. Guards: a pure tail-append is excluded (LCP lands near the end → xMeasured ≈ input → no effect); a client-side inbound rewrite is excluded (its earlier divergence would understate and mis-credit the fold).

REPORTING 3 — per-line WIRE / UPSTREAM / RESTART sections (5bdc878a6)

handleAcpCache now renders per-line sections for three invalidation buckets that already carried per-event data but were shown only as aggregate counts in CACHE INVALIDATION: WIRE SWITCHES, UPSTREAM SWITCHES, RESTART/REFORK. Each is gated on count > 0 exactly like the existing MODEL / KEY / PROMPT / PROJECTION sections, so a session that never hits one stays byte-identical. Lets the floor-3 cross-check ("is seam sample #3416 one of the 10 restart/refork events?") run straight off the report.

REPORTING 4 — disjoint ⚠ suspected-seam vs ✎ client-rewrite sections (4db43b05b)

detectSeam pushes BOTH the #3023 client-side mid-history rewrite AND the legacy suspected-rebuild fall-through into ONE shared capped ring (led.seamEvents). formatSeam's ⚠ section iterated every event without filtering, so a host-owned client rewrite was also listed under the "suspected" header (double-listed) — contaminating suspect triage. The section now lists only non-client events; the ✎ section (already filtered to e.client) is unchanged. Counters were always clean (seamSuspects vs clientInserts); only the per-line listing leaked. Web session page already distinguishes them inline, so it was unaffected.

REPORTING 5 — fold-attribution X / V / Vraw / Vp log (cfb2d31a0, 3a1541c32)

One [acp-compress-obs] fold-attribution X(...) · V · Vraw · Vp line per fold carrying the actual X per new block plus the live-view vs raw-inbound totals. Cross-referenced against the ledger's billed input it separates the two inflation sources (unit scale vs view identity) in a single capture. Pure logging.

Behavior changes (disclosed)

  • FIX 1: for stateless-client proxy sessions, fold shrink% rises from a degenerate ~1% to its true value (~40–50%); the degenerate-fold gate (DEGENERATE_FOLD_COVERAGE = 0.8) and the maxShrink advisory go from effectively-never-firing to firing correctly; postCtx + the next-request cache ceiling are now computed from the live view. Unchanged whenever messages == compressMessages (which is exactly what every pre-existing unit test exercises).
  • FIX 2: genuinely-fold-caused mid-history breaks move from the "unattributed seam" bucket into the fold/compRepay bucket — this IS the fix (expected to move the reported samples out of ⚠ CACHE SEAM). Bounded residual: a mid-history break co-occurring with an unrelated pending fold could be credited to that fold (capped by remainder; no longer flagged as a seam). Gated to non-client mid-list breaks only.

Tests

  • FIX 1: tests/issue3307-fold-view-basis.test.ts (5 cases: V tracks live-not-raw; Vp === V − S same-view net change; multi-level folding keeps tracking the live view; re-folding tracks the already-shrunk live view; baseline messages == compressMessages unchanged).
  • FIX 2: tests/issue3307-fold-attribution-wire.test.ts (3 cases: inflated-X mid-history fold is attributed; a pure tail-append with a pending inflated-X fold is NOT credited; a client-side inbound rewrite stays client-side, not fold-credited).
  • REPORTING 3: 2 regression tests in tests/cache-ledger.test.ts.
  • REPORTING 4: tests/issue3307-seam-suspect-vs-client.test.ts.
  • Local: 32 targeted tests green + tsc --noEmit clean on this head. Full-suite verdict = remote CI.

Status

kernel-change: none (host-side only). Refs #3307 (not closing). The mechanism fix is implemented and unit-verified on synthetic bodies; final closure awaits the reporter's real-provider re-capture confirming the original 21 samples move out of ⚠ CACHE SEAM (the new Vraw/X logs make that a single grep). Issue stays open until that verification lands.

wire/upstream/restart already carried per-event data in the cache report
(switchEvents / restartEvents) but handleAcpCache only ever rendered their
aggregate counts in CACHE INVALIDATION — no per-line timestamps, so
seam-vs-restart cross-checks (e.g. #3416 273->28 vs the 10 restart/refork
events) were impossible to run off the report. Surface them exactly like the
existing MODEL/KEY/PROMPT/PROJECTION sections. Refs #3307.
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-16_acp-cache-wire-upstream-restart-per-line (fad2033)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4459-pr.3333.f22ae30

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3333.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 按 "acp-cache WIRE UPSTREAM RESTART switch sections render" 检索 open+closed issue/PR,无同类渲染工作;本 PR 正确引用父 issue #3307(不关闭,#3307 的第三道缝根因仍在等 reporter 证据)。

这是 [PR] 跟踪 issue,按协议核对代码 + CI,不重做。结论如下(八问):

  1. 是什么问题:/acp-cache 报告里 WIRE/UPSTREAM/RESTART 三类失效只有聚合计数、没有逐行时间戳,导致 [cache] Similar cache seams still detectable after #1548 / #1551 / #1554: v0.1.197, 21 samples / 1.14M tok | [cache] #1548 / #1551 / #1554 之后仍可测到同类缓存接缝:v0.1.197,21 样本 / 1.14M tok #3307 floor 3 建议的交叉核对("seam 样本 真模型 gate 盲区:#3401 带病全绿——补齐 DSH 插件车道 + pi/codex/claude/opencode 客户端车道 #3416 是不是 10 次 restart/refork 之一")做不了——而 model/key/prompt 切换却有逐行明细。
  2. 什么原因导致的:事件数据本来就算好了(switchEvents/restartEvents,ledger 测试一直在断言),只是 handleAcpCache 只把 MODEL/KEY/PROMPT/PROJECTION 接上了逐行渲染,这三个桶漏接了。纯渲染缺口,不是数据缺口。
  3. 影响范围与严重程度:仅诊断输出面;触发条件 = 会话真的发生过 wire/upstream 切换或 daemon 重启重计费。严重度 = 可观测性缺失(交叉核对要人肉对时间戳),无任何功能/计费行为影响。
  4. 怎么修复的:复用现成的 formatModelSwitches(三个桶都是同一 ModelSwitchStats 形状),各按 count > 0 门控插入到 CACHE INVALIDATION 之前——与既有 PROJECTION 段([Bug] Responses self-heal 切换投影规则后,提示词指纹变化被误归因为宿主改写 #2867)完全同构。
  5. 是否有回归问题:审过以下既有行为,均保持——① 桶为空时输出逐字节不变(门控同既有段落);② tail() 是纯追加 helper(stability+seam),无截断交互;③ 无 wire/message-shape、config、kernel、update.ts、阈值/时序改动;④ master 漂移核查:fix(opencode-native): proxy 不可用时超窗请求本地拒发,不再静默直连撞上游 400 #3318 虽与 master 领先 3 个提交,但两个触碰文件在 master 上与 PR base blob 完全一致(已比对),无冲突风险。结论:无回归。
  6. 是否破坏缓存:不涉及(未触碰缓存/重试/游标/去重逻辑)。
  7. 是否改了设计方向:不涉及;web 会话页对齐(src/web/client.ts)按 PR 描述刻意留作后续,不在本 PR 范围。
  8. 是否引入了新的配置:不涉及。

验证证据:

判定:代码可合(diff 干净、最小、同构扩展、测试有牙);CI 待 flake 重跑转绿后全绿。合并动作留给人工。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

重跑 #2 结果:还是 win22-1 红,但换了一个测试——抖动家族再添一员,已触发第三次重跑。

轮次 红分片 失败测试 签名
#1 (5bdc878) windows-latest, node22, shard 1 tests/issue1452-socket-hardening.test.ts:405 retired stall guard (#1706/#1714) first chunk should have flowed through, duration_ms ≈62.7s
#2 (重跑) windows-latest, node22, shard 1(其余全绿) tests/lineage-reconcile-boot.test.ts:468 AC12 multi-hop grandchild AssertionError notStrictEqual,expected true / actual true(断言"两者应不同",实际相同),duration_ms ≈6.4s

两轮红的分片完全相同、失败文件完全不同、且都与本 PR diff(cache-ledger 渲染 +38/-2)零交集;两个文件在同 head 的 ubuntu 各分片均为绿。本地定向复跑:stall guard 单测通过、AC12 单测通过(node --import tsx --test --test-name-pattern AC12 tests/lineage-reconcile-boot.test.ts → 1/1 pass)。

这与已关闭的 #3259(同一 stall-guard 测试同签名)、#3279(win node22 runner 活体 HTTP/fs 定时抖动,4 个测试文件跨 2 轮 CI,本地全绿)是同一环境抖动家族;lineage-reconcile-boot 是该家族新出现的一个成员。按惯例不另开 issue(#3259/#3279 已覆盖该家族且均已关闭,是否重新立项请你定)。

已触发第三次仅重跑失败 job,结果出来我再补一条结论。若三轮仍红,建议人工看一眼 win22-1 runner 或考虑给该分片加自动二次确认(#3279 建议方向 1)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

重跑 #3 全绿——CI 判定完成,本 PR 可合。

无阻塞项,等你合并。若后续 win22-1 再抖,建议考虑 #3279 里的方向 1(该分片自动二次确认)。

)

detectSeam pushes both the #3023 client-side rewrite and the legacy suspected-rebuild fall-through into one shared capped ring (led.seamEvents); formatSeam's CACHE SEAM section iterated every event without filtering, so a host-owned client rewrite was also listed under the 'suspected mid-history prefix breaks' header. List only non-client events there so the two sections stay disjoint.
@ranxianglei ranxianglei changed the title acp-cache: render per-line WIRE/UPSTREAM/RESTART switch sections (#3307) acp-cache report: per-line switch sections + disjoint suspect/client seam sections (#3307) Oct 16, 2026

@rahul05ranjan rahul05ranjan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against runtime — no node_modules on my machine, so I couldn't run the suite directly; instead I lifted formatSeam and formatModelSwitches verbatim off this branch into a standalone script and drove them with adversarial reports (12 assertions, all green), including an A/B against the pre-PR suspect loop.

Disjointness fix (src/cache-ledger.ts:1979-1986): reproduced both the bug and the fix. With a shared ring holding one client event (.client set) and one suspect event, the old loop listed both under ⚠ — exactly the double-listing described in the body — while the new filter((e) => !e.client) keeps each seq under exactly one header. I checked both directions: client seq absent from ⚠, suspect seq absent from ✎.

Edge worth knowing (not a blocker): the suspectEvents.length === 0 branch now also fires when suspect forensics were captured but later evicted from the shared 8-slot SEAM_EVENTS_CAP ring by a burst of client rewrites. The "retained" wording is honest so it reads fine, but a session with heavy client churn will show the ⚠ disclaimer even though body-pair evidence once existed. If that ever bites during #3307 triage, a separate suspect-only ring would fix it — fine as a follow-up.

Per-line sections (src/cache-ledger.ts:1919-1921): confirmed the events were already computed in the report builder (lines 1821-1823), so this is render-only as claimed. Checked: restart events render ? → (restart) (null from falls to "?"), summary caps at SWITCH_LIST_CAP with the … N earlier switch(es) omitted note, and all-zero buckets produce "" so a session without these events stays byte-identical. Ordering matches the sibling sections, and tail() at line 1906 is pure concatenation, so the new text can't get clipped.

tests/issue3307-seam-suspect-vs-client.test.ts driving both arms in one session is the right shape — it pins the exact invariant this restores.

…on forensics

#3307's unattributed re-pay survives even when every seam fold has a real
start_ref, so the V' fallback is not the trigger: decomposeSample drops the
fold when bestX = prefixTokensBeforeRef(start) >= billed input, zeroing
structuralExcess. That value is in-memory (tokenSnapshot) and never persisted,
so it cannot be reconstructed from the blocks table after the fact.

Emit one [acp-compress-obs] fold-attribution line per fold carrying the actual
X per block plus the same-space view tokens V/Vp. Cross-referenced against the
ledger's billed input it separates the two candidate inflation sources: unit
scale (kernel chars/4 estimate vs provider-billed tokens) vs view identity
(snapshot tracked on a larger view than the shrunken outbound). Pure logging;
no behavior change.
@ranxianglei ranxianglei changed the title acp-cache report: per-line switch sections + disjoint suspect/client seam sections (#3307) acp-cache report: per-line switch sections + disjoint seam sections + fold-attribution X logging (#3307) Oct 16, 2026
awork added 3 commits October 18, 2026 12:16
…end (#3307 P1)

#3307 P1: the observability denominator V used (compressMessages ?? messages), which in proxy mode is the client un-shrunken full-history resend -- ~50x the billed input for a stateless client (codex_webui). Every real fold read as a ~1% shrink and postCtx/ceiling/the degenerate-fold gate all ran off the wrong basis.

V now measures the LIVE context being managed (ctx.messages / processedMessages, what gets billed next); applyCompression still runs on compressMessages (raw boundaries, #2627) so fold geometry is unchanged. The fold-attribution log now reports V(live) vs Vraw(inbound) so one capture shows the collapse.

Behavior change (intended, disclosed): in stateless-client proxy sessions shrink% rises from the degenerate ~1% to its true value; the degenerate-fold gate (DEGENERATE_FOLD_COVERAGE=0.8) and the maxShrink advisory go from effectively-never-firing to firing correctly; postCtx and the next-request cache ceiling are computed from the live view. Unchanged when there is no separate raw resend (messages == compressMessages), which is what every existing unit test exercises.

Regression: tests/issue3307-fold-view-basis.test.ts pins V = live view across single/multi-level folds, verifies Vp === V - S (same-view net change), and keeps the messages==compressMessages baseline unchanged.
…nflated snapshot (#3307 P2-A)

#3307 P2-A: the snapshot-based fold prefix X (prefixTokensBeforeRef over the client un-shrunken resent history) exceeds the billed input for a stateless proxy client, so decomposeSample zeroes structuralExcess and the fold re-pay falls into the unattributed seam bucket. When a genuine bili-side mid-history break is present, measure the stable prefix off the wire (byte-LCP of this outbound vs the previous, scaled to billed input) and min() it into X before decomposeSample, so the fold is credited again instead of crying a false seam.

Gated to mid-list breaks only (a tail append lands the LCP near the end -> xMeasured ~= input -> no effect) and never on a client-side inbound rewrite (#3023), whose earlier divergence would understate and mis-credit the fold. Host-side only -- no kernel change; decomposeSample is called as-is with the corrected firstFoldStartTokens.

Regression: tests/issue3307-fold-attribution-wire.test.ts pins attribution for the inflated-X mid-history fold plus both guards (tail-append not credited; client rewrite stays client-side, not fold).
@ranxianglei ranxianglei changed the title acp-cache report: per-line switch sections + disjoint seam sections + fold-attribution X logging (#3307) [PR] acp-cache report: per-line switch sections + disjoint seam sections + fold-attribution X logging (#3307) Oct 19, 2026
…shrank it) (#3307)

The multi-level P1 test held the live view constant across folds, so it did not cover the re-fold topology the #3307 GO called out (a fold whose live view already embeds an earlier summary and is smaller than the raw resend). Add a case that shrinks ctx.messages after fold-1 and asserts fold-2 V tracks that current view, never the raw resend or a stale snapshot.
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4572(master 合并,merge commit 保留原作者署名)。本 PR 关单收尾;回滚点为 #4572 单提交 revert。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants