Repository navigation
[PR] acp-cache report: per-line switch sections + disjoint seam sections + fold-attribution X logging (#3307) - #3333
ranxianglei wants to merge 7 commits into
Conversation
wire/upstream/restart already carried per-event data in the cache report (switchEvents / restartEvents) but handleAcpCache only ever rendered their aggregate counts in CACHE INVALIDATION — no per-line timestamps, so seam-vs-restart cross-checks (e.g. #3416 273->28 vs the 10 restart/refork events) were impossible to run off the report. Surface them exactly like the existing MODEL/KEY/PROMPT/PROJECTION sections. Refs #3307.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4459-pr.3333.f22ae30This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3333.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 按 "acp-cache WIRE UPSTREAM RESTART switch sections render" 检索 open+closed issue/PR,无同类渲染工作;本 PR 正确引用父 issue #3307(不关闭,#3307 的第三道缝根因仍在等 reporter 证据)。 这是 [PR] 跟踪 issue,按协议核对代码 + CI,不重做。结论如下(八问):
验证证据:
判定:代码可合(diff 干净、最小、同构扩展、测试有牙);CI 待 flake 重跑转绿后全绿。合并动作留给人工。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 重跑 #2 结果:还是 win22-1 红,但换了一个测试——抖动家族再添一员,已触发第三次重跑。
两轮红的分片完全相同、失败文件完全不同、且都与本 PR diff(cache-ledger 渲染 +38/-2)零交集;两个文件在同 head 的 ubuntu 各分片均为绿。本地定向复跑: 这与已关闭的 #3259(同一 stall-guard 测试同签名)、#3279(win node22 runner 活体 HTTP/fs 定时抖动,4 个测试文件跨 2 轮 CI,本地全绿)是同一环境抖动家族;lineage-reconcile-boot 是该家族新出现的一个成员。按惯例不另开 issue(#3259/#3279 已覆盖该家族且均已关闭,是否重新立项请你定)。 已触发第三次仅重跑失败 job,结果出来我再补一条结论。若三轮仍红,建议人工看一眼 win22-1 runner 或考虑给该分片加自动二次确认(#3279 建议方向 1)。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 重跑 #3 全绿——CI 判定完成,本 PR 可合。
无阻塞项,等你合并。若后续 win22-1 再抖,建议考虑 #3279 里的方向 1(该分片自动二次确认)。 |
) detectSeam pushes both the #3023 client-side rewrite and the legacy suspected-rebuild fall-through into one shared capped ring (led.seamEvents); formatSeam's CACHE SEAM section iterated every event without filtering, so a host-owned client rewrite was also listed under the 'suspected mid-history prefix breaks' header. List only non-client events there so the two sections stay disjoint.
rahul05ranjan
left a comment
There was a problem hiding this comment.
Verified against runtime — no node_modules on my machine, so I couldn't run the suite directly; instead I lifted formatSeam and formatModelSwitches verbatim off this branch into a standalone script and drove them with adversarial reports (12 assertions, all green), including an A/B against the pre-PR suspect loop.
Disjointness fix (src/cache-ledger.ts:1979-1986): reproduced both the bug and the fix. With a shared ring holding one client event (.client set) and one suspect event, the old loop listed both under ⚠ — exactly the double-listing described in the body — while the new filter((e) => !e.client) keeps each seq under exactly one header. I checked both directions: client seq absent from ⚠, suspect seq absent from ✎.
Edge worth knowing (not a blocker): the suspectEvents.length === 0 branch now also fires when suspect forensics were captured but later evicted from the shared 8-slot SEAM_EVENTS_CAP ring by a burst of client rewrites. The "retained" wording is honest so it reads fine, but a session with heavy client churn will show the ⚠ disclaimer even though body-pair evidence once existed. If that ever bites during #3307 triage, a separate suspect-only ring would fix it — fine as a follow-up.
Per-line sections (src/cache-ledger.ts:1919-1921): confirmed the events were already computed in the report builder (lines 1821-1823), so this is render-only as claimed. Checked: restart events render ? → (restart) (null from falls to "?"), summary caps at SWITCH_LIST_CAP with the … N earlier switch(es) omitted note, and all-zero buckets produce "" so a session without these events stays byte-identical. Ordering matches the sibling sections, and tail() at line 1906 is pure concatenation, so the new text can't get clipped.
tests/issue3307-seam-suspect-vs-client.test.ts driving both arms in one session is the right shape — it pins the exact invariant this restores.
…on forensics #3307's unattributed re-pay survives even when every seam fold has a real start_ref, so the V' fallback is not the trigger: decomposeSample drops the fold when bestX = prefixTokensBeforeRef(start) >= billed input, zeroing structuralExcess. That value is in-memory (tokenSnapshot) and never persisted, so it cannot be reconstructed from the blocks table after the fact. Emit one [acp-compress-obs] fold-attribution line per fold carrying the actual X per block plus the same-space view tokens V/Vp. Cross-referenced against the ledger's billed input it separates the two candidate inflation sources: unit scale (kernel chars/4 estimate vs provider-billed tokens) vs view identity (snapshot tracked on a larger view than the shrunken outbound). Pure logging; no behavior change.
…view-factor forensics
…end (#3307 P1) #3307 P1: the observability denominator V used (compressMessages ?? messages), which in proxy mode is the client un-shrunken full-history resend -- ~50x the billed input for a stateless client (codex_webui). Every real fold read as a ~1% shrink and postCtx/ceiling/the degenerate-fold gate all ran off the wrong basis. V now measures the LIVE context being managed (ctx.messages / processedMessages, what gets billed next); applyCompression still runs on compressMessages (raw boundaries, #2627) so fold geometry is unchanged. The fold-attribution log now reports V(live) vs Vraw(inbound) so one capture shows the collapse. Behavior change (intended, disclosed): in stateless-client proxy sessions shrink% rises from the degenerate ~1% to its true value; the degenerate-fold gate (DEGENERATE_FOLD_COVERAGE=0.8) and the maxShrink advisory go from effectively-never-firing to firing correctly; postCtx and the next-request cache ceiling are computed from the live view. Unchanged when there is no separate raw resend (messages == compressMessages), which is what every existing unit test exercises. Regression: tests/issue3307-fold-view-basis.test.ts pins V = live view across single/multi-level folds, verifies Vp === V - S (same-view net change), and keeps the messages==compressMessages baseline unchanged.
…nflated snapshot (#3307 P2-A) #3307 P2-A: the snapshot-based fold prefix X (prefixTokensBeforeRef over the client un-shrunken resent history) exceeds the billed input for a stateless proxy client, so decomposeSample zeroes structuralExcess and the fold re-pay falls into the unattributed seam bucket. When a genuine bili-side mid-history break is present, measure the stable prefix off the wire (byte-LCP of this outbound vs the previous, scaled to billed input) and min() it into X before decomposeSample, so the fold is credited again instead of crying a false seam. Gated to mid-list breaks only (a tail append lands the LCP near the end -> xMeasured ~= input -> no effect) and never on a client-side inbound rewrite (#3023), whose earlier divergence would understate and mis-credit the fold. Host-side only -- no kernel change; decomposeSample is called as-is with the corrected firstFoldStartTokens. Regression: tests/issue3307-fold-attribution-wire.test.ts pins attribution for the inflated-X mid-history fold plus both guards (tail-append not credited; client rewrite stays client-side, not fold).
…shrank it) (#3307) The multi-level P1 test held the live view constant across folds, so it did not cover the re-fold topology the #3307 GO called out (a fold whose live view already embeds an earlier summary and is smaller than the raw resend). Add a case that shrinks ctx.messages after fold-1 and asserts fold-2 V tracks that current view, never the raw resend or a stale snapshot.
What
Two fixes plus three reporting/observability improvements, all on the same #3307 surface (acp-cache attribution + report rendering). No wire/message-shape change, no config key, no kernel change, no
update.tstouch.Root cause (shared by both fixes)
A stateless proxy client (e.g.
codex_webuioverresponses) resends its FULL history every turn, and bili rebuilds the outbound by collapsing folded ranges into summaries. Two quantities were being measured over the client's un-shrunken inbound view instead of the compressed/billed view:firstFoldStartTokens(prefixTokensBeforeRef(start_ref)over the in-memorytokenSnapshot, which is built on the same un-shrunk inbound) → lands ≥ billed input →decomposeSamplezeroesstructuralExcess→ the fold's re-pay becomes unattributed → it falls into the ⚠ CACHE SEAM bucket as a false "suspected mid-history prefix break".FIX 1 (P1) — measure fold-shrink V over the live view (
e31661f74)src/stream.ts: the fold-shrink denominator V now countsctx.messages(the live/processed view = the billed basis) instead ofctx.compressMessages ?? ctx.messages(the raw resend).applyCompressionstill runs on the raw history (fold geometry must map to raw message boundaries, #2627) — only the observability denominator changed. This aligns with #1911's own stated intent ("V is the pre-fold LIVE view size"); the bug was that production's??preferred the bloated raw over the correct live view.FIX 2 (P2-A) — attribute fold re-pay via the wire-measured prefix (
da6834ee9)src/cache-ledger.ts recordCacheSample: when a genuine bili-side mid-history break is present (substantive residual, both outbound bodies captured & uncapped, not a client-side inbound rewrite #3023, mid-list shape), the stable prefix is measured off the wire — byte-LCP of this outbound vs the previous, scaled to billed input — andmin()'d into X beforedecomposeSample, so the fold is credited again instead of crying a false seam. Host-side only;decomposeSampleis called as-is with the correctedfirstFoldStartTokens. Guards: a pure tail-append is excluded (LCP lands near the end →xMeasured ≈ input→ no effect); a client-side inbound rewrite is excluded (its earlier divergence would understate and mis-credit the fold).REPORTING 3 — per-line WIRE / UPSTREAM / RESTART sections (
5bdc878a6)handleAcpCachenow renders per-line sections for three invalidation buckets that already carried per-event data but were shown only as aggregate counts inCACHE INVALIDATION:WIRE SWITCHES,UPSTREAM SWITCHES,RESTART/REFORK. Each is gated oncount > 0exactly like the existing MODEL / KEY / PROMPT / PROJECTION sections, so a session that never hits one stays byte-identical. Lets the floor-3 cross-check ("is seam sample #3416 one of the 10 restart/refork events?") run straight off the report.REPORTING 4 — disjoint ⚠ suspected-seam vs ✎ client-rewrite sections (
4db43b05b)detectSeampushes BOTH the #3023 client-side mid-history rewrite AND the legacy suspected-rebuild fall-through into ONE shared capped ring (led.seamEvents).formatSeam's ⚠ section iterated every event without filtering, so a host-owned client rewrite was also listed under the "suspected" header (double-listed) — contaminating suspect triage. The section now lists only non-client events; the ✎ section (already filtered toe.client) is unchanged. Counters were always clean (seamSuspectsvsclientInserts); only the per-line listing leaked. Web session page already distinguishes them inline, so it was unaffected.REPORTING 5 — fold-attribution X / V / Vraw / Vp log (
cfb2d31a0,3a1541c32)One
[acp-compress-obs] fold-attribution X(...) · V · Vraw · Vpline per fold carrying the actual X per new block plus the live-view vs raw-inbound totals. Cross-referenced against the ledger's billed input it separates the two inflation sources (unit scale vs view identity) in a single capture. Pure logging.Behavior changes (disclosed)
DEGENERATE_FOLD_COVERAGE = 0.8) and the maxShrink advisory go from effectively-never-firing to firing correctly;postCtx+ the next-request cache ceiling are now computed from the live view. Unchanged whenevermessages == compressMessages(which is exactly what every pre-existing unit test exercises).remainder; no longer flagged as a seam). Gated to non-client mid-list breaks only.Tests
tests/issue3307-fold-view-basis.test.ts(5 cases: V tracks live-not-raw;Vp === V − Ssame-view net change; multi-level folding keeps tracking the live view; re-folding tracks the already-shrunk live view; baselinemessages == compressMessagesunchanged).tests/issue3307-fold-attribution-wire.test.ts(3 cases: inflated-X mid-history fold is attributed; a pure tail-append with a pending inflated-X fold is NOT credited; a client-side inbound rewrite stays client-side, not fold-credited).tests/cache-ledger.test.ts.tests/issue3307-seam-suspect-vs-client.test.ts.tsc --noEmitclean on this head. Full-suite verdict = remote CI.Status
kernel-change:none (host-side only). Refs #3307 (not closing). The mechanism fix is implemented and unit-verified on synthetic bodies; final closure awaits the reporter's real-provider re-capture confirming the original 21 samples move out of ⚠ CACHE SEAM (the newVraw/Xlogs make that a single grep). Issue stays open until that verification lands.