Skip to content

fix(cache): quarantine unmeasured cache reports + attribute invalidation causes (P0+P1, CLI) - #1545

Closed
ranxianglei wants to merge 2 commits into
2026-09-28_acp-cache-model-switch-attributionfrom
2026-09-28_cache-attribution-p0-p1-cli
Closed

ranxianglei wants to merge 2 commits into
2026-09-28_acp-cache-model-switch-attributionfrom
2026-09-28_cache-attribution-p0-p1-cli

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Follow-up to #1536 (compression cache hit-rate collapse). Implements the owner's scoped plan: P0 fix first → add attribution for what can't be fixed → CLI first, Web UI as a separate change. Generalizes and extends #1535's model-switch attribution.

⚠️ Stacked on #1540 — base is 2026-09-28_acp-cache-model-switch-attribution, not master. Merge #1540 first; this diff shows only the delta on top of it.

P0 — stop conflating unmeasured with 0% (measurement correctness)

Providers that omit cache-hit tokens (cachedTokens === undefined) were booked as cached=0, so their whole billed prefix landed in the kernel's unexplained ttlRepay residual and the hit rate read 0% even when nothing was wrong. Such samples are now quarantined out of the closure totals (own unmeasuredSamples / unmeasuredInput counters), dropped from the rendered line set, and excluded from the false-positive cache-collapse warning. The kernel identity still closes exactly over the measurable subset.

P1 — attribute the non-fixable causes (generalizing #1535)

#1535 added a model-switch bucket. This generalizes it to a full target identity and adds the other structural-invalidation causes found in the #1536 diagnosis:

  • wire-protocol switch (e.g. anthropic ↔ openai-completions relay)
  • upstream-origin switch (rotating free relays busts the prefix even for the same model + wire)
  • proxy-restart / refork (#499) via a process boot-id marker

Each cause charges only its sample's stable-prefix residual (tr), so compRepay stays booked to its fold and the kernel closure never breaks. Upstream TTL/eviction stays the single remaining residual (deliberately not over-split into unmeasurable sub-causes).

Surface (CLI-first, per owner decision)

  • /acp-cache text gains a CACHE INVALIDATION breakdown section.
  • __bili/cache-report JSON gains wireSwitches / upstreamSwitches / restartDrops / unmeasured / invalidation.
  • Web UI is deferred to a separate PR (owner: "web 的单独拆分").

Behavior changes (disclosed)

  • [acp-usage] now logs (no cache report) instead of cached=0 (cache hit 0%) when a provider omits cache tokens (known samples unchanged).
  • warnCacheCollapse no longer fires for non-reporting providers (removes a false-positive collapse alert that was part of the same conflation).

Verification

  • npm run typecheck ✓
  • npm test → 3040 pass / 0 fail (12 skipped = gated e2e); +5 regression tests in tests/cache-ledger.test.ts (unknown quarantine, wire switch, upstream switch, restart, CLI breakdown)
  • npm run build ✓
  • Existing tests unaffected (they pass numeric cached; single-process so no false restart flag; feat: attribute model-switch cache drops in acp-cache report and web UI #1540's MODEL SWITCHES text unchanged)

Fixes #1536

…attribute invalidation causes

#1536: the ledger booked every sample whose provider omitted cache-hit
tokens as cached=0, so a whole billed prefix landed in the unexplained
ttlRepay residual and the hit rate read 0% even though nothing was wrong.

P0 (measurement correctness): core.ts recordUsage normalizes undefined
cachedTokens to null; cache-ledger.ts quarantines such samples out of the
closure totals (own unknownSamples/unknownInput counters), drops them from
the rendered line set, and skips the false-positive cache-collapse warning.
The kernel identity still closes exactly over the measurable subset.

P1 (attribution for the non-fixable causes), generalizing #1535's
model-switch bucket to a full target identity (model | wire protocol |
upstream origin) plus a proxy-restart/refork bucket (#499) via a process
boot-id marker. Each cause charges only its sample's stable-prefix residual
(tr) so compRepay stays booked to its fold and the closure never breaks.

CLI (/acp-cache) gains a CACHE INVALIDATION breakdown section; the
__bili/cache-report JSON gains wireSwitches/upstreamSwitches/restartDrops/
unmeasured/invalidation. Web UI is intentionally deferred to a separate PR.

Behavior changes (disclosed):
- [acp-usage] now logs "(no cache report)" instead of "cached=0 (cache hit
  0%)" when the provider omits cache tokens (known samples unchanged).
- warnCacheCollapse no longer fires for non-reporting providers (removes a
  false-positive collapse alert that was part of the same conflation).

Tests: tests/cache-ledger.test.ts +5 regression tests (unknown quarantine,
wire switch, upstream switch, restart, CLI breakdown). typecheck + full
suite (3040 pass) + build green on this branch.
…lugin-pipe usage path (#1536)

Review finding on this PR: the ledger has two entry points and only one was fixed. applyUsageSample (src/plugin.ts) — used by every plugin-mode pipe (pipePluginChatWithStrip / pipePluginResponsesWithStrip / pipePluginJson, the prepared.pluginMode branch of forward()) — still booked cachedTokens ?? 0, fed warnCacheCollapse with 0 for non-reporting providers, and passed no protocol/upstream identity. So P0 conflation and false-positive collapse warnings stayed live for all ACP-native agent sessions, and P1 wire/upstream attribution was blind there (only model + restart buckets worked).

- applyUsageSample: normalize undefined -> null (mirrors recordUsage in loop/core.ts), gate warnCacheCollapse on a real report, pass cached/protocol/upstreamOrigin into recordCacheSample; log marker unified to '(no cache report)' across both paths (was 'cached=n/a' in plugin mode).
- pipes take an optional trailing upstreamOrigin and forward it.
- server.ts: hoist the fetched-origin computation above the plugin branch (targetOrigin) and share it between the three session-carrying pipe calls and the compress-loop ctx (removes the shadowing local).
- tests: 3 regression tests (quarantine closure over the measurable subset, collapse-watch suppression, wire+upstream buckets via the plugin path); update the #695 log-text test to the unified marker.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewed #1545 against its stacked base (#1540 branch) — direction, regression surface, code, tests, diff. Found one real gap in the PR's own coverage, fixed it directly on the branch (commit 118209b3), and filed one separate issue for a pre-existing adjacent gap (#1547).

① Original issue (#1536) — direction ✓. The scoped plan (P0 measurement correctness → P1 attribution for the non-fixable causes → CLI first, Web deferred, stacked on #1540) matches the owner decision in #1536 verbatim. P0 root cause correctly identified and fixed at the producer: cachedTokens === undefined booked as 0 made the whole billed prefix land in the unexplained ttlRepay residual → false 0% hit rate; now normalized to null at the sample source instead of filtering symptoms downstream.

② Regression audit — behaviors touched, verdicts:

  • [acp-usage] loop-path log: known samples byte-identical; non-reporting providers print (no cache report) instead of cached=0 (cache hit 0%) — disclosed ✓
  • warnCacheCollapse state machine (sawHigh/lowRun/warned): gating the call on a real report leaves state consistent — fewer observations, no corruption ✓
  • Kernel closure identity: totals now close over the measurable subset only; residual === 0 / balanced === true still exact (regression test pins it). Intended change, disclosed: hit rate no longer diluted by unmeasurable input ✓
  • linesOmitted semantics unchanged; unknown lines stay in the store, filtered only at render ✓
  • feat: attribute model-switch cache drops in acp-cache report and web UI #1540's MODEL SWITCHES section text untouched ✓
  • stats.cachedTokens / cacheSamples accumulation unchanged on both paths ✓
  • Pipe signature changes are optional trailing params — all ~121 existing test call sites compile unchanged; session-less call sites (title-gen / side passthrough) untouched, accounting stays off there ✓

③ Code — one gap found, fixed on branch (118209b3). The ledger has two entry points; the PR fixed only one. applyUsageSample (src/plugin.ts) — used by every plugin-mode pipe (pipePluginChatWithStrip / pipePluginResponsesWithStrip / pipePluginJson, i.e. the prepared.pluginMode branch of forward(), which early-returns before the compress loop) — still booked cachedTokens ?? 0, fed warnCacheCollapse with 0, and passed no protocol/upstream identity. Consequence: for all ACP-native agent sessions (bili opencode / pi / dsh …) the P0 conflation and the false-positive collapse warning stayed live, and wire/upstream attribution was blind there (only model + restart buckets worked). Fixed: undefined→null normalization mirroring recordUsage in loop/core.ts, collapse-watch gated on a real report, protocol + upstreamOrigin plumbed through the three pipes (optional trailing param) from a single targetOrigin computed once in forward() — this also removed the shadowing let upstreamOrigin local in the streaming rewriter branch.

④ Test validity. The five new tests in the PR reproduce their bugs against the old code (teeth confirmed). My fix adds three more, each failing on the pre-fix path: quarantine closure over the measurable subset via the plugin pipe, collapse-watch suppression for non-reporting providers, wire+upstream buckets populated through applyUsageSample. Local verification on the pushed head: typecheck ✓, 3043 pass / 0 fail (12 skipped = gated e2e), build ✓. GitHub e2e suites were green on the previous head; they re-run on 118209b3.

⑤ Diff cleanliness. Original delta tight and single-concern; my fix commit touches only the gap (src/plugin.ts, src/server.ts, tests/usage-observability.test.ts) and is clearly labeled.

Behavior changes introduced by my fix commit (disclosed):

  • Plugin-mode [acp-usage] log: cached=n/a → (no cache report) — unified marker across both paths for the same condition (old plugin wording was honest but divergent from the loop path).
  • Plugin-mode ledger samples now carry protocol/upstream: wire/upstream invalidation buckets become active for agent sessions (additive signal; previously always 0 there).

New issue filed (pre-existing, separate root cause): #1547 — the non-streaming rewriter branch of forward() updates session.stats directly but never calls recordCacheSample, so stream:false sessions produce zero ledger lines and are blind to /acp-cache + the collapse diagnostics (predates this PR; nudge triggering unaffected). Repro + suggested fix inside.

Verdict: mergeable once #1540 lands (base is that branch — merge order matters). Merging stays human-side.

中文摘要:审查了 #1545(P0 隔离未上报样本 + 失效归因),发现并直接在分支上修复了一个真实缺口——插件模式(ACP-native agent)的用量入口 applyUsageSample 仍把未上报的缓存记为 0、且不带 wire/upstream 身份,导致 P0/P1 对所有 agent 会话失效;已补归一化、告警门控与参数透传(+3 个回归测试,本地 3043 全绿),并为"非流式路径从不写 cache ledger"这一既有独立问题单开了 #1547;#1540 合入后本 PR 可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Follow-up on the pushed fix head 118209b3:

  • e2e suites: 5/5 green (codex / opencode-native / dsh / pi lanes + hermetic registry).
  • One CI note so nobody waits on it: the main CI job (typecheck/test/build, ubuntu+windows × node 22/24) is triggered by pull_request: branches: [master] only — since this PR's base is feat: attribute model-switch cache drops in acp-cache report and web UI #1540's branch, it has never run here and will run once the base flips to master after feat: attribute model-switch cache drops in acp-cache report and web UI #1540 merges. That gap is covered by local verification on the same head: typecheck ✓, full suite 3043 pass / 0 fail ✓, build ✓ (the diff is pure TS logic with no OS-specific surface, so the windows/node-24 matrix adds little risk beyond what those runs will confirm later anyway).

Nothing else outstanding from this review round.

@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review: ✅ mergeable — this is the load-bearing layer of the stack, and the math holds.

What I verified (the two questions that matter)

  1. Closure only over the measurable subset. Unknown samples (cached === null) are decomposed with cached=0 but every derived bucket (missed/nc/cr/tr) is forced to 0 and excluded from agg input/cached — so the kernel identity input = cached + nc + cr + tr + residual stays exact over what IS measured (r.totals.residual === 0 pinned). A non-reporting provider no longer books its whole billed prefix as a fake 0%-hit ttlRepay residual.
  2. Buckets don't leak. model/wire/upstream/restart each charge only this sample's tr; attributedMissed is the union (charged once, never twice), so invalidation.remaining = tr − attributedMissed stays ≥ 0. Overlapping switches display in both buckets by design and say so.
  • Fold-cursor known-gate: an unmeasured sample between fold and next measured sample advances neither the cursor nor the owner's T — I found this gate had NO test pinning it (mutation M2: 0 red), so I added one and pushed it to the stack head (test(cache): pin the fold-cursor known-gate…, now red-under-mutation / green clean).
  • settleUsageReport is the single shared path; all three response shapes (loop SSE, plugin pipes, non-stream) settle through it and can no longer drift per shape.

Non-blocking observations

  • restarted swallows silently if the FIRST post-restart sample is unmeasured (lastBoot updates unconditionally). The re-bill then lands in remaining instead of restart — attribution accuracy only, rare combo, acceptable.
  • agg.requests counts unmeasured samples while token sums exclude them — informational field, intentional.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Verified your deep-review points against the stack head, independently. One clarification up front: the commit you pushed (test(cache): pin the fold-cursor known-gate…, tip 1e04ba03) and settleUsageReport live on branch 2026-09-28_nonstream-cache-ledger = open PR #1550 (base = this PR's branch), not on #1545's head — which is still my fix commit 118209b3. So the stack is #1540 → #1545 → #1550, and #1545's own diff is untouched by your push.

Verified at stack head 1e04ba03:

  • settleUsageReport really is the single shared path — recordUsage (loop SSE), applyUsageSample (plugin pipes) and the non-streaming rewriter arm of forward() all settle through it. I audited the two migrated call sites line-by-line for behavioral equivalence: stat-update gates, the total > 0 || reportedCached !== null sample gate, and the compress-credit netting are preserved exactly (including the plugin edge where inputTokens === undefined ⇒ promptInputTotal returns 0, so the old cachedTokens !== undefined && total > 0 block could never have fired without the input gate either). Intentional deltas, all parity-direction:
    • non-stream zero/negative-total reports no longer clobber lastInputTokens or retire overflowArmTokens ([Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793 parity; pinned by your unit test);
    • non-stream outputTokens now accrues even when the input total is absent (previously dropped inside the typeof total === "number" guard — degenerate case only);
    • non-stream turns now emit the [acp-usage] log line and feed warnCacheCollapse (previously never watched there) — gated on reportedCached !== null, so no false positives for non-reporting providers.
  • Fold-cursor pin confirmed: tests/cache-ledger.test.ts "unknown-cache sample must not eat a pending fold" — asserts the unknown line is quarantined out of the rendered set while line 3 keeps foldSeq=1 and the owner's T > 0; the mutation comment names exactly the red variant (drop known &&). Matches your M2 finding.
  • Your two non-blocking observations stand as assessed: first-post-restart-unmeasured swallows into remaining, and agg.requests counting unmeasured samples — both acceptable, no action taken.

Local pre-flight at 1e04ba03: typecheck ✓ · full suite 3048 pass / 0 fail / 12 skipped (+5 tests over #1545's head) · build ✓. GitHub e2e suites on that head: 5/5 green; PR #1550 reports mergeable: true. (Main CI job still waits on the base flipping to master, same trigger semantics as noted before.)

#1547 status: left open — #1550's body carries Closes #1547, so it will close itself when that PR lands. No separate closure needed.

Merge order when you're ready: #1540 → #1545 → #1550.

中文摘要:核对了你推的 fold-cursor 测试与 settleUsageReport 共享结算路径(在 #1550 分支上,非本 PR),逐行验证了两个迁移调用点的行为等价性与三处有意的 parity 变更,本地 typecheck/3048 项测试/build 全绿、e2e 5/5,#1547 由 #1550 合并时自动关闭,三个 PR 按 #1540→#1545→#1550 顺序可合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Closed as superseded by the combined PR #1558 (issue #1557) — the three were mutually dependent (#1545 references #1540's attribution symbols 20×) and are now one reviewable unit. All semantics preserved; review comments carry over. Web UI portion split out for ework (tracked in #1557).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant