Skip to content

feat: combined cache attribution stack — model-switch drops, unmeasured quarantine, non-streaming settle (#1557) - #1558

Merged
ranxianglei merged 8 commits into
masterfrom
2026-09-28_cache-attribution-combined
Sep 28, 2026
Merged

ranxianglei merged 8 commits into
masterfrom
2026-09-28_cache-attribution-combined

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #1557

Supersedes #1540 / #1545 / #1550 (mutually dependent stack — #1545 references #1540's attribution symbols 20×; three-in-one here per owner decision). Web UI portion split out for a separate PR by ework (tracked in #1557).

What's inside (semantics unchanged from the three originals, all deep-reviewed)

  1. Model-switch attribution (sw line flag, both-sides-known gate, A→B→A, tr-only charging, legacy ledger normalization) — from feat: attribute model-switch cache drops in acp-cache report and web UI #1540.
  2. P0 unmeasured quarantine + P1 invalidation buckets (identity closure exact over the measurable subset, pinned residual===0; wire/upstream/restart buckets, union-once, remaining >= 0) — from fix(cache): quarantine unmeasured cache reports + attribute invalidation causes (P0+P1, CLI) #1545.
  3. Non-streaming ledger settle (single shared settle path across loop-SSE / plugin pipe / stream:false; atomic; no double count) + fold-cursor pinning test (unknown sample between fold and measured sample must not eat compRepay attribution) — from fix(cache): settle non-streaming rewriter turns into the cache ledger (#1547) #1550.

Review provenance

Verification (head 3dc1358)

  • typecheck ✓ · cache-ledger 30/30 · issue1547-nonstream 4/4 · usage-observability 7/7 · web-sessions (master state) 6/6
  • Remote CI runs full suite.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-28_cache-attribution-combined (fec8a2d)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1520-pr.1558.bf4c78b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1558.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到。这是已完成的合并 PR(取代 #1540/#1545/#1550),我不重做,按 review 流程核验:拉分支核对代码、查 CI、跑定向测试,结论稍后回帖。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — verified head 3dc1358 (branch 2026-09-28_cache-attribution-combined, base master 13a00139, fresh). Verdict: mergeable (human merge). Details:

① Issue direction (#1557 = #1535 + #1536 + #1547)

All three root causes are fixed at the source, not masked:

② Regression audit (existing behavior on touched paths)

  • recordCacheSample has no remaining external callers — all three stats+ledger sites route through settleUsageReport; equivalence verified path-by-path (gate total>0 || reportedCached!==null ≡ old total>0; outputTokens still unconditional; compressCredit netting preserved).
  • Persistence: ledger stays v=1; new line/agg fields are optional-additive; getCacheLedger normalizes legacy ledgers on load (missing agg keys → 0). Old sessions load NaN-free (tested).
  • Stream/plugin wire path: byte-identical for reporting providers; only log wording changes when a provider omits cache tokens.
  • linesOmitted math safe: led.lines is never trimmed in storage (only the render cap applies).
  • No config-surface, session-identity, updater, or acp-kernel-pin changes.

Behavior changes (old → new, intentional)

  1. Unmeasured samples: booked as 0%-hit input → quarantined (unk lines excluded from aggregates, closure totals, hitPct seeding). Why: P0 accounting distortion (SSH 端口转发使用不同本地端口时 Web UI 被 Origin 校验拒绝 #1536).
  2. warnCacheCollapse: fired for non-reporting providers (cached treated as 0) → skipped when no cache data (all three paths). Why: false-positive collapse alarms.
  3. [acp-usage] log: cached=0 (cache hit 0%) → (no cache report) when provider omits cache tokens. Log-only.
  4. Non-streaming JSON turns: stats-only (invisible to /acp-cache) → now produce ledger samples; additionally zero-total usage reports no longer retire overflowArmTokens / reset lastInputTokens — aligns with the [Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793 guard already present in the stream/plugin paths. Why: single settle semantics (cache ledger: non-streaming rewriter path never records usage samples (ledger blind for stream:false sessions) #1547).
  5. /stats payload: gains per-session lastModel / modelSwitches / switchMissedTokens (additive JSON; consumers ignore unknown fields).
  6. /acp-cache report: gains MODEL SWITCHES + CACHE INVALIDATION sections.

③ Code

Clean. Restart detection via per-process BOOT_ID is honestly best-effort; identity flags gate on both-sides-known so A→B→A round-trips and legacy lines can't misattribute; remaining = max(0, tr − attributed) keeps residuals non-negative; union-once prevents double-charging across cause buckets. No as any / @ts-ignore; logging goes through the proper channels.

④ Test validity

Toothed, not tautological: pinned residual===0 against exact aggregate math under heavy sampling; the fold-cursor pinning test nails "unknown sample between fold and measured sample must not eat compRepay"; e2e non-streaming settle through a live server; legacy-normalization regression test; collapse-watch gating test. Each new test reproduces its original bug.

⑤ Diff cleanliness

8 files, every line on-purpose. One minor note: history carries one base-update merge commit (a30ee8d1) — cosmetic, harmless if merged squash/rebase. The README sentence about the web column ships in the split-out PR below, so no doc rot.

Verification (head 3dc1358)

  • Local: typecheck ✓ · full suite 3065 tests — 3053 pass / 0 fail / 12 skip · targeted 41/41 (matches your claim)
  • Remote CI on head: all 9 workflow runs green (full test matrix incl. Windows, E2E codex/pi/opencode/dsh, hermetic registry)

Web UI follow-up (owner-assigned split from #1557)

Delivered: PR #1561 — model-switch column in both session tables, branched off this branch's head 3dc1358f (stacked on this PR). It's the web hunks of #1540 applied onto the combined stack with zero drift (verified src/web/ + tests/web-sessions.test.ts identical between #1540's branch point and this head). Local: typecheck ✓, web-sessions 7/7, full suite 3066 tests / 0 fail. After this PR merges, #1561 needs its base re-targeted to master.


中文摘要:核验了本合并 PR 的完整代码与本地/远端测试——三处缓存归因修复均打在根因上、无意外回归、六项行为变更已逐条披露(旧值→新值),可以合并;Web UI 部分已按 owner 指示拆为独立 PR #1561(基于本分支头)。

ework-agent and others added 8 commits September 28, 2026 19:40
…attribute invalidation causes

#1536: the ledger booked every sample whose provider omitted cache-hit
tokens as cached=0, so a whole billed prefix landed in the unexplained
ttlRepay residual and the hit rate read 0% even though nothing was wrong.

P0 (measurement correctness): core.ts recordUsage normalizes undefined
cachedTokens to null; cache-ledger.ts quarantines such samples out of the
closure totals (own unknownSamples/unknownInput counters), drops them from
the rendered line set, and skips the false-positive cache-collapse warning.
The kernel identity still closes exactly over the measurable subset.

P1 (attribution for the non-fixable causes), generalizing #1535's
model-switch bucket to a full target identity (model | wire protocol |
upstream origin) plus a proxy-restart/refork bucket (#499) via a process
boot-id marker. Each cause charges only its sample's stable-prefix residual
(tr) so compRepay stays booked to its fold and the closure never breaks.

CLI (/acp-cache) gains a CACHE INVALIDATION breakdown section; the
__bili/cache-report JSON gains wireSwitches/upstreamSwitches/restartDrops/
unmeasured/invalidation. Web UI is intentionally deferred to a separate PR.

Behavior changes (disclosed):
- [acp-usage] now logs "(no cache report)" instead of "cached=0 (cache hit
  0%)" when the provider omits cache tokens (known samples unchanged).
- warnCacheCollapse no longer fires for non-reporting providers (removes a
  false-positive collapse alert that was part of the same conflation).

Tests: tests/cache-ledger.test.ts +5 regression tests (unknown quarantine,
wire switch, upstream switch, restart, CLI breakdown). typecheck + full
suite (3040 pass) + build green on this branch.
…lugin-pipe usage path (#1536)

Review finding on this PR: the ledger has two entry points and only one was fixed. applyUsageSample (src/plugin.ts) — used by every plugin-mode pipe (pipePluginChatWithStrip / pipePluginResponsesWithStrip / pipePluginJson, the prepared.pluginMode branch of forward()) — still booked cachedTokens ?? 0, fed warnCacheCollapse with 0 for non-reporting providers, and passed no protocol/upstream identity. So P0 conflation and false-positive collapse warnings stayed live for all ACP-native agent sessions, and P1 wire/upstream attribution was blind there (only model + restart buckets worked).

- applyUsageSample: normalize undefined -> null (mirrors recordUsage in loop/core.ts), gate warnCacheCollapse on a real report, pass cached/protocol/upstreamOrigin into recordCacheSample; log marker unified to '(no cache report)' across both paths (was 'cached=n/a' in plugin mode).
- pipes take an optional trailing upstreamOrigin and forward it.
- server.ts: hoist the fetched-origin computation above the plugin branch (targetOrigin) and share it between the three session-carrying pipe calls and the compress-loop ctx (removes the shadowing local).
- tests: 3 regression tests (quarantine closure over the measurable subset, collapse-watch suppression, wire+upstream buckets via the plugin path); update the #695 log-text test to the unified marker.
…loop/plugin call sites (#1547)

recordUsage (loop/core.ts) and applyUsageSample (plugin.ts) each inlined the same input-side stats update + recordCacheSample sequence with subtly drifted guards. Extract settleUsageReport into cache-ledger.ts (next to recordCacheSample, which it calls) so all three response shapes can settle through one path.
…#1547)

The non-streaming JSON branch of forward() updated session.stats directly but never recorded a ledger sample, so stream:false sessions produced zero lines in /acp-cache and __bili/cache-report and were blind to every hit-rate / invalidation-attribution feature. Route its usage capture through settleUsageReport with protocol + targetOrigin, and give the branch the [acp-usage] log line and collapse watch the other two shapes already have.
…eview of #1550)

The new non-streaming settle line lacked the imageUsageSuffix marker that
the loop and plugin settle sites already carry (#695 parity claimed in flag
e). imageUsageSuffix is a pure read (empty unless the session has saved
image tokens), so appending it changes no state.
…st not eat a pending fold (#1536)

Mutation check: dropping the known-gate on the consumedFoldSeq advance
leaves this red ('measured sample owns the fold compRepay'); clean tree
48/48.
…ed quarantine + invalidation buckets, non-streaming ledger settle (#1535 #1536 #1547, web UI split out)
@ranxianglei
ranxianglei force-pushed the 2026-09-28_cache-attribution-combined branch from 3dc1358 to fec8a2d Compare September 28, 2026 11:41
@ranxianglei

Copy link
Copy Markdown
Owner Author

Conflict resolved (rebased onto current master, head fec8a2d, pushed — remote CI arbitrates):

README.md (the only real conflict): master's condensation deletions stand in full (owner's editorial intent). The branch's single genuinely-new doc block — the #1535 MODEL SWITCHES explanation — was re-grafted in master's condensed style into the existing /acp-cache FAQ answer (report semantics: switch-charged residual, per-event from → to / hit % / tokens, full vs last-8 summary, web column note). Everything else in the conflict region (hermes/opencode-2.x verbose sections master deleted) stays deleted.

src/plugin.ts / src/server.ts: auto-merged cleanly across all 8 commits (no manual hunks — master's movement was disjoint).

Gates run locally before push: cache-ledger + issue1547-nonstream + usage-observability + web-sessions(master state) + plugin-protocol = 63/63. Full suite on remote CI.

Merge order unchanged: this PR → repoint #1560 to master → merge #1560 → close #1557.

@ranxianglei
ranxianglei merged commit ae24ebf into master Sep 28, 2026
13 checks passed
ranxianglei added a commit that referenced this pull request Sep 29, 2026
Supersedes PR #1560: its stacked base branch (2026-09-28_cache-attribution-combined) was squash-merged to master via #1558, leaving head cf81056 permanently CONFLICTING. This carries the byte-identical 5-file web delta (+230/-8, verified against stacked head a30ee8d) onto current master. The switch accounting it displays shipped in #1558.
@ranxianglei
ranxianglei deleted the 2026-09-28_cache-attribution-combined branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[combined] Cache attribution stack: model-switch drops + unmeasured quarantine/invalidation buckets + non-streaming ledger settle

1 participant