Repository navigation
feat: attribute model-switch cache drops in acp-cache report and web UI - #1540
ranxianglei wants to merge 1 commit into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1484-pr.1540.eec9e70This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1540.tgz
npm install -g packageInstall instructions are refreshed on each push. |
c460c4b to
c9c9dbf
Compare
Review of #1540 — approved, mergeable (rebased onto current master). ① Issue / direction. Targets the root cause, not a symptom. A mid-session model switch invalidates the provider prefix cache, so the next request re-bills the whole stable prefix — and in the acp-cache ledger those tokens were silently absorbed into the kernel's generic ② Regression review (existing-behavior inventory). All changes are additive; no unintended regression found:
③ Code review.
④ Test validity. All 6 new cases in
Minor doc nit (non-blocking). The PR body says " Verdict: Mergeable once CI is green on the rebased head 中文摘要:模型切换掉缓存的归因实现正确、测试扎实、改动全部为增量(内核桶与恒等式未动、旧账本原地归一化、web 表列数一致);唯一实质问题是分支基于旧 master(server.ts 在 master 上另有 clientError 改动但区域不相交),我已 rebase 到最新 master 并复跑 typecheck/test/build 全绿后强推(新 head c9c9dbf),CI 变绿即可合并。 |
c9c9dbf to
70360af
Compare
70360af to
4c3f40b
Compare
|
Deep review: ✅ mergeable as-is (bottom of the stack, self-contained). What I verified
Non-blocking
|
Concur — I re-verified your two load-bearing points against the rebased head ( ① Both lanes stamp ② Event list is render-independent. Confirmed on the code side. Agree on the non-blocking nit: Final state: single commit rebased onto live master → head 中文摘要:独立复核了你深评的两个关键点—— |
Closes #1535
What & why
Switching models mid-session invalidates the provider's prefix cache: the next request re-bills the entire stable prefix under the new model. In the acp-cache ledger those tokens landed in the kernel's generic
ttlbucket — indistinguishable from upstream TTL expiry / provider-side eviction (the kernel cannot tell them apart, by design). This PR attributes them explicitly: each model switch gets its own count and token total, in both the/acp-cachetext report and the web UI.How
src/cache-ledger.tsrecordCacheSamplereadssession.metadata.lastModel(already stamped per bound request insrc/server.tsbefore prepare; metadata persists across restarts) and flags a line (sw) when its model differs from the previous sample's KNOWN model; unknown/missing models never flag.ttlRepay= missed − new content − compRepay). Deliberately NOT cr+tr: compRepay stays booked to its fold's T, so a switch coinciding with a fold never double-counts a token. Honest by construction — if the switch didn't actually drop the cache, the residual ≈ 0.agg.switches/agg.switchMissed(never pruned); per-event list built from surviving lines.BiliCacheReport extends CacheReport { modelSwitches },readModelSwitchStats()(read-only, no ledger bootstrap).handleAcpCachenow ends with aMODEL SWITCHESsection (always shown, incl.none observed;fulllists every event, summary the last 8).WebSessionSummarygains optionalmodelSwitches/switchMissedTokens(emitted only when > 0); sessions table + overview recent list gain one column (count · dropped tokens, tooltip explains the attribution); i18n zh/en./__bili/statsalso gainslastModel/modelSwitches/switchMissedTokensfor endpoint consistency.Behavior changes (all additive)
/acp-cachetext report now ALWAYS ends with a newMODEL SWITCHESsection (zero case printsnone observed)./__bili/cache-reportJSON gainsmodelSwitches {count, missedTokens, events[]}on each session report./__bili/statssessions gainlastModel/modelSwitches/switchMissedTokens.Pre-flight (head c460c4b, base origin/master 86e0b68 / v0.1.163)
npm run typecheck— cleannpm test— 3019 tests: 3007 pass / 0 fail / 12 skippednpm run build— OK