Skip to content

feat: per-request cache reconciliation ledger (acp_cache tool) - #804

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-15_cache-report
Sep 18, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-15_cache-report

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Issue #800 (part 2/3 of three-repo change; kernel core: ranxianglei/acp-kernel#297)

What

Prove with data that compression does not wreck the prompt cache. Every provider usage report is decomposed into an exact additive identity:

missed_i = input_i − cached_i = newContent_i + compRepay_i + ttlRepay_i

so the grand ledger always closes against provider-reported totals (identity check OK — IN = C + NC + CR + TR, printed in every report — a broken implementation cannot hide). compRepay is capped at the structural excess beyond the fold's divergence point (X), so compression is only charged for re-pay it actually caused; everything else in the stable prefix lands honestly in ttl/other (TTL expiry / eviction).

Changes

  • src/cache-ledger.ts (new): session-scoped ledger persisted in session.metadata.cacheLedger — 512-line ring + exact running aggregates (survive ring eviction and restart) + fold events carrying measured T/h/k counters.
    • Folds detected from state.blocks diff against a persisted high-water mark: proxy mode eagerly at the applyRanges site (with V/V′ geometry), plugin mode lazily at usage time (pi creates blocks locally; bootstrap guard ignores pre-existing blocks).
    • X (divergence token offset) via tokenSnapshot prefix sums over ref order (kernel contract: refs are chronological, never reused).
    • Incremental math delegates to kernel decomposeSample — the same function buildCacheReport uses internally — so streaming and batch reports agree bit-for-bit (test asserts equality on identical raw data).
  • Tool dispatch: executeProxyTool (OpenAI/Responses) and executeAnthropicProxyTool (Anthropic) both handle the new read-only acp_cache tool → formatted report text. Registered to the model in all proxy-mode wires via the kernel arrays.
  • Management endpoint: GET /__bili/cache-report?session=<id> (defaults to most recent) → JSON {reports:[{id,report}]}; loopback+origin guards apply as with /__bili/stats.
  • Recorders: recordUsage (proxy mode, per upstream round) and applyUsageSample (plugin mode, per client request) feed the ledger with normalized cached-inclusive input totals.
  • package.json: acp-kernel pinned 0.0.72 → 0.0.73 (first release containing feat: cache reconciliation core (acp_cache tool + buildCacheReport) acp-kernel#297)。

Pre-flight

  • typecheck clean; 1519 pass / 0 fail (4 existing exact tool-list assertions updated — acp_cache intentionally joins the registered read-only set); build OK.

Known limitations (documented in module header)

  • absorb/decompress-triggered invalidations land in the ttl/other bucket (they are not folds).
  • X is best-effort: tokenSnapshot holds first-render estimates, and provider-side tokenization differs slightly.
  • Proxy mode samples per upstream round; plugin mode samples per client request — the identity holds either way.

⚠️ CI will be red until acp-kernel@0.0.73 publishes: merge ranxianglei/acp-kernel#297 first, then cut the kernel release (v0.0.73). Lockfile left stale on purpose — CI uses npm install (not npm ci) and re-resolves once the version exists.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — code verified good; CI is red solely because of the unpublished dependency.

What I verified

  1. Fetched 2026-09-15_cache-report (cbc298c) and read every one of the 10 changed files (+532/−8). Diff is clean — each line serves the stated purpose, no stray churn.

  2. Since CI cannot install acp-kernel@0.0.73 yet, I built acp-kernel from PR test: normalize macOS /var->/private/var tmpdir paths in launcher rewrite tests #297's exact head (sha 51baf8d), packed it as v0.0.73, and ran the full pre-flight against this branch locally:

    • typecheck: clean (all new kernel imports — decomposeSample, buildCacheReport, computeFoldEconomics, formatCacheReport, summarizeFoldEconomics, CacheReport/CacheTotals/FoldEvent types — resolve)
    • test: 1617 pass / 0 fail / 2 skipped (my count differs from the pre-flight's 1519 by ~98 — environment-dependent subtest counts; both runs are zero-fail)
    • build: OK; cacheLedger + decomposeSample confirmed inlined in the 2.88 MB bundle

    Conclusion: this branch will go green the moment acp-kernel@0.0.73 exists on npm. No code changes needed.

  3. Spot-checked the integration seams:

    • GET /__bili/cache-report sits under the /__bili/ admin prefix, so the tunnel-marker 403, loopback gate, and trusted-origin gate all apply (src/server.ts:539-564); the startsWith route match handles the query string correctly.
    • Both recorders feed normalized cached-inclusive totals (promptInputTotal) and skip zero-total samples on both paths ([Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793 convention) — proxy mode in recordUsage (src/loop/core.ts), plugin mode in applyUsageSample (src/plugin.ts).
    • handleAcpCache uses loggerLog, not console.error ✓.

Non-blocking observations (noted, no action taken)

  • When several unconsumed folds are pending at sample time, consumedFoldSeq advances past all of them even though decomposeSample may attribute re-pay to only one (the others keep T=0). Acceptable edge case — proxy mode always emits a usage round between folds — and the grand identity still closes either way.
  • profile { w: 1.0, r: 0.1, q: 4.0 } is hard-coded in buildSessionCacheReport; fine for v1, config-sourced later if needed.
  • FOLD_CAP=256 eviction drops per-fold economics from the report while agg-level cr/tr stay exact — consistent with the documented ring semantics.

Remaining blockers, in order

  1. Merge feat: cache reconciliation core (acp_cache tool + buildCacheReport) acp-kernel#297 (still open — human-only merge).
  2. Cut the kernel release v0.0.73 from master after test: normalize macOS /var->/private/var tmpdir paths in launcher rewrite tests #297 lands (I can prepare that release branch/PR as soon as it merges).
  3. Before merging this PR: refresh package-lock.json on the branch. PR CI uses npm install (re-resolves fine), but release.yml uses npm ci, which hard-fails on a lock/package mismatch — merging with the intentionally-stale lock would leave master broken for every subsequent release, and AGENTS.md requires the lockfile to stay in sync with the exact pin. Once 0.0.73 is live on npm I can push that one commit to the branch.

📝 中文摘要:代码审查通过,并已用内核 PR #297 的精确 head 本地全量验证(typecheck 干净、1617 个测试全部通过、构建成功);当前 CI 红仅因 acp-kernel@0.0.73 未发布。下一步:合并内核 PR #297 → 发布 v0.0.73 → 在本分支刷新 lockfile(release CI 用 npm ci,锁文件与 package.json 不一致会直接失败)→ 即可合并本 PR。

@ranxianglei

Copy link
Copy Markdown
Owner Author

解决冲突 解决ci失败

Issue #800: prove with data that compression does not wreck the prompt
cache. Every provider usage report is decomposed into an exact additive
identity — missed = new content + compression re-pay + ttl/other — so the
grand ledger always closes against the provider-reported totals.

- src/cache-ledger.ts: session-scoped ledger in metadata.cacheLedger
  (512-line ring + exact aggregates + fold events with measured T/h/k);
  folds detected from state.blocks diff (proxy mode eager at
  applyRanges, plugin mode lazy at usage time), X via tokenSnapshot
  prefix sums; incremental math delegates to acp-kernel decomposeSample
  so streaming and batch reports agree bit-for-bit.
- executeProxyTool / executeAnthropicProxyTool dispatch the new read-only
  acp_cache tool (kernel-defined) -> formatted report.
- GET /__bili/cache-report?session=<id> management endpoint (JSON).
- recordUsage (proxy mode) and applyUsageSample (plugin mode) feed the
  ledger; acp-kernel pinned to the release containing PR
  ranxianglei/acp-kernel#297.
@ranxianglei
ranxianglei force-pushed the 2026-09-15_cache-report branch from cbc298c to 8abb9bc Compare September 18, 2026 01:30
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-15_cache-report (8abb9bc)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-804

Each push to this PR publishes a new version under the pr-804 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr804.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Done — conflicts resolved and CI fully green. Pushed rebased head 8abb9bc to the PR branch.

Root causes

  1. Conflict: the branch was based on a1b24aa (v0.1.114); master has since moved to 0cfde7a (v0.1.118). In particular master's cdda2ea (from fix: bump acp-kernel 0.0.74 → 0.0.77 (hold acp_cache off wire until adoption) #915) bumped the kernel pin to 0.0.77 and added an UNADOPTED_KERNEL_TOOLS hold in src/compress-tool.ts filtering acp_cache off every wire surface ("until adoption lands"), which collides head-on with this PR's pin change and tool-list test updates.
  2. CI red: every check run on the old head cbc298c started 2026-09-15T10:23Z — the moment the PR opened, i.e. before acp-kernel@0.0.73 existed on npm. All jobs died at install time; the failures were never about the code.

What the rebase does

  • Rebased onto current master 0cfde7a; single commit, 11 files, +540/−34, every line purpose-related.
  • Adoption decision: this PR is the execution path for acp_cache (executeProxyTool dispatch in src/loop/core.ts + src/stream.ts, /__bili/cache-report endpoint), so I removed master's UNADOPTED_KERNEL_TOOLS hold — the tool goes back on all four wire surfaces and PROXY_TOOL_NAMES aliases the kernel set again. Without this, merging would ship a proxy that serves the tool schema… then answers calls with [Unknown proxy tool].
  • Kept master's kernel pin 0.0.77 (supersedes 0.0.73; carries the cache-reconciliation core from 0.0.75+). package.json now matches master exactly.
  • Lockfile blocker (floor 3) cleared: post-rebase lockfile already carries 0.0.77 matching the pin — npm install produces zero diff, so release.yml's npm ci will not fail.

Evidence

  • Local pre-flight on the rebased tree: typecheck clean; full suite 1750 pass / 0 fail / 2 skipped; build OK with acp_cache inlined in dist/index.js.
  • GitHub CI on new head 8abb9bc: all 8 checks success — test (ubuntu 22/24, windows 22/24), build-artifact, version-guard, real-codex zero-token e2e, codex MCP registration.
  • PR state: mergeable: true, mergeable_state: clean.

#804

Rebase 到最新 master(v0.1.118)解决了冲突:保留 kernel 0.0.77 固定版本、去掉 master 临时加上的 acp_cache 屏蔽(本 PR 正是它的执行路径,随本 PR 一起上线),lockfile 已同步;原 CI 全红是因为旧分支基于旧代码且当时 acp-kernel@0.0.73 尚未发布导致安装失败——现在 8 项 CI 全绿、无冲突,可以合并(合并由你操作)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant