Skip to content

fix(cache): settle non-streaming rewriter turns into the cache ledger (#1547) - #1550

Closed
ranxianglei wants to merge 4 commits into
2026-09-28_cache-attribution-p0-p1-clifrom
2026-09-28_nonstream-cache-ledger
Closed

ranxianglei wants to merge 4 commits into
2026-09-28_cache-attribution-p0-p1-clifrom
2026-09-28_nonstream-cache-ledger

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #1547

What

The cache ledger had exactly two entry points: recordUsage (streaming compress loop, src/loop/core.ts) and applyUsageSample (plugin pipes, src/plugin.ts). The non-streaming rewriter branch of forward() (src/server.ts, else arm of if (prepared.stream) — JSON responses with compress injection) updated session.stats directly but never recorded a ledger sample. stream:false sessions therefore produced zero lines in /acp-cache and __bili/cache-report, and every hit-rate / invalidation-attribution feature (including the #1536 P0/P1 work this branch builds on) was blind to them.

Changes

  1. refactor (50278b82): extract settleUsageReport(session, { total, reportedCached, output?, protocol?, upstream? }) into src/cache-ledger.ts — the shared input-side settle path (stats update + recordCacheSample). recordUsage and applyUsageSample now call it; behavior-neutral (typecheck + full suite green before and after extraction; guards audited line-by-line).
  2. fix (bb4ef7ac): the non-streaming else arm settles through the same helper, passing protocol: prepared.protocol and upstream: targetOrigin (the origin computation added by fix(cache): quarantine unmeasured cache reports + attribute invalidation causes (P0+P1, CLI) #1545), so all three response shapes (plugin SSE, loop SSE, loop JSON) feed one ledger. The branch also gets the [acp-usage] log line and the warnCacheCollapse watch the other two shapes already have.

Behavior changes (flagged per policy)

# Old New Why
a stream:false turns record zero ledger samples one sample per successful turn the fix itself
b a zero-total usage report clobbered lastInputTokens→0, lastInputTokensSource, and retired the overflow arm preserved aligns with #793 zero-total semantics already used by streaming/plugin paths
c cacheSamples incremented even at total===0 requires total>0 aligns with the other two shapes
d collapse watch skipped non-streaming sessions covered the issue names the watch opt-out as part of the problem
e no [acp-usage] log line for non-streaming turns logged #695 parity across settle sites
f outputTokens only added when input fields were present added whenever output is reported aligns with loop/plugin (output independent of input); degenerate case only
g cached-without-input (degenerate): nothing done settled as an input=0 sample aligns with the loop's degenerate handling

Nudge / emergency-truncate triggering is unaffected: it reads session.stats, whose values are unchanged for every live report shape.

Stacking dependency

Builds on #1545's branch: consumes its targetOrigin computation and the extended recordCacheSample(cached: number | null, protocol, upstream) signature. Please merge after #1545 (or rebase onto master once it lands).

Tests

  • New tests/issue1547-nonstream-cache-ledger.test.ts: server-level repro of the issue (a stream:false turn through the built proxy now yields a ledger line and populates __bili/cache-report; a follow-up zero-total turn keeps lastInputTokens and records no sample) + 3 unit tests for settleUsageReport (attribution fields, credit netting + quarantine, zero-total no-clobber).
  • Pre-flight on this head: typecheck clean; full suite 3047 pass / 0 fail / 12 skip (e2e-gated); build OK; e2e codex preflight OK. The full ACP_TEST_E2E run needs a local upstream at 127.0.0.1:8199 which this sandbox does not have — the base commit fails identically there, so CI (which has the secret upstream) is the authoritative e2e gate for this PR.

ework-agent added 2 commits September 28, 2026 12:56
…loop/plugin call sites (#1547)

recordUsage (loop/core.ts) and applyUsageSample (plugin.ts) each inlined the same input-side stats update + recordCacheSample sequence with subtly drifted guards. Extract settleUsageReport into cache-ledger.ts (next to recordCacheSample, which it calls) so all three response shapes can settle through one path.
…#1547)

The non-streaming JSON branch of forward() updated session.stats directly but never recorded a ledger sample, so stream:false sessions produced zero lines in /acp-cache and __bili/cache-report and were blind to every hit-rate / invalidation-attribution feature. Route its usage capture through settleUsageReport with protocol + targetOrigin, and give the branch the [acp-usage] log line and collapse watch the other two shapes already have.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review started. Pulling the branch (bb4ef7ac, stacked on #1545's 2026-09-28_cache-attribution-p0-p1-cli) and walking the PR-review gate: direction vs #1547 → regression inventory of the three settle sites → code/test/diff review. Will report findings here with a Chinese summary when done.

…eview of #1550)

The new non-streaming settle line lacked the imageUsageSuffix marker that
the loop and plugin settle sites already carry (#695 parity claimed in flag
e). imageUsageSuffix is a pure read (empty unless the session has saved
image tokens), so appending it changes no state.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — fix is correct, minimal, and well-tested. Mergeable once the stack lands (order: #1540 → #1545 → #1550). I added one small direct fix during review (below). Details:

① Direction (issue first). Verified #1547's triage against both master and #1545's head: the root cause is real and correctly located — the non-streaming else arm of forward() was the only settle site that updated session.stats without touching the cache ledger, so stream:false sessions were invisible to /acp-cache, __bili/cache-report, and everything built on them (#1536 attribution). Fixing at the shared settle layer (helper) rather than adding a third divergent inline block is the right layer — it also removes the future drift risk between the three sites.

② Regression audit (existing behaviors on touched paths):

  • src/loop/core.ts recordUsage: neutral. Stats conditions are byte-equivalent after extraction (total>0 gates input/lastInput/source/overflow-arm; reportedCached!==null && total>0 for cached/cacheSamples), outputTokens update stays outside the gate exactly as before, the [acp-usage] round N line is unchanged, and the recordCacheSample gate + fields (at/input/cached/out/protocol/upstreamOrigin) are identical.
  • src/plugin.ts applyUsageSample: neutral. Because total = sample.inputTokens !== undefined ? promptInputTotal(...) : 0, total>0 ⟹ inputTokens defined, so the old cached-increment condition is logically identical to the new helper gate; the old record gate (inputTokens !== undefined && total > 0) ≡ the new outer if. The warnCacheCollapse call and zero-total warn come from fix(cache): quarantine unmeasured cache reports + attribute invalidation causes (P0+P1, CLI) #1545 and are untouched here.
  • src/server.ts non-streaming arm: intentional, all seven flags verified in the diff — (a) zero→one ledger sample per successful turn (the fix); (b) zero-total no longer clobbers lastInputTokens/source or retires the overflow arm ([Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793 parity); (c) cacheSamples now requires total>0; (d) collapse watch covered; (e) [acp-usage] line added; (f) outputTokens now added whenever output is reported (was nested inside the total check); (g) cached-without-input settles as an input=0 sample (matches the loop's degenerate handling). All align this site with the other two rather than inventing new semantics.
  • Nudge / emergency-truncate: unaffected — they read session.stats, whose values are unchanged for every live report shape.
  • Double-recording check: compressLoopResponsesJson writes nothing to stats.; the outer else arm remains the single settle point for JSON responses.
  • Safety at the new call sites: warnCacheCollapse early-returns below MIN_INPUT (8000), so the billed=0 path can't divide; recordCacheSample computes hit% only when known && input > 0, so the (g) input=0 sample records cleanly and unknown reports still quarantine into unknownInput.

③ Code. Helper signature/behavior audited line-by-line (single definition, callers keep their own gates/log/watch/output handling); targetOrigin comes from #1545's computation and is in scope; no as any, no config surface, no version bump, no lockfile churn.

④ Test validity. tests/issue1547-nonstream-cache-ledger.test.ts has teeth: the server-level test drives a real built proxy with a stream:false turn through a fake OpenAI upstream and asserts stats, the report endpoint, the ledger line (hitPct 90, proto/upstream attribution), then a follow-up zero-total turn asserting lastInputTokens preserved and no second sample — removing the settle call or the zero-total guard would fail it. The 3 unit tests pin attribution fields, credit netting + quarantine, and zero-total no-clobber. (Unit test 4 documents that a direct helper call with (0,null) still pushes an input=0 line — correct as internal semantics; the caller gates mean live paths never reach that shape, which the server test pins.)

⑤ Diff cleanliness. 5 files, 2 commits (refactor / fix cleanly separated), every line on-topic.

Verification evidence. CI on this head ran the 5 e2e lanes (real opencode/dsh/pi/codex vs deterministic fake upstream + hermetic registry e2e) — all green. The main ci.yml matrix did not run because its trigger is pull_request: branches:[master] and this PR's base is the stacked branch; e2e jobs only build, they don't typecheck or run the suite. So I ran the missing gates locally on the head: typecheck clean, full suite 3047 pass / 0 fail / 12 skip (exactly matching the PR's claim), build OK, new test file 4/4. Note for merge time: once #1545 lands, rebase onto master so ci.yml runs on the final head before merging.

Direct fix added during review (per review policy, committed straight to the branch): the new non-streaming [acp-usage] line lacked the imageUsageSuffix marker that both other settle sites carry, so flag (e)'s "#695 parity" wasn't fully met. imageUsageSuffix is a pure read (empty string unless the session has saved image tokens), so appending it changes no state. Commit 800856eb ("fix: complete [acp-usage] log parity for stream:false turns") — 2 lines, typecheck clean, tests green. The e2e lanes will re-run on the new head.

中文摘要:修复了非流式(stream:false)请求从不写入缓存账本的缺陷(三个结算点统一走共享 helper),行为变更 (a)–(g) 均已在 diff 中逐项核实为有意对齐、无意外回归,本地补齐了 CI 未跑的 typecheck+全量测试(3047 通过)并顺手补全了日志的 img-saved 字段,待 #1545 合入、rebase 到 master 触发主 CI 后即可合并。

…st not eat a pending fold (#1536)

Mutation check: dropping the known-gate on the consumedFoldSeq advance
leaves this red ('measured sample owns the fold compRepay'); clean tree
48/48.
@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review: ✅ mergeable — cleanly rebased stack head, 48/48 local (incl. my new pin), typecheck clean.

What I verified

  • The cache ledger: non-streaming rewriter path never records usage samples (ledger blind for stream:false sessions) #1547 fix itself: the non-streaming branch now routes through settleUsageReport — stream:false sessions finally feed the ledger (previously stats-only: requests: 0 in cache-report while stats accumulated). The log-line parity commit makes all three shapes emit identical [acp-usage] lines, so post-fold cliffs are attributable from logs on every shape.
  • Settle atomicity: the whole ledger push is synchronous — no interleaving hazard for concurrent requests on one session; the later withSessionLock sections (response rewrite) are untouched.
  • Order-of-operations preserved: warnCacheCollapse still runs only for measured samples, zero-total guard ([Bug] Anthropic 流式请求被中断后,可能把 lastInputTokens 错误覆盖为 0 #793) still protects lastInputTokens, outputTokens still added exactly once per response (no double-count with the shared path — the shared path doesn't touch output stats).
  • e2e (issue1547) drives the real server non-streaming branch and asserts ledger lines + invalidation attribution end-to-end.

Stack order: #1540 → #1545 → #1550 (this PR carries the fold-cursor pin test on its head — merge the stack in order and it all lands together).

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Confirmed against the new head 1e04ba03 (your fold-cursor pin) — everything matches your read:

Nothing outstanding from my side — ready to merge in order whenever you are.

中文摘要:已在新头 1e04ba0(含你的 fold-cursor 回归 pin)上独立复核——typecheck 干净、全量 3048 通过、CI 五条 e2e 全绿,pin 测试断言有牙(mutation 会红),无新增问题;按 #1540 → #1545 → #1550 顺序合并即可,合并时留意主 ci.yml 需在落到 master 后的最终头上跑绿。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Closed as superseded by the combined PR #1558 (issue #1557) — the three were mutually dependent (#1545 references #1540's attribution symbols 20×) and are now one reviewable unit. All semantics preserved; review comments carry over. Web UI portion split out for ework (tracked in #1557).

ranxianglei pushed a commit that referenced this pull request Sep 28, 2026
…eview of #1550)

The new non-streaming settle line lacked the imageUsageSuffix marker that
the loop and plugin settle sites already carry (#695 parity claimed in flag
e). imageUsageSuffix is a pure read (empty unless the session has saved
image tokens), so appending it changes no state.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant