Repository navigation
[PR] fix(cache): make fold re-pay attribution (#3307 P2-A) work on the relay lane it was built for - #4751
[PR] fix(cache): make fold re-pay attribution (#3307 P2-A) work on the relay lane it was built for#4751ranxianglei wants to merge 7 commits into
Conversation
…3307 P2-A works on wrapped lanes #3307 P2-A measured the stable prefix off the wire inside recordCacheSample by pairing this request's outbound body with the previous settled one. It read both pending slots straight off session.transient, but #4430 moved the relay-lane pending evidence into the per-request AsyncLocalStorage capture (noteForwardedBody/noteInboundMessages write via pendingCapture). On every wrapped lane - streaming, loop rounds, non-streaming - the transient slot was always undefined, so the whole compensation block silently died exactly on its target lane: stateless proxy clients resending full history kept booking fold re-pay into the unattributed seam bucket. Read curB/curIn through pendingCapture (same scope as their writers); the settled baselines (prevB/prevIn) stay on transient by design. Reading both together keeps the #3023 clientBreak guard effective on wrapped lanes. Regression tests drive each request cycle through its own withCacheCapture scope, mirroring relay.ts forward(); the positive case fails pre-fix.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9436-pr.4751.21e0d04This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4751.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 按失败签名检索 open+closed issue/PR(#3307 P2-A 机制家族:recordCacheSample fold re-pay 归因、pendingCapture vs transient 证据槽),唯一命中是本 PR。本 PR 即 #4731 的修复,同机制家族无并行条目。 标签:bug, scope/wiring, p2(缺陷为真 bug——P2-A 补偿块在 relay 车道一直是死代码;改动位于请求管道/relay 捕获槽;严重度 p2=归因报表失真、有绕行、非缓存泄露、不影响计费)。 八问
测试有效性新套件让每个请求周期走自己的
本地验证(定向,未跑全量)
CI 状态与已执行动作(请注意)
判定代码审查通过:根因定位准确、修法最小(两行 + 注释)、测试双向钉住机制、无回归、无配置面变更。待新 head 2d7a674 的 CI 全绿即可合并(预期由 ci-fixes 覆盖在册簇;若残留红,继续走 #4599 跟踪,不另开 issue)。 |
…onflict resolution) Resolves the single conflict in src/cache-ledger.ts: master already landed a cleaner #4731 fix (e253a29) that reads the pending evidence pair through the pendingEvidencePair helper. Take master's version for the conflicting hunk so the canonical fix wins; this PR's net contribution is now its regression test tests/issue4731-fold-attribution-relay-lane.test.ts (relay-lane fold attribution, #3023 client-rewrite guard, tail-append negative), which passes alongside master's issue4731-cache-capture-pair.test.ts (8/8).
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved and pushed. PR #4751 was dirty against master. I merged What the conflict actually was. Master already landed a cleaner fix for the same issue #4731 — commit Resolution. I took master's version for the conflicting hunk in Disposition note (your call). Since master already fixed #4731 more completely — its 结构化八问:
判定:冲突已解决并推送,PR 现可合并(mergeable, 待 review)。因 master 已更完整修复 #4731,此 PR 现仅剩一个额外回归测试,建议 owner 决定合并(补测试)或关闭(重复)。 |
[ework] CI fix verified green ✓ |
(ai: vllm-qwen/qwen3.8-27b) This re-trigger was stale — the conflict was already resolved in my previous run. I verified the live state first: the PR was What I did this run: brought the PR fully up to date so the "behind" state is cleared and the blocker can't re-fire.
Live state after push: 结构化八问:
判定:冲突早已解决(上一轮);本轮把 PR 同步到最新 master,现已 |
[ework] CI fix verified green ✓ |
…suite The #4731 source fix (fold re-pay attribution reading the #4430 ALS capture on the relay lane) is already on master (e253a29) with its own regression suite (issue4731-cache-capture-pair.test.ts). This file's positive mid-history-break and #3023 client-rewrite cases are strict subsets of that suite's; only the tail-append guard on the wrapped lane is new here. Drop the two near-duplicates and the now-unused CoreMessage/noteInboundMessages imports.
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE — the PR is now a clean, minimal, test-only addition.
Key finding: the source fix is already on master
The fix this PR describes (fold re-pay attribution reading the #4430 ALS capture on the relay lane) was already merged to master by a parallel agent as e253a29d55 ("fix(cache-ledger): read the wire fold-attribution pair from the ALS capture on the relay lane (#4731)"), and issue #4731 is closed.
After this branch merged master (13f735862), the PR's source diff is empty — src/cache-ledger.ts is byte-identical to master. The only remaining diff is the regression test file tests/issue4731-fold-attribution-relay-lane.test.ts.
What I did (fix on the spot)
The test file's 3 cases were 2/3 duplicate of master's existing tests/issue4731-cache-capture-pair.test.ts:
- the positive mid-history-break case — a strict subset of master's test 1 (master's also asserts
cr > 0+ the transient sanity); - the #3023 client-rewrite negative — a strict subset of master's test 2 (master's also asserts
clientInserts === 1); - the tail-append guard on the wrapped lane — unique (master's relay-lane suite has no tail-append; the loop-lane twin lives in
issue3307-fold-attribution-wire.test.ts).
已顺手修:trimmed the test file to the one unique tail-append guard (dropped the two near-duplicates + the now-unused CoreMessage/noteInboundMessages imports) + commit 146e928
Verified at the new head: all 9 tests green (the trimmed file + master's 5 capture-pair + 3 wire tests); typecheck clean. The positive case has teeth — reverting the pendingEvidencePair read back to session.transient?.sentBody makes it fail.
8问
- 是什么问题:#3307 P2-A 的折叠 re-pay 归因在 relay 车道上失效——
recordCacheSample读的是session.transient?.sentBody,但 #4430 已把 relay 车道的 pending 证据挪进 ALS capture,该槽位在 wrapped 车道恒为 undefined,整个补偿块在目标车道上死掉,re-pay 一直记进未归因 seam 桶。 - 什么原因导致的:#4430 把 relay 车道证据迁移进 ALS 后,#3307 的归因块没跟着改读路径,仍直读 transient(curB/curIn 同病)。
- 影响范围与严重程度:仅 relay(wrapped)车道——无状态代理客户端重发全量历史的场景。属观测/归因精度问题(acp_cache 折叠经济学失真),不影响压缩正确性,p2 级体验/观测瑕疵。
- 怎么修复的:新增
pendingEvidencePair(session)访问器,从单一来源(ALS capture 活跃取 capture,否则 transient)同时取{sentBody, inBody};curB/curIn 必须一起改(只改 curB 会在 wrapped 车道旁路 #3023 clientBreak 守卫);prevB/prevIn 留在 transient(settled 基线住那里)。 - 是否有回归问题:无。loop 车道(不走 ALS)行为字节级不变;stale transient body 不再误配对;#3023 client-rewrite 守卫在 wrapped 车道仍生效——master 的
issue4731-cache-capture-pair.test.ts5 个 case 全部钉住。 - 是否破坏缓存:不涉及。只改归因读取来源,不动缓存键/重试/增量游标/去重键。
- 是否改了设计方向:不涉及。只是把归因块的证据读取对齐到 #4430 的 capture 架构,无转向。
- 是否引入了新的配置:不涉及。
Note: the PR body's "Fix"/"Behavior change" sections are now partially stale (the fix already shipped via e253a29d55), but the described behavior is correct and verified. Safe to merge — it only adds the one unique regression test.
[bot] 🏷 PR #4751 review — done. Verdict: APPROVE (posted as a review comment; see PR thread). Key finding — the source fix is already on master. The fix this PR describes (fold re-pay attribution reading the #4430 ALS capture on the relay lane) was already merged by a parallel agent as Fix on the spot: the test file's 3 cases were 2/3 duplicate of master's existing 已顺手修:trimmed the test file to the one unique tail-append guard + commit 146e928 Review posting note: a formal APPROVE is rejected by GitHub with "Can not approve your own pull request" (the PR is authored by the same PR state: head 8问
Coordination issue: I searched open issues/PRs for a review-lane coordination hub and couldn't identify a specific one with confidence — the only coordination hub named in AGENTS.md is #4599 (CI 总控), which is for CI failures (this PR's CI is green), not review status. Point me at the exact coordination issue number and I'll post the one-line report there. |
Closes #4731
What
#3307 P2-A (de193f4) added wire-measured fold attribution to
recordCacheSample: when a big unexplained miss coincides with a pending fold, pair this request's outbound body against the previous settled one, take the byte-LCP, and min() the measured pre-fold prefix intofirstFoldStartTokensso the re-pay credits the fold instead of falling into the unattributed seam bucket.The block read its pending evidence straight off
session.transient. But #4430 moved the relay-lane pending evidence into the per-request AsyncLocalStorage capture —noteForwardedBody/noteInboundMessageswrite viapendingCapture(session), which returns the ALS capture when one is active for this session. So on every wrapped lane the transient slot was always undefined and the compensation block silently never ran — exactly on its target lane (stateless proxy clients resending full history through the relay). The sibling reads #4430 migrated (detectSeam's sent/in slots, settleUsageReport's keyFp/effFp/slot) all usependingCapture; these two lines were the only stragglers.Fix
Two lines in
src/cache-ledger.ts(recordCacheSample): readcurB/curInthroughpendingCapture(session)instead ofsession.transient. Both must move together: with onlycurBfixed, theclientBreakguard (#3023) would see an undefinedcurInand be bypassed on wrapped lanes, mis-crediting client-side rewrites. The settled baselines (prevB/prevIn=settledBody/settledInBody) stay on transient by design — settleUsageReport moves them there.Side effect (the issue's secondary risk, resolved by the same change): a mixed wrapped/unwrapped session can no longer pair a stale body left in transient by an earlier unwrapped call with the current usage — each settle now reads only its own scope's slot.
Behavior change (intended)
On ALL relay-driven traffic (streaming, loop rounds, non-streaming —
runCompressLoopruns insideforwardCaptured, so loop rounds are wrapped too):firstFoldStartTokenslowered to the wire-measured value); tail-appends and client-side inbound rewrites still do NOT credit the fold (guards unchanged).Unwrapped lanes (plugin pipe) already worked and are byte-for-byte unchanged (
pendingCapturefalls back tosession.transientthere).No billing or cache behavior changes — attribution/reporting only. No config surface touched.
Tests
New
tests/issue4731-fold-attribution-relay-lane.test.tsmirrors the existing #3307 suite but drives each request cycle through its OWNwithCacheCapturescope (exactly whatrelay.ts forward()does per request):curIncomes from the same capture (a curB-only fix would fail this);Existing suites re-run green: issue3307-fold-attribution-wire, issue3307-seam-suspect-vs-client, cache-capture-pairing, cache-seam-detector, cache-ledger, issue2202-coverage-accrual (78/78). Typecheck clean.