Skip to content

[PR] fix(cache): pair overlapping forwards with their own request evidence - #4430

Closed
ReineLan wants to merge 1 commit into
ranxianglei:masterfrom
ReineLan:2026-10-23_cache-request-pairing
Closed

ReineLan wants to merge 1 commit into
ranxianglei:masterfrom
ReineLan:2026-10-23_cache-request-pairing

Conversation

@ReineLan

Copy link
Copy Markdown
Contributor

Closes #4429

Category A — cache forensics. Two in-flight forwards sharing a Session currently overwrite the pending body/inbound/image slots. If B sends before A completes, A's bill can be attached to B's digest and send-time facts, and A's settlement can consume B's evidence.

This change scopes pending captures to the async forward, including internal compression rounds. Completed evidence stays on the session; reversed completion order clears comparison continuity rather than inventing a prefix seam. Aborted captures release their pending references. The JSON settlement path uses the original send capture instead of re-noting a body without its original credential/parameter facts. Image learning retains the relay's first-round capture order and drops unbilled facts when advancing an internal round.

Optional capture ID/round and out-of-order evidence fields are exposed in cache reports. Legacy non-forward callers retain their API. The existing semantic classification rules, wire bodies, folding policy, session identity and baseline-writer authority are unchanged. In particular this does not resolve #1916 or prove the cause of #4328's production misses. Generated thinking-token canonicalization and capped-body attribution changes from the larger local staging patch were deliberately excluded to avoid overlapping #4053 and independent scopes.

Validation on master 50e2506 + head:

  • Negative control on unmodified master a2d8cb9: original four-case HTTP overlap matrix fails; SSE cases pair the wrong body digest or lose it.
  • 107/107 targeted tests pass, zero skips: eight real HTTP proxy/plugin × SSE/JSON × completion-order cases, request-local unit/cleanup/round/image cases, cache ledger/seam/key/effort/non-stream/body stability regressions, and hermetic image-billing e2e. The fake upstream records exact received bodies; each bill is identified by its output-token count and checked against body SHA-256, credential and generation fingerprints.
  • npm run typecheck passed; after rebasing, tsc --noEmit --project tsconfig.json passed again.
  • npm run build passed after rebase, including dist import/external checks.
  • Two manual review passes; git diff --check clean.

The 4-phase real-provider Codex suite has not been run and remains a pre-merge requirement for the touched request pipeline. Remote full CI test lanes on this pushed head are authoritative and must be checked; local targeted results do not imply full-suite green.

Config surface: no config fields, flags, defaults, persistence version, or kernel changes. No live installation/configuration changes were made.

@ranxianglei ranxianglei changed the title fix(cache): pair overlapping forwards with their own request evidence [PR] fix(cache): pair overlapping forwards with their own request evidence Oct 23, 2026
@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Oct 23, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 独立检索了 open+closed 的 issue/PR(按 pending capture / body digest / mispair / cache forensics 等失败签名,不只标题):最近的缓存取证类 PR——#4053(host 生成的 thinkingTokens 不进 client-origin 证据)、#4391(body stability 用 marker-blind 视图)、#4132(anthropic lane cached tokens 双计)、#4074(cache_creation 合成终帧)——机制各不相同,本 PR 专修「同一 Session 上两个 in-flight forward 互相覆盖 pending body/inbound/image 槽位」这一请求证据所有权问题,与它们无重叠。issue #4429 的自查重结论与我的独立检索一致。

标签: bug scope/wiring p2 — 缺陷类;触碰的是请求管道上的取证/计费证据面(forward 路径),非 kernel 压缩决策面;影响是误导性的缓存归因 + 可能学错按路由的图片成本,属确定性噪音/成本问题,非可用性停摆、非缓存泄露类,故 p2。

正在做完整 review(方向关 → 回归清点 → 代码审查 → 测试有效性 → CI 状态),结论按结构化八问回复。

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: zhipuai-lb/glm-5.3)

已在 #4558 落地(ALS capture 与 #4053 证据剥离正交并集)。本 PR 关单收尾;回滚点为 #4558 单提交 revert。

ranxianglei pushed a commit that referenced this pull request Oct 25, 2026
…apture on the relay lane (#4731)

#3307 P2-A's xMeasured block read the pending body/inbound pair from
session.transient.sentBody/inBody, but since #4430 the relay lane keeps
that pending evidence inside the withCacheCapture AsyncLocalStorage
capture — transient.sentBody is never written there, so the fold-cost
attribution fix was dead on exactly its target lane (stateless proxy
clients) while the loop lane kept working.

Add pendingEvidencePair(): both sides (outbound body + inbound view)
from ONE source — the capture when present, else session.transient
(loop lane, byte-identical fallback) — because a mixed read can pair an
old body with new usage and mis-attribute a seam. Settled baselines
(settledBody/settledInBody) stay on transient on every lane.

Also document the round>=2 conservative settle (noteForwardedBody drops
pending.inBody on retries, so settledInBody loses its baseline):
diagnostic-only by design, semantics unchanged.

Regression: tests/issue4731-cache-capture-pair.test.ts pins the
relay-lane fire, the #3023 client-rewrite guard with capture-held
evidence, the no-mis-pair negative (stale transient body + capture
usage), the loop-lane fallback, and the same-source pair contract.
ranxianglei pushed a commit that referenced this pull request Nov 13, 2026
…suite

The #4731 source fix (fold re-pay attribution reading the #4430 ALS capture
on the relay lane) is already on master (e253a29) with its own regression
suite (issue4731-cache-capture-pair.test.ts). This file's positive
mid-history-break and #3023 client-rewrite cases are strict subsets of that
suite's; only the tail-append guard on the wrapped lane is new here. Drop the
two near-duplicates and the now-unused CoreMessage/noteInboundMessages imports.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

2 participants