Repository navigation
Conversation
Correlate exact egress fingerprints, socket generations and send attempts with terminal usage-field presence. Preserve reported numeric zero versus absent cache usage without logging raw payloads or headers. This adds diagnostics, not a provider cache-miss fix.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 检索了 open+closed 的 issue 与近期 PR(按主题 "Responses WebSocket cache diagnostics" / "cached tokens zero" / "cache miss"):没有其它在途的 WS 车道诊断 PR。与 #4328(Codex auto-review 短空闲低命中调查)、#5124(Codex 大规模 cache miss)是同症状家族但角色不同——那两个是未结的调查/报告,本 PR 是为该类问题补的脱敏观测手段,不修根因、不关闭它们,PR 描述里也明确声明了这一点;#5497 是 generation-boundary 误报的另一条线,与本 PR 无交互。判定:作为独立的诊断工具 PR 处理。 标签: 审核进行中:已读完整 diff(2 文件 +203/-0,head |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR Review(第一轮)— 方向、回归、代码、测试均过,无阻塞发现;剩一个 gating 决策 + CI 全量绿① 方向(原 issue 关):#4328 / #5124 是两个未结调查,同一症状家族——Codex 车道偶发 cache hit 掉零,现有日志分不清「本地改写 / 重连 / 上游未命中」。本 PR 在 WS 出站边界补脱敏观测:socket generation + 单调 send 序列关联、frame/等价全量请求/settings/response-id 的 SHA-256 指纹、usage 三态语义(reported-zero / missing / malformed)。方向正确:先测量后修因、指纹而非留原始载荷、数据面零改动;「不关闭两个调查、不坐实 provider 侧原因」的声明诚实。 ② 回归审查(既有行为逐项清点):
③ 代码审查:
④ 测试有效性:真 ws WebSocketServer + 真 验证:本地 checkout head 结构化八问
🔶 需要决策(PR 为何 draft——作者明确请求)
判定:代码与测试达标、无阻塞发现;待 (a) owner 对上方 gating 拍板、(b) 全量 CI 绿。CI 结果出来后我会补一条跟进。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 跟进(head
八问判定更新:第 5 问(回归)维持「无」——唯一红项已排除归因于本 PR;其余七问不变。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Reviewed end-to-end at head 1276d990a (base master@23a826d3788a). Scope is exactly 2 files vs the merge-base: src/responses-ws.ts (+36) and new tests/responses-ws-cache-diagnostics.test.ts (+167). No kernel/, no config surface, no persistence/wire/version changes — verified against the merge-base diff.
Correctness — verified, no defects.
- Purely additive. Every hunk inserts observation before an existing statement without altering control flow:
connectionGeneration++after socket creation (src/responses-ws.ts:362), fingerprint consts before the exchange promise (:449-452), asendcounter + debug log beforesocket.send(:468-475), and a terminal-usage block betweenreceivedEvents++andoutput.observe(:521-535). Request serialization, forwarded frames, checkpoint handling, retry decisions, and backpressure are untouched. - Counters are correctly scoped: monotonic per transport instance, never reset in
close(), so generations/sends stay distinguishable across reconnects and continuation retries. - Presence logic is right:
Object.hasOwndistinguishes absent vs present;tokens()(:525) maps non-finite/malformed →null, so reported-zero ≠ missing ≠ malformed — the exact distinction the motivation needs. - Privacy holds: logs emit only counters, enum event types, booleans, numeric usage, and SHA-256 hex digests; IDs are hashed, headers/URLs/payloads/connection-key never appear.
Wire fidelity — pinned by tests. The suite asserts the EXACT bytes the mock server receives equal the expected frames across full/delta/settings-change/reconnect, plus exact SSE bytes, so any future drift that changes egress bytes fails. Secret-scanning assertions (private-prompt / private-auth-key / … absent from logs) directly enforce the privacy claim. Real regression tests, not tautologies.
One substantive finding (no defect, but a design decision): the fingerprints are computed unconditionally, even when debug logging is off. Every /responses WS fetch now does a full-body JSON.stringify + SHA-256 (requestHash, :451), a canonical()+SHA-256 of settings (:452), and a SHA-256 of the frame inside sendCreate (:473). Cost scales with input size — and this project's normal case is MB-scale marathon contexts, so it is a recurring hot-path tax, not negligible. DiagnosticLog (:20) is an unguarded callback and the hashing runs before this.log(...), so no caller-side debug gate suppresses it. The author already raised this under "Performance / why draft" and asks to agree on bounded use / a gate / measured overhead before production — I am formalizing that as a decision below rather than deciding it here.
Minor: the body has a "Performance / why draft" section but the PR is not in draft state — either the framing is stale or it should be draft pending the perf decision. Non-blocking.
Verdict: code is correct, tested, and safe to ship functionally. I am not approving yet — the sole open item is the unconditional hot-path hashing, a gating/performance decision for the owner. Routed to the 🔶 需要决策 comment below; applying the needs-decision label.
|
🔶 需要决策 PR #5615 代码正确、测试扎实、无缺陷。唯一待你拍板:新增的缓存诊断指纹是「无条件」计算的——即使 debug 日志关闭,每个 八问(第 3/5 问按选项拆)
选项对比
请拍板 A/B/C(或直接给别的方向)。在你定之前我不 approve,PR 维持 blocked。我倾向 B:既保住取证价值,又把热路径成本封顶,且不加配置面。 |
(ai: vllm-qwen/qwen3.8-27b) 审查完成 #5615(head 1276d99,base master@23a826d37):代码正确、无缺陷——出站/SSE 字节与脱敏隐私均被新测试钉死(精确字节断言 + secret 扫描);唯一开放项「诊断指纹无条件做全量 body 序列化 + 3×SHA-256 的热路径成本」已按 🔶 需要决策 在 GitHub 给出选项 A/B/C(推荐 B:常开+限幅),已加 needs-decision 标签(GitHub 已加;镜像本已带),未 approve,待 owner 拍板。CI 镜像缓存显示仍在跑(暂无失败)。注:workdir 无 add-label.sh,标签经 API 直接施加。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
复审(新 head a9e07f7c5)— 本 PR 自身改动逐字节未变,维持「待决策、不 approve」
核对新 head:a9e07f7c5 是一条 “Merge branch 'master'” 提交——作者把分支重新同步到最新 master。相对上一轮我审的 head,本 PR 自己的两个文件(src/responses-ws.ts +36、tests/responses-ws-cache-diagnostics.test.ts +167)逐字节未变(git diff 为空);这次新增的只是从 master 合并进来的无关 dsh 变更(src/dsh-channel.ts 及若干 dsh 测试,现已在 master 上),不属于本 PR 贡献。对当前 master(merge-base c9ba4856)的真实 diff 仍恰好是这 2 个文件、纯新增 203 行——无 kernel/、无配置面、无 wire/持久化/版本变更。
结论沿用第一轮:代码正确、无缺陷——出站/SSE 的精确字节断言钉死了 wire fidelity,secret 扫描钉死了脱敏隐私,测试能兜住回归。唯一开放项不变:诊断指纹是无条件计算的——每个 /responses WS 请求即使 debug 日志关闭也会做一次全量 body 序列化 + 三次 SHA-256,成本随输入规模线性增长(本项目常态是 MB 级马拉松上下文)。这是取舍而非缺陷:作者自己在 “Performance / why draft” 里提出、要求进生产前先定策略。已在我上一条 🔶 需要决策 评论给出选项 A/B/C(推荐 B:常开但限幅),等 owner 拍板。
CI:镜像缓存显示新 head 的测试矩阵仍在排队/运行中(19 pending、0 failed、11 skipped),尚未全绿——此处不断言绿。
判定:仍 不 approve。这不是代码问题,而是待决策的热路径成本策略;needs-decision 标签保留。owner 定下方向后我再据此 approve,或按 B 把昂贵的全量 input 哈希改成长度受限载荷。
(ai: vllm-qwen/qwen3.8-27b) 复审新 head |
request_sha256 serialized and hashed the whole effective request a second time on every send, including delta sends. A synthetic benchmark on Node 24 put that at about 2.4 ms per MB of body, run synchronously. The field added little. On full sends it hashed the same text as frame_sha256. On delta sends, ResponsesWsHistory.continuation() has already matched the settings and every prior input item, so mode=delta attests the same thing. The hash also changes every turn, so it cannot show prefix stability. The remaining send fields are unchanged. The test now pins the exact key set of the send record.
|
(ai: anthropic/claude-fable-5-2) On the open A/B/C decision about the hashing cost: commit Cost.
Why remove instead of truncate.
What changed. 4 insertions and 4 deletions across Verification on The PR body is updated to match: the fingerprint list no longer mentions the full request, and the "Performance / why draft" section is replaced with the numbers above. The decision is still yours. If you prefer A, C, or a bound on |
…-cache-diagnostics
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 已独立核验作者新提交 核实结果
仍待你拍板的一点:去掉最贵字段后残余成本 = full send 时的 CI 镜像缓存:head |
|
Landed via batch PR #6482 (merged) — closing as superseded. |
Summary
Add redacted diagnostics at the final Responses WebSocket send boundary and raw terminal-event boundary.
Related: #4328 and #5124. This PR does not close either investigation or establish a provider-side cause. It is independent of #5497.
Motivation and limits
In a local billion-context 0.1.212 Pi/Codex session, one successful append-only delta request reported 18,479 input tokens and zero cached tokens. The effective-prefix evidence, unchanged settings, and existing connection logs did not identify a local rewrite or reconnect. Other 20 delta sends in the 26-request sample had nonzero hits.
The installed usage parser preserves absent-versus-numeric-zero usage, but the original terminal WebSocket frame was not retained. These diagnostics make the next recurrence easier to correlate without retaining raw payloads. They cannot prove upstream cache residency, routing, eviction, or correctness of provider usage reporting.
Privacy and wire fidelity
The new records contain only allowlisted counters, event types, numeric usage, booleans, and SHA-256 digests. They do not add raw prompts, reasoning, authorization/session headers, URLs, cache keys, checkpoint/response IDs, or error details to logs. In particular, the connection reuse key is never logged because it contains canonical headers.
Send records describe send attempts, not upstream acknowledgement or delivery. Counters are per transport instance; retain the existing outer session/connection log context.
Tests assert exact serialized egress and forwarded SSE bytes, not just parsed-object equivalence.
Performance
This is temporary diagnostic instrumentation, not a production cache optimization or a cache-miss fix.
The diagnostics are always on. They add no gate and no configuration.
Commit
30ee96bfremoved therequest_sha256field. It was the only part that serialized the full effective request a second time, and it ran on every send, including delta sends. The remaining per-send work is:frame_sha256: one SHA-256 over the frame string thatsendCreatealready builds. On a delta send, that string is the delta only.settings_sha256: canonical serialization and SHA-256 of the non-input settings.previous_response_idand responseidstrings.Synthetic benchmark of the three computations (Node v24.21.0, one machine, median of 15 runs, settings of roughly 90 KB). It was not measured in a live proxy.
request_sha256frame_sha256on a full sendsettings_sha256The field was removed rather than truncated for these reasons:
frame_sha256.ResponsesWsHistory.continuation()has already compared the settings and every prior input item, somode=deltaattests the same thing.One cost remains:
frame_sha256on a full send scales with the body size, as the table shows. Full sends happen on the first request, after a reconnect, and when a continuation does not apply.No live proxy/session was modified or restarted, and no provider experiment was run.
Config surface
None: no new or changed config fields, environment variables, or CLI flags. Records use the existing diagnostic logger callback.
No persistence-format, wire-protocol, package-version, or kernel-source changes.
Verification
Refreshed onto upstream master
1ee687bc; checked head1d59e828.node node_modules/typescript/bin/tsc --noEmit --project tsconfig.json— passed.npm run build— passed, including dist import/external checks.git diff --check upstream/master...HEAD— passed.30ee96bf(after removingrequest_sha256):tsc --noEmit, the five targeted files (74/74), andgit diff --checkon that commit's changes were rerun and passed.npm run buildwas not rerun.Targeted tests use the storage sandbox preload and mock upstreams. Full CI and live Codex-backend reproduction have not been run locally.
Category A — prefix-cache diagnostics.