Skip to content

[PR] chore: add redacted Responses WebSocket cache diagnostics - #5615

Closed
xulongwu4 wants to merge 5 commits into
ranxianglei:masterfrom
xulongwu4:2026-11-03_codex-ws-cache-diagnostics
Closed

xulongwu4 wants to merge 5 commits into
ranxianglei:masterfrom
xulongwu4:2026-11-03_codex-ws-cache-diagnostics

Conversation

@xulongwu4

@xulongwu4 xulongwu4 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add redacted diagnostics at the final Responses WebSocket send boundary and raw terminal-event boundary.

  • Correlate socket generations and monotonic send-attempt sequences, including continuation retries.
  • Fingerprint exact serialized egress frames, non-input settings, and checkpoint/response IDs.
  • Record numeric input/output/cached usage alongside explicit field-presence flags. Preserve the distinction between reported zero, missing usage, and malformed values.
  • Leave request serialization, forwarded response frames, checkpoint handling, and retry decisions unchanged.

Related: #4328 and #5124. This PR does not close either investigation or establish a provider-side cause. It is independent of #5497.

Motivation and limits

In a local billion-context 0.1.212 Pi/Codex session, one successful append-only delta request reported 18,479 input tokens and zero cached tokens. The effective-prefix evidence, unchanged settings, and existing connection logs did not identify a local rewrite or reconnect. Other 20 delta sends in the 26-request sample had nonzero hits.

The installed usage parser preserves absent-versus-numeric-zero usage, but the original terminal WebSocket frame was not retained. These diagnostics make the next recurrence easier to correlate without retaining raw payloads. They cannot prove upstream cache residency, routing, eviction, or correctness of provider usage reporting.

Privacy and wire fidelity

The new records contain only allowlisted counters, event types, numeric usage, booleans, and SHA-256 digests. They do not add raw prompts, reasoning, authorization/session headers, URLs, cache keys, checkpoint/response IDs, or error details to logs. In particular, the connection reuse key is never logged because it contains canonical headers.

Send records describe send attempts, not upstream acknowledgement or delivery. Counters are per transport instance; retain the existing outer session/connection log context.

Tests assert exact serialized egress and forwarded SSE bytes, not just parsed-object equivalence.

Performance

This is temporary diagnostic instrumentation, not a production cache optimization or a cache-miss fix.

The diagnostics are always on. They add no gate and no configuration.

Commit 30ee96bf removed the request_sha256 field. It was the only part that serialized the full effective request a second time, and it ran on every send, including delta sends. The remaining per-send work is:

  • frame_sha256: one SHA-256 over the frame string that sendCreate already builds. On a delta send, that string is the delta only.
  • settings_sha256: canonical serialization and SHA-256 of the non-input settings.
  • SHA-256 of the short previous_response_id and response id strings.

Synthetic benchmark of the three computations (Node v24.21.0, one machine, median of 15 runs, settings of roughly 90 KB). It was not measured in a live proxy.

Body size Removed request_sha256 frame_sha256 on a full send settings_sha256
2.3 MB 5.4 ms 1.3 ms 0.6 ms
8.8 MB 20.9 ms 5.2 ms 0.6 ms
34.9 MB 151.8 ms 39.0 ms 0.3 ms

The field was removed rather than truncated for these reasons:

  • On a full send it hashed the same text as frame_sha256.
  • On a delta send, ResponsesWsHistory.continuation() has already compared the settings and every prior input item, so mode=delta attests the same thing.
  • It changed every turn as the input grew, so it could not show prefix stability.
  • A hash of a bounded prefix could read "unchanged" when the tail changed.

One cost remains: frame_sha256 on a full send scales with the body size, as the table shows. Full sends happen on the first request, after a reconnect, and when a continuation does not apply.

No live proxy/session was modified or restarted, and no provider experiment was run.

Config surface

None: no new or changed config fields, environment variables, or CLI flags. Records use the existing diagnostic logger callback.

No persistence-format, wire-protocol, package-version, or kernel-source changes.

Verification

Refreshed onto upstream master 1ee687bc; checked head 1d59e828.

  • node node_modules/typescript/bin/tsc --noEmit --project tsconfig.json — passed.
  • npm run build — passed, including dist import/external checks.
  • Five targeted files — 74/74 passed:
    node --import ./tests/preload-storage-baseline.mjs --import tsx --test \
      tests/responses-ws-cache-diagnostics.test.ts \
      tests/responses-websocket.test.ts \
      tests/codex-responses-ws.test.ts \
      tests/responses-ws-http-fallback.test.ts \
      tests/usage-observability.test.ts
  • Eight new tests cover streaming/nonstreaming missing-versus-zero usage, malformed-value redaction, exact wire/SSE bytes, settings changes, deltas, reconnects, continuation recovery, all terminal event types, and exclusion of nonterminal usage.
  • git diff --check upstream/master...HEAD — passed.
  • Head 30ee96bf (after removing request_sha256): tsc --noEmit, the five targeted files (74/74), and git diff --check on that commit's changes were rerun and passed. npm run build was not rerun.

Targeted tests use the storage sandbox preload and mock upstreams. Full CI and live Codex-backend reproduction have not been run locally.

Category A — prefix-cache diagnostics.

Correlate exact egress fingerprints, socket generations and send attempts with terminal usage-field presence. Preserve reported numeric zero versus absent cache usage without logging raw payloads or headers. This adds diagnostics, not a provider cache-miss fix.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 检索了 open+closed 的 issue 与近期 PR(按主题 "Responses WebSocket cache diagnostics" / "cached tokens zero" / "cache miss"):没有其它在途的 WS 车道诊断 PR。与 #4328(Codex auto-review 短空闲低命中调查)、#5124(Codex 大规模 cache miss)是同症状家族但角色不同——那两个是未结的调查/报告,本 PR 是为该类问题补的脱敏观测手段,不修根因、不关闭它们,PR 描述里也明确声明了这一点;#5497 是 generation-boundary 误报的另一条线,与本 PR 无交互。判定:作为独立的诊断工具 PR 处理。

标签: codex scope/wiring p3(+系统自带 pr)— 诊断面落在 Responses WebSocket 出站边界(协议接线域),非缺陷修复故不打 bug;当前为 draft、无用户可见行为变更,定 p3。

审核进行中:已读完整 diff(2 文件 +203/-0,head 1d59e828)、逐段核对 src/responses-ws.ts 集成点与新测试文件,正在本地跑定向测试并核对 CI。结论会按八问体例跟进回复。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

PR Review(第一轮)— 方向、回归、代码、测试均过,无阻塞发现;剩一个 gating 决策 + CI 全量绿

① 方向(原 issue 关):#4328 / #5124 是两个未结调查,同一症状家族——Codex 车道偶发 cache hit 掉零,现有日志分不清「本地改写 / 重连 / 上游未命中」。本 PR 在 WS 出站边界补脱敏观测:socket generation + 单调 send 序列关联、frame/等价全量请求/settings/response-id 的 SHA-256 指纹、usage 三态语义(reported-zero / missing / malformed)。方向正确:先测量后修因、指纹而非留原始载荷、数据面零改动;「不关闭两个调查、不坐实 provider 侧原因」的声明诚实。

② 回归审查(既有行为逐项清点):

③ 代码审查:

  • requestHash 哈希的是原始 body(等价全量请求)而非 continuation 后的 request——full 模式 requestHash==frameHash(有测试钉),delta 模式给出等价全量指纹,语义正确。
  • settingsHash 复用既有 canonical()(排序键、去 undefined,即连接 key 的序列化器),key 序无关,一致性好。
  • connectionGeneration 在握手 resolve 前自增(失败尝试也计数)——作为关联计数器可接受。
  • terminal 事件列表与函数后半的逐字面枚举重复——既有风格本就多处重复这些字面量,抽常量会扩大 diff,不强求。
  • 诊断调用位于 message handler try/catch 内(logger 抛错会终止交换)——base 已有同类暴露(retry 路径日志同样在 try 内),且 context.log 是 bili 自身非抛 logger,非新风险类,不修。
  • 外部提交严标准(命名/死代码/魔法串/错误处理/风格一致/测试质量)逐项过。

④ 测试有效性:真 ws WebSocketServer + 真 ResponsesWsUpstream(被测单元不是 mock);断言是字节精确(egress rows deepEqual、SSE 字符串相等)而非解析对象等价;隐私断言扫全部日志找 prompt/reasoning/auth/query/session-header/id 泄漏;usage 矩阵覆盖 stream/nonstream × zero/missing/malformed;reconnect、continuation 恢复、settings 变更、非终态排除各有专例,负向(NOT trigger)方向钉住,符合 #3617 gate 纪律。是真测试:未来任何人改帧序列化,deepEqual rows 立刻红。

验证:本地 checkout head 1d59e828 实跑五个目标测试文件——74/74 通过,与 PR 声明一致。全量回归以远程 CI 为准(head 1d59e828 各 shard 正在排队/运行,结果出来我补一条)。

结构化八问

  1. 是什么问题:不是缺陷修复——为 [cache investigation] Codex auto-review: short-idle low hits despite a long stable outbound prefix #4328/Massive cache misses within Codex | codex 内 cache 大规模 miss #5124(Codex 车道 cache hit 掉零、现有日志无法归因)补脱敏关联观测手段。
  2. 什么原因导致:不适用(诊断工具);直接动因是 usage parser 已区分 missing vs numeric-zero,但原始 terminal WS 帧不留存,下次复发无法关联。
  3. 影响范围与严重程度:仅 debug 级日志内容 + 每轮 CPU(额外一次全量 body stringify + 2–3 次 sha256);无用户可见行为变化;内部实现 + 潜在性能成本(见决策项)。
  4. 怎么修复的:指纹而非留存——SHA-256 摘要(frame/等价全量请求/settings/response-id)+ 计数(socket generation、send 序列、含重试)+ usage presence flags,全部走既有 logger callback,数据面零改动。
  5. 是否有回归问题:无——序列化/转发字节/checkpoint/重试决策逐项核实未变且被字节精确测试钉住;唯一漂移是无条件哈希计算(性能维度,决策项)。
  6. 是否破坏缓存:否——缓存/重试/游标/去重策略一概未触碰,诊断记录纯只读观测。
  7. 是否改了设计方向:否——单 writer transport 实例模型不变,计数语义 per-transport-instance,外层 session/connection 日志上下文保留。
  8. 是否引入新配置:否——无新字段/env/CLI flag,复用既有 diagnostic logger。

🔶 需要决策(PR 为何 draft——作者明确请求)

  • 背景一句话:全量请求序列化 + 哈希目前无条件执行,即使 debug 输出关闭。
  • 待决问题:生产化前如何 gating。
  • 选项:
  • 推荐:B(作者按此小改即可出 draft)。不决定的默认行为:PR 保持 draft,不自动合并,等待裁决。

判定:代码与测试达标、无阻塞发现;待 (a) owner 对上方 gating 拍板、(b) 全量 CI 绿。CI 结果出来后我会补一条跟进。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

CI 跟进(head 1d59e828,全量已跑完):

  • 主测试矩阵(CI run 41651621504):25/26 shard 绿,唯一红 = test (windows-latest, 22, 2),失败用例 abort x proxy-openai-sse: client abort mid-stream destroys the upstream request(tests/wire-exit-matrix.test.ts,AssertionError upstream socket(s) must be destroyed after the client abort (closed=0),耗时 11.0s)。
  • 归因判定:与本 PR 无关的时序 flake。 该用例属 OpenAI SSE 车道的客户端中止断连路径,本 PR diff 只碰 src/responses-ws.ts + 新增 WS 诊断测试,未触及该路径任何代码;本地 Linux 同 head 实跑该文件 7/7 通过;master 今日最近一次全量 CI(9c97d756)为绿——不是继承红。同 shard 日志里还有并发测试的 SQLITE_BUSY 噪音,符合 Windows runner 负载下的 socket-teardown 竞态特征。
  • 14:42 前后那批 cancelled job 是重触发时旧批次被并发取消(cancel-in-progress),其替代批次全部绿(含三个 zero-token e2e 车道)。
  • 建议:合并前对该失败 job(或整个 CI run)做一次 re-run 即可;re-run 绿之后,本 PR 唯一的剩余阻塞点就是上一条评论里的 🔶 gating 决策(推荐 B:哈希计算挂既有 debug 开关)。

八问判定更新:第 5 问(回归)维持「无」——唯一红项已排除归因于本 PR;其余七问不变。

@xulongwu4
xulongwu4 marked this pull request as ready for review November 4, 2026 12:42

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

Reviewed end-to-end at head 1276d990a (base master@23a826d3788a). Scope is exactly 2 files vs the merge-base: src/responses-ws.ts (+36) and new tests/responses-ws-cache-diagnostics.test.ts (+167). No kernel/, no config surface, no persistence/wire/version changes — verified against the merge-base diff.

Correctness — verified, no defects.

  • Purely additive. Every hunk inserts observation before an existing statement without altering control flow: connectionGeneration++ after socket creation (src/responses-ws.ts:362), fingerprint consts before the exchange promise (:449-452), a send counter + debug log before socket.send (:468-475), and a terminal-usage block between receivedEvents++ and output.observe (:521-535). Request serialization, forwarded frames, checkpoint handling, retry decisions, and backpressure are untouched.
  • Counters are correctly scoped: monotonic per transport instance, never reset in close(), so generations/sends stay distinguishable across reconnects and continuation retries.
  • Presence logic is right: Object.hasOwn distinguishes absent vs present; tokens() (:525) maps non-finite/malformed → null, so reported-zero ≠ missing ≠ malformed — the exact distinction the motivation needs.
  • Privacy holds: logs emit only counters, enum event types, booleans, numeric usage, and SHA-256 hex digests; IDs are hashed, headers/URLs/payloads/connection-key never appear.

Wire fidelity — pinned by tests. The suite asserts the EXACT bytes the mock server receives equal the expected frames across full/delta/settings-change/reconnect, plus exact SSE bytes, so any future drift that changes egress bytes fails. Secret-scanning assertions (private-prompt / private-auth-key / … absent from logs) directly enforce the privacy claim. Real regression tests, not tautologies.

One substantive finding (no defect, but a design decision): the fingerprints are computed unconditionally, even when debug logging is off. Every /responses WS fetch now does a full-body JSON.stringify + SHA-256 (requestHash, :451), a canonical()+SHA-256 of settings (:452), and a SHA-256 of the frame inside sendCreate (:473). Cost scales with input size — and this project's normal case is MB-scale marathon contexts, so it is a recurring hot-path tax, not negligible. DiagnosticLog (:20) is an unguarded callback and the hashing runs before this.log(...), so no caller-side debug gate suppresses it. The author already raised this under "Performance / why draft" and asks to agree on bounded use / a gate / measured overhead before production — I am formalizing that as a decision below rather than deciding it here.

Minor: the body has a "Performance / why draft" section but the PR is not in draft state — either the framing is stale or it should be draft pending the perf decision. Non-blocking.

Verdict: code is correct, tested, and safe to ship functionally. I am not approving yet — the sole open item is the unconditional hot-path hashing, a gating/performance decision for the owner. Routed to the 🔶 需要决策 comment below; applying the needs-decision label.

@ranxianglei

Copy link
Copy Markdown
Owner

🔶 需要决策
(ai: vllm-qwen/qwen3.8-27b)

PR #5615 代码正确、测试扎实、无缺陷。唯一待你拍板:新增的缓存诊断指纹是「无条件」计算的——即使 debug 日志关闭,每个 /responses WebSocket 请求也会做一次全量 body 序列化 + 三次 SHA-256。这是作者自己在「Performance / why draft」里提出、请求进生产前定的事。诊断的价值恰恰在于捕捉「不会预告」的复发(#4328/#5124 的偶发前缀缓存 miss),所以「要不要一直开着 / 怎么限成本」是取舍,不该由我替你定。

八问(第 3/5 问按选项拆)

  1. 是什么问题:给 Responses WS 车道加脱敏缓存诊断(send/terminal 关联 + 帧/设置/用量指纹),用于下次偶发缓存 miss 定位,不是修 bug。
  2. 什么原因导致的:原终端帧没被留存,现有连接日志无法区分本地改写 vs 上游未命中,故补观测面。
  3. 影响范围与严重程度(按选项):
    • A 直接合入常开:所有 /responses WS 请求(codex 车道为主)每请求多一次全量序列化 + 3×SHA-256,成本随输入规模线性增长;本项目常态是 MB 级马拉松上下文,属持续热路径开销。功能/可用性/数据/安全零影响。
    • B 常开但限幅(只对长度受限载荷做哈希,保留小体积 settings_sha256):最坏 CPU 有上界,仍保持常开可抓突发复发;超大 body 的全量 request_sha256 精度略降。
    • C 新增开关门控(默认关):稳态零成本;但需 owner 批准新配置面(config surface 属 owner 设计区),且突发复发若未提前开启就抓不到,削弱取证目标。
  4. 怎么修复的(推荐 B 的思路):保留常开以保住「抓不预告复发」这一核心价值,把昂贵的全量 input 哈希改成对长度受限前缀哈希(settings_sha256 已能廉价覆盖「设置未变」这一关键信号),把最坏成本封顶;几行改动,不动架构、不加配置。
  5. 回归风险(按选项):A/B 均不改任何既有行为(纯观测,测试已钉死出站/SSE 字节),回归风险低;C 引入新配置默认值,需走配置面纪律并补 en/zh 文档。
  6. 是否破坏缓存:不涉及——不动 bili 自身任何缓存/重试/游标/去重逻辑(本 PR 只观测 provider 侧缓存用量)。
  7. 是否改了设计方向:无架构转向,仅新增一个观测面;真正的开放问题就是这个观测面的成本策略。
  8. 是否引入了新的配置:PR 本身没有(已核实)。只有选 C 才新增开关,那属 owner 门控的配置面变更。

选项对比

选项 稳态成本 能否抓突发复发 改动量 需 owner 批配置
A 常开(现状) 高(随上下文规模) ✅ 始终 0 否
B 常开+限幅(推荐) 低且有界 ✅ 始终 几行 否
C 开关门控(默认关) 零 ⚠️ 需提前开 中(含配置面) 是

请拍板 A/B/C(或直接给别的方向)。在你定之前我不 approve,PR 维持 blocked。我倾向 B:既保住取证价值,又把热路径成本封顶,且不加配置面。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) 审查完成 #5615(head 1276d99,base master@23a826d37):代码正确、无缺陷——出站/SSE 字节与脱敏隐私均被新测试钉死(精确字节断言 + secret 扫描);唯一开放项「诊断指纹无条件做全量 body 序列化 + 3×SHA-256 的热路径成本」已按 🔶 需要决策 在 GitHub 给出选项 A/B/C(推荐 B:常开+限幅),已加 needs-decision 标签(GitHub 已加;镜像本已带),未 approve,待 owner 拍板。CI 镜像缓存显示仍在跑(暂无失败)。注:workdir 无 add-label.sh,标签经 API 直接施加。

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

复审(新 head a9e07f7c5)— 本 PR 自身改动逐字节未变,维持「待决策、不 approve」

核对新 head:a9e07f7c5 是一条 “Merge branch 'master'” 提交——作者把分支重新同步到最新 master。相对上一轮我审的 head,本 PR 自己的两个文件(src/responses-ws.ts +36、tests/responses-ws-cache-diagnostics.test.ts +167)逐字节未变(git diff 为空);这次新增的只是从 master 合并进来的无关 dsh 变更(src/dsh-channel.ts 及若干 dsh 测试,现已在 master 上),不属于本 PR 贡献。对当前 master(merge-base c9ba4856)的真实 diff 仍恰好是这 2 个文件、纯新增 203 行——无 kernel/、无配置面、无 wire/持久化/版本变更。

结论沿用第一轮:代码正确、无缺陷——出站/SSE 的精确字节断言钉死了 wire fidelity,secret 扫描钉死了脱敏隐私,测试能兜住回归。唯一开放项不变:诊断指纹是无条件计算的——每个 /responses WS 请求即使 debug 日志关闭也会做一次全量 body 序列化 + 三次 SHA-256,成本随输入规模线性增长(本项目常态是 MB 级马拉松上下文)。这是取舍而非缺陷:作者自己在 “Performance / why draft” 里提出、要求进生产前先定策略。已在我上一条 🔶 需要决策 评论给出选项 A/B/C(推荐 B:常开但限幅),等 owner 拍板。

CI:镜像缓存显示新 head 的测试矩阵仍在排队/运行中(19 pending、0 failed、11 skipped),尚未全绿——此处不断言绿。

判定:仍 不 approve。这不是代码问题,而是待决策的热路径成本策略;needs-decision 标签保留。owner 定下方向后我再据此 approve,或按 B 把昂贵的全量 input 哈希改成长度受限载荷。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) 复审新 head a9e07f7c5:本 PR 自己的两个文件(src/responses-ws.ts + 诊断测试)相对上一轮逐字节未变,新增的只是一条 master 再合并(带入现已在 master 上的无关 dsh 变更)。结论沿用第一轮——代码正确、无缺陷(出站/SSE 精确字节 + secret 扫描钉死);唯一开放项仍是「诊断指纹无条件做全量 body 序列化 + 3×SHA-256」的热路径成本策略,维持 🔶 需要决策(A/B/C,推荐 B)、needs-decision 标签保留、不 approve。CI 镜像缓存显示新 head 矩阵仍在排队/运行(19 pending / 0 failed),未断言绿。

request_sha256 serialized and hashed the whole effective request a second
time on every send, including delta sends. A synthetic benchmark on
Node 24 put that at about 2.4 ms per MB of body, run synchronously.

The field added little. On full sends it hashed the same text as
frame_sha256. On delta sends, ResponsesWsHistory.continuation() has
already matched the settings and every prior input item, so mode=delta
attests the same thing. The hash also changes every turn, so it cannot
show prefix stability.

The remaining send fields are unchanged. The test now pins the exact
key set of the send record.
@xulongwu4

Copy link
Copy Markdown
Contributor Author

(ai: anthropic/claude-fable-5-2)

On the open A/B/C decision about the hashing cost: commit 30ee96bf reaches B's goal (always on, bounded cost, no config surface) by a different route. It removes request_sha256 instead of truncating it.

Cost. request_sha256 was the only field that serialized the full effective request a second time, and it ran on every send, including delta sends. Synthetic benchmark (Node v24.21.0, one machine, median of 15 runs, not a live proxy):

Body size Removed request_sha256 frame_sha256 on a full send settings_sha256
2.3 MB 5.4 ms 1.3 ms 0.6 ms
8.8 MB 20.9 ms 5.2 ms 0.6 ms
34.9 MB 151.8 ms 39.0 ms 0.3 ms

frame_sha256 hashes the frame string that sendCreate already builds. On a delta send that string is the delta only, so the full-send column applies only to the first request, reconnects, and sends where a continuation does not apply.

Why remove instead of truncate.

  • On a full send the field hashed the same text as frame_sha256.
  • On a delta send, ResponsesWsHistory.continuation() has already compared the settings and every prior input item, so mode=delta attests the same thing.
  • It changed every turn as the input grew, so it could not show prefix stability.
  • A hash of a bounded prefix could read "unchanged" when the tail changed.

What changed. 4 insertions and 4 deletions across src/responses-ws.ts and tests/responses-ws-cache-diagnostics.test.ts. The other send fields and the terminal record are unchanged. The test now pins the exact key set of the send record, so a reintroduced full-request hash fails it.

Verification on 30ee96bf. The edited test failed before the source change and passes after it. The five targeted files pass 74/74, and tsc --noEmit passes. npm run build and the full suite were not run locally. No CI run is visible for this head yet.

The PR body is updated to match: the fingerprint list no longer mentions the full request, and the "Performance / why draft" section is replaced with the numbers above.

The decision is still yours. If you prefer A, C, or a bound on frame_sha256 for full sends, this commit can be reverted or extended.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 已独立核验作者新提交 30ee96bf(当前 head 9967322371)——声明属实,改动范围干净:

核实结果

  • 真实 PR 范围(对当前 master 58a87962)仍恰好 2 个文件:src/responses-ws.ts +35、tests/responses-ws-cache-diagnostics.test.ts +168,共 +203;无 kernel/、无配置面、无 wire/持久化/版本变更。这次新增的巨量 diff 全是 master 自身推进被再合并带进来的,与本 PR 无关。
  • request_sha256 及其全量 body 二次序列化+哈希已删除;保留 frame_sha256(哈希 sendCreate 已构建的 frameText,delta 时极小)、settings_sha256(非 input,小)、previous_response_id_sha256 与整段 terminal 记录——与作者描述逐行一致。
  • 「删除优于截断」的理由成立:full send 时它与 frame_sha256 同值;delta 时 continuation() 已比对过全部前置 input+settings,mode=delta 已背书前缀稳定;且它随 input 增长每轮都变、本就无法显示前缀稳定。顺带说一句:这比我第一轮建议的「限幅截断」更对——截断版会在尾部变化时误读成「未变」。
  • 测试升级到位:新增断言钉死 send 记录的精确 key 集合(Object.keys(row).sort() = connection/frame_sha256/mode/phase/previous_response_id_sha256/send/settings_sha256),重新引入全量请求哈希会直接挂掉;精确出站字节、secret 脱敏扫描、present-flag 三态、重连 generation、continuation 重试→resend 关联等原有覆盖全部保留。

仍待你拍板的一点:去掉最贵字段后残余成本 = full send 时的 frame_sha256(作者基准 34.9MB≈39ms,但 full send 仅首包/重连/续传失败才出现,稳态 delta 很便宜)+ 小体积 settings_sha256。这是「常开(B) vs 门控(C)」的取舍,不是缺陷——按权限规则,这个成本策略只能由你(owner)定,我不 approve、也不把作者的自述当作你的决定。你确认 B(或改要 A/C/对 full-send 的 frame_sha256 再限幅)后我立即据此 approve 或让作者调整。

CI 镜像缓存:head 9967322371 仍在跑(8 pending / 0 failed / 22 done),未断言绿。

@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6482 (merged) — closing as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants