Repository navigation
[PR] fix: Codex/opencode WebSocket delta turns no longer rebuild the full history before compression (#4804) - #4817
Conversation
- export ResponsesWsSession so the onMessage admission seam is testable directly (same seam as the issue's offline repro harness) - tests/issue4804-ws-admission-boundary.test.ts expresses the acceptance behavior: full replay above the old 100 MiB cap reaches dispatch; small previous_response_id continuation over the reconstructed cap reaches dispatch; checkpoint accounting retains a completed turn whose combined size crosses the old cap. Verified to fail on unfixed code with the exact production signatures (request_too_large 413, previous_response_not_found). Controls (small turns pass, foreign id refused 400) stay green.
…4804) The Responses WebSocket lanes (codex-responses-ws, responses-ws) measured the RECONSTRUCTED full history against a hardcoded 100 MiB cap before dispatch ran, so a long session whose processed outbound was far below the cap died with 413 request_too_large on the next turn even when the client sent only a few MiB of delta; checkpoint accounting shared the same cap and cleared checkpoints near it, stranding continuations with previous_response_not_found. #3202 made the HTTP budgets configurable but explicitly left the WS surface fixed at 100 MiB. - ws maxPayload, reconstructed-body admission, upstream frame/backpressure/ output guards and checkpoint accounting all resolve limitsReceiveBytes() (BILI_RECEIVE_BODY_BYTES > network.receiveBodyBytes > 5 GiB) - zero new config fields - one full serialization per turn: admission measures the same wire string the pipeline envelope pushes (was two) Behavior changes (old -> new, defaults): - WS wire frame cap 100 MiB -> 5 GiB - WS reconstructed-body admission cap 100 MiB -> 5 GiB - checkpoint retention cap 100 MiB -> 5 GiB (near-cap checkpoints survive) - 413 message names the live budget instead of a constant "100 MiB" Restore the old value with BILI_RECEIVE_BODY_BYTES=104857600 (this also sets the HTTP receive budget, as it always did).
…4804) The codec comment claimed codex "never sends previous_response_id", but the #4804 real-session capture shows another codex-family build sending previous_response_id with a tiny input tail as its normal operating mode. Scope the claim to the 0.147.0 capture it came from; both shapes are live production traffic.
…help-merge sweep)
master window commits 293ae65/de5e03659 bumped kernel/package.json to 0.0.165 without regenerating the lockfiles; lock-sync re-resolution then rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and fails. Pure recorded-version sync via npm install --package-lock-only; no dependency or semantic change.
…ry reconstruction (#4804) A previous_response_id continuation used to expand into the complete reconstructed history before admission/processing: every turn re-materialized all retained originals (O(N) JSON rebuild plus repeated serialization), and once the reconstruction crossed the byte budget the turn was refused with 413 before ACP ever ran — stranding sessions whose model view had already been compressed to <1 MiB. The client's actual wire frame was tens of KiB; the cost was entirely bili's internal expansion. The delta now rides the unified transcript: - On an eligible continuation (matching checkpoint id, unified storage readable, all prior outputs echoable), the session dispatches the small body [prior outputs, new input] instead of the O(N) reconstruction and stamps the envelope with the commit-time identity signature of the dispatched body's last core message (non-enumerable symbol property). - prepare-responses renders the outbound prefix from the server's own transcript rows (buildAnchoredView: same loader/anchor machinery as snapshot-render, no chain walk — the client vouches for the prefix by protocol) and appends the delta tail verbatim. - The seam is verified against the stored row's identity signature; any mismatch (or missing store / non-echoable output / rollout off) falls back to the legacy full-reconstruction path, and an engine-side seam miss answers 400 previous_response_not_found — the client's standard recovery (full replay) re-anchors losslessly. Nothing is dropped: the store keeps every original. - Checkpoints chain (base/tail/tailSig) so steady-state turns grow memory O(1); the legacy expand() walks the chain only on fallback. - Echoing the prior outputs into the delta body also gets them ingested into the unified transcript (rows are request-side only), keeping the server-rendered prefix complete across restarts. Behavior changes (disclosed): - New rollout feature wsDeltaContinuation, DEFAULT OFF (0%): zero behavior change until ramped; force on via BILI_ROLLOUT_WS_DELTA_CONTINUATION=1 or rollout.features.wsDeltaContinuation. - When ON and eligible: the upstream request for a delta turn carries the server-rendered folded prefix plus echoed outputs plus the tail instead of the client-resent full history (removes the internal O(N) materialization and its 413 window). - A new 400 previous_response_not_found can be returned on a delta turn when the transcript seam cannot be verified; the client recovers with a full replay (no data loss). Tests: tests/issue4804-ws-delta-anchor.test.ts (engine/history/session units, both gate directions) and tests/issue4804-ws-delta-e2e.test.ts (real proxy + fake WS upstream: delta keeps the conversation without resending it; unknown checkpoint -> 400 -> lossless replay).
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6271-pr.4817.cde034aThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4817.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复——本 PR 是 #4804(WS 车道长会话在压缩前被 100 MiB 拦死)的根治 Part 2,与 #4808(同 issue 的止血 PR,仍 open)是「根治 supersede 止血」关系,非重复立项;按 PR 描述合并本 PR 时应关闭 #4808。另有 #4788/#3643 追踪 RSS/耗时基线,属后续验证跟踪,不重复。 标签: bug / codex / opencode / scope/wiring / p1(缺陷修复;涉及 Codex 与 OpenCode V2 两条 WS 车道;改动面在请求管道/WS 桥接层;底层缺陷会让长会话续轮直接 413 死掉、会话搁浅,属"不能用"级)。 状态: 已拉取 head |
(ai: vllm-qwen/qwen3.8-27b) 查重: 同机制已登记,非新问题——系统报的 kernel-guard 红落在旧 head ① 是什么问题:CI 门禁 kernel-guard 红——PR diff 因 sweep 带入的 lockfile 版本同步提交( ② 什么原因导致的:Kernel Boundary 规则「任何触碰 ③ 影响范围与严重程度:仅 CI 门禁级,无用户面影响;只影响本 PR 的可合并判定,不影响任何已发布行为。 ④ 怎么修复的:零代码改动——PR 正文补 ⑤ 是否有回归问题:无。剩余 10 个 test 分片红经逐日志核对全部为继承性 born-red(master/ci-fixes 同款,哨兵正在 ci-fixes 上修,#4599 floors 270/271):#1492 B preflight ×3、#847 exhaustion ×3、#4125 give-up reason ×3、storage-engine-proofs full-mode restart(google)×3、storage-render golden(google)×3;另 1 条 Windows EBUSY teardown flake(issue4732 测试自己的 rmSync 撞 SQLite 锁,与本 PR diff 零交集)已登记 #4599 待哨兵裁定。按 CI 协调协议不在本分支修 master 级失败;ci-fixes 收敛后 re-merge 即绿(或等哨兵 help-merge sweep)。 ⑥ 是否破坏缓存:不涉及(本轮零代码改动)。 ⑦ 是否改了设计方向:不涉及。 ⑧ 是否引入了新的配置:不涉及(rollout 开关 判定:本 lane 收工——唯一可操作项(kernel-guard)已修复并实测绿,无需再推新 commit;剩余红全部继承/环境因素,等哨兵在 ci-fixes 收敛后本分支 re-merge 即转全绿,届时可请 owner 合并。 |
(ai: vllm-qwen/qwen3.8-27b) 查重: 与 attempt 1 同一继承集群,非新问题——本轮是同一批 10 个 test 分片的重跑结果(新 job,09:09Z 起跑),失败签名与首轮日志审计逐条一致,确认确定性继承而非 flake 风暴;ci-fixes 仍为 ① 是什么问题:10 个 test 分片红 = master/ci-fixes 继承的 born-red 集群 + 1 条 Windows teardown 缺陷。 ② 什么原因导致的:全部失败源自 master 上既有的代码/测试(preflight 措辞漂移钉死 #1492/#847/#4125、storage golden 漂移家族 ×2),与本 PR diff 零交集(本 PR 只动 responses-ws / snapshot-render / prepare-responses / ws-bridge / knobs / rollout / persist 纯 getter)。其中 ③ 影响范围与严重程度:仅 CI 门禁级,无用户面影响;只影响本 PR 的可合并判定。 ④ 怎么修复的:本轮零代码改动——按 CI 协调协议(2026-10-24),master 级失败不在此分支修,统一收敛 ci-fixes(哨兵已接管,#4599 floors 270/271);本分支已含当前 ci-fixes tip,修复落笔后 re-merge(或等哨兵 help-merge sweep)即全绿。不开 decision-request issue:这些失败无一需要设计裁决,且 owner 政策明令禁止为 CI 问题开新 issue,状态一律在 #4599 汇报(本 lane 两条评论均已留痕,非静默等待)。 ⑤ 是否有回归问题:无。本 PR 自身车道全绿(kernel-guard ✅、lock-sync ✅、unused-exports ✅、static-gates ✅×2、全部 guard ✅、e2e workflow ×13 ✅、新增 #4804 套件 30/30 ok);重跑未引入任何新失败签名。 ⑥ 是否破坏缓存:不涉及。 ⑦ 是否改了设计方向:不涉及。 ⑧ 是否引入了新的配置:本轮零改动。一项依赖披露:kernel-guard 目前靠 PR 正文的 判定:auto-fix lane(attempt 2/2)收工——本分支无可操作项(全部红为继承/环境因素,修复所有权在 ci-fixes 哨兵);#4814 + 哨兵修复落 master/ci-fixes 后 re-merge 转全绿,届时请 owner 合并。 |
…4-ws-delta-continuation # Conflicts: # kernel/package-lock.json # package-lock.json
…4-ws-delta-continuation
…k so the 400 full-replay recovery is reachable (#4817) The #4804 recovery branch in responses-ws.ts answers a seam-miss delta with the client's standard 400 previous_response_not_found frame and clears the checkpoint chain, but it only fires when dispatch lets the SnapshotAnchorFallbackError through. server.ts's dispatch catch used to swallow every error into a generic 503 — the 400 branch was dead code in production (the unit test stubs dispatch to rethrow directly), the checkpoint stayed live, and the client retried the same deterministic miss forever. Rethrow for exactly the request the WS session stamped (WS_ANCHORED_REQ_PROP); plain-HTTP lanes keep the 503 shape. Adds the end-to-end happy-path pin: a delta continuation anchors through the REAL dispatch and completes without any error frame. The miss path itself stays unit-pinned; driving it through the real pipeline needs a same-session side-lane primitive the harness lacks (a second WS peer's transcript rows are invisible to the main peer's snapshot view) — recorded in the PR thread.
|
(ai: zhipuai-lb/glm-5.3) 跟进:审查发现的「400 恢复分支是死代码」已修复并推到本 PR 分支(4baf5b3)。 修了什么:server.ts 的 dispatch catch 把所有错误吞成通用 503,导致 #4804 里精心写的「锚点失配 → 400 previous_response_not_found → 客户端全量重放」恢复分支在生产永远走不到(单测是靠 stub dispatch 直接 rethrow 才测到的)。现在对 WS 会话打过标记的请求(WS_ANCHORED_REQ_PROP)原样 rethrow,普通 HTTP 车道保持 503 形状。 测试:新增端到端 happy-path 钉(增量续写经真实 dispatch 锚定成功、无任何 error 帧);失配路径维持单测钉。想把失配路径也做成真管线 e2e 时发现测试基建缺口:同一 session id 的第二个 WS peer 写的 transcript 行,主 peer 的快照视图看不到(每连接独立会话上下文),且轮间折叠发生在下一次 dispatch 内部(在 seam 检查之后)——两种确定性失配手段都构造不出来。这个缺口先记录在案,不阻塞本 PR。 WS 家族回归 30/30 绿,typecheck 干净。 |
(ai: vllm-qwen/qwen3.8-27b) CI lane 确认(head 八问(本轮为 CI 状态确认,零代码改动): **判定:分支全绿,可合并。**请 owner 合并;按 PR 正文约定,#4808 在本 PR 合并后冗余,可一并关闭。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.871-master.e91e113 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Part 2 of 2 for #4804 (root fix). Supersedes PR #4808: this branch already contains its stopgap commits, so merging this PR makes #4808 redundant — please close #4808 when merging this one.
What was broken
A
previous_response_idcontinuation expanded into the complete reconstructed history before admission and processing: every turn re-materialized all retained originals (O(N) JSON rebuild plus repeated serialization), and once that reconstruction crossed the byte budget the turn died with 413request_too_largebefore ACP compression ever ran — stranding sessions whose model view had already been compressed to <1 MiB. The client's actual wire frame was tens of KiB (real capture: 37,739 B); the cost was entirely internal expansion.How it works
The delta rides the unified transcript:
src/responses-ws.ts): on an eligible continuation (matching checkpoint id, unified storage readable, all prior outputs echoable), dispatches the small body[prior outputs, new input]instead of the O(N) reconstruction, and stamps the envelope with the commit-time identity signature of the dispatched body's last core message (non-enumerable symbol property, zero wire pollution).src/server/prepare-responses.ts+buildAnchoredViewinsrc/server/snapshot-render.ts): renders the outbound prefix from the server's own transcript rows — same loader/anchor machinery as snapshot-render, no chain walk (the client vouches for the prefix by protocol) — and appends the delta tail verbatim.previous_response_not_found— the client's standard recovery (full replay) re-anchors losslessly. Nothing is dropped: the store keeps every original.base/tail/tailSig) make steady-state turns grow memory O(1);expand()walks the chain only on fallback. Echoing prior outputs into the delta body also gets them ingested into the unified transcript (rows are request-side only), keeping the server-rendered prefix complete across restarts.Behavior changes (disclosed)
ws-bridge.tsmaxPayload)limitsReceiveBytes()(default 5 GiB,network.receiveBodyBytes/BILI_RECEIVE_BODY_BYTES)responses-ws.ts)wsDeltaContinuation, DEFAULT OFF (0%)BILI_ROLLOUT_WS_DELTA_CONTINUATION=1orrollout.features.wsDeltaContinuation)previous_response_not_foundwhen the transcript seam can't be verifiedWith the rollout OFF (the merged default) the only effective changes are the first four rows — identical to PR #4808. Eligibility additionally requires
storage.unified: "full"(shadow mode reads return null; no 400-chatter).Config surface
No new config fields. Reuses
network.receiveBodyBytes/BILI_RECEIVE_BODY_BYTES(Part 1) and the existing owner-gated rollout registry (new entrywsDeltaContinuation, percent 0). Env vars:BILI_ROLLOUT_WS_DELTA_CONTINUATIONfollows the establishedBILI_ROLLOUT_*convention.Item-ID evidence (thread floor 8)
v1 is deliberately client-ID-independent: the anchor is a server-side identity signature computed at commit time on the exact dispatched body (same converter both sides ⇒ byte-exact parity). The reporter's stability data (10/10 items across 17 replay comparisons; but 8/128 assistant messages omitting
idin the natural thread) confirms ID-keying cannot carry alone — recorded here as future hardening, not implemented.Tests
tests/issue4804-ws-delta-anchor.test.ts(new, 14): engine units (happy/covered/mismatch/no-probe/empty/unparsable), chained-checkpoint expand both directions + over-budget clear, session-level gate matrix (eligible delta / 400-recovery / rollout-off legacy / non-echoable-output legacy), tailSig determinism. Both gate directions (Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617).tests/issue4804-ws-delta-e2e.test.ts(new, 2): real proxy + fake WS upstream, unified storage enabled — delta turn reaches upstream carrying prefix + echo + tail although the client resent none (marker-order assertions); unknown checkpoint → 400 → full replay recovers losslessly.kernel-change: none — this branch makes zero changes under kernel/ (an earlier lockfile conflict from a swept ci-fixes sync was resolved by taking master's kernel 0.0.166 regeneration verbatim).
Review notes (§7.4)
Client↔bili wire shape unchanged; bili→upstream content is what compression already produced. No config schema, persistence format, or kernel changes. Touched the session/compression-judgment path with a new rollout-gated behavior, so this stays a human merge decision.