Skip to content

[PR] fix(codex-lane): context-only prewarm probe relay + hosted web_search_call ordering (#4594, #7296) - #7322

Open
ranxianglei wants to merge 2 commits into
masterfrom
2026-11-19_codex-lane-root-cause
Open

ranxianglei wants to merge 2 commits into
masterfrom
2026-11-19_codex-lane-root-cause

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Root-cause fix for the codex-lane batch (aggregation #7314): the two previously separate Codex Responses-wire defects, fixed in one PR per the root-cause aggregation contract.

#4594 — context-only prewarm ran as an ACP turn; probe relay bypassed the upstream proxy

src/responses-ws.ts only treated generate:false with an empty input as a prewarm probe. Codex's startup/idle-thread warmup sends generate:false plus its developer/user context items, so it fell through into the ACP pipeline as a zero-message turn: it deactivated every fold block (0 active blocks saved), re-injected the whole codexForgedSummaries list, and inflated the request with ACP tools. The probe relay also egressed on the bare transport, bypassing the upstream proxy that pipeline turns use.

Fix: a generate:false request that replays no conversation turn (empty input, or only developer/system/user messages) is now a probe — forwarded verbatim through the relay's upstream proxy, never run as a turn. History prewarms keep riding the pipeline. ws-bridge now exposes the resolved proxy (egressProxyFor) instead of a boolean so the tunnel guard and the probe share one decision.

Cherry-picked from #4595 (author z0d0dOr) with -x, rebased onto current master.

#7296 — ordering repair split a reasoning item from its hosted web_search_call

responsesToCore represents hosted call items as contentType: "reasoning"; repairResponsesAssistantOrdering mistook the hosted call for a second reasoning and inserted a synthetic user separator between the real reasoning and the call, so the Responses server rejected the web_search_call ("provided without its required 'reasoning' item").

Fix: a reasoning-content core whose rawResponsesItem sidecar carries a hosted-call wire type is classified as a tool-call (phase 2), keeping reasoning → hosted call in the same run. Separators still land only between distinct runs.

Cherry-picked from #7302 (author ci-check) with -x.

Rebase note (#4594 commit)

src/inline-ws.ts and src/inline-engine.ts postdate #4595's base and called installWebSocketBridge with the old isEgressProxied?: (url) => boolean signature. The rebase adapts them to the new egressProxyFor?: (url) => string | undefined signature (plus one fixture in tests/ws-bridge-pending-close.test.ts). No behavior change beyond what #4595 intended — the inline WS bridge's prewarm probes now egress through the upstream proxy too.

Behavior changes (intended, old → new)

  1. Probe recognition: generate:false + empty input → generate:false + no conversation turn (empty, or only developer/system/user messages).
  2. Probe egress: always direct → through the configured upstream proxy (same route pipeline turns use).
  3. Context-only prewarm no longer counted in stats.requests (parity with empty-input probes, which were never counted).
  4. reasoning → hosted web_search_call no longer split by a synthetic user separator (distinct runs still separated).

Verification

Fixes #4594
Fixes #7296

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-19_codex-lane-root-cause (e7b3a51)

Latest build — install this exact version

npm install -g billion-context@0.0.11114-pr.7322.1376053

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
e7b3a51 billion-context@0.0.11114-pr.7322.1376053 build
6da9871 billion-context@0.0.11000-pr.7322.da14ad6 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7322.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(codex-lane): context-only prewarm probe relay + hosted web_search_call ordering (#4594, #7296) [PR] fix(codex-lane): context-only prewarm probe relay + hosted web_search_call ordering (#4594, #7296) Nov 19, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Picked up — reviewing PR #7322 (root-cause aggregation fix for the codex-lane batch #7314: #4594 + #7296 in one PR).

Duplicate check: not a duplicate. This is the single consolidated fix PR for the codex-lane aggregation #7314. The earlier PRs #4595 and #7302 are labeled superseded (their commits live here via cherry-pick -x); both are still open on GitHub and will need owner close after this merges — I won't close them. No other open PR targets #4594 or #7296.

Labels: the source PR already carries bug, p1, codex, scope/wiring, root-cause (verified live on GitHub) — p1 driven by #4594 (fold blocks wiped on every codex warmup is a context-management failure), #7296 alone would be p2. The mirror's label endpoint is not implemented in the ework shim, so no mirror-side labels to add.

Review in progress: diff read, cherry-pick provenance, regression paths, CI lanes, then the structured verdict.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review verdict: mergeable — CI green on head 6da9871e232 (30 checks: 13 success, 17 skipped, 0 failed), verified live minutes ago. PR is mergeable=true, currently blocked only on review.

What I independently verified (not a re-read of the issue body):

  1. Direction (both fixes hit the root cause, not the symptom).
  2. Code (full diff read, 9 files, +219/−27).
    • isPrewarmProbe (src/responses-ws.ts:100-103): empty input is vacuously true (preserves the original bug: Responses degenerate retry overlooks custom tool calls and drops completed-attempt usage #1862 behavior); assistant/function_call/reasoning items stay on the pipeline (negative test present).
    • Probe egress: proxyDispatcher(egressProxyFor?.(upstreamUrl)) — proxyDispatcher(undefined) → undefined (direct, no-op); memoized per proxy identity (no per-probe agent churn); the callback resolves via the same resolveProxyDecision as pipeline turns (src/server.ts:825) and the tunnel guard.
    • ws-bridge signature change isEgressProxied: boolean → egressProxyFor: string | undefined: all 5 call sites updated (server.ts, inline-engine.ts, 2 test files); tunnel guard !== undefined is boolean-identical; zero remaining references to the old name.
    • Ordering repair: phase logic sound — [reasoning, hosted, hosted] stays one run; [hosted, reasoning'] still gets a separator (phase 2 > 0); real reasoning/function_call/message classification unchanged.
    • Sidecar survival: the kernel fold path performs no cloning (checked kernel/src/block-ops.ts, compress.ts, pipeline.ts); survivors keep object identity, so rawResponsesItem reaches the repair.
    • Host RESPONSES_HOSTED_CALL_TYPES exactly mirrors the kernel case list (kernel/src/wire/responses.ts:545-551).
  3. Tests (quality, not just green). Both fixes carry positive + negative gate pairs per Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617: context-only prewarm → probe (no block deactivation, no forged-summary re-injection) AND history prewarm stays on the pipeline; web_search_call keeps its run with the real reasoning AND a genuine second reasoning is still separated.
  4. Provenance. 4f648c25839 is byte-identical to [PR] fix: keep hosted web_search_call in the same run as its reasoning (#7296) #7302's 2a2a9e923ae (author ci-check preserved); 6da9871e232 = [PR] fix: route codex context-only prewarms to the probe relay via the upstream proxy #4595's cdda1b3b301 (author z0d0dOr preserved) + exactly the rebase adaptations documented in the PR body (src/inline-engine.ts, src/inline-ws.ts, one test fixture). Cherry-pick trailers present.
  5. Regression sweep (every touched path). Empty-input probe preserved; normal turns unchanged (generate===false gate); tunnel-guard boolean identical; inline WS bridge semantics preserved (its probes now also egress via the proxy — disclosed in the rebase note); the orphan-skip rule no longer drops a hosted call that is the sole survivor of a run that had a body (strictly an improvement — the old drop was a silent wire-item loss); both compression modes considered (this lane is proxy-mode territory; plugin mode untouched). No regressions found.
  6. Diff cleanliness. Every line relates to the two fixes; no whitespace churn, no unrelated files.

Behavior changes — all four disclosed in the PR body, all verified as intended (probe recognition widened; probe egress direct → proxy; context-only prewarm no longer in stats.requests; reasoning → hosted call no longer split).

Drive-by filed: #7327 — the host's hosted-call type set duplicates the kernel's responsesToCore case list, which the kernel does not export. If the kernel ever adds a hosted-call type, the host set drifts and #7296 recurs for that type (loud server rejection, not silent corruption). Issue carries options A (kernel export, gated) / B (parity canary test) with recommendation.

Post-merge note (for you): #4595 and #7302 are marked superseded but still open — per policy I don't close issues; please close them after merging this.


八问(中文结构化结论)

  1. 是什么问题:Codex Responses 线路上两个缺陷——(a) Codex 的上下文预热(generate:false + developer/user 上下文)被当成 ACP 回合执行:停用全部折叠块、重注入全部伪造摘要、请求被 ACP 工具膨胀,且 probe 中继绕过上游代理直连;(b) 顺序修复把被展平为 contentType:"reasoning" 的 hosted web_search_call 误判为第二段 reasoning,在真实 reasoning 与调用之间插入合成 user 分隔符,Responses 服务端拒绝该调用("provided without its required 'reasoning' item")。
  2. 什么原因导致的:(a) probe 判定只认 generate:false + 空 input,带上下文的预热落入流水线成为零消息回合;probe fetch 没有 dispatcher,走裸传输。(b) responsesToCore 把 7 种 hosted 调用统一展平成 contentType:"reasoning" core,而顺序修复只按 core 的 contentType 分类,没有看 rawResponsesItem sidecar 携带的线上类型。
  3. 影响范围与严重程度:(a) 所有配置了上游代理的 Codex 会话:每次启动/空闲预热都搅动折叠状态、膨胀请求、走错出口路由——p1(与源 issue 评级一致);(b) 使用 hosted web_search 等能力、且折叠边界落在调用附近的 Codex 会话:服务端拒绝回合——p2(需特定条件)。
  4. 怎么修复的:(a) probe 识别放宽为「generate:false + 不重放任何对话回合」(空 input 或仅 developer/system/user 消息),probe 与流水线回合共用同一个 resolveProxyDecision 解析出的上游代理(ws-bridge 从布尔改为暴露 egressProxyFor,隧道守卫与 probe 共用一个决策);(b) 带 hosted-call 线上类型的 reasoning 内容 core 按 tool-call(phase 2)分类,reasoning → hosted call 保持同一 run,分隔符仍只落在不同 run 之间。
  5. 是否有回归问题:逐路径审过——空 input probe 保留(vacuous every);重放历史的预热仍走流水线(负向测试钉住);正常回合不变(generate===false 门);隧道守卫布尔语义相同(!== undefined);5 个 bridge 调用点全部更新、无残留旧名;真实 reasoning/function_call/message 排序不变;孤儿跳过规则不再丢弃 hosted call(改进——旧行为是静默丢线上条目);两种压缩模式都考虑(本车道仅 proxy 模式)。未发现回归。
  6. 是否破坏缓存:否。缓存键、前缀字节、折叠状态格式均未动。probe 从此完全不触碰会话状态(之前它记录幻影回合、移动折叠锚点——那正是 bug)。stats.requests 不再计入上下文预热(已披露;与空 input probe 对齐)。
  7. 是否改了设计方向:否。probe 中继与顺序修复机制本身未变,只放宽了识别谓词与出口路由;ws-bridge 签名变更(布尔→代理字符串)是服务于单一共享决策的机械重构。
  8. 是否引入了新的配置:无新 env/config/默认值。

收工判定:可合并。 CI 在 6da9871e232 全绿,八问全部干净,drive-by 已立项(#7327);合并后请关闭已 superseded 的 #4595 / #7302。

ci-check and others added 2 commits November 21, 2026 01:33
)

responsesToCore flattens hosted calls (web_search_call, file_search_call, mcp_call, ...) into contentType:"reasoning" cores. The Responses ordering repair classified them as a second reasoning item and inserted a user separator between the real reasoning and the hosted call, splitting the assistant run — the server then rejects the web_search_call "without its required 'reasoning' item".

Classify a flattened hosted-call core (whose rawResponsesItem sidecar carries a hosted-call wire type) as tool-call (phase 2) so the reasoning -> hosted call run stays intact. The separator still lands between distinct runs.

Closes #7296

(cherry picked from commit 2a2a9e9)
…tream proxy (#4594)

A generate:false request that replays no conversation turn (empty input, or
only developer/system/user messages) is a prewarm probe: forward it verbatim
instead of running it as a zero-message ACP turn, which deactivated every fold
block and re-injected the whole forged-summary list. History prewarms keep
riding the pipeline. The probe relay now egresses through the same upstream
proxy the relay resolves; ws-bridge exposes the resolved proxy instead of a
boolean so the tunnel guard and the probe share one decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit cdda1b3)
@ranxianglei
ranxianglei force-pushed the 2026-11-19_codex-lane-root-cause branch from 6da9871 to e7b3a51 Compare November 20, 2026 17:34
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Merge conflict resolved — PR rebased onto current master and pushed.

What I did this run:

  1. Verified live state first (not from memory): PR head 6da9871e232 sat 2 commits above merge-base 6453feeeb84 while master had moved 61 commits since; the head carried no [ework:ci-held] marker, so no dead mid-session run to finish.
  2. Rebased 2026-11-19_codex-lane-root-cause onto origin/master (b10c1a0d4ee). Commit 1 ([Bug]: Codex Responses ordering inserts a user separator before a hosted web search call #7296 ordering fix) applied clean; commit 2 (Codex WS context-only prewarm runs as an ACP turn: folds wiped, all forged summaries re-injected; probe relay skips the upstream proxy #4594 probe relay) hit exactly one conflict — the import block of src/responses-ws.ts (master added sseEventData to the sse-util import; the PR added the proxyDispatcher import directly above it, so the hunks overlapped).
  3. Resolved as the union of both intents: kept import { proxyDispatcher } from "./upstream-proxy.js" plus the sseEventData-extended sse-util import. Everything else auto-merged; I read the merged region to confirm the PR's isPrewarmProbe predicate and the probe fetch's dispatcher: proxyDispatcher(...) argument now coexist with master's refactored warmup SSE read loop (sseEventData block parsing, decode/finalize reordering) — both mechanisms intact.
  4. Verified locally: npm run typecheck clean; targeted suites green — 126 tests, 0 failures across codex-responses-ws, issue564-responses-ordering, ws-bridge-pending-close, issue2627-turn-sep-coverage, issue5139-turn-separator-ranges, invariant-judge, fold-round2-shape, responses-ws-http-fallback, ws-bridge.
  5. Pushed: fetched again immediately before push (remote head unchanged — no concurrent work on the branch), then force-pushed the rebased head e7b3a5120dd. This is the sanctioned conflict-rebase force-push and the branch's final head, so it carries no [ework:ci-held] marker → it is this branch's one full CI pass.

Post-push live state (mirror cache, fetched this run): CI in progress on e7b3a5120dd — 13 checks in progress, 2 success, 15 skipped, 0 failures so far. I am not asserting green until the shards finish; the full-suite verdict is CI's.

Scope check: the diff vs master is identical in scope to the reviewed PR — same 9 files, +219/−27, no new lines introduced by the rebase. Both cherry-pick commits keep their original authors (z0d0dOr, ci-check) and -x trailers.


七问(中文结构化结论)

  1. 是什么问题:PR [PR] fix(codex-lane): context-only prewarm probe relay + hosted web_search_call ordering (#4594, #7296) #7322 与 master 产生合并冲突(merge state dirty),阻塞合并。
  2. 什么原因导致的:PR 基于 6453feeeb84 开出,此后 master 前进了 61 个提交;其中 SSE 工具重构(sse-util 新增 sseEventData、warmup 读循环重写)与本 PR 在 src/responses-ws.ts 的 import 区相邻改动,产生唯一一处文本冲突。
  3. 影响范围与严重程度:仅阻塞本 PR 合并,不影响任何运行时行为;冲突面 = 1 个文件 1 处 import 块。
  4. 怎么修复的:rebase 到当前 master(b10c1a0d4ee);冲突按双方意图并集解决(保留 proxyDispatcher import + 带 sseEventData 的 sse-util import),其余改动(isPrewarmProbe、probe fetch 的 dispatcher: 参数)与 master 的 SSE 循环重构自动合并,已逐行核对合并后区域。
  5. 是否有回归问题:无。typecheck 干净;9 个定向测试套件 126 例全绿(含 PR 自带的正/负门控回归测试);对 master 的 diff 与已审核版本完全同范围(9 文件 +219/−27,rebase 未引入任何新改动行)。
  6. 是否破坏缓存:否——本次只动了 import 合并与分支历史,未触碰任何缓存/前缀/折叠逻辑。
  7. 是否改了设计方向:否。两个 cherry-pick 提交保留原作者与 -x trailer,行为与已审核版本一致;无新增配置项。

收工判定:冲突已解、最终 head e7b3a5120dd 已推送,CI 在新 head 上运行中(当前 0 失败);全量结果以 CI 为准,若红本车道(attempt 2/6)会再次处理。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants