Skip to content

fix: extend strict-echo repair to the Responses wire (#1479) - #1482

Merged
ranxianglei merged 3 commits into
masterfrom
2026-09-27_responses-strict-echo
Sep 27, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-09-27_responses-strict-echo

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

#762's strict-echo repair was chat-completions-only:

  • normalizeStrictEchoReasoning ran in prepareOpenai only;
  • normalizeStrictEchoBody (the loop re-request path) guarded on Array.isArray(body.messages) and no-opped on Responses bodies (input[]);
  • prepareResponses never normalized at all.

So after a compress fold, a Responses-wire request whose assistant run lost its reasoning item went out un-repaired and strict-echo gateways rejected it with reasoning_content_missing (issue log: 7/7 rejections were round-2 re-requests right after a fold; strict-echo: injected fired 0x while reasoning-pair-violated fired 798x). Meanwhile the #684 canary used an immediate-precedence check that also flagged healthy [reasoning, message, function_call] turns — most of that 798x noise.

Changes

  • src/strict-echo.ts — new normalizeStrictEchoResponsesInput(input, enabled, log, sessionId): walks the flat input as assistant runs (reasoning / message(assistant) / function_call / custom_tool_call / function_call_output); for each run carrying ≥1 tool call and 0 reasoning items it injects one blank {type:"reasoning", summary:[{type:"summary_text", text:""}]} item at the run start. Same gate as the chat path: isStrictReasoningEcho (learned 400 flag or deepseek origin/model) and at least one reasoning item present anywhere (thinking-session criterion — non-thinking sessions stay byte-identical). No-op returns the same array reference.
    normalizeStrictEchoBody now patches input[] as well as messages[], so the single loop call site covers both wires.
  • src/server.ts prepareResponses — repairs rebuiltInput before forwarding, gated by the same isStrictReasoningEcho(session, upstreamOrigin, modelIdOf(parsed)), ordered ahead of the canary so a repaired body cannot self-fire its own sentinel (same rationale as the DeepSeek thinking mode: residual "reasoning_content must be passed back" 400 on the first post-fold request (0.1.109 / acp-kernel 0.0.69) — the forwarded body is not persisted #762 comment in prepareOpenai).
  • src/loop/core.ts — every buildRequest body now goes through withStrictEchoRepair, including the truncation-continuation and degenerate-turn retries, which were never normalized even for the chat wire (same root cause, closed here).
  • Canary warnResponsesReasoningPairs — rewritten from immediate-precedence to run-based counting (a run with tool calls but no reasoning = violation), reporting the orphaned function_call ids per issue option 3.

Behavior changes (old → new)

  1. Responses-wire outbound bodies change when strict-echo applies (learned flag or deepseek origin/model): a blank reasoning item is inserted at orphan run starts. This is the fix — previously those bytes were absent and the gateway 400'd. Non-thinking sessions and non-strict origins are untouched (no-op path verified by tests).
  2. reasoning-pair-violated sentinel fires LESS often: old = every function_call not immediately preceded by a reasoning item (flagged healthy [reasoning, message, function_call] turns — the issue's chronic-noise complaint); new = runs with tool calls but no reasoning item at all. Log prefix kept for grep continuity; message format changed to name the offending call ids.
  3. Loop retry paths (truncation/degenerate) are now strict-echo-normalized too — previously unnormalized for every wire. Only observable when the flag is armed AND a retry fires with an orphan run; otherwise byte-identical.

Config surface

None. No config fields added/renamed/re-semantics'd; the existing isStrictReasoningEcho gate (learned flag + deepseek statics) is reused unchanged.

Tests

  • tests/reasoning-echo.test.ts: unit coverage for the new injector (disabled/no-reasoning/healthy → same array ref; orphan run → exactly one blank at run start, all other items reference-equal, input not mutated; multi-orphan; normalizeStrictEchoBody on input[]), plus the canary tests rewritten for run-based semantics (healthy turns silent, orphaned runs warn + named).
  • tests/loop-strict-echo-responses.test.ts (new): full arc through the real proxy on the Responses wire — request 1 ships the orphan run and 400s (sentinel fires once, pre-repair), the 400 learns the flag, request 2's main-path forward carries the blank echo, and the loop re-request (whose compress ack pair forms its own run) carries it too; asserts zero orphan runs in both repaired bodies.

Pre-flight on this head: npm run typecheck clean · npm test 2918 pass / 0 fail / 12 skip · npm run build OK (dist-import-annotations OK).

Fixes #1479

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-27_responses-strict-echo (b084912)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1426-pr.1482.3b0e557

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1482.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Picking up this PR review.

Fetched 2026-09-27_responses-strict-echo — single commit 34b2970f on top of current master, no version bump (correct for a content fix). Scope:

  • src/strict-echo.ts (+80): new normalizeStrictEchoResponsesInput + normalizeStrictEchoBody now patches input[]
  • src/server.ts (+56): prepareResponses repairs rebuiltInput before forward
  • src/loop/core.ts (+20): every buildRequest body routed through withStrictEchoRepair
  • tests/reasoning-echo.test.ts (+161), new tests/loop-strict-echo-responses.test.ts (+219)

Running the full pass now: original-issue (#1479) direction check → regression inventory on the touched paths → code/invariant review → test-quality check → local pre-flight (typecheck/test/build). Will report a per-behavior verdict + merge recommendation next.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #1482 审查结论:可以合并(需人工合并,不适用 auto-merge)

① 方向(原 issue #1479) — 根因确认成立。#762 的 strict-echo 修复只覆盖 chat-completions(messages[]),Responses 线(input[])完全没有修复路径:normalizeStrictEchoBody 对 input[] 是 no-op、prepareResponses 从不做归一化、loop 重请求同样是盲区。fold 后 assistant run 丢失 reasoning item → strict-echo 网关 400 reasoning_content_missing。方案(选项 1+3:加 Responses 修复孪生 + 哨兵改 run-based)打到核心而非治标。✓

② 回归审查(逐项既有行为 + 结论)

  • chat 路径(normalizeStrictEchoReasoning / prepareOpenai):未改动,字节级不变。✓
  • normalizeStrictEchoBody:chat messages[] 分支逻辑与原先完全一致;仅新增 input[] 分支;无 patch 时仍返回同一对象引用(no-op 契约保持)。✓
  • prepareResponses:修复在转发前执行且受 isStrictReasoningEcho 门控;非 strict origin/model 或非思考会话 → 返回原数组,出站体字节不变。✓
  • 哨兵 warnResponsesReasoningPairs:改 run-based,触发更少(旧 immediate-precedence 把健康 [reasoning,message,function_call] 轮全标了 = 那 798× 噪声的来源)。仅日志不改线;前缀 reasoning-pair-violated: 保留,grep 连续。细微点:现在也统计 custom_tool_call(旧只算 function_call)——在"纯 custom_tool_call orphan run"这一窄场景会比旧版多触发一次,但那是旧版漏掉的真实违规,且与修复端一致,属更准确而非回归。✓
  • src/loop/core.ts:四个 buildRequest 站点(主重请求 + truncation/degenerate/no-thinking-retry)统一走 withStrictEchoRepair;门控关闭时 no-op。补齐了之前只有主重请求被归一化的盲区。✓
  • 两种压缩模式都受益(修复在 wire-body 层,与 pluginMode 无关;插入的是 reasoning item,不与 plugin 模式的 tool/nudge 抑制冲突)。✓

③ 代码审查

④ 测试有效性

  • 单测有牙:健康 [reasoning,message,function_call] 断言 0 行日志(正是误报回归的复现)、多 call run 静默、orphan 点名 call_id、双 orphan 断言精确文案 3 tool-call item(s) in 2 assistant run(s) + 三个 id、非思考会话静默。注入器:disabled/no-reasoning/healthy 均返回同引用;orphan 断言恰好一个空 reasoning 且在 run 起点 + 其余项引用相等 + input.length===7(不可变)+ 日志匹配;body 包装器分别处理 messages[]/input[]。✓
  • 新 e2e tests/loop-strict-echo-responses.test.ts:hermetic 全流程(进程内假 upstream + 真 proxy),驱动完整生命周期——req1 带 orphan 发出未修(标志未置位)→ 400 → learner 置位 → req2 主路径带空 echo(orphanRunCount===0)→ compress tool 触发 loop → 重请求也带空 echo。杀手断言:reasoning-pair-violated 恰好触发 1 次(仅修复前),同时钉死"误报减少"+"修复后 body 不会自触哨兵"。我单独跑过,确认真实 PASS(不是 skip)。✓

⑤ diff 整洁度:5 文件全部与本 PR 目的直接相关,无跑题文件、无生成物/锁文件连带、无大段空白重排;无 version bump(content 分支正确);config surface 零变更。✓

本地 pre-flight(PR head 34b2970f):typecheck ✓ · 2918 pass / 0 fail / 12 skip ✓ · build ✓(dist-import-annotations: OK,index.js 仍依赖-free)。

新鲜基座验证(master 已前进 5 个 commit,含并发的 #1474 adapter-responses.ts ingress-id-scope 改动):我在临时分支把本 PR rebase 到当前 master tip(v0.1.160)——rebase 干净无冲突,rebase 后 HEAD 上 typecheck ✓ · 2919 pass / 0 fail / 12 skip ✓(比原来多 1 个是 #1475 的新测试,与本 PR 两个新测试共存全绿)· build ✓。两处 Responses 管线相邻但正交(#1474 在 ingress 只重写 msg-proxy-* 超长 id、保 provider rs_*/fc_* 字节不变;本 PR 是出站注入无 id 的空 reasoning item),合成无碍。

行为变更披露(策略要求)

  1. Responses 出站体在 strict-echo 生效时于 orphan run 起点插入空 reasoning item(旧:缺失→400;新:存在)。非 strict/非思考会话完全不变。
  2. reasoning-pair-violated 触发更少(见②)。
  3. loop 三条重试路径现也被 strict-echo 归一化(旧:这些路径在所有 wire 上都未归一化)。仅在标志置位且该重试带 orphan run 时可观察。

自动合并判定:按 §7.4,此 PR 属跨两种压缩模式的 wire/message-shape 变更(向出站 Responses body 注入 item)→ 必须人工合并,不适用 auto-merge。但所有检查在新基座上全绿、diff 干净、引用 Fixes #1479、测试充分——可合并。

遗留/已知限制(非 bug,不另行立项):空 reasoning 对 DeepSeek thinking 是被接受的形式(hermes-agent #15527);若未来出现要求回传真实 reasoning 内容而非"存在性"的网关,空 echo 不满足——但没有实证案例,且 learn-on-400 只对已因缺 reasoning 而 400 过的网关生效,故不构成本 PR 缺陷。选项 2(fold 保留 provider 真实 reasoning)是更重的 kernel-view 手术,选项 1 已彻底解决报告的 400,维持不实现。

需要的话我可以把已验证的 rebase 版本 force-push 到 2026-09-27_responses-strict-echo 以保线性历史;否则当前分支无冲突,可直接合并。

中文摘要:修复了 strict-echo 修复缺失 Responses 线(及 loop 重试路径)导致的 fold 后 reasoning_content_missing 400,并把哨兵从误报的 immediate-precedence 改为 run-based;代码、测试、以及在最新 master 上的 rebase pre-flight 全部通过,可以(人工)合并。

#762's blank-echo injection only covered chat-completions bodies: normalizeStrictEchoReasoning ran in prepareOpenai only, and normalizeStrictEchoBody no-opped on Responses bodies (input[]). After a compress fold, a Responses-wire request whose assistant run lost its reasoning item was rejected by strict-echo gateways with reasoning_content_missing, while the #684 canary simultaneously fired on healthy [reasoning, message, function_call] turns (immediate-precedence check).

- strict-echo.ts: new normalizeStrictEchoResponsesInput injects one blank {type:"reasoning", summary:[{type:"summary_text", text:""}]} item at the start of each assistant run carrying a tool call but no reasoning item (same gate as the chat path: isStrictReasoningEcho + at least one reasoning item present); normalizeStrictEchoBody now patches input[] as well as messages[].
- server.ts prepareResponses: repair rebuiltInput before forwarding, ahead of the canary so a repaired body does not self-fire it.
- loop core.ts: every buildRequest body (main re-request plus the truncation/degenerate retries, which were never normalized) goes through the same gate.
- canary warnResponsesReasoningPairs: immediate-precedence -> run-based counting, reporting the orphaned function_call ids. Behavior change: it now fires LESS often - the false positives on healthy turns are gone; log prefix 'reasoning-pair-violated' kept for grep continuity.

Fixes #1479
@ranxianglei
ranxianglei force-pushed the 2026-09-27_responses-strict-echo branch from 34b2970 to e28746c Compare September 27, 2026 07:46
- custom_tool_call_output joins the run-item set in
  normalizeStrictEchoResponsesInput (and the test helper): an interleaved
  [call, output, call] layout no longer splits a reasoning-bearing run
  into a false orphan (latent today — bili and codex emit grouped
  layouts — but the same class the PR repairs).
- WIRE_RULES gains WC-009 (responses: thinking-mode providers reject
  resent assistant runs with no reasoning item, 400 code 11155) with a
  validation-parity clause in validateResponsesBody and a gate probe —
  the permanent-ledger discipline (#762/#1479 provenance).
@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep-review verdict: MERGEABLE — plus two review fixes landed at 1d5ff03 (both flagged by the review, applied on this branch):

  1. custom_tool_call_output was missing from the run-item set (src/strict-echo.ts + test helper): an interleaved [call, output, call] layout would split a reasoning-bearing run into a false orphan → bogus blank injection mid-turn. Latent today (bili/codex emit grouped layouts) but exactly the class this PR repairs. One-line completion + a regression test pinning the interleaved shape.
  2. WC-009 added to the wire-constraint ledger (tests/wire-contract-fakes.ts + gate probe): thinking-mode providers reject resent assistant runs with no reasoning item (400 code 11155 reasoning_content_missing), provenance DeepSeek thinking mode: residual "reasoning_content must be passed back" 400 on the first post-fold request (0.1.109 / acp-kernel 0.0.69) — the forwarded body is not persisted #762/[Bug] #762's strict-echo repair is chat-completions-only: Responses-wire requests still 400 with reasoning_content_missing after a compress fold #1479, with a validation-parity clause in validateResponsesBody — the permanent-ledger discipline the fixing PR owes.

Review evidence (delegate + verified): repair is insert-only and reference-preserving; #283 intact (insert-before, never append); idempotent; all four loop retry sites normalized (chat-wire retries were previously unnormalized too — bonus fix); both compression modes covered (prepareResponses + loop); same config gate, no new config surface; mutation-verified (killing the Responses injector reddens 5 tests incl. the full-arc).

Gates on 1d5ff03: typecheck ✓ · targeted 52/52 ✓ · golden 5/5 ✓ · full suite 2920 pass / 0 fail (12 e2e-gated skips).

Non-blocking follow-ups noted for the record: incremental-mode (previous_response_id) sessions have no in-input reasoning so the gate stays closed (conservative, disclosed); model-issued exotic call types (local_shell_call etc.) are neither run items nor calls (nothing emits them today).

…1479)

Review follow-up to 1d5ff03: the inRun-set completion updated the repair (src/strict-echo.ts), the e2e helper, and the WC-009 parity fake, but missed the runtime canary warnResponsesReasoningPairs (src/server.ts). There a custom_tool_call_output still broke the run, so an interleaved [reasoning, function_call, custom_tool_call_output, function_call] split into a false orphan: the canary would fire reasoning-pair-violated on a healthy turn while the repair correctly stayed silent, and it would disagree with the WC-009 fake. Add custom_tool_call_output to the canary in-run no-count branch (mirrors the other three sites) + a regression test pinning canary/repair/WC-009 parity (mutation-verified: reverting the line reddens the test). Green: typecheck / 2921 pass 0 fail 12 skip / build.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 复核 1d5ff03 两个修复 + 补上其中一处遗漏(已直接修在分支上)

先确认你落的两个修复都正确:

但复核发现修复①只改了 4 处 run 定义里的 3 处,漏了运行时哨兵。 全仓只有两处真正建模"strict-echo assistant run":repair(src/strict-echo.ts)和哨兵 warnResponsesReasoningPairs(src/server.ts)。你把 repair、e2e helper、WC-009 fake 三处的类型表都补了 custom_tool_call_output,唯独 src/server.ts:2551 的哨兵仍是 if (t === "function_call_output") continue;——custom_tool_call_output 会落到 closeRun() 把 run 拆开。后果:对 [reasoning, function_call(c1), custom_tool_call_output(c1), function_call(c2)] 这种 interleaved 布局,repair 正确地保持静默,但哨兵会把它拆成 [reasoning,fc](有 reasoning,OK)+ [fc(c2)](无 reasoning → 假 orphan)→ 对一个健康轮打出 reasoning-pair-violated;而且它会和 WC-009 的 validation-parity fake(fake 已含 ctc_output)对同一输入给出相反判定。这正是本 PR 要消灭的那类误报,且破坏了我上一轮认可的"哨兵兵↔repair↔WC-009 一致"。坦白说:我第一轮也漏了这个——当时夸了两端一致,却没逐处核对每张类型表是否都含 ctc_output。

已在分支上直接补掉(commit b0849126,已 push):

  • src/server.ts:2551:哨兵 in-run 分支补 || t === "custom_tool_call_output",与其余三处镜像。
  • tests/reasoning-echo.test.ts:加一条哨兵侧回归测试,断言同一 interleaved 布局下 warnResponsesReasoningPairs 0 行日志,钉死 canary/repair/WC-009 三者一致。
  • mutation 验证:临时回退该行 → 新测试 not ok(fail 1);恢复后转绿。测试有牙。
  • pre-flight(HEAD=b0849126):typecheck ✓ · 全套 2921 pass / 0 fail / 12 skip(比你报的 2920 多 1 = 我这条新测试)· build ✓(dist-import-annotations: OK)。
  • 另做了一次全仓 sweep:除上述两处外,其余提到这些类型的文件(output-steering.ts/responses-tool-output.ts/tool-pair-order.ts/compress-loop-responses.ts/plugin.ts/adapter-responses.ts)要么本就同时列两种 output,要么建模的是别的关注点(call 检测/buffer/tool-pair 顺序),都不是漏网的 reasoning-run 站点。四处现已全部一致。

基座新鲜度:PR 现在基于 v0.1.160(30b1574b),master tip 又前进了一个 commit 0b718003(#1490 cache-ledger-full-history,只动 src/cache-ledger.ts+其测试)。与本 PR 零文件重叠、无语义交互,可干净合并;若要我 rebase 到最新 master tip 也可以说一声,但当前无冲突。

结论:仍为 MERGEABLE(按 §7.4,wire/message-shape 变更跨两种压缩模式 → 人工合并,不适用 auto-merge)。两处落地的修复正确,我补上的哨兵遗漏也已修好并验证,canary/repair/WC-009 现完全一致。

中文摘要:复核了你落的两个修复(repair inRun 补全 + WC-009 入账)均正确,但发现修复①漏改了运行时哨兵 warnResponsesReasoningPairs(4 处 run 定义只改了 3 处),会导致 interleaved custom_tool_call_output 布局下哨兵误报且与 WC-009 fake 判定相悖;我已直接在分支补上这一行 + 回归测试(mutation 验证过、全套 2921 绿),四处 run 定义现已完全一致,可以(人工)合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] #762's strict-echo repair is chat-completions-only: Responses-wire requests still 400 with reasoning_content_missing after a compress fold

1 participant