Skip to content

[PR] fix(#5951): replay a stream cut that lands after reasoning-only output - #5956

Merged
ranxianglei merged 4 commits into
ranxianglei:masterfrom
vladk1m0:2026-11-08_reasoning-replay-after-cut
Nov 9, 2026
Merged

ranxianglei merged 4 commits into
ranxianglei:masterfrom
vladk1m0:2026-11-08_reasoning-replay-after-cut

Conversation

@vladk1m0

@vladk1m0 vladk1m0 commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

A stream cut that lands inside the reasoning phase is currently a dead turn: the host has a thinking fragment, no answer, no tool call, and the pipe answers with a truncation error. Two paths lead there — a clean EOF and a read error — and neither re-issues the request.

  chat pipe, cut detected
    if (!sawTerminal && …) {
-     if (await retryZeroByteCut()) continue;        // requires nothing forwarded at all
+     if ((await retryZeroByteCut()) || (await retryReasoningOnlyCut())) continue;
    }
    try { step = await reader.read(); }
-   catch (e) { throw e; }                          // straight to the truncation signal
+   catch (e) { if (await retryReasoningOnlyCut(e)) continue; throw e; }

retryReasoningOnlyCut() replays once when the only thing the client has seen is reasoning: no answer text, no tool call, no partial argument stream, no released markup, no stripped echo. It shares the single stream-level re-issue budget with the #2171 zero-byte path, so a second cut still surfaces the #721 signal instead of looping. Anything the host could merge badly keeps the old behaviour.

TruncationDiag gains readError, so a cut that is not replayed names the socket error instead of only classification: "no-terminal-seen".

Evidence

Before: the reasoning-only cut reaches the host as:

[plugin] upstream stream read failed (openai): terminated — truncation signal path
err: upstream stream ended before a completion event; this turn may be incomplete
diag: {"cause":"read-error","events":18,"bytes":4168,"visibleChars":0,"eofBufferBytes":0,"upstreamStatus":200}

After: that same shape is re-issued once and, if the retry answers, the host receives the answer and the reasoning together.

tests/issue5951-reasoning-only-cut-replay.test.ts   7 pass   (3 replays, 3 keep-the-signal, 1 diag)
tests/issue2171-zero-byte-truncation-retry.test.ts  …  pass   (unchanged)
tests/issue2328-terminal-diag.test.ts               …  pass   (unchanged)
npm run typecheck                                   clean
npm run build                                       exit 0, dist-external OK
full suite (BILI_TEST_LEAK_GUARD=0)                 10445 tests, 13 failures

The 13 failures are pre-existing and environmental, not caused by this change: the nine affected files give 501 tests / 488 pass / 13 fail both with the patch stashed and applied (tests/test-suite-lock.test.ts CLI leak guard, opencode-v2, pi-native, opencode-native, issue5627, issue3304, launcher, issue3544, identity-harvest).

Merge Danger

Door: two-way — the replay path is additive and gated by a predicate that only accepts reasoning-only cuts; reverting it restores the previous error behaviour with no state to migrate.
Blast Radius: narrow — one function plus one diag field, chat pipe only; the responses pipe (codex) keeps its behaviour because replaying item ids there needs its own review.

Closes #5951

…-only output

A cut that lands inside the reasoning phase used to be a dead turn: the
client held a thinking fragment, no answer and no tool call, and the pipe
handed the host a truncation error instead of re-sending the request.

retryZeroByteCut() is the only replay path for a cut stream and its
zero-visible predicate rejects anything that already forwarded reasoning;
the read-error branch never attempted a replay at all.

Add retryReasoningOnlyCut(): a one-shot replay for a cut (EOF or read
error) whose only client-visible output was reasoning. It shares the
single stream-level re-issue budget with the ranxianglei#2171 zero-byte path and
reuses the mid-turn framing so the retry's blocks land after the ones the
client already saw. Answer text, tool calls, partial argument streams,
released markup, a stripped echo or a forwarded content block still keep
the ranxianglei#721 in-band signal.

TruncationDiag gains readError so a cut that is not replayed names its
cause instead of only its classification.
@ranxianglei ranxianglei changed the title fix(#5951): replay a stream cut that lands after reasoning-only output [PR] fix(#5951): replay a stream cut that lands after reasoning-only output Nov 8, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: opencode/qwen3.8-27b)

Duplicate screening: not a duplicate — this is a deliberate narrow extension of the merged zero-byte-cut retry (#2171, shipped as PR #2173). Related-but-different threads exist but none share the mechanism: #2516 keeps unsigned thinking out of client history on re-fetch (different path, no interaction here), #4712/#4681 hold round≥2 signed thinking off the wire until its signature lands (KDD #10 territory). No other open PR targets #5951.

Labels applied: bug + scope/wiring + p2 (kept the existing pr). Defect class; the change lives in the plugin pipe's upstream-stream transport/retry + client forwarding (protocol-adaptation boundary); p2 because it's a specific-condition dead turn (a transient upstream cut landing inside the reasoning phase) with a manual re-prompt workaround — not total unavailability, not a cache leak or data loss. No client label: the repro is generic OpenAI-wire plugin traffic and no single host can be pinned.

CI (live, mirror cache, fetched 2026-11-08T13:15Z): 30 checks — 0 failed, 12 passed, 6 skipped (env-gated e2e lanes), 12 still running. Merge state reads "blocked" = required checks in progress, not a conflict (the PR is clean/mergeable). No failures so far; I'll follow up once the suite settles.

结构化八问

  1. 是什么问题:上游流在「只输出了推理(reasoning/thinking)、还没出正文或工具调用」时被切断,这一轮就变成死轮——宿主只拿到一段思考片段,没有答案、没有工具调用,只能收到截断错误,用户得重新提问。
  2. 什么原因导致的:既有的重发路径 retryZeroByteCut(plugin lanes: auto-retry zero-client-visible upstream truncations (frequent since 2026-10-05, cf. #2154) #2171)其零可见谓词刻意排除「已转发过推理」的流(sawThinking),而读错误分支此前根本不尝试重发、直接吐截断信号——于是落在推理阶段的切断没有任何一条路会重新发请求。
  3. 影响范围与严重程度:仅插件车道(openai/google 增量式 wire);触发条件是「上游在推理阶段瞬时断流、且此前未转发任何可操作内容」。命中时该轮完全失败(可手动重提),其余轮次不受影响。属特定条件下的可用性缺陷(p2),非全局不可用、非缓存泄露/数据丢失。
  4. 怎么修复的:新增窄范围重发 retryReasoningOnlyCut——仅当客户端只看到推理(无正文/工具调用/参数流/释放标记/剥离回声)时重发一次,复用既有单次重发预算与中试帧结构;读错误分支也接入同一条重发;并给 TruncationDiag 增加 readError 字段,让未被重发的切断能报出具体 socket 错误而非只有 classification。
  5. 是否有回归问题:未发现回归。既有行为逐项核对保持——零字节切断(plugin lanes: auto-retry zero-client-visible upstream truncations (frequent since 2026-10-05, cf. #2154) #2171)、退化空轮([BUG] In some cases, with Codex's auto model + DeepSeek, it will suddenly terminate automatically. #732/workbuddy报错Empty stream: upstream gateway sent only placeholder chunks without any model output (chunks=0, bytes=0) #821)、tag-echo 空轮(Plugin passthrough leaves the client an empty turn when the model echoes a render tag #870)三条重发的谓词与预算互锁不变;anthropic wire 经 blocksForwarded 被排除(思考块本身即已转发块,索引/签名归客户端);proxy 模式走独立路径不受影响(relay.ts:1808 prepared.pluginMode 门控);推理字节仍走 identity filter 逐字转发(KDD release v0.1.3 #10 不破)。非阻塞观察见下。
  6. 是否破坏缓存:不涉及。改动在流传输/重发层,不触碰前缀缓存、增量游标、去重键;重发为整包重发同一请求(与既有两条重发一致),命中温缓存,无新增失效策略。
  7. 是否改了设计方向:否。是既有「单次重发预算 + 中试帧结构」机制的一次窄扩展(新增一个谓词入口),方向与 plugin lanes: auto-retry zero-client-visible upstream truncations (frequent since 2026-10-05, cf. #2154) #2171/[BUG] In some cases, with Codex's auto model + DeepSeek, it will suddenly terminate automatically. #732/workbuddy报错Empty stream: upstream gateway sent only placeholder chunks without any model output (chunks=0, bytes=0) #821 一致,不动架构/边界/数据流。
  8. 是否引入了新的配置:无新 env/config/默认值,纯代码行为新增,配置面无变化。

Non-blocking findings

  • Nit (responses diag): src/plugin.ts:4674 — the responses pipe's read-error emit calls buildTruncationDiag("read-error") without passing the error, so the codex/responses lane won't populate readError even though its signature now accepts it (the chat lane does, src/plugin.ts:3794). One-line fix: pass e. Flagging rather than committing since I can't push to your fork branch.
  • Google wire coverage: the predicate incidentally covers the Google wire too (delta-based, no content_block_start, so blocksForwarded stays 0). The issue/evidence is scoped to OpenAI. Not a regression (strictly better than the dead-turn baseline), but under the evidence-permitlist discipline you may want either a Google-specific test or an explicit protocol gate.
  • Test gap (Anthropic exclusion): the load-bearing Anthropic exclusion (the blocksForwarded > 0 guard) isn't pinned by a test. A negative case — an Anthropic thinking block forwarded, then cut → refetches === 0 — would lock the intent that index/signature accounting stays client-owned.
  • Note only (zero-byte read error): the read-error catch tries only retryReasoningOnlyCut, not retryZeroByteCut, so a zero-byte cut arriving as a read error (vs a clean EOF) still isn't retried. That's pre-existing behavior (read errors always rethrew before this change), not a regression, and out of scope here.

Verdict

Solid, well-scoped, well-tested change. The predicate, the shared single re-issue budget, and the Anthropic exclusion all check out against the surrounding state machine, and the 7 new tests have teeth (3 positive replays, 3 keep-the-signal negatives including budget-exhaustion, 1 diag). Mergeable pending green CI. Every finding above is non-blocking — none warrant holding the merge.

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 8, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: opencode/qwen3.8-27b)

Following up on my review above (it stands in full). One addition on CI: I re-checked through an independent source after the suite kept running — still zero failures across every run of the head commit (de3bd431), with the latest test/static-gate lanes still in progress (a few earlier runs show as superseded/cancelled from a re-trigger, not failures). So the picture is unchanged: no failing lane, green not yet reached.

Verdict unchanged — mergeable pending green CI; the four findings remain non-blocking. I'll post the final green/red call once the suite actually settles (if red, I'll isolate the failing lane and check inherited-master-red vs. this PR's fault first). No other action taken: PR left open for owner merge, no issues opened, nothing pushed to your fork branch.

@ranxianglei

Copy link
Copy Markdown
Owner

Non-blocking findings should be fixed, conflicts should be resolved

ci-check added 2 commits November 9, 2026 09:47
…nglei#5956 conflict resolution)

Resolves the single src/plugin.ts conflict in the chat-pipe read-error catch
block by keeping BOTH sides' intent: master's noteTruncatedUpstream(session,
protocol, "read-error") session note AND the PR's buildTruncationDiag(
"read-error", e) that names the failed read in the truncation diag.
…e its read error

Non-blocking finding from conflict resolution: the ranxianglei#5951 change added the
optional readError parameter to BOTH pipes' buildTruncationDiag definitions,
but only the chat pipe's read-error call site was updated to pass the caught
error. The responses pipe's definition gained the parameter and the
conditional field, yet its call site never passed e, leaving the readError
diag field dead in that lane. Pass e so a non-replayed responses cut names
its cause, symmetric with the chat pipe.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Blocker + what I did this run (upstream PR #5956)

The PR was mergeable_state: dirty — its head carried only the #5951 fix on top of an old merge-base (fe963d7f5c / #5932), while master had advanced past it to e7b3d2821f. I merged current master into the branch and pushed back to the contributor fork (verified the fork branch still pointed at de3bd43141 first → clean fast-forward, no force-push, nothing clobbered). New head: a9a1301e41.

Conflict resolution (exactly 1 conflict, src/plugin.ts): in the chat-pipe read-error catch block, master had added noteTruncatedUpstream(session, protocol, "read-error") while the PR changed buildTruncationDiag("read-error") → buildTruncationDiag("read-error", e). I kept both (union of intent): the session note AND the readError naming. Every other hunk auto-merged cleanly.

Non-blocking finding fixed (per your note): the #5951 change added the optional readError parameter to both pipes' buildTruncationDiag definitions but only wired the chat-pipe call site. The responses-pipe call site never passed e, so its readError diag field was dead code in that lane. I wired it (buildTruncationDiag("read-error", e)) so a non-replayed responses cut names its cause, symmetric with the chat pipe. Shipped as a separate labeled commit.

Verification: typecheck green · #5951 regression test 7/7 · targeted truncation/diag/stream-error suite 63/63. New CI run on the pushed head: all gate checks already green (kernel-guard, version-guard, lock-sync, unused-exports, config-surface-guard, …), remaining test/e2e lanes queued, zero failures. Merge state is now blocked (= awaiting CI completion + required approvals), not a conflict.

⚠️ Behavior change (flagged per policy): responses-lane truncation diagnostic — old value: readError field absent (param was dead); new value: readError present, naming the failed read (e.g. Error/ECONNRESET: ...). Additive to the in-band error meta only — no control-flow, retry-budget, or client wire-shape change. Reason: completes the #5951 intent symmetrically with the chat lane.


结构化八问(中文,供 reviewer 核查)

  1. 是什么问题:PR [PR] fix(#5951): replay a stream cut that lands after reasoning-only output #5956 与 master 合并冲突(dirty)——PR 头只带 Replay a stream cut that lands after reasoning-only output #5951 修复、基于旧 merge-base([PR] Over-window requests no longer sent raw upstream: three forwarding gates metered by actual outbound body size + 32 MiB c | 超窗请求不再裸发上游:三道转发闸按真实出站体计量 + 大请求 32MiB 成本带准入 (#5000 #4543 #5885 #5794) #5932),master 已前进到 e7b3d28。
  2. 什么原因导致的:PR 分支落后多个版本窗口;master 在 src/plugin.ts 的 read-error 分支新增了 noteTruncatedUpstream 会话记录行,与 PR 对同一 hunk 的 buildTruncationDiag(..., e) 改动相撞。
  3. 影响范围与严重程度:仅合并层面,无外部行为影响;冲突解决前 PR 无法进入 CI 全量判定(「特定条件下有问题」=合并前不可合)。
  4. 怎么修复的:把当前 master merge 进 PR 分支;唯一冲突取两侧意图并集(保留 master 会话记录行 + PR 的 readError 命名);另修一处非阻塞缺陷——responses 车道 read-error 发射点补传 e,使其新增的 readError 诊断字段真正生效(与 chat 车道对称)。
  5. 是否有回归问题:审过四条截断发射点(chat/responses × EOF/read-error),现均同时带会话记录 + 正确 cause;hadTextOtherThanTextFields 那处 drain 守卫变更来自 master 本身(已在 master 过 CI)。定向测试 63/63 + Replay a stream cut that lands after reasoning-only output #5951 回归 7/7 通过,未见破坏既有行为。
  6. 是否破坏缓存:不涉及——仅截断信号 meta 里加一个诊断字段,不触及缓存/重试游标/去重键。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:冲突已解、非阻塞项已修、本地定向验证全绿,已推回 fork(head = a9a1301e41),mergeable=true;等 CI 全量跑完 + owner 审批即可合。我这边无需再动,除非某条 CI 车道转红(届时按 CI-fail 流程处理)。

@ranxianglei
ranxianglei merged commit e2ff3a0 into ranxianglei:master Nov 9, 2026
52 checks passed
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1113-master.e2ff3a0 (rolling master tag).

npm install -g billion-context@0.0.1113-master.e2ff3a0  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replay a stream cut that lands after reasoning-only output

2 participants