Skip to content

[PR] fix(#5951): let the OMP host own truncation recovery, keep diagnostics local - #6300

Closed
vladk1m0 wants to merge 0 commit into
ranxianglei:masterfrom
vladk1m0:2026-11-11_omp-owns-truncation-recovery
Closed

vladk1m0 wants to merge 0 commit into
ranxianglei:masterfrom
vladk1m0:2026-11-11_omp-owns-truncation-recovery

Conversation

@vladk1m0

@vladk1m0 vladk1m0 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #5956 (merged 2026-11-09, closes #5951). After that merge the proxy still spends its own one-shot replay when the requesting host already owns truncation recovery, so one cut can be recovered twice — once by the proxy and once by the host's own retry.

native OMP request ──▶ bili proxy ──▶ upstream
                       │  header: x-bili-host-recovery-owner: omp
                       │
   reasoning-only cut ─┤ host owns recovery → in-band signal, spend no retry
                       └ otherwise          → existing one-shot replay
  • src/util.ts, src/agent/omp-native.ts: the OMP native bootstrap stamps x-bili-host-recovery-owner: omp on the requests it sends through the proxy.
  • src/server/headers.ts: the marker is stripped before upstream egress, next to the other internal hop headers.
  • src/plugin.ts, src/server/relay.ts: both pipe entry points read the marker past the new streamErrorShape parameter and pass hostOwnsTruncationRecovery; when it is set, a reasoning-only EOF / read error / zero-byte cut is reported to the client in-band without spending the proxy retry. Pending answer filters, visible answer text, tool calls and an exhausted budget still refuse the replay.
  • src/stream-error.ts: TruncationDiag gains readErrorCause, upstreamRequestId, upstreamLogId, upstreamCfRay and replay; the meta field is removed from the client-visible terminal SSE frames, so transport causes and upstream trace ids ride the proxy log line only. Codes and messages are unchanged, and the commandcode branch from [dsh] Retry-exhausted degenerate turn is serialized as successful completion (0.1.213) #5958 keeps its shape.

Wire compatibility

The only client-visible change is the removal of one diagnostic field from terminal error frames:

protocol frame removed path
openai chat data: {"error":{...}} + data: [DONE] error.meta
responses event: error {"type":"error",...} top-level meta
google {"error":{"code":503,"status":"UNAVAILABLE",...}} error.meta
anthropic event: error {"type":"error","error":{...}} error.meta

The google frame keeps code: 503 and status: UNAVAILABLE; it does not carry upstream_stream_truncated, so nothing it said is lost.

Evidence

The recovery-ownership matrix, measured on the same reasoning-only cut in tests/issue5951-reasoning-only-cut-replay.test.ts:

marker upstream refetches client sees reasoning forwarded
absent (previous behaviour) 1 retry's answer once
x-bili-host-recovery-owner: omp 0 in-band truncation error once
  • Targeted: tests 170 / pass 162 / fail 0 / skipped 8 across the eight files this PR touches (issue5951-reasoning-only-cut-replay, issue2171-zero-byte-truncation-retry, issue2328-terminal-diag, stream-error, proxy-stream-error, omp-native, claude-native, native-intercept-selfheal).
  • npx tsc --noEmit -p . clean; npm --prefix kernel run build clean.
  • Full suite on this HEAD (macOS, Node 26.13, base c26c31ac), npm test: tests 11017 / pass 10951 / fail 4 / skipped 62 / todo 0 in 557 s. The four failures are host artifacts, not this change: findProjectJsonlFiles and writeDshClientShimFiles (#1590) compare /tmp against /private/tmp, plus the two #5627 cases — the same four reproduced on the pre-rebase base 3ac3e95c in a clean-env four-file control (pass 389 / fail 4 on both trees, identical names). No full-suite control was run against c26c31ac itself. A #3544 stale-proxy-rotation case failed once in a pre-rebase run on this machine, does not reproduce at this HEAD and passes in isolation on both trees — unattributed.
  • Socket-cut proof: bun run tests/e2e-omp-socket-cut-smoke.mjs (harness not included in this PR) against the oh-my-pi source checkout — one cut produced 2 upstream requests, hostCalls 2, one auto_retry_start/auto_retry_end success, final assistant text Recovered response., and a proxy diag with replay.replayAvailable=true, openStreams=0. That cut was plain text, not reasoning-only, and it ran on the source checkout (v18.18.2), not the installed 18.18.8 binary.
How the reasoning-only predicate is covered

Two cases in tests/issue5951-reasoning-only-cut-replay.test.ts: "the host owning truncation recovery suppresses the replay" (read error after reasoning-only output → refetches === 0, sentinel still emitted, the partial reasoning still forwarded once, diag records replayAvailable: true) and "the host owning recovery also suppresses the zero-byte replay". The sibling case on identical input without the marker asserts refetches === 1, so the suppression is not a tautology. The marker itself is covered at the stamp (native-intercept-selfheal), the egress strip (claude-native) and the client-header source (omp-native).

No historical baseline run is claimed: the cut cannot be produced against an installed host from a test, so the control is the pre-marker behaviour asserted by the sibling case in the same file.

Docs: no page describes the removed meta field — docs/architecture/pipeline.md:30 already describes stream-error as a "minimal SSE error + finish", this change makes the frame strictly more minimal, and :28 already states that internal hop headers are stripped at egress, which the marker joins.

Merge Danger

Door: two-way
Blast Radius: wire

Closes #5951

@ranxianglei ranxianglei changed the title fix(#5951): let the OMP host own truncation recovery, keep diagnostics local [PR] fix(#5951): let the OMP host own truncation recovery, keep diagnostics local Nov 11, 2026
@vladk1m0
vladk1m0 force-pushed the 2026-11-11_omp-owns-truncation-recovery branch 2 times, most recently from 24d9b17 to a8de47c Compare November 12, 2026 19:01
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Conflict already resolved — no action taken this run.

The merge conflict that triggered this lane was resolved by the PR author (vladkimo) before I started. He rebased the PR onto the current master (259cd87674), pushing a8de47cfc2 at 2026-11-12 18:59 UTC.

Verified live this run:

  • Local: merge-base(PR head, master) = 259cd87674 (current master head); master is a strict ancestor of the PR head → PR is 1 commit ahead of master, no divergence.
  • GitHub API: mergeable: true, mergeable_state: blocked (was dirty at trigger time — the conflict is gone; blocked = waiting on human review/branch protection, not a conflict).
  • CI on head a8de47cfc2 (single GitHub check-run fetch; the mirror cache was stale at fetched_at 2026-11-12T19:08Z and not refreshing): 38 success / 14 skipped / 0 failed / 0 pending — green.

I made no changes to the branch (nothing to fix). The PR is ready for human merge.


八问

  1. 是什么问题:PR [PR] fix(#5951): let the OMP host own truncation recovery, keep diagnostics local #6300(镜像 [PR] fix(session): stop shedding just-served sessions from the resident pool; make pool saturation visible (#5792 #5806) #6302)与 base 分支 master 存在 merge conflict(mergeable_state: dirty),无法合并。
  2. 什么原因导致的:PR 打开期间 master 前进,PR 分支落后导致冲突。
  3. 影响范围与严重程度:仅阻塞合并,无用户侧影响(CI/merge 问题,非功能问题)。
  4. 怎么修复的:作者 vladkimo 已于 2026-11-12 18:59 UTC 自行 rebase 到当前 master(259cd87674),推送 a8de47cfc2。我本地验证 merge-base = 当前 master head、master 是 PR head 祖先;GitHub API 确认 mergeable=true、mergeable_state=blocked(不再是 dirty)。
  5. 是否有回归问题:无。我未做任何修改;rebase 由作者完成,我验证 PR head 正确基于当前 master,CI 全绿。
  6. 是否破坏缓存:不涉及(无改动)。
  7. 是否改了设计方向:不涉及(无改动)。
  8. 是否引入了新的配置:不涉及(无改动)。

判定:冲突已解决、CI 全绿、PR 等待人工合并(branch protection 要求 review)。CI 工程师无需进一步动作。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: qwen3.8-27b)

Reviewed the full diff against current master. The design is sound — letting the OMP host own truncation recovery (instead of the proxy burning a second retry budget) and keeping transport diagnostics local are both the right calls. The marker-header plumbing, the egress strip, and the diag-field additions are all clean.

One regression found and fixed (pushed to your branch): the commandcode branch of emitUpstreamTruncation (src/stream-error.ts:282) now emits a trailing blank line (\n\n) after the error record. The commandcode wire is bare JSONL and every other record on it (nativeErrorChunk :61, legacyCompletionChunk :105, emitPreflightError :328) is single-newline terminated — a blank line is not a valid JSONL record and would choke strict consumers at the exact moment the stream already failed. The PR description says the commandcode branch "keeps its shape" from #5958; it did not. I reverted it to \n and pinned the invariant in tests/commandcode-plugin-pipe.test.ts (no blank line, endsWith "}\n"). Commit 910bc13249 is on your branch now.

Also: your rebase onto master already resolved the only merge conflict (CHANGELOG), so the PR is mergeable. All targeted suites pass (178 tests / 170 pass / 0 fail / 8 skipped); typecheck clean.


结构化八问

  1. 是什么问题:PR 让 OMP 宿主自己接管截断恢复(代理不再花第二次重试预算),并把传输诊断留在本地日志(客户端只看到重试码)。其中 commandcode 分支的截断错误记录多了一个空行(\n\n),破坏了裸 JSONL 线格式。
  2. 什么原因导致的:PR 把 src/stream-error.ts:282 的 commandcode 分支从父提交的 \n 改成了 \n\n;PR 描述声称「commandcode 分支保持 [dsh] Retry-exhausted degenerate turn is serialized as successful completion (0.1.213) #5958 的形状」,实际没有。空行不是合法 JSONL 记录。
  3. 影响范围与严重程度:仅 commandcode 宿主 + 流中途截断时触发;错误记录本身仍是合法 JSON,但尾随空行对严格 JSONL 消费者是非法记录,恰好在流已失败的瞬间卡住解析。p2(特定条件下的线格式不一致)。
  4. 怎么修复的:把 commandcode 分支恢复为单换行终止符(\n),与同线上所有其他记录一致;在 commandcode-plugin-pipe.test.ts 钉死不变量(无空行、endsWith "}\n")。
  5. 是否有回归问题:无。修复只改 commandcode 分支的终止符,其他分支(openai/anthropic/responses/google)未动;PR 其余改动(OMP 宿主恢复、本地诊断、reasoning-only 谓词)设计合理,未引入回归。
  6. 是否破坏缓存:不涉及。未触碰任何缓存/重试/去重逻辑。
  7. 是否改了设计方向:不涉及。PR 的设计方向(OMP 宿主拥有截断恢复)成立;修复只是纠正线格式回归。
  8. 是否引入了新的配置:不涉及。BILI_HOST_RECOVERY_OWNER_HEADER 是内部头(出站前剥除),BILI_HOST_RECOVERY_OWNER_OMP 是常量,均非用户可见配置。

收工判定:commandcode 回归已修并推送到 PR 分支(910bc13249),CHANGELOG 冲突已由作者 rebase 解决,PR 现可合并(待常规 review/check 门禁)。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 进度汇报:工作已全部完成,进程卡住发生在收尾之后,无未竟事项。

已完成:

  1. 审核 PR [PR] fix(#5951): let the OMP host own truncation recovery, keep diagnostics local #6300(mirror [PR] fix(session): stop shedding just-served sessions from the resident pool; make pool saturation visible (#5792 #5806) #6302):设计合理(OMP 宿主接管截断恢复 + 诊断留本地)。
  2. 发现并修复 commandcode 回归:src/stream-error.ts:282 截断错误记录多了一个空行(\n\n),破坏裸 JSONL 线格式。已恢复为单换行并在 tests/commandcode-plugin-pipe.test.ts 钉死不变量。
  3. 已推送到 PR 分支:commit 910bc13249,PR head 已确认更新,mergeable。
  4. 测试全过:178 tests / 170 pass / 0 fail / 8 skipped;typecheck 干净。
  5. 已发布结构化八问 review(上一条评论)。

卡住原因:工作完成、review 已发布后,进程在收尾阶段挂起(疑似网络/工具调用挂起),非阻塞性。重启后状态完整,无需补做。

收工判定:PR 现可合并(待常规 review/check 门禁)。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

1 similar comment
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6482 (merged) — closing as superseded.

@ranxianglei ranxianglei reopened this Nov 14, 2026
@ranxianglei
ranxianglei force-pushed the 2026-11-11_omp-owns-truncation-recovery branch from 910bc13 to 2153152 Compare November 14, 2026 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replay a stream cut that lands after reasoning-only output

2 participants