Repository navigation
[PR] fix(handoff): persist a completed reply when the client hangs up - #4861
yanhuojiyusheng wants to merge 3 commits into
Conversation
The handoff snapshot was gated on the client socket still being writable (relay passed canSave=() => !res.destroyed, and both pipes required !res.destroyed before reply.save()). Clients that close the connection the instant they hold the terminal frame — pi does exactly this — therefore lost every assistant reply: the pipe threw 'This operation was aborted' after sawTerminal=true, took the client-abort branch and returned without saving. Persistence now answers 'did the upstream turn finish?', not 'is the socket still open?': the pipes save on sawTerminal/replyCompleted even from the client-abort path, and the relay no longer couples canSave to res.destroyed. Partial turns (no terminal) are still never captured. Refs ranxianglei#4247
bbe33aa to
63e13be
Compare
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 检索了 open/closed 的 issue 与 PR(按 handoff/persist/abort 关键词 + 标题匹配):唯一在途的相关工作是本 PR 自身;#4247/#4248 家族(handoff 落最新回复)是前置工作、已合并,本 PR 修的是其遗留的 socket 耦合条件;#4629(codex forged-summary ledger)与 #4045(codex forge tail)机制不同。无并列的修复 PR 或 issue。 标签: 开始独立审核:核对分支代码、既有行为回归面、测试有效性,结论稍后按八问给出。 |
(ai: vllm-qwen/qwen3.8-27b) 独立审核结论(八问)① 是什么问题:pi 这类「拿到终帧立刻挂断」的客户端,单轮会话的助手回复永远进不了 handoff 快照( ② 什么原因导致的:持久化判定被错误地耦合到客户端 socket 存活上,而不是上游回合是否完成。三层耦合:relay 用 ③ 影响范围与严重程度:所有在拿到终帧后立即关连接的插件车道客户端(实测主要是 pi 单轮;issue 普查显示 pi han=false 全灭,匿名代理模式 534/541 正常)。能用但交接面缺最新回复,客户端自身有完整转录、下一轮请求可恢复 → p2(已打标 bug/p2/pi/scope/wiring)。 ④ 怎么修复的:把判据从「socket 还活着吗」换成「上游回合完成了吗」。两条 pipe 在正常出口和 client-abort catch 分支都按 ⑤ 是否有回归问题:逐项清点过触碰路径的既有行为——(a) 无终帧的半截回复永不落盘:保持(新测试负例钉住);(b) truncated 信号路径: ⑥ 是否破坏缓存:不涉及。 ⑦ 是否改了设计方向:不改。#4247/#4248 建立的 handoff 机制原样保留,只修正了触发谓词(socket liveness → upstream completion),wire/配置/持久化格式/会话身份均未动。 ⑧ 是否引入了新的配置:不涉及。 独立验证证据(非转述 PR 声明)
收工判定代码与行为均成立,diff 最小(+12/−2 源码)、单一线性提交、注释带 issue 引用符合仓库风格,未发现需要顺手修的小瑕疵。可合——但按 §7.4,持久化语义变更属人工合并范畴(PR 作者自己也标了),请 owner 手动合并:#4861 。CI 全量结果以推送头为准。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE — independent end-to-end pass on head bd9091d45 (the post-master-merge state), not a rehash of the earlier threads: the conflict-resolution merge was verified by diff, not by commit message (merge-base == current master ee0dcc7c4, so the two-dot diff IS exactly what lands: 4 files, +201/−4, nothing from master dropped by the resolution), and every claim below was re-derived from code plus local runs on this head.
Correctness. The predicate swap is right and safe:
save()internal guards (src/server/handoff-response.ts:39-43: discarded / empty / stale request-counter / lastMessages snapshot / canSave) make cross-request double-save impossible and within-request save idempotent.createHandoffReplyhas exactly one production call site (src/server/relay.ts:273), so dropping the socket-coupledcanSavechanges nothing else.retryEmptyTurnresetssawTerminal=falsewhen a retry takes over (src/plugin.ts:2771), so the new predicates can never fire for a superseded attempt. In the responses pipereplyCompletedis set only after the retry gate passed (src/plugin.ts:4426); failed/incomplete terminals discard instead — no "saved then truncation signal" divergence.- Normal exit XOR catch per request; truncated ⇒ !sawTerminal keeps save/discard mutually exclusive. Wire bytes untouched (truncation-signal emission order preserved); no config / persistence-format / identity change.
Tests. Real harness (listen(0), hermetic fake upstream, real server via startServer). Case 1 reproduces the exact pi shape (upstream holds after [DONE], client self-destructs on [DONE]); case 2 pins the no-terminal invariant with teeth ("half an ans" would appear in the handoff only if partial content were persisted). The handoff-response harness change moves the abort-case teeth from the socket guard to the terminal gate — not weakened, now matches production. I reproduced RED→GREEN myself (fix stashed → the PR's own test fails with the AssertionError quoted in the PR body; restored → green), ran the abort suite 5× consecutively (stable), plus handoff-response + plugin-protocol + loop-core = 65/65 and clean typecheck on this head. CI per mirror cache at bd9091d: 0 failed (10 success / 13 fork-lane skips / static gates still queued at fetch time) — final verdict belongs to the CI lanes on the pushed head.
One non-blocking finding — fixed on-spot, needs your push (or maintainer edits). Both pipes' upstream-read-error catch branch (client STILL connected) skipped persistence entirely: a turn whose terminal frame already landed but whose trailing bytes died in flight (LB reset right after [DONE]) still loses its reply. Same mechanism family as #4860 — persistence must answer "did upstream finish?", not "how did the stream end?" — rarer trigger than the pi hang-up, pre-existing on master, non-blocking for this PR's scope. Fixed as commit e59daeb (two mirrored save lines + bidirectional RST regression pair; RED without the source change, stable 5× locally) but pushing to your fork was rejected (permission denied; maintainer edits are off). Two ways to land it: apply the patch below to the branch, or enable "Allow maintainer edits" on the PR and I will re-push e59daeb.
diff --git a/src/plugin.ts b/src/plugin.ts
index 53413b4ce..ff93ebaa2 100644
--- a/src/plugin.ts
+++ b/src/plugin.ts
@@ -3674,6 +3674,10 @@ export async function pipePluginChatWithStrip(
/* client half-gone; the emission below is best-effort too */
}
loggerLog("warn", `[plugin] upstream stream read failed (${protocol}): ${String(e instanceof Error ? e.message : e)} — truncation signal path`);
+ // Same rule as the client-abort branch: the terminal already arrived, so
+ // the answer is complete even though the trailing bytes died in flight
+ // (e.g. an LB reset right after [DONE]).
+ if (sawTerminal && !sawReplyError) reply?.save();
emitUpstreamTruncation(res, protocol, finalFinishReason !== undefined, log, buildTruncationDiag("read-error"), transparentTruncationApplies(session));
return;
} finally {
@@ -4543,6 +4547,9 @@ export async function pipePluginResponsesWithStrip(
/* client half-gone; the emission below is best-effort too */
}
loggerLog("warn", `[plugin] upstream stream read failed (responses): ${String(e instanceof Error ? e.message : e)} — truncation signal path`);
+ // Same rule as the client-abort branch: response.completed was fully
+ // processed, so the reply belongs in the handoff even if the tail died.
+ if (replyCompleted) reply?.save();
emitUpstreamTruncation(res, "responses", false, log, buildTruncationDiag("read-error"), transparentTruncationApplies(session));
return;
} finally {
diff --git a/tests/handoff-client-abort.test.ts b/tests/handoff-client-abort.test.ts
index 6565092b9..bd18a3314 100644
--- a/tests/handoff-client-abort.test.ts
+++ b/tests/handoff-client-abort.test.ts
@@ -41,13 +41,15 @@ interface Harness {
/** Upstream that writes `head` and then holds the stream open — the real
* slow-model shape: the client has its terminal frame while the connection
- * is still alive. */
-async function startHarness(head: string[], opts: { hold?: boolean; doneAfterMs?: number } = {}): Promise<Harness> {
+ * is still alive. `hook` gets the server-side response once the head is
+ * written so a test can kill the connection from the upstream side. */
+async function startHarness(head: string[], opts: { hold?: boolean; doneAfterMs?: number; hook?: (res: http.ServerResponse) => void } = {}): Promise<Harness> {
const upstream = http.createServer((req, res) => {
req.on("data", () => {});
req.on("end", () => {
res.writeHead(200, { "content-type": "text/event-stream" });
for (const line of head) res.write(line);
+ opts.hook?.(res);
if (opts.doneAfterMs !== undefined) {
// [DONE] lands after the client has had time to hang up, so the
// pipe's write of that frame hits a dead socket.
@@ -185,3 +187,78 @@ test("does not persist a reply the upstream never terminated", async () => {
await h.close();
}
});
+
+// Upstream-side death (hard RST instead of a clean close) exercises the pipe's
+// upstream-read-error branch with the client still connected — the mirror of
+// the client-hang-up shape above. Same rule must hold: persistence answers
+// "did the upstream turn finish?", never "how did the stream end?".
+
+async function pollHandoff(h: Harness, label: string, until: (md: string) => boolean): Promise<string> {
+ const deadline = Date.now() + 5_000;
+ let md = "";
+ while (Date.now() < deadline) {
+ md = await handoffMd(h, label);
+ if (until(md)) return md;
+ await new Promise((r) => setTimeout(r, 100));
+ }
+ return md;
+}
+
+test("keeps the reply when upstream resets right after the terminal frame", async () => {
+ // The whole turn incl. [DONE] is delivered, then the upstream connection
+ // hard-resets (the LB-kill shape) while the client stays connected. The
+ // turn completed upstream, so the reply must land in the handoff.
+ let serverRes: http.ServerResponse | undefined;
+ const h = await startHarness([roleChunk(), textChunk("final answer"), terminalChunk(), "data: [DONE]\n\n"], { hold: true, hook: (r) => { serverRes = r; } });
+ try {
+ const req = request(h, "handoff-upstream-rst-after-terminal", payload(true));
+ let seen = "";
+ req.on("response", (res) => {
+ res.on("data", (c: Buffer) => {
+ seen += c.toString("utf8");
+ // The client saw [DONE], so bili has already written it — from
+ // here on every further read on the upstream side fails.
+ if (seen.includes("data: [DONE]") && serverRes && serverRes.socket && !serverRes.destroyed) {
+ serverRes.socket.resetAndDestroy();
+ }
+ });
+ });
+ req.on("error", () => {});
+ req.end(JSON.stringify(payload(true)));
+ const deadline = Date.now() + 5_000;
+ while (Date.now() < deadline && !seen.includes("data: [DONE]")) await new Promise((r) => setTimeout(r, 25));
+ assert.match(seen, /final answer/);
+ const md = await pollHandoff(h, "handoff-upstream-rst-after-terminal", (m) => m.includes("final answer"));
+ assert.match(md, /### assistant/);
+ assert.match(md, /final answer/);
+ } finally {
+ await h.close();
+ }
+});
+
+test("does not persist when upstream resets mid-turn before any terminal frame", async () => {
+ // Negative direction of the same gate: the reset lands before any terminal
+ // frame, so the turn never completed upstream and nothing may be saved.
+ let serverRes: http.ServerResponse | undefined;
+ const h = await startHarness([roleChunk(), textChunk("half an ans")], { hold: true, hook: (r) => { serverRes = r; } });
+ try {
+ const req = request(h, "handoff-upstream-rst-mid-turn", payload(true));
+ let seen = "";
+ req.on("response", (res) => {
+ res.on("data", (c: Buffer) => {
+ seen += c.toString("utf8");
+ if (seen.includes("half an ans") && serverRes && serverRes.socket && !serverRes.destroyed) {
+ serverRes.socket.resetAndDestroy();
+ }
+ });
+ });
+ req.on("error", () => {});
+ req.end(JSON.stringify(payload(true)));
+ const deadline = Date.now() + 5_000;
+ while (Date.now() < deadline && !seen.includes("half an ans")) await new Promise((r) => setTimeout(r, 25));
+ const md = await pollHandoff(h, "handoff-upstream-rst-mid-turn", () => false);
+ assert.doesNotMatch(md, /half an ans/);
+ } finally {
+ await h.close();
+ }
+});Behavior change disclosure (old→new). (a) This PR (already disclosed in its body): upstream-completed turn with dead client socket — old: dropped from the handoff; new: persisted. (b) The follow-up fix above extends the same rule to the upstream-dies-after-terminal path — old: dropped; new: persisted. Partial turns (no terminal frame) are still never persisted — pinned by negative tests in both suites.
八问
- 是什么问题: pi 这类「读到终止帧立刻挂断」的客户端,上游已正常完成回合,但助手回复进不了交接快照(handoffResponse 缺失)。
- 什么原因导致的: 落盘判据选错了维度——绑在「客户端 socket 是否还活着」而非「上游这一轮是否完成」;relay 的 canSave、两条 pipe 的正常出口、abort catch 三处同向耦合。判据本身错了,不是 pi 行为异常。
- 影响范围与严重程度: 所有读到终帧即断连的插件车道客户端(实测 pi 单轮全灭;匿名代理模式的短命客户端同型中招);能用但交接面缺最新回复,客户端自身转录完整、重连+下一轮可恢复 → p2。
- 怎么修复的: 判据换成上游完成态(chat 看 sawTerminal、responses 看 replyCompleted,均叠加 !错误标志),正常出口与 client-abort catch 双路径落盘;relay 去掉 socket 耦合。修在决策点本身,不是中途拦截症状。
- 是否有回归问题: 无。逐项清点:无终帧不落盘(负例钉住)、truncated 与 save 结构互斥、超尺寸 discard / 错误事件 / JSON 分支守卫 / snapshot 失效双守卫全在 save() 内部未动且各有测试、跨请求双 save 不可能(stale guard + 出口互斥)、wire 字节不变(信号发射顺序保留)、side-passthrough 不建回复对象不变。
- 是否破坏缓存: 不涉及(会话元数据,不进前缀缓存材料,不动重试/去重键)。
- 是否改了设计方向: 不改;把 #4247 家族的隐含契约显式化——落盘回答「回合是否完成」,与客户端存活解耦。
- 是否引入新配置: 不涉及。
判定:可合。 按 §7.4 持久化语义变更属人工合并范畴,请自行合并 #4861 (建议连同上面的 follow-up 修复一起落地)。
(ai: vllm-qwen/qwen3.8-27b) 审核完成:APPROVE 已提交(GitHub review #5731532796)。对 head bd9091d(合入 master 后的状态)独立复审:冲突解决经 diff 验证未丢 master 内容(merge-base == master,落地 diff = 4 文件 +201/−4);三层判据替换与 save() 内部守卫逐项复核成立;RED→GREEN 亲手复现、abort 套件连跑 5 次稳定、定向套件 65/65、typecheck 干净;CI 镜像缓存 0 failed。已顺手修:两条 pipe 的 upstream-read-error 分支(客户端仍连着)在终帧已到后同样漏保存——commit e59daeb(两行镜像 save + 双向 RST 回归对)已备好,但推作者 fork 被拒(permission denied,maintainer edits 关闭),完整补丁附在 review 正文,作者可直接 apply 或开启 maintainer edits 由我重推。判定:可合(§7.4 持久化语义变更 → 请 owner 手动合并)。 |
|
Landed via batch PR #6482 (merged) — closing as superseded. |
…xianglei#4861) The client-abort save paths land, but the relay still passed canSave=() => !res.destroyed, so a finished reply was dropped the instant pi hung up on the terminal frame. Persistence now keys on the upstream turn finishing; partial turns stay uncaptured.
Fixes #4860
What
metadata.handoffResponsewas dropped for pi single-turn sessions even though the answer reached the client. Persisting the reply was conditioned on the client socket still being alive; pi closes immediately after reading the terminal frame, so the save never happened.Instrumented run before the fix:
Upstream had already finished (
finish_reason=stop); the write after the terminal frame ([DONE]/ trailing frames) threwAbortError, landing in the plugin pipe's catch branch:src/plugin.ts—if (res.destroyed || res.writableEnded) { log?.("client aborted mid-stream"); return; }→ returns withoutsave()and withoutdiscard().if (sawTerminal && !sawReplyError && !res.destroyed && !res.writableEnded) reply?.save();src/server/relay.ts:271—createHandoffReply(prepared.session, () => !res.destroyed), same coupling one layer up.Change (+12/−2 source)
Persistence now depends on the upstream turn completing, not on the client socket:
src/server/relay.ts:271createHandoffReply(prepared.session)— drop thecanSavesocket guardsrc/plugin.ts(chat pipe)if (sawTerminal && !sawReplyError) reply?.save();at loop end and in the client-abort catch branch beforereturnsrc/plugin.ts(responses pipe)if (replyCompleted) reply?.save();in both the normal exit and the abort catch branchA reply with no terminal frame is still never persisted — unchanged. Both plugin pipes (chat + responses) are covered; the relay path (JSON branch) shares the same
HandoffReplyobject and needs no separate change. Nothing on the wire, in config, in the persistence format, or in session identity is touched.Tests
tests/handoff-client-abort.test.ts(2 cases):role+content+finish_reasonthen[DONE]and holds the socket; the client destroys itself right after it seesdata: [DONE]→ the handoff must contain### assistant+ the reply text.AssertionError: The input did not match the regular expression /### assistant/), GREEN after.tests/handoff-response.test.ts:16harness now callscreateHandoffReply(session)(same shape as production); its abort / oversize / snapshot-invalidation cases still pass.Verification (on the rebased head)
node --import tsx --test tests/handoff-response.test.ts tests/handoff-client-abort.test.ts→ 20/20 passtests/plugin-protocol + loop-core + cli-export + web-handoff-roles + aux-proxy-handoff + e2e-export-handoff + plugin-native-binding→ 67/67 passnpm run typecheck→ clean;npm run build→ OK (build-info: commit=63e13bec)ACP_PORT=18913 node dist/index.js pi -p --no-session --mode text "Reply with exactly: PI-HANDOFF-OK"→ printsPI-HANDOFF-OK; session01a17884-76d1-734c-af33-daa352b04040(agent=pi,requests=1) now persistshandoffResponse: {"request":1,"messages":[{"id":"reply_text_1","role":"assistant","contentType":"text","text":"PI-HANDOFF-OK"}]}.Reviewer notes
63e13bec, rebased onto current master05c379ef.AddLabelsToLabelable):bugp2piscope/wiring.versionintentionally untouched (content commit).