Skip to content

[PR] feat: plugin-lane stream-termination observability (#2328) - #2356

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-07_plugin-sse-observability
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-07_plugin-sse-observability

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Closes #2328

Follow-up observability for the plugin-lane SSE strip pipes (pipePluginChatWithStrip / pipePluginResponsesWithStrip) — the lanes #2323's CRLF-tear bug came from, which had no raw-SSE capture and no termination context on their in-band truncation frame. All three checklist items share one failure domain (plugin-lane stream-termination observability) and ship together. No compression-semantics change.

What

  • Q1 — raw-SSE capture. When diagnostics.dumpSse (env ACP_DUMP_SSE or file key) is set, each pipe tees the upstream stream into <dir>/<ts>-<sid>-plugin-{chat,responses}-raw.sse, mirroring the proxy lane's <ts>-<sid>-raw.sse. The parse branch is untouched; the tee branch feeds the existing dumpStreamToFile. Default OFF; only the first-attempt stream is captured (a retry/refetch second stream is not — documented limitation).
  • Q2 — termination metadata. The in-band upstream_stream_truncated error frame gains a metadata-only diagnostics object: classification, sawTerminal, chunksReceived, bytesReceived, unparsedFrames, residualBufferLen, lastEventTypes (last 8), upstreamStatus, upstreamContentType, retryNote. server.ts supplies status/content-type (already in hand at every call site).
  • Q4 — classification. Distinguishes why the terminal was never seen: missing-terminal (clean EOF, nothing unparseable) vs unrecognized-terminal (≥1 frame failed JSON.parse) vs incomplete-trailing-event (dangling partial frame left in buffer at EOF). Surfaced in both the frame and the proxy log line — directly answering the issue's "上游没发终端 vs 终端到了但解析被拒" split.

Config surface

No new config. Q1 reuses the existing diagnostics.dumpSse directory convention exactly as the proxy lane and loop lane already do — an existing mechanism expresses this precisely, so no new field/section/env var was invented (Configuration Surface Discipline). The diag plumbing is internal (a trailing optional arg), not a config knob.

⚠️ Behavior change (flagged per policy)

The plugin-pipe truncation error frame now always carries a diagnostics object (old: none). Old→new: {"error":{...,"message":"..."}} → {"error":{...,"message":"...","diagnostics":{...}}} (responses nests top-level rather than under error). Rationale: counts/classification are zero-cost and strictly more informative → on by default; only upstreamStatus/upstreamContentType are conditional (present when the caller supplies them — server.ts always does). Risk is low: these are bili-generated error-frame fields, never user intent or tool-call arguments (the #1039 wire-fidelity bound is untouched), and clients ignore unknown JSON keys. emitUpstreamTruncation stays byte-identical when called without meta (no other caller passes one).

Sequencing / hot-file note (updated post-rebase)

Source bugfix #2323 shipped as PR #2329 and merged to master (2e32ea428) while this PR was open; this branch has been rebased onto current master including #2329, with the two pipe read-loop edits reconciled by union of intent: #2329's stateful CR-holding parser (pendingFinal + finalizeSseLineEndings EOF re-drive) is preserved verbatim, and #2328's counters layer on top — chunksReceived/bytesReceived count real upstream chunks only (the else branch), so the synthetic EOF finalize pass never counts as a chunk, and retryFired flags the one-shot retry in the done branch exactly as before. As predicted pre-rebase, after #2329 lands the CRLF-tear class simply stops appearing in the unrecognized-terminal bucket: the torn fragments now resolve and parse on the finalize re-drive.

Verification

  • Re-verified on the rebased head (47965cf49): npm run typecheck clean; targeted suites green — issue2328-plugin-sse-observability (9/9), issue2171-zero-byte-truncation-retry + plugin-empty-turn-retry, issue1452-socket-hardening, issue2323-sse-crlf-cross-chunk — 55/55 total. All pre-existing suites use .includes() on code/message, unaffected by the added field.
  • Full regression delegated to CI.

Wire-shape change (error-frame metadata) → human merge per §7.4, no auto-merge.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-07_plugin-sse-observability (8d05bb9)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2529-pr.2356.c23af24

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2356.tgz
npm install -g package

Install instructions are refreshed on each push.

Q1 raw-SSE capture: the Responses/Chat strip pipes tee the upstream stream
into diagnostics.dumpSse (reused, default OFF) as <ts>-<sid>-plugin-{chat,
responses}-raw.sse, mirroring the proxy lane.

Q2 termination metadata: the in-band truncation error frame now carries a
metadata-only diagnostics object (chunk/byte counts, last event types,
residual buffer length, sawTerminal, upstream status/content-type, retry
disposition). Byte-identical when no metadata is supplied.

Q4 classification: distinguishes why the terminal was never seen —
missing-terminal vs unrecognized-terminal vs incomplete-trailing-event —
surfaced in both the frame and the proxy log line.

Closes #2328
@ranxianglei
ranxianglei force-pushed the 2026-10-07_plugin-sse-observability branch from 47965cf to 8d05bb9 Compare October 7, 2026 15:58
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 冲突已解决:分支已 rebase 到最新 master(2495ad25d,含 #2329 及后续 #2332/#2302 合入),PR 现为 mergeable: true(无冲突),CI 全量检查在新 head 8d05bb943 上运行中。PR 正文的 sequencing 一节已同步更新为 rebase 后状态。

结构化七问

  1. 是什么问题:系统转发的上游 PR 冲突通知——不是功能缺陷。[PR] feat: plugin-lane stream-termination observability (#2328) #2356 分支与 master 分叉,GitHub 判定不可合并(dirty)。
  2. 什么原因导致的:fix: hold back trailing CR so SSE terminators do not tear across chunks (#2323) #2329([Bug] Responses SSE 的 CRLF 跨 chunk 归一化会拆坏多行 data,产生 upstream_stream_truncated(v0.1.187) #2323 的 CRLF-tear 修复)先合入 master,把 src/plugin.ts 两条 strip pipe 的读循环重构为 pendingFinal + finalizeSseLineEndings EOF 重驱结构;本 PR 的诊断计数器是加在旧结构循环上的,同区域双改必然冲突。PR 正文早已声明该依赖("if fix: hold back trailing CR so SSE terminators do not tear across chunks (#2323) #2329 merges first this branch needs a rebase")——这是预期内的排序事件,不是新 bug。
  3. 怎么修复的:rebase 到当前 master tip,按「union of intent」解冲突:fix: hold back trailing CR so SSE terminators do not tear across chunks (#2323) #2329 的新循环结构原样保留,本 PR 计数器叠加其上——chunksReceived/bytesReceived 只计入真实上游 chunk(else 分支),合成的 EOF finalize 重驱不计为 chunk;retryFired 在 done 分支的 one-shot retry 处标记。其余文件(server.ts / stream-error.ts / 测试)自动合并干净,无手工改动。force-push 的是功能分支(非 master)。
  4. 是否有回归问题:逐项审过既有行为——① fix: hold back trailing CR so SSE terminators do not tear across chunks (#2323) #2329 的 CR 尾持 + EOF 重驱逻辑逐字节未动;② plugin lanes: auto-retry zero-client-visible upstream truncations (frequent since 2026-10-05, cf. #2154) #2171 零字节截断重试、[BUG] In some cases, with Codex's auto model + DeepSeek, it will suddenly terminate automatically. #732/workbuddy报错Empty stream: upstream gateway sent only placeholder chunks without any model output (chunks=0, bytes=0) #821 空轮重试路径未变(仅新增 retryFired 记录位);③ 截断帧语义不变:已核实 emitUpstreamTruncation 全部调用点只有 plugin.ts 这 4 处,其他 lane 不传 meta → 帧字节级一致;④ 默认行为零变化(dumpSse 默认 OFF)。定向验证:typecheck clean,5 个定向套件 55/55 绿(issue2328×9、issue2171、plugin-empty-turn-retry、issue1452、issue2323);全量回归交给 CI(正在跑)。一个良性交互:rebase 后 CRLF-tear 类不再落入 unrecognized-terminal 桶(碎片在 finalize 重驱中解析成功)——正是 PR 正文预告的行为。
  5. 是否破坏缓存:不涉及——未触碰前缀缓存、会话身份、压缩节奏或 message-ref/id 逻辑。
  6. 是否改了设计方向:没有。仍是 [Enhancement] plugin-lane 流终止可观测性:Responses/Chat pipe 缺 raw-SSE dump 开关 + 终止元数据诊断 #2328 原定范围(Q1 raw-SSE 捕获 + Q2 终止元数据 + Q4 分类),机制不变,仅计数器挂点随 fix: hold back trailing CR so SSE terminators do not tear across chunks (#2323) #2329 的循环重构平移。
  7. 是否引入新的配置:无。Q1 复用既有 diagnostics.dumpSse;diag 是内部可选尾参,非配置项。

判定:冲突已清,分支基于最新 master,可合并状态待 CI 转绿。wire-shape 变更(truncation 帧新增 diagnostics 字段,仅 plugin-lane 截断路径)按 §7.4 属 human merge——CI 绿后请人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 这条冲突通知是过期状态,无需再次 rebase——刚核实过:

  • master 仍停在我上次 rebase 的基点 2495ad25d,没有新提交(新通知应是对 force-push 前 head 或重算中间态的旧判定);
  • GitHub 实时状态:mergeable: true、mergeable_state: blocked(blocked = CI 检查未齐,不是 dirty/冲突);本地 merge-tree 模拟与 master 合并 0 冲突;
  • CI 在当前 head 8d05bb943 上:23 项已过、4 项运行中、0 失败。

按「冲突解完状态会自动转干净,不重复 rebase」的纪律,这里不做无意义的二次 rebase。等剩余 4 项 CI 转绿即可人工合并(wire-shape 变更按 §7.4 保持 human merge)。若之后 master 再前进产生真实冲突,我再处理并附七问。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #2328 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

@ranxianglei ranxianglei changed the title feat: plugin-lane stream-termination observability (#2328) [PR] feat: plugin-lane stream-termination observability (#2328) Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] plugin-lane 流终止可观测性:Responses/Chat pipe 缺 raw-SSE dump 开关 + 终止元数据诊断

1 participant