Skip to content

fix: reject incomplete/failed Responses JSON summaries in preflight (#2309) - #2311

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-07_preflight-incomplete-json
Oct 7, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-07_preflight-incomplete-json

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2309

What

The SSE path already treats response.incomplete / response.failed as unusable summary terminals (#780/#784); the JSON path had no terminal check. An HTTP-200 application/json Responses body declaring status: "incomplete" (incomplete_details.reason: "max_output_tokens") sailed through requestSummary() whenever its partial text cleared MIN_SUMMARY_CHARS (50) — the fragment then entered a compression block and the history forwarded to the business model. Observed in production with an 824-char partial.

How

  • extractSummaryText (Responses section): explicit status === "incomplete" | "failed" now returns empty whatever the text length, across both the flat output_text shortcut and the output[] walk. Omitted status stays accepted — compat gateways that never send it are untouched (context window problem continues #987 class).
  • diagnoseEmptySummary: new responsesTerminalDiagnosis names the terminal precisely with the raw reason/error embedded verbatim (the upstream returned an incomplete Responses summary (status=incomplete, reason=max_output_tokens)), running BEFORE extractStreamError so a failed response is not claimed by the weaker bare-error framing. Shape guard: bad status without a recognizable Responses output shape keeps the generic diagnosis.
  • emptySummaryIsSizeDriven: reason=max_output_tokens now classifies as size-driven → halving recovery ([Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 semantics; the production case was exactly this reason). Other reasons keep the transient bounded same-span re-draw path.

Scope is deliberately Responses-only: openai/anthropic JSON paths accepting finish_reason=length/stop_reason=max_tokens partials is consistent with their SSE leniency (#764/#780 design — those wires do not require terminals); only Responses was asymmetric (SSE strict vs JSON lenient).

Behavior changes (old → new)

  1. Previously-accepted partials are now rejected. A 200 JSON Responses body with explicit status=incomplete|failed used to be accepted when its text ≥ 50 chars; it now enters the existing unusable-summary chain (same-span re-draw for transient reasons, halving for size-driven, fail-fast + dead-end cooldown when nothing recovers). Completed and status-omitted bodies behave exactly as before.
  2. Diagnosis string change. For a status=incomplete JSON body the operator-facing string changes from the upstream returned a plain-JSON completion with empty content (status=incomplete) to the upstream returned an incomplete Responses summary (status=incomplete[, reason=<raw>]). The old wording was factually wrong whenever partial text rode the body. This string appears in proxy logs and the 502 fail-fast message; one context window problem continues #987 pinned test assertion updated accordingly (openai/anthropic empty-completion strings unchanged).
  3. No model budget, compression-range, persistence-format, or wire-shape changes. No new config surface (nothing added to billion-context.json, no new env vars, no CLI flags).

Evidence

New suite tests/preflight-incomplete-json.test.ts (unit + integration through the real proxy):

  • extractor rejects both JSON shapes for explicit bad terminals; completed/status-omitted/other protocols unchanged;
  • diagnosis strings pinned; classifier routing pinned (max_output_tokens → size-driven, others → transient);
  • proxy AND plugin modes: first summary answer = the issue's exact 95-char incomplete JSON, later answers completed → request recovers 200, forwarded wire contains the full summary and NOT the partial text, no dead-end marker left;
  • transient variant (no reason): same-span re-draw (identical span length), then recovery;
  • persistent-incomplete: 502 preflight_compress_failed carrying the new diagnosis, bounded summary calls, zero forwards, dead-end marker armed, cooldown spends zero upstream calls.

Pre-patch run of this suite on unpatched behavior: 6/6 fail (three on exactly the rejected partial text must NOT reach the forwarded history, one on systemic incomplete summaries must fail-fast, got 200). Post-patch: 6/6 pass, plus all pre-existing preflight suites green (104 tests across the 15 files importing src/preflight.ts) and typecheck + build clean.

…2309)

The SSE path already treats response.incomplete / response.failed as
unusable summary terminals (#780/#784); the JSON path had no terminal check,
so an HTTP-200 application/json body declaring status=incomplete with
incomplete_details.reason=max_output_tokens sailed through requestSummary
whenever its partial text cleared MIN_SUMMARY_CHARS (50) and entered a
compression block plus the forwarded history.

- extractSummaryText: explicit status=incomplete/failed now rejects the body
  whatever its text length, across both the flat output_text shortcut and
  the output[] walk. Omitted status stays accepted (compat gateways).
- diagnoseEmptySummary: new responsesTerminalDiagnosis names the terminal
  precisely with the raw reason/error embedded verbatim, running before
  extractStreamError so a failed response is not claimed by the weaker
  bare-error framing.
- emptySummaryIsSizeDriven: reason=max_output_tokens classifies as
  size-driven (halving recovery, #726 semantics); other reasons keep the
  transient bounded re-draw path.
- Regression tests: extractor/diagnosis/classifier units plus proxy AND
  plugin-mode integration (first answer incomplete JSON, later answers
  complete): partial text never reaches the wire; persistent-incomplete
  fails fast carrying the diagnosis and arms the dead-end cooldown.
- Updated the #987 pinned assertion whose input (status=incomplete JSON)
  now routes to the precise terminal diagnosis instead of 'plain-JSON
  completion with empty content' (that string is false when partial text
  rides the body).
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-07_preflight-incomplete-json (eec422b)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2444-pr.2311.d89e8cc

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2311.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #2311 审查结论(fix: reject incomplete/failed Responses JSON summaries in preflight,Closes #2309)

已拉取分支核对代码、rebase 到当前 master 74cf524 独立复核、跑定向测试。逐问如下:

① 是什么问题:Responses 摘要以 HTTP-200 JSON 返回且声明 status:"incomplete"/"failed" 时,只要部分正文 ≥ MIN_SUMMARY_CHARS(50),preflight 就当作可用摘要,截断正文进入压缩块并随转发历史发给业务模型。SSE 路径早已拒收同一终态(#780/#784),JSON 路径漏了对称的终态校验——这是根因所在,不是表面现象。

② 什么原因导致的:extractSummaryText() 的 Responses 分支只抽文本、不看 status;requestSummary() 拿到后仅做最小长度检查。终态校验合同只实现在 SSE 侧(extractSummaryFromSse 的 invalid 判定),JSON.parse 成功后直接进入抽取——#780/#784 的完整性规则没覆盖这条腿。修法打在源头(抽取前拒收),不是在中途过滤字节,符合「疏不堵」原则。

③ 怎么修复的:三处协同——(a) 抽取前拒收显式 incomplete|failed,同时覆盖 output[] 遍历与 output_text 快捷分支;省略 status 的兼容响应保持接受。(b) 新增 responsesTerminalDiagnosis 精确命名终态并原文嵌入 reason/error,先于 extractStreamError 运行(防 failed 响应被较弱的 bare-error 措辞抢走)+ shape guard(无 Responses 形状保留通用诊断)。(c) emptySummaryIsSizeDriven 增加 reason=max_output_tokens → 走 #726 减半恢复,其他 reason 走既有有界重绘。范围刻意仅限 Responses:openai/anthropic 的 JSON 接受 finish_reason=length 类部分与其 SSE 宽松设计一致,不构成同类缺口,未动。

④ 是否有回归问题:无。逐项过过的既有行为:

  • completed 与省略 status 的响应行为不变;openai/anthropic/google 三个早退分支不受影响(新代码在它们之后才执行)。
  • SSE 两处 extractSummaryText 调用点:line 551 的 {output:[o.item]} wrapper 确无顶层 status;line 554 的 response.completed 传入的 o.response 实际是带顶层 status:"completed" 的完整响应对象(PR 自述「均不携带顶层 status」此处措辞不精确),但 "completed" 不在拒绝集内,且该处有 if (full) 守卫、SSE 侧见 response.incomplete/failed 事件类型也已先置 invalid 短路——结论(零影响)成立,只是理由应修正为「completed 不被拒」。
  • 诊断顺序变更仅影响带坏终态的 Responses body;bare {error} 对象(无 status)仍落回 extractStreamError,顺序不变。
  • 高危模式逐项:无语义漂移、无 fail-fast 变吞错、无阈值/时序漂移;唯一输出变化是那条被披露的诊断串(有意变更)。

⑤ 是否破坏缓存:不涉及。不触碰前缀缓存、重试预算、dead-end 冷却键或去重逻辑;冷却机制原样复用并有测试钉住(marker 置位、cooldown 期零上游调用)。

⑥ 是否改了设计方向:不涉及。这是既有终态校验合同在 JSON 路径上的补齐,机制/数据流/架构均未转向。

⑦ 是否引入新配置:不涉及。无新 env/config/默认值/CLI flag。

行为变更披露(旧→新):

  1. 此前被接收的 incomplete/failed JSON 部分摘要(≥50 字符)→ 现一律拒收,进入既有不可用摘要链路(transient 同 span 有界重绘 / size-driven 减半 / 持续失败 fail-fast + dead-end 冷却)。触发条件仅为 body 显式声明坏终态,completed 与省略 status 流量零影响。
  2. status=incomplete JSON body 诊断串:plain-JSON completion with empty content (status=incomplete) → incomplete Responses summary (status=incomplete[, reason=<raw>])(出现在 proxy 日志与 502 fail-fast message;旧串对携带部分正文的 body 是事实错误)。context window problem continues #987 一处固定断言随之更新;openai/anthropic 的 empty-completion 串不变。

验证证据(我在 rebase 后的当前 master 上独立复跑,非沿用作者自述):

  • rebase 干净:PR 基于 8500672,落后当前 master 132 commit;src/preflight.ts 期间被改 11 次但 diff 原样套用(53/5/321 行不变)。
  • typecheck 干净;build 成功 + dist-import-annotations: OK(zod-free 不变量)。
  • 全部 19 个引用 src/preflight.ts 的测试文件 148 项全绿(作者自述「15 文件 104 项」是基于旧 base 的数字,master 之后又加了文件,我的数字覆盖之)。
  • 回归测试确有牙:把行为逻辑回退(仅保留 export 让 import 可用)后,新套件 6/6 全败且失败签名与作者声称完全一致——extractor 期望 '' 实得 95 字符 PARTIAL、诊断串新旧不符、proxy/plugin「the rejected partial text must NOT reach the forwarded history」、persistent「systemic incomplete summaries must fail-fast, got 200」。恢复补丁后 6/6 通过。

⚠️ 一个与本 PR 无关的独立发现(master 自身已红,非本 PR 引入):CI 的 unused-exports 门在 PR head 上失败,报 INTERNAL_ONLY src/launcher.ts: codexRewriteInsertionIndex。我切到干净的 origin/master(不含本 PR)跑同一门,同样失败——即当前 master 的既有状态;本 PR 完全不碰 launcher.ts,不应归因于它。溯源:该函数在 5336c0700(real-codex persona CI)被导出供外部消费,但当前 master tip 除 launcher.ts 自身外已无任何引用(grep 确认),故被判 internal-only。处置建议:不要塞进本 PR(会污染 preflight 修动的 diff,违反一 PR 一 scope / 净 diff 规则);宜单独小修,或在已在改 launcher.ts 的在途 PR #2306 里顺带处理——若确为纯内部函数,去掉行首 export 即可让门转绿。(按 loop-guard,此链为 agent 发起,我不自动立项,留你决定。)

判定:从代码 / 回归 / 测试有效性看,本 PR 正确、完整、可合并。合并前两项前置卫生:(1) 作者 rebase 到最新 master(现落后 132 commit,preflight.ts 是热文件);(2) 上面那个 master 既有的 unused-exports 红门需另行处理,否则 CI 无法全绿。合并仍由人工执行:#2311

@ranxianglei
ranxianglei merged commit bc30d58 into master Oct 7, 2026
23 of 24 checks passed
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.104-master.bc30d58 (rolling master tag).

npm install -g billion-context@0.0.104-master.bc30d58  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] preflight:JSON Responses 的 incomplete 摘要仍被接收,与 SSE 终态校验不一致

1 participant