Skip to content

fix(plugin): retry a turn whose only visible output was a stripped tag echo (#870) - #871

Merged
ranxianglei merged 9 commits into
ranxianglei:masterfrom
joshwapohlmann:2026-09-16_870-plugin-empty-turn-retry
Sep 17, 2026
Merged

ranxianglei merged 9 commits into
ranxianglei:masterfrom
joshwapohlmann:2026-09-16_870-plugin-empty-turn-retry

Conversation

@joshwapohlmann

Copy link
Copy Markdown
Contributor

Problem

The plugin path strips render tags from every forwarded chunk. When the model echoes a ref annotation as the entire visible content of a turn, the tag is removed, the emptied chunk is suppressed, and the client receives a turn with neither text nor a tool call, which the harness aborts. Releasing the echo is not an option: #463 deliberately swallows a pure tag echo, and #674 plus the degenerate-turn warning exist so that imitated tags never reach the client.

The adapter path (proxy mode with the compress tool injected) already answers this with a one-shot re-fetch; the plugin's direct pipe had no equivalent, so the defect only bit plugin-mode sessions.

Fix

On a turn that ends cleanly (stop, end_turn, stop_sequence) with zero visible text, zero tool calls and no error finish reason, the pipe now re-issues the same request once with a continuation nudge and relays the retry in the original framing.

  • src/degenerate-retry.ts (new): DEGENERATE_RETRY_NUDGE, injectContinuationNudge(protocol, body) and makeContinuationRefetch(opts), moved out of src/loop/core.ts so both paths share one nudge and one body builder.
  • src/wire-body.ts (new): appendTrailingUserText(protocol, body, text), extracted from the fake-completion hint, which now delegates to it.
  • src/plugin.ts: pipePluginChatWithStrip takes a refetch callback, tracks degenerateRetried / inRetry / retryIndexOffset / blocksForwarded, and gates on a clean terminal reason set.

Two details the tests forced, both about not corrupting the client's view of the turn: the openai terminal is the choices[].finish_reason chunk rather than [DONE] (using [DONE] let the first attempt's terminal reach the client twice), and the gate runs after the event is processed, because a coalesced chunk can carry both content and the finish reason and would otherwise be judged empty and dropped. An anthropic retry lands at "index":1 with a single message_start / message_stop, and an unissuable retry presents the original terminal with valid JSON.

If the retry is also empty, the original empty turn is passed through and the existing [degenerate-turn] warning fires, so the contract that keeps the annotation away from the client is unchanged.

Tests

tests/plugin-empty-turn-retry.test.ts (8, against the pipe with a fake response and a fake refetch): retries once on an echo-only turn and the client sees the retry's prose with exactly one [DONE] and one finish_reason; no retry when visible text, a tool call or content_filter ends the turn; an empty retry passes the empty turn through and warns; an unissuable retry presents the original terminal as valid JSON; anthropic keeps one message_start / message_stop with the retry at index 1; and the nudge-body shapes per wire.

Verification

Full proxy suite on the branch: 1669 tests, 1661 passing, the same six environmental failures upstream shows. npm run build clean (dist 2.89 MB), npx tsc --noEmit -p tsconfig.build.json clean. The branch is deliberately not part of the local installation, which carries the wire and the fold budget; the reproduction it answers was a live plugin-mode session whose turn ended with no text and no tool call.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Filed note on the Gemini half, deliberately not in this branch.

The commit that teaches the gate the Gemini terminal spelling (candidates[].finishReason: "STOP") and gives the trailing-nudge writer a contents[].parts[] branch cannot typecheck on its own: with master's protocol unions ("anthropic" | "responses" in src/plugin.ts, "anthropic" | "openai" in src/wire-body.ts) those === "google" comparisons are TS2367 errors, and the union widening is what the Google wire PR (#867) does. So this branch is the retry fix for the wires master supports today, and the Gemini coverage belongs here as a small follow-up once #867 has landed.

Branch head is now 8d0a089, rebased onto d650db7: npx tsc --noEmit -p tsconfig.build.json clean, tests/plugin-empty-turn-retry.test.ts 8/8. The diff is the three commits the PR describes; nothing else moved.

The other half of the same stall is a separate problem and a separate PR: a render tag wrapped around the model's whole turn now leaves an empty turn instead of orphan markup (#872), which is the shape this retry exists to re-ask.

@joshwapohlmann

joshwapohlmann commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Live verification in the reporter's own session, on the locally installed build of this change, after the proxy was restarted onto it at 18:04:48Z.

Counted per hour in that machine's bili.log for 2026-09-16, one line per event:

hour strips retries [degenerate-turn] warnings
15:00Z 156 0 110
16:00Z 468 0 169
17:00Z 1498 0 404
18:00Z 98 2 0

[degenerate-turn] is the residual-stall signature: a turn reached the client with no visible text and no tool call.

  • The retry fired twice in the 18:00Z hour, both in the reporter's own session, at 18:05:28.144Z and 18:09:34.174Z. No warning followed either, so both turns came back usable.
  • Both echo shapes occur in that traffic and both recovered: 18:05:28.016Z stripped an unbalanced wrapped head (tokens="1" text="text with no close), 18:09:37.309Z stripped a balanced one (tokens="1" text="text">), which is what a plain tag echo leaves behind.
  • One turn watched end to end: head stripped at 18:05:28.016Z, retry at 18:05:28.144Z, the retry's own request upstream at 18:05:32.477Z (processTurn 516 msgs, then the forward POST).
  • A second live session was asked to stand in for the model and re-trigger the original stall deliberately; it recovered.

Before this went upstream, both shapes were re-proved against the installed bundle with a mock upstream: two upstream requests, the second carrying the continuation nudge, and a readable tool call reaching the client.

joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
…anxianglei#870, PR ranxianglei#871)

The retry gate read only the Anthropic (message_delta stop_reason) and OpenAI
(choices[].finish_reason) terminals, so an echo-only Gemini turn — whose
terminal is the candidate that closes the stream, finishReason STOP — was
never re-asked and still reached the host empty. The gate now reads that
spelling as well, and appendTrailingUserText carries the nudge on the Google
wire's own turn shape (a text part appended to the last user content) instead
of reporting an unusable body.
@joshwapohlmann
joshwapohlmann force-pushed the 2026-09-16_870-plugin-empty-turn-retry branch from 14ec41c to 817a8cf Compare September 16, 2026 18:22
@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Adding the third pipe. The Responses passthrough had no gate at all, so an echo-only codex subagent turn still reached the host empty: 8 of one day's degenerate turns in a single session, 56 across all sessions.

One design note, because this pipe's terminal is not the chat wires' terminal. A Responses turn ends on a FAMILY — response.output_text.done, response.content_part.done, response.output_item.done — decided by response.completed, and every one of those events carries ids the client already holds. So the retry holds the family until the completion decides the turn: on a degenerate completion the held events and that completion are dropped, the agent's own body is re-issued with the same continuation nudge, and the retry is reframed onto the ids the client already holds — its response.created and its added events are dropped, and item_id, output_index and the completed output ids are rewritten to the first attempt's. A healthy turn releases the held events in place, so holding never reorders a turn.

Wired at all three Responses passthrough sites: plugin mode, the native-compact tee, and proxy mode without injection.

Tests: five in tests/plugin-empty-turn-retry.test.ts — the retry with the ids rewritten (one response.created, one response.completed, no id from the retry reaching the client), the healthy release order (output_text.done still precedes response.completed), a turn whose text arrives only in the done event (no retry, because the client would have received it), a function call, and a failed terminal. 13/13 in that file, tsc --noEmit clean. Commit 817a8cf.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Reviewed the rebased head (8d0a089, onto d650db7) against the #870 contract. Verdict: logic is correct, no regressions vs master, ready for human merge. Details below.

Verification (local, on rebased head)

  • tsc --noEmit clean; build succeeds (dist bundle ~2.9 MB).
  • Full suite A/B: PR head 1672 tests / 1669 pass / 1 fail / 2 skipped vs master d650db7 1664 / 1661 / 1 / 2. Delta is exactly the 8 new tests, all passing. The single failure (#857 D: upward self-heal trusts usage baselines only) fails byte-identically on clean master — pre-existing, unrelated to this change (also explains the red test jobs in CI on both master and this head; all non-test jobs are green here). One early run showed a second transient failure that did not reproduce in any later run (PR head ×2, master ×1).
  • All 8 tests in tests/plugin-empty-turn-retry.test.ts pass in isolation.

Constraint-by-constraint check (#870 OP)

  • Gate placement — the terminal gate runs after the event is processed (src/plugin.ts, loop tail: process → count text → gate → write-or-discard), so a coalesced chunk carrying content and the finish reason counts its own text before the turn can be called empty. Correct.
  • At most one retry — degenerateRetried is set before await refetch(), so even a failed/null refetch consumes the one shot and passes the original terminal through. Correct.
  • Clean-terminal whitelist — CLEAN_TURN_REASONS = {stop, end_turn, stop_sequence}; max_tokens/refusal/safety/content_filter/tool_use never re-prompt. Correct.
  • OpenAI framing — gating on the finish_reason chunk (not [DONE]) means the first attempt's terminal is dropped before its [DONE] is ever read (buf reset + reader swap inside retryEmptyTurn), so the client sees exactly one finish_reason and one [DONE]. Correct.
  • Anthropic framing — retryFraming consumes only the retry's message_start; block indices are offset by blocksForwarded via top-level-index rewriting on content_block_(start|delta|stop) data lines. Single start/stop, retry content lands after the client's own blocks. Correct. offsetRetryIndices degrades safely on multi-line data (unparseable line → untouched) and is a no-op outside retry / for OpenAI chunks (no top-level index).
  • Retry-also-empty — the retry's own terminal hits the gate again, degenerateRetried short-circuits it, the empty turn passes through, and the existing [degenerate-turn] warning fires post-loop (maybeWarnDegenerate, wire plugin-passthrough-*). Correct.
  • Truncation (Plugin mode: upstream stream truncated before terminal event → client gets bare cut-off SSE (no completion/error event) #721) — takeover resets sawTerminal, so a cut retry stream still raises the in-band truncation signal. Correct.
  • Nudge injection — appendTrailingUserText is a verbatim extraction of the old injectFakeCompletionHint body (behavior-preserving split into src/wire-body.ts); merges into the last user turn, appends a fresh one only when the body ends assistant-side, null on unusable bodies. Unit-pinned. Correct.
  • Wiring — both server.ts call sites pass the agent's own forwarded body (same variable resolveFakeCompletion uses), buildForwardHeaders(headers), upstream URL, dispatcher, and the client abort signal; null from makeContinuationRefetch (aborted / unbuildable body / non-ok / exception) always falls back to passthrough. Responses wire deliberately untouched — matches scope.

Non-blocking observations

  1. Thinking-only turns also trigger the retry (visible-text counter ignores reasoning_content, which is forwarded verbatim). Reading the [BUG] In some cases, with Codex's auto model + DeepSeek, it will suddenly terminate automatically. #732/workbuddy报错Empty stream: upstream gateway sent only placeholder chunks without any model output (chunks=0, bytes=0) #821 precedent, this seems intentional — those hosts aborted on thinking-only turns too — but flagging it explicitly since the OP describes the tag-echo shape. Each such turn costs one extra full request, bounded to one per turn.
  2. When the retry takes over, a filter tail held at turn-end is swallowed with the dropped terminal output (previously flushTails would have delivered it at EOF). Only affects turns whose entire visible content is tag-shaped — i.e. exactly what plugin 模式 openai SSE:混合文本字段 chunk 因单字段纯标签 echo 被整体丢弃(丢失同 chunk 真实输出) #463 wants gone — so this reads as more correct, not a regression.
  3. On takeover the first attempt's reader is abandoned without cancel()/releaseLock() (the finally releases only the current reader). The upstream body is already ~drained (terminal seen), worst case one briefly-held socket; cosmetic.
  4. Optional coverage for a follow-up: retry-stream truncation (Plugin mode: upstream stream truncated before terminal event → client gets bare cut-off SSE (no completion/error event) #721 signal after takeover) and the coalesced content+finish_reason chunk both trace correctly in code but have no dedicated test.
  5. Gemini exclusion (waiting on feat(proxy): serve the Google/Gemini wire (#866) #867's protocol-union widening) and the adjacent fix(loop): swallow a render tag wrapped around the model's turn #872 (tag wrapping the whole turn — same empty-turn shape, covered by this retry) are sensible scoping calls; noted.

Diff hygiene: 8 files, every line on-purpose, no lockfile churn, version untouched, CHANGELOG entry present. Merge stays human-side per repo rules.

中文摘要:修复了插件透传模式下模型整轮只吐出被剥离的 render tag、导致客户端拿到空回合而中断的问题(干净终止且零可见文本/零工具调用时,用 agent 自己的 body 追加一次续写 nudge 重发,严格限一次重试并保住两种 wire 的帧结构);已逐条核对 #870 的全部约束并在 rebase 后的分支上验证 typecheck/测试/构建(唯一失败项在 master 上同样失败,属既有问题),可以合并。

joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
Review follow-ups on ranxianglei#871.

The retry swaps in a new reader while the first attempt's stream is already
terminal and drained; the old one is now closed explicitly in both pipes
instead of being left for the garbage collector to release.

Two tests the reviewer asked for: a chunk that coalesces content with the
finish reason counts its own text (and, when that coalesced content is an echo,
still retries exactly once — the ordering the gate depends on), and a retry
stream that is cut still raises the ranxianglei#721 truncation signal, because takeover
resets the terminal flag.
joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
Review follow-ups on ranxianglei#871.

The retry swaps in a new reader while the first attempt's stream is already
terminal and drained; the old one is now closed explicitly in both pipes
instead of being left for the garbage collector to release.

Two tests the reviewer asked for: a chunk that coalesces content with the
finish reason counts its own text (and, when that coalesced content is an echo,
still retries exactly once — the ordering the gate depends on), and a retry
stream that is cut still raises the ranxianglei#721 truncation signal, because takeover
resets the terminal flag.
@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Three things: the head moved, your five observations answered, and two of them acted on.

The head moved since your review

You reviewed 8d0a089. The branch is now 817a8cf plus one follow-up commit, and the new work is the third pipe: the Responses passthrough had no gate at all, so an echo-only codex subagent turn still reached the host empty — 8 of one day's degenerate turns in a single session, 56 across all sessions. Your "Responses wire deliberately untouched — matches scope" was accurate for the head you saw, not for the branch now.

Its terminal is a family rather than one event, so the retry holds response.output_text.done / response.content_part.done / response.output_item.done until response.completed decides the turn: on a degenerate completion the held events and that completion are dropped, the agent's own body is re-issued with the same nudge, and the retry is reframed onto the ids the client already holds (its response.created and its added events dropped, item_id, output_index and the completed output ids rewritten to the first attempt's). A healthy turn releases the held events in place, so holding never reorders a turn. Wired at all three Responses passthrough sites. Five tests pin it, 13/13 in tests/plugin-empty-turn-retry.test.ts.

Your five observations

  1. Thinking-only turns also re-trigger — intended, and I would rather keep it. The hosts aborted on thinking-only turns for the same reason (nothing visible to them), which is what [BUG] In some cases, with Codex's auto model + DeepSeek, it will suddenly terminate automatically. #732/workbuddy报错Empty stream: upstream gateway sent only placeholder chunks without any model output (chunks=0, bytes=0) #821 fixed, and the cost is bounded: one extra request, once per turn, and only after a clean terminal with zero visible text.
  2. Held tail swallowed with the dropped terminal — agreed, more correct per plugin 模式 openai SSE:混合文本字段 chunk 因单字段纯标签 echo 被整体丢弃(丢失同 chunk 真实输出) #463. A tail still held at turn end is by construction tag-shaped, which is exactly what that contract removes; your reading is the one I intended.
  3. Abandoned reader — fixed. Both pipes now close the first attempt's reader explicitly before swapping in the retry's, instead of relying on the finally (which by then only owns the new one).
  4. The two coverage gaps — added, both, as you suggested:
    • a chunk that coalesces content with the finish reason counts its own text — asserted twice, once with prose (no retry, the prose delivered) and once with an echo as that coalesced content (exactly one retry, tag never leaks). The second is the one that actually pins the ordering your review traced: process, then count, then gate.
    • a retry stream that is cut still raises the Plugin mode: upstream stream truncated before terminal event → client gets bare cut-off SSE (no completion/error event) #721 truncation signal, plus the assertion that the client is still terminated — that is the sawTerminal reset you checked in code.
  5. Scope calls — agreed on both. The Gemini wire stays out until feat(proxy): serve the Google/Gemini wire (#866) #867's protocol-union bump exists, and the adjacent fix(loop): swallow a render tag wrapped around the model's turn #872 turn shape is covered by this retry rather than by a second mechanism.

What I would look at on the re-review

The third pipe is the part you have not seen, so that is where a second pass earns its keep: the id rewriting, the drop-on-takeover of the held family, and the framing parity with your points about response.created. Everything else in the PR is unchanged apart from the reader fix and the three tests above.

Nothing else is pending on our side; merge stays human-side per the repo rules. Thanks for the constraint-by-constraint trace — the coalesced-chunk ordering is exactly the kind of thing that looks correct in code and fails in production.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

One more commit on the branch, and it is a bug this PR's own third pipe introduced — found by proving the pipe against the installed bundle rather than the build.

A Responses turn whose only output is an echoed render tag now reaches the retry correctly, but the retry's response.output_item.done was passed to the client still carrying the retry's own item.id. Everything else had already been reframed onto the first attempt's ids — response.created, the deltas' item_id, the completed output ids — so the client held one item and then watched its identity change mid-turn, which is the shape this pipe exists to prevent.

rewriteRetryIds now rewrites the nested item.id too, and the regression pin emits that frame from the retry. Before the change the assertion on it fails, and the reason is worth naming: the fixture had never produced a response.output_item.done at all, so the assertion that the retry's ids are rewritten was passing over a frame it never saw. The hole was in the pin, not in the assertion.

How it surfaced is the part I would keep: the chat pipe's prover cannot see this path, so I wrote the Responses twin (prove-installed-responses-turn.mjs), which drives one plugin-mode turn through the installed CLI against a mock Responses upstream and reads the client's own bytes. On the pre-fix bundle it printed retry ids present: YES (1 frame(s)) with the offending event; on the rebuilt bundle, no, with the completed output carrying the item id the client already held. Unit tests passed on both, which is the point: the assembled ids look right in the pipe's own state and wrong at the wire.

@joshwapohlmann

joshwapohlmann commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Gap found in production: a turn that ends in visible orphan markup does not retry

The gate counts visibleTextChars from what the client actually receives post-strip (src/plugin.ts:1041, :1049, :1100), so a turn whose only text the filter swallows retries correctly. A turn whose orphan markup the filter declines to swallow — the unterminated render-tag case, released at the swallow cap — leaves visibleTextChars > 0, and the gate returns false at src/plugin.ts:885.

Two production occurrences on 2026-09-16, both on the OpenAI wire with DeepSeek:

when session evidence
17:29:09Z architect 01a0a0cb-4a90-74db-8e19-814e80cd4e20 recorded as the fixture at tests/tag-echo.test.ts:612-628
19:36-19:41Z 01a0ab88-a02e-7459-b278-56d7dbfe9746 the client received m00693</cp> plus a ||DSML||-wrapped parameter block, found no tool call in it, and stalled until interrupted

The second occurrence is the first one again: the model opens a render tag and writes the whole turn inside it with the attributes still open, so no close is ever written and no tag regex can match the span.

Proposed condition, in the shape of the existing one: treat the turn as empty when everything the client received was markup the filter declined to swallow. src/plugin.ts already carries the predicates to decide that (mayStartRenderTag, mayStartMarkerLine, the pending-span state at :1091), so the gate can ask whether the surviving text holds any non-markup content instead of whether it is non-empty.

Second condition: surface the failure instead of ending empty

Both occurrences shared a worse property than the missing retry: the turn stalled rather than failing, so nothing reached the client until the user interrupted by hand. An empty turn is indistinguishable from a model that simply produced nothing, and the session looks idle while it is dead.

After the retry has been spent, a turn that ends in orphan markup with no tool call should raise an in-band error. The channel exists: src/stream-error.ts already emits a signal when a stream ends without a terminal event, and the client renders it.

Not implemented in this branch yet. The wire bytes of the second occurrence are gone — the proxy log records events rather than bodies, and the payload store keeps only the proxy's own transformed view — but the client-visible residue survives in that session's transcript, so a fixture has to be reconstructed from the client's view rather than recorded off the stream. The ||DSML|| marker is the model's own envelope: it appears nowhere in src/ or in the shipped bundle.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Third occurrence, 2026-09-16T20:46:05Z: per call, not per turn

Same session as the second occurrence (01a0ab88-a02e-7459-b278-56d7dbfe9746), this time on a hub call: the parameter block opened normally and then closed with a stray </parameter> inside it, the intent parameter carried ||DSML||, and the call's real parameters were absent, so nothing executed. The model re-issued the same call correctly on the next attempt and it ran.

Two properties this adds:

  • per call, not per turn — the call immediately before it, in the same minute and of the same kind, arrived intact;
  • not tool-specific — the first occurrence was a read, this one a hub.

In all three cases the client showed a stalled session rather than an error, which is the second condition above.

Where the bytes are not, for anyone chasing a fourth: bili.log (and its rotated .old) records events rather than bodies; the payload store keeps the proxy's post-transform view of a conversation, and a turn the client discards never enters a later request; the harness's http-400-requests/ dumps full bodies but only for 400s, and this is a parse failure in the client rather than a 400.

joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
…xianglei#870, PR ranxianglei#871)

A render tag whose close never arrives is dropped by the filter while its
interior survives as the turn's only visible text: the host receives orphan
markup, finds no tool call in it, and stalls. The gate measured that interior
as visible output, so the degenerate-turn retry never fired.

Count the text a dropped tag leaves behind as markup instead, at both the chat
wire's rewrite sites, and re-issue the turn. The recorded architect-session
turn is the fixture, reconstructed from the client-visible residue because the
wire bytes are not kept anywhere.
@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Implemented in a008628.

The gate now counts the text a dropped tag leaves behind as markup rather than as visible output, at both of the chat wire's rewrite sites, so a turn whose only visible content is a surviving tag interior is re-issued. Measured behaviour: that interior has already reached the client by the time the terminal arrives, so the retry cannot erase it — what the fix guarantees is that the turn carries content the host can act on, which is what the stall was missing.

Fixture unclosedEchoTurn() in tests/plugin-empty-turn-retry.test.ts, reconstructed from the client-visible residue of the second occurrence because the wire bytes are kept nowhere. It fails before the change (no re-issue) and passes after. tests/plugin-empty-turn-retry.test.ts 17/17, tests/tag-echo.test.ts 44/44 including the recorded architect-session fixture, tests/plugin-passthrough-tag-strip-chat.test.ts 20/20 — the #463/#673 pins are untouched.

Still open, unchanged: the in-band error when the retry is also degenerate, and the same gate on the Responses pipe, whose heldVisibleChars has the twin of this hole.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Evidence for the second condition, from the session that hit it five times

Reported by 01a0ab88-a02e-7459-b278-56d7dbfe9746, which saw the shape five times on 2026-09-16 between 20:44Z and 20:50Z, all on hub calls:

  • the client surfaced nothing: no parse error, no warning, no error line. The residue appeared as ordinary assistant text, the real parameters were absent, and the turn ended normally;
  • the first occurrence stalled until the user interrupted, the later ones did not — the same corruption produces two different client-visible outcomes, and neither is reported as a failure;
  • the trigger is the marker being emitted, not the tool being called: writing it verbatim into a message reproduced the fault immediately, losing that message's recipient parameter and truncating the body at that position.

That last point locates the cause away from this repository. The marker is DeepSeek's own envelope vocabulary — fullwidth vertical bars, U+FF5C, its special-token delimiter — and the proxy neither writes nor handles it: zero occurrences in src/ and in the shipped bundle. A serving stack mishandling its own envelope explains the corruption. What this repo owes the user is that the resulting turn is not silently empty, which is the second condition above.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

fc8a350 ("reach the degenerate-turn retry on the Google wire too") is blocked on the wire branch, not pending

That commit sits in integration but never reached this branch, which is why refreshing integration conflicts: it edits src/plugin.ts and tests/plugin-empty-turn-retry.test.ts, the same files as the fix above.

Replaying it here does not compile, and the errors name the reason: src/plugin.ts(1147), src/wire-body.ts(37) — This comparison appears to be unintentional because the types '"anthropic" | "responses"' and '"google"' have no overlap. Its google branch needs "google" in the WireProtocol union, which the wire branch (#866) defines and this branch does not.

So it belongs here only after the wire lands, in the same sequencing as the kernel dependency in #867. Before replaying it, check whether the wire branch already wires the retry for the google path — that commit was written against integration, where both are present, so the two may be partly the same change.

@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

Checked the sequencing question above, and it is not redundant: the wire branch has no src/wire-body.ts at all, and its src/plugin.ts has no retryEmptyTurn. It carries the google wire's suppression path (googleFrameHasNonText) but never wired the retry to it. So fc8a350 is a genuine addition that needs the wire under it, in that order.

joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
…nxianglei#870, PR ranxianglei#871)

A turn the host never sees any content from is indistinguishable from a model
that produced nothing, and the client reports neither, so a dead session reads
as an idle one. The degenerate-turn gate already retried once; when that retry
degenerates as well, the client now gets the in-band stream error the host would
never surface, and the retry's own terminal is dropped so the error closes the
turn.

Observed on 2026-09-16 by the session that hit the corruption five times: no
parse error, no warning, the residue as ordinary assistant text.
@joshwapohlmann

Copy link
Copy Markdown
Contributor Author

The Responses pipe has the same hole; the measurements for whoever takes it

Tested rather than assumed: an unclosed render tag on the Responses wire does not retry. The probe is the chat one's shape on that wire (responsesUnclosedTurn()), and the re-issue never fires.

Four attempts to mirror the chat fix failed, and the measurements say why that pipe's accounting is not symmetric:

  • heldVisibleChars counts the done-family, which repeats the turn's text, so it is not a second measure of what the client saw; the deltas are.
  • For an unclosed tag the deltas' filtered text is empty — the span is still pending — so visibleTextChars is 0 and the text arrives through flushTail, which neither counter includes.
  • The held path strips with stripResponsesText, not the shared filter, so a drop there sets no flag.
  • Dropping the heldVisibleChars > 0 clause breaks a pinned test, so that clause is load-bearing for the echo case.

Net: a correct fix must measure the attempt's text once, across the delta, the flush tail and the held family. Left out rather than half-done. The chat wire, which every observed failure was on, has both halves working.

joshwapohlmann pushed a commit to joshwapohlmann/billion-context that referenced this pull request Sep 16, 2026
Resolved: the gate keeps the released-markup measure and drops the
`degenerateRetried ||` guard (that guard made the spent-retry error branch
unreachable) while taking integration's newer CLEAN_TURN_REASONS object form;
the changelog keeps integration's ranxianglei#872/ranxianglei#868 bullets and the branch's copy of the
ranxianglei#871 bullet, which carries the sentences this change added.
@ranxianglei

Copy link
Copy Markdown
Owner

ci失败

@ranxianglei

Copy link
Copy Markdown
Owner

继续

Joshwa Pohlmann added 3 commits September 17, 2026 14:25
…g echo

Plugin-mode turns that end with no visible text and no tool call reach the host
as an empty completed turn, which it then aborts (ranxianglei#673 logs them as degenerate).
The trigger is the render-tag echo: the model imitates the annotation line it saw
in its prompt, the tag filter strips it, and where the echo was the whole turn
nothing is left to send. The filter's own rules are not negotiable — ranxianglei#463 pins
that an all-echo turn is dropped, ranxianglei#673 pins that a typo'd tag never leaks and
that the emptiness is reported — so the turn is recovered rather than redefined.

The pipe re-issues the agent's own request once, with the continuation nudge
ranxianglei#732/ranxianglei#821 already uses for the compress loop, when a turn reaches its terminal
(openai finish_reason chunk, anthropic message_delta) with zero visible text, no
tool call and a clean completion reason. The retry's content is spliced into the
same client turn: the first attempt's terminal event is dropped, its message is
never re-opened, and the retry's anthropic content blocks are offset past the
ones the client already saw. A retry that also comes back empty, or cannot be
issued at all, leaves today's behaviour exactly as it was: the empty turn passes
through and [degenerate-turn] warns.

injectFakeCompletionHint's per-wire body mutation moves to wire-body.ts so both
post-hoc mutations share one implementation, and the retry nudge text moves to
degenerate-retry.ts so the loop and the pipe quote one string.
Joshwa Pohlmann and others added 6 commits September 17, 2026 14:25
…ianglei#870, ranxianglei#732/ranxianglei#821)

The chat pipes re-issue the agent's own body once when a turn reaches its
terminal with nothing visible — the render-tag echo case the filter empties.
The Responses pipe had no such gate, so an echo-only codex subagent turn still
reached the host empty: 8 of one day's degenerate turns in a single session and
56 across all sessions.

Its terminal is a family rather than one event, so the retry HOLDS
response.output_text.done / response.content_part.done /
response.output_item.done until response.completed decides the turn. On a
degenerate completion the held events and that completion are dropped, the body
is re-issued with the same continuation nudge, and the retry is reframed onto
the ids the client already holds: its response.created and added events are
dropped, and item_id, output_index and the completed output ids are rewritten
to the first attempt's. A healthy turn releases the held events in place, so
holding never reorders a turn.

Wired at all three Responses passthrough sites (plugin mode, the native-compact
tee, and proxy mode without injection); the plugin chat pipes' refetch already
carries the URL and headers this needs.
Review follow-ups on ranxianglei#871.

The retry swaps in a new reader while the first attempt's stream is already
terminal and drained; the old one is now closed explicitly in both pipes
instead of being left for the garbage collector to release.

Two tests the reviewer asked for: a chunk that coalesces content with the
finish reason counts its own text (and, when that coalesced content is an echo,
still retries exactly once — the ordering the gate depends on), and a retry
stream that is cut still raises the ranxianglei#721 truncation signal, because takeover
resets the terminal flag.
The retry reframed the response id, the completed output ids and the `item_id`
on deltas, but `response.output_item.*` carries the item's identity nested as
`item.id`. That frame is released to the client, so without this the client
watched the item it held be replaced by the retry's mid-turn.

Found by proving the pipe against the installed bundle rather than the build: a
Responses turn whose only output is an echoed render tag, driven through the real
CLI, reached the client with `item.id = msg_second` while every other id carried
the first attempt's. The regression pin now emits that frame from the retry.
…xianglei#870, PR ranxianglei#871)

A render tag whose close never arrives is dropped by the filter while its
interior survives as the turn's only visible text: the host receives orphan
markup, finds no tool call in it, and stalls. The gate measured that interior
as visible output, so the degenerate-turn retry never fired.

Count the text a dropped tag leaves behind as markup instead, at both the chat
wire's rewrite sites, and re-issue the turn. The recorded architect-session
turn is the fixture, reconstructed from the client-visible residue because the
wire bytes are not kept anywhere.
…nxianglei#870, PR ranxianglei#871)

A turn the host never sees any content from is indistinguishable from a model
that produced nothing, and the client reports neither, so a dead session reads
as an idle one. The degenerate-turn gate already retried once; when that retry
degenerates as well, the client now gets the in-band stream error the host would
never surface, and the retry's own terminal is dropped so the error closes the
turn.

Observed on 2026-09-16 by the session that hit the corruption five times: no
parse error, no warning, the residue as ordinary assistant text.
…lei#870, PR ranxianglei#871)

Two CI regressions found while reviewing ranxianglei#871's red test jobs:

1. The degenerate-turn re-issue fired on the native-compaction trigger
   request (the resetAfterSuccess passthrough), appending the continuation
   nudge AFTER compaction_trigger and breaking the wire shape (ranxianglei#283: the
   trigger must stay the last input item). makeContinuationRefetch now
   refuses to re-issue a body whose final input item is a trigger.

2. The new gates retried turns the model left genuinely bare — zero output
   of any kind — double-billing an upstream empty completion. Both plugin
   pipes now require proof of invisible-but-real output (a stripped
   render-tag/marker echo, released markup, or thought/reasoning), matching
   the boundary the compress loop has kept since ranxianglei#732/ranxianglei#821.
@ranxianglei
ranxianglei force-pushed the 2026-09-16_870-plugin-empty-turn-retry branch from cffedd1 to 86abe3a Compare September 17, 2026 06:49
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

CI failure — root-caused, fixed, re-pushed. All jobs green on 86abe3a.

Reproduced all three red test jobs locally with an A/B against base d650db7. Two were genuine regressions introduced by this PR's new degenerate-turn retry; one was a stale-base artifact:

1. native-compact-terminal — "trigger stays final" (PR-introduced). The resetAfterSuccess passthrough serves the native-compaction trigger request through pipePluginResponsesWithStrip with the new continuation refetch. The trigger's empty-output completed response hit the retry gate, and the re-issued body appended the nudge after compaction_trigger — breaking the wire shape (#283: the trigger must stay the last input item). In production that risks double-compaction / malformed follow-up requests. Fix: makeContinuationRefetch now refuses to re-issue a body whose final input item is a compaction trigger (lazy-parsed at retry time, so healthy turns pay nothing). This mirrors the guard the proxy-mode nudge path already carries (isCompactionTrigger, src/server.ts).

2. responses-tool-images — plugin-mode TypeError (PR-introduced). The new gate also retried turns where the model produced nothing at all — no text, no reasoning, no tool call — i.e. the upstream's own empty completion. That double-bills every empty completion and appends a stray user turn to the forwarded request (the endsWith crash came from filtering that appended item). The compress loop has kept a stricter boundary since #732/#821: only invisible-but-real output retries, never a genuinely bare turn. Fix: both plugin pipes now require evidence of invisible-but-real output before retrying — chat pipe: thought, a stripped tag/marker echo, or released markup; Responses pipe: reasoning, or a stripped echo seen in deltas / done-family events.

3. #857 D: upward self-heal trusts usage baselines only — pre-existing, not from this PR. Fails byte-identically on the clean base; already fixed on master by 323d8c1f (post-#852 DeepSeek 1M window table). Resolved by the rebase below.

What changed: rebased onto current master 55da3f0, plus one commit 86abe3a — 40 lines in src/degenerate-retry.ts + src/plugin.ts, nothing else. Pre-flight: typecheck ✓, full suite 1688/1688 ✓, build ✓. CI on 86abe3a: all 8 jobs green, including the four test-matrix jobs that were red. The PR's own 18 retry-contract tests pass unmodified.

Ready for merge whenever.

修了什么问题:PR 新增的空轮重试有两个回归——(1) 会对 native-compaction 触发请求重发、把 nudge 追加到 compaction_trigger 之后破坏 wire 形状;(2) 对模型完全无输出的空 completion 也重试(双倍计费)。已在 86abe3a 修复(trigger 结尾的 body 一律不重发;重试门前要求存在"不可见但真实"的输出证据),并 rebase 到最新 master(第三个失败是旧基线问题,master 已修)。本地全量 1688 测试全绿、CI 8 个 job 全绿,可以合并。

@ranxianglei
ranxianglei merged commit a7ed529 into ranxianglei:master Sep 17, 2026
8 checks passed
ranxianglei pushed a commit that referenced this pull request Sep 17, 2026
A model imitating the render tag can write its whole turn — tool call
included — where the opening's attributes are still open, so the attribute
list never terminates and no close tag is ever written (recorded shape:
`<acp tokens="1" text="text` immediately followed by the turn's own
markup, architect session 01a0a0cb, 2026-09-16T17:29:09). Every tag pattern
misses the span at once: attribute classes stop at `<`, and the value opened
with a quote is never closed. The filter dropped the opening and released the
payload, and the agent received a completed turn of orphan
parameter/invoke/calls closers with no tool call in it, stalling until nudged.

The stripper now recognises the wrapped shape by form — the attribute list
runs into a `<` (BROKEN_ATTRS), or carries an odd number of quotes — and
swallows the span whole to a loose close tag or to the end of the turn, in
stripAcpTags and in the streaming filter alike, with a budget of its own so
the #644 rule for a plain opening's over-long tail stays intact. The turn
reaches the client empty, which the degenerate-turn retry (#870/#871) re-asks
for.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants