Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions TECHNICAL-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,3 +375,35 @@ parse failures) forward as-is and log `[tag-echo] raw exit` so new leak shapes
become visible without changing the wire. Pinned by
`tests/unified-acp-invariants.test.ts` (eight tests, both lanes).


## Degenerate-turn retry escalation — two nudges, then an honest failure (#2176)

The empty-turn re-issue (`src/degenerate-retry.ts`, fired by both plugin
pipes when a turn terminates with nothing client-visible — the chronic
tag-echo shape) is bounded at `DEGENERATE_RETRY_MAX_ATTEMPTS = 2`:

1. **First re-issue — the plain continuation nudge** (#732/#821 wording,
unchanged): "continue, take your next concrete action".
2. **Second re-issue — the anti-echo escalation**: names the behavior to
stop (message-reference tags are stripped before anyone reads them) and
the output to produce instead (plain prose or a tool call). Mentioning
the tags is deliberate: the turn is already lost at that point, and the
instruction rides a one-shot trailing user message that is never
persisted nor replayed.

After both re-issues degenerate, each pipe keeps its existing honest
terminal: the chat pipe emits the #870 in-band error (a COMPLETED turn
carrying the error text), the Responses pipe lets the retry's own
completion close the turn. The compress-loop lane keeps its original
one-shot bound — its degenerate rounds are tool-result rounds, not user
turns, and were not part of the #2176 report.

**Observability** (#2176): every fire increments
`session.stats.degenerateRetries`, every unrecoverable turn increments
`session.stats.degenerateExhausted` (both persisted, restart-stable), and
`GET /__bili/status` exposes the aggregate as `degenerate: { sessions,
retries, exhausted }`. The exhausted counter is gated behind the same
visible-text predicates as the retry itself, so a recovered escalation is
never miscounted. Nonzero `exhausted` is the signal that the escalation
did not save the turn — the tail is first-cause territory (injection
format), not retry-count territory.
55 changes: 45 additions & 10 deletions src/degenerate-retry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,31 @@ import { appendTrailingUserText } from "./wire-body.js";
// turn and the host aborts it. The pipe can still re-issue the SAME body
// (plus this nudge) because it holds the upstream URL and the forwarded
// headers, and it can splice the retry's content into the open client stream.
// Bounded to one attempt per turn; the body mutation never touches session
// state, so the nudge is neither persisted nor replayed on the client's next
// request.
// Bounded: the first re-issue carries the plain continuation nudge, a second
// (final) re-issue carries the anti-echo escalation (#2176: chronic tag-echo
// where the model keeps answering with markup only). The body mutation never
// touches session state, so the nudge is neither persisted nor replayed on
// the client's next request.
export const DEGENERATE_RETRY_NUDGE =
"[billion-context] Your previous response ended with no visible text and no tool call. Continue now: take your next concrete action.";

/** #2176: the second (escalated) re-issue. The first nudge says "continue";
* a model that answers with markup again is imitating the render tags it saw
* in its prompt, so this one names the behavior to stop and the output to
* produce instead. Mentioning the tags here is deliberate: at this point the
* turn is already lost, and the instruction rides a one-shot trailing user
* message that is never persisted. */
export const DEGENERATE_RETRY_NUDGE_ESCALATED =
"[billion-context] Your last two responses contained only message-reference tags with no visible text and no tool call — those tags are stripped before anyone reads them. Do not output them. Answer now in plain prose, or make a tool call.";

/** #2176: re-issues per degenerate turn (1 = plain nudge, 2 = escalated). */
export const DEGENERATE_RETRY_MAX_ATTEMPTS = 2;

/** The retry body: the forwarded body with the continuation nudge appended as a
* trailing user turn. Null when the body cannot carry one. */
export function injectContinuationNudge(protocol: WireProtocol, body: string | Buffer): string | null {
return appendTrailingUserText(protocol, body, DEGENERATE_RETRY_NUDGE);
* trailing user turn. Null when the body cannot carry one. `escalated` selects
* the #2176 anti-echo nudge (second and final attempt). */
export function injectContinuationNudge(protocol: WireProtocol, body: string | Buffer, escalated = false): string | null {
return appendTrailingUserText(protocol, body, escalated ? DEGENERATE_RETRY_NUDGE_ESCALATED : DEGENERATE_RETRY_NUDGE);
}

export interface ContinuationRetryOpts {
Expand All @@ -47,9 +62,10 @@ export interface ContinuationRetryOpts {
* function for the stream pipe: it yields the retry's response body, or null
* when the retry cannot be issued (unbuildable body, upstream error, client
* already gone) — the caller then passes the original empty turn through
* unchanged. */
export function makeContinuationRefetch(opts: ContinuationRetryOpts): () => Promise<ReadableStream<Uint8Array> | null> {
return async () => {
* unchanged. Callers that escalate pass `true` for the #2176 anti-echo nudge
* (the caller tracks attempt count; the closure is stateless). */
export function makeContinuationRefetch(opts: ContinuationRetryOpts): (escalated?: boolean) => Promise<ReadableStream<Uint8Array> | null> {
return async (escalated = false) => {
if (opts.signal.aborted) return null;
// A body whose final input item is a compaction trigger must never be
// re-issued: appending the nudge after it breaks the wire shape (#283:
Expand All @@ -67,7 +83,7 @@ export function makeContinuationRefetch(opts: ContinuationRetryOpts): () => Prom
/* unparseable body: injectContinuationNudge below reports and skips */
}
}
const retryBody = injectContinuationNudge(opts.protocol, opts.body);
const retryBody = injectContinuationNudge(opts.protocol, opts.body, escalated);
if (retryBody === null) {
opts.log("warn", `[${opts.label}] [plugin] degenerate-terminal retry skipped: request body carries no turn array`);
return null;
Expand Down Expand Up @@ -103,3 +119,22 @@ export function makeContinuationRefetch(opts: ContinuationRetryOpts): () => Prom
}
};
}

/** #2176: aggregate view of the degenerate-retry counters carried in
* session.stats, surfaced by /__bili/status so the chronic tag-echo tail is
* measurable in the field (fired re-issues and turns no re-issue could
* save). Sessions without counters are skipped from the session count. */
export function degenerateRetrySummary(sessions: Array<{ stats?: { degenerateRetries?: number; degenerateExhausted?: number } }>): { sessions: number; retries: number; exhausted: number } {
let touched = 0;
let retries = 0;
let exhausted = 0;
for (const s of sessions) {
const r = s.stats?.degenerateRetries ?? 0;
const e = s.stats?.degenerateExhausted ?? 0;
if (r === 0 && e === 0) continue;
touched += 1;
retries += r;
exhausted += e;
}
return { sessions: touched, retries, exhausted };
}
88 changes: 56 additions & 32 deletions src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import { ccrEnabled, ccrLoopConfig, contentStoreOf, retrieveToolName } from "./s
import { imageUsageSuffix } from "./image-compress.js";
import { emitStreamError, emitUpstreamTruncation } from "./stream-error.js";
import { degenerateTurnWarning } from "./degenerate-turn.js";
import { DEGENERATE_RETRY_MAX_ATTEMPTS } from "./degenerate-retry.js";
import { PANEL_BOX_FOOTER } from "./acp-panel.js";
import { describeAdvisory, getAdvisoryState } from "./advisory.js";
import { describeUpdateReady, getUpdateVisibility } from "./update-notes.js";
Expand Down Expand Up @@ -1709,10 +1710,11 @@ const ANTHROPIC_BLOCK_EVENT = /^content_block_(start|delta|stop)$/;
* tiny input_tokens would clobber lastInputTokens and break compression
* triggering for the main conversation.
*
* `refetch` supplies the one-shot degenerate-turn retry (#732/#821): when the
* turn reaches its terminal with nothing visible — the tag-echo case, where
* the filter empties the only text block so the host aborts an empty turn —
* the pipe re-issues the request through it and splices the retry's content
* `refetch` supplies the degenerate-turn retry (#732/#821, #2176 escalation):
* when the turn reaches its terminal with nothing visible — the tag-echo
* case, where the filter empties the only text block so the host aborts an
* empty turn — the pipe re-issues the request through it (up to twice: a
* plain nudge, then an anti-echo escalation) and splices the retry's content
* into the client stream the first attempt already opened. Omit it for the
* plain pass-through. */
export async function pipePluginChatWithStrip(
Expand All @@ -1721,7 +1723,7 @@ export async function pipePluginChatWithStrip(
protocol: WireProtocol,
session?: Session,
log?: (msg: string) => void,
refetch?: () => Promise<ReadableStream<Uint8Array> | null>,
refetch?: (escalated?: boolean) => Promise<ReadableStream<Uint8Array> | null>,
upstreamOrigin?: string,
// True when the outbound request carried the [ACP absorb] instruction:
// the model may then answer with a tool call written as prose, and the
Expand Down Expand Up @@ -1849,45 +1851,51 @@ export async function pipePluginChatWithStrip(
// terminal event is dropped when the retry takes over, so the client sees
// one turn: its framing stays open, and the retry's content blocks are
// shifted past the ones already streamed.
let degenerateRetried = false;
let degenerateRetries = 0;
let truncationRetried = false;
let forwardedAny = false;
let inRetry = false;
let retryIndexOffset = 0;
let blocksForwarded = 0;
/** One-shot re-issue when a turn reaches its terminal with nothing visible:
* the tag-echo case, where the filter empties the only text block and the
* host aborts an empty completed turn. Returns true when the retry stream
* took over, in which case the caller drops the terminal event of the
* attempt it came from. Consults truncationRetried: a request whose stream
* already spent its one re-issue on a zero-visible cut (#2171) cannot also
* re-send on a degenerate completion — one re-issue per request, total. */
/** Degenerate-turn re-issues (#732/#821), #2176-escalated to at most
* DEGENERATE_RETRY_MAX_ATTEMPTS: the first carries the plain continuation
* nudge, a second carries the anti-echo escalation. Returns true when the
* retry stream took over, in which case the caller drops the terminal
* event of the attempt it came from. Consults truncationRetried: a request
* whose stream already spent its re-issue on a zero-visible cut (#2171)
* cannot also re-send on a degenerate completion. */
const retryEmptyTurn = async (reason: string | undefined): Promise<boolean> => {
if (refetch === undefined || truncationRetried) return false;
// Markup released from a held span carries nothing the host can act on:
// an unclosed render tag stalls the turn exactly like an empty one.
if (visibleTextChars > releasedMarkupChars || sawToolUse) return false;
if (reason === undefined || !CLEAN_TURN_REASONS.has(reason)) return false;
if (res.destroyed || res.writableEnded) return false;
if (degenerateRetried) {
// The retry degenerated too. An empty turn is indistinguishable from a
// model that produced nothing and the session reads as idle while it is
// dead, so the client gets an error the host would never surface (#870).
log?.("[plugin] degenerate terminal turn again after the retry; emitting an in-band error (#870)");
if (degenerateRetries >= DEGENERATE_RETRY_MAX_ATTEMPTS) {
// Every re-issue degenerated too. An empty turn is indistinguishable
// from a model that produced nothing and the session reads as idle
// while it is dead, so the client gets an error the host would never
// surface (#870).
log?.("[plugin] degenerate terminal turn again after the continuation nudges; emitting an in-band error (#870)");
if (session) session.stats.degenerateExhausted = (session.stats.degenerateExhausted ?? 0) + 1;
// #870 deliberately chose a COMPLETED turn carrying the error text (visible to the host); keep the legacy shape here regardless of the global streamErrorShape default.
emitStreamError(res, protocol, "the turn degenerated again after the continuation nudge", undefined, "completion");
emitStreamError(res, protocol, "the turn degenerated again after the continuation nudges", undefined, "completion");
return true;
}
// A turn the model left genuinely bare — no thought, no stripped echo,
// no released markup — is the upstream's own empty answer, not a stall:
// re-issuing it double-bills an empty completion (#732/#821 keep the
// same boundary in the compress loop).
if (!sawThinking && !sawStrippedEcho && releasedMarkupChars === 0) return false;
degenerateRetried = true;
log?.("[plugin] degenerate terminal turn (no usable output); retrying once with a continuation nudge (#732/#821)");
const escalated = degenerateRetries > 0;
degenerateRetries += 1;
if (session) session.stats.degenerateRetries = (session.stats.degenerateRetries ?? 0) + 1;
log?.(escalated
? "[plugin] degenerate terminal turn again; escalating with the anti-echo nudge (#2176)"
: "[plugin] degenerate terminal turn (no usable output); retrying with a continuation nudge (#732/#821)");
let next: ReadableStream<Uint8Array> | null = null;
try {
next = await refetch();
next = await refetch(escalated);
} catch (e) {
log?.(`[plugin] degenerate-terminal retry failed (${e instanceof Error ? e.message : String(e)}); passing the empty turn through`);
return false;
Expand Down Expand Up @@ -1929,7 +1937,7 @@ export async function pipePluginChatWithStrip(
* the #721 in-band error (partial content must not be regenerated). */
const retryZeroByteCut = async (): Promise<boolean> => {
if (refetch === undefined) return false;
if (degenerateRetried || truncationRetried) return false;
if (degenerateRetries > 0 || truncationRetried) return false;
// Zero-visible predicate: no framing opened, no content block started,
// no prose accumulated or held by the tag filter, no unparseable frame
// forwarded verbatim, no partial SSE event left in the buffer.
Expand Down Expand Up @@ -2691,15 +2699,16 @@ function googleFrameHasNonText(ev: Record<string, unknown>): boolean {
* #732/#821 parity with pipePluginChatWithStrip: when the turn's completion
* reports a clean status with nothing visible — the echoed render tag was the
* only thing the model emitted, and the filter emptied it — the agent's own
* body is re-issued ONCE with a continuation nudge instead of leaving the host
* body is re-issued with a continuation nudge (up to twice: plain, then the
* #2176 anti-echo escalation) instead of leaving the host
* with an empty completed turn. The retry is reframed onto the ids the client
* already holds, so the client still sees one turn. */
export async function pipePluginResponsesWithStrip(
stream: ReadableStream<Uint8Array>,
res: ServerResponse,
session?: Session,
log?: (msg: string) => void,
refetch?: () => Promise<ReadableStream<Uint8Array> | null>,
refetch?: (escalated?: boolean) => Promise<ReadableStream<Uint8Array> | null>,
upstreamOrigin?: string,
// See pipePluginChatWithStrip: whole-field tool-call emission drop,
// gated on the request carrying the [ACP absorb] instruction (m00885).
Expand Down Expand Up @@ -2758,7 +2767,7 @@ export async function pipePluginResponsesWithStrip(
// output_item.added releases them early instead — clients require
// done(itemN) before added(itemN+1) (#1061) — so by the terminal only the
// last item's family can still be held.
let degenerateRetried = false;
let degenerateRetries = 0;
let truncationRetried = false;
let forwardedAny = false;
let inRetry = false;
Expand Down Expand Up @@ -2794,7 +2803,7 @@ export async function pipePluginResponsesWithStrip(
* keeps the #721 in-band error. */
const retryZeroByteCut = async (): Promise<boolean> => {
if (refetch === undefined) return false;
if (degenerateRetried || truncationRetried) return false;
if (degenerateRetries > 0 || truncationRetried) return false;
// Zero-visible predicate: no item framing held, no status observed, no
// prose accumulated or held by the filters, no verbatim frame
// forwarded, no partial SSE event left in the buffer.
Expand Down Expand Up @@ -2955,8 +2964,9 @@ export async function pipePluginResponsesWithStrip(
* Returns true when the retry stream took over, in which case the caller
* drops the held done-family events AND the completion it came from. */
const retryEmptyTurn = async (status: string | undefined): Promise<boolean> => {
// truncationRetried: one re-issue per request, total — see the chat-pipe twin.
if (degenerateRetried || truncationRetried || refetch === undefined) return false;
// truncationRetried: see the chat-pipe twin — a zero-visible cut (#2171)
// spends the budget a degenerate completion cannot also spend.
if (refetch === undefined || truncationRetried) return false;
if (visibleTextChars > 0 || heldVisibleChars > 0 || sawFunctionCall) return false;
if (status !== "completed") return false;
if (res.destroyed || res.writableEnded) return false;
Expand All @@ -2965,11 +2975,25 @@ export async function pipePluginResponsesWithStrip(
// double-bills an empty completion (#732/#821 keep the same boundary in
// the compress loop).
if (!sawReasoning && !sawStrippedEcho) return false;
degenerateRetried = true;
log?.("[plugin] degenerate terminal turn (no visible output); retrying once with a continuation nudge (#732/#821)");
if (degenerateRetries >= DEGENERATE_RETRY_MAX_ATTEMPTS) {
// Both re-issues degenerated: unlike the chat pipe (#870 in-band
// error) this lane lets the retry's own completion close the turn;
// count the exhaustion so /__bili/status can see the tail (#2176).
// Checked AFTER the visible-text gates so a recovered escalation
// (this same guard runs at its terminal too) is not miscounted.
log?.("[plugin] degenerate terminal turn again after the continuation nudges (#2176)");
if (session) session.stats.degenerateExhausted = (session.stats.degenerateExhausted ?? 0) + 1;
return false;
}
const escalated = degenerateRetries > 0;
degenerateRetries += 1;
if (session) session.stats.degenerateRetries = (session.stats.degenerateRetries ?? 0) + 1;
log?.(escalated
? "[plugin] degenerate terminal turn again; escalating with the anti-echo nudge (#2176)"
: "[plugin] degenerate terminal turn (no visible output); retrying with a continuation nudge (#732/#821)");
let next: ReadableStream<Uint8Array> | null = null;
try {
next = await refetch();
next = await refetch(escalated);
} catch (e) {
log?.(`[plugin] degenerate-terminal retry failed (${e instanceof Error ? e.message : String(e)}); passing the empty turn through`);
return false;
Expand Down
Loading
Loading