From 684a295a1e7ad89c9faba3a17e942f6708e13fd1 Mon Sep 17 00:00:00 2001 From: awork Date: Sat, 17 Oct 2026 05:02:08 +0800 Subject: [PATCH 1/4] fix: book upstream-stated overflow counts into the fold target (#3450) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An upstream overflow 400 that states the rejected payload's own counts ('you requested N tokens (M in the messages, K in the completion)') fed only its WINDOW into the one-shot arm — and discarded M and K entirely. The arm booked a ceiling as if it were a measurement, so preflight saw 'payload ~window, deficit 0' on an underestimating local meter and forwarded a payload the upstream then billed over the wall (messages AND completion against the same window), killing the turn. - inspectContextOverflow now extracts requestedTotal/requestedMessages/ requestedCompletion from provider-specific phrasings (OpenAI/DeepSeek messages+completion split; Anthropic 'prompt is too long'; Gemini 'input token count') — a wrong guess is worse than no guess, so each pattern stays provider-specific. - The arm books the measured MESSAGE count (a ceiling is not a size); the side-request guard keeps the CEILING via overflowArmTokens. - The proven completion share is reserved off the wall for every payload-vs-wall judgment while the arm lives (the foldable quantity is MESSAGES); it retires with the arm on the next real usage report. - One wall per request on the refold path: prepare/nudge and preflight judge on min(declared, upstream-stated) instead of two different walls. - commitFold nets each fold out of the input baseline like the live path does, so an armed floor at-or-above the target descends and the loop can exit on fit instead of burning budget or stalling. - One-time warning when the upstream-stated window differs from the window in use, naming both values (operator hint: modelContextLimit). Behavior changes (disclosed): armed baseline is the upstream measurement when stated (was: the stated window); preflight judges on window minus the proven completion reservation while armed; lastInputTokens nets down during preflight folds (was monotonic-increase within a run); new persisted stat fields overflowArmCompletionReserve / overflowArmMismatchWarned (legacy files: absent = legacy behavior). Tests: parser cases incl. the verbatim incident body + regression guards; hermetic e2e for honest-fail (uncloseable gap surfaces as 400, never a masked over-forward) and in-request rescue (fold under the message budget, reservation retires with the arm). issue1195 B updated to the new arm value (measured 130000 vs ceiling 128000). --- CONFIGURATION.md | 1 + CONFIGURATION.zh-CN.md | 1 + src/cache-ledger.ts | 5 + src/persist.ts | 5 + src/preflight.ts | 14 +- src/server.ts | 9 +- src/server/handle.ts | 16 +- src/server/relay.ts | 38 +++- src/session.ts | 15 ++ src/util.ts | 41 +++- tests/context-overflow.test.ts | 95 ++++++++ tests/e2e-issue3450-stated-counts.test.ts | 255 ++++++++++++++++++++++ tests/issue1195-overflow-refold.test.ts | 6 +- 13 files changed, 489 insertions(+), 12 deletions(-) create mode 100644 tests/e2e-issue3450-stated-counts.test.ts diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 0f4160037e..c0abc3d3a5 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -1259,6 +1259,7 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t - **Default:** *(the model's native window)* - **Status:** ACTIVE - **Description:** The context window size, in tokens. This is the **denominator** the engine uses for its usage ratio (`usage = tokens / modelContextLimit`) — it is **not** a truncation cap. Accepts an absolute number (`200000`) or a percent string (`"80%"` = 80% of the model's native window, resolved from the built-in table or models.dev registry). When omitted at every level, the native window is used. This is the highest-priority source for the model limit; it overrides the built-in table, the legacy per-model `context` field, and the top-level `modelContextLimit`. Note it also serves as the **hard preflight wall**: once a payload reaches this value the proxy proactively folds context before forwarding, and if folding cannot bring it under, the request fails fast instead of being sent upstream. To keep day-to-day context small while still letting large reads burst up to the native window, keep `modelContextLimit` at its native value and use `maxContextLimit` as the soft target instead (see [Soft target with elastic headroom](#soft-target-with-elastic-headroom-1122)). +- **Overflow-arm behavior (#3450).** When upstream rejects a payload with a stated-window error, the proxy arms that rejection for the retry: if the window declared here differs from the one upstream states, it logs a one-time warning naming both values and folds against the smaller of the two; if the error body also states token counts (e.g. `you requested N tokens (M in the messages, K in the completion)`), `M` is booked as the measured payload size (a ceiling is not a size) and `K` is subtracted from this wall for fit judgments while the arm lives — upstream proved it bills the completion share against the same budget. The next real usage report retires the arm and the reservation together. If your declaration is wrong, set this key to the true window: the warning goes away and the fold target follows. #### `outputHeadroomMaxPct` diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 11a4f9a397..0209fef40d 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -1262,6 +1262,7 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 - **默认值:** *(模型的原始窗口)* - **状态:** ACTIVE - **说明:** 上下文窗口大小,以 token 为单位。它是引擎用于计算使用率比例的**分母**(`usage = tokens / modelContextLimit`)—— 它**不是**截断上限。接受绝对数值(`200000`)或百分比字符串(`"80%"` = 模型原始窗口的 80%,从内置表或 models.dev 注册表解析)。在每个层级都省略时,使用原始窗口。这是模型上限的最高优先级来源;它会覆盖内置表、旧版按模型的 `context` 字段以及顶层的 `modelContextLimit`。注意它同时也是**预检压缩的硬墙**:一旦载荷达到该值,代理会在转发前主动折叠上下文;若折叠后仍超出,请求会直接快速失败而不是发往上游。若希望日常上下文保持较小、同时允许大读取任务突发到原始窗口,请让 `modelContextLimit` 保持为原始窗口值,改用 `maxContextLimit` 作为软目标(见[软目标与弹性余量](#软目标与弹性余量-1122))。 +- **溢出武装行为 (#3450)。** 当上游以"窗口超限"错误拒绝载荷时,代理会把这次拒绝"武装"给下一次尝试:若此处声明的窗口与上游给出的数值不一致,会记一条一次性警告(同时给出两个值)并按两者中较小者折叠;若错误正文还给出了 token 计数(如 `you requested N tokens (M in the messages, K in the completion)`),则把 `M` 记为实测载荷大小(上限不是尺寸),并在该武装存续期间从本墙中扣除 `K` 再判断载荷是否装得下——上游已证明完成份额与输入共用同一预算。下一次真实的 usage 报告会同时退役该武装与预留。如果你的声明有误,把本键设为真实窗口即可:警告消失,折叠目标随之修正。 #### `outputHeadroomMaxPct` diff --git a/src/cache-ledger.ts b/src/cache-ledger.ts index 4162bfb3ca..8d9486886b 100644 --- a/src/cache-ledger.ts +++ b/src/cache-ledger.ts @@ -1524,7 +1524,12 @@ export function settleUsageReport( // anchor immediately — no need to wait out its prepare budget. clearPostRebuildAnchor(session); // #1110: a real usage report retires the one-shot overflow arm. + // #3450: the proven completion reservation retires with it — it was + // evidence about the rejected payload's turn; a recovered session + // returns to judging on the declared window (a repeat overflow + // re-arms both in one event). delete session.stats.overflowArmTokens; + delete session.stats.overflowArmCompletionReserve; } // #1933 F1: roll the pending pair forward to the current turn's value — // it will be consumed by the NEXT report (or cleared when the lane diff --git a/src/persist.ts b/src/persist.ts index 1f631053fa..00ef411af3 100644 --- a/src/persist.ts +++ b/src/persist.ts @@ -143,6 +143,8 @@ interface PersistedSession { lastInputTokensSource?: string; lastInputTokensOrigin?: string; overflowArmTokens?: number; + overflowArmCompletionReserve?: number; + overflowArmMismatchWarned?: boolean; contextTokens?: number; contextTokensSource?: string; // #2129: measurement-state fields buildRecord persists via spread — @@ -2488,6 +2490,9 @@ function buildSession(parsed: PersistedSession): Session { lastInputTokensSource: stats.lastInputTokensSource === "usage" || stats.lastInputTokensSource === "estimate" || stats.lastInputTokensSource === "overflow-arm" ? stats.lastInputTokensSource : undefined, // #1110: one-shot overflow arm — legacy files lack it; absent = no arm. overflowArmTokens: typeof stats.overflowArmTokens === "number" && Number.isFinite(stats.overflowArmTokens) && stats.overflowArmTokens > 0 ? stats.overflowArmTokens : undefined, + // #3450: proven completion reservation + one-time warn flag — legacy files lack both. + overflowArmCompletionReserve: typeof stats.overflowArmCompletionReserve === "number" && Number.isFinite(stats.overflowArmCompletionReserve) && stats.overflowArmCompletionReserve > 0 ? stats.overflowArmCompletionReserve : undefined, + overflowArmMismatchWarned: stats.overflowArmMismatchWarned === true ? true : undefined, // In-memory only — a fresh process has no pending compress fold. compressCreditTokens: 0, contextTokens: Math.max(0, stats.contextTokens ?? parsed.contextTokens ?? 0), diff --git a/src/preflight.ts b/src/preflight.ts index d0397e51d1..2858aa55e1 100644 --- a/src/preflight.ts +++ b/src/preflight.ts @@ -2070,8 +2070,20 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa // char count against it instead of the token-based credit. const compressed = deps.session.stats.compressCreditTokens - creditBefore; const folded = baselineKnown ? compressed : messages.filter((message) => effectiveMessageIds.includes(message.id)).reduce((total, message) => total + (message.text ?? "").length, 0); + const summaryCost = defaultCountTokens(summary); currentTokens = Math.max(0, currentTokens - folded + countText(summary)); - deps.session.stats.lastInputTokens += defaultCountTokens(summary); + // #3450: net the fold out of the input baseline like the live path + // (stream.ts) does — the summary re-enters the payload but the + // folded mass leaves it. Without this an armed/usage floor at-or- + // above the target never descends and the loop cannot exit on fit + // (it only ever burned budget or stalled). Token caliber matches + // only when baselineKnown; the char-based regime keeps the legacy + // summary-cost addition. + if (baselineKnown) { + deps.session.stats.lastInputTokens = Math.max(0, deps.session.stats.lastInputTokens - Math.max(0, folded - summaryCost)); + } else { + deps.session.stats.lastInputTokens += summaryCost; + } appliedThisRound += 1; result.compressedRanges += 1; return true; diff --git a/src/server.ts b/src/server.ts index fddc3bf4cd..f8c8e5089c 100644 --- a/src/server.ts +++ b/src/server.ts @@ -2995,7 +2995,12 @@ export async function preflightCompressIfNeeded( instanceId: string, ): Promise { const session = prepared.session; - const limit = config.modelContextLimit; + // #3450: when an upstream-stated overflow proved this provider bills the + // COMPLETION against the same wall ("N in the messages, N in the + // completion"), the foldable quantity is MESSAGES — every payload-vs-wall + // judgment below runs on window − reservation instead of the raw window. + const completionReserve = typeof session.stats.overflowArmCompletionReserve === "number" && session.stats.overflowArmCompletionReserve > 0 ? session.stats.overflowArmCompletionReserve : 0; + const limit = Math.max(0, config.modelContextLimit - completionReserve); const overflowTarget = prepared.protocol === "responses" && isCodexClient(req.headers) && codexCompactMode() === "intercept" ? limit * CODEX_COMPACT_HEALTH_RATIO : limit; @@ -3314,7 +3319,7 @@ export async function preflightCompressIfNeeded( // provider-billed baseline and the (calibrated) local estimate — so a // false trigger is diagnosable from the log alone instead of requiring a // cross-reference between gate and nudge lines. - log("warn", `[${session.id}] context ${tokenCount} tokens reached preflight target ${compressionTarget}${growthArmed ? " (auto-fold growth floor)" : ""} (model window ${limit}, model=${model}; usage-baseline=${baselineFloor > 0 ? baselineFloor : "none"} local-est=${Math.round(calibratedText)}${kFactor !== undefined ? ` raw=${Math.round(textEstimate + overheadEstimate)} k̂=${kFactor.toFixed(2)}` : ""}); preflight compressing before forward`); + log("warn", `[${session.id}] context ${tokenCount} tokens reached preflight target ${compressionTarget}${growthArmed ? " (auto-fold growth floor)" : ""}${completionReserve > 0 ? ` (window ${config.modelContextLimit} minus proven completion reservation ${completionReserve})` : ""} (model window ${limit}, model=${model}; usage-baseline=${baselineFloor > 0 ? baselineFloor : "none"} local-est=${Math.round(calibratedText)}${kFactor !== undefined ? ` raw=${Math.round(textEstimate + overheadEstimate)} k̂=${kFactor.toFixed(2)}` : ""}); preflight compressing before forward`); // #300: stamp the chain marker so a downstream bili skips these // summarization calls too (preflight always processes). const { upstreamUrl, headers, proxyUrl } = buildForwardTarget(req, opts, route, affinity, instanceId); diff --git a/src/server/handle.ts b/src/server/handle.ts index 84aa9e128f..40f51f9552 100644 --- a/src/server/handle.ts +++ b/src/server/handle.ts @@ -2359,6 +2359,17 @@ async function handleRequest( logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, forwardBody); return { body: forwardBody, prepared: null }; } + // #3450: ONE wall per request — on the refold path the + // prepare/nudge stage must judge against the same window the + // preflight stage folds on. min(declared, stated): the + // operator/host declaration and the upstream's own proof bound + // each other; neither may outrank the other within one + // request (the two-walls split let "usage=105%" and + // "target 1048576" coexist in the same second). Downward only + // — a stated ceiling above the declaration never loosens it. + if (overflowWindow !== undefined && typeof reqConfig.modelContextLimit === "number" && reqConfig.modelContextLimit > 0 && overflowWindow < reqConfig.modelContextLimit) { + reqConfig = { ...reqConfig, modelContextLimit: overflowWindow }; + } prepared = await runPrepare(); // [#2926] anthropic Threads lane: prepare mirrored the turn and // may have armed a reforge (mirror estimate crossed the window @@ -2440,7 +2451,10 @@ async function handleRequest( res, opts, core, - overflowWindow !== undefined ? { ...reqConfig, modelContextLimit: overflowWindow } : reqConfig, + // #3450: no per-stage override anymore — the wall was + // unified above (min of declared and upstream-stated), + // so prepare/nudge and preflight judge on one number. + reqConfig, nativeWindow, requestModel, resolvedNativeWindow, diff --git a/src/server/relay.ts b/src/server/relay.ts index b4fdfa92a8..9f4a5b44c9 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -1028,11 +1028,22 @@ export async function forward( // re-sent the same oversized history every turn — 400 forever. // Clamping to min(stated, reserved) keeps the arm upstream-truth // when the two agree, and only bites when a reservation exists. + // + // #3450: when the body states the REJECTED PAYLOAD'S OWN size + // ("N in the messages"), book THAT as the armed baseline — the + // window is a ceiling, not a measurement; arming at the ceiling + // made preflight see "payload ~window, deficit 0" while the real + // payload sat above its local estimate and the re-forward died. + // The #3738 clamp then caps that measurement at the judging limit + // (a stated count exceeds the reserved limit exactly when the + // completion share pushed the total over the window). + const measured = typeof info.requestedMessages === "number" && info.requestedMessages > 0 ? info.requestedMessages : undefined; const judgingLimit = typeof s.metadata.effectiveContextLimit === "number" && s.metadata.effectiveContextLimit > 0 ? s.metadata.effectiveContextLimit : 0; - const armedWindow = judgingLimit > 0 && judgingLimit < info.window ? judgingLimit : info.window; - s.stats.lastInputTokens = armedWindow; + const unclampedBaseline = measured ?? info.window; + const armedBaseline = judgingLimit > 0 && judgingLimit < unclampedBaseline ? judgingLimit : unclampedBaseline; + s.stats.lastInputTokens = armedBaseline; s.stats.lastInputTokensSource = "overflow-arm"; // #2313: the rejection came from THIS upstream — stamp the // evidence origin so the #1933 F2 route gate (and the #2313 @@ -1040,9 +1051,26 @@ export async function forward( // instead of demoting it against a stale settle origin. s.stats.lastInputTokensOrigin = normalizeUpstreamOrigin(upstreamUrl); // #1110: record the arm SEPARATELY so the side-request guard - // can read it without ever touching the nudge baseline. - s.stats.overflowArmTokens = armedWindow; - log("warn", `[${s.id}] upstream context overflow (model=${reqModel ?? "unknown"}) — window ${info.window} stated upstream; armed emergency shrink at ${armedWindow}${armedWindow === info.window ? "" : ` (clamped to the output-headroom-reserved judging limit ${judgingLimit})`}, declared window unchanged (#987)`); + // can read it without ever touching the nudge baseline. The guard + // keeps the CEILING even when the baseline books a measured or + // clamped value (#3450) — it must never clamp below what the + // upstream proved, and it reserves each side request's own output + // from this value itself, so feeding it a pre-reserved number + // would double-subtract the headroom. + s.stats.overflowArmTokens = info.window; + // #3450: the completion share proves this provider bills OUTPUT + // against the same wall — remember it so preflight folds against + // window − completion (the foldable quantity is MESSAGES). + const reserve = typeof info.requestedCompletion === "number" && info.requestedCompletion > 0 ? info.requestedCompletion : undefined; + if (reserve !== undefined) s.stats.overflowArmCompletionReserve = reserve; + // #3450: one-time mismatch warning — the upstream-stated window + // differs from the window bili actually judges on. + const declaredWindow = typeof s.metadata.effectiveContextLimit === "number" ? s.metadata.effectiveContextLimit : 0; + if (declaredWindow > 0 && declaredWindow !== info.window && !s.stats.overflowArmMismatchWarned) { + s.stats.overflowArmMismatchWarned = true; + log("warn", `[${s.id}] upstream-stated window ${info.window} differs from the window in use (${declaredWindow}, model=${reqModel ?? "unknown"}) — folding targets min of the two; set compress.modelContextLimit if the declaration is wrong (#3450)`); + } + log("warn", `[${s.id}] upstream context overflow (model=${reqModel ?? "unknown"}) — window ${info.window} stated upstream${measured !== undefined ? `, rejected payload ~${measured} tokens stated` : ""}${reserve !== undefined ? `, completion reservation ${reserve} tokens` : ""}; armed emergency shrink at ${armedBaseline}${armedBaseline < unclampedBaseline ? ` (clamped to the judging limit ${judgingLimit}, #3738)` : ""}, declared window unchanged (#987)`); } else { // No window number stated — nothing to learn (and #987 // removed the learner anyway), but the rejection itself is diff --git a/src/session.ts b/src/session.ts index 111b4b7568..25d0c08ddf 100644 --- a/src/session.ts +++ b/src/session.ts @@ -335,6 +335,16 @@ export type Session = { * usage report (one-shot, per the #554 contract) and on compression * reset. Absent on legacy session files → treated as no arm. */ overflowArmTokens?: number; + /** #3450: completion-channel reservation PROVEN by an upstream-stated + * overflow ("N in the completion" billed against the same wall) — + * preflight folds against window − reserve because the foldable + * quantity is MESSAGES. Set alongside the overflow arm; cleared on + * native-compaction reset like the arm itself. Absent/zero → no + * reservation (legacy behavior). */ + overflowArmCompletionReserve?: number; + /** #3450: one-time mismatch warning already emitted for this session + * (upstream-stated window ≠ window in use). Provenance only. */ + overflowArmMismatchWarned?: boolean; /** Tokens compressed THIS turn whose fold has not yet materialized in * an upstream usage report (the post-compress re-request re-sends the * UNFOLDED history for prefix-cache reasons, so its usage report @@ -1247,6 +1257,11 @@ export function resetSessionCompression(session: Session): void { // longer bounds this session; drop it so the guard falls back to the // declared window instead of clamping to a stale ceiling. delete session.stats.overflowArmTokens; + // #3450: same boundary — the proven completion reservation described the + // pre-compaction payload lineage, and the rebuilt conversation re-arms the + // one-time mismatch warning. + delete session.stats.overflowArmCompletionReserve; + delete session.stats.overflowArmMismatchWarned; // #728: the pre-compaction outbound payload is gone — the old estimate // (measured against the pre-compaction wire) would read high and blind // the nudge fallback early; let the next prepare* re-measure. diff --git a/src/util.ts b/src/util.ts index 8319688a0c..2fc9dd5935 100644 --- a/src/util.ts +++ b/src/util.ts @@ -236,6 +236,18 @@ export interface ContextOverflowInfo { * number is present. #987: this arms the one-shot emergency shrink — it * is never persisted as a learned window. */ window?: number; + /** #3450: total token count the upstream says it RECEIVED, when the body + * states it ("you requested N tokens") — the rejected payload's own size, + * NOT the window; arming at the ceiling instead of this measurement is + * what turned a real deficit into "deficit 0". */ + requestedTotal?: number; + /** #3450: message-channel share of the requested total ("N in the + * messages") — the foldable quantity an over-window provider bills. */ + requestedMessages?: number; + /** #3450: completion-channel share ("N in the completion") — proves this + * provider bills OUTPUT against the same wall as input, so preflight + * must fold against window − reservation. */ + requestedCompletion?: number; /** Truncated error-body text, for logging. */ message: string; } @@ -312,6 +324,25 @@ function parseOverflowWindow(text: string): number | undefined { return undefined; } +/** #3450: best-effort extraction of the REJECTED PAYLOAD'S OWN token counts + * from an overflow body — distinct from parseOverflowWindow (the ceiling). A + * wrong guess is worse than no guess, so each pattern is provider-specific. */ +function parseOverflowCounts(text: string): { total?: number; messages?: number; completion?: number } { + // OpenAI/DeepSeek: "...you requested 1160518 tokens (904518 in the + // messages, 256000 in the completion)." + let m = text.match(/requested\s+(\d[\d,]*)\s*tokens?\s*\(\s*(\d[\d,]*)\s*in the messages,\s*(\d[\d,]*)\s*in the completion\s*\)/i); + if (m) return { total: toTokenNumber(m[1]), messages: toTokenNumber(m[2]), completion: toTokenNumber(m[3]) }; + // Anthropic: "prompt is too long: 130000 tokens > 128000 maximum" — the + // first number IS the prompt (messages) size; no completion split stated. + m = text.match(/prompt is too long:\s*(\d[\d,]*)\s*tokens?\s*>/i); + if (m) return { total: toTokenNumber(m[1]), messages: toTokenNumber(m[1]) }; + // Gemini: "The input token count (1012345) exceeds the maximum number of + // tokens allowed (1048576)." + m = text.match(/input token count\s*\((\d[\d,]*)\)\s*exceeds/i); + if (m) return { total: toTokenNumber(m[1]), messages: toTokenNumber(m[1]) }; + return {}; +} + /** Inspect an upstream response for a context-overflow error. `status` is the * HTTP status; `bodyText` is the (usually small) error body. 400/413 and any * 5xx are checked against the recognized context-too-long markers; the marker @@ -325,7 +356,15 @@ export function inspectContextOverflow(status: number, bodyText: string): Contex if (!bodyText) return { isOverflow: false, message }; const isOverflow = CONTEXT_OVERFLOW_PATTERNS.some((p) => p.test(bodyText)); if (!isOverflow) return { isOverflow: false, message }; - return { isOverflow: true, window: parseOverflowWindow(bodyText), message }; + const counts = parseOverflowCounts(bodyText); + return { + isOverflow: true, + window: parseOverflowWindow(bodyText), + ...(counts.total !== undefined ? { requestedTotal: counts.total } : {}), + ...(counts.messages !== undefined ? { requestedMessages: counts.messages } : {}), + ...(counts.completion !== undefined ? { requestedCompletion: counts.completion } : {}), + message, + }; } /** Default cap on the output-headroom reservation, as a fraction of the diff --git a/tests/context-overflow.test.ts b/tests/context-overflow.test.ts index bb3a9e3eaf..a0f32fbe6f 100644 --- a/tests/context-overflow.test.ts +++ b/tests/context-overflow.test.ts @@ -74,6 +74,101 @@ test("Anthropic: 'prompt is too long: X tokens > Y maximum' → window is Y", () assert.equal(info.window, 128000); // the maximum, not the 130000 total }); +// #3450: when the body ALSO states the rejected payload's own counts, they are +// distinct from the ceiling — arming the shrink at the ceiling instead of the +// measurement is what turned a real deficit into "deficit 0". Each pattern is +// provider-specific; a wrong guess is worse than no guess. +test("DeepSeek/OpenAI: 'requested N tokens (M in the messages, K in the completion)' → all three counts (#3450)", () => { + // Verbatim from incident #3450. + const body = JSON.stringify({ + error: { + code: "context_length_exceeded", + message: + "This model's maximum context length is 1048576 tokens. However, you requested 1160518 tokens (904518 in the messages, 256000 in the completion). Please reduce the length of the inputs or output.", + }, + }); + const info = inspectContextOverflow(400, body); + assert.equal(info.isOverflow, true); + assert.equal(info.window, 1048576); // the ceiling + assert.equal(info.requestedTotal, 1160518); + assert.equal(info.requestedMessages, 904518); + assert.equal(info.requestedCompletion, 256000); +}); + +test("DeepSeek/OpenAI counts survive comma grouping (#3450)", () => { + const body = JSON.stringify({ + error: { + code: "context_length_exceeded", + message: "you requested 1,160,518 tokens (904,518 in the messages, 256,000 in the completion).", + }, + }); + const info = inspectContextOverflow(400, body); + assert.equal(info.isOverflow, true); + assert.equal(info.requestedTotal, 1160518); + assert.equal(info.requestedMessages, 904518); + assert.equal(info.requestedCompletion, 256000); +}); + +test("Anthropic 'prompt is too long' states the prompt count — no completion split (#3450)", () => { + const body = JSON.stringify({ + type: "error", + error: { type: "invalid_request_error", message: "prompt is too long: 130000 tokens > 128000 maximum" }, + }); + const info = inspectContextOverflow(400, body); + assert.equal(info.window, 128000); + assert.equal(info.requestedTotal, 130000); + assert.equal(info.requestedMessages, 130000); // input side only + assert.equal(info.requestedCompletion, undefined); +}); + +test("Gemini 'input token count (N) exceeds' states the input count (#3450)", () => { + const info = inspectContextOverflow( + 400, + '{"error":{"message":"The input token count (1012345) exceeds the maximum number of tokens allowed (1048576)."}}', + ); + assert.equal(info.isOverflow, true); + assert.equal(info.window, 1048576); + assert.equal(info.requestedTotal, 1012345); + assert.equal(info.requestedMessages, 1012345); + assert.equal(info.requestedCompletion, undefined); +}); + +test("overflow body WITHOUT stated counts → no requested* fields (#3450 regression guard)", () => { + // The legacy phrasing must keep behaving exactly as before (selfheal T3 + // relies on it): window only, no invented counts. + const body = JSON.stringify({ + error: { + code: "context_length_exceeded", + message: "This model's maximum context length is 8192 tokens. However, your messages resulted in 13000 tokens.", + }, + }); + const info = inspectContextOverflow(400, body); + assert.equal(info.isOverflow, true); + assert.equal(info.window, 8192); + assert.equal(info.requestedTotal, undefined); + assert.equal(info.requestedMessages, undefined); + assert.equal(info.requestedCompletion, undefined); +}); + +test("legacy OpenAI 'requested N tokens (your request used ...)' does NOT match the counts pattern (#3450)", () => { + // The chat-completions phrasing splits inputs/output differently — it must + // not be misread as a messages/completion pair. + const body = JSON.stringify({ + error: { + message: + "This model's maximum context length is 128000 tokens. However, you requested 132096 tokens (your request used 20 inputs and 4096 output tokens). Please reduce the length of the inputs or output.", + type: "invalid_request_error", + code: "context_length_exceeded", + }, + }); + const info = inspectContextOverflow(400, body); + assert.equal(info.isOverflow, true); + assert.equal(info.window, 128000); + assert.equal(info.requestedTotal, undefined); + assert.equal(info.requestedMessages, undefined); + assert.equal(info.requestedCompletion, undefined); +}); + test("context_length_exceeded code with comma-grouped window", () => { const body = JSON.stringify({ error: { code: "context_length_exceeded", message: "exceeds the limit of 1,048,576 tokens" } }); const info = inspectContextOverflow(400, body); diff --git a/tests/e2e-issue3450-stated-counts.test.ts b/tests/e2e-issue3450-stated-counts.test.ts new file mode 100644 index 0000000000..1a2decb350 --- /dev/null +++ b/tests/e2e-issue3450-stated-counts.test.ts @@ -0,0 +1,255 @@ +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import test from "node:test"; + +process.env.NODE_ENV = "test"; + +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { listSessions } from "../src/session.ts"; + +// #3450: an upstream-stated overflow body carries more than its window — the +// rejected payload's OWN counts ("N in the messages, K in the completion"). +// The old code booked the WINDOW as the armed baseline and threw the counts +// away, so preflight saw "payload ~window, deficit 0" on an underestimating +// local meter and forwarded a payload the upstream then billed over the wall +// (messages AND completion against 1048576) — killing the turn. Now: +// 1. the measured message count is what gets armed (a ceiling is not a size); +// 2. the proven completion share is reserved off the wall for fit judgments; +// 3. prepare/nudge and preflight judge on ONE wall per request (min of +// declared and upstream-stated); +// 4. when declaration ≠ stated, a one-time warning names both values. + +const DEEPSEEK_OVERFLOW_BODY = JSON.stringify({ + error: { + code: "context_length_exceeded", + message: + "This model's maximum context length is 1048576 tokens. However, you requested 1160518 tokens (904518 in the messages, 256000 in the completion). Please reduce the length of the inputs or output.", + }, +}); + +// A small-window variant where the fold CAN close the gap within the request: +// stated 30000, rejected payload 35000 = 28000 messages + 7000 completion → +// message budget 23000. +const SMALL_WINDOW_OVERFLOW_BODY = JSON.stringify({ + error: { + code: "context_length_exceeded", + message: "This model's maximum context length is 30000 tokens. However, you requested 35000 tokens (28000 in the messages, 7000 in the completion). Please reduce the length of the inputs or output.", + }, +}); + +function okSse(): string { + return ( + `event: message_start\ndata: ${JSON.stringify({ type: "message_start", message: { id: "m1", role: "assistant", usage: { input_tokens: 5000 } } })}\n\n` + + `event: content_block_start\ndata: ${JSON.stringify({ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } })}\n\n` + + `event: content_block_delta\ndata: ${JSON.stringify({ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "ok" } })}\n\n` + + `event: content_block_stop\ndata: ${JSON.stringify({ type: "content_block_stop", index: 0 })}\n\n` + + `event: message_delta\ndata: ${JSON.stringify({ type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 3 } })}\n\n` + + `event: message_stop\ndata: ${JSON.stringify({ type: "message_stop" })}\n\n` + ); +} + +const SUMMARY_TEXT = + "PREFLIGHT SUMMARY: the segment covered a multi-step debugging session. Key decisions: chose the preflight approach over lossy truncation because the payload must stay coherent. Files touched: src/a.ts:10, src/b.ts:20. Outcome: fixed and verified by tests."; + +function summaryJson(): string { + return JSON.stringify({ + id: "msg_summary", + type: "message", + role: "assistant", + model: "claude-test", + content: [{ type: "text", text: SUMMARY_TEXT }], + stop_reason: "end_turn", + usage: { input_tokens: 500, output_tokens: 50 }, + }); +} + +function bigConversation(): Array<{ role: string; content: string }> { + const msgs: Array<{ role: string; content: string }> = []; + for (let i = 0; i < 12; i++) { + const role = i % 2 === 0 ? "user" : "assistant"; + msgs.push({ role, content: `Message ${i} of the long conversation. ` + `MARKER_${i}_content_`.repeat(250) }); + } + return msgs; +} + +function proxyBaseOptions(upstreamPort: number, window: number): ProxyOptions { + return { + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { [`http://127.0.0.1:${upstreamPort}`]: { models: { "claude-test": { context: window } } } }, + modelContextLimit: window, + kernelConfig: defaultConfig(window), + compress: { injectTool: true, injectNudge: true }, + compat: { roles: {} }, + streamErrorShape: "protocol", + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + advisoryCheck: false, + releaseNotesCheck: false, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions; +} + +test("e2e #3450 T1: stated counts arm at the MEASURED size, reserve the completion share, warn once on mismatch — and the uncloseable gap fails honestly (no silent over-forward)", async () => { + // Declared 1M (the dsh-native shape: host declares its own window, sitting + // at the ceiling with no margin). Upstream proves: wall 1048576, rejected + // payload 904518 messages + 256000 completion. The local conversation is + // only ~16k tokens — folding it cannot close a 904k-measured gap, so the + // turn must end in a visible 400, NOT a "fitsWindow=true" forward that + // dies upstream. + const bodies: string[] = []; + const upstream = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + const raw = Buffer.concat(chunks).toString("utf8"); + bodies.push(raw); + let parsed: { stream?: boolean } = {}; + try { + parsed = JSON.parse(raw); + } catch { + parsed = {}; + } + if (parsed.stream === false) { + res.writeHead(200, { "content-type": "application/json" }); + res.end(summaryJson()); + return; + } + res.writeHead(400, { "content-type": "application/json" }); + res.end(DEEPSEEK_OVERFLOW_BODY); + }); + }); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const proxy = await startServer(proxyBaseOptions(upstreamPort, 1_000_000)); + await once(proxy, "listening"); + const proxyPort = (proxy.address() as { port: number }).port; + + try { + const url = `http://127.0.0.1:${proxyPort}/bili/http://127.0.0.1:${upstreamPort}/v1/messages`; + const body = JSON.stringify({ model: "claude-test", max_tokens: 1024, stream: true, messages: bigConversation() }); + const headers = { "content-type": "application/json", "x-acp-session": "t3450-arm-sess" }; + + const r1 = await fetch(url, { method: "POST", headers, body }); + assert.equal(r1.status, 400, "the measured gap is uncloseable locally — the rejection surfaces, it is not masked by an over-window forward"); + await r1.text(); + const s = listSessions().find((x) => x.id === "t3450-arm-sess"); + assert.ok(s, "session exists"); + // 1. armed baseline = the upstream-stated MESSAGE count, not the window. + assert.ok(s!.stats.lastInputTokens < 904_518, `armed below the stated 904518 (folds netted out): ${s!.stats.lastInputTokens}`); + assert.ok(s!.stats.lastInputTokens > 800_000, `armed near the stated 904518, not the 1M declaration: ${s!.stats.lastInputTokens}`); + assert.equal(s!.stats.lastInputTokensSource, "overflow-arm"); + // 2. the completion share is reserved off the wall while the arm lives. + assert.equal(s!.stats.overflowArmCompletionReserve, 256_000); + // 3. the side-request guard keeps the CEILING even though the baseline + // books the measurement. + assert.equal(s!.stats.overflowArmTokens, 1_048_576); + // 4. declaration (1M) ≠ stated (1048576) → one-time mismatch warning flag. + assert.equal(s!.stats.overflowArmMismatchWarned, true); + // nothing learned into the declared window (#987 contract unchanged). + assert.equal(s!.metadata.confirmedContextLimits, undefined); + } finally { + proxy.close(); + proxy.closeIdleConnections?.(); + await once(proxy, "close"); + upstream.close(); + upstream.closeIdleConnections?.(); + await once(upstream, "close"); + } +}); + +test("e2e #3450 T2: with a small stated window the same counts rescue the turn IN-REQUEST and the reservation retires with the arm", async () => { + // Declared 40k, upstream states wall 30k and a rejected payload of + // 28000 messages + 7000 completion → message budget 23k. The local + // conversation (~16k) CAN reach under 23k once folds net out of the + // armed floor, so the refold path must converge on fit and re-send within + // the same request. + let streamingCall = 0; + let summaryCalls = 0; + const bodies: string[] = []; + const upstream = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + const raw = Buffer.concat(chunks).toString("utf8"); + bodies.push(raw); + let parsed: { stream?: boolean } = {}; + try { + parsed = JSON.parse(raw); + } catch { + parsed = {}; + } + if (parsed.stream === false) { + summaryCalls += 1; + res.writeHead(200, { "content-type": "application/json" }); + res.end(summaryJson()); + return; + } + if (streamingCall === 0) { + res.writeHead(400, { "content-type": "application/json" }); + res.end(SMALL_WINDOW_OVERFLOW_BODY); + } else { + res.writeHead(200, { "content-type": "text/event-stream" }); + res.end(okSse()); + } + streamingCall += 1; + }); + }); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const proxy = await startServer(proxyBaseOptions(upstreamPort, 40_000)); + await once(proxy, "listening"); + const proxyPort = (proxy.address() as { port: number }).port; + + try { + const url = `http://127.0.0.1:${proxyPort}/bili/http://127.0.0.1:${upstreamPort}/v1/messages`; + const body = JSON.stringify({ model: "claude-test", max_tokens: 1024, stream: true, messages: bigConversation() }); + const headers = { "content-type": "application/json", "x-acp-session": "t3450-rec-sess" }; + + const r1 = await fetch(url, { method: "POST", headers, body }); + assert.equal(r1.status, 200, "#1195/#3450: folded under the message budget (window − completion) and re-sent within the same request"); + assert.equal(r1.headers.get("content-type"), "text/event-stream"); + await r1.text(); + assert.ok(summaryCalls >= 1, "preflight ran compress summary calls"); + const lastForward = bodies[bodies.length - 1]; + assert.ok(lastForward.includes(SUMMARY_TEXT), "the forwarded body carries the fold summary"); + const s = listSessions().find((x) => x.id === "t3450-rec-sess"); + assert.ok(s, "session exists"); + // The retry's real usage report replaces the armed baseline and retires + // BOTH the arm and its completion reservation (#1110 lifecycle). + assert.equal(s!.stats.lastInputTokens, 5000, "the retry's usage report already replaced the armed baseline"); + assert.equal(s!.stats.lastInputTokensSource, "usage"); + assert.equal(s!.stats.overflowArmTokens, undefined, "the one-shot arm retired with the recovery"); + assert.equal(s!.stats.overflowArmCompletionReserve, undefined, "the proven reservation retires WITH the arm"); + assert.equal(s!.stats.overflowArmMismatchWarned, true, "the one-time warning flag survives recovery (it is provenance)"); + assert.equal(s!.metadata.confirmedContextLimits, undefined, "nothing learned into the declared window"); + } finally { + proxy.close(); + proxy.closeIdleConnections?.(); + await once(proxy, "close"); + upstream.close(); + upstream.closeIdleConnections?.(); + await once(upstream, "close"); + } +}); diff --git a/tests/issue1195-overflow-refold.test.ts b/tests/issue1195-overflow-refold.test.ts index 6c70b5a166..0339f2a2c4 100644 --- a/tests/issue1195-overflow-refold.test.ts +++ b/tests/issue1195-overflow-refold.test.ts @@ -225,8 +225,10 @@ test("e2e #1195 B: an unfoldable payload passes the ORIGINAL overflow 400 throug const s = listSessions().find((x) => x.id === "refold-b"); assert.ok(s, "session exists"); - assert.equal(s!.stats.lastInputTokens, 128000, "the arm still fires — the next turn folds if it can"); - assert.equal(s!.stats.overflowArmTokens, 128000, "#1110: the arm is recorded"); + // #3450: the body states the rejected prompt's own size (130000) — the + // arm books THAT measurement; the side-request guard keeps the CEILING. + assert.equal(s!.stats.lastInputTokens, 130000, "the arm still fires at the upstream-measured size — the next turn folds if it can"); + assert.equal(s!.stats.overflowArmTokens, 128000, "#1110: the arm is recorded (ceiling, unchanged by #3450)"); assert.equal(s!.metadata.confirmedContextLimits, undefined, "#987: nothing learned"); }); From 6cc78a650baf0f49c9bbc34bfc5c1ff9639234db Mon Sep 17 00:00:00 2001 From: awork Date: Sat, 17 Oct 2026 14:20:19 +0800 Subject: [PATCH 2/4] =?UTF-8?q?fix:=20drop=20double-netting=20in=20commitF?= =?UTF-8?q?old=20=E2=80=94=20applyRanges=20already=20nets=20the=20gross=20?= =?UTF-8?q?fold=20(#3450=20trace=20correction)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/preflight.ts | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git a/src/preflight.ts b/src/preflight.ts index 2858aa55e1..22e50bb9fc 100644 --- a/src/preflight.ts +++ b/src/preflight.ts @@ -2070,20 +2070,13 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa // char count against it instead of the token-based credit. const compressed = deps.session.stats.compressCreditTokens - creditBefore; const folded = baselineKnown ? compressed : messages.filter((message) => effectiveMessageIds.includes(message.id)).reduce((total, message) => total + (message.text ?? "").length, 0); - const summaryCost = defaultCountTokens(summary); currentTokens = Math.max(0, currentTokens - folded + countText(summary)); - // #3450: net the fold out of the input baseline like the live path - // (stream.ts) does — the summary re-enters the payload but the - // folded mass leaves it. Without this an armed/usage floor at-or- - // above the target never descends and the loop cannot exit on fit - // (it only ever burned budget or stalled). Token caliber matches - // only when baselineKnown; the char-based regime keeps the legacy - // summary-cost addition. - if (baselineKnown) { - deps.session.stats.lastInputTokens = Math.max(0, deps.session.stats.lastInputTokens - Math.max(0, folded - summaryCost)); - } else { - deps.session.stats.lastInputTokens += summaryCost; - } + // The apply path (stream.ts applyRanges) already netted the gross + // fold out of lastInputTokens and credited compressCreditTokens; + // add the re-entering summary's cost back so the baseline tracks + // the post-fold payload size exactly once (#3450 trace: arm 1048576 + // → −gross +summary = 1047168 after one fold). + deps.session.stats.lastInputTokens += defaultCountTokens(summary); appliedThisRound += 1; result.compressedRanges += 1; return true; From e49b2c217e1c555c9ee845e513daa03e441a444b Mon Sep 17 00:00:00 2001 From: awork Date: Tue, 20 Oct 2026 00:37:55 +0800 Subject: [PATCH 3/4] =?UTF-8?q?fix:=20apply=20armed=20completion=20reserve?= =?UTF-8?q?=20min-wise=20against=20the=20stated=20wall=20(#3450=20=C3=97?= =?UTF-8?q?=20#3738=20merge=20regression)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebasing this branch over #3738 exposed a joint defect neither PR's own tests cover: handle.ts rewrites config.modelContextLimit IN PLACE with the output-headroom reservation before preflight runs (its [headroom] line says 'nudge/preflight judge input against reserved'), so subtracting the demonstrated completion share from that already-reserved value counts the completion share TWICE. On any session with an output-budget reservation whose overflow 400 states a completion count, the preflight wall collapsed toward zero (converge fixture: 40,000 - 60,000 -> 0) and the rescue folded nothing. Verified by the new case-3 convergence run failing on the naive merge and passing after this fix. The reservation now applies MIN-wise to the upstream-stated wall: limit = min(reservedLimit, statedWindow - completionReserve) while armed, falling back to the untouched limit otherwise. The incident arithmetic (stated 1,048,576 / messages 841,304 / completion 256,000, reserved limit 792,576) yields exactly 792,576; the no-reservation path is byte-identical to before. Test reconciliations in tests/fix-arm-headroom-clamp.test.ts (each with an inline rationale): case 1's guard now pins the stated CEILING (the side-request guard reserves each request's own output from that value itself, so feeding it the pre-reserved limit would double-subtract); case 2 pins lastInputTokens at the stated message count 841,304 (since #3450 the measurement is booked when present; the no-reservation clamp stays a no-op above it); case 3's guard pin likewise moves to the ceiling. --- src/server.ts | 15 ++++++++++++++- tests/fix-arm-headroom-clamp.test.ts | 22 ++++++++++++++++++---- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/src/server.ts b/src/server.ts index f8c8e5089c..7fe15b7f87 100644 --- a/src/server.ts +++ b/src/server.ts @@ -2999,8 +2999,21 @@ export async function preflightCompressIfNeeded( // COMPLETION against the same wall ("N in the messages, N in the // completion"), the foldable quantity is MESSAGES — every payload-vs-wall // judgment below runs on window − reservation instead of the raw window. + // The reservation applies MIN-wise to the stated wall, never subtracted + // from config.modelContextLimit: handle.ts rewrites that value IN PLACE + // with the output-headroom reservation (its [headroom] line: "nudge/ + // preflight judge input against ${reserved}"), so subtracting the + // demonstrated completion share again would count it twice and collapse + // the wall toward zero whenever a stated count lands on a reserved + // session (#3450 × #3738 merge regression; case 3 of + // tests/fix-arm-headroom-clamp.test.ts pins the recovery). In the rare + // stated-below-declared corner the min() over-folds slightly — the safe + // direction (availability over context retention). + const statedWall = typeof session.stats.overflowArmTokens === "number" && session.stats.overflowArmTokens > 0 ? session.stats.overflowArmTokens : 0; const completionReserve = typeof session.stats.overflowArmCompletionReserve === "number" && session.stats.overflowArmCompletionReserve > 0 ? session.stats.overflowArmCompletionReserve : 0; - const limit = Math.max(0, config.modelContextLimit - completionReserve); + const limit = completionReserve > 0 && statedWall > 0 + ? Math.max(0, Math.min(config.modelContextLimit, statedWall - completionReserve)) + : config.modelContextLimit; const overflowTarget = prepared.protocol === "responses" && isCodexClient(req.headers) && codexCompactMode() === "intercept" ? limit * CODEX_COMPACT_HEALTH_RATIO : limit; diff --git a/tests/fix-arm-headroom-clamp.test.ts b/tests/fix-arm-headroom-clamp.test.ts index 0064bba1b9..8e6f2568d3 100644 --- a/tests/fix-arm-headroom-clamp.test.ts +++ b/tests/fix-arm-headroom-clamp.test.ts @@ -218,7 +218,15 @@ test("the stated-window arm is clamped to the headroom-reserved judging limit", reserved, "this limb: the clamp is exactly the headroom reservation", ); - assert.equal(s.stats.overflowArmTokens, s.stats.lastInputTokens, "overflowArmTokens mirrors the arm"); + // Since #3450 the guard input intentionally diverges from the nudge + // baseline: it keeps the RAW stated ceiling because the side-request + // guard reserves each request's own output FROM this value itself — + // feeding it the pre-reserved limit would double-subtract the headroom. + assert.equal( + s.stats.overflowArmTokens, + WINDOW, + "guard keeps the stated ceiling — it self-reserves headroom (#3450)", + ); // The request that produced the rejection must actually have carried the // output budget that causes the reservation — otherwise the test would be @@ -255,10 +263,13 @@ test("the stated-window arm is the upstream truth when no headroom reservation a WINDOW, "control: with no output budget no headroom reservation is recorded", ); + // Since #3450 the stated MESSAGE COUNT is booked when the body carries + // one — stronger upstream evidence than the ceiling. The no-reservation + // clamp stays a no-op: nothing is ever shrunk below the measurement. assert.equal( s.stats.lastInputTokens, - WINDOW, - "with no reservation the arm stays at the exact stated window", + 841_304, + "with no reservation the arm books the stated message count (≤ the stated window)", ); assert.ok(calls.length > 0, "the upstream was reached"); assert.notEqual(calls[0].usage, MAX_OUTPUT, "control: no max_tokens was forwarded"); @@ -469,7 +480,10 @@ test("next-turn preflight converges under the clamped arm (the recovery the clam `arm ${s.stats.lastInputTokens} must sit at the reserved judging limit ${CONVERGE_LIMIT} — ` + "above it, the next turn's fold walk demands phantom savings", ); - assert.equal(s.stats.overflowArmTokens, CONVERGE_LIMIT, "overflowArmTokens mirrors the arm"); + // Since #3450 the guard input diverges from the nudge baseline: it keeps + // the RAW stated ceiling (this fixture's stated count equals it) — the + // guard reserves each request's own output from that value itself. + assert.equal(s.stats.overflowArmTokens, CONVERGE_WINDOW, "guard keeps the stated ceiling (#3450)"); // Turn 2: the client re-sends the same history. The armed meter engages // preflight, the fold walk lands the payload under the reserved limit, From 5b91fb5755b159d149b88f1cd4faf04b6cf9426d Mon Sep 17 00:00:00 2001 From: awork Date: Tue, 20 Oct 2026 11:48:15 +0800 Subject: [PATCH 4/4] test: pin the #3450 in-flight refold rescue in the arm-clamp converge case MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebasing onto master surfaced an interaction this PR's own base predated: #3739's case 3 (merged 2026-10-19, postdating this PR's base) pins the PRE-#3450 refold behavior — fold against the bare stated window, retry rejected again by the completion share, 400 surfaced until the next turn. Since #3450 the same-request refold folds against window − proven completion share, so the rejected request itself is rescued (client sees the retry's 200) and the next turn converges on the stored folds. The stronger behavior is the point of #3450; update the case to pin it: turn 1 asserts the in-flight rescue + paired arm/reservation retirement on the settled usage report, turn 2 keeps the no-loop convergence. --- tests/fix-arm-headroom-clamp.test.ts | 75 ++++++++++++++++++---------- 1 file changed, 48 insertions(+), 27 deletions(-) diff --git a/tests/fix-arm-headroom-clamp.test.ts b/tests/fix-arm-headroom-clamp.test.ts index 8e6f2568d3..b6ca5c10aa 100644 --- a/tests/fix-arm-headroom-clamp.test.ts +++ b/tests/fix-arm-headroom-clamp.test.ts @@ -279,12 +279,21 @@ test("the stated-window arm is the upstream truth when no headroom reservation a -// --- Case 3: the recovery the clamp restores — next-turn convergence --- +// --- Case 3: the recovery the clamp restores — same-request rescue, then +// next-turn convergence --- // // Cases 1-2 pin the invariant (arm <= judging limit). This case pins the // BEHAVIOR that invariant exists for: after an overflow 400 arms the shrink, -// the NEXT turn's preflight folds the payload down to the reserved judging -// limit and the forward succeeds — instead of looping on 400s. +// the payload gets folded down to the reserved judging limit and the forward +// succeeds — instead of looping on 400s. +// +// Since #3450 the 400 body's stated counts sharpen that recovery: the +// same-request refold folds against window − proven completion share (not the +// bare window), so when the provider bills messages+completion against one +// wall, the REJECTED request itself is rescued (the client sees the retry's +// 200, not the 400); the next turn then converges on the stored folds. +// (Pre-#3450 the refold targeted the bare window, the completion share pushed +// the retry back over the wall, and the 400 surfaced until the next turn.) // // The mock prices ~10% denser than the proxy's own estimator // (defaultCountTokens * 1.1): the class of provider-vs-heuristic tokenizer @@ -401,7 +410,7 @@ function convergeUpstream(): { server: http.Server; calls: CCall[] } { return { server, calls }; } -test("next-turn preflight converges under the clamped arm (the recovery the clamp restores)", async (t) => { +test("stated-count overflow is rescued within the SAME request, then converges next turn (the recovery #3450 restores)", async (t) => { _setStoreForTest(new SessionStore({ enabled: false })); setRegistryForTest({}); const { server: upstream, calls } = convergeUpstream(); @@ -467,28 +476,41 @@ test("next-turn preflight converges under the clamped arm (the recovery the clam assert.ok(s.stats.lastInputTokens > 0, "the estimate baseline is positive"); // Turn 1: the oversized turn rides the forward-once-for-evidence path - // (estimate baseline, no usage-grade floor) and hits the real 400. + // (estimate baseline, no usage-grade floor) and hits the real 400. The + // same-request refold then folds against window − proven completion share + // (100,000 − 60,000 = 40,000 here — the same number as the clamped judging + // limit), so the retry fits the mock's acceptance band and the client sees + // the rescue's 200 instead of the 400. const r1 = await post(convergeBig()); - assert.equal(r1.status, 400, "the oversized forward is rejected upstream"); + assert.equal(r1.status, 200, "the stated-count refold rescues the request in-flight"); await r1.arrayBuffer(); + { + const mains = calls.filter((c) => c.stream); + const r1fwd = mains[mains.length - 1]; + assert.ok(r1fwd.raw.includes(CONVERGE_SUMMARY_TEXT), "the folded summary rides the retry wire"); + assert.ok(r1fwd.raw.includes("MARKER_23_"), "the recent tail survives the fold"); + let distinct = 0; + for (let i = 0; i < 24; i++) { + if (r1fwd.raw.includes(`MARKER_${i}_content_`)) distinct += 1; + } + assert.ok( + distinct <= 12, + `substantial folding expected: ${distinct}/24 conversation turns still raw on the wire`, + ); + } s = getSession("limb-arm-converge"); - assert.ok(s, "session survives the 400"); - assert.equal(s.stats.lastInputTokensSource, "overflow-arm", "the rejection arms the emergency shrink"); - assert.equal( - s.stats.lastInputTokens, - CONVERGE_LIMIT, - `arm ${s.stats.lastInputTokens} must sit at the reserved judging limit ${CONVERGE_LIMIT} — ` + - "above it, the next turn's fold walk demands phantom savings", - ); - // Since #3450 the guard input diverges from the nudge baseline: it keeps - // the RAW stated ceiling (this fixture's stated count equals it) — the - // guard reserves each request's own output from that value itself. - assert.equal(s.stats.overflowArmTokens, CONVERGE_WINDOW, "guard keeps the stated ceiling (#3450)"); - - // Turn 2: the client re-sends the same history. The armed meter engages - // preflight, the fold walk lands the payload under the reserved limit, - // and the forward succeeds — the recovery the clamp restores. - const summariesBefore = calls.filter((c) => !c.stream && c.maxTok > 200).length; + assert.ok(s, "session survives the rescued turn"); + // The 400 armed the shrink (clamped baseline, ceiling guard, proven + // reservation); the rescued response's usage report then retired the pair + // in one settle — #3450's paired-retirement contract. + assert.equal(s.stats.lastInputTokensSource, "usage", "the rescued usage report retires the arm"); + assert.ok(s.stats.lastInputTokens > 0, "the baseline is replaced by the real usage figure"); + assert.equal(s.stats.overflowArmTokens, undefined, "#1129: the arm sidecar retires with the arm"); + assert.equal(s.stats.overflowArmCompletionReserve, undefined, "#3450: the reservation retires with the arm"); + + // Turn 2: the client re-sends the same history. The folds stored by the + // in-flight rescue ride the reconciliation, the payload stays under the + // reserved limit, and the forward succeeds again — no loop. const r2 = await post(convergeBig()); assert.equal(r2.status, 200, "the next turn converges instead of looping"); await r2.arrayBuffer(); @@ -504,12 +526,11 @@ test("next-turn preflight converges under the clamped arm (the recovery the clam distinct <= 12, `substantial folding expected: ${distinct}/24 conversation turns still raw on the wire`, ); - const summariesAfter = calls.filter((c) => !c.stream && c.maxTok > 200).length; - assert.ok(summariesAfter > summariesBefore, "preflight ran real summary calls to fold"); s = getSession("limb-arm-converge"); assert.ok(s, "session survives the successful turn"); - assert.equal(s.stats.lastInputTokensSource, "usage", "the successful usage report retires the arm"); - assert.equal(s.stats.overflowArmTokens, undefined, "#1129: the arm sidecar retires with the arm"); + assert.equal(s.stats.lastInputTokensSource, "usage", "judging stays on real usage after the rescue"); + assert.equal(s.stats.overflowArmTokens, undefined, "no stale arm lingers past the settled usage"); + assert.equal(s.stats.overflowArmCompletionReserve, undefined, "no stale reservation lingers past the settled usage"); await close(); });