diff --git a/CHANGELOG.md b/CHANGELOG.md index 5fb2a85334..165555b625 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -496,8 +496,8 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Fixes +- **Session input baselines no longer outlive their measurement context — failure arms adopt the billing-caliber reading, and a model switch retires trusted baselines (#2683)**: two producers of stale baselines in one mechanism family. (1) `armFailureShrink` (no-usage relay 5xx / network failure) adopted the char-count UPPER BOUND of the outbound payload as `session.stats.lastInputTokens` — a 1.7–4× over-reading persisted as the session baseline, inflating preflight entry budgets, responses-lane output headroom and displays, and surviving a model/window change until the next usage report (live case: 441,193 armed against a 202,885-token payload; after a 1M→400k model switch the phantom meter read 142% and two ~3-minute emergency folds ran on a payload already at 62% of the new window). The arm now adopts the same BILLING-CALIBER reading the preflight trigger uses (k̂-scaled chars/4 + wire overhead + image reserve), and its log names the provenance (`(k̂-calibrated outbound reading)` / `(raw outbound reading)`). This supersedes #1498-F1's bound floor: the kernel truncate-loop consumer that justified it no longer exists (every processTurn feed is quarantined since #1492/#1839; preflight runs noEmergencyTruncate), so the bound's only live effect was phantom mass. **Behavior change (old→new):** the armed value on a no-usage failure drops from the char-count upper bound to the billing-caliber estimate — it may now land BELOW a future bill on dense JSON/CJK routes (chars/4 undercounts, #553); acceptable because estimate-grade values are quarantined from every decision floor and k̂ closes most of the gap where route+model provenance matches. (2) A usage-grade baseline measured under model X kept deciding requests for model Y: only ORIGIN changes demoted it (#1933 F2), so after a mid-session model switch the foreign bill pinned the preflight loop's decisionTokens above the fold target forever — folding until budget exhaustion although the payload fit. A real model change (both sides known; unknown→known never counts) now retires the trusted-grade baseline at the per-request model-stamp site (`retireCrossModelBaselines`, logged with both models and the retired value) BEFORE prepare sizes the nudge and before the trigger runs; the next usage report re-anchors. Estimate-grade values survive the switch (byte-scale properties of the payload, not of the billing model); same-model window moves keep the bill valid, so a same-model turn never retires it. Tests: `tests/issue2683-cross-model-baseline.test.ts` (G1 cross-model retirement → zero summary calls + re-anchor; G2 arm scale band [chars/8, char-count) + provenance clause; G3 control: same-model over-window bill NOT retired — floor stands and the fitting payload forwards via the #4288 fit-forward, current master's same-model semantics) and the re-scaled #1498-F2 assertion in `tests/issue1493-arm-outbound-payload.test.ts`. - **Anthropic Threads continue-turns survive the compress loop: re-requests continue from the latest upstream response id instead of re-sending the client's already-consumed `previous_message_id` (#2701)**: with `thread: {type:"continue", previous_message_id}` (Claude Code 2.1.295 via Threads), the conversation lives server-side and the request body carries only the turn's 1–2 delta messages. The loop's internal round-2 fetch CONSUMES the client's continue target, and the legacy round-2 body — a full-history rebuild from `adapter.buildRequest(coreMessages, ...)` re-spreading the client's `thread` field — re-sent exactly that consumed target: upstream 400 `"The requested `previous_message_id` has already been continued. Each response `id` may be continued at most once"`, turn dead, fold lost. Worse, that rebuild re-stamps system via `appendSystemText(systemPrompt, originalSystem)` — different bytes than the steady prepare path (`injectSystem` + `#1085`-anchored stamp) ever sends — poisoning the per-thread fingerprint: after every fold the NEXT client continue mismatched with `"differing field(s): cache_control, system, tools"` (the class-2 replay 400s; corrects #2648's P3 note — folds ARE inherited by the server thread once a create delivers them, the limitation was fold-only-deliverable-at-create plus this drift). Four coordinated changes, anthropic-only, zero new knobs: (1) the adapter records EVERY round's `message_start` id (`threadTailId` + `threadResponseIds`, exposed as optional `CompressLoopAdapter` hooks) and reports the tail id on the final terminal `message_delta`; (2) on a threads-CONTINUE turn, every re-request (main re-request, the three nudge retries, the degraded thinking-strip retry) is rebuilt delta-shaped — `{...requestBody, thread: {…, previous_message_id: latestRoundId}, messages: coreToAnthropic(unsent delta)}` — where the delta carries ONLY the loop's own client-invisible additions since the last accepted send (proxy tool_results + retrieval injections; assistant output already lives server-side in r_k; visibility markers deliberately excluded — a server thread is append-only and would carry the client noise forever); system/tools/every other byte stay identical to round 1's body (fingerprint stable); acceptance bookkeeping rolls the accepted window when a round's response starts; (3) the #413 blind truncation retry is barred once the sent body's continue target is provably consumed (a response id was observed since the send) — a byte-identical re-send would 400; pre-start truncations keep the retry; (4) `src/server/thread-tracking.ts` persists every observed id into `session.metadata.anthropicThreadTail` (`{tail, consumed[]}`, capped 20, recorded in the loop's `finally` so consumed ids survive even turns that die mid-round), and `prepareAnthropic` rewrites an inbound continue whose target sits in the consumed set to the tracked tail (unknown ids pass through untouched — restart/foreign-route safe; a `create` resets tracking). Thread CREATE turns keep the legacy full rebuild: a create re-sends the whole (possibly folded) history — the only channel that can deliver a fold to the server-side thread. Known limitation (documented in code): a fold executed during a CONTINUE turn is not reflected server-side until the next create (“silent lie” — the loop's local view is folded, the thread keeps the pre-fold text; emergency truncation stays impossible on continues for the same reason). Tests: `tests/issue2701-threads-continue.test.ts` — re-request continues from `msg_r1` with a single-message tool_result delta and byte-identical system/tools; ids persisted as `{tail: msg_r2, consumed: [msg_r1]}`; terminal `message_delta` carries the tail id; pre-start truncation keeps the blind retry (byte-identical re-send); post-start truncation bars it (no re-fetch, client-visible error); create turns keep the full-rebuild path; thread-tracking unit tests (roll, cap, rewrite-to-tail, unknown-id passthrough, create reset). - - **Outgoing bodies are scrubbed for unpaired surrogate escapes at every send seam — JSON.stringify hides the half as escape text, so the existing guard could never see it (#2565, #816/#828/#1615 family, fourth site)**: a conversation could become permanently stuck against a strict upstream — every request rejected with a non-retryable 400 `invalid_request`, every other conversation on the same provider/key fine. Two defects combined. (1) Ingest: a summary can hold a real lone half (a model that copies literal `\uXXXX` prose out of tool output — our ACP-status panel text is emitted that way — has it re-decoded along the summarizer path), and it was then stored verbatim and re-emitted on every later request. (2) Wire: `scrubLoneSurrogates()` existed in `src/text-safe.ts` but its only caller was `summaryFingerprintLine` (`src/stream.ts`, the log fingerprint line) — the outgoing body was never scrubbed, and applying it at the send seam is not enough, because `JSON.stringify` renders a lone half as the **six-character escape text** `\uXXXX`: by the time the body is a string there is no code unit left for the code-unit regex to find. Now `src/text-safe.ts` also exports `scrubLoneSurrogateEscapes()` — walks backslash runs so an escape only starts on an **odd**-length run (doubled `\\uXXXX`, i.e. literal prose a model quoted, is left byte-identical), keeps a well-formed high+low escape pair verbatim, and rewrites an unpaired half to the `\uFFFD` escape — plus `scrubLoneSurrogatesOnWire()` (that, over the existing real-unit scrub) which is applied at all four serialization seams: `src/loop/core.ts` `fetchUpstream`'s `wireBodyStr`, `src/server/relay.ts` (the final-send seam, before `applyResign` so the signature covers the sent bytes, and the context-overflow refold re-send), and `src/reasoning-guard.ts`'s round re-send; `kernel/src/compress.ts` scrubs summaries at `applyCompression` ingest so the half is never stored in the first place (shared `scrubLoneSurrogates()` in `kernel/src/truncate.ts`). Buffer bodies (raw passthrough) are deliberately untouched. Tests: `tests/text-safe.test.ts` pins the escape scanner (lone high/low escape → `\uFFFD`, valid escape pair kept, doubled backslash prose kept, ordinary escapes `\n` `\t` `\u0041` `\u4e2d` kept, short hex and trailing backslash kept), `tests/wire-lone-surrogate-scrub.test.ts` is an e2e against a mock upstream (poisoned body still gets a 200 and no unpaired half reaches the upstream, valid pairs survive, U+FFFD present; a clean body is not over-scrubbed), `kernel/tests/scrub-lone-surrogates.test.ts` covers the code-unit scrub and the ingest scrub. - **Claude Code forks resolve their parent from the transcript, refs are never re-issued after inheritance, large appends no longer read as rewrites (#2426)**: an SDK `forkSession` followed by a plain resume (opencode-claude retrying a turn after a usage-limit error) carries no `--fork-session` SessionStart declaration, and the first forked request differs from the parent's last request in its tail (resume-time `` / skill diffs) — so the byte-exact `findResumeParent` rejected the true parent (equal length, last message differs) and adopted a far older ancestor: 18 folded blocks were not inherited, ~1871 messages were folded again, and `unannounced-rewrite` fired. Three independent fixes: (1) `src/claude-transcript-lineage.ts` reads the `forkedFrom` marker Claude Code stamps on every fork transcript line (`/projects/*/.jsonl`, header-only read ≤ 256 KB, realpath confined to the projects root, cycle-safe, depth ≤ 8) and walks it to the nearest ancestor bili already knows; the #1486 resume-inheritance block asks it first for requests identified by `x-claude-code-session-id` (anthropic, main lane, loopback only) and falls back to the prefix match when no transcript / marker / known ancestor exists. (2) Adoption and `applyCompactionArchive` pruning now reserve the parent's ref space (`bili:ref-floor` sentinel in `messageRefs.byRaw`, excluded from rewrite accounting) so a number that denoted message X in the parent can never be re-issued in the child. (3) `detectUnannouncedHistoryRewrite` also requires `knownIncoming / knownBefore < 0.5`, so a turn that appends many messages is no longer reported as an unannounced rewrite while a real rewrite still is. No new config or env knobs. One existing test that hard-coded the child reusing the parent's ref numbers (`tests/fork-ccr-adoption.test.ts`) now asserts numbering above the parent's highest ref. diff --git a/src/server.ts b/src/server.ts index f06ee81a2b..e133bd01d2 100644 --- a/src/server.ts +++ b/src/server.ts @@ -3531,7 +3531,7 @@ export function outboundPayloadBreakdown( opts: ProxyOptions, route: ReturnType, reqUrl: string, -): { textEstimate: number; overheadEstimate: number; imageTokens: number; payloadEstimate: number; armEstimate: number } { +): { textEstimate: number; overheadEstimate: number; imageTokens: number; payloadEstimate: number; armEstimate: number; armCalibrated: boolean } { const billingUpstream = route?.rewrittenUrl ?? (/^https?:\/\//i.test(reqUrl) ? reqUrl : opts.upstream); const imageTokens = imageReserveFor(prepared.session, prepared.protocol, prepared.parsedBody ?? prepared.body, opts, billingUpstream); // #1498-F2: a kernel-transform failure leaves processedMessages empty while @@ -3540,17 +3540,25 @@ export function outboundPayloadBreakdown( const msgs = prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages; const textEstimate = estimateCoreMessages(msgs); const overheadEstimate = estimateWireOverhead(prepared.protocol, (prepared.parsedBody as Record | undefined) ?? prepared.body); - // #1498-F1: the arm value is not only a preflight floor — the kernel's - // tool-result truncation loop consumes it as its remaining-depth input, so - // an optimistic text estimate (chars/4 undercounts dense JSON/code by up to - // ~4x, #553) stops the loop one candidate early and re-breaks the #604 - // relay rescue. The arming quantity is therefore floored by the text char - // bound, which never undershoots and deliberately does NOT count image - // base64 — image-heavy payloads keep the billing-accurate payloadEstimate - // (#1493) while text-dominated near-window payloads arm deep enough for the - // truncation loop to clear hidden upstream tolerances (#604). - const armEstimate = Math.max(textEstimate, estimateCoreMessagesUpper(msgs)) + overheadEstimate + imageTokens; - return { textEstimate, overheadEstimate, imageTokens, payloadEstimate: textEstimate + overheadEstimate + imageTokens, armEstimate }; + // #2683 (supersedes the #1498-F1 bound floor): the arm adopts a + // BILLING-CALIBER READING of this send — the same k̂-scaled outbound + // formula the preflight trigger uses (#1933 F1 / #2117 B) plus the image + // reserve — NOT the char-count upper bound. The bound was floored in + // because the kernel's tool-result truncation loop once consumed the arm + // as its remaining-depth input (#604 rescue); since the #1492/#1839 reader + // quarantine NO processTurn feed passes an estimate-grade arm into any + // band (preflight runs noEmergencyTruncate), so the bound's only live + // effect was phantom mass: a 1.7–4× over-reading persisted as the session + // baseline, inflating preflight budgets, output headroom and displays + // until the next usage report landed. A reading may miss LOW (chars/4 + // undercounts dense JSON/code, #553) — acceptable because estimate-grade + // values are quarantined from every decision floor, and k̂ closes most of + // the gap where route+model provenance matches. + const kFactor = currentCalibrationFactor(prepared.session.stats, prepared.session.metadata?.lastModel); + const armRaw = textEstimate + overheadEstimate; + const armText = applyEstimateCalibration(armRaw, kFactor, prepared.session.stats.calibratedEstimateOrigin, normalizeUpstreamOrigin(billingUpstream)); + const armEstimate = Math.round(armText + imageTokens); + return { textEstimate, overheadEstimate, imageTokens, payloadEstimate: textEstimate + overheadEstimate + imageTokens, armEstimate, armCalibrated: armText !== armRaw }; } /** #2078: `parsed` carries the caller's pre-parsed send body so forward() does diff --git a/src/server/handle.ts b/src/server/handle.ts index 0166ebde6b..79dae613e6 100644 --- a/src/server/handle.ts +++ b/src/server/handle.ts @@ -20,7 +20,7 @@ import { hostIdForLog, maskHeadersForLog, maskUrlForLog, maskUrlsInText } from " import { bodyMightCarryImages, buildIncomingImageIndex, pruneRetrieveImgExports, refreshIncomingImageIndex, type IndexedImage } from "../image-restore.js"; import { durableMessageToolPaths, withDurableProtection } from "../durable-message-guards.js"; import { biliToolsDeclaredOnWire, countBiliToolUses, declaresMcpBiliToolsOnWire, evaluateSelfHealRound, pluginLaneDegraded, pluginLaneRestore } from "../session-self-heal.js"; -import { acquireInFlight, getSessionAsync, hasProcessedState, markDirty, peekSession, releaseInFlight, storeEffectiveConfig, tickPostRebuildAnchor, withSessionLock, type Session } from "../session.js"; +import { acquireInFlight, getSessionAsync, hasProcessedState, markDirty, peekSession, releaseInFlight, retireCrossModelBaselines, storeEffectiveConfig, tickPostRebuildAnchor, withSessionLock, type Session } from "../session.js"; import { buildCompressSystemPrompt } from "../compress-tool.js"; import { storeEffectiveSearchPlanAware } from "../decompress-shared.js"; import { externalSummaryEnabled, messageTimestampSections } from "../external-summary-surface.js"; @@ -2883,6 +2883,18 @@ async function handleRequest( // plugin sessions, so wire-mode panels fell back to a hardcoded 200K. session.metadata.effectiveContextLimit = nativeWindow; session.metadata.reservedContextLimit = reqConfig.modelContextLimit; + // #2683: a real model change between consecutive requests invalidates + // the billing-grade baseline measured under the previous model — retire + // it here, before prepare sizes the nudge and before forward() runs the + // preflight trigger. lastModel is only ever stamped with a known id, so + // an absent prev is "unknown", not a switch; same-model window moves + // keep the bill valid and never fire this. + if (reqModelId !== undefined && session.metadata.lastModel !== undefined && session.metadata.lastModel !== reqModelId) { + const retired = retireCrossModelBaselines(session); + if (retired > 0) { + log("warn", `[${session.id}] model switched ${session.metadata.lastModel} -> ${reqModelId} — retired cross-model input baseline (${retired} tokens billed under ${session.metadata.lastModel}); next usage report re-anchors (#2683)`); + } + } // #955 runtime-info: record the model id + window source for this // session so /__bili/plugin/status can show them pre-first-request and // post-hoc forensics can tell which source sized the window. diff --git a/src/server/relay.ts b/src/server/relay.ts index 7e34790cf8..e40c94b770 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -146,25 +146,25 @@ function installV1ResponseFramer(res: http.ServerResponse): void { }) as typeof res.end; } -/** #604/#1839: record an upstream failure that will never report usage - * (relay/gateway 5xx, network-level failure). #604 raises - * session.stats.lastInputTokens to a local estimate of the wire body so the - * next prepare() lands in the kernel's emergency band (truncate.threshold = - * 0.95) and truncates large tool results server-side — the rescue that breaks - * the relay-5xx deadlock (#1493 refines the arm to the OUTBOUND payload size - * so fitting payloads don't over-trigger). The arm STAYS load-bearing: - * removing it re-deadlocks exactly those sessions. - * #1839: the arm stays ESTIMATE-grade and the ghost it once caused is killed - * at the reader, not by deleting the arm — while any usage-grade anchor - * exists, effectiveTokenCount's anchor branch prefers lastUsageGradeTokens - * over every estimate-grade value, so the arm can no longer reach the nudge - * denominator, the display or the preflight floors (pre-fix, one aborted turn - * armed 719521 against a real input of 143419 and the inflated denominator - * persisted ~23 min). Anchor-less sessions (fresh / silent backends) size on - * the per-turn min(localInputEstimate, raw) views, where the arm guarantees a - * near-window payload still crosses the emergency band on retry. The next - * real usage report overwrites it. */ -function armFailureShrink(prepared: Prepared, log: (level: string, msg: string) => void, reason: string, est: number): void { +/** #604/#1839/#2683: record an upstream failure that will never report usage + * (relay/gateway 5xx, network-level failure) by raising + * session.stats.lastInputTokens to a local READING of the outbound payload — + * the billing-caliber estimate outboundPayloadBreakdown publishes (#2683: + * k̂-scaled chars/4 + wire overhead + image reserve, NOT the char-count upper + * bound, which overstated the baseline 1.7–4× and forced phantom folds after + * model/window changes). The arm STAYS ESTIMATE-grade: since the #1492/#1839 + * reader quarantine no decision floor trusts an estimate-grade value (the + * preflight floors, effectiveTokenCount's fast path and statusInputBaseline + * all gate on source ∈ {usage, overflow-arm}), so its live effects are + * sizing/budget contexts only — preflight entry budget scaling, the + * responses-lane output headroom, the codex-compact gate, persistence and + * displays. (The #604 era's second consumer — feeding the kernel's + * tool-result truncation loop as remaining depth — is gone: every processTurn + * feed is quarantined and preflight runs noEmergencyTruncate.) The next real + * usage report overwrites it. A mid-session model switch does NOT retire an + * estimate-grade arm (byte-scale property of the payload) — it retires only + * usage/overflow-arm-grade baselines measured under the previous model. */ +function armFailureShrink(prepared: Prepared, log: (level: string, msg: string) => void, reason: string, est: number, calibrated?: boolean): void { const s = prepared.session; if (!Number.isFinite(est) || est <= 0) return; // An upstream-proven ceiling (overflow-arm) plus its route-origin evidence @@ -184,7 +184,7 @@ function armFailureShrink(prepared: Prepared, log: (level: string, msg: string) // The error path returns before forward()'s trailing markDirty — the // arm must schedule its OWN save or it is lost on restart. markDirty(s); - log("warn", `[${s.id}] ${reason} with no usage report — armed emergency shrink with local estimate ${est} tokens`); + log("warn", `[${s.id}] ${reason} with no usage report — armed emergency shrink with local estimate ${est} tokens${calibrated ? " (k̂-calibrated outbound reading)" : " (raw outbound reading)"}`); } } @@ -1228,7 +1228,10 @@ async function forwardCaptured( if (noteCircuitFailFastLog(upstreamUrl)) { log("warn", `[${prepared?.session.id ?? "passthrough"}] [acp-proxy] upstream circuit open — failing fast without contacting ${upstreamUrl}: ${circuitBlock}`); } - if (prepared && req.method !== "GET" && req.method !== "HEAD") armFailureShrink(prepared, log, "upstream circuit open", outboundPayloadBreakdown(prepared, opts, route, req.url ?? "").armEstimate); + if (prepared && req.method !== "GET" && req.method !== "HEAD") { + const arm = outboundPayloadBreakdown(prepared, opts, route, req.url ?? ""); + armFailureShrink(prepared, log, "upstream circuit open", arm.armEstimate, arm.armCalibrated); + } if (prepared && prepared.attachedRetrievals && prepared.attachedRetrievals.length > 0) dropRetrievals(prepared.session, prepared.attachedRetrievals.map((i) => i.ref), "upstream circuit open"); const lastKind = upstreamCircuitLastFailureKind(upstreamUrl); const retryAfter = upstreamCircuitRetryAfterSeconds(upstreamUrl); @@ -1293,7 +1296,10 @@ async function forwardCaptured( recordUpstreamCircuitFailure(upstreamUrl, error, proxyUrl !== undefined, failure.message); // #604: a network-level failure (socket reset, timeout abort) also never // reports usage — arm the emergency shrink like the 5xx branch below. - if (prepared && req.method !== "GET" && req.method !== "HEAD") armFailureShrink(prepared, log, "network failure", outboundPayloadBreakdown(prepared, opts, route, req.url ?? "").armEstimate); + if (prepared && req.method !== "GET" && req.method !== "HEAD") { + const arm = outboundPayloadBreakdown(prepared, opts, route, req.url ?? ""); + armFailureShrink(prepared, log, "network failure", arm.armEstimate, arm.armCalibrated); + } // [#1343] no response means the attached full text never reached the model — // drop-and-log it (a corrective note surfaces on the next qualifying request). if (prepared && prepared.attachedRetrievals && prepared.attachedRetrievals.length > 0) dropRetrievals(prepared.session, prepared.attachedRetrievals.map((i) => i.ref), "upstream network failure"); @@ -1310,7 +1316,10 @@ async function forwardCaptured( // must be strictly smaller than what was just rejected. recordUpstreamConnection(upstreamUrl, proxyUrl, error, skippedProxy); - if (prepared && req.method !== "GET" && req.method !== "HEAD") armFailureShrink(prepared, log, "network failure", outboundPayloadBreakdown(prepared, opts, route, req.url ?? "").armEstimate); + if (prepared && req.method !== "GET" && req.method !== "HEAD") { + const arm = outboundPayloadBreakdown(prepared, opts, route, req.url ?? ""); + armFailureShrink(prepared, log, "network failure", arm.armEstimate, arm.armCalibrated); + } if (prepared && prepared.attachedRetrievals && prepared.attachedRetrievals.length > 0) dropRetrievals(prepared.session, prepared.attachedRetrievals.map((i) => i.ref), "upstream network failure"); // #4601: a non-string (verbatim passthrough) body was reported as 0.0 MiB — measure it. const sentBytes = typeof wireBody === "string" ? Buffer.byteLength(wireBody) : wireBody.byteLength; @@ -2084,7 +2093,8 @@ async function forwardCaptured( // already armed by the overflow path above, and armFailureShrink's // overflow-arm guard keeps this estimate from demoting that ceiling (#2484). if (prepared?.session && upstream.status >= 500) { - armFailureShrink(prepared, log, `upstream ${upstream.status}`, outboundPayloadBreakdown(prepared, opts, route, req.url ?? "").armEstimate); + const arm = outboundPayloadBreakdown(prepared, opts, route, req.url ?? ""); + armFailureShrink(prepared, log, `upstream ${upstream.status}`, arm.armEstimate, arm.armCalibrated); } // #174: always log a non-2xx upstream response (status + request-id + // body snippet) — a 4xx/5xx with zero log trace is a diagnostic diff --git a/src/session.ts b/src/session.ts index 33c9833c52..76da632069 100644 --- a/src/session.ts +++ b/src/session.ts @@ -1014,6 +1014,39 @@ export function displayContextBest(session: { * scenario (estimate-grade overshoot pinning the shown reference) lives on * as a kernel test. */ +/** #2683: a mid-session MODEL switch invalidates billing-grade baselines. A + * usage report (or overflow evidence) measured under model X bills X's + * tokenizer; judging model Y's requests against it forces phantom folds — + * case: 566,322 billed under a 1M-window model judged against a 400k window + * after the switch fired two ~3-minute emergency folds on a payload that + * already fit at 62% of the new window. Retires exactly the model-specific + * readings: lastInputTokens while usage/overflow-arm grade (plus its origin + * stamp), the usage anchor twin lastUsageGradeTokens, the context mirror, and + * the one-shot overflow arm. The last-real-usage TIMESTAMP + * (metadata.contextTokensAt) is KEPT — it is forensic display data ("when did + * the last real bill arrive"), not a decision input. Estimate-grade values + * SURVIVE: they are byte-scale properties of the payload itself, not of the + * model that bills it (and estimate-grade values are quarantined from every + * decision floor already). The calibration ring keys by model and rolls over + * on its own; the cache ledger keeps its history. Same-model window changes + * do NOT retire: the bill stays valid when only the declared window moves. + * Returns the retired trusted baseline (0 when nothing trusted stood). */ +export function retireCrossModelBaselines(session: Session): number { + const st = session.stats; + const trusted = st.lastInputTokensSource === "usage" || st.lastInputTokensSource === "overflow-arm"; + if (!trusted || !(st.lastInputTokens > 0)) return 0; + const retired = st.lastInputTokens; + st.lastInputTokens = 0; + st.lastInputTokensSource = undefined; + st.lastInputTokensOrigin = undefined; + st.lastUsageGradeTokens = undefined; + st.contextTokens = 0; + st.contextTokensSource = undefined; + delete st.overflowArmTokens; + markDirty(session); + return retired; +} + /** #1595: name the third no-usage shape — an upstream SUCCESS that completes * without reporting input usage. Transport failures and upstream 5xx already * log their own armFailureShrink arms; this one used to be silent, so a diff --git a/tests/context-provenance-outbound.test.ts b/tests/context-provenance-outbound.test.ts index 8aabeeaecd..9fb65c363f 100644 --- a/tests/context-provenance-outbound.test.ts +++ b/tests/context-provenance-outbound.test.ts @@ -117,7 +117,11 @@ for (const wire of ["anthropic", "openai", "responses", "google"] as const) { h.setUsage(false); if (scenario === "upstream-failure") h.setFail(); now = 1791049003000; - await h.send([...history, "prior response", "x".repeat(7500)], scenario === "model-switch" ? "model-switched" : "model-test"); + // #2683: the failure arm adopts a billing-caliber reading (chars/4-class, + // k̂-scaled) instead of the char-count bound — size the growth so that + // reading still outgrows the 10000-token fixture bill and the + // replace-stale-usage mechanic under test keeps firing. + await h.send([...history, "prior response", "x".repeat(60_000)], scenario === "model-switch" ? "model-switched" : "model-test"); const after = await h.status(); assert.equal(after.contextTokensSource, "estimate"); assert(after.contextTokens > 18000, JSON.stringify(after)); @@ -125,8 +129,15 @@ for (const wire of ["anthropic", "openai", "responses", "google"] as const) { assert.equal(after.contextTokensAt, now); assert.notEqual(after.contextGeneration, before.contextGeneration); assert.equal(session.metadata.contextTokensAt, usageAt, "prepare cannot refresh the last real usage time"); - assert.equal(session.stats.lastUsageGradeTokens, 10000); - assert.equal(session.stats.lastInputTokensSource, scenario === "upstream-failure" ? "estimate" : "usage"); + // #2683: a MODEL switch retires the cross-model usage reading — the bill was + // measured under the previous model's tokenizer, so it loses decision + // authority (baseline + anchor) until the next usage report. The raw sample + // meter (cumulative inputTokens) and the last-usage timestamp are forensic + // only, so they survive the switch. Same-model turns keep the reading: a + // no-usage success keeps the last real bill, and a failed turn whose payload + // outgrew it replaces it with the outbound reading. + assert.equal(session.stats.lastUsageGradeTokens, scenario === "model-switch" ? undefined : 10000); + assert.equal(session.stats.lastInputTokensSource, scenario === "upstream-failure" ? "estimate" : scenario === "model-switch" ? undefined : "usage"); assert.equal(after.inputTokens, 10000); assert.equal(h.forwarded.length, 2); t.diagnostic(JSON.stringify({ wire, scenario, before: before.contextTokens, after: after.contextTokens, source: after.contextTokensSource, usageAt, estimateAt: after.contextTokensAt })); diff --git a/tests/issue1493-arm-outbound-payload.test.ts b/tests/issue1493-arm-outbound-payload.test.ts index e7729f675a..80254cf6b5 100644 --- a/tests/issue1493-arm-outbound-payload.test.ts +++ b/tests/issue1493-arm-outbound-payload.test.ts @@ -263,9 +263,21 @@ test("#1498-F2: transform-failure fallback arms at the RAW view, not wire overhe const s2 = listSessions().find((x) => x.id === s1!.id); assert.equal(s2?.stats.lastInputTokensSource, "estimate", "armed baseline is tagged as an estimate"); + // #2683: the arm adopts the billing-caliber READING of the raw view + // (chars/4-class + wire overhead), not the char-count UPPER bound — + // the bound overstated the session baseline up to 4× and survived + // model/window changes as phantom mass. The regression this case pins + // ("the raw view is measured, not overhead-only") holds at the lower + // scale: well above wire-overhead-only arming (hundreds of tokens), + // well below the char count. + const armed = s2?.stats.lastInputTokens ?? 0; assert.ok( - (s2?.stats.lastInputTokens ?? 0) >= TEXT_CHARS, - `fallback arm must measure the RAW view (>= ${TEXT_CHARS} char bound), got ${s2?.stats.lastInputTokens} — overhead-only arming never rescues the #604 deadlock`, + armed >= TEXT_CHARS / 8, + `fallback arm must measure the RAW view (>= ${TEXT_CHARS / 8} chars/4-class), got ${armed} — overhead-only arming is the #1498-F2 regression`, + ); + assert.ok( + armed < TEXT_CHARS, + `fallback arm must NOT adopt the char-count upper bound (< ${TEXT_CHARS}), got ${armed} (#2683)`, ); } finally { proxy.close(); diff --git a/tests/issue2683-cross-model-baseline.test.ts b/tests/issue2683-cross-model-baseline.test.ts new file mode 100644 index 0000000000..f2435b61ff --- /dev/null +++ b/tests/issue2683-cross-model-baseline.test.ts @@ -0,0 +1,275 @@ +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import test from "node:test"; + +process.env.NODE_ENV = "test"; +// #1688: pin the legacy single-attempt failure budget so a socket drop +// surfaces as bili's 502 immediately (the shape G2 arms from). +process.env.BILI_REPLAY_RETRY_MAX = "1"; + +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { _resetSessionsForTest, listSessions, type Session } from "../src/session.ts"; +import { setLogCapture } from "../src/logger.ts"; + +// #2683: session input baselines outlived their measurement context — +// (A) armFailureShrink adopted the char-count UPPER BOUND as the session +// baseline (1.7–4× over-reading surviving as phantom mass), and +// (B) a usage-grade baseline measured under model X kept deciding requests +// for model Y after a mid-session model switch (case: 566,322 billed under a +// 1M-window model judged against a 400k window → two ~3-minute emergency +// folds on a payload already at 62% of the new window). The fixes: the arm +// adopts the billing-caliber READING (k̂-scaled outbound estimate + image +// reserve), and a real model change retires the trusted-grade baseline before +// prepare sizes the nudge and before the preflight trigger runs. + +type Msg = { role: string; content: unknown }; + +const LINE = (i: number) => + `const handler_${i} = (req: Request, res: Response) => { res.status(200).json({ status: "ok", id: ${i}, ts: Date.now() }); };`; +const HEAVY = (i: number) => `CODE_${i}_` + LINE(i).repeat(62); + +function baseConversation(): Msg[] { + const msgs: Msg[] = []; + for (let i = 0; i < 12; i++) { + msgs.push({ role: i % 2 === 0 ? "user" : "assistant", content: i < 6 ? HEAVY(i) : `note_${i}_short tail ${i}` }); + } + return msgs; +} + +/** Streaming-call verdicts by 1-based index; anything unmatched gets 200 SSE. */ +type Verdict = { kind: "ok"; input?: number } | { kind: "destroy" }; + +function makeRelay(script: Verdict[]) { + const received: Buffer[] = []; + let streamingCall = 0; + let nonStream = 0; + const server = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + const raw = Buffer.concat(chunks); + received.push(raw); + let parsed: Record = {}; + try { parsed = JSON.parse(raw.toString("utf8")); } catch { /* non-JSON */ } + if (parsed.stream !== true) { + nonStream += 1; + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify({ + id: "msg_summary", type: "message", role: "assistant", model: "claude-relay", + content: [{ type: "text", text: "PREFLIGHT SUMMARY: this segment is a cross-model baseline fixture whose every marker is derivable from its turn index." }], stop_reason: "end_turn", + usage: { input_tokens: 500, output_tokens: 50 }, + })); + return; + } + streamingCall += 1; + const verdict = script[streamingCall - 1]; + if (verdict?.kind === "destroy") { req.socket.destroy(); return; } + const input = verdict && verdict.kind === "ok" ? (verdict.input ?? 5000) : 5000; + res.writeHead(200, { "content-type": "text/event-stream" }); + res.end( + `event: message_start\ndata: ${JSON.stringify({ type: "message_start", message: { id: "m1", role: "assistant", usage: { input_tokens: input } } })}\n\n` + + `event: content_block_start\ndata: ${JSON.stringify({ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } })}\n\n` + + `event: content_block_delta\ndata: ${JSON.stringify({ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "ok" } })}\n\n` + + `event: content_block_stop\ndata: ${JSON.stringify({ type: "content_block_stop", index: 0 })}\n\n` + + `event: message_delta\ndata: ${JSON.stringify({ type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 3 } })}\n\n` + + `event: message_stop\ndata: ${JSON.stringify({ type: "message_stop" })}\n\n`, + ); + }); + }); + return { server, received, nonStreamCalls: () => nonStream }; +} + +interface Harness { + logs: string[]; + relay: ReturnType; + post: (model: string, messages: Msg[]) => Promise; + session: () => Session; + close: () => Promise; +} + +async function startHarness(sessionId: string, script: Verdict[], models: Record): Promise { + const logs: string[] = []; + setLogCapture((level, msg) => { logs.push(`${level} ${msg}`); }); + _resetSessionsForTest(); + const relay = makeRelay(script); + relay.server.listen(0, "127.0.0.1"); + await once(relay.server, "listening"); + const upstreamPort = (relay.server.address() as { port: number }).port; + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const window = Object.values(models)[0].context; + const proxy = await startServer({ + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { [`http://127.0.0.1:${upstreamPort}`]: { models } }, + modelContextLimit: window, + kernelConfig: defaultConfig(window), + compress: { injectTool: true, injectNudge: true }, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: true, + debug: false, + passthrough: false, + compat: { roles: {} }, streamErrorShape: "protocol", passthroughSource: null, autoRestartOnUpdate: false, updateTag: "latest", advisoryCheck: false, releaseNotesCheck: false, + autoUpdate: false, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions); + await once(proxy, "listening"); + const url = `http://127.0.0.1:${(proxy.address() as { port: number }).port}/bili/http://127.0.0.1:${upstreamPort}/v1/messages`; + const headers = { "content-type": "application/json", "x-acp-session": sessionId }; + const post = (model: string, messages: Msg[]) => fetch(url, { + method: "POST", + headers, + body: JSON.stringify({ model, max_tokens: 1024, stream: true, system: "You are a helpful assistant.", messages }), + }); + const session = (): Session => { + const s = listSessions().find((x) => x.id === sessionId); + assert.ok(s, "session exists"); + return s!; + }; + const close = async () => { + await new Promise((r) => proxy.close(() => r())); + await new Promise((r) => relay.server.close(() => r())); + }; + return { logs, relay, post, session, close }; +} + +test("#2683 G1: a model switch retires the cross-model usage baseline — no phantom fold, no phantom nudge meter (e2e)", async () => { + const id = "sess-2683-g1"; + const WINDOW = 400_000; + const h = await startHarness(id, [ + { kind: "ok", input: 566_000 }, // turn 1 under claude-a bills OVER the window + { kind: "ok", input: 6_000 }, // turn 2 under claude-b fits comfortably + ], { "claude-a": { context: WINDOW }, "claude-b": { context: WINDOW } }); + try { + const base = baseConversation(); + const r1 = await h.post("claude-a", base); + assert.equal(r1.status, 200, "turn 1 succeeds"); + await r1.text(); + let s = h.session(); + assert.equal(s.stats.lastInputTokens, 566_000, "usage-grade baseline after turn 1"); + assert.equal(s.stats.lastInputTokensSource, "usage"); + + const msgs2: Msg[] = [...base, { role: "assistant", content: "ok, done." }, { role: "user", content: "step two?" }]; + const r2 = await h.post("claude-b", msgs2); + assert.equal(r2.status, 200, "turn 2 recovers"); + await r2.text(); + + // The switch fired BEFORE the trigger: the 566k bill of claude-a was + // retired, so claude-b's fitting payload judged on its OWN estimate + // never crossed the preflight threshold — zero summarization calls. + // Pre-fix, the trusted floor pinned decisionTokens at 566,322 ≥ the + // ~380k target: the loop folded until its budget ran out (~3 minutes + // live) although the payload sat at ~12k tokens. + assert.equal(h.relay.nonStreamCalls(), 0, "no preflight fold on the switched turn"); + assert.ok( + h.logs.some((l) => l.includes(`[${id}] model switched claude-a -> claude-b`) && l.includes("retired cross-model input baseline (566000")), + "retirement logged with the retired value and both models", + ); + // The phantom 142% meter (566000/400000) must be gone: nothing may size + // off the foreign-model bill anymore. + assert.ok(!h.logs.some((l) => l.includes("(566000/")), "no nudge/preflight line still divides by the switched-out bill"); + + s = h.session(); + assert.equal(s.stats.lastInputTokens, 6_000, "the next usage report re-anchors the baseline"); + assert.equal(s.stats.lastInputTokensSource, "usage"); + } finally { + await h.close(); + } +}); + +test("#2683 G2: a no-usage network failure arms at the billing-caliber reading, not the char-count bound (e2e)", async () => { + const id = "sess-2683-g2"; + const WINDOW = 400_000; + const TEXT_CHARS = 300_000; // dense ASCII: char-count 300k vs chars/4 ≈ 75k + const h = await startHarness(id, [ + { kind: "destroy" }, // turn 1: socket drop, no usage report + { kind: "ok", input: 5_000 }, // turn 2: retry recovers + ], { "claude-m": { context: WINDOW } }); + try { + const body: Msg[] = [{ role: "user", content: "x".repeat(TEXT_CHARS) }]; + const r1 = await h.post("claude-m", body); + assert.equal(r1.status, 502, "network failure surfaces as bili's 502"); + await r1.text(); + + const s1 = h.session(); + assert.equal(s1.stats.lastInputTokensSource, "estimate", "armed baseline is tagged as an estimate"); + const armed = s1.stats.lastInputTokens; + // Raw-view scale (well above wire-overhead-only arming, the #1498-F2 + // regression)… + assert.ok( + armed >= TEXT_CHARS / 8, + `arm measures the raw view (>= ${TEXT_CHARS / 8}), got ${armed}`, + ); + // …but NOT the char-count upper bound (old arm ≥ 300,000 + overhead — + // the 1.7–4× phantom mass this issue reports). + assert.ok( + armed < TEXT_CHARS, + `arm must not adopt the char-count bound (< ${TEXT_CHARS}), got ${armed}`, + ); + assert.ok( + h.logs.some((l) => l.includes("armed emergency shrink with local estimate") && l.includes("(raw outbound reading)")), + "provenance clause present in the arm log (no k̂ learned yet)", + ); + + const r2 = await h.post("claude-m", body); + assert.equal(r2.status, 200, "retry recovers"); + await r2.text(); + assert.equal(h.relay.nonStreamCalls(), 0, "fitting retry forwards without folding"); + const s2 = h.session(); + assert.equal(s2.stats.lastInputTokens, 5_000, "real usage report overwrote the armed value"); + } finally { + await h.close(); + } +}); + +test("#2683 G3: control — a SAME-MODEL over-window bill is NOT retired: the floor stands and the fitting payload forwards (#4288) (e2e)", async () => { + const id = "sess-2683-g3"; + const WINDOW = 100_000; + const h = await startHarness(id, [ + { kind: "ok", input: 150_000 }, // system+tools-heavy upstream: bills far above the window + { kind: "ok", input: 150_000 }, + ], { "claude-m": { context: WINDOW } }); + try { + // Heavy older messages (foldable by the kernel) but a payload whose + // OWN estimate (~11k tokens) sits far under the budget — only the + // trusted 150k bill can trip the trigger. The control: the retirement + // must NOT fire without a model change. On current master the + // discriminator is the session state itself — #4288's fit-forward + // forwards a payload that fits by its own estimate whether or not the + // trusted floor stands, so the summary-call count no longer tells the + // two cases apart (pre-#4288 master folded here via the floored + // comparator; that behavior was deliberately replaced). + const base = baseConversation(); + const r1 = await h.post("claude-m", base); + assert.equal(r1.status, 200, "turn 1 succeeds"); + await r1.text(); + assert.equal(h.session().stats.lastInputTokens, 150_000, "trusted baseline stands"); + + const msgs2: Msg[] = [...base, { role: "assistant", content: "ok." }, { role: "user", content: "next step?" }]; + const r2 = await h.post("claude-m", msgs2); + await r2.text(); + assert.ok(!h.logs.some((l) => l.includes("model switched")), "no spurious switch on a same-model turn"); + assert.equal(h.session().stats.lastInputTokensSource, "usage", "baseline NOT retired without a model change"); + assert.equal(h.session().stats.lastInputTokens, 150_000, "trusted bill survives the same-model turn"); + assert.equal(r2.status, 200, "same-model turn forwards"); + // Current master (#4288): a payload that fits by its own estimate + // forwards even under a higher same-model usage baseline — the + // fit-forward exit outranks the floored comparator, so this turn + // burns zero summary calls (genuine overflow is rescued by the + // overflow-arm if upstream rejects, not by a blind fold here). + assert.equal(h.relay.nonStreamCalls(), 0, "fit-forward on the payload's own estimate (#4288) — no blind fold"); + assert.ok( + h.logs.some((l) => l.includes("payload fits by its own estimate") && l.includes("forwarding (#4288)")), + "#4288 fit-forward logged for the same-model turn", + ); + } finally { + await h.close(); + } +}); diff --git a/tests/notice-registry.test.ts b/tests/notice-registry.test.ts index 6d3b60edb4..b3b0d0e8a4 100644 --- a/tests/notice-registry.test.ts +++ b/tests/notice-registry.test.ts @@ -17,6 +17,11 @@ const cfgPath = path.join(fs.mkdtempSync(path.join(os.tmpdir(), "notice-reg-cfg- process.env.DSH_HOME = fs.mkdtempSync(path.join(os.tmpdir(), "notice-reg-dsh-")); process.env.XDG_STATE_HOME = stateDir; process.env.BILI_CONFIG_FILE = cfgPath; +// #4011 lesson: this suite asserts the pure-sqlite default notice is visible, +// which requires the pure-sqlite SELECTION (unified "sqlite" or absent — the +// gate is unifiedSelectionIsSqlite, not the resolved mode). The suite baseline +// preload pins "legacy" for the broad corpus, so pin the selection explicitly. +process.env.BILI_STORAGE_UNIFIED = "sqlite"; const { NOTICES, visibleNotices, SQLITE_DEFAULT_NOTICE_ID } = await import("../src/notices.ts"); const { IOT_INCOMPAT_NOTICE_ID, PCK_NOTICE_ID, WM_FASTPATH_INTRO_ID, dismissUpgradeNotice } = await import("../src/pck-notice.ts");