From 174eeed52a11b07d2d8a186238099f982a6fda0a Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 12:42:27 -0400 Subject: [PATCH 01/31] fix(inference): a compile refused for budget holds its turn until compression makes it fit A compile that can't fit the budget throws context-manager's OverBudgetError before any provider call, for example after an immediate budget decrease. The error policy retried it into the same refusal within seconds and ended the turn. Compression then made the context fit, but nothing ran the turn again, so the agent stayed silent until its next wake. Now the refusal holds the turn (holdForBudgetRefusal), for a resident or a conversation agent whose context manager has compression pending: - The hold is its own per-agent record and never touches the provider gate, so the maintenance loop keeps ticking the agent and its compression calls are admitted as usual. The refused start gives back its turn token and provider admission, so a held agent is idle to the scheduler and to a surgery. - Later wakes join the held requests. Nothing compiles while the hold lasts. - A release check after every maintenance pass, every drain kick and every surgery runs a dry-run compile at the live budget. Once it fits, the held requests go to the queue together, for one fresh compile, and the agent's context gets a system note saying why the turn waited. As a system message it neither wakes the agent nor counts as conversational input for routing. - The breaker's drain kick runs when the hold starts, and again after each check while compression is pending, if the maintenance timer is off. - The hold ends after a fixed 30 minutes (BUDGET_HOLD_MS), or at once when compression has nothing left to do. Its turn's refusals then take the error policy's path, as before, and the failure's reason says how long the turn was held. - A turn a dry run released, refused again, holds again with its first refusal's start and deadline. - An ephemeral run, whose caller is waiting, and a refusal with nothing pending keep the error policy's path. Observable: the logical turn stays open while held, so it settles when the held turn ends. New traces inference:held and inference:hold_ended. healthSnapshot() gives each agent's budgetHold. The context-refusal ops alert goes out when the hold starts. The breaker's drain kick moved into kickOverBudgetDrain, unchanged for the exhaustion path. Tests: test/budget-refusal-hold.test.ts, eleven cases: - the turn runs once compression makes it fit, with no new wake, and its traces; - the release note, its metadata and its routing; - later wakes joining, with no compile while held; - nothing pending at the refusal; - nothing left to compress; - the deadline; - the maintenance timer off; - a held agent's status, its idleness for a surgery, and the surgery's check; - a released turn whose compile refuses again keeping its first deadline; - an ephemeral run's refusal; - the hold at stop. The hold's fields are read with optional chaining where a framework built from the prototype alone, as some tests build one, reaches them. --- .../budget-refusal-holds-turn.fixed.md | 24 ++ src/framework.ts | 390 +++++++++++++++++- src/types/trace.ts | 37 ++ test/budget-refusal-hold.test.ts | 386 +++++++++++++++++ 4 files changed, 818 insertions(+), 19 deletions(-) create mode 100644 changelog.d/budget-refusal-holds-turn.fixed.md create mode 100644 test/budget-refusal-hold.test.ts diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md new file mode 100644 index 000000000..388a41909 --- /dev/null +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -0,0 +1,24 @@ +- A turn whose compile is refused for budget waits for compression and then + runs, instead of ending (#304). The refusal comes before any provider call, + for example after an immediate budget decrease. Before, the error policy + retried it into the same refusal within seconds and ended the turn, and once + compression had made the context fit, nothing ran it again, so the agent + stayed silent until its next wake. Now a resident's or a conversation + agent's turn is held while its context manager has compression pending, and + it runs as one fresh compile once a dry-run compile fits, checked after + every maintenance pass, drain kick and surgery. Later wakes join it. The + hold never closes the provider gate, so compression keeps running. It ends + after 30 minutes, or at once when compression has nothing left to do, and + the turn then fails as a refused compile did, its reason saying how long it + was held. An ephemeral run, and a refusal with nothing pending, fail as + before. + - **What the agent sees:** when the held turn runs, a system note in its + context says why it waited: "[budget-hold] Your context didn't fit its + budget (100,201 tokens against 48,568), so your turn waited 7 min while + compression caught up. It fits now." + - **What hosts see:** the logical turn stays open, so it settles when the + held turn ends, up to 30 minutes later, rather than as exhausted within + seconds. New traces `inference:held` and `inference:hold_ended`, and + `healthSnapshot()` gives each agent's `budgetHold` (or null). The + `context-refusal` ops alert goes out when the hold starts, and says the + turn is held until its deadline. diff --git a/src/framework.ts b/src/framework.ts index 013277a59..988bc947b 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1107,6 +1107,46 @@ const PROVIDER_ACCELERATION_DEFAULT_COOLDOWN_MS = 65_000; * the run instead (nonOwningProviderWait). */ const PROVIDER_ACCELERATION_MAX_COOLDOWN_MS = 10 * 60_000; const PROVIDER_ACCELERATION_JITTER_MS = 5_000; +/** How long a turn whose compile was refused for budget waits for compression + * to make its context fit (#304). In the incident that asked for the hold, + * compression took 7 minutes from the first refusal to a fit, so 30 minutes + * leaves a large backlog room. A hold already ends at once when compression + * has nothing left to do (checkBudgetHold), so this bounds only a + * compression that keeps working and never makes the context fit. A fixed + * bound rather than an option: no host has needed another, and a stall past + * it is context-manager's readiness to report, not a deadline to tune. */ +const BUDGET_HOLD_MS = 30 * 60_000; +/** How a budget hold ended: its context fits now; its deadline passed; + * compression has nothing left to do and it still doesn't fit; or the + * framework stopped or the agent left. */ +type BudgetHoldOutcome = 'fit' | 'deadline' | 'nothing_pending' | 'stopped'; +/** A wait in words: seconds under a minute, else whole minutes. */ +function describeWait(ms: number): string { + return ms < 60_000 ? `${Math.max(1, Math.round(ms / 1000))} s` : `${Math.round(ms / 60_000)} min`; +} +/** A turn held because its compile was refused for budget (#304, + * holdForBudgetRefusal). */ +interface BudgetHold { + startedAt: number; + /** The deadline: BUDGET_HOLD_MS after the first refusal. */ + until: number; + /** The requests the hold took, the refused one first, for one fresh + * compile once the context fits. */ + heldRequests: InferenceRequest[]; + errorType: 'over_budget' | 'context_refusal'; + /** The refusal's message, bounded. */ + reason: string; + /** The refused compile's budget and size, when context-manager gave them. */ + budget?: number; + actual?: number; + /** The deadline's timer. */ + timer: ReturnType; + /** With the maintenance timer off and the provider gate closed to + * compression: when the hold looks again. */ + recheck?: ReturnType; + /** A release check (a dry-run compile) is running. */ + checking: boolean; +} /** Longest delay one setTimeout can represent (2^31-1 ms, about 24.8 days); * a longer timer fires almost at once, so longer waits re-arm in steps. */ const MAX_TIMER_DELAY_MS = 2_147_483_647; @@ -1646,6 +1686,21 @@ export class AgentFramework { private providerAccelerationCooldowns: Map = new Map(); private providerAccelerationRecoveries: Map = new Map(); private providerAccelerationLastRecovery: Map = new Map(); + /** Turns held while compression makes the context fit (#304), per agent. */ + private budgetHolds: Map = new Map(); + /** BUDGET_HOLD_MS, an instance field only so tests can shorten it. */ + private budgetHoldMs = BUDGET_HOLD_MS; + /** A hold that released its turn because a dry run fit, until that turn's + * compile fits too: a refusal of it holds again with the first refusal's + * start and deadline, since the dry run can fit where the compile, with + * its injections, doesn't. */ + private budgetHoldCarry: Map = new Map(); + /** A hold that ended without a fit, until its turn's compile fits or the + * turn ends: that turn's refusals take the error policy's path, as before + * #304, and say how long the turn was held. */ + private budgetHoldSpent: Map = new Map(); + /** What runs when an agent's drain kick ends (kickOverBudgetDrain). */ + private overBudgetDrainAfter: Map void>> = new Map(); private providerAccelerationDefaultCooldownMs = PROVIDER_ACCELERATION_DEFAULT_COOLDOWN_MS; private providerAccelerationJitterMs = PROVIDER_ACCELERATION_JITTER_MS; private providerHoldHook: ProviderHoldHook | undefined; @@ -2724,6 +2779,9 @@ export class AgentFramework { // A stopped host must never hang behind its own cooldown. this.cancelProviderAdmission(); + // Nor behind a turn held for budget: its requests go with the stop, as a + // provider hold's do. + for (const name of [...(this.budgetHolds?.keys() ?? [])]) this.endBudgetHold(name, 'stopped'); // An open logical turn doesn't survive a stop: a start after a restart // on this instance opens a new one rather than continuing the old turn's // routing (openLogical). A resume still continues its turn, with its @@ -3116,6 +3174,271 @@ export class AgentFramework { return `${error.message} (the provider asked to wait ${asked}; an ephemeral run doesn't sleep a stated wait ` + `longer than ${cap}, so it ends now)`; } + /** + * A compile refused for budget (#304): hold the turn while compression makes + * the context fit, instead of retrying it into the same refusal and ending + * it. The refusal comes before any provider call, and compression is what + * changes the outcome, so the hold never touches the provider gate: the + * maintenance loop keeps ticking the agent, its auxiliary calls are + * admitted as usual, and nothing parks at the gate on the hold's behalf. The + * turn token goes with the failed start, so a held agent is idle to the + * scheduler and to a surgery. Later wakes join the held requests + * (processInferenceRequests), and a release check (checkBudgetHold) after + * every maintenance pass, every drain kick and every surgery sends them to + * the queue together, for one fresh compile, once a dry run fits. + * + * Only a turn that comes through the scheduler holds: a resident's or a + * conversation agent's. An ephemeral run's caller is waiting on it, so its + * refusal keeps the error policy's path. So does a refusal while the + * context manager has nothing pending, since compression can't change it, + * and one of a turn whose hold ended without a fit (budgetHoldSpent). + */ + private holdForBudgetRefusal(agent: Agent, error: Error, trigger?: InferenceRequest): boolean { + const { errorType } = this.classifyInferenceError(error); + if (errorType !== 'over_budget' && errorType !== 'context_refusal') return false; + if (!this.budgetHolds || this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; + if (this.budgetHoldSpent.has(agent.name)) return false; + const existing = this.budgetHolds.get(agent.name); + if (existing) { + if (trigger && !existing.heldRequests.some((r) => this.sameInferenceRequest(r, trigger))) existing.heldRequests.push(trigger); + return true; + } + if (!this.compressionPending(agent)) return false; + const now = Date.now(); + // A turn a dry run released, refused again, keeps its first refusal's + // start and deadline. + const carried = this.budgetHoldCarry.get(agent.name); + this.budgetHoldCarry.delete(agent.name); + const startedAt = carried && carried.until > now ? carried.startedAt : now; + const until = carried && carried.until > now ? carried.until : now + this.budgetHoldMs; + const figures = error as Error & { budget?: unknown; actual?: unknown }; + const hold: BudgetHold = { + startedAt, + until, + heldRequests: trigger ? [trigger] : [], + errorType, + reason: boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS), + ...(typeof figures.budget === 'number' ? { budget: figures.budget } : {}), + ...(typeof figures.actual === 'number' ? { actual: figures.actual } : {}), + timer: setTimeout(() => this.endBudgetHold(agent.name, 'deadline'), Math.min(until - now, MAX_TIMER_DELAY_MS)), + checking: false, + }; + hold.timer.unref?.(); + this.budgetHolds.set(agent.name, hold); + this.emitTrace({ + type: 'inference:held', + agentName: agent.name, + errorType, + error: hold.reason, + ...(hold.budget !== undefined ? { budget: hold.budget } : {}), + ...(hold.actual !== undefined ? { actual: hold.actual } : {}), + until, + heldRequests: hold.heldRequests.length, + }); + const deadline = new Date(until).toISOString(); + console.error(`[budget-hold] agent=${agent.name} compile refused (${errorType}): ${hold.reason} — ` + + `turn held until ${deadline} while compression catches up`); + // The operator hears of the refusal when it happens, as at exhaustion + // before; the per-(agent, kind) cooldown keeps repeats down. + this.opsAlert( + 'context-refusal', + agent.name, + `compile refused (${errorType}): ${hold.reason} — the turn is held until ${deadline} while compression ` + + 'catches up, and runs once the context fits; if it never does, raise contextBudgetTokens or overBudgetGraceRatio.', + { skipLog: true }, + ); + // After the refused start has given back its provider admission + // (startAgentStream's finally): until then the gate reads busy, and the + // kick would wait for the next look instead of running now. + const kick = setTimeout(() => this.compressForBudgetHold(agent), 0); + kick.unref?.(); + return true; + } + + /** Whether the agent's context manager has compression pending, the only + * thing that can make a refused compile fit without a new budget. */ + private compressionPending(agent: Agent): boolean { + try { + return !agent.getContextManager().isReady(); + } catch { + return false; + } + } + + /** Every held turn looks again (checkBudgetHold). */ + private checkBudgetHolds(): void { + if (!this.budgetHolds?.size) return; + for (const name of [...this.budgetHolds.keys()]) void this.checkBudgetHold(name); + } + + /** + * Whether a held turn's context fits now, by a dry-run compile at the live + * budget: it refuses as the turn's compile would, and sends, settles and + * commits nothing (Agent.compileWithInjections). If it fits, the hold ends + * and its turn runs. If it doesn't and compression has nothing left to do, + * the hold ends without a fit. Otherwise it goes on, and with the + * maintenance timer off, which leaves the drain kick as the only thing + * that compresses, it kicks the drain again. + */ + private async checkBudgetHold(agentName: string): Promise { + const hold = this.budgetHolds.get(agentName); + const agent = this.agents.get(agentName); + if (!hold || hold.checking || !agent) return; + hold.checking = true; + let fits: boolean; + try { + await agent.compileWithInjections(undefined, undefined, { dryRun: true }); + fits = true; + } catch (error) { + const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); + // Any other failure is the turn's own compile to meet and report. + fits = errorType !== 'over_budget' && errorType !== 'context_refusal'; + } finally { + hold.checking = false; + } + if (this.budgetHolds.get(agentName) !== hold || this.agents.get(agentName) !== agent) return; + if (fits) { + this.endBudgetHold(agentName, 'fit'); + } else if (!this.compressionPending(agent)) { + this.endBudgetHold(agentName, 'nothing_pending'); + } else if (this.maintenanceIntervalMs <= 0) { + this.compressForBudgetHold(agent); + } + } + + /** + * Kick the drain for a held turn (kickOverBudgetDrain), then look again. A + * provider gate closed to compression would park the kick's calls there, + * where a surgery withdraws them (#352): instead, with the maintenance timer + * off, the hold looks again after an interval; with it on, the next pass's + * check does. + */ + private compressForBudgetHold(agent: Agent): void { + const hold = this.budgetHolds.get(agent.name); + if (!hold) return; + if (this.providerGateBlocked(agent.name)) { + if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { + hold.recheck = setTimeout(() => { + hold.recheck = undefined; + void this.checkBudgetHold(agent.name); + }, DEFAULT_MAINTENANCE_INTERVAL_MS); + hold.recheck.unref?.(); + } + return; + } + this.kickOverBudgetDrain(agent.name, agent, () => void this.checkBudgetHold(agent.name)); + } + + /** + * End a budget hold. Its requests go to the queue together, for one fresh + * compile, unless it ended with a stop or the agent's leaving. After a fit, + * the agent's context gets a note saying why its turn waited, stored as a + * system message: it doesn't wake the agent, and it isn't conversational + * input, so it moves no turn's routing (isConversationalInjection). After a + * deadline, or with nothing left to compress, the turn's refusals take the + * error policy's path, as before #304 (budgetHoldSpent). + */ + private endBudgetHold(agentName: string, outcome: BudgetHoldOutcome): void { + const hold = this.budgetHolds?.get(agentName); + if (!hold) return; + clearTimeout(hold.timer); + if (hold.recheck) clearTimeout(hold.recheck); + this.budgetHolds.delete(agentName); + const waitedMs = Date.now() - hold.startedAt; + this.emitTrace({ + type: 'inference:hold_ended', + agentName, + outcome, + waitedMs, + heldRequests: hold.heldRequests.length, + }); + console.error(`[budget-hold] agent=${agentName} hold ended (${outcome}) after ${waitedMs}ms; ` + + (outcome === 'stopped' + ? `${hold.heldRequests.length} held request(s) dropped with it` + : `${hold.heldRequests.length} held request(s) queued for one fresh compile`)); + const agent = this.agents.get(agentName); + if (outcome === 'stopped' || !agent) return; + if (outcome === 'fit') { + this.budgetHoldCarry.set(agentName, { startedAt: hold.startedAt, until: hold.until }); + const size = hold.budget !== undefined && hold.actual !== undefined + ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` + : ''; + try { + agent.getContextManager().addMessage( + 'user', + [{ + type: 'text', + text: `[budget-hold] Your context didn't fit its budget${size}, so your turn waited ` + + `${describeWait(waitedMs)} while compression caught up. It fits now.`, + }], + { system: true, kind: 'budget-hold' } as MessageMetadata, + ); + } catch (err) { + console.error(`[budget-hold] could not record the release note for ${agentName}: ${describeFailureWithFrames(err)}`); + } + } else { + this.budgetHoldSpent.set(agentName, { outcome, waitedMs }); + } + const requests = hold.heldRequests.length > 0 ? hold.heldRequests : [{ + agentName, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now(), + } satisfies InferenceRequest]; + this.pendingRequests.push(...requests); + } + + /** A refusal of a turn whose hold ended without a fit, its reason saying + * how long the turn waited and why it ended. The error's name is kept, so + * it still classifies as the refusal it is. */ + private afterBudgetHold(agentName: string, error: Error): Error { + const spent = this.budgetHoldSpent?.get(agentName); + if (!spent) return error; + const { errorType } = this.classifyInferenceError(error); + if (errorType !== 'over_budget' && errorType !== 'context_refusal') return error; + const why = spent.outcome === 'deadline' + ? `the hold's ${describeWait(this.budgetHoldMs)} bound passed` + : 'compression had nothing left to do'; + const held = new Error(`${error.message} (the turn was held ${describeWait(spent.waitedMs)} while compression ran; ${why} and it still didn't fit)`); + held.name = error.name; + if (error.stack !== undefined) held.stack = error.stack; + return held; + } + + /** + * The OverBudget deadlock breaker's drain kick (PR #58): a compile refused + * for budget runs no compression of its own, since the drain is driven by + * activity the refusal prevents, so kick the strategy's drain directly. + * Bounded ticks, best-effort, never throws, one kick per agent at a time; + * `after` runs once the kick ends, the one already running if there is one. + */ + private kickOverBudgetDrain(agentName: string, agent: Agent, after?: () => void): void { + if (after) { + const waiting = this.overBudgetDrainAfter.get(agentName); + if (waiting) waiting.push(after); + else this.overBudgetDrainAfter.set(agentName, [after]); + } + if (this.overBudgetDrainInFlight.has(agentName)) return; + this.overBudgetDrainInFlight.add(agentName); + void (async () => { + let ticks = 0; + try { + const cm = agent.getContextManager(); + while (ticks < 8) { + await cm.tick(); + ticks++; + } + console.error(`[inference-failed] drain kicked for ${agentName} (OverBudget breaker, ${ticks} ticks)`); + } catch (err) { + console.error(`[inference-failed] drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); + } finally { + this.overBudgetDrainInFlight.delete(agentName); + } + const waiting = this.overBudgetDrainAfter?.get(agentName); + if (waiting) { + this.overBudgetDrainAfter.delete(agentName); + for (const run of waiting) run(); + } + })(); + } + /** When a hold's timer next fires: at its deadline, stepped past what one * timer can hold. A hold with no deadline (a wait held until released, or * the stand-in for unreadable records) asks again as often as unreadable @@ -3413,6 +3736,9 @@ export class AgentFramework { this.maintenancePass = pass; void pass.finally(() => { if (this.maintenancePass === pass) this.maintenancePass = null; + // Every held turn looks again after every pass, whether or not its + // agent ticked in it: once compression is done, the pass skips it. + this.checkBudgetHolds(); }); } @@ -7614,6 +7940,10 @@ export class AgentFramework { } } } + // The surgery may have moved or rewritten the store under a held turn: + // the history now may fit, or may not. Look again rather than wait for + // the next maintenance pass. + this.checkBudgetHolds(); }; } @@ -9385,6 +9715,8 @@ export class AgentFramework { `${cooldown.waitModel !== undefined ? ` on ${cooldown.waitModel}` : ''}: ` + `${cooldown.heldRequests.length} held request(s) dropped with it (the recorded wait stays)`); } + // So does a budget hold: its requests are for an agent that is gone. + this.endBudgetHold(agentName, 'stopped'); this.emitTrace({ type: 'mcpl:conversation-disposed', agentName, @@ -10255,6 +10587,16 @@ export class AgentFramework { continue; } + // A turn held for budget takes later wakes into the one fresh compile + // its release check queues (#304). No compile while it lasts: the + // context hasn't changed but by compression, which the check watches. + const budgetHold = this.budgetHolds?.get(agentName); + if (budgetHold) { + for (const req of requests) { + if (!budgetHold.heldRequests.some((r) => this.sameInferenceRequest(r, req))) budgetHold.heldRequests.push(req); + } + continue; + } const providerCooldown = this.providerAccelerationCooldowns?.get(agentName); if (providerCooldown) { // A provider wait binds its own model; with a different primary model @@ -11707,6 +12049,10 @@ export class AgentFramework { } } + // The turn's compile fit and its stream exists: a hold that released + // it, or that ended without a fit, is over. + this.budgetHoldCarry?.delete(agent.name); + this.budgetHoldSpent?.delete(agent.name); const handle = this.driveStream( agent, stream, @@ -11728,7 +12074,7 @@ export class AgentFramework { // a hook/compile/stream-setup failure must not leave "typing…" stuck. // (Retries below restart it; stopTyping is idempotent.) this.channelRegistry?.stopTyping(); - const err = error instanceof Error ? error : new Error(String(error)); + const err = this.afterBudgetHold(agent.name, error instanceof Error ? error : new Error(String(error))); this.emitTrace({ type: 'inference:failed', agentName: agent.name, @@ -11737,6 +12083,15 @@ export class AgentFramework { }); agent.reset(); + if (this.holdForBudgetRefusal(agent, err, trigger)) { + // A compile refused for budget waits for compression, not the error + // policy (#304): the hold's release check owns the later compile. The + // logical turn stays open, settled by that compile, and this start's + // token and provider admission go with it (startAgentStream's + // finally), so the held agent is idle meanwhile. + this.eventGate?.onInferenceEnded(agent.name); + return false; + } if (ownsProviderGate && this.holdProviderAcceleration(agent, err, trigger)) { // Capacity scheduling is not poisoned history. The cooldown timer owns // one later fresh compile; no immediate retry or exhaustion marker. @@ -11785,6 +12140,9 @@ export class AgentFramework { speech: '', error: statedWait === 'fail' ? this.statedWaitNotSlept(err) : err.message, }, routing); + // A turn whose hold ended without a fit has now failed: the next + // refusal is a new turn's, and holds. + this.budgetHoldSpent?.delete(agent.name); this.emitTrace({ type: 'inference:exhausted', agentName: agent.name, @@ -15419,6 +15777,17 @@ export class AgentFramework { }; })(), lastInference: this.lastInferenceAt.get(name) ?? null, + budgetHold: (() => { + const hold = this.budgetHolds?.get(name); + return hold === undefined ? null : { + since: hold.startedAt, + until: hold.until, + heldRequests: hold.heldRequests.length, + errorType: hold.errorType, + ...(hold.budget !== undefined ? { budget: hold.budget } : {}), + ...(hold.actual !== undefined ? { actual: hold.actual } : {}), + }; + })(), refusalStats: this.refusalStats.get(name) ?? null, })), }; @@ -15740,24 +16109,7 @@ export class AgentFramework { { skipLog: true }, ); } - if (agent && overBudget && !this.overBudgetDrainInFlight.has(agentName)) { - this.overBudgetDrainInFlight.add(agentName); - void (async () => { - let ticks = 0; - try { - const cm = agent.getContextManager(); - while (ticks < 8) { - await cm.tick(); - ticks++; - } - console.error(`[inference-failed] drain kicked for ${agentName} (OverBudget breaker, ${ticks} ticks)`); - } catch (err) { - console.error(`[inference-failed] drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); - } finally { - this.overBudgetDrainInFlight.delete(agentName); - } - })(); - } + if (agent && overBudget) this.kickOverBudgetDrain(agentName, agent); // (3) Hard-down escalation on repeated identical failure. if (streak >= this.inferenceFailureEscalationThreshold) { diff --git a/src/types/trace.ts b/src/types/trace.ts index b1526bbcb..9300c08a5 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -93,6 +93,43 @@ export type TraceEvent = /** The provider's own error code, when it sent one (at most 128 characters). */ providerErrorCode?: string; }) + | (TraceEventBase & { + /** + * A turn whose compile was refused for budget is held while compression + * makes its context fit, instead of failing (#304). Its logical turn + * stays open; `inference:hold_ended` follows, and then the turn's fresh + * compile, or after a hold that ended without a fit, its exhaustion. + */ + type: 'inference:held'; + agentName: string; + /** 'over_budget', or 'context_refusal' when no summary covers a span the compile would drop. */ + errorType: 'over_budget' | 'context_refusal'; + /** The refusal's message, bounded (300 characters). */ + error: string; + /** The refused compile's budget and size in tokens, when context-manager gave them. */ + budget?: number; + actual?: number; + /** The hold's deadline (epoch ms), 30 minutes after the first refusal. */ + until: number; + /** Requests the hold has taken so far, the refused one included. */ + heldRequests: number; + }) + | (TraceEventBase & { + type: 'inference:hold_ended'; + agentName: string; + /** + * 'fit': a dry-run compile fits, and the held requests are queued for + * one fresh compile. 'deadline': the 30 minutes passed. 'nothing_pending': + * compression has nothing left to do and the context still doesn't fit. + * After those two the requests are queued too, and the turn fails as a + * refused compile did before the hold. 'stopped': the framework stopped + * or the agent left, and the requests went with it. + */ + outcome: 'fit' | 'deadline' | 'nothing_pending' | 'stopped'; + /** Since the first refusal. */ + waitedMs: number; + heldRequests: number; + }) // Streaming inference lifecycle | (TraceEventBase & { diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts new file mode 100644 index 000000000..d91a1a173 --- /dev/null +++ b/test/budget-refusal-hold.test.ts @@ -0,0 +1,386 @@ +/** + * A compile refused for budget holds its turn (#304). An immediate budget + * decrease, or a turn that grew the context past the budget, makes the next + * compile refuse with context-manager's OverBudgetError before any provider + * call. Before the hold, the error policy retried it into the same refusal + * within seconds and the turn ended; compression then freed the space, but + * nothing ran the turn again, so the resident stayed silent until its next + * wake. Now the turn waits while compression runs, and runs once a dry-run + * compile fits: later wakes join it, and a deadline, or compression with + * nothing left to do, ends it as a refused compile ended before. + */ + +import { describe, it } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { ContextManager } from '@animalabs/context-manager'; +import type { EventResponse, Module, ProcessEvent, TraceEvent } from '../src/index.js'; +import { AgentFramework } from '../src/index.js'; +import { createMockResponse, MockMembrane } from './helpers/mock-membrane.js'; + +async function pollUntil(cond: () => boolean, timeoutMs: number): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (cond()) return true; + await new Promise((r) => setTimeout(r, 25)); + } + return cond(); +} + +function stored(framework: AgentFramework): Array<{ text: string; metadata: Record }> { + const cm = framework.getAgent('prime')?.getContextManager(); + const msgs = (cm?.queryMessages({}).messages ?? []) as unknown as Array<{ + content: Array<{ type: string; text?: string }>; + metadata?: Record; + }>; + return msgs.map((m) => ({ + text: (m.content ?? []).filter((b) => b.type === 'text' && typeof b.text === 'string').map((b) => b.text).join('\n'), + metadata: m.metadata ?? {}, + })); +} +const storedText = (framework: AgentFramework): string => stored(framework).map((m) => m.text).join('\n'); + +/** + * The context manager's compression backlog: every compile is refused until + * `fitAt` ticks have run (or `fit` is set), and pending work is reported + * until then, or until `readyAt` ticks, whichever is sooner, as a strategy + * reports a backlog. The maintenance loop and the drain kick tick it. + */ +interface Backlog { ticks: number; fitAt: number; readyAt: number; fit: boolean } +const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; +const pending = (b: Backlog): boolean => !fits(b) && b.ticks < b.readyAt; + +type Patchable = Record<'compile' | 'tick' | 'isReady', (...args: unknown[]) => unknown>; + +async function withBacklog(backlog: Partial, run: (b: Backlog) => Promise): Promise { + const b: Backlog = { ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, ...backlog }; + const proto = ContextManager.prototype as unknown as Patchable; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + proto.compile = async function (this: unknown, ...args: unknown[]) { + if (!fits(b)) { + throw Object.assign( + new Error('Compile plan would exceed hard budget: head=41200 tail=8100 middle=62000 budget=48568'), + { name: 'OverBudgetError', budget: 48_568, actual: 100_201 }, + ); + } + return original.compile.apply(this, args); + }; + proto.tick = async function () { b.ticks++; }; + proto.isReady = function () { return !pending(b); }; + try { + return await run(b); + } finally { + Object.assign(proto, original); + } +} + +/** Stores each external message for the agent and asks for a turn, as a + * channel module does. */ +class InboxModule implements Module { + readonly name = 'inbox'; + async start(): Promise {} + async stop(): Promise {} + getTools() { return []; } + async handleToolCall(): Promise { throw new Error('no tools'); } + async onProcess(event: ProcessEvent): Promise { + if (event.type !== 'external-message') return {}; + return { + addMessages: [{ participant: 'User', content: (event as { content: unknown }).content as never }], + requestInference: true, + }; + } +} + +async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: number }) { + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-')); + const membrane = new MockMembrane(); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.' }], + modules: [new InboxModule()], + syncIntervalMs: 0, + maintenanceIntervalMs: options.maintenanceIntervalMs, + // The default policy's three retries, without its 1, 2 and 4 second waits. + errorPolicy: { maxRetries: 3, onInferenceError: (_error, _agent, attempt) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + if (options.holdMs !== undefined) (framework as unknown as { budgetHoldMs: number }).budgetHoldMs = options.holdMs; + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + const send = (text: string) => framework.pushEvent({ + type: 'external-message', + source: 'test', + content: [{ type: 'text', text }], + metadata: {}, + triggerInference: true, + } as unknown as ProcessEvent); + framework.start(); + const of = (type: string) => traces.filter((e) => e.type === type) as unknown as Array>; + return { + framework, + membrane, + send, + of, + close: async () => { + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + }, + }; +} + +describe('a compile refused for budget holds its turn (#304)', () => { + it('runs the refused turn once compression makes the context fit, with no new wake', async () => { + await withBacklog({ fitAt: 12 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => fits(backlog), 10_000), 'compression should catch up'); + const answered = await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000); + assert.ok(answered, 'the refused turn should run once the context fits, with no new message to wake it'); + assert.equal(t.membrane.calls.length, 1, 'one provider call: the fresh compile after the hold, never a refused one'); + + const held = t.of('inference:held'); + assert.equal(held.length, 1); + assert.equal(held[0]!.errorType, 'over_budget'); + assert.equal(held[0]!.budget, 48_568); + assert.equal(held[0]!.actual, 100_201); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + assert.deepEqual(t.of('inference:exhausted'), [], 'the turn never failed'); + } finally { + await t.close(); + } + }); + }); + + it('tells the agent why its turn waited, in a system note that moves no routing', async () => { + await withBacklog({ fitAt: 12 }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); + const notes = stored(t.framework).filter((m) => m.metadata.kind === 'budget-hold'); + assert.equal(notes.length, 1); + assert.match(notes[0]!.text, /^\[budget-hold\] Your context didn't fit its budget \(100,201 tokens against 48,568\), so your turn waited \d+ s while compression caught up\. It fits now\.$/); + assert.equal(notes[0]!.metadata.system, true); + // A system message is not conversational input: even in a channel the + // turn engaged, it re-pins nothing (#333's arrival rule). + const routing = { engaged: new Set(['test:channel']) }; + const repin = (t.framework as unknown as { repinChannel: (m: unknown, r: unknown) => string | undefined }) + .repinChannel({ ...notes[0]!.metadata, channelId: 'test:channel' }, routing); + assert.equal(repin, undefined); + // The note comes before the turn's compile, so the request carries it. + const request = JSON.stringify(t.membrane.calls[0]); + assert.ok(request.includes('[budget-hold]'), 'the held turn\'s request includes the note'); + } finally { + await t.close(); + } + }); + }); + + it('takes later wakes into the held turn, for one compile', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('First message'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + t.send('Second message, while held'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Second message'), 5_000)); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(t.membrane.calls.length, 0, 'no compile while held, so no provider call'); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.equal(t.membrane.calls.length, 1, 'both wakes ran as one turn'); + const request = JSON.stringify(t.membrane.calls[0]); + assert.ok(request.includes('First message') && request.includes('Second message, while held')); + assert.equal(t.of('inference:held').length, 1, 'the second wake joined the hold, and started none'); + assert.equal(t.of('inference:failed').length, 1, 'only the first refusal: the second wake never compiled while held'); + } finally { + await t.close(); + } + }); + }); + + it('keeps today\'s failure when nothing is pending: compression cannot change it', async () => { + await withBacklog({ readyAt: 0 }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000), 'the turn fails, as before the hold'); + assert.deepEqual(t.of('inference:held'), []); + assert.equal(t.of('inference:exhausted')[0]!.errorType, 'over_budget'); + assert.ok(storedText(t.framework).includes('[inference-failed]'), 'the agent gets the failure marker'); + } finally { + await t.close(); + } + }); + }); + + it('ends the hold when compression has nothing left to do and the context still does not fit', async () => { + // The maintenance pass at start runs 8 ticks before the turn: the + // backlog is still pending at the refusal, and done 12 ticks later. + await withBacklog({ readyAt: 20 }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['nothing_pending']); + const exhausted = t.of('inference:exhausted')[0]!; + assert.equal(exhausted.errorType, 'over_budget'); + assert.match(String(exhausted.error), /the turn was held \d+ s while compression ran; compression had nothing left to do and it still didn't fit/); + assert.equal(t.membrane.calls.length, 0); + } finally { + await t.close(); + } + }); + }); + + it('ends the hold at its deadline, and the turn fails as a refused compile did', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 400 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); + assert.match(String(t.of('inference:exhausted')[0]!.error), /the turn was held \d+ s while compression ran; the hold's 1 s bound passed and it still didn't fit/); + assert.equal(t.membrane.calls.length, 0); + } finally { + await t.close(); + } + }); + }); + + it('keeps compressing with the maintenance timer off: the drain kick runs again until it fits', async () => { + await withBacklog({ fitAt: 20 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 0 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), + 'the kick goes on past its 8 ticks while compression is pending'); + assert.ok(backlog.ticks >= 20); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + }); + + it('leaves the held agent idle: its status shows the hold, and a surgery is admitted and looks again', async () => { + await withBacklog({}, async (backlog) => { + // One maintenance pass at start, then none for an hour: only the + // surgery can bring the release check. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await new Promise((r) => setTimeout(r, 100)); + const agents = (t.framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: Record | null }> }).agents; + const hold = agents.find((a) => a.name === 'prime')!.budgetHold; + assert.ok(hold, 'the status names the hold'); + assert.equal(hold!.errorType, 'over_budget'); + assert.equal(hold!.heldRequests, 1); + assert.equal((t.framework as unknown as { activeTurnTokens: Map }).activeTurnTokens.has('prime'), false, + 'no turn token is kept for the held turn'); + backlog.fit = true; + await t.framework.switchBranch('main'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + 'the surgery brought the release check forward'); + const after = (t.framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: unknown }> }).agents; + assert.equal(after.find((a) => a.name === 'prime')!.budgetHold, null); + } finally { + await t.close(); + } + }); + }); + + it('holds a released turn again with its first deadline when its own compile still refuses', async () => { + // The dry run fits at once, but the turn's compile refuses twice more, + // as a compile with injections can where the dry run fits. + const proto = ContextManager.prototype as unknown as Patchable; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + let refusals = 0; + proto.compile = async function (this: unknown, ...args: unknown[]) { + const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; + if (!dryRun && refusals < 3) { + refusals++; + throw Object.assign(new Error('Compile plan would exceed hard budget: budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 49_000 }); + } + return original.compile.apply(this, args); + }; + proto.tick = async function () {}; + proto.isReady = function () { return false; }; + try { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + const held = t.of('inference:held'); + assert.equal(held.length, 3, 'each refusal of the released turn held it again'); + assert.ok(held.every((e) => e.until === held[0]!.until), 'with the first refusal\'s deadline'); + assert.deepEqual(t.of('inference:exhausted'), []); + } finally { + await t.close(); + } + } finally { + Object.assign(proto, original); + } + }); + + it('leaves an ephemeral run to fail as before: its caller is waiting', async () => { + await withBacklog({}, async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-eph-')); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [], + modules: [], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + errorPolicy: { maxRetries: 3, onInferenceError: (_error, _agent, attempt) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + try { + const { agent, contextManager } = await framework.createEphemeralAgent({ + name: 'worker', model: 'test-model', systemPrompt: 'Do the task.', allowedTools: 'all', + }); + contextManager.addMessage('user', [{ type: 'text', text: 'Run once.' }]); + const run = framework.runEphemeralToCompletion(agent, contextManager); + framework.start(); + // Bounded, so a run that holds fails here rather than hanging the file. + let timer: ReturnType | undefined; + const bound = new Promise((_resolve, reject) => { + timer = setTimeout(() => reject(new Error('the run was still waiting after 5 s')), 5_000); + }); + try { + await assert.rejects(Promise.race([run, bound]), /exceed hard budget/); + } finally { + clearTimeout(timer); + } + assert.deepEqual(traces.filter((e) => e.type === 'inference:held'), []); + } finally { + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + }); + + it('drops the hold at stop', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + let closed = false; + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await t.close(); + closed = true; + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); + assert.equal(t.membrane.calls.length, 0); + } finally { + if (!closed) await t.close(); + } + }); + }); +}); From 13bd4f4965de9ff771acd7852bab8d41799d9dc2 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 13:45:55 -0400 Subject: [PATCH 02/31] fix(inference): the budget hold's note is an injection; its check tells a fit from a failed dry run, runs again when asked mid-run, and paces its kicks The round on an internal review of 174eeed (sw-af304, 23 issues). The note: - The note that tells the agent why its turn waited is no longer stored. It is an injection of the released turn's compile alone (budgetHoldNotes), consumed when the turn's stream exists. The context manager adds injections after it selects within the budget, so the note can't be what refuses the compile, it is in the request only when the compile fit, a release the compile doesn't confirm adds nothing, and nothing stored can move a turn's routing. The release check: - A dry run that fails for a reason other than the budget ends the hold as check_failed rather than fit, and the turn's own compile meets that failure. - One check runs at a time. A check asked for while one runs marks it to run again: the running dry run may predate the surgery or pass that asked, so its verdict is set aside. No check runs while a surgery holds the store; the surgery's release asks for one. - Undo and redo ask for a check, as surgeries do. - With the maintenance timer off, a check that finds compression still pending kicks the drain again after an interval (budgetHoldRecheckMs, the timer's 5 s default), not at once. A backlog that doesn't move costs a kick and a dry run an interval, not a loop. The hold's state: - A released turn refused after its hold's deadline is spent (afterBudgetHold), so it fails as a hold that reached its bound does, rather than holding afresh. - No hold starts once stop() has closed provider admission. - What a hold keeps for an agent's name (the carry, the spent mark, the note) is cleared at stop, at disposal and when the turn fails, as well as when its stream starts. - inference:held carries the hold's start (since). Text: the changelog fragment and the trace docs say the refused attempt still emits inference:failed, as a provider hold's does, name check_failed, and describe the note as an injection. Tests: budget-refusal-hold now has 18 tests: - the note's request, that nothing is stored, and that the next turn carries none; - the later-wakes test waits for both requests in the hold; - later turns holding afresh after a failed and after a completed held turn; - check_failed; - the timer-off kicks paced, and a backlog that never moves; - the closed provider gate's look an interval later; - a dry run set aside when a surgery asks mid-run; - an undo's check; - a released turn's carried start, and its refusal past the deadline; - stop and disposal clearing the hold's state. A sequential mock membrane gives each turn its own response. --- .../budget-refusal-holds-turn.fixed.md | 35 +- src/framework.ts | 211 ++++++++---- src/types/trace.ts | 30 +- test/budget-refusal-hold.test.ts | 301 +++++++++++++++--- 4 files changed, 446 insertions(+), 131 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 388a41909..e8957bbe9 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -4,21 +4,24 @@ retried it into the same refusal within seconds and ended the turn, and once compression had made the context fit, nothing ran it again, so the agent stayed silent until its next wake. Now a resident's or a conversation - agent's turn is held while its context manager has compression pending, and - it runs as one fresh compile once a dry-run compile fits, checked after - every maintenance pass, drain kick and surgery. Later wakes join it. The - hold never closes the provider gate, so compression keeps running. It ends - after 30 minutes, or at once when compression has nothing left to do, and - the turn then fails as a refused compile did, its reason saying how long it - was held. An ephemeral run, and a refusal with nothing pending, fail as - before. - - **What the agent sees:** when the held turn runs, a system note in its - context says why it waited: "[budget-hold] Your context didn't fit its - budget (100,201 tokens against 48,568), so your turn waited 7 min while - compression caught up. It fits now." - - **What hosts see:** the logical turn stays open, so it settles when the - held turn ends, up to 30 minutes later, rather than as exhausted within - seconds. New traces `inference:held` and `inference:hold_ended`, and - `healthSnapshot()` gives each agent's `budgetHold` (or null). The + agent's turn is held while its context manager has compression pending. It + runs as one fresh compile once a dry-run compile fits, checked after every + maintenance pass, every drain kick the hold starts, and every surgery, undo + and redo. Later wakes join it. The hold never closes the provider gate, so + compression keeps running. It ends 30 minutes after the refusal that + started it, or at once when compression has nothing left to do, and the + turn then fails as a refused compile did, its reason saying how long it was + held. An ephemeral run, and a refusal with nothing pending, fail as before. + - **What the agent sees:** the held turn's compile carries a note saying + why it waited, for that compile alone: "[budget-hold] Your context didn't + fit its budget (100,201 tokens against 48,568), so this turn waited 7 min + while compression caught up. It fits now." It's never stored. + - **What hosts see:** the refused attempt still emits `inference:failed`, as + a provider hold's does, so a module that brackets typing with it stops + typing while the turn waits, and the turn's next start emits + `inference:started` again. The logical turn stays open, so it settles when + the held turn ends, up to 30 minutes later, rather than as exhausted + within seconds. New traces `inference:held` and `inference:hold_ended`, + and `healthSnapshot()` gives each agent's `budgetHold` (or null). The `context-refusal` ops alert goes out when the hold starts, and says the turn is held until its deadline. diff --git a/src/framework.ts b/src/framework.ts index 988bc947b..80325b7d3 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1116,10 +1116,11 @@ const PROVIDER_ACCELERATION_JITTER_MS = 5_000; * bound rather than an option: no host has needed another, and a stall past * it is context-manager's readiness to report, not a deadline to tune. */ const BUDGET_HOLD_MS = 30 * 60_000; -/** How a budget hold ended: its context fits now; its deadline passed; - * compression has nothing left to do and it still doesn't fit; or the - * framework stopped or the agent left. */ -type BudgetHoldOutcome = 'fit' | 'deadline' | 'nothing_pending' | 'stopped'; +/** How a budget hold ended: its context fits now; its dry run failed for + * another reason, which the turn's own compile then meets; its deadline + * passed; compression has nothing left to do and it still doesn't fit; or + * the framework stopped or the agent left. */ +type BudgetHoldOutcome = 'fit' | 'check_failed' | 'deadline' | 'nothing_pending' | 'stopped'; /** A wait in words: seconds under a minute, else whole minutes. */ function describeWait(ms: number): string { return ms < 60_000 ? `${Math.max(1, Math.round(ms / 1000))} s` : `${Math.round(ms / 60_000)} min`; @@ -1141,11 +1142,15 @@ interface BudgetHold { actual?: number; /** The deadline's timer. */ timer: ReturnType; - /** With the maintenance timer off and the provider gate closed to - * compression: when the hold looks again. */ + /** With the maintenance timer off: when the hold next compresses or looks + * again (budgetHoldRecheckMs). */ recheck?: ReturnType; /** A release check (a dry-run compile) is running. */ checking: boolean; + /** A check was asked for while one ran: the running one's verdict may + * predate what prompted it (a surgery, a compression pass), so it is + * set aside and the check runs again. */ + again: boolean; } /** Longest delay one setTimeout can represent (2^31-1 ms, about 24.8 days); * a longer timer fires almost at once, so longer waits re-arm in steps. */ @@ -1701,6 +1706,15 @@ export class AgentFramework { private budgetHoldSpent: Map = new Map(); /** What runs when an agent's drain kick ends (kickOverBudgetDrain). */ private overBudgetDrainAfter: Map void>> = new Map(); + /** The note a released turn's compile carries, saying why the turn waited: + * an injection of that compile alone (beginAgentTurn), set when a hold + * releases its turn and cleared once the turn's stream exists. */ + private budgetHoldNotes: Map = new Map(); + /** With the maintenance timer off, how long a held turn waits between + * compressing and looking again: the timer's own default interval, so a + * backlog that doesn't move costs a kick and a dry run every 5 s, not a + * loop. An instance field only so tests can shorten it. */ + private budgetHoldRecheckMs = DEFAULT_MAINTENANCE_INTERVAL_MS; private providerAccelerationDefaultCooldownMs = PROVIDER_ACCELERATION_DEFAULT_COOLDOWN_MS; private providerAccelerationJitterMs = PROVIDER_ACCELERATION_JITTER_MS; private providerHoldHook: ProviderHoldHook | undefined; @@ -2780,8 +2794,12 @@ export class AgentFramework { // A stopped host must never hang behind its own cooldown. this.cancelProviderAdmission(); // Nor behind a turn held for budget: its requests go with the stop, as a - // provider hold's do. + // provider hold's do, and nothing a hold left for its agent's name + // outlives the stop. for (const name of [...(this.budgetHolds?.keys() ?? [])]) this.endBudgetHold(name, 'stopped'); + this.budgetHoldCarry?.clear(); + this.budgetHoldSpent?.clear(); + this.budgetHoldNotes?.clear(); // An open logical turn doesn't survive a stop: a start after a restart // on this instance opens a new one rather than continuing the old turn's // routing (openLogical). A resume still continues its turn, with its @@ -3184,19 +3202,22 @@ export class AgentFramework { * turn token goes with the failed start, so a held agent is idle to the * scheduler and to a surgery. Later wakes join the held requests * (processInferenceRequests), and a release check (checkBudgetHold) after - * every maintenance pass, every drain kick and every surgery sends them to - * the queue together, for one fresh compile, once a dry run fits. + * every maintenance pass, every drain kick the hold starts, and every + * surgery, undo and redo sends them to the queue together, for one fresh + * compile, once a dry run fits. * * Only a turn that comes through the scheduler holds: a resident's or a * conversation agent's. An ephemeral run's caller is waiting on it, so its * refusal keeps the error policy's path. So does a refusal while the - * context manager has nothing pending, since compression can't change it, - * and one of a turn whose hold ended without a fit (budgetHoldSpent). + * context manager has nothing pending, since compression can't change it; + * one of a turn whose hold ended without a fit (budgetHoldSpent); and one + * once stop() has closed provider admission. */ private holdForBudgetRefusal(agent: Agent, error: Error, trigger?: InferenceRequest): boolean { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return false; - if (!this.budgetHolds || this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; + if (!this.budgetHolds || this.providerAdmissionClosed) return false; + if (this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; if (this.budgetHoldSpent.has(agent.name)) return false; const existing = this.budgetHolds.get(agent.name); if (existing) { @@ -3206,11 +3227,12 @@ export class AgentFramework { if (!this.compressionPending(agent)) return false; const now = Date.now(); // A turn a dry run released, refused again, keeps its first refusal's - // start and deadline. + // start and deadline (a carry past its deadline is spent already: + // afterBudgetHold). const carried = this.budgetHoldCarry.get(agent.name); this.budgetHoldCarry.delete(agent.name); - const startedAt = carried && carried.until > now ? carried.startedAt : now; - const until = carried && carried.until > now ? carried.until : now + this.budgetHoldMs; + const startedAt = carried ? carried.startedAt : now; + const until = carried ? carried.until : now + this.budgetHoldMs; const figures = error as Error & { budget?: unknown; actual?: unknown }; const hold: BudgetHold = { startedAt, @@ -3220,8 +3242,9 @@ export class AgentFramework { reason: boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS), ...(typeof figures.budget === 'number' ? { budget: figures.budget } : {}), ...(typeof figures.actual === 'number' ? { actual: figures.actual } : {}), - timer: setTimeout(() => this.endBudgetHold(agent.name, 'deadline'), Math.min(until - now, MAX_TIMER_DELAY_MS)), + timer: setTimeout(() => this.endBudgetHold(agent.name, 'deadline'), Math.min(Math.max(0, until - now), MAX_TIMER_DELAY_MS)), checking: false, + again: false, }; hold.timer.unref?.(); this.budgetHolds.set(agent.name, hold); @@ -3232,18 +3255,19 @@ export class AgentFramework { error: hold.reason, ...(hold.budget !== undefined ? { budget: hold.budget } : {}), ...(hold.actual !== undefined ? { actual: hold.actual } : {}), + since: startedAt, until, heldRequests: hold.heldRequests.length, }); const deadline = new Date(until).toISOString(); - console.error(`[budget-hold] agent=${agent.name} compile refused (${errorType}): ${hold.reason} — ` + + console.error(`[budget-hold] agent=${agent.name} compile refused (${errorType}): ${hold.reason} \u2014 ` + `turn held until ${deadline} while compression catches up`); // The operator hears of the refusal when it happens, as at exhaustion // before; the per-(agent, kind) cooldown keeps repeats down. this.opsAlert( 'context-refusal', agent.name, - `compile refused (${errorType}): ${hold.reason} — the turn is held until ${deadline} while compression ` + + `compile refused (${errorType}): ${hold.reason} \u2014 the turn is held until ${deadline} while compression ` + 'catches up, and runs once the context fits; if it never does, raise contextBudgetTokens or overBudgetGraceRatio.', { skipLog: true }, ); @@ -3272,37 +3296,60 @@ export class AgentFramework { } /** - * Whether a held turn's context fits now, by a dry-run compile at the live - * budget: it refuses as the turn's compile would, and sends, settles and - * commits nothing (Agent.compileWithInjections). If it fits, the hold ends - * and its turn runs. If it doesn't and compression has nothing left to do, - * the hold ends without a fit. Otherwise it goes on, and with the - * maintenance timer off, which leaves the drain kick as the only thing - * that compresses, it kicks the drain again. + * Whether a held turn's context fits now, by a dry-run compile at the + * agent's live budget, the one its turn's compile plans at: it refuses as + * that compile would, and sends, settles and commits nothing + * (Agent.compileWithInjections). If it fits, the hold ends and its turn + * runs. If it fails for another reason, the hold ends too, and the turn's + * own compile meets that failure ('check_failed'). If it refuses and + * compression has nothing left to do, the hold ends without a fit. + * Otherwise it goes on, and with the maintenance timer off, which leaves + * the hold's own drain kicks as all that compresses, it kicks the drain + * again after an interval (budgetHoldRecheckMs). + * + * One check runs at a time. A check asked for while one runs marks it to + * run again: the running dry run may predate a surgery or a pass that + * prompted the new one, so its verdict is set aside. None runs while a + * surgery holds the store; the surgery's release asks for one. */ private async checkBudgetHold(agentName: string): Promise { - const hold = this.budgetHolds.get(agentName); + const hold = this.budgetHolds?.get(agentName); const agent = this.agents.get(agentName); - if (!hold || hold.checking || !agent) return; + if (!hold || !agent) return; + if (hold.checking) { + hold.again = true; + return; + } + if (this.surgeryHold) return; hold.checking = true; - let fits: boolean; + let verdict: 'fit' | 'refused' | 'check_failed'; try { - await agent.compileWithInjections(undefined, undefined, { dryRun: true }); - fits = true; - } catch (error) { - const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); - // Any other failure is the turn's own compile to meet and report. - fits = errorType !== 'over_budget' && errorType !== 'context_refusal'; + do { + hold.again = false; + try { + await agent.compileWithInjections(undefined, undefined, { dryRun: true }); + verdict = 'fit'; + } catch (error) { + const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); + verdict = errorType === 'over_budget' || errorType === 'context_refusal' ? 'refused' : 'check_failed'; + } + } while (hold.again && this.budgetHolds.get(agentName) === hold && !this.surgeryHold); } finally { hold.checking = false; } if (this.budgetHolds.get(agentName) !== hold || this.agents.get(agentName) !== agent) return; - if (fits) { - this.endBudgetHold(agentName, 'fit'); + // A surgery began while the last dry run ran: its release looks again. + if (this.surgeryHold) return; + if (verdict !== 'refused') { + this.endBudgetHold(agentName, verdict); } else if (!this.compressionPending(agent)) { this.endBudgetHold(agentName, 'nothing_pending'); - } else if (this.maintenanceIntervalMs <= 0) { - this.compressForBudgetHold(agent); + } else if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { + hold.recheck = setTimeout(() => { + hold.recheck = undefined; + this.compressForBudgetHold(agent); + }, this.budgetHoldRecheckMs); + hold.recheck.unref?.(); } } @@ -3314,14 +3361,14 @@ export class AgentFramework { * check does. */ private compressForBudgetHold(agent: Agent): void { - const hold = this.budgetHolds.get(agent.name); - if (!hold) return; + const hold = this.budgetHolds?.get(agent.name); + if (!hold || this.agents.get(agent.name) !== agent) return; if (this.providerGateBlocked(agent.name)) { if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { hold.recheck = setTimeout(() => { hold.recheck = undefined; void this.checkBudgetHold(agent.name); - }, DEFAULT_MAINTENANCE_INTERVAL_MS); + }, this.budgetHoldRecheckMs); hold.recheck.unref?.(); } return; @@ -3331,15 +3378,27 @@ export class AgentFramework { /** * End a budget hold. Its requests go to the queue together, for one fresh - * compile, unless it ended with a stop or the agent's leaving. After a fit, - * the agent's context gets a note saying why its turn waited, stored as a - * system message: it doesn't wake the agent, and it isn't conversational - * input, so it moves no turn's routing (isConversationalInjection). After a - * deadline, or with nothing left to compress, the turn's refusals take the - * error policy's path, as before #304 (budgetHoldSpent). + * compile, unless it ended with a stop or the agent's leaving, which take + * the requests and the hold's other state with them. + * - After a fit, or a dry run that failed for another reason, the turn's + * compile carries a note saying why the turn waited (budgetHoldNotes). + * It is an injection of that compile alone, which the context manager + * adds after selecting within the budget: it can't be what refuses the + * compile, it is in the request only when the compile fit, it is never + * stored, and it moves no routing. + * - After a deadline, or with nothing left to compress, the turn's + * refusals take the error policy's path, as before #304 + * (budgetHoldSpent). */ private endBudgetHold(agentName: string, outcome: BudgetHoldOutcome): void { const hold = this.budgetHolds?.get(agentName); + if (outcome === 'stopped') { + // Whatever the agent's name carries outlives no stop and no disposal: + // a later agent of the name starts with none of it. + this.budgetHoldCarry?.delete(agentName); + this.budgetHoldSpent?.delete(agentName); + this.budgetHoldNotes?.delete(agentName); + } if (!hold) return; clearTimeout(hold.timer); if (hold.recheck) clearTimeout(hold.recheck); @@ -3358,24 +3417,13 @@ export class AgentFramework { : `${hold.heldRequests.length} held request(s) queued for one fresh compile`)); const agent = this.agents.get(agentName); if (outcome === 'stopped' || !agent) return; - if (outcome === 'fit') { + if (outcome === 'fit' || outcome === 'check_failed') { this.budgetHoldCarry.set(agentName, { startedAt: hold.startedAt, until: hold.until }); const size = hold.budget !== undefined && hold.actual !== undefined ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` : ''; - try { - agent.getContextManager().addMessage( - 'user', - [{ - type: 'text', - text: `[budget-hold] Your context didn't fit its budget${size}, so your turn waited ` + - `${describeWait(waitedMs)} while compression caught up. It fits now.`, - }], - { system: true, kind: 'budget-hold' } as MessageMetadata, - ); - } catch (err) { - console.error(`[budget-hold] could not record the release note for ${agentName}: ${describeFailureWithFrames(err)}`); - } + this.budgetHoldNotes.set(agentName, `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + + `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`); } else { this.budgetHoldSpent.set(agentName, { outcome, waitedMs }); } @@ -3386,13 +3434,19 @@ export class AgentFramework { } /** A refusal of a turn whose hold ended without a fit, its reason saying - * how long the turn waited and why it ended. The error's name is kept, so - * it still classifies as the refusal it is. */ + * how long the turn waited and why it ended. A released turn refused + * after its hold's deadline passed is such a turn too. The error's name + * is kept, so it still classifies as the refusal it is. */ private afterBudgetHold(agentName: string, error: Error): Error { - const spent = this.budgetHoldSpent?.get(agentName); - if (!spent) return error; const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return error; + const carried = this.budgetHoldCarry?.get(agentName); + if (carried && carried.until <= Date.now()) { + this.budgetHoldCarry.delete(agentName); + this.budgetHoldSpent.set(agentName, { outcome: 'deadline', waitedMs: Date.now() - carried.startedAt }); + } + const spent = this.budgetHoldSpent?.get(agentName); + if (!spent) return error; const why = spent.outcome === 'deadline' ? `the hold's ${describeWait(this.budgetHoldMs)} bound passed` : 'compression had nothing left to do'; @@ -7745,6 +7799,8 @@ export class AgentFramework { params: { turnIndex: checkpoint.turnIndex }, result: { sourceBranch: currentBranch.name, targetBranch: undoBranchName }, }); + // The store moved: a held turn's context may fit now (checkBudgetHold). + this.checkBudgetHolds(); return { undone: true, @@ -7807,6 +7863,8 @@ export class AgentFramework { ...(requester ? { requester } : {}), result: { sourceBranch: currentBranch.name, targetBranch: branchName }, }); + // The store moved: a held turn's context may fit now (checkBudgetHold). + this.checkBudgetHolds(); return { redone: true, @@ -11964,6 +12022,20 @@ export class AgentFramework { } } + // A turn a budget hold released tells the agent why it waited (#304). + // Injections are added after the context manager selects within the + // budget, so the note can't be what refuses this compile, and it is in + // the request only when the compile fit. It is never stored. + const holdNote = this.budgetHoldNotes?.get(agent.name); + if (holdNote !== undefined) { + const noteInjection: ContextInjection = { + namespace: 'framework:budget-hold', + position: 'afterUser', + content: [{ type: 'text', text: holdNote }], + }; + injections = injections ? [...injections, noteInjection] : [noteInjection]; + } + if (trigger?.ephemeralSystemPrompt) { const silentInjection: ContextInjection = { namespace: 'framework:silent-heartbeat', @@ -12050,9 +12122,10 @@ export class AgentFramework { } // The turn's compile fit and its stream exists: a hold that released - // it, or that ended without a fit, is over. + // it, or that ended without a fit, is over, and its note was carried. this.budgetHoldCarry?.delete(agent.name); this.budgetHoldSpent?.delete(agent.name); + this.budgetHoldNotes?.delete(agent.name); const handle = this.driveStream( agent, stream, @@ -12140,9 +12213,11 @@ export class AgentFramework { speech: '', error: statedWait === 'fail' ? this.statedWaitNotSlept(err) : err.message, }, routing); - // A turn whose hold ended without a fit has now failed: the next - // refusal is a new turn's, and holds. + // The turn has failed, after a hold or not: the next refusal is a new + // turn's, holds afresh, and carries no note of this one's wait. this.budgetHoldSpent?.delete(agent.name); + this.budgetHoldCarry?.delete(agent.name); + this.budgetHoldNotes?.delete(agent.name); this.emitTrace({ type: 'inference:exhausted', agentName: agent.name, diff --git a/src/types/trace.ts b/src/types/trace.ts index 9300c08a5..129af0524 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -96,9 +96,11 @@ export type TraceEvent = | (TraceEventBase & { /** * A turn whose compile was refused for budget is held while compression - * makes its context fit, instead of failing (#304). Its logical turn - * stays open; `inference:hold_ended` follows, and then the turn's fresh - * compile, or after a hold that ended without a fit, its exhaustion. + * makes its context fit, instead of failing (#304). It follows the + * refused attempt's `inference:failed`, as a provider hold does. Its + * logical turn stays open; `inference:hold_ended` follows, and then + * the turn's fresh compile, or after a hold that ended without a fit, + * its exhaustion. */ type: 'inference:held'; agentName: string; @@ -109,7 +111,10 @@ export type TraceEvent = /** The refused compile's budget and size in tokens, when context-manager gave them. */ budget?: number; actual?: number; - /** The hold's deadline (epoch ms), 30 minutes after the first refusal. */ + /** When the hold started (epoch ms): this refusal, or for a released + * turn refused again, the refusal that first held it. */ + since: number; + /** The hold's deadline (epoch ms), 30 minutes after `since`. */ until: number; /** Requests the hold has taken so far, the refused one included. */ heldRequests: number; @@ -119,14 +124,17 @@ export type TraceEvent = agentName: string; /** * 'fit': a dry-run compile fits, and the held requests are queued for - * one fresh compile. 'deadline': the 30 minutes passed. 'nothing_pending': - * compression has nothing left to do and the context still doesn't fit. - * After those two the requests are queued too, and the turn fails as a - * refused compile did before the hold. 'stopped': the framework stopped - * or the agent left, and the requests went with it. + * one fresh compile. 'check_failed': the dry run failed for a reason + * other than the budget, and the requests are queued for the turn's + * own compile to meet it. 'deadline': the 30 minutes passed. + * 'nothing_pending': compression has nothing left to do and the context + * still doesn't fit. After those two the requests are queued too, and + * the turn fails as a refused compile did before the hold. 'stopped': + * the framework stopped or the agent left, and the requests went with + * it. */ - outcome: 'fit' | 'deadline' | 'nothing_pending' | 'stopped'; - /** Since the first refusal. */ + outcome: 'fit' | 'check_failed' | 'deadline' | 'nothing_pending' | 'stopped'; + /** Since the hold started (`inference:held`'s `since`). */ waitedMs: number; heldRequests: number; }) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index d91a1a173..bcf4fed6d 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -18,7 +18,19 @@ import { join } from 'node:path'; import { ContextManager } from '@animalabs/context-manager'; import type { EventResponse, Module, ProcessEvent, TraceEvent } from '../src/index.js'; import { AgentFramework } from '../src/index.js'; -import { createMockResponse, MockMembrane } from './helpers/mock-membrane.js'; +import type { NormalizedRequest, YieldingStream } from '@animalabs/membrane'; +import { createMockResponse, MockMembrane, MockYieldingStream } from './helpers/mock-membrane.js'; + +/** One queued response per stream, so each turn gets its own. */ +class SequentialStreamMembrane extends MockMembrane { + private streamIdx = 0; + override streamYielding(request: NormalizedRequest, _options?: unknown): YieldingStream { + this.calls.push(request); + const stream = new MockYieldingStream(this.responses.slice(this.streamIdx, ++this.streamIdx)); + this.lastStream = stream; + return stream; + } +} async function pollUntil(cond: () => boolean, timeoutMs: number): Promise { const deadline = Date.now() + timeoutMs; @@ -46,20 +58,37 @@ const storedText = (framework: AgentFramework): string => stored(framework).map( * The context manager's compression backlog: every compile is refused until * `fitAt` ticks have run (or `fit` is set), and pending work is reported * until then, or until `readyAt` ticks, whichever is sooner, as a strategy - * reports a backlog. The maintenance loop and the drain kick tick it. + * reports a backlog. The maintenance loop and the drain kick tick it. A + * compile decides at its start, as the context manager selects at its start; + * a dry run then waits for `dryRunGate` when one is set, and throws + * `dryRunError` once when one is set. */ -interface Backlog { ticks: number; fitAt: number; readyAt: number; fit: boolean } +interface Backlog { + ticks: number; fitAt: number; readyAt: number; fit: boolean; + dryRuns: number; compiles: number; + dryRunGate?: Promise; dryRunError?: Error; +} const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; const pending = (b: Backlog): boolean => !fits(b) && b.ticks < b.readyAt; type Patchable = Record<'compile' | 'tick' | 'isReady', (...args: unknown[]) => unknown>; async function withBacklog(backlog: Partial, run: (b: Backlog) => Promise): Promise { - const b: Backlog = { ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, ...backlog }; + const b: Backlog = { ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, dryRuns: 0, compiles: 0, ...backlog }; const proto = ContextManager.prototype as unknown as Patchable; const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; proto.compile = async function (this: unknown, ...args: unknown[]) { - if (!fits(b)) { + const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; + if (dryRun) b.dryRuns++; + else b.compiles++; + const fitsNow = fits(b); + if (dryRun && b.dryRunGate) await b.dryRunGate; + if (dryRun && b.dryRunError) { + const error = b.dryRunError; + b.dryRunError = undefined; + throw error; + } + if (!fitsNow) { throw Object.assign( new Error('Compile plan would exceed hard budget: head=41200 tail=8100 middle=62000 budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 100_201 }, @@ -93,10 +122,32 @@ class InboxModule implements Module { } } -async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: number }) { +/** The framework's internals these tests read or shorten. */ +interface Internal { + budgetHoldMs: number; + budgetHoldRecheckMs: number; + budgetHolds: Map; + budgetHoldCarry: Map; + budgetHoldSpent: Map; + budgetHoldNotes: Map; + activeTurnTokens: Map; + pendingRequests: Array<{ agentName: string }>; + providerGateBlocked: (agentName: string) => boolean; + disposeConversationAgent: (agentName: string) => void; +} + +const heldRequests = (framework: AgentFramework): number | undefined => { + const agents = (framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: { heldRequests: number } | null }> }).agents; + return agents.find((a) => a.name === 'prime')?.budgetHold?.heldRequests; +}; + +async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: number; recheckMs?: number; responses?: number }) { const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-')); - const membrane = new MockMembrane(); + const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); + for (let i = 1; i < (options.responses ?? 1); i++) { + membrane.pushResponse(createMockResponse([{ type: 'text', text: `Answered turn ${i + 1}` }])); + } const framework = await AgentFramework.create({ storePath: join(tempDir, 'store'), membrane: membrane.asMembrane(), @@ -107,7 +158,9 @@ async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: n // The default policy's three retries, without its 1, 2 and 4 second waits. errorPolicy: { maxRetries: 3, onInferenceError: (_error, _agent, attempt) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, }); - if (options.holdMs !== undefined) (framework as unknown as { budgetHoldMs: number }).budgetHoldMs = options.holdMs; + const internal = framework as unknown as Internal; + if (options.holdMs !== undefined) internal.budgetHoldMs = options.holdMs; + if (options.recheckMs !== undefined) internal.budgetHoldRecheckMs = options.recheckMs; const traces: TraceEvent[] = []; framework.onTrace((e) => traces.push(e)); const send = (text: string) => framework.pushEvent({ @@ -121,6 +174,7 @@ async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: n const of = (type: string) => traces.filter((e) => e.type === type) as unknown as Array>; return { framework, + internal, membrane, send, of, @@ -155,25 +209,21 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('tells the agent why its turn waited, in a system note that moves no routing', async () => { + it('tells the agent why its turn waited, in that turn\'s compile alone', async () => { await withBacklog({ fitAt: 12 }, async () => { - const t = await residentTurn({ maintenanceIntervalMs: 50 }); + const t = await residentTurn({ maintenanceIntervalMs: 50, responses: 2 }); try { t.send('Are you there?'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); - const notes = stored(t.framework).filter((m) => m.metadata.kind === 'budget-hold'); - assert.equal(notes.length, 1); - assert.match(notes[0]!.text, /^\[budget-hold\] Your context didn't fit its budget \(100,201 tokens against 48,568\), so your turn waited \d+ s while compression caught up\. It fits now\.$/); - assert.equal(notes[0]!.metadata.system, true); - // A system message is not conversational input: even in a channel the - // turn engaged, it re-pins nothing (#333's arrival rule). - const routing = { engaged: new Set(['test:channel']) }; - const repin = (t.framework as unknown as { repinChannel: (m: unknown, r: unknown) => string | undefined }) - .repinChannel({ ...notes[0]!.metadata, channelId: 'test:channel' }, routing); - assert.equal(repin, undefined); - // The note comes before the turn's compile, so the request carries it. - const request = JSON.stringify(t.membrane.calls[0]); - assert.ok(request.includes('[budget-hold]'), 'the held turn\'s request includes the note'); + const first = JSON.stringify(t.membrane.calls[0]); + assert.match(first, /\[budget-hold\] Your context didn't fit its budget \(100,201 tokens against 48,568\), so this turn waited \d+ s while compression caught up\. It fits now\./); + // An injection, never stored: no later turn sees it, and no stored + // message could move a turn's routing (#333's arrival rule). + assert.ok(!storedText(t.framework).includes('[budget-hold]'), 'the note is not stored'); + t.send('And now?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered turn 2'), 5_000)); + assert.equal(t.membrane.calls.length, 2); + assert.ok(!JSON.stringify(t.membrane.calls[1]).includes('[budget-hold]'), 'the next turn carries no note'); } finally { await t.close(); } @@ -187,16 +237,15 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('First message'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); t.send('Second message, while held'); - assert.ok(await pollUntil(() => storedText(t.framework).includes('Second message'), 5_000)); - await new Promise((r) => setTimeout(r, 200)); + assert.ok(await pollUntil(() => heldRequests(t.framework) === 2, 5_000), 'the second wake joins the hold'); assert.equal(t.membrane.calls.length, 0, 'no compile while held, so no provider call'); + assert.equal(t.of('inference:failed').length, 1, 'only the first refusal: the second wake never compiled while held'); backlog.fit = true; assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); assert.equal(t.membrane.calls.length, 1, 'both wakes ran as one turn'); const request = JSON.stringify(t.membrane.calls[0]); assert.ok(request.includes('First message') && request.includes('Second message, while held')); assert.equal(t.of('inference:held').length, 1, 'the second wake joined the hold, and started none'); - assert.equal(t.of('inference:failed').length, 1, 'only the first refusal: the second wake never compiled while held'); } finally { await t.close(); } @@ -252,9 +301,57 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('keeps compressing with the maintenance timer off: the drain kick runs again until it fits', async () => { + it('holds a later turn afresh after a hold that ended in failure, and after one that ran', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 400 }); + try { + // A turn whose hold passes its bound fails ... + t.send('First'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + // ... and the next turn's refusal holds again, with its own deadline. + t.send('Second'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 2, 5_000), 'a failed turn\'s spent hold does not pass to the next turn'); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + // After a held turn that ran, a later refusal starts a new hold, not + // the released turn's. + backlog.fit = false; + const before = Date.now(); + t.send('Third'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 3, 5_000)); + const third = t.of('inference:held')[2]!; + assert.ok((third.since as number) >= before, 'its own start, not the released turn\'s'); + assert.ok((third.until as number) >= before + 400); + } finally { + await t.close(); + } + }); + }); + + it('runs again when a dry run fails for another reason, and the turn\'s own compile meets it', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + // The context now fits, but the check's dry run fails for another + // reason: the hold doesn't call that a fit, and lets the turn's + // compile find out. + backlog.dryRunError = new Error('store read failed'); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['check_failed']); + assert.equal(backlog.dryRunError, undefined, 'the dry run met the error'); + assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]'), 'the turn that ran says why it waited'); + } finally { + await t.close(); + } + }); + }); + + it('paces compression with the maintenance timer off: the drain kick runs again until it fits', async () => { await withBacklog({ fitAt: 20 }, async (backlog) => { - const t = await residentTurn({ maintenanceIntervalMs: 0 }); + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 50 }); try { t.send('Are you there?'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), @@ -267,6 +364,44 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('with the maintenance timer off, a backlog that never moves costs a kick an interval, not a loop', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 100 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await new Promise((r) => setTimeout(r, 550)); + // The first kick at once, then one an interval: about 6 in 550 ms. + assert.ok(backlog.ticks <= 8 * 8, `${backlog.ticks} ticks in 550 ms`); + assert.ok(backlog.dryRuns <= 8, `${backlog.dryRuns} dry runs in 550 ms`); + assert.ok(backlog.ticks >= 16, 'it does kick again'); + } finally { + await t.close(); + } + }); + }); + + it('with the maintenance timer off and the provider gate closed, looks again an interval later', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 50 }); + const open = t.internal.providerGateBlocked.bind(t.framework); + let blocked = true; + t.internal.providerGateBlocked = (name: string) => (name === 'prime' && blocked) || open(name); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(backlog.ticks, 0, 'no kick while the gate is closed: its calls would park there'); + backlog.fit = true; + blocked = false; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + 'the hold looked again once an interval passed'); + } finally { + await t.close(); + } + }); + }); + it('leaves the held agent idle: its status shows the hold, and a surgery is admitted and looks again', async () => { await withBacklog({}, async (backlog) => { // One maintenance pass at start, then none for an hour: only the @@ -275,27 +410,69 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - await new Promise((r) => setTimeout(r, 100)); + assert.ok(await pollUntil(() => !t.internal.activeTurnTokens.has('prime'), 3_000), 'no turn token is kept for the held turn'); const agents = (t.framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: Record | null }> }).agents; const hold = agents.find((a) => a.name === 'prime')!.budgetHold; assert.ok(hold, 'the status names the hold'); assert.equal(hold!.errorType, 'over_budget'); assert.equal(hold!.heldRequests, 1); - assert.equal((t.framework as unknown as { activeTurnTokens: Map }).activeTurnTokens.has('prime'), false, - 'no turn token is kept for the held turn'); backlog.fit = true; await t.framework.switchBranch('main'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), 'the surgery brought the release check forward'); - const after = (t.framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: unknown }> }).agents; - assert.equal(after.find((a) => a.name === 'prime')!.budgetHold, null); + assert.equal(heldRequests(t.framework), undefined); + } finally { + await t.close(); + } + }); + }); + + it('sets aside a dry run that was running when a surgery asked to look again', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => backlog.ticks >= 16 && !t.internal.activeTurnTokens.has('prime'), 3_000)); + // A dry run starts, decides it doesn't fit, and is held there. + let release!: () => void; + backlog.dryRunGate = new Promise((resolve) => { release = resolve; }); + const dryRuns = backlog.dryRuns; + (t.framework as unknown as { checkBudgetHolds: () => void }).checkBudgetHolds(); + assert.ok(await pollUntil(() => backlog.dryRuns === dryRuns + 1, 2_000)); + // The context now fits, and a surgery asks to look again. + backlog.fit = true; + await t.framework.switchBranch('main'); + backlog.dryRunGate = undefined; + release(); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + 'the stale verdict was set aside, and a fresh dry run released the turn'); + assert.ok(backlog.dryRuns >= dryRuns + 2); } finally { await t.close(); } }); }); - it('holds a released turn again with its first deadline when its own compile still refuses', async () => { + it('looks again after an undo', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => backlog.ticks >= 16 && !t.internal.activeTurnTokens.has('prime'), 3_000)); + backlog.fit = true; + const undone = t.framework.undoLastTurn('prime'); + assert.equal(undone.undone, true, 'the refused start left a checkpoint to undo'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + 'the undo brought the release check forward'); + } finally { + await t.close(); + } + }); + }); + + it('holds a released turn again with its first refusal\'s start and deadline when its own compile still refuses', async () => { // The dry run fits at once, but the turn's compile refuses twice more, // as a compile with injections can where the dry run fits. const proto = ContextManager.prototype as unknown as Patchable; @@ -318,7 +495,8 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); const held = t.of('inference:held'); assert.equal(held.length, 3, 'each refusal of the released turn held it again'); - assert.ok(held.every((e) => e.until === held[0]!.until), 'with the first refusal\'s deadline'); + assert.ok(held.every((e) => e.since === held[0]!.since), 'with the first refusal\'s start'); + assert.ok(held.every((e) => e.until === held[0]!.until), 'and its deadline'); assert.deepEqual(t.of('inference:exhausted'), []); } finally { await t.close(); @@ -328,6 +506,35 @@ describe('a compile refused for budget holds its turn (#304)', () => { } }); + it('fails a released turn refused after its hold\'s deadline, rather than holding it afresh', async () => { + const proto = ContextManager.prototype as unknown as Patchable; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + // The dry run fits, but the released turn's compile starts only after + // the deadline (a slow queue), and refuses. + let real = 0; + proto.compile = async function (this: unknown, ...args: unknown[]) { + const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; + if (dryRun) return original.compile.apply(this, args); + if (++real === 2) await new Promise((r) => setTimeout(r, 500)); + throw Object.assign(new Error('Compile plan would exceed hard budget: budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 49_000 }); + }; + proto.tick = async function () {}; + proto.isReady = function () { return false; }; + try { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 300 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.equal(t.of('inference:held').length, 1, 'no second hold past the first one\'s deadline'); + assert.match(String(t.of('inference:exhausted')[0]!.error), /the hold's 1 s bound passed and it still didn't fit/); + } finally { + await t.close(); + } + } finally { + Object.assign(proto, original); + } + }); + it('leaves an ephemeral run to fail as before: its caller is waiting', async () => { await withBacklog({}, async () => { const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-eph-')); @@ -367,7 +574,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('drops the hold at stop', async () => { + it('drops the hold, its requests and what it left for the name, at stop and at disposal', async () => { await withBacklog({}, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); let closed = false; @@ -377,10 +584,32 @@ describe('a compile refused for budget holds its turn (#304)', () => { await t.close(); closed = true; assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); + assert.deepEqual(t.internal.pendingRequests.filter((r) => r.agentName === 'prime'), [], 'the held request went with the stop'); assert.equal(t.membrane.calls.length, 0); + for (const map of [t.internal.budgetHolds, t.internal.budgetHoldCarry, t.internal.budgetHoldSpent, t.internal.budgetHoldNotes]) { + assert.equal(map.size, 0); + } } finally { if (!closed) await t.close(); } }); + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 300 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + t.internal.budgetHoldSpent.set('prime', { outcome: 'deadline', waitedMs: 1 }); + t.internal.disposeConversationAgent('prime'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); + assert.equal(t.internal.budgetHolds.size, 0); + assert.equal(t.internal.budgetHoldSpent.size, 0, 'a later agent of the name starts with none of it'); + // Past the hold's deadline: nothing is queued for an agent that's gone. + await new Promise((r) => setTimeout(r, 450)); + assert.deepEqual(t.internal.pendingRequests.filter((r) => r.agentName === 'prime'), []); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); + } finally { + await t.close(); + } + }); }); }); From 41afd1178751932d71a30b8de721f76cce8e4757 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 13:54:41 -0400 Subject: [PATCH 03/31] test(inference): the budget hold's note is in the released turn's first compile only, and stays out of the store and the turn's routing Two tests asked for in Ada-1017's read of the round (room-203 #97000). - A held turn whose first stream after the release restarts through a fresh compile (physicalWindowTokens, #92): the first compile carries the note, the restart's does not. The note is consumed when the turn's first stream exists. - A held turn on a channel (an mcpl:channel-incoming message on discord:guild:chanA): nothing of the note is stored, in any message or its metadata, and every start of the turn, the refused one and the released one, has the channel as its locus. --- test/budget-refusal-hold.test.ts | 93 +++++++++++++++++++++++++++++++- 1 file changed, 92 insertions(+), 1 deletion(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index bcf4fed6d..58e0ae990 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -16,7 +16,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { ContextManager } from '@animalabs/context-manager'; -import type { EventResponse, Module, ProcessEvent, TraceEvent } from '../src/index.js'; +import type { EventResponse, Module, ProcessEvent, ToolDefinition, ToolResult, TraceEvent } from '../src/index.js'; import { AgentFramework } from '../src/index.js'; import type { NormalizedRequest, YieldingStream } from '@animalabs/membrane'; import { createMockResponse, MockMembrane, MockYieldingStream } from './helpers/mock-membrane.js'; @@ -122,6 +122,36 @@ class InboxModule implements Module { } } +/** A tool whose result is large, and a first round whose usage reports a + * prompt near a 200k window, so the round's continuation restarts the stream + * through a fresh compile (physicalWindowTokens, #92). */ +class BlobToolModule implements Module { + readonly name = 'canned'; + async start(): Promise {} + async stop(): Promise {} + getTools(): ToolDefinition[] { + return [{ name: 'fetch', description: 'blob', inputSchema: { type: 'object', properties: {} } }]; + } + async handleToolCall(): Promise { + return { success: true, data: { blob: 'x'.repeat(30_000) } }; + } + async onProcess(event: ProcessEvent): Promise { + if (event.type !== 'external-message') return {}; + return { + addMessages: [{ participant: 'User', content: (event as { content: unknown }).content as never }], + requestInference: true, + }; + } +} +function nearCapToolResponse() { + const r = createMockResponse([ + { type: 'text', text: 'Working' }, + { type: 'tool_use', id: 'call_1', name: 'canned--fetch', input: {} }, + ], 'tool_use'); + (r as { usage: unknown }).usage = { inputTokens: 5_000, outputTokens: 5, cacheReadTokens: 195_000 }; + return r; +} + /** The framework's internals these tests read or shorten. */ interface Internal { budgetHoldMs: number; @@ -230,6 +260,67 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('puts the note into the released turn\'s first compile only: a restart inside the turn compiles without it', async () => { + await withBacklog({ fitAt: 12 }, async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-restart-')); + const membrane = new SequentialStreamMembrane(); + membrane.pushResponse(nearCapToolResponse()); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered after the restart' }])); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.', allowedTools: 'all', maxTokens: 4_000, physicalWindowTokens: 200_000 }], + modules: [new BlobToolModule()], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + errorPolicy: { maxRetries: 3, onInferenceError: (_error, _agent, attempt) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + try { + framework.start(); + framework.pushEvent({ + type: 'external-message', source: 'test', content: [{ type: 'text', text: 'fetch it' }], metadata: {}, triggerInference: true, + } as unknown as ProcessEvent); + assert.ok(await pollUntil(() => storedText(framework).includes('Answered after the restart'), 10_000)); + assert.ok(traces.some((e) => e.type === 'inference:held'), 'the turn was held first'); + assert.ok(traces.some((e) => e.type === 'inference:stream_restarted'), 'and restarted inside the turn'); + assert.equal(membrane.calls.length, 2); + assert.ok(JSON.stringify(membrane.calls[0]).includes('[budget-hold]'), 'the first compile after the release carries the note'); + assert.ok(!JSON.stringify(membrane.calls[1]).includes('[budget-hold]'), 'the restart\'s compile does not'); + } finally { + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + }); + + it('keeps the note out of the store and out of the turn\'s routing', async () => { + await withBacklog({ fitAt: 12 }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + // A message on a channel, so the turn has a locus to keep. + await (t.framework as unknown as { handleMcplChannelIncoming: (e: unknown) => Promise }).handleMcplChannelIncoming({ + type: 'mcpl:channel-incoming', serverId: 'discord', channelId: 'discord:guild:chanA', messageId: 'm-1', + author: { id: 'U1', name: 'owner' }, content: [{ type: 'text', text: 'Are you there?' }], + timestamp: new Date().toISOString(), metadata: {}, triggerInference: true, + }); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); + assert.equal(t.of('inference:held').length, 1); + assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]')); + // Every message in the store, every metadata field included. + const everything = JSON.stringify(t.framework.getAgent('prime')!.getContextManager().queryMessages({}).messages); + assert.ok(!everything.includes('budget-hold'), 'nothing of the note is stored'); + // The refused start and the released one: both on the message's channel. + const started = t.of('inference:started'); + assert.ok(started.length >= 2); + assert.ok(started.every((e) => e.channelId === 'discord:guild:chanA'), 'the released turn keeps its locus'); + } finally { + await t.close(); + } + }); + }); + it('takes later wakes into the held turn, for one compile', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); From 452fdbbe52de0fc4b3a20e9171c6b5426dac15d3 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 14:07:24 -0400 Subject: [PATCH 04/31] fix(inference): the budget hold's check reruns once after a refusal, reads pending at the dry run's start, and every end but a stop leaves the note From Beth-2389's read of 13bd4f4 (room-203 #97114), with her probe. - The rerun could keep a fit from landing. A check set its verdict aside whenever another was asked for during its dry run, and every maintenance pass asks. With dry runs slower than the asks, every dry run was overlapped and nothing landed until the deadline. Now a check runs one dry run. It runs one more only if that one refused and a check was asked for meanwhile, and takes the second verdict. A fit stands even if it is stale: the released turn's compile decides, and holds again with the carried deadline if it refuses. A refusal the second run reaches is looked at by the next pass. - A refusal ends the hold as nothing_pending only if compression had nothing pending when that dry run began. Read after the dry run, a compression that finished meanwhile turned a stale refusal into a false nothing_pending. The new test hit this before the fix: the outcome was nothing_pending, not fit. - Every end but a stop sets the note, so a turn that fits after its hold's deadline carries how long it waited, and not an earlier release's wait. - budgetHoldCarry's comment, and the carry test's, give the reason the turn's compile can refuse where the dry run fit: content reaching it after the dry run (the turn start's deferred flush, a coalesced batch's assembly, messages stored between). It is not injections, which go in after the select. - The routing test from 41afd11 failed as written: with no channel registry the locus is null, so no start has one. It now reads the turn's routing at the released start and after the compile that carried the note (trigger channel, locus, engaged, pin, destination, arrivals): they match, and the trigger channel is the message's. Tests, 22 in budget-refusal-hold: - a fit lands within 3 s while checks come every 50 ms against 300 ms dry runs (Beth's probe); - a turn that fits after its hold's deadline carries the note; - the routing test as above. --- src/framework.ts | 83 ++++++++++++++++++++------------ test/budget-refusal-hold.test.ts | 79 +++++++++++++++++++++++++----- 2 files changed, 121 insertions(+), 41 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 80325b7d3..ed1dd3944 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1147,9 +1147,9 @@ interface BudgetHold { recheck?: ReturnType; /** A release check (a dry-run compile) is running. */ checking: boolean; - /** A check was asked for while one ran: the running one's verdict may - * predate what prompted it (a surgery, a compression pass), so it is - * set aside and the check runs again. */ + /** A check was asked for while one ran: a refusal the running one reaches + * may predate what prompted the ask (a surgery, a compression pass), so + * it runs one more dry run (checkBudgetHold). */ again: boolean; } /** Longest delay one setTimeout can represent (2^31-1 ms, about 24.8 days); @@ -1697,8 +1697,10 @@ export class AgentFramework { private budgetHoldMs = BUDGET_HOLD_MS; /** A hold that released its turn because a dry run fit, until that turn's * compile fits too: a refusal of it holds again with the first refusal's - * start and deadline, since the dry run can fit where the compile, with - * its injections, doesn't. */ + * start and deadline. The dry run can fit where the turn's compile then + * doesn't, since content can reach that compile after the dry run: the + * turn start's flush of deferred messages, a coalesced batch's assembly, + * messages stored between the check and the start. */ private budgetHoldCarry: Map = new Map(); /** A hold that ended without a fit, until its turn's compile fits or the * turn ends: that turn's refusals take the error policy's path, as before @@ -3307,10 +3309,15 @@ export class AgentFramework { * the hold's own drain kicks as all that compresses, it kicks the drain * again after an interval (budgetHoldRecheckMs). * - * One check runs at a time. A check asked for while one runs marks it to - * run again: the running dry run may predate a surgery or a pass that - * prompted the new one, so its verdict is set aside. None runs while a - * surgery holds the store; the surgery's release asks for one. + * One check runs at a time. A check asked for while one runs marks it: a + * refusal it then reaches may predate what prompted the ask (a surgery, a + * pass that compressed), so it runs one more dry run and takes that + * verdict. A fit stands even if it is stale: the released turn's own + * compile decides, and holds again with the carried deadline if it + * refuses. Only one more, so asks that come faster than a dry run takes + * can't keep a verdict from landing; a refusal the second run reaches is + * looked at again by the next pass. None runs while a surgery holds the + * store; the surgery's release asks for one. */ private async checkBudgetHold(agentName: string): Promise { const hold = this.budgetHolds?.get(agentName); @@ -3323,17 +3330,20 @@ export class AgentFramework { if (this.surgeryHold) return; hold.checking = true; let verdict: 'fit' | 'refused' | 'check_failed'; + // Whether compression was pending when the dry run that decides began: a + // refusal says nothing left to do only of a context that had none then. + // (Read after it, a compression that finished meanwhile would turn a + // stale refusal into a false nothing_pending.) + let pendingAtStart: boolean; try { - do { + hold.again = false; + pendingAtStart = this.compressionPending(agent); + verdict = await this.budgetHoldDryRun(agent); + if (verdict === 'refused' && hold.again && this.budgetHolds.get(agentName) === hold && !this.surgeryHold) { hold.again = false; - try { - await agent.compileWithInjections(undefined, undefined, { dryRun: true }); - verdict = 'fit'; - } catch (error) { - const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); - verdict = errorType === 'over_budget' || errorType === 'context_refusal' ? 'refused' : 'check_failed'; - } - } while (hold.again && this.budgetHolds.get(agentName) === hold && !this.surgeryHold); + pendingAtStart = this.compressionPending(agent); + verdict = await this.budgetHoldDryRun(agent); + } } finally { hold.checking = false; } @@ -3342,7 +3352,7 @@ export class AgentFramework { if (this.surgeryHold) return; if (verdict !== 'refused') { this.endBudgetHold(agentName, verdict); - } else if (!this.compressionPending(agent)) { + } else if (!pendingAtStart) { this.endBudgetHold(agentName, 'nothing_pending'); } else if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { hold.recheck = setTimeout(() => { @@ -3353,6 +3363,18 @@ export class AgentFramework { } } + /** One dry-run compile at the agent's live budget, as the release check + * reads it: it fits, it is refused for budget, or it fails otherwise. */ + private async budgetHoldDryRun(agent: Agent): Promise<'fit' | 'refused' | 'check_failed'> { + try { + await agent.compileWithInjections(undefined, undefined, { dryRun: true }); + return 'fit'; + } catch (error) { + const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); + return errorType === 'over_budget' || errorType === 'context_refusal' ? 'refused' : 'check_failed'; + } + } + /** * Kick the drain for a held turn (kickOverBudgetDrain), then look again. A * provider gate closed to compression would park the kick's calls there, @@ -3380,12 +3402,13 @@ export class AgentFramework { * End a budget hold. Its requests go to the queue together, for one fresh * compile, unless it ended with a stop or the agent's leaving, which take * the requests and the hold's other state with them. - * - After a fit, or a dry run that failed for another reason, the turn's - * compile carries a note saying why the turn waited (budgetHoldNotes). - * It is an injection of that compile alone, which the context manager - * adds after selecting within the budget: it can't be what refuses the - * compile, it is in the request only when the compile fit, it is never - * stored, and it moves no routing. + * - The turn's next compile carries a note saying why it waited + * (budgetHoldNotes), whatever the hold's end: one that fits after a + * deadline says how long it waited too. It is an injection of that + * compile alone, which the context manager adds after selecting within + * the budget: it can't be what refuses the compile, it is in the request + * only when the compile fit, it is never stored, and it moves no + * routing. * - After a deadline, or with nothing left to compress, the turn's * refusals take the error policy's path, as before #304 * (budgetHoldSpent). @@ -3417,13 +3440,13 @@ export class AgentFramework { : `${hold.heldRequests.length} held request(s) queued for one fresh compile`)); const agent = this.agents.get(agentName); if (outcome === 'stopped' || !agent) return; + const size = hold.budget !== undefined && hold.actual !== undefined + ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` + : ''; + this.budgetHoldNotes.set(agentName, `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + + `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`); if (outcome === 'fit' || outcome === 'check_failed') { this.budgetHoldCarry.set(agentName, { startedAt: hold.startedAt, until: hold.until }); - const size = hold.budget !== undefined && hold.actual !== undefined - ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` - : ''; - this.budgetHoldNotes.set(agentName, `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + - `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`); } else { this.budgetHoldSpent.set(agentName, { outcome, waitedMs }); } diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 58e0ae990..34321dbdf 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -60,13 +60,13 @@ const storedText = (framework: AgentFramework): string => stored(framework).map( * until then, or until `readyAt` ticks, whichever is sooner, as a strategy * reports a backlog. The maintenance loop and the drain kick tick it. A * compile decides at its start, as the context manager selects at its start; - * a dry run then waits for `dryRunGate` when one is set, and throws - * `dryRunError` once when one is set. + * a dry run then waits for `dryRunGate` and `dryRunMs` when they are set, + * and throws `dryRunError` once when one is set. */ interface Backlog { ticks: number; fitAt: number; readyAt: number; fit: boolean; dryRuns: number; compiles: number; - dryRunGate?: Promise; dryRunError?: Error; + dryRunGate?: Promise; dryRunError?: Error; dryRunMs?: number; } const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; const pending = (b: Backlog): boolean => !fits(b) && b.ticks < b.readyAt; @@ -83,6 +83,7 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr else b.compiles++; const fitsNow = fits(b); if (dryRun && b.dryRunGate) await b.dryRunGate; + if (dryRun && b.dryRunMs) await new Promise((r) => setTimeout(r, b.dryRunMs)); if (dryRun && b.dryRunError) { const error = b.dryRunError; b.dryRunError = undefined; @@ -298,8 +299,24 @@ describe('a compile refused for budget holds its turn (#304)', () => { it('keeps the note out of the store and out of the turn\'s routing', async () => { await withBacklog({ fitAt: 12 }, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); + // The routing the released turn starts with, and the routing it has + // once its compile, the one carrying the note, has been sent. + type Routing = { triggerChannel?: string; locus?: string; engaged: Set; pin?: string; destination: string | null; arrivals: unknown[] }; + const routingNow = () => { + const r = (t.framework as unknown as { turnRouting: Map }).turnRouting.get('prime'); + return r && { + triggerChannel: r.triggerChannel, locus: r.locus, engaged: [...r.engaged].sort(), pin: r.pin, + destination: r.destination, arrivals: r.arrivals.length, + }; + }; + const atStart: unknown[] = []; + const afterCompile: unknown[] = []; + t.framework.onTrace((e) => { + if (e.type === 'inference:started') atStart.push(routingNow()); + if (e.type === 'inference:completed') afterCompile.push(routingNow()); + }); try { - // A message on a channel, so the turn has a locus to keep. + // A message on a channel, so the turn has a trigger channel to keep. await (t.framework as unknown as { handleMcplChannelIncoming: (e: unknown) => Promise }).handleMcplChannelIncoming({ type: 'mcpl:channel-incoming', serverId: 'discord', channelId: 'discord:guild:chanA', messageId: 'm-1', author: { id: 'U1', name: 'owner' }, content: [{ type: 'text', text: 'Are you there?' }], @@ -307,14 +324,15 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); assert.equal(t.of('inference:held').length, 1); - assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]')); + assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]'), 'the released compile carried the note'); // Every message in the store, every metadata field included. const everything = JSON.stringify(t.framework.getAgent('prime')!.getContextManager().queryMessages({}).messages); assert.ok(!everything.includes('budget-hold'), 'nothing of the note is stored'); - // The refused start and the released one: both on the message's channel. - const started = t.of('inference:started'); - assert.ok(started.length >= 2); - assert.ok(started.every((e) => e.channelId === 'discord:guild:chanA'), 'the released turn keeps its locus'); + // The released start (the last one) and the routing after its compile. + assert.equal(afterCompile.length, 1); + const released = atStart[atStart.length - 1] as { triggerChannel?: string }; + assert.equal(released.triggerChannel, 'discord:guild:chanA', 'the released turn starts from the message\'s channel'); + assert.deepEqual(afterCompile[0], released, 'and the compile with the note leaves its routing as it was'); } finally { await t.close(); } @@ -545,6 +563,44 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('lands a fit while checks are asked for faster than a dry run takes', async () => { + // Beth-2389's probe: dry runs of 300 ms, a pass every 50 ms, each asking. + await withBacklog({ dryRunMs: 300 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 6_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await new Promise((r) => setTimeout(r, 400)); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + 'the turn runs within 3 s of the fit, not at the deadline'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + }); + + it('tells a turn that fits after its hold\'s deadline why it waited', async () => { + await withBacklog({}, async (backlog) => { + // No passes after the first, so nothing looks between the hold's first + // check and its deadline. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, holdMs: 400 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => backlog.dryRuns >= 1, 2_000)); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); + assert.match(JSON.stringify(t.membrane.calls[0]), /\[budget-hold\][^"]*so this turn waited \d+ s while compression caught up/); + assert.deepEqual(t.of('inference:exhausted'), []); + } finally { + await t.close(); + } + }); + }); + it('looks again after an undo', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); @@ -564,8 +620,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); it('holds a released turn again with its first refusal\'s start and deadline when its own compile still refuses', async () => { - // The dry run fits at once, but the turn's compile refuses twice more, - // as a compile with injections can where the dry run fits. + // The dry run fits at once, but the turn's compile refuses twice more, as + // it can when content reaches it after the dry run (the turn start's + // flush of deferred messages, a coalesced batch's assembly). const proto = ContextManager.prototype as unknown as Patchable; const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; let refusals = 0; From d8e33258f0b7b55d6b13775bcf2cacd6cd7f6d8d Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 14:47:58 -0400 Subject: [PATCH 05/31] fix(inference): a budget hold keeps a batch's turn, lets withdrawn wakes go, survives the closing-fork reaper, clears on a failed start, and reopens the gate for a continuation Greptile's review of 452fdbb on #361 found five, each checked here and each with a test. - A batch-only start that reaches its compile has had its batches assembled and their content stored. Its held request now loses the batch-only markers, keeping its subject. Marked batch-only, the released start would assemble again: if a later batch of the subject rendered empty, the start would withdraw the turn the stored content was owed. - cancelCoalescedWakes filters a budget hold's requests by subject, as it filters a provider hold's. A withdrawn wake no longer stays in the hold, where it would decide the released turn's prose mode. - The closing-fork reaper counts a budget hold as unfinished work. A closing conversation's closure turn held for budget has no stream and no pending request, so the reaper disposed it and dropped the hold. - startAgentStream's outer catch, where a start fails outside its compile (a checkpoint write, locus resolution), clears what a hold kept for the agent's name (clearBudgetHoldState), as the inner terminal path does. - A released continuation (a budget restart, a guard retry) opens the event gate again when it resumes. The refusal closed the turn at the gate, and a continuation skips the gate's start because its predecessor would have kept the gate open, so the resumed turn ran with the gate closed and its events did not buffer (budgetHoldGateClosed). Tests, 27 in budget-refusal-hold: - a batch-only start's held request is an ordinary cause, with its subject kept; - a withdrawn coalesced wake leaves the hold; - the reaper leaves a held closure turn; - a released start whose checkpoint write fails leaves no hold state; - a held budget restart resumes with the agent inferring at the gate. --- src/framework.ts | 53 ++++++++--- test/budget-refusal-hold.test.ts | 151 +++++++++++++++++++++++++++++++ 2 files changed, 193 insertions(+), 11 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index ed1dd3944..43a742901 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1706,6 +1706,9 @@ export class AgentFramework { * turn ends: that turn's refusals take the error policy's path, as before * #304, and say how long the turn was held. */ private budgetHoldSpent: Map = new Map(); + /** Agents whose released budget hold held a continuation, which opens the + * event gate again when it resumes (endBudgetHold, beginAgentTurn). */ + private budgetHoldGateClosed: Set = new Set(); /** What runs when an agent's drain kick ends (kickOverBudgetDrain). */ private overBudgetDrainAfter: Map void>> = new Map(); /** The note a released turn's compile carries, saying why the turn waited: @@ -2802,6 +2805,7 @@ export class AgentFramework { this.budgetHoldCarry?.clear(); this.budgetHoldSpent?.clear(); this.budgetHoldNotes?.clear(); + this.budgetHoldGateClosed?.clear(); // An open logical turn doesn't survive a stop: a start after a restart // on this instance opens a new one rather than continuing the old turn's // routing (openLogical). A resume still continues its turn, with its @@ -3221,9 +3225,17 @@ export class AgentFramework { if (!this.budgetHolds || this.providerAdmissionClosed) return false; if (this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; if (this.budgetHoldSpent.has(agent.name)) return false; + // A batch-only start that reached its compile had its batches assembled + // and their content stored first (beginAgentTurn): the held request is an + // ordinary cause now. Left marked batch-only, the released start would + // assemble again, find nothing new if a later batch of the subject + // renders empty, and withdraw the turn that content was owed. + const held = trigger?.coalescingBatch + ? { ...trigger, coalescingBatch: undefined, coalescingBatchSubjects: undefined } + : trigger; const existing = this.budgetHolds.get(agent.name); if (existing) { - if (trigger && !existing.heldRequests.some((r) => this.sameInferenceRequest(r, trigger))) existing.heldRequests.push(trigger); + if (held && !existing.heldRequests.some((r) => this.sameInferenceRequest(r, held))) existing.heldRequests.push(held); return true; } if (!this.compressionPending(agent)) return false; @@ -3239,7 +3251,7 @@ export class AgentFramework { const hold: BudgetHold = { startedAt, until, - heldRequests: trigger ? [trigger] : [], + heldRequests: held ? [held] : [], errorType, reason: boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS), ...(typeof figures.budget === 'number' ? { budget: figures.budget } : {}), @@ -3418,9 +3430,7 @@ export class AgentFramework { if (outcome === 'stopped') { // Whatever the agent's name carries outlives no stop and no disposal: // a later agent of the name starts with none of it. - this.budgetHoldCarry?.delete(agentName); - this.budgetHoldSpent?.delete(agentName); - this.budgetHoldNotes?.delete(agentName); + this.clearBudgetHoldState(agentName); } if (!hold) return; clearTimeout(hold.timer); @@ -3453,9 +3463,23 @@ export class AgentFramework { const requests = hold.heldRequests.length > 0 ? hold.heldRequests : [{ agentName, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now(), } satisfies InferenceRequest]; + // The refusal closed the turn at the event gate. A held continuation (a + // budget restart, a guard retry) skips the gate's start when it resumes, + // since its predecessor would have kept the gate open: this one didn't, + // so its start opens it again (beginAgentTurn). + if (requests.some((r) => resumesLogicalTurn(r.reason))) this.budgetHoldGateClosed.add(agentName); this.pendingRequests.push(...requests); } + /** What a hold keeps for an agent's name beyond the hold itself: cleared + * when the turn it was for has run or failed, and at stop and disposal. */ + private clearBudgetHoldState(agentName: string): void { + this.budgetHoldCarry?.delete(agentName); + this.budgetHoldSpent?.delete(agentName); + this.budgetHoldNotes?.delete(agentName); + this.budgetHoldGateClosed?.delete(agentName); + } + /** A refusal of a turn whose hold ended without a fit, its reason saying * how long the turn waited and why it ended. A released turn refused * after its hold's deadline passed is such a turn too. The error's name @@ -9877,11 +9901,13 @@ export class AgentFramework { // Fallback reaper: a closing fork whose closure inference never ran // (request dropped as stale, inference policy veto) would otherwise stay - // registered forever. No active stream + no pending request = done. + // registered forever. No active stream + no pending request = done. A + // closure turn held for budget is pending too, in its hold. for (const agentName of [...this.closingConversationAgents]) { if ( !this.activeStreams.has(agentName) && - !this.pendingRequests.some((r) => r.agentName === agentName) + !this.pendingRequests.some((r) => r.agentName === agentName) && + !this.budgetHolds?.has(agentName) ) { this.disposeConversationAgent(agentName); } @@ -10349,6 +10375,9 @@ export class AgentFramework { for (const cooldown of this.providerAccelerationCooldowns.values()) { cooldown.heldRequests = cooldown.heldRequests.filter((r) => r.coalescingSubject !== subject); } + for (const hold of this.budgetHolds?.values() ?? []) { + hold.heldRequests = hold.heldRequests.filter((r) => r.coalescingSubject !== subject); + } } /** @@ -11724,6 +11753,8 @@ export class AgentFramework { stopReason: 'exhausted', speech: '', error: error instanceof Error ? error.message : String(error), }, this.openLogical.get(agent.name)); } + // The start's turn has failed: what a budget hold kept for it goes too. + this.clearBudgetHoldState(agent.name); throw error; } finally { if (!tokenHandedOff && this.activeTurnTokens.get(agent.name) === turnToken) { @@ -11980,7 +12011,8 @@ export class AgentFramework { }); // A budget restart continues the same logical inference window. The // predecessor keeps EventGate liveness until its successor terminates. - if (!resumesTurn) { + // A continuation a budget hold released does open it: the hold closed it. + if (!resumesTurn || this.budgetHoldGateClosed?.delete(agent.name)) { this.eventGate?.onInferenceStarted(agent.name); } this.lastInferenceAt.set(agent.name, { ...this.lastInferenceAt.get(agent.name), startedAt: Date.now() }); @@ -12149,6 +12181,7 @@ export class AgentFramework { this.budgetHoldCarry?.delete(agent.name); this.budgetHoldSpent?.delete(agent.name); this.budgetHoldNotes?.delete(agent.name); + this.budgetHoldGateClosed?.delete(agent.name); const handle = this.driveStream( agent, stream, @@ -12238,9 +12271,7 @@ export class AgentFramework { }, routing); // The turn has failed, after a hold or not: the next refusal is a new // turn's, holds afresh, and carries no note of this one's wait. - this.budgetHoldSpent?.delete(agent.name); - this.budgetHoldCarry?.delete(agent.name); - this.budgetHoldNotes?.delete(agent.name); + this.clearBudgetHoldState(agent.name); this.emitTrace({ type: 'inference:exhausted', agentName: agent.name, diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 34321dbdf..aa29618e0 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -166,6 +166,9 @@ interface Internal { providerGateBlocked: (agentName: string) => boolean; disposeConversationAgent: (agentName: string) => void; } +type HeldRequest = { agentName: string; reason: string; coalescingBatch?: boolean; coalescingBatchSubjects?: string[]; coalescingSubject?: string }; +const holdOf = (t: { internal: Internal }) => + (t.internal.budgetHolds as unknown as Map).get('prime'); const heldRequests = (framework: AgentFramework): number | undefined => { const agents = (framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: { heldRequests: number } | null }> }).agents; @@ -722,6 +725,154 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('holds a batch-only start\'s request as an ordinary cause: its batches were already stored', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + try { + // A batch-only wake reaches its compile, which is refused. + t.internal.pendingRequests.push({ + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), + coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'], + } as unknown as { agentName: string }); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + const held = holdOf(t)!.heldRequests; + assert.equal(held.length, 1); + assert.equal(held[0]!.coalescingBatch, undefined, 'no longer batch-only, so the released start keeps its turn'); + assert.equal(held[0]!.coalescingBatchSubjects, undefined); + assert.equal(held[0]!.coalescingSubject, 'zz-subject', 'its subject stays, for a cancellation'); + } finally { + await t.close(); + } + }); + }); + + it('lets a withdrawn coalesced wake leave a hold, as it leaves a provider hold', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + t.internal.pendingRequests.push({ + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), coalescingSubject: 'zz-retracted', + } as unknown as { agentName: string }); + assert.ok(await pollUntil(() => holdOf(t)?.heldRequests.length === 2, 5_000), 'the wake joins the hold'); + (t.framework as unknown as { cancelCoalescedWakes: (subject: string) => void }).cancelCoalescedWakes('zz-retracted'); + assert.deepEqual(holdOf(t)!.heldRequests.map((r) => r.coalescingSubject), [undefined], 'the withdrawn wake is gone, the turn\'s own stays'); + } finally { + await t.close(); + } + }); + }); + + it('keeps a closing conversation\'s held closure turn from the reaper', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + const fw = t.framework as unknown as { + conversationRouter: unknown; lastConversationSweep: number; closingConversationAgents: Set; + sweepExpiredConversations: () => void; agents: Map; + }; + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + // As a fork closing with its closure turn held: no stream, no pending request. + fw.conversationRouter = { expired: () => [] }; + fw.lastConversationSweep = 0; + fw.closingConversationAgents.add('prime'); + fw.sweepExpiredConversations(); + assert.ok(fw.agents.has('prime'), 'the held closure turn is unfinished work'); + assert.ok(holdOf(t), 'and its hold stays'); + } finally { + fw.closingConversationAgents.delete('prime'); + fw.conversationRouter = undefined; + await t.close(); + } + }); + }); + + it('clears what a hold kept when the released start fails outside its compile', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + const fw = t.framework as unknown as { recordTurnCheckpoint: (name: string) => void }; + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => !t.internal.activeTurnTokens.has('prime'), 3_000)); + // The released start's checkpoint write fails, as a full disk would. + const original = fw.recordTurnCheckpoint; + let failed = 0; + fw.recordTurnCheckpoint = function (this: unknown, name: string) { + if (failed++ === 0) throw new Error('ENOSPC: no space left on device'); + return original.call(this, name); + }; + backlog.fit = true; + await t.framework.switchBranch('main'); + assert.ok(await pollUntil(() => failed > 0 && t.internal.budgetHolds.size === 0, 3_000)); + await new Promise((r) => setTimeout(r, 100)); + for (const map of [t.internal.budgetHoldCarry, t.internal.budgetHoldSpent, t.internal.budgetHoldNotes]) { + assert.equal(map.size, 0, 'nothing of the failed turn\'s hold is left for the next'); + } + } finally { + await t.close(); + } + }); + }); + + it('opens the event gate again when a held continuation resumes', async () => { + const proto = ContextManager.prototype as unknown as Patchable; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + // The turn's first compile fits, and its restart's fresh compile is + // refused once: the hold takes a context_budget_restart. + let real = 0; + proto.compile = async function (this: unknown, ...args: unknown[]) { + const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; + if (!dryRun && ++real === 2) { + throw Object.assign(new Error('Compile plan would exceed hard budget: budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 49_000 }); + } + return original.compile.apply(this, args); + }; + proto.tick = async function () {}; + proto.isReady = function () { return false; }; + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-gate-')); + const membrane = new SequentialStreamMembrane(); + membrane.pushResponse(nearCapToolResponse()); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered after the restart' }])); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.', allowedTools: 'all', maxTokens: 4_000, physicalWindowTokens: 200_000 }], + modules: [new BlobToolModule()], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + gate: { config: { policies: [], default: 'skip' } }, + errorPolicy: { maxRetries: 3, onInferenceError: (_error: Error, _agent: string, attempt: number) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const gate = (framework as unknown as { eventGate: { inferenceDiagnostics(): { inferring: string[] } } }).eventGate; + const traces: TraceEvent[] = []; + const inferringDuringResume: string[][] = []; + framework.onTrace((e) => { + traces.push(e); + // Sampled while the resumed stream runs (by inference:completed the + // gate has closed the turn again). + if (e.type === 'inference:content_block' && traces.some((x) => x.type === 'inference:hold_ended')) { + inferringDuringResume.push(gate.inferenceDiagnostics().inferring); + } + }); + try { + framework.start(); + framework.getAgent('prime')!.getContextManager().addMessage('user', [{ type: 'text', text: 'fetch it' }]); + (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ agentName: 'prime', reason: 'external-message', source: 'test', timestamp: Date.now() }); + assert.ok(await pollUntil(() => storedText(framework).includes('Answered after the restart'), 10_000)); + const held = traces.filter((e) => e.type === 'inference:held'); + assert.equal(held.length, 1, 'the restart\'s refused compile was held'); + assert.ok(inferringDuringResume.length >= 1); + assert.ok(inferringDuringResume.every((names) => names.includes('prime')), 'the resumed turn is inferring at the gate, so its events buffer'); + } finally { + await framework.stop(); + Object.assign(proto, original); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + it('drops the hold, its requests and what it left for the name, at stop and at disposal', async () => { await withBacklog({}, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); From bc3bb8e159e6f34d6c080468e38736238ea9ba6c Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Wed, 14 Oct 2026 15:08:13 -0400 Subject: [PATCH 06/31] test(inference): a held continuation's resumed turn leaves the event gate closed once it ends Beth-2389's suggestion on d8e3325 (room-203 #97370): the gate test also checks that the agent is no longer inferring at the gate once the resumed turn ends, so the reopening leaves no turn open at the gate, where it would buffer every agent's events. (The gate's inferring set makes a second open idempotent, so this checks the close, not a count.) --- test/budget-refusal-hold.test.ts | 3 +++ 1 file changed, 3 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index aa29618e0..5322054ac 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -866,6 +866,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(held.length, 1, 'the restart\'s refused compile was held'); assert.ok(inferringDuringResume.length >= 1); assert.ok(inferringDuringResume.every((names) => names.includes('prime')), 'the resumed turn is inferring at the gate, so its events buffer'); + // And closed once the turn ends: the reopening leaves no turn open at + // the gate, where it would buffer every agent's events. + assert.ok(await pollUntil(() => !gate.inferenceDiagnostics().inferring.includes('prime'), 2_000), 'the gate closes when the turn ends'); } finally { await framework.stop(); Object.assign(proto, original); From 442c5f0718b8df49ca30cc709f72813cdc3040c7 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 07:50:25 -0400 Subject: [PATCH 07/31] fix(inference): the budget hold releases on context-manager's preview, as one record per held turn From Anarchid's review of bc3bb8e on #361 (R1-1 to R1-10). R1-1: the release check took a dry compile that resolved as a fit. context-manager documents that a dry compile doesn't throw an over-budget plan at its plan check (SelectOptions.dryRun), and that previewContext reports feasibility. Just over the budget the dry compile resolved while the turn's committing compile refused, so the hold released into the refusal and held again at compile speed. With the real autobiographical strategy at such a budget, the old hold was held 66 times in a second. - The check now takes the preview's verdict on the agent's next compile, at the budget that compile plans at (Agent.nextCompileBudget, which keeps the tool-result guard's reservation). The preview is synchronous. - A strategy with no preview, or one whose preview needs adaptive resolution, gets the dry compile, whose plan check such a strategy doesn't skip. Only that path awaits, so only it keeps an in-flight field (BudgetHold.checking). The hold is one record per agent with a state (BudgetHold.state: holding, released, spent), in place of the hold map plus the carry, spent, note and gate-closed maps. - released replaces the carry: a refusal of the released turn re-enters the same record, with its first start and deadline. - spent replaces the spent mark. - The note and the continuation flag are fields on it. - The record goes when the turn's stream exists, when the turn fails (inner and outer catch), at a policy veto, and at stop and disposal (settleBudgetHold, dropBudgetHold). - The check's again/rerun and pendingAtStart machinery is gone. pending is read before the verdict, which matters only on the awaiting fallback. A signature (budgetHoldSignature: the strategy's progress snapshot, the agent's runtime settings, and a count of surgeries, undos and redos) decides when the hold looks again. - R1-10: a check is skipped while the signature is unchanged since the last refusal. - A released turn refused again isn't released again until the signature changes. That covers any check/compile mismatch: content that reaches the compile after the check, or a plan the preview can't see (a kv-unified prefix hash). - Undefined, so every check runs, for a strategy that reports no progress. The smaller findings: - R1-6: a held refusal drops the checkpoint its start recorded (dropTurnCheckpoint), so the released start's is the turn's one undo step. It is recorded after the inputs that joined the hold, so undo keeps them. - R1-5: a released or spent record whose turn the inference policy declines ends there. Its logical turn is settled and its record goes, so no note or deadline reaches a later turn. - R1-7: the refused start's own request loses its coalescing subject as well as its batch markers. Its content was stored before its compile, so no cancellation withdraws the turn, and a hold always keeps a cause. A start with no request gets a budget-hold-release request when it is held, not at release. - With the maintenance timer off, a held agent gets one interval that kicks the drain (or with the provider gate closed, only looks) every budgetHoldRecheckMs. - R1-3: BUDGET_HOLD_MS's comment and the changelog now say the bound also ends a compression that is pending but can't move, and that the operator hears of it from the context-refusal alert at each hold's start, at most once per 15 minutes. - R1-2: the changelog says a restart drops the hold, as it drops a turn in flight. Left as they are, with Ada-1017's reasons: - R1-2: in-flight work survives no restart anywhere in the framework. - R1-3: no stall rule: the alert already says it, and the bound ends it. - R1-9: one cause per turn, not a lost or misrouted one. R1-4 and R1-8 come with the merge order: #352 first, and main's context-manager floor. Tests, 34 in budget-refusal-hold: - The harness's verdict is now the preview, whose contract is to say whether the context fits. The dry compile refuses as the committing one would only on the fallback path, as a hierarchical strategy's does. - A new describe runs the check against the real autobiographical strategy, unpatched but for a stubbed tick: the band where the dry compile resolves and the plan doesn't fit, and the fallback where the preview needs adaptive resolution. - New tests cover: the skip; a re-refused turn held until new evidence; the surgery count across a straddling dry compile; one undo step; the policy veto; a cancellation that leaves the turn its cause; the first kick; and a released turn not shown as waiting. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 33 +- src/agent.ts | 13 +- src/framework.ts | 538 ++++++++++-------- src/types/trace.ts | 19 +- test/budget-refusal-hold.test.ts | 489 +++++++++++----- 5 files changed, 708 insertions(+), 384 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index e8957bbe9..ee72a693f 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -4,18 +4,33 @@ retried it into the same refusal within seconds and ended the turn, and once compression had made the context fit, nothing ran it again, so the agent stayed silent until its next wake. Now a resident's or a conversation - agent's turn is held while its context manager has compression pending. It - runs as one fresh compile once a dry-run compile fits, checked after every - maintenance pass, every drain kick the hold starts, and every surgery, undo - and redo. Later wakes join it. The hold never closes the provider gate, so - compression keeps running. It ends 30 minutes after the refusal that - started it, or at once when compression has nothing left to do, and the - turn then fails as a refused compile did, its reason saying how long it was - held. An ephemeral run, and a refusal with nothing pending, fail as before. + agent's turn is held while its context manager has compression pending. + Later wakes join it. The hold never closes the provider gate, so + compression keeps running. An ephemeral run, and a refusal with nothing + pending, fail as before. + - **When it runs:** as one fresh compile, once context-manager's preview of + that compile says it fits. A dry compile can't say so, since it doesn't + throw an over-budget plan at its plan check. A strategy with no preview + gets the dry compile. The hold looks after every maintenance pass, every + drain kick it starts, and every surgery, undo and redo, and again only + once something that could make the context fit has changed: compression's + progress, the agent's budget or runtime settings, or the store, by a + surgery. A turn released and refused again, as one is when content + reaches its compile after the check, is held again with its first + deadline, and isn't released again until one of those changes. + - **When it ends without running:** 30 minutes after the refusal that + started it, or at once when compression has nothing pending, and the turn + then fails as a refused compile did, its reason saying how long it was + held. The 30 minutes also bound a compression that is pending but can't + move, such as one whose model keeps failing. The hold lives in memory, as + a turn in flight does: a restart drops it, and the agent's next wake + compiles afresh. - **What the agent sees:** the held turn's compile carries a note saying why it waited, for that compile alone: "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min - while compression caught up. It fits now." It's never stored. + while compression caught up. It fits now." It's never stored. One held + turn is one undo step, which keeps the messages that arrived while it + waited. - **What hosts see:** the refused attempt still emits `inference:failed`, as a provider hold's does, so a module that brackets typing with it stops typing while the turn waits, and the turn's next start emits diff --git a/src/agent.ts b/src/agent.ts index e6e0129c5..e97741b66 100644 --- a/src/agent.ts +++ b/src/agent.ts @@ -17,6 +17,9 @@ import { DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, } from './tool-result-history.js'; +/** ContextManager.compile's own budget when it is given none. */ +const CONTEXT_MANAGER_DEFAULT_BUDGET: TokenBudget = { maxTokens: 100_000, reserveForResponse: 4_000 }; + export interface ActivationRequestOptions { /** The silent heartbeat tick this request answers, if any. */ silentHeartbeat?: SilentHeartbeatTick; @@ -322,11 +325,17 @@ export class Agent { const resolved = this.resolveBudget(budget); const reserve = this.toolResultGuard.pendingWireReserveTokens; if (reserve === 0) return resolved; - // Mirrors ContextManager.compile's own default when no budget is set. - const base = resolved ?? { maxTokens: 100_000, reserveForResponse: 4_000 }; + const base = resolved ?? CONTEXT_MANAGER_DEFAULT_BUDGET; return { ...base, reserveForResponse: base.reserveForResponse + reserve }; } + /** The budget the agent's next compile plans at (compileBudget), for a + * preview of that compile: context-manager's own default when the agent + * sets none, as its compile then gets. */ + nextCompileBudget(): TokenBudget { + return this.compileBudget(undefined) ?? CONTEXT_MANAGER_DEFAULT_BUDGET; + } + /** Structural compatibility keeps lightweight test/host ContextManager * doubles working while making the new capability optional at runtime. */ private getHotContextSettings(): HotContextSettingsStatus | null { diff --git a/src/framework.ts b/src/framework.ts index 43a742901..5665e5b2b 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1110,47 +1110,63 @@ const PROVIDER_ACCELERATION_JITTER_MS = 5_000; /** How long a turn whose compile was refused for budget waits for compression * to make its context fit (#304). In the incident that asked for the hold, * compression took 7 minutes from the first refusal to a fit, so 30 minutes - * leaves a large backlog room. A hold already ends at once when compression - * has nothing left to do (checkBudgetHold), so this bounds only a - * compression that keeps working and never makes the context fit. A fixed - * bound rather than an option: no host has needed another, and a stall past - * it is context-manager's readiness to report, not a deadline to tune. */ + * leaves a large backlog room. A hold ends sooner only when compression has + * nothing pending (checkBudgetHold). So this bounds a compression that keeps + * working and never makes the context fit, and one that is pending but + * can't move: a merge behind a compression hold, a compression model that + * keeps failing and being retried, the speculative cap. The operator hears + * of either from the context-refusal alert at each hold's start, at most + * once per 15 minutes. A fixed bound rather than an option: no host has + * needed another. */ const BUDGET_HOLD_MS = 30 * 60_000; -/** How a budget hold ended: its context fits now; its dry run failed for +/** How a budget hold ended: its context fits now; its check failed for * another reason, which the turn's own compile then meets; its deadline - * passed; compression has nothing left to do and it still doesn't fit; or - * the framework stopped or the agent left. */ + * passed; compression has nothing pending and it still doesn't fit; or the + * framework stopped or the agent left. */ type BudgetHoldOutcome = 'fit' | 'check_failed' | 'deadline' | 'nothing_pending' | 'stopped'; /** A wait in words: seconds under a minute, else whole minutes. */ function describeWait(ms: number): string { return ms < 60_000 ? `${Math.max(1, Math.round(ms / 1000))} s` : `${Math.round(ms / 60_000)} min`; } -/** A turn held because its compile was refused for budget (#304, - * holdForBudgetRefusal). */ +/** An agent's turn held because its compile was refused for budget (#304, + * holdForBudgetRefusal), from the first refusal until the turn's stream + * exists or the turn ends. */ interface BudgetHold { + /** holding: the turn waits, later wakes join it, and the release check + * looks. released: its requests are queued for one fresh compile, and a + * refusal of that compile holds again. spent: it ended without a fit (its + * deadline passed, or compression had nothing pending), and the turn's + * refusals take the error policy's path (afterBudgetHold). */ + state: 'holding' | 'released' | 'spent'; startedAt: number; /** The deadline: BUDGET_HOLD_MS after the first refusal. */ until: number; - /** The requests the hold took, the refused one first, for one fresh - * compile once the context fits. */ + /** While holding, the requests it took, the refused one first, for one + * fresh compile once the context fits. */ heldRequests: InferenceRequest[]; errorType: 'over_budget' | 'context_refusal'; - /** The refusal's message, bounded. */ + /** The last refusal's message, bounded. */ reason: string; - /** The refused compile's budget and size, when context-manager gave them. */ - budget?: number; - actual?: number; - /** The deadline's timer. */ - timer: ReturnType; - /** With the maintenance timer off: when the hold next compresses or looks - * again (budgetHoldRecheckMs). */ - recheck?: ReturnType; - /** A release check (a dry-run compile) is running. */ + /** The last refused compile's budget and size, when context-manager gave them. */ + budget: number | undefined; + actual: number | undefined; + /** While holding: the deadline's timer, and with the maintenance timer off + * the interval at which it kicks the drain and looks (budgetHoldRecheckMs). */ + timer?: ReturnType; + recheck?: ReturnType; + /** A release check's dry compile is running (checkBudgetHold). */ checking: boolean; - /** A check was asked for while one ran: a refusal the running one reaches - * may predate what prompted the ask (a surgery, a compression pass), so - * it runs one more dry run (checkBudgetHold). */ - again: boolean; + /** The signature (budgetHoldSignature) at the check that last refused, or + * at a refusal of the turn this hold released: the hold looks again only + * once it changes. */ + refusedAt?: string; + /** Once its wait ends: the note the turn's compile carries, saying why it waited. */ + note?: string; + /** Once its wait ends: its requests include a continuation, whose start + * opens the event gate again (beginAgentTurn). */ + gateClosed: boolean; + /** Once spent: how it ended. */ + ended?: { outcome: 'deadline' | 'nothing_pending'; waitedMs: number }; } /** Longest delay one setTimeout can represent (2^31-1 ms, about 24.8 days); * a longer timer fires almost at once, so longer waits re-arm in steps. */ @@ -1695,31 +1711,15 @@ export class AgentFramework { private budgetHolds: Map = new Map(); /** BUDGET_HOLD_MS, an instance field only so tests can shorten it. */ private budgetHoldMs = BUDGET_HOLD_MS; - /** A hold that released its turn because a dry run fit, until that turn's - * compile fits too: a refusal of it holds again with the first refusal's - * start and deadline. The dry run can fit where the turn's compile then - * doesn't, since content can reach that compile after the dry run: the - * turn start's flush of deferred messages, a coalesced batch's assembly, - * messages stored between the check and the start. */ - private budgetHoldCarry: Map = new Map(); - /** A hold that ended without a fit, until its turn's compile fits or the - * turn ends: that turn's refusals take the error policy's path, as before - * #304, and say how long the turn was held. */ - private budgetHoldSpent: Map = new Map(); - /** Agents whose released budget hold held a continuation, which opens the - * event gate again when it resumes (endBudgetHold, beginAgentTurn). */ - private budgetHoldGateClosed: Set = new Set(); /** What runs when an agent's drain kick ends (kickOverBudgetDrain). */ private overBudgetDrainAfter: Map void>> = new Map(); - /** The note a released turn's compile carries, saying why the turn waited: - * an injection of that compile alone (beginAgentTurn), set when a hold - * releases its turn and cleared once the turn's stream exists. */ - private budgetHoldNotes: Map = new Map(); - /** With the maintenance timer off, how long a held turn waits between - * compressing and looking again: the timer's own default interval, so a - * backlog that doesn't move costs a kick and a dry run every 5 s, not a - * loop. An instance field only so tests can shorten it. */ + /** With the maintenance timer off, how often a held turn kicks the drain + * and looks again: the timer's own default interval. An instance field + * only so tests can shorten it. */ private budgetHoldRecheckMs = DEFAULT_MAINTENANCE_INTERVAL_MS; + /** Surgeries, undos and redos so far: one can make a held turn's context + * fit whatever compression has done (budgetHoldSignature). */ + private storeSurgeries = 0; private providerAccelerationDefaultCooldownMs = PROVIDER_ACCELERATION_DEFAULT_COOLDOWN_MS; private providerAccelerationJitterMs = PROVIDER_ACCELERATION_JITTER_MS; private providerHoldHook: ProviderHoldHook | undefined; @@ -2801,11 +2801,7 @@ export class AgentFramework { // Nor behind a turn held for budget: its requests go with the stop, as a // provider hold's do, and nothing a hold left for its agent's name // outlives the stop. - for (const name of [...(this.budgetHolds?.keys() ?? [])]) this.endBudgetHold(name, 'stopped'); - this.budgetHoldCarry?.clear(); - this.budgetHoldSpent?.clear(); - this.budgetHoldNotes?.clear(); - this.budgetHoldGateClosed?.clear(); + for (const name of [...(this.budgetHolds?.keys() ?? [])]) this.dropBudgetHold(name); // An open logical turn doesn't survive a stop: a start after a restart // on this instance opens a new one rather than continuing the old turn's // routing (openLogical). A resume still continues its turn, with its @@ -3210,57 +3206,81 @@ export class AgentFramework { * (processInferenceRequests), and a release check (checkBudgetHold) after * every maintenance pass, every drain kick the hold starts, and every * surgery, undo and redo sends them to the queue together, for one fresh - * compile, once a dry run fits. + * compile, once context-manager says the context fits. * * Only a turn that comes through the scheduler holds: a resident's or a * conversation agent's. An ephemeral run's caller is waiting on it, so its * refusal keeps the error policy's path. So does a refusal while the * context manager has nothing pending, since compression can't change it; - * one of a turn whose hold ended without a fit (budgetHoldSpent); and one - * once stop() has closed provider admission. + * one of a turn whose hold is spent; and one once stop() has closed + * provider admission. + * + * The agent's one record of its held turn (BudgetHold) lasts from the first + * refusal until that turn's stream exists or the turn ends. A refusal of + * the turn it released holds again on the same record, with the first + * refusal's start and deadline. It lives in memory, as a turn in flight + * does: a restart drops it with its requests, and the agent's next wake + * compiles afresh. */ - private holdForBudgetRefusal(agent: Agent, error: Error, trigger?: InferenceRequest): boolean { + private holdForBudgetRefusal( + agent: Agent, + error: Error, + trigger: InferenceRequest | undefined, + /** The turn checkpoint the refused start recorded, by turn index. */ + checkpoint: number | undefined, + ): boolean { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return false; if (!this.budgetHolds || this.providerAdmissionClosed) return false; if (this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; - if (this.budgetHoldSpent.has(agent.name)) return false; - // A batch-only start that reached its compile had its batches assembled - // and their content stored first (beginAgentTurn): the held request is an - // ordinary cause now. Left marked batch-only, the released start would - // assemble again, find nothing new if a later batch of the subject - // renders empty, and withdraw the turn that content was owed. - const held = trigger?.coalescingBatch - ? { ...trigger, coalescingBatch: undefined, coalescingBatchSubjects: undefined } - : trigger; const existing = this.budgetHolds.get(agent.name); - if (existing) { - if (held && !existing.heldRequests.some((r) => this.sameInferenceRequest(r, held))) existing.heldRequests.push(held); + if (existing?.state === 'spent') return false; + // The refused start's own request is an ordinary cause now, whatever it + // was: its batches were assembled and their content stored before its + // compile (beginAgentTurn), so the turn is owed. Neither an empty later + // batch of its subject nor a cancellation of the subject withdraws it, as + // neither withdraws a turn already running. A start with no request of + // its own gets one. + const own: InferenceRequest = trigger + ? { ...trigger, coalescingSubject: undefined, coalescingBatch: undefined, coalescingBatchSubjects: undefined } + : { agentName: agent.name, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now() }; + if (existing?.state === 'holding') { + if (!existing.heldRequests.some((r) => this.sameInferenceRequest(r, own))) existing.heldRequests.push(own); return true; } if (!this.compressionPending(agent)) return false; + // One held turn is one undo step: the refused start's checkpoint goes, and + // the released start records the turn's, after the inputs that joined it. + if (checkpoint !== undefined) this.dropTurnCheckpoint(agent.name, checkpoint); const now = Date.now(); - // A turn a dry run released, refused again, keeps its first refusal's - // start and deadline (a carry past its deadline is spent already: - // afterBudgetHold). - const carried = this.budgetHoldCarry.get(agent.name); - this.budgetHoldCarry.delete(agent.name); - const startedAt = carried ? carried.startedAt : now; - const until = carried ? carried.until : now + this.budgetHoldMs; const figures = error as Error & { budget?: unknown; actual?: unknown }; - const hold: BudgetHold = { - startedAt, - until, - heldRequests: held ? [held] : [], - errorType, - reason: boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS), - ...(typeof figures.budget === 'number' ? { budget: figures.budget } : {}), - ...(typeof figures.actual === 'number' ? { actual: figures.actual } : {}), - timer: setTimeout(() => this.endBudgetHold(agent.name, 'deadline'), Math.min(Math.max(0, until - now), MAX_TIMER_DELAY_MS)), - checking: false, - again: false, + const hold: BudgetHold = existing ?? { + state: 'holding', startedAt: now, until: now + this.budgetHoldMs, heldRequests: [], + errorType, reason: '', budget: undefined, actual: undefined, checking: false, gateClosed: false, }; + hold.state = 'holding'; + hold.heldRequests = [own]; + hold.errorType = errorType; + hold.reason = boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS); + hold.budget = typeof figures.budget === 'number' ? figures.budget : undefined; + hold.actual = typeof figures.actual === 'number' ? figures.actual : undefined; + hold.note = undefined; + hold.gateClosed = false; + // The check released this turn and its compile was refused, so the two + // disagreed: content reached the compile after the check, or the compile + // planned what the preview can't see (a kv-unified prefix). The hold + // doesn't release it again on the same evidence, but once compression + // moves, the budget or settings change, or a surgery runs + // (budgetHoldSignature). + hold.refusedAt = existing ? this.budgetHoldSignature(agent) : undefined; + hold.timer = setTimeout(() => this.releaseBudgetHold(agent.name, 'deadline'), Math.min(Math.max(0, hold.until - now), MAX_TIMER_DELAY_MS)); hold.timer.unref?.(); + // With the maintenance timer off, nothing else compresses or looks again: + // the hold kicks the drain and looks, at the timer's own default interval. + if (this.maintenanceIntervalMs <= 0) { + hold.recheck = setInterval(() => this.compressForBudgetHold(agent), this.budgetHoldRecheckMs); + hold.recheck.unref?.(); + } this.budgetHolds.set(agent.name, hold); this.emitTrace({ type: 'inference:held', @@ -3269,11 +3289,11 @@ export class AgentFramework { error: hold.reason, ...(hold.budget !== undefined ? { budget: hold.budget } : {}), ...(hold.actual !== undefined ? { actual: hold.actual } : {}), - since: startedAt, - until, + since: hold.startedAt, + until: hold.until, heldRequests: hold.heldRequests.length, }); - const deadline = new Date(until).toISOString(); + const deadline = new Date(hold.until).toISOString(); console.error(`[budget-hold] agent=${agent.name} compile refused (${errorType}): ${hold.reason} \u2014 ` + `turn held until ${deadline} while compression catches up`); // The operator hears of the refusal when it happens, as at exhaustion @@ -3309,74 +3329,76 @@ export class AgentFramework { for (const name of [...this.budgetHolds.keys()]) void this.checkBudgetHold(name); } + /** A surgery, undo or redo moved the store: a held turn's context may fit + * now, whatever compression has done, so every hold looks again. */ + private checkBudgetHoldsAfterSurgery(): void { + this.storeSurgeries++; + this.checkBudgetHolds(); + } + /** - * Whether a held turn's context fits now, by a dry-run compile at the - * agent's live budget, the one its turn's compile plans at: it refuses as - * that compile would, and sends, settles and commits nothing - * (Agent.compileWithInjections). If it fits, the hold ends and its turn - * runs. If it fails for another reason, the hold ends too, and the turn's - * own compile meets that failure ('check_failed'). If it refuses and - * compression has nothing left to do, the hold ends without a fit. - * Otherwise it goes on, and with the maintenance timer off, which leaves - * the hold's own drain kicks as all that compresses, it kicks the drain - * again after an interval (budgetHoldRecheckMs). - * - * One check runs at a time. A check asked for while one runs marks it: a - * refusal it then reaches may predate what prompted the ask (a surgery, a - * pass that compressed), so it runs one more dry run and takes that - * verdict. A fit stands even if it is stale: the released turn's own - * compile decides, and holds again with the carried deadline if it - * refuses. Only one more, so asks that come faster than a dry run takes - * can't keep a verdict from landing; a refusal the second run reaches is - * looked at again by the next pass. None runs while a surgery holds the - * store; the surgery's release asks for one. + * Whether a held turn's context fits now, by context-manager's verdict on + * the agent's next compile (budgetHoldPreview, else budgetHoldDryRun). If + * it fits, the hold ends and its turn runs. If the verdict failed for + * another reason, the hold ends too, and the turn's own compile meets that + * failure ('check_failed'). If it refuses and compression had nothing + * pending, the hold ends without a fit. Otherwise it goes on, and doesn't + * look again until compression moves, the budget or settings change, or a + * surgery runs (BudgetHold.refusedAt). None runs while a surgery holds the + * store; the surgery's release asks. */ private async checkBudgetHold(agentName: string): Promise { const hold = this.budgetHolds?.get(agentName); const agent = this.agents.get(agentName); - if (!hold || !agent) return; - if (hold.checking) { - hold.again = true; - return; - } - if (this.surgeryHold) return; - hold.checking = true; - let verdict: 'fit' | 'refused' | 'check_failed'; - // Whether compression was pending when the dry run that decides began: a - // refusal says nothing left to do only of a context that had none then. - // (Read after it, a compression that finished meanwhile would turn a - // stale refusal into a false nothing_pending.) - let pendingAtStart: boolean; - try { - hold.again = false; - pendingAtStart = this.compressionPending(agent); - verdict = await this.budgetHoldDryRun(agent); - if (verdict === 'refused' && hold.again && this.budgetHolds.get(agentName) === hold && !this.surgeryHold) { - hold.again = false; - pendingAtStart = this.compressionPending(agent); + if (hold?.state !== 'holding' || !agent || this.surgeryHold) return; + const signature = this.budgetHoldSignature(agent); + if (signature !== undefined && signature === hold.refusedAt) return; + // Read before the verdict: a compression that ends while a dry compile + // runs would otherwise make that compile's refusal look final. + const pending = this.compressionPending(agent); + let verdict = this.budgetHoldPreview(agent); + if (verdict === undefined) { + // Only the dry compile awaits, so only it can overlap another check. + if (hold.checking) return; + hold.checking = true; + try { verdict = await this.budgetHoldDryRun(agent); + } finally { + hold.checking = false; } - } finally { - hold.checking = false; - } - if (this.budgetHolds.get(agentName) !== hold || this.agents.get(agentName) !== agent) return; - // A surgery began while the last dry run ran: its release looks again. - if (this.surgeryHold) return; - if (verdict !== 'refused') { - this.endBudgetHold(agentName, verdict); - } else if (!pendingAtStart) { - this.endBudgetHold(agentName, 'nothing_pending'); - } else if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { - hold.recheck = setTimeout(() => { - hold.recheck = undefined; - this.compressForBudgetHold(agent); - }, this.budgetHoldRecheckMs); - hold.recheck.unref?.(); + if (this.budgetHolds.get(agentName) !== hold || hold.state !== 'holding') return; + if (this.agents.get(agentName) !== agent || this.surgeryHold) return; } + if (verdict !== 'refused') this.releaseBudgetHold(agentName, verdict); + else if (!pending) this.releaseBudgetHold(agentName, 'nothing_pending'); + else hold.refusedAt = signature; } - /** One dry-run compile at the agent's live budget, as the release check - * reads it: it fits, it is refused for budget, or it fails otherwise. */ + /** + * Context-manager's verdict on a held turn's next compile, at the budget it + * plans at (Agent.nextCompileBudget): its preview's, which reports an + * over-budget plan as not fitting. A dry compile can't give that verdict, + * since it doesn't throw an over-budget plan at the plan check + * (SelectOptions.dryRun in context-manager). Undefined for a strategy with + * no preview, and for one whose preview needs adaptive resolution: the + * dry compile decides there (budgetHoldDryRun). + */ + private budgetHoldPreview(agent: Agent): 'fit' | 'refused' | 'check_failed' | undefined { + const cm = agent.getContextManager() as unknown as { + previewContext?: (budget: TokenBudget) => { fits?: unknown } | null; + }; + if (typeof cm.previewContext !== 'function') return undefined; + try { + const preview = cm.previewContext(agent.nextCompileBudget()); + if (!preview) return undefined; + return preview.fits === true ? 'fit' : 'refused'; + } catch (error) { + return error instanceof Error && error.message.includes('requires adaptiveResolution') ? undefined : 'check_failed'; + } + } + + /** The verdict of a strategy with no preview: a dry compile, whose plan + * check such a strategy doesn't skip (a hierarchical one's doesn't). */ private async budgetHoldDryRun(agent: Agent): Promise<'fit' | 'refused' | 'check_failed'> { try { await agent.compileWithInjections(undefined, undefined, { dryRun: true }); @@ -3387,55 +3409,56 @@ export class AgentFramework { } } + /** What can make a held turn's context fit: compression's progress, the + * agent's runtime settings (its budget, and the tail and pace that shape + * the plan), and a surgery on the store. A check that refused isn't + * repeated until one of them changes. Undefined when the strategy reports + * no progress, so that every check runs. */ + private budgetHoldSignature(agent: Agent): string | undefined { + try { + const progress = this.contextProgress(agent.getContextManager()); + return progress === undefined + ? undefined + : JSON.stringify([progress, agent.getRuntimeSettings(), this.storeSurgeries]); + } catch { + return undefined; + } + } + /** * Kick the drain for a held turn (kickOverBudgetDrain), then look again. A * provider gate closed to compression would park the kick's calls there, - * where a surgery withdraws them (#352): instead, with the maintenance timer - * off, the hold looks again after an interval; with it on, the next pass's - * check does. + * where a surgery withdraws them (#352): instead it only looks, and the + * next pass or interval kicks. */ private compressForBudgetHold(agent: Agent): void { - const hold = this.budgetHolds?.get(agent.name); - if (!hold || this.agents.get(agent.name) !== agent) return; + if (this.budgetHolds?.get(agent.name)?.state !== 'holding' || this.agents.get(agent.name) !== agent) return; if (this.providerGateBlocked(agent.name)) { - if (this.maintenanceIntervalMs <= 0 && !hold.recheck) { - hold.recheck = setTimeout(() => { - hold.recheck = undefined; - void this.checkBudgetHold(agent.name); - }, this.budgetHoldRecheckMs); - hold.recheck.unref?.(); - } + void this.checkBudgetHold(agent.name); return; } this.kickOverBudgetDrain(agent.name, agent, () => void this.checkBudgetHold(agent.name)); } /** - * End a budget hold. Its requests go to the queue together, for one fresh - * compile, unless it ended with a stop or the agent's leaving, which take - * the requests and the hold's other state with them. + * End a held turn's wait: its requests go to the queue together, for one + * fresh compile, and the record stays until that turn's stream exists or + * the turn ends (settleBudgetHold). * - The turn's next compile carries a note saying why it waited - * (budgetHoldNotes), whatever the hold's end: one that fits after a - * deadline says how long it waited too. It is an injection of that - * compile alone, which the context manager adds after selecting within - * the budget: it can't be what refuses the compile, it is in the request - * only when the compile fit, it is never stored, and it moves no - * routing. - * - After a deadline, or with nothing left to compress, the turn's - * refusals take the error policy's path, as before #304 - * (budgetHoldSpent). + * (BudgetHold.note). It is an injection of that compile alone, which the + * context manager adds after selecting within the budget: it can't be + * what refuses the compile, it is in the request only when the compile + * fit, it is never stored, and it moves no routing. + * - After a fit, or a check that failed otherwise, the turn is released: a + * refusal of its compile holds again, with the same start and deadline. + * - After the deadline, or with nothing pending, the hold is spent: the + * turn's refusals take the error policy's path, as before #304, and say + * how long the turn was held (afterBudgetHold). */ - private endBudgetHold(agentName: string, outcome: BudgetHoldOutcome): void { + private releaseBudgetHold(agentName: string, outcome: Exclude): void { const hold = this.budgetHolds?.get(agentName); - if (outcome === 'stopped') { - // Whatever the agent's name carries outlives no stop and no disposal: - // a later agent of the name starts with none of it. - this.clearBudgetHoldState(agentName); - } - if (!hold) return; - clearTimeout(hold.timer); - if (hold.recheck) clearTimeout(hold.recheck); - this.budgetHolds.delete(agentName); + if (hold?.state !== 'holding') return; + this.clearBudgetHoldTimers(hold); const waitedMs = Date.now() - hold.startedAt; this.emitTrace({ type: 'inference:hold_ended', @@ -3445,59 +3468,77 @@ export class AgentFramework { heldRequests: hold.heldRequests.length, }); console.error(`[budget-hold] agent=${agentName} hold ended (${outcome}) after ${waitedMs}ms; ` + - (outcome === 'stopped' - ? `${hold.heldRequests.length} held request(s) dropped with it` - : `${hold.heldRequests.length} held request(s) queued for one fresh compile`)); - const agent = this.agents.get(agentName); - if (outcome === 'stopped' || !agent) return; - const size = hold.budget !== undefined && hold.actual !== undefined - ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` - : ''; - this.budgetHoldNotes.set(agentName, `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + - `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`); + `${hold.heldRequests.length} held request(s) queued for one fresh compile`); if (outcome === 'fit' || outcome === 'check_failed') { - this.budgetHoldCarry.set(agentName, { startedAt: hold.startedAt, until: hold.until }); + hold.state = 'released'; } else { - this.budgetHoldSpent.set(agentName, { outcome, waitedMs }); + hold.state = 'spent'; + hold.ended = { outcome, waitedMs }; } - const requests = hold.heldRequests.length > 0 ? hold.heldRequests : [{ - agentName, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now(), - } satisfies InferenceRequest]; + const size = hold.budget !== undefined && hold.actual !== undefined + ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` + : ''; + hold.note = `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + + `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`; // The refusal closed the turn at the event gate. A held continuation (a // budget restart, a guard retry) skips the gate's start when it resumes, // since its predecessor would have kept the gate open: this one didn't, // so its start opens it again (beginAgentTurn). - if (requests.some((r) => resumesLogicalTurn(r.reason))) this.budgetHoldGateClosed.add(agentName); - this.pendingRequests.push(...requests); + hold.gateClosed = hold.heldRequests.some((r) => resumesLogicalTurn(r.reason)); + this.pendingRequests.push(...hold.heldRequests); + hold.heldRequests = []; + } + + /** A stop, or the agent's leaving: its held turn goes, with its requests, + * as a provider hold's do, and nothing of it outlives the agent. */ + private dropBudgetHold(agentName: string): void { + const hold = this.budgetHolds?.get(agentName); + if (!hold) return; + this.budgetHolds.delete(agentName); + if (hold.state !== 'holding') return; + this.clearBudgetHoldTimers(hold); + const waitedMs = Date.now() - hold.startedAt; + this.emitTrace({ + type: 'inference:hold_ended', + agentName, + outcome: 'stopped', + waitedMs, + heldRequests: hold.heldRequests.length, + }); + console.error(`[budget-hold] agent=${agentName} hold ended (stopped) after ${waitedMs}ms; ` + + `${hold.heldRequests.length} held request(s) dropped with it`); } - /** What a hold keeps for an agent's name beyond the hold itself: cleared - * when the turn it was for has run or failed, and at stop and disposal. */ - private clearBudgetHoldState(agentName: string): void { - this.budgetHoldCarry?.delete(agentName); - this.budgetHoldSpent?.delete(agentName); - this.budgetHoldNotes?.delete(agentName); - this.budgetHoldGateClosed?.delete(agentName); + /** The turn a released or spent hold was for has its stream, or has ended: + * the record goes. A holding record is a turn still waiting, and stays. */ + private settleBudgetHold(agentName: string): void { + if (this.budgetHolds?.get(agentName)?.state !== 'holding') this.budgetHolds?.delete(agentName); } - /** A refusal of a turn whose hold ended without a fit, its reason saying - * how long the turn waited and why it ended. A released turn refused - * after its hold's deadline passed is such a turn too. The error's name - * is kept, so it still classifies as the refusal it is. */ + private clearBudgetHoldTimers(hold: BudgetHold): void { + clearTimeout(hold.timer); + clearInterval(hold.recheck); + hold.timer = undefined; + hold.recheck = undefined; + } + + /** A refusal of a turn whose hold is spent, its reason saying how long the + * turn waited and why it ended. A released turn refused after its hold's + * deadline passed is such a turn too. The error's name is kept, so it + * still classifies as the refusal it is. */ private afterBudgetHold(agentName: string, error: Error): Error { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return error; - const carried = this.budgetHoldCarry?.get(agentName); - if (carried && carried.until <= Date.now()) { - this.budgetHoldCarry.delete(agentName); - this.budgetHoldSpent.set(agentName, { outcome: 'deadline', waitedMs: Date.now() - carried.startedAt }); - } - const spent = this.budgetHoldSpent?.get(agentName); - if (!spent) return error; - const why = spent.outcome === 'deadline' + const hold = this.budgetHolds?.get(agentName); + if (hold?.state === 'released' && hold.until <= Date.now()) { + hold.state = 'spent'; + hold.ended = { outcome: 'deadline', waitedMs: Date.now() - hold.startedAt }; + } + if (hold?.state !== 'spent' || !hold.ended) return error; + const why = hold.ended.outcome === 'deadline' ? `the hold's ${describeWait(this.budgetHoldMs)} bound passed` : 'compression had nothing left to do'; - const held = new Error(`${error.message} (the turn was held ${describeWait(spent.waitedMs)} while compression ran; ${why} and it still didn't fit)`); + const held = new Error(`${error.message} (the turn was held ${describeWait(hold.ended.waitedMs)} while compression ran; ${why} and it still didn't fit)`); held.name = error.name; if (error.stack !== undefined) held.stack = error.stack; return held; @@ -7847,7 +7888,7 @@ export class AgentFramework { result: { sourceBranch: currentBranch.name, targetBranch: undoBranchName }, }); // The store moved: a held turn's context may fit now (checkBudgetHold). - this.checkBudgetHolds(); + this.checkBudgetHoldsAfterSurgery(); return { undone: true, @@ -7911,7 +7952,7 @@ export class AgentFramework { result: { sourceBranch: currentBranch.name, targetBranch: branchName }, }); // The store moved: a held turn's context may fit now (checkBudgetHold). - this.checkBudgetHolds(); + this.checkBudgetHoldsAfterSurgery(); return { redone: true, @@ -8048,7 +8089,7 @@ export class AgentFramework { // The surgery may have moved or rewritten the store under a held turn: // the history now may fit, or may not. Look again rather than wait for // the next maintenance pass. - this.checkBudgetHolds(); + this.checkBudgetHoldsAfterSurgery(); }; } @@ -8586,7 +8627,7 @@ export class AgentFramework { // Turn checkpoint internals // ========================================================================== - private recordTurnCheckpoint(agentName: string): void { + private recordTurnCheckpoint(agentName: string): number { const turnIndex = this.turnCounters.get(agentName) ?? 0; this.turnCounters.set(agentName, turnIndex + 1); @@ -8607,6 +8648,16 @@ export class AgentFramework { } this.saveTurnCheckpoints(agentName, checkpoints); + return turnIndex; + } + + /** Drop the agent's newest turn checkpoint, if it is the one recorded for + * this turn index (holdForBudgetRefusal). */ + private dropTurnCheckpoint(agentName: string, turnIndex: number): void { + const checkpoints = this.getTurnCheckpoints(agentName); + if (checkpoints.at(-1)?.turnIndex !== turnIndex) return; + checkpoints.pop(); + this.saveTurnCheckpoints(agentName, checkpoints); } /** @@ -9821,7 +9872,7 @@ export class AgentFramework { `${cooldown.heldRequests.length} held request(s) dropped with it (the recorded wait stays)`); } // So does a budget hold: its requests are for an agent that is gone. - this.endBudgetHold(agentName, 'stopped'); + this.dropBudgetHold(agentName); this.emitTrace({ type: 'mcpl:conversation-disposed', agentName, @@ -10701,7 +10752,7 @@ export class AgentFramework { // its release check queues (#304). No compile while it lasts: the // context hasn't changed but by compression, which the check watches. const budgetHold = this.budgetHolds?.get(agentName); - if (budgetHold) { + if (budgetHold?.state === 'holding') { for (const req of requests) { if (!budgetHold.heldRequests.some((r) => this.sameInferenceRequest(r, req))) budgetHold.heldRequests.push(req); } @@ -10856,6 +10907,16 @@ export class AgentFramework { // A context-budget restart inherits the predecessor's EventGate liveness. // If policy drops the queued successor, no driveStream remains to release it. if (budgetRestart) this.eventGate?.onInferenceEnded(agentName); + // A turn a budget hold released, declined now, won't run: its logical + // turn, open since the refusal, ends here, and the hold's record goes + // with it, so nothing of it reaches the agent's next turn. + const heldTurn = this.budgetHolds?.get(agentName); + if (heldTurn && heldTurn.state !== 'holding') { + this.budgetHolds.delete(agentName); + this.settleAgent(agentName, { + stopReason: 'exhausted', speech: '', error: 'the inference policy declined a turn held for budget', + }, this.openLogical.get(agentName)); + } const gate = this.providerGates.get(agentName); if (gate?.primaryPending && this.providerAccelerationRecoveries.has(agentName)) { gate.primaryPending = false; @@ -11753,8 +11814,8 @@ export class AgentFramework { stopReason: 'exhausted', speech: '', error: error instanceof Error ? error.message : String(error), }, this.openLogical.get(agent.name)); } - // The start's turn has failed: what a budget hold kept for it goes too. - this.clearBudgetHoldState(agent.name); + // The start's turn has failed: a budget hold's record of it goes too. + this.settleBudgetHold(agent.name); throw error; } finally { if (!tokenHandedOff && this.activeTurnTokens.get(agent.name) === turnToken) { @@ -11894,8 +11955,9 @@ export class AgentFramework { } // Record turn checkpoint before inference (only on first attempt, not retries) + let checkpoint: number | undefined; if (attempt === 0 && trigger?.reason !== 'tool_result_guard_retry') { - this.recordTurnCheckpoint(agent.name); + checkpoint = this.recordTurnCheckpoint(agent.name); this.redoStacks.delete(agent.name); // new work invalidates redo } @@ -12012,7 +12074,13 @@ export class AgentFramework { // A budget restart continues the same logical inference window. The // predecessor keeps EventGate liveness until its successor terminates. // A continuation a budget hold released does open it: the hold closed it. - if (!resumesTurn || this.budgetHoldGateClosed?.delete(agent.name)) { + const releasedHold = this.budgetHolds?.get(agent.name); + let reopensGate = false; + if (releasedHold && releasedHold.state !== 'holding' && releasedHold.gateClosed) { + releasedHold.gateClosed = false; + reopensGate = true; + } + if (!resumesTurn || reopensGate) { this.eventGate?.onInferenceStarted(agent.name); } this.lastInferenceAt.set(agent.name, { ...this.lastInferenceAt.get(agent.name), startedAt: Date.now() }); @@ -12081,7 +12149,8 @@ export class AgentFramework { // Injections are added after the context manager selects within the // budget, so the note can't be what refuses this compile, and it is in // the request only when the compile fit. It is never stored. - const holdNote = this.budgetHoldNotes?.get(agent.name); + const heldTurn = this.budgetHolds?.get(agent.name); + const holdNote = heldTurn && heldTurn.state !== 'holding' ? heldTurn.note : undefined; if (holdNote !== undefined) { const noteInjection: ContextInjection = { namespace: 'framework:budget-hold', @@ -12178,10 +12247,7 @@ export class AgentFramework { // The turn's compile fit and its stream exists: a hold that released // it, or that ended without a fit, is over, and its note was carried. - this.budgetHoldCarry?.delete(agent.name); - this.budgetHoldSpent?.delete(agent.name); - this.budgetHoldNotes?.delete(agent.name); - this.budgetHoldGateClosed?.delete(agent.name); + this.settleBudgetHold(agent.name); const handle = this.driveStream( agent, stream, @@ -12212,7 +12278,7 @@ export class AgentFramework { }); agent.reset(); - if (this.holdForBudgetRefusal(agent, err, trigger)) { + if (this.holdForBudgetRefusal(agent, err, trigger, checkpoint)) { // A compile refused for budget waits for compression, not the error // policy (#304): the hold's release check owns the later compile. The // logical turn stays open, settled by that compile, and this start's @@ -12271,7 +12337,7 @@ export class AgentFramework { }, routing); // The turn has failed, after a hold or not: the next refusal is a new // turn's, holds afresh, and carries no note of this one's wait. - this.clearBudgetHoldState(agent.name); + this.settleBudgetHold(agent.name); this.emitTrace({ type: 'inference:exhausted', agentName: agent.name, @@ -15908,7 +15974,7 @@ export class AgentFramework { lastInference: this.lastInferenceAt.get(name) ?? null, budgetHold: (() => { const hold = this.budgetHolds?.get(name); - return hold === undefined ? null : { + return hold?.state !== 'holding' ? null : { since: hold.startedAt, until: hold.until, heldRequests: hold.heldRequests.length, diff --git a/src/types/trace.ts b/src/types/trace.ts index 129af0524..8b0134a5e 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -123,15 +123,16 @@ export type TraceEvent = type: 'inference:hold_ended'; agentName: string; /** - * 'fit': a dry-run compile fits, and the held requests are queued for - * one fresh compile. 'check_failed': the dry run failed for a reason - * other than the budget, and the requests are queued for the turn's - * own compile to meet it. 'deadline': the 30 minutes passed. - * 'nothing_pending': compression has nothing left to do and the context - * still doesn't fit. After those two the requests are queued too, and - * the turn fails as a refused compile did before the hold. 'stopped': - * the framework stopped or the agent left, and the requests went with - * it. + * 'fit': context-manager's preview of the turn's next compile says it + * fits (a dry compile does, for a strategy with no preview), and the + * held requests are queued for one fresh compile. 'check_failed': that + * check failed for a reason other than the budget, and the requests are + * queued for the turn's own compile to meet it. 'deadline': the 30 + * minutes passed. 'nothing_pending': compression has nothing pending + * and the context still doesn't fit. After those two the requests are + * queued too, and the turn fails as a refused compile did before the + * hold. 'stopped': the framework stopped or the agent left, and the + * requests went with it. */ outcome: 'fit' | 'check_failed' | 'deadline' | 'nothing_pending' | 'stopped'; /** Since the hold started (`inference:held`'s `since`). */ diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 5322054ac..f09c47bb4 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -5,9 +5,11 @@ * call. Before the hold, the error policy retried it into the same refusal * within seconds and the turn ended; compression then freed the space, but * nothing ran the turn again, so the resident stayed silent until its next - * wake. Now the turn waits while compression runs, and runs once a dry-run - * compile fits: later wakes join it, and a deadline, or compression with - * nothing left to do, ends it as a refused compile ended before. + * wake. Now the turn waits while compression runs, and runs once + * context-manager's preview says the context fits: later wakes join it, and a + * deadline, or compression with nothing pending, ends it as a refused compile + * ended before. The last describe runs the release check against the real + * autobiographical strategy. */ import { describe, it } from 'node:test'; @@ -15,8 +17,8 @@ import assert from 'node:assert/strict'; import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; -import { ContextManager } from '@animalabs/context-manager'; -import type { EventResponse, Module, ProcessEvent, ToolDefinition, ToolResult, TraceEvent } from '../src/index.js'; +import { AutobiographicalStrategy, ContextManager, PassthroughStrategy } from '@animalabs/context-manager'; +import type { EventResponse, InferencePolicy, Module, ProcessEvent, ToolDefinition, ToolResult, TraceEvent } from '../src/index.js'; import { AgentFramework } from '../src/index.js'; import type { NormalizedRequest, YieldingStream } from '@animalabs/membrane'; import { createMockResponse, MockMembrane, MockYieldingStream } from './helpers/mock-membrane.js'; @@ -55,54 +57,90 @@ function stored(framework: AgentFramework): Array<{ text: string; metadata: Reco const storedText = (framework: AgentFramework): string => stored(framework).map((m) => m.text).join('\n'); /** - * The context manager's compression backlog: every compile is refused until - * `fitAt` ticks have run (or `fit` is set), and pending work is reported - * until then, or until `readyAt` ticks, whichever is sooner, as a strategy - * reports a backlog. The maintenance loop and the drain kick tick it. A - * compile decides at its start, as the context manager selects at its start; - * a dry run then waits for `dryRunGate` and `dryRunMs` when they are set, - * and throws `dryRunError` once when one is set. + * The context manager's compression backlog: the context fits once `fitAt` + * ticks have run (or `fit` is set), and pending work is reported until then, + * or until `readyAt` ticks, whichever is sooner, as a strategy reports a + * backlog (`alwaysPending` keeps it reported). The maintenance loop and the + * drain kick tick it. The strategy's progress snapshot moves with each tick, + * unless `stalled`, and when the context comes to fit, as a real one's does + * when the compression that makes it fit lands. + * - The release check's verdict is the context manager's preview, whose + * contract is to say whether the context fits: here it says `fits`, and + * throws `previewError` once when one is set. + * - `previewless` is a strategy with no preview: the check falls back to a + * dry compile, which refuses as the committing one would (as a + * hierarchical strategy's does), decided at its start as a select decides, + * after `dryRunGate` and `dryRunMs` when they are set. + * - A committing compile is refused while the context doesn't fit, and + * `refuseCompiles` more times whatever it does, as one is when content + * reaches it after the check; `slowCompile` makes the nth wait first. + * The last describe runs the real strategy instead. */ interface Backlog { ticks: number; fitAt: number; readyAt: number; fit: boolean; - dryRuns: number; compiles: number; - dryRunGate?: Promise; dryRunError?: Error; dryRunMs?: number; + alwaysPending: boolean; stalled: boolean; previewless: boolean; refuseCompiles: number; + previews: number; dryRuns: number; dryRunsRunning: number; mostDryRunsRunning: number; compiles: number; + previewError?: Error; dryRunGate?: Promise; dryRunMs?: number; slowCompile?: { nth: number; ms: number }; } const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; -const pending = (b: Backlog): boolean => !fits(b) && b.ticks < b.readyAt; +const pending = (b: Backlog): boolean => b.alwaysPending || (!fits(b) && b.ticks < b.readyAt); +const overBudget = (actual = 100_201) => Object.assign( + new Error('Compile plan would exceed hard budget: head=41200 tail=8100 middle=62000 budget=48568'), + { name: 'OverBudgetError', budget: 48_568, actual }, +); -type Patchable = Record<'compile' | 'tick' | 'isReady', (...args: unknown[]) => unknown>; +type Patchable = Record<'compile' | 'tick' | 'isReady' | 'previewContext', (...args: unknown[]) => unknown>; async function withBacklog(backlog: Partial, run: (b: Backlog) => Promise): Promise { - const b: Backlog = { ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, dryRuns: 0, compiles: 0, ...backlog }; + const b: Backlog = { + ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, + alwaysPending: false, stalled: false, previewless: false, refuseCompiles: 0, + previews: 0, dryRuns: 0, dryRunsRunning: 0, mostDryRunsRunning: 0, compiles: 0, + ...backlog, + }; const proto = ContextManager.prototype as unknown as Patchable; - const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady, previewContext: proto.previewContext }; + const passthrough = PassthroughStrategy.prototype as unknown as { getProgressSnapshot?: () => unknown }; proto.compile = async function (this: unknown, ...args: unknown[]) { - const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; - if (dryRun) b.dryRuns++; - else b.compiles++; - const fitsNow = fits(b); - if (dryRun && b.dryRunGate) await b.dryRunGate; - if (dryRun && b.dryRunMs) await new Promise((r) => setTimeout(r, b.dryRunMs)); - if (dryRun && b.dryRunError) { - const error = b.dryRunError; - b.dryRunError = undefined; - throw error; + if ((args[2] as { dryRun?: boolean } | undefined)?.dryRun === true) { + b.dryRuns++; + b.mostDryRunsRunning = Math.max(b.mostDryRunsRunning, ++b.dryRunsRunning); + try { + const fitsNow = fits(b); + if (b.dryRunGate) await b.dryRunGate; + if (b.dryRunMs) await new Promise((r) => setTimeout(r, b.dryRunMs)); + if (!fitsNow) throw overBudget(); + } finally { + b.dryRunsRunning--; + } + return original.compile.apply(this, args); } - if (!fitsNow) { - throw Object.assign( - new Error('Compile plan would exceed hard budget: head=41200 tail=8100 middle=62000 budget=48568'), - { name: 'OverBudgetError', budget: 48_568, actual: 100_201 }, - ); + b.compiles++; + if (b.slowCompile?.nth === b.compiles) await new Promise((r) => setTimeout(r, b.slowCompile!.ms)); + if (!fits(b)) throw overBudget(); + if (b.refuseCompiles > 0) { + b.refuseCompiles--; + throw overBudget(49_000); } return original.compile.apply(this, args); }; + proto.previewContext = function () { + b.previews++; + if (b.previewError) { + const error = b.previewError; + b.previewError = undefined; + throw error; + } + return b.previewless ? null : { fits: fits(b) }; + }; proto.tick = async function () { b.ticks++; }; proto.isReady = function () { return !pending(b); }; + passthrough.getProgressSnapshot = function () { return { chunksCompressed: b.stalled ? 0 : b.ticks, fits: fits(b) }; }; try { return await run(b); } finally { Object.assign(proto, original); + delete passthrough.getProgressSnapshot; } } @@ -158,9 +196,6 @@ interface Internal { budgetHoldMs: number; budgetHoldRecheckMs: number; budgetHolds: Map; - budgetHoldCarry: Map; - budgetHoldSpent: Map; - budgetHoldNotes: Map; activeTurnTokens: Map; pendingRequests: Array<{ agentName: string }>; providerGateBlocked: (agentName: string) => boolean; @@ -168,14 +203,16 @@ interface Internal { } type HeldRequest = { agentName: string; reason: string; coalescingBatch?: boolean; coalescingBatchSubjects?: string[]; coalescingSubject?: string }; const holdOf = (t: { internal: Internal }) => - (t.internal.budgetHolds as unknown as Map).get('prime'); + (t.internal.budgetHolds as unknown as Map).get('prime'); const heldRequests = (framework: AgentFramework): number | undefined => { const agents = (framework.healthSnapshot() as { agents: Array<{ name: string; budgetHold: { heldRequests: number } | null }> }).agents; return agents.find((a) => a.name === 'prime')?.budgetHold?.heldRequests; }; -async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: number; recheckMs?: number; responses?: number }) { +async function residentTurn(options: { + maintenanceIntervalMs: number; holdMs?: number; recheckMs?: number; responses?: number; inferencePolicy?: InferencePolicy; +}) { const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-')); const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); @@ -189,6 +226,7 @@ async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: n modules: [new InboxModule()], syncIntervalMs: 0, maintenanceIntervalMs: options.maintenanceIntervalMs, + ...(options.inferencePolicy ? { inferencePolicy: options.inferencePolicy } : {}), // The default policy's three retries, without its 1, 2 and 4 second waits. errorPolicy: { maxRetries: 3, onInferenceError: (_error, _agent, attempt) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, }); @@ -440,20 +478,20 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('runs again when a dry run fails for another reason, and the turn\'s own compile meets it', async () => { + it('runs the turn when the check fails for another reason, and the turn\'s own compile meets it', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - // The context now fits, but the check's dry run fails for another + // The context now fits, but the check's preview fails for another // reason: the hold doesn't call that a fit, and lets the turn's // compile find out. - backlog.dryRunError = new Error('store read failed'); + backlog.previewError = new Error('store read failed'); backlog.fit = true; assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['check_failed']); - assert.equal(backlog.dryRunError, undefined, 'the dry run met the error'); + assert.equal(backlog.previewError, undefined, 'the preview met the error'); assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]'), 'the turn that ran says why it waited'); } finally { await t.close(); @@ -476,6 +514,21 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('kicks the drain at once when the hold starts', async () => { + // No maintenance timer, and an interval longer than the test: only the + // hold's first kick can compress. + await withBacklog({ fitAt: 8 }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 60_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + }); + it('with the maintenance timer off, a backlog that never moves costs a kick an interval, not a loop', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 100 }); @@ -485,7 +538,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { await new Promise((r) => setTimeout(r, 550)); // The first kick at once, then one an interval: about 6 in 550 ms. assert.ok(backlog.ticks <= 8 * 8, `${backlog.ticks} ticks in 550 ms`); - assert.ok(backlog.dryRuns <= 8, `${backlog.dryRuns} dry runs in 550 ms`); + assert.ok(backlog.previews <= 8, `${backlog.previews} checks in 550 ms`); assert.ok(backlog.ticks >= 16, 'it does kick again'); } finally { await t.close(); @@ -539,45 +592,50 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('sets aside a dry run that was running when a surgery asked to look again', async () => { - await withBacklog({}, async (backlog) => { - const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + it('falls back to a dry compile for a strategy with no preview, one at a time, and lands a fit', async () => { + // Beth-2389's probe: dry compiles of 300 ms, a pass every 50 ms, each + // asking. Compression finishes, and makes the context fit, while one runs. + await withBacklog({ previewless: true, dryRunMs: 300, fitAt: 40, readyAt: 40 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 6_000 }); try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - assert.ok(await pollUntil(() => backlog.ticks >= 16 && !t.internal.activeTurnTokens.has('prime'), 3_000)); - // A dry run starts, decides it doesn't fit, and is held there. - let release!: () => void; - backlog.dryRunGate = new Promise((resolve) => { release = resolve; }); - const dryRuns = backlog.dryRuns; - (t.framework as unknown as { checkBudgetHolds: () => void }).checkBudgetHolds(); - assert.ok(await pollUntil(() => backlog.dryRuns === dryRuns + 1, 2_000)); - // The context now fits, and a surgery asks to look again. - backlog.fit = true; - await t.framework.switchBranch('main'); - backlog.dryRunGate = undefined; - release(); - assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), - 'the stale verdict was set aside, and a fresh dry run released the turn'); - assert.ok(backlog.dryRuns >= dryRuns + 2); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), + 'the turn runs soon after the fit, not at the deadline'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit'], + 'a refusal decided before compression finished is not taken for nothing pending'); + assert.ok(backlog.dryRuns >= 1 && backlog.previews >= 1, 'the preview had nothing to say, and a dry compile decided'); + assert.equal(backlog.mostDryRunsRunning, 1, 'one dry compile at a time'); } finally { await t.close(); } }); }); - it('lands a fit while checks are asked for faster than a dry run takes', async () => { - // Beth-2389's probe: dry runs of 300 ms, a pass every 50 ms, each asking. - await withBacklog({ dryRunMs: 300 }, async (backlog) => { - const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 6_000 }); + it('on the fallback path, looks again at the next pass after a surgery a dry compile straddled', async () => { + // Compression makes no progress, so only a surgery changes what the + // check compares. + await withBacklog({ previewless: true, stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { t.send('Are you there?'); - assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - await new Promise((r) => setTimeout(r, 400)); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.dryRuns >= 1 && backlog.dryRunsRunning === 0, 5_000)); + // A surgery brings a check, whose dry compile decides it doesn't fit + // and is held there. + let release!: () => void; + backlog.dryRunGate = new Promise((resolve) => { release = resolve; }); + const dryRuns = backlog.dryRuns; + await t.framework.switchBranch('main'); + assert.ok(await pollUntil(() => backlog.dryRuns === dryRuns + 1, 2_000)); + // The context now fits, and a second surgery asks while that dry + // compile runs. backlog.fit = true; + await t.framework.switchBranch('main'); + backlog.dryRunGate = undefined; + release(); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), - 'the turn runs within 3 s of the fit, not at the deadline'); - assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + 'the stale refusal stood only until the next pass, which the second surgery had given something new to look at'); + assert.equal(backlog.mostDryRunsRunning, 1); } finally { await t.close(); } @@ -592,7 +650,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - assert.ok(await pollUntil(() => backlog.dryRuns >= 1, 2_000)); + assert.ok(await pollUntil(() => backlog.previews >= 1, 2_000)); backlog.fit = true; assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); @@ -605,16 +663,21 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); it('looks again after an undo', async () => { - await withBacklog({}, async (backlog) => { - const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + await withBacklog({ fit: true }, async (backlog) => { + // One maintenance pass at start, then none for an hour: only the undo + // can bring the release check. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, responses: 2 }); try { + t.send('A turn that fits'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + backlog.fit = false; t.send('Are you there?'); - assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - assert.ok(await pollUntil(() => backlog.ticks >= 16 && !t.internal.activeTurnTokens.has('prime'), 3_000)); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + assert.ok(await pollUntil(() => !t.internal.activeTurnTokens.has('prime'), 3_000)); backlog.fit = true; const undone = t.framework.undoLastTurn('prime'); - assert.equal(undone.undone, true, 'the refused start left a checkpoint to undo'); - assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), + assert.equal(undone.undone, true, 'the earlier turn left a checkpoint to undo'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered turn 2'), 3_000), 'the undo brought the release check forward'); } finally { await t.close(); @@ -622,24 +685,32 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('holds a released turn again with its first refusal\'s start and deadline when its own compile still refuses', async () => { - // The dry run fits at once, but the turn's compile refuses twice more, as - // it can when content reaches it after the dry run (the turn start's - // flush of deferred messages, a coalesced batch's assembly). - const proto = ContextManager.prototype as unknown as Patchable; - const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; - let refusals = 0; - proto.compile = async function (this: unknown, ...args: unknown[]) { - const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; - if (!dryRun && refusals < 3) { - refusals++; - throw Object.assign(new Error('Compile plan would exceed hard budget: budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 49_000 }); + it('makes one held turn one undo step, which keeps the inputs that joined it', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('First message'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + t.send('Second message, while held'); + assert.ok(await pollUntil(() => heldRequests(t.framework) === 2, 5_000)); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.equal(t.framework.getUndoRedoState('prime').undoDepth, 1, 'the refused start left no undo step of its own'); + assert.equal(t.framework.undoLastTurn('prime').undone, true); + const text = storedText(t.framework); + assert.ok(!text.includes('Answered once it fit'), 'the undo takes the reply'); + assert.ok(text.includes('First message') && text.includes('Second message, while held'), 'and keeps both inputs'); + } finally { + await t.close(); } - return original.compile.apply(this, args); - }; - proto.tick = async function () {}; - proto.isReady = function () { return false; }; - try { + }); + }); + + it('holds a released turn again with its first refusal\'s start and deadline when its own compile still refuses', async () => { + // The check says it fits at once, but the turn's compile refuses three + // times, as it can when content reaches it after the check (the turn + // start's flush of deferred messages, a coalesced batch's assembly). + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: 3 }, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { t.send('Are you there?'); @@ -652,38 +723,93 @@ describe('a compile refused for budget holds its turn (#304)', () => { } finally { await t.close(); } - } finally { - Object.assign(proto, original); - } + }); + }); + + it('does not release a turn its compile refused again until compression moves or a surgery runs', async () => { + // The check says it fits, the turn's compile is refused every time, and + // compression makes no progress. + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 2, 5_000), 'released once, refused, and held again'); + const compiles = backlog.compiles; + await new Promise((r) => setTimeout(r, 600)); + assert.equal(t.of('inference:held').length, 2, 'no release on the same evidence, pass after pass'); + assert.equal(backlog.compiles, compiles, 'and no compile refused for it'); + await t.framework.switchBranch('main'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 3, 3_000), + 'a surgery is new evidence: the hold looks, releases, and holds again'); + } finally { + await t.close(); + } + }); + }); + + it('looks again only once something that could make the context fit has changed', async () => { + await withBacklog({ stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + let previews = backlog.previews; + await new Promise((r) => setTimeout(r, 500)); + assert.equal(backlog.previews, previews, 'ten passes, and no check while nothing changed'); + // A budget for the agent changes what its next compile plans at. (The + // change previews once itself, as its preflight.) + t.framework.updateAgentRuntimeSettings('prime', { contextBudgetTokens: 200_000 }); + previews = backlog.previews; + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'it looks once the budget changes'); + previews = backlog.previews; + await new Promise((r) => setTimeout(r, 300)); + assert.equal(backlog.previews, previews); + backlog.stalled = false; + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'and once compression moves'); + } finally { + await t.close(); + } + }); }); it('fails a released turn refused after its hold\'s deadline, rather than holding it afresh', async () => { - const proto = ContextManager.prototype as unknown as Patchable; - const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; - // The dry run fits, but the released turn's compile starts only after - // the deadline (a slow queue), and refuses. - let real = 0; - proto.compile = async function (this: unknown, ...args: unknown[]) { - const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; - if (dryRun) return original.compile.apply(this, args); - if (++real === 2) await new Promise((r) => setTimeout(r, 500)); - throw Object.assign(new Error('Compile plan would exceed hard budget: budget=48568'), { name: 'OverBudgetError', budget: 48_568, actual: 49_000 }); - }; - proto.tick = async function () {}; - proto.isReady = function () { return false; }; - try { + // The check says it fits, but the released turn's compile starts only + // after the deadline (a slow queue), and refuses. + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, slowCompile: { nth: 2, ms: 500 } }, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50, holdMs: 300 }); try { t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000)); + assert.equal(heldRequests(t.framework), undefined, 'a released turn is not waiting, so the status shows no hold'); assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); assert.equal(t.of('inference:held').length, 1, 'no second hold past the first one\'s deadline'); assert.match(String(t.of('inference:exhausted')[0]!.error), /the hold's 1 s bound passed and it still didn't fit/); } finally { await t.close(); } - } finally { - Object.assign(proto, original); - } + }); + }); + + it('ends a released turn the inference policy declines: nothing of its hold reaches a later turn', async () => { + await withBacklog({}, async (backlog) => { + let decline = false; + const t = await residentTurn({ maintenanceIntervalMs: 50, inferencePolicy: { shouldInfer: () => !decline } }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + decline = true; + backlog.fit = true; + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1 && t.internal.budgetHolds.size === 0, 3_000), + 'released, declined, and its record gone'); + assert.equal(t.membrane.calls.length, 0); + decline = false; + t.send('A later turn'); + assert.ok(await pollUntil(() => t.membrane.calls.length === 1, 3_000)); + assert.ok(!JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]'), 'the later turn carries no note of the declined one\'s wait'); + } finally { + await t.close(); + } + }); }); it('leaves an ephemeral run to fail as before: its caller is waiting', async () => { @@ -725,8 +851,8 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('holds a batch-only start\'s request as an ordinary cause: its batches were already stored', async () => { - await withBacklog({}, async () => { + it('holds a batch-only start\'s request as an ordinary cause, which no cancellation withdraws', async () => { + await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); try { // A batch-only wake reaches its compile, which is refused. @@ -739,7 +865,13 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(held.length, 1); assert.equal(held[0]!.coalescingBatch, undefined, 'no longer batch-only, so the released start keeps its turn'); assert.equal(held[0]!.coalescingBatchSubjects, undefined); - assert.equal(held[0]!.coalescingSubject, 'zz-subject', 'its subject stays, for a cancellation'); + // Its content was stored before its compile: a cancellation of its + // subject takes neither the turn nor its only cause. + (t.framework as unknown as { cancelCoalescedWakes: (subject: string) => void }).cancelCoalescedWakes('zz-subject'); + assert.equal(holdOf(t)!.heldRequests.length, 1, 'the turn keeps its own cause'); + backlog.fit = true; + await t.framework.switchBranch('main'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); } finally { await t.close(); } @@ -789,7 +921,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('clears what a hold kept when the released start fails outside its compile', async () => { + it('drops the hold\'s record when the released start fails outside its compile', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); const fw = t.framework as unknown as { recordTurnCheckpoint: (name: string) => void }; @@ -806,11 +938,8 @@ describe('a compile refused for budget holds its turn (#304)', () => { }; backlog.fit = true; await t.framework.switchBranch('main'); - assert.ok(await pollUntil(() => failed > 0 && t.internal.budgetHolds.size === 0, 3_000)); - await new Promise((r) => setTimeout(r, 100)); - for (const map of [t.internal.budgetHoldCarry, t.internal.budgetHoldSpent, t.internal.budgetHoldNotes]) { - assert.equal(map.size, 0, 'nothing of the failed turn\'s hold is left for the next'); - } + assert.ok(await pollUntil(() => failed > 0 && t.internal.budgetHolds.size === 0, 3_000), + 'nothing of the failed turn\'s hold is left for the next'); } finally { await t.close(); } @@ -888,9 +1017,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); assert.deepEqual(t.internal.pendingRequests.filter((r) => r.agentName === 'prime'), [], 'the held request went with the stop'); assert.equal(t.membrane.calls.length, 0); - for (const map of [t.internal.budgetHolds, t.internal.budgetHoldCarry, t.internal.budgetHoldSpent, t.internal.budgetHoldNotes]) { - assert.equal(map.size, 0); - } + assert.equal(t.internal.budgetHolds.size, 0); } finally { if (!closed) await t.close(); } @@ -900,11 +1027,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - t.internal.budgetHoldSpent.set('prime', { outcome: 'deadline', waitedMs: 1 }); t.internal.disposeConversationAgent('prime'); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); - assert.equal(t.internal.budgetHolds.size, 0); - assert.equal(t.internal.budgetHoldSpent.size, 0, 'a later agent of the name starts with none of it'); + assert.equal(t.internal.budgetHolds.size, 0, 'a later agent of the name starts with none of it'); // Past the hold's deadline: nothing is queued for an agent that's gone. await new Promise((r) => setTimeout(r, 450)); assert.deepEqual(t.internal.pendingRequests.filter((r) => r.agentName === 'prime'), []); @@ -915,3 +1040,111 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); }); + +describe('the release check takes context-manager\'s verdict on the real strategy (#304)', () => { + /** + * A resident whose autobiographical strategy holds sixteen messages of + * mixed sizes, none compressed: compression is pending, and stays so, since + * nothing here can compress (the tick is stubbed: there is no compression + * model). The verdict's path, the preview and both compiles, is the + * strategy's own, unpatched. + */ + async function realResident(adaptiveResolution: boolean) { + const proto = ContextManager.prototype as unknown as { tick: () => Promise }; + const tick = proto.tick; + proto.tick = async () => {}; + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-real-')); + const membrane = new SequentialStreamMembrane(); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ + name: 'prime', model: 'test-model', systemPrompt: 'You are prime.', contextBudgetTokens: 400_000, maxTokens: 1_000, + strategy: new AutobiographicalStrategy({ adaptiveResolution, targetChunkTokens: 300, recentWindowTokens: 400 }), + }], + modules: [], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + errorPolicy: { maxRetries: 3, onInferenceError: (_error: Error, _agent: string, attempt: number) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + const cm = framework.getAgent('prime')!.getContextManager(); + for (let i = 0; i < 16; i++) { + cm.addMessage(i % 2 === 0 ? 'User' : 'prime', [{ type: 'text', text: `m${i} ` + 'lorem ipsum dolor sit amet '.repeat(i % 3 === 0 ? 500 : 60) }]); + } + const budget = (maxTokens: number) => ({ maxTokens, reserveForResponse: 1_000 }); + return { + framework, + membrane, + of: (type: string) => traces.filter((e) => e.type === type) as unknown as Array>, + preview: (maxTokens: number) => (cm as unknown as { + previewContext: (b: { maxTokens: number; reserveForResponse: number }) => { fits: boolean; finalTokens: number }; + }).previewContext(budget(maxTokens)), + dryCompileResolves: (maxTokens: number) => cm.compile(budget(maxTokens), undefined, { dryRun: true } as never).then(() => true, () => false), + setBudget: (tokens: number) => framework.updateAgentRuntimeSettings('prime', { contextBudgetTokens: tokens, immediate: true }, { allowInfeasible: true }), + wake: () => (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ + agentName: 'prime', reason: 'external-message', source: 'test', timestamp: Date.now(), + }), + close: async () => { + await framework.stop(); + proto.tick = tick; + rmSync(tempDir, { recursive: true, force: true }); + }, + }; + } + + it('keeps holding at a budget where a dry compile resolves but the plan does not fit', async () => { + const t = await realResident(true); + try { + // Just over the budget, the adaptive strategy's dry compile resolves: + // it doesn't throw an over-budget plan at its plan check. The preview + // says the plan doesn't fit, and the turn's committing compile refuses. + assert.equal(t.preview(400_000).fits, true); + const plan = t.preview(400_000).finalTokens; + let band: number | undefined; + for (let b = plan; b < plan * 1.05 && band === undefined; b += Math.max(1, Math.round(plan / 1_000))) { + if (await t.dryCompileResolves(b) && !t.preview(b).fits) band = b; + } + assert.ok(band !== undefined, 'there is a budget where the dry compile resolves and the plan does not fit'); + t.setBudget(band); + t.framework.start(); + t.wake(); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000), 'the turn\'s compile is refused there'); + await new Promise((r) => setTimeout(r, 1_000)); + assert.equal(t.of('inference:held').length, 1, 'held once, never released into the same refusal'); + assert.deepEqual(t.of('inference:hold_ended'), []); + assert.equal(t.membrane.calls.length, 0); + t.setBudget(400_000); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), 'a budget it fits releases it'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + + it('falls back to a dry compile where the preview needs adaptive resolution', async () => { + const t = await realResident(false); + try { + // The hierarchical strategy refuses an over-budget context in its + // preview and in a dry compile alike. A context that fits has no fold + // plan to preview, and the dry compile decides. + let fitsFrom = 10_000; + while (!(await t.dryCompileResolves(fitsFrom)) && fitsFrom < 400_000) fitsFrom += 1_000; + assert.ok(fitsFrom < 400_000); + assert.throws(() => t.preview(fitsFrom), /requires adaptiveResolution/); + t.setBudget(Math.round(fitsFrom * 0.8)); + t.framework.start(); + t.wake(); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + await new Promise((r) => setTimeout(r, 500)); + assert.deepEqual(t.of('inference:hold_ended'), [], 'it stays held while the context does not fit'); + t.setBudget(fitsFrom + 2_000); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit'], 'the dry compile said it fits'); + } finally { + await t.close(); + } + }); +}); From bfcff594229628850bcb6107e4b14518b4d919ae Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 08:23:09 -0400 Subject: [PATCH 08/31] test(inference): a redo brings a held turn's release check forward The redo's ask (checkBudgetHoldsAfterSurgery at redo) had no test of its own: with no maintenance passes after the first, an undo while a turn is held finds no fit, and a redo once it would fit releases the turn. Undoing the redo's ask now fails this test. The hold file has 35 tests. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index f09c47bb4..2a1df0424 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -685,6 +685,31 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('looks again after a redo', async () => { + await withBacklog({ fit: true }, async (backlog) => { + // No passes after the first: only the undo and the redo bring checks. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, responses: 2 }); + try { + t.send('A turn that fits'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + backlog.fit = false; + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + assert.ok(await pollUntil(() => !t.internal.activeTurnTokens.has('prime'), 3_000)); + // An undo while the turn is held, whose own check finds no fit yet ... + assert.equal(t.framework.undoLastTurn('prime').undone, true); + assert.ok(await pollUntil(() => backlog.previews >= 2, 2_000)); + // ... and a redo once it would fit. + backlog.fit = true; + assert.equal(t.framework.redo('prime').redone, true); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered turn 2'), 3_000), + 'the redo brought the release check forward'); + } finally { + await t.close(); + } + }); + }); + it('makes one held turn one undo step, which keeps the inputs that joined it', async () => { await withBacklog({}, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); From 76d4a2edcaac55608288d412cbec6b9996a36e2b Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 09:44:11 -0400 Subject: [PATCH 09/31] test(inference): the hold's drain kick stops when a surgery takes the store mid-tick, and the surgery waits for it From R1-4 of Anarchid's review of bc3bb8e on #361: the existing surgery test's ticks resolve synchronously, so nothing showed what #352's kick does when a surgery arrives while a tick is in flight. The test lets the start pass finish, then makes each tick wait on a gate. A refusal holds the turn, and the hold's kick enters its first tick, with the hold's check waiting on it. A branch switch arrives. After 200 ms it hasn't switched, since it waits for the tick in flight. A kick asked for while the surgery holds the store starts nothing and leaves no callback behind. Once the tick is let go, the kick stops there with no second tick, the surgery completes, the kick's callback has run, and the turn is still held. The hold file has 36 tests. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 40 ++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 2a1df0424..6bb36fbac 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -592,6 +592,46 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('stops the hold\'s drain kick when a surgery takes the store mid-tick, and the surgery waits for it', async () => { + await withBacklog({}, async (backlog) => { + // One maintenance pass at start, then none for an hour. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + const fw = t.framework as unknown as { + maintenancePass: unknown; + overBudgetDrainAfter: Map; + kickOverBudgetDrain: (name: string, agent: unknown, after?: () => void) => void; + }; + const proto = ContextManager.prototype as unknown as Patchable; + try { + assert.ok(await pollUntil(() => fw.maintenancePass === null, 3_000), 'the start pass is done'); + // From here a tick waits to be let go, as a compression call does. + let letGo!: () => void; + const gate = new Promise((resolve) => { letGo = resolve; }); + proto.tick = async function () { backlog.ticks++; await gate; }; + const before = backlog.ticks; + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.ticks === before + 1, 5_000), + 'the hold\'s kick is in its first tick'); + assert.equal(fw.overBudgetDrainAfter.get('prime')?.length, 1, 'with the hold\'s check waiting on it'); + let switched = false; + const surgery = t.framework.switchBranch('main').then(() => { switched = true; }); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(switched, false, 'the surgery waits for the tick in flight'); + // A kick asked for while the surgery holds the store starts nothing + // and leaves no callback behind. + fw.kickOverBudgetDrain('prime', t.framework.getAgent('prime'), () => {}); + assert.equal(fw.overBudgetDrainAfter.get('prime')?.length, 1); + letGo(); + await surgery; + assert.equal(backlog.ticks, before + 1, 'the kick stopped once the surgery held the store: no tick after it'); + assert.equal(fw.overBudgetDrainAfter.has('prime'), false, 'the kick ran its callback when it ended'); + assert.ok(t.internal.budgetHolds.size === 1, 'the turn is still held'); + } finally { + await t.close(); + } + }); + }); + it('falls back to a dry compile for a strategy with no preview, one at a time, and lands a fit', async () => { // Beth-2389's probe: dry compiles of 300 ms, a pass every 50 ms, each // asking. Compression finishes, and makes the context fit, while one runs. From 48aaf950b469f5735fa8005e3d0e2b7c986f057d Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 10:25:13 -0400 Subject: [PATCH 10/31] test(inference): the mid-tick surgery test lets its tick go when it fails Undoing the kick's surgery guard (registering a callback before declining) failed the test's assertion, but the gated tick was never let go, so the stop in its finally waited on the kick forever and the file never exited. The test now lets the tick go in its finally, so such a failure shows as that test failing. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 6bb36fbac..dc7e935d8 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -602,10 +602,12 @@ describe('a compile refused for budget holds its turn (#304)', () => { kickOverBudgetDrain: (name: string, agent: unknown, after?: () => void) => void; }; const proto = ContextManager.prototype as unknown as Patchable; + // Let go in the finally too, so a failed assertion fails the test + // rather than leaving the tick, and the stop that waits for it, hanging. + let letGo = () => {}; try { assert.ok(await pollUntil(() => fw.maintenancePass === null, 3_000), 'the start pass is done'); // From here a tick waits to be let go, as a compression call does. - let letGo!: () => void; const gate = new Promise((resolve) => { letGo = resolve; }); proto.tick = async function () { backlog.ticks++; await gate; }; const before = backlog.ticks; @@ -627,6 +629,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(fw.overBudgetDrainAfter.has('prime'), false, 'the kick ran its callback when it ended'); assert.ok(t.internal.budgetHolds.size === 1, 'the turn is still held'); } finally { + letGo(); await t.close(); } }); From 2abb9b4004ea08c9b28d5fc1cc8c8921e6326089 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:00:24 -0400 Subject: [PATCH 11/31] fix(inference): the budget hold's signature covers every strategy and removed messages, a release keeps its evidence, and a released turn with nothing pending spends its hold From an internal review of 22bb6ca7..48aaf95. Its 29 issues: 17 fixed here, 5 intended, 5 not real and 2 not worth it. Code: - A strategy that reports no progress snapshot now gives the signature whether it still has compression pending. Before, its signature was undefined, so a released turn its compile refused again was released again at every look. The hold now waits for compression to finish, a settings change or a surgery. - The signature takes the agent's message count, so a message removed by any path (hide, removeMessage, a module's removals) brings a look. An edit that shrinks a message is seen at the next compression step or surgery. - A release stamps lookedAt (formerly refusedAt) with the signature it was decided on, and a refusal of the released turn keeps it. Before, the refusal stamped the signature read after the compile, which could take in a compression step the compile never saw and hide the fit it brought. - afterBudgetHold spends a released record when its turn is refused with nothing pending, as it does past the bound, so the failure says when the turn was first held and why the hold ended. - The note is composed at injection (budgetHoldNote), with the wait counted to then. It no longer says that compression caught up, since an undo, redo or surgery can release a hold too. The failure reason says when the turn was first held, which stays true whatever the time went to. - kickOverBudgetDrain's lines name their caller: a hold's kicks log as [budget-hold], the breaker's as [inference-failed]. - Comments now say that stop and disposal take a hold still waiting with its requests. Requests already released are queued wakes, left as any. Tests, 42 in budget-refusal-hold: - new: a no-progress strategy's re-refused turn is held until compression finishes; a compression step during a released turn's refused compile brings a look; a removed message brings a look; a released turn refused with nothing pending fails with that reason; no hold begins for a compile refused once stop() has closed admission; a context refusal holds as an over-budget one does; - strengthened: the first test checks the 30-minute bound; the gate-closed test sees the interval's look release the turn before the gate opens; the fallback test asserts the overlap it depends on; the timer-off test waits for its second kick and bounds the checks by the elapsed intervals; the real-strategy tests see a check run, through a call-through spy on previewContext, before they wait; the undo test asserts what the held compile contains. Judged not to change, with reasons in the review's ledger: - an undo while a turn is held: it rewinds past that turn's messages, as it does past any queued wake's, and the held turn runs on what the undo left. Ada-1017's call; the class is a framework issue she is filing; - a look asked during the fallback's dry compile is dropped, and the next pass looks again; - hold_ended at release, not again when a released turn is spent; - check_failed when an undo leaves the strategy to reinitialize; - a released hold bounded only at a later refusal. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 31 ++-- src/framework.ts | 127 +++++++------ test/budget-refusal-hold.test.ts | 175 ++++++++++++++++-- 3 files changed, 254 insertions(+), 79 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index ee72a693f..1e13f2868 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -12,25 +12,29 @@ that compile says it fits. A dry compile can't say so, since it doesn't throw an over-budget plan at its plan check. A strategy with no preview gets the dry compile. The hold looks after every maintenance pass, every - drain kick it starts, and every surgery, undo and redo, and again only - once something that could make the context fit has changed: compression's - progress, the agent's budget or runtime settings, or the store, by a - surgery. A turn released and refused again, as one is when content - reaches its compile after the check, is held again with its first - deadline, and isn't released again until one of those changes. + drain kick it starts, and every surgery, undo and redo. It looks again + only once something that could make the context fit has changed: + compression's progress (for a strategy that reports none, whether it + still has compression pending), the agent's message count, its runtime + settings, or the store, by a surgery. An edit that shrinks a message is + seen at the next compression step or surgery. A turn released and refused + again, as one is when content reaches its compile after the check, is + held again with its first deadline. It isn't released again until one of + those changes. - **When it ends without running:** 30 minutes after the refusal that started it, or at once when compression has nothing pending, and the turn - then fails as a refused compile did, its reason saying how long it was + then fails as a refused compile did, its reason saying when it was first held. The 30 minutes also bound a compression that is pending but can't move, such as one whose model keeps failing. The hold lives in memory, as a turn in flight does: a restart drops it, and the agent's next wake compiles afresh. - **What the agent sees:** the held turn's compile carries a note saying why it waited, for that compile alone: "[budget-hold] Your context didn't - fit its budget (100,201 tokens against 48,568), so this turn waited 7 min - while compression caught up. It fits now." It's never stored. One held - turn is one undo step, which keeps the messages that arrived while it - waited. + fit its budget (100,201 tokens against 48,568), so this turn waited + 7 min. It fits now." It's never stored. One held turn is one undo step, + which keeps the messages that arrived while it waited. An undo while a + turn is held rewinds past that turn's messages, as it does past any queued + wake's, and the held turn then runs on what the undo left. - **What hosts see:** the refused attempt still emits `inference:failed`, as a provider hold's does, so a module that brackets typing with it stops typing while the turn waits, and the turn's next start emits @@ -38,5 +42,6 @@ the held turn ends, up to 30 minutes later, rather than as exhausted within seconds. New traces `inference:held` and `inference:hold_ended`, and `healthSnapshot()` gives each agent's `budgetHold` (or null). The - `context-refusal` ops alert goes out when the hold starts, and says the - turn is held until its deadline. + `context-refusal` ops alert is raised when the hold starts, at most once + per 15 minutes for an agent, and says the turn is held until its + deadline. A hold's drain kicks log as `[budget-hold]`. diff --git a/src/framework.ts b/src/framework.ts index f4b620151..9af50dca7 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1137,12 +1137,10 @@ interface BudgetHold { recheck?: ReturnType; /** A release check's dry compile is running (checkBudgetHold). */ checking: boolean; - /** The signature (budgetHoldSignature) at the check that last refused, or - * at a refusal of the turn this hold released: the hold looks again only - * once it changes. */ - refusedAt?: string; - /** Once its wait ends: the note the turn's compile carries, saying why it waited. */ - note?: string; + /** The signature (budgetHoldSignature) at the hold's last verdict: a check + * that refused, or one that released a turn its compile then refused. The + * hold looks again only once the signature changes. */ + lookedAt?: string; /** Once its wait ends: its requests include a continuation, whose start * opens the event gate again (beginAgentTurn). */ gateClosed: boolean; @@ -2797,9 +2795,10 @@ export class AgentFramework { // A stopped host must never hang behind its own cooldown. this.cancelProviderAdmission(); - // Nor behind a turn held for budget: its requests go with the stop, as a - // provider hold's do, and nothing a hold left for its agent's name - // outlives the stop. + // Nor behind a turn held for budget: a hold still waiting goes with its + // requests, as a provider hold's do, and every hold's record goes. Requests + // a hold has already released are queued wakes, which a stop leaves + // queued as it leaves any. for (const name of [...(this.budgetHolds?.keys() ?? [])]) this.dropBudgetHold(name); // An open logical turn doesn't survive a stop: a start after a restart // on this instance opens a new one rather than continuing the old turn's @@ -3289,15 +3288,12 @@ export class AgentFramework { hold.reason = boundFailureText(String(error.message), FAILURE_SUMMARY_MAX_CHARS); hold.budget = typeof figures.budget === 'number' ? figures.budget : undefined; hold.actual = typeof figures.actual === 'number' ? figures.actual : undefined; - hold.note = undefined; hold.gateClosed = false; - // The check released this turn and its compile was refused, so the two - // disagreed: content reached the compile after the check, or the compile - // planned what the preview can't see (a kv-unified prefix). The hold - // doesn't release it again on the same evidence, but once compression - // moves, the budget or settings change, or a surgery runs - // (budgetHoldSignature). - hold.refusedAt = existing ? this.budgetHoldSignature(agent) : undefined; + // A released turn refused again keeps lookedAt, the signature its release + // was decided on: the check and the compile disagreed (content reached the + // compile after the check, or the compile planned what the preview can't + // see, such as a kv-unified prefix), so the hold doesn't release it again + // on the same evidence, only once something in the signature changes. hold.timer = setTimeout(() => this.releaseBudgetHold(agent.name, 'deadline'), Math.min(Math.max(0, hold.until - now), MAX_TIMER_DELAY_MS)); hold.timer.unref?.(); // With the maintenance timer off, nothing else compresses or looks again: @@ -3368,16 +3364,15 @@ export class AgentFramework { * another reason, the hold ends too, and the turn's own compile meets that * failure ('check_failed'). If it refuses and compression had nothing * pending, the hold ends without a fit. Otherwise it goes on, and doesn't - * look again until compression moves, the budget or settings change, or a - * surgery runs (BudgetHold.refusedAt). None runs while a surgery holds the - * store; the surgery's release asks. + * look again until something in its signature changes (BudgetHold.lookedAt). + * None runs while a surgery holds the store; the surgery's release asks. */ private async checkBudgetHold(agentName: string): Promise { const hold = this.budgetHolds?.get(agentName); const agent = this.agents.get(agentName); if (hold?.state !== 'holding' || !agent || this.surgeryHold) return; const signature = this.budgetHoldSignature(agent); - if (signature !== undefined && signature === hold.refusedAt) return; + if (signature !== undefined && signature === hold.lookedAt) return; // Read before the verdict: a compression that ends while a dry compile // runs would otherwise make that compile's refusal look final. const pending = this.compressionPending(agent); @@ -3394,9 +3389,16 @@ export class AgentFramework { if (this.budgetHolds.get(agentName) !== hold || hold.state !== 'holding') return; if (this.agents.get(agentName) !== agent || this.surgeryHold) return; } - if (verdict !== 'refused') this.releaseBudgetHold(agentName, verdict); - else if (!pending) this.releaseBudgetHold(agentName, 'nothing_pending'); - else hold.refusedAt = signature; + if (verdict === 'refused' && pending) { + hold.lookedAt = signature; + } else if (verdict === 'refused') { + this.releaseBudgetHold(agentName, 'nothing_pending'); + } else { + // The evidence this release rests on, so that a refusal of the released + // turn holds it until the evidence changes (holdForBudgetRefusal). + hold.lookedAt = signature; + this.releaseBudgetHold(agentName, verdict); + } } /** @@ -3434,17 +3436,22 @@ export class AgentFramework { } } - /** What can make a held turn's context fit: compression's progress, the - * agent's runtime settings (its budget, and the tail and pace that shape - * the plan), and a surgery on the store. A check that refused isn't - * repeated until one of them changes. Undefined when the strategy reports - * no progress, so that every check runs. */ + /** + * What can make a held turn's context fit: compression's progress (for a + * strategy that reports none, whether it still has compression pending), + * the agent's message count (a message removed by any path), the agent's + * runtime settings (its budget, and the tail and pace that shape the plan), + * and surgeries on the store. A verdict isn't taken again until one of them + * changes. An edit that shrinks a message changes none of them, and is seen + * at the next compression step or surgery. Undefined, so that the check + * runs, when one of them can't be read, as while the strategy waits to + * reinitialize for a new branch. + */ private budgetHoldSignature(agent: Agent): string | undefined { try { - const progress = this.contextProgress(agent.getContextManager()); - return progress === undefined - ? undefined - : JSON.stringify([progress, agent.getRuntimeSettings(), this.storeSurgeries]); + const cm = agent.getContextManager(); + const progress = this.contextProgress(cm) ?? { pending: this.compressionPending(agent) }; + return JSON.stringify([progress, cm.getMessageCount(), agent.getRuntimeSettings(), this.storeSurgeries]); } catch { return undefined; } @@ -3462,7 +3469,7 @@ export class AgentFramework { void this.checkBudgetHold(agent.name); return; } - this.kickOverBudgetDrain(agent.name, agent, () => void this.checkBudgetHold(agent.name)); + this.kickOverBudgetDrain(agent.name, agent, () => void this.checkBudgetHold(agent.name), 'hold'); } /** @@ -3470,7 +3477,7 @@ export class AgentFramework { * fresh compile, and the record stays until that turn's stream exists or * the turn ends (settleBudgetHold). * - The turn's next compile carries a note saying why it waited - * (BudgetHold.note). It is an injection of that compile alone, which the + * (budgetHoldNote). It is an injection of that compile alone, which the * context manager adds after selecting within the budget: it can't be * what refuses the compile, it is in the request only when the compile * fit, it is never stored, and it moves no routing. @@ -3478,7 +3485,7 @@ export class AgentFramework { * refusal of its compile holds again, with the same start and deadline. * - After the deadline, or with nothing pending, the hold is spent: the * turn's refusals take the error policy's path, as before #304, and say - * how long the turn was held (afterBudgetHold). + * when the turn was first held (afterBudgetHold). */ private releaseBudgetHold(agentName: string, outcome: Exclude): void { const hold = this.budgetHolds?.get(agentName); @@ -3500,11 +3507,6 @@ export class AgentFramework { hold.state = 'spent'; hold.ended = { outcome, waitedMs }; } - const size = hold.budget !== undefined && hold.actual !== undefined - ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` - : ''; - hold.note = `[budget-hold] Your context didn't fit its budget${size}, so this turn ` + - `waited ${describeWait(waitedMs)} while compression caught up. It fits now.`; // The refusal closed the turn at the event gate. A held continuation (a // budget restart, a guard retry) skips the gate's start when it resumes, // since its predecessor would have kept the gate open: this one didn't, @@ -3514,8 +3516,18 @@ export class AgentFramework { hold.heldRequests = []; } - /** A stop, or the agent's leaving: its held turn goes, with its requests, - * as a provider hold's do, and nothing of it outlives the agent. */ + /** The note a held turn's compile carries once its wait has ended, saying + * why it waited, with the wait counted to now (beginAgentTurn). */ + private budgetHoldNote(hold: BudgetHold): string { + const size = hold.budget !== undefined && hold.actual !== undefined + ? ` (${hold.actual.toLocaleString('en-US')} tokens against ${hold.budget.toLocaleString('en-US')})` + : ''; + return `[budget-hold] Your context didn't fit its budget${size}, so this turn waited ` + + `${describeWait(Date.now() - hold.startedAt)}. It fits now.`; + } + + /** A stop, or the agent's leaving: the agent's hold record goes, and a hold + * still waiting takes its requests with it, as a provider hold's do. */ private dropBudgetHold(agentName: string): void { const hold = this.budgetHolds?.get(agentName); if (!hold) return; @@ -3555,15 +3567,22 @@ export class AgentFramework { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return error; const hold = this.budgetHolds?.get(agentName); - if (hold?.state === 'released' && hold.until <= Date.now()) { - hold.state = 'spent'; - hold.ended = { outcome: 'deadline', waitedMs: Date.now() - hold.startedAt }; + if (hold?.state === 'released') { + // A released turn refused once its bound has passed, or with nothing + // pending that could change the refusal, has spent its hold. + const agent = this.agents.get(agentName); + const outcome = hold.until <= Date.now() ? 'deadline' + : agent && !this.compressionPending(agent) ? 'nothing_pending' : undefined; + if (outcome) { + hold.state = 'spent'; + hold.ended = { outcome, waitedMs: Date.now() - hold.startedAt }; + } } if (hold?.state !== 'spent' || !hold.ended) return error; const why = hold.ended.outcome === 'deadline' ? `the hold's ${describeWait(this.budgetHoldMs)} bound passed` : 'compression had nothing left to do'; - const held = new Error(`${error.message} (the turn was held ${describeWait(hold.ended.waitedMs)} while compression ran; ${why} and it still didn't fit)`); + const held = new Error(`${error.message} (the turn was first held ${describeWait(hold.ended.waitedMs)} ago; ${why} and it still didn't fit)`); held.name = error.name; if (error.stack !== undefined) held.stack = error.stack; return held; @@ -3576,7 +3595,7 @@ export class AgentFramework { * Bounded ticks, best-effort, never throws, one kick per agent at a time; * `after` runs once the kick ends, the one already running if there is one. */ - private kickOverBudgetDrain(agentName: string, agent: Agent, after?: () => void): void { + private kickOverBudgetDrain(agentName: string, agent: Agent, after?: () => void, by: 'breaker' | 'hold' = 'breaker'): void { if (this.surgeryHold) return; if (after) { const waiting = this.overBudgetDrainAfter.get(agentName); @@ -3593,15 +3612,19 @@ export class AgentFramework { await cm.tick(); ticks++; } + // A hold's kicks, every interval with the maintenance timer off, are + // its own work, not failures. + const tag = by === 'hold' ? '[budget-hold]' : '[inference-failed]'; + const label = by === 'hold' ? 'budget hold' : 'OverBudget breaker'; console.error( - `[inference-failed] drain kicked for ${agentName} (OverBudget breaker, ${ticks} ticks` + + `${tag} drain kicked for ${agentName} (${label}, ${ticks} ticks` + `${ticks < 8 ? ', stopped while a surgery holds the store' : ''})`, ); } catch (err) { if (err instanceof MaintenanceWithdrawnError) { - console.error(`[inference-failed] drain kicked for ${agentName} (OverBudget breaker, ${ticks} ticks, stopped: ${err.message})`); + console.error(`${by === 'hold' ? '[budget-hold]' : '[inference-failed]'} drain kicked for ${agentName} (${by === 'hold' ? 'budget hold' : 'OverBudget breaker'}, ${ticks} ticks, stopped: ${err.message})`); } else { - console.error(`[inference-failed] drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); + console.error(`${by === 'hold' ? '[budget-hold]' : '[inference-failed]'} drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); } } finally { this.overBudgetDrainInFlight.delete(agentName); @@ -12438,7 +12461,7 @@ export class AgentFramework { // budget, so the note can't be what refuses this compile, and it is in // the request only when the compile fit. It is never stored. const heldTurn = this.budgetHolds?.get(agent.name); - const holdNote = heldTurn && heldTurn.state !== 'holding' ? heldTurn.note : undefined; + const holdNote = heldTurn && heldTurn.state !== 'holding' ? this.budgetHoldNote(heldTurn) : undefined; if (holdNote !== undefined) { const noteInjection: ContextInjection = { namespace: 'framework:budget-hold', diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index dc7e935d8..5f973adab 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -80,13 +80,22 @@ interface Backlog { ticks: number; fitAt: number; readyAt: number; fit: boolean; alwaysPending: boolean; stalled: boolean; previewless: boolean; refuseCompiles: number; previews: number; dryRuns: number; dryRunsRunning: number; mostDryRunsRunning: number; compiles: number; + /** Dry compiles that began before the context fit and ended after. */ + dryRunsAcrossFit: number; + /** The strategy reports no progress snapshot at all. */ + noProgress: boolean; + /** When set, the progress snapshot reports this instead of the ticks. */ + progress?: number; + /** The refusal's error name: OverBudgetError unless set. */ + refusalName?: string; previewError?: Error; dryRunGate?: Promise; dryRunMs?: number; slowCompile?: { nth: number; ms: number }; } const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; const pending = (b: Backlog): boolean => b.alwaysPending || (!fits(b) && b.ticks < b.readyAt); +let refusalName = 'OverBudgetError'; const overBudget = (actual = 100_201) => Object.assign( new Error('Compile plan would exceed hard budget: head=41200 tail=8100 middle=62000 budget=48568'), - { name: 'OverBudgetError', budget: 48_568, actual }, + { name: refusalName, budget: 48_568, actual }, ); type Patchable = Record<'compile' | 'tick' | 'isReady' | 'previewContext', (...args: unknown[]) => unknown>; @@ -96,8 +105,10 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr ticks: 0, fitAt: Infinity, readyAt: Infinity, fit: false, alwaysPending: false, stalled: false, previewless: false, refuseCompiles: 0, previews: 0, dryRuns: 0, dryRunsRunning: 0, mostDryRunsRunning: 0, compiles: 0, + dryRunsAcrossFit: 0, noProgress: false, ...backlog, }; + refusalName = b.refusalName ?? 'OverBudgetError'; const proto = ContextManager.prototype as unknown as Patchable; const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady, previewContext: proto.previewContext }; const passthrough = PassthroughStrategy.prototype as unknown as { getProgressSnapshot?: () => unknown }; @@ -109,6 +120,7 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr const fitsNow = fits(b); if (b.dryRunGate) await b.dryRunGate; if (b.dryRunMs) await new Promise((r) => setTimeout(r, b.dryRunMs)); + if (!fitsNow && fits(b)) b.dryRunsAcrossFit++; if (!fitsNow) throw overBudget(); } finally { b.dryRunsRunning--; @@ -135,12 +147,17 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr }; proto.tick = async function () { b.ticks++; }; proto.isReady = function () { return !pending(b); }; - passthrough.getProgressSnapshot = function () { return { chunksCompressed: b.stalled ? 0 : b.ticks, fits: fits(b) }; }; + if (!b.noProgress) { + passthrough.getProgressSnapshot = function () { + return { chunksCompressed: b.progress ?? (b.stalled ? 0 : b.ticks), fits: fits(b) }; + }; + } try { return await run(b); } finally { Object.assign(proto, original); delete passthrough.getProgressSnapshot; + refusalName = 'OverBudgetError'; } } @@ -273,6 +290,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(held[0]!.errorType, 'over_budget'); assert.equal(held[0]!.budget, 48_568); assert.equal(held[0]!.actual, 100_201); + assert.equal((held[0]!.until as number) - (held[0]!.since as number), 30 * 60_000, 'a fixed 30-minute bound'); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); assert.deepEqual(t.of('inference:exhausted'), [], 'the turn never failed'); } finally { @@ -288,7 +306,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('Are you there?'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); const first = JSON.stringify(t.membrane.calls[0]); - assert.match(first, /\[budget-hold\] Your context didn't fit its budget \(100,201 tokens against 48,568\), so this turn waited \d+ s while compression caught up\. It fits now\./); + assert.match(first, /\[budget-hold\] Your context didn't fit its budget \(100,201 tokens against 48,568\), so this turn waited \d+ s\. It fits now\./); // An injection, never stored: no later turn sees it, and no stored // message could move a turn's routing (#333's arrival rule). assert.ok(!storedText(t.framework).includes('[budget-hold]'), 'the note is not stored'); @@ -428,7 +446,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['nothing_pending']); const exhausted = t.of('inference:exhausted')[0]!; assert.equal(exhausted.errorType, 'over_budget'); - assert.match(String(exhausted.error), /the turn was held \d+ s while compression ran; compression had nothing left to do and it still didn't fit/); + assert.match(String(exhausted.error), /the turn was first held \d+ s ago; compression had nothing left to do and it still didn't fit/); assert.equal(t.membrane.calls.length, 0); } finally { await t.close(); @@ -443,7 +461,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); - assert.match(String(t.of('inference:exhausted')[0]!.error), /the turn was held \d+ s while compression ran; the hold's 1 s bound passed and it still didn't fit/); + assert.match(String(t.of('inference:exhausted')[0]!.error), /the turn was first held \d+ s ago; the hold's 1 s bound passed and it still didn't fit/); assert.equal(t.membrane.calls.length, 0); } finally { await t.close(); @@ -535,11 +553,14 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - await new Promise((r) => setTimeout(r, 550)); - // The first kick at once, then one an interval: about 6 in 550 ms. - assert.ok(backlog.ticks <= 8 * 8, `${backlog.ticks} ticks in 550 ms`); - assert.ok(backlog.previews <= 8, `${backlog.previews} checks in 550 ms`); - assert.ok(backlog.ticks >= 16, 'it does kick again'); + const since = Date.now(); + assert.ok(await pollUntil(() => backlog.ticks >= 16, 5_000), 'it does kick again'); + await new Promise((r) => setTimeout(r, 300)); + // The first kick at once, then one an interval of 100 ms: never more + // kicks, or checks, than the intervals that have passed allow. + const intervals = Math.ceil((Date.now() - since) / 100) + 1; + assert.ok(backlog.ticks <= 8 * (intervals + 1), `${backlog.ticks} ticks in ${intervals} intervals`); + assert.ok(backlog.previews <= intervals + 1, `${backlog.previews} checks in ${intervals} intervals`); } finally { await t.close(); } @@ -558,9 +579,11 @@ describe('a compile refused for budget holds its turn (#304)', () => { await new Promise((r) => setTimeout(r, 200)); assert.equal(backlog.ticks, 0, 'no kick while the gate is closed: its calls would park there'); backlog.fit = true; + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), + 'the hold looked again an interval later, with the gate still closed, and released the turn'); + assert.equal(backlog.ticks, 0); blocked = false; - assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), - 'the hold looked again once an interval passed'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); } finally { await t.close(); } @@ -649,6 +672,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { 'a refusal decided before compression finished is not taken for nothing pending'); assert.ok(backlog.dryRuns >= 1 && backlog.previews >= 1, 'the preview had nothing to say, and a dry compile decided'); assert.equal(backlog.mostDryRunsRunning, 1, 'one dry compile at a time'); + assert.ok(backlog.dryRunsAcrossFit >= 1, 'a dry compile began before the fit and ended after it, which is what this test is for'); } finally { await t.close(); } @@ -697,7 +721,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { backlog.fit = true; assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); - assert.match(JSON.stringify(t.membrane.calls[0]), /\[budget-hold\][^"]*so this turn waited \d+ s while compression caught up/); + assert.match(JSON.stringify(t.membrane.calls[0]), /\[budget-hold\][^"]*so this turn waited \d+ s\. It fits now/); assert.deepEqual(t.of('inference:exhausted'), []); } finally { await t.close(); @@ -722,6 +746,12 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(undone.undone, true, 'the earlier turn left a checkpoint to undo'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered turn 2'), 3_000), 'the undo brought the release check forward'); + // The undo took the earlier turn, and with it the held turn's input, + // stored after that turn's checkpoint. The held turn then runs on what + // the undo left, as any queued wake does after an undo. + const released = JSON.stringify(t.membrane.calls[1]); + assert.ok(released.includes('A turn that fits'), 'the undone turn\'s input is still there'); + assert.ok(!released.includes('Are you there?'), 'and the held turn\'s own input is not'); } finally { await t.close(); } @@ -840,6 +870,110 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('for a strategy that reports no progress, does not release a re-refused turn again until compression finishes', async () => { + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, noProgress: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 2, 5_000), 'released once, refused, and held again'); + const compiles = backlog.compiles; + await new Promise((r) => setTimeout(r, 600)); + assert.equal(t.of('inference:held').length, 2, 'no release pass after pass, with no progress to go on'); + assert.equal(backlog.compiles, compiles); + // Compression finishes, which the hold sees, so it looks once more. + backlog.alwaysPending = false; + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 2, 3_000), 'it looks again once nothing is pending'); + } finally { + await t.close(); + } + }); + }); + + it('looks again when compression moved while a released turn compiled, though that compile was refused', async () => { + // Compression's progress is frozen but for one step, which lands while + // the released turn's (slow) compile runs, and that compile is refused. + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: 2, progress: 0, slowCompile: { nth: 2, ms: 300 } }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => backlog.compiles === 2, 5_000), 'released, and its compile has begun'); + backlog.progress = 1; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), + 'the step that landed during the refused compile brought a look, which released the turn again'); + assert.equal(t.of('inference:held').length, 2); + } finally { + await t.close(); + } + }); + }); + + it('looks again when a message is removed while held', async () => { + await withBacklog({ stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + await new Promise((r) => setTimeout(r, 200)); + const previews = backlog.previews; + await new Promise((r) => setTimeout(r, 300)); + assert.equal(backlog.previews, previews, 'nothing changed, so no look'); + const cm = t.framework.getAgent('prime')!.getContextManager(); + const messages = cm.queryMessages({}).messages; + cm.removeMessage(messages[messages.length - 1]!.id); + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'a removed message brings a look'); + } finally { + await t.close(); + } + }); + }); + + it('ends a released turn refused with nothing pending as a hold with nothing left to do', async () => { + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, slowCompile: { nth: 2, ms: 300 } }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => backlog.compiles === 2, 5_000), 'released, and its compile has begun'); + // By the time that compile is refused, compression has nothing pending. + backlog.alwaysPending = false; + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.equal(t.of('inference:held').length, 1, 'not held again'); + assert.match(String(t.of('inference:exhausted')[0]!.error), + /the turn was first held \d+ s ago; compression had nothing left to do and it still didn't fit/); + } finally { + await t.close(); + } + }); + }); + + it('starts no hold for a compile refused once stop() has closed admission', async () => { + await withBacklog({ slowCompile: { nth: 1, ms: 300 } }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + t.send('Are you there?'); + assert.ok(await pollUntil(() => backlog.compiles === 1, 5_000), 'the turn\'s compile has begun'); + // stop() closes admission while that compile runs, and it is then refused. + await t.close(); + await new Promise((r) => setTimeout(r, 400)); + assert.deepEqual(t.of('inference:held'), [], 'no hold begins once stop has closed admission'); + }); + }); + + it('holds a context refusal as it holds an over-budget one', async () => { + await withBacklog({ fitAt: 12, refusalName: 'UncoveredDropError' }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 10_000)); + const held = t.of('inference:held'); + assert.equal(held.length, 1); + assert.equal(held[0]!.errorType, 'context_refusal'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + assert.deepEqual(t.of('inference:exhausted'), []); + } finally { + await t.close(); + } + }); + }); + it('fails a released turn refused after its hold\'s deadline, rather than holding it afresh', async () => { // The check says it fits, but the released turn's compile starts only // after the deadline (a slow queue), and refuses. @@ -1118,9 +1252,16 @@ describe('the release check takes context-manager\'s verdict on the real strateg * strategy's own, unpatched. */ async function realResident(adaptiveResolution: boolean) { - const proto = ContextManager.prototype as unknown as { tick: () => Promise }; + const proto = ContextManager.prototype as unknown as { tick: () => Promise; previewContext: (...args: unknown[]) => unknown }; const tick = proto.tick; proto.tick = async () => {}; + // A spy that counts previews and calls through, to see that a check ran. + const previewContext = proto.previewContext; + let previews = 0; + proto.previewContext = function (this: unknown, ...args: unknown[]) { + previews++; + return previewContext.apply(this, args); + }; const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-real-')); const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); @@ -1155,9 +1296,11 @@ describe('the release check takes context-manager\'s verdict on the real strateg wake: () => (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ agentName: 'prime', reason: 'external-message', source: 'test', timestamp: Date.now(), }), + previews: () => previews, close: async () => { await framework.stop(); proto.tick = tick; + proto.previewContext = previewContext; rmSync(tempDir, { recursive: true, force: true }); }, }; @@ -1178,8 +1321,10 @@ describe('the release check takes context-manager\'s verdict on the real strateg assert.ok(band !== undefined, 'there is a budget where the dry compile resolves and the plan does not fit'); t.setBudget(band); t.framework.start(); + const before = t.previews(); t.wake(); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000), 'the turn\'s compile is refused there'); + assert.ok(await pollUntil(() => t.previews() > before, 3_000), 'and the release check has looked'); await new Promise((r) => setTimeout(r, 1_000)); assert.equal(t.of('inference:held').length, 1, 'held once, never released into the same refusal'); assert.deepEqual(t.of('inference:hold_ended'), []); @@ -1204,8 +1349,10 @@ describe('the release check takes context-manager\'s verdict on the real strateg assert.throws(() => t.preview(fitsFrom), /requires adaptiveResolution/); t.setBudget(Math.round(fitsFrom * 0.8)); t.framework.start(); + const before = t.previews(); t.wake(); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => t.previews() > before, 3_000), 'the release check has looked'); await new Promise((r) => setTimeout(r, 500)); assert.deepEqual(t.of('inference:hold_ended'), [], 'it stays held while the context does not fit'); t.setBudget(fitsFrom + 2_000); From aafdc9ea48f7f184d2fc8e1f06d7ca31142f8964 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:02:35 -0400 Subject: [PATCH 12/31] test(inference): the hold's context-refusal alert goes out at its start; its webhook post is what the cooldown limits The first hold test now asserts the context-refusal ops:alert at the hold's start, with its 'held until' wording. The changelog and BUDGET_HOLD_MS's comment now say that the alert's trace goes out at every hold's start, and that only its webhook post is limited to once per 15 minutes for an agent (opsAlert). Co-Authored-By: Claude Opus 5.5 --- changelog.d/budget-refusal-holds-turn.fixed.md | 7 ++++--- src/framework.ts | 6 +++--- test/budget-refusal-hold.test.ts | 3 +++ 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 1e13f2868..c531c31f8 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -42,6 +42,7 @@ the held turn ends, up to 30 minutes later, rather than as exhausted within seconds. New traces `inference:held` and `inference:hold_ended`, and `healthSnapshot()` gives each agent's `budgetHold` (or null). The - `context-refusal` ops alert is raised when the hold starts, at most once - per 15 minutes for an agent, and says the turn is held until its - deadline. A hold's drain kicks log as `[budget-hold]`. + `context-refusal` ops alert goes out when the hold starts (its + `ops:alert` trace each time, its webhook post at most once per 15 minutes + for an agent), and says the turn is held until its deadline. A hold's + drain kicks log as `[budget-hold]`. diff --git a/src/framework.ts b/src/framework.ts index 9af50dca7..03588c704 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1096,9 +1096,9 @@ const PROVIDER_ACCELERATION_JITTER_MS = 5_000; * working and never makes the context fit, and one that is pending but * can't move: a merge behind a compression hold, a compression model that * keeps failing and being retried, the speculative cap. The operator hears - * of either from the context-refusal alert at each hold's start, at most - * once per 15 minutes. A fixed bound rather than an option: no host has - * needed another. */ + * of either from the context-refusal alert at each hold's start, whose + * webhook post goes at most once per 15 minutes for an agent. A fixed bound + * rather than an option: no host has needed another. */ const BUDGET_HOLD_MS = 30 * 60_000; /** How a budget hold ended: its context fits now; its check failed for * another reason, which the turn's own compile then meets; its deadline diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 5f973adab..d3dee707e 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -291,6 +291,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(held[0]!.budget, 48_568); assert.equal(held[0]!.actual, 100_201); assert.equal((held[0]!.until as number) - (held[0]!.since as number), 30 * 60_000, 'a fixed 30-minute bound'); + const alerts = t.of('ops:alert').filter((e) => e.kind === 'context-refusal'); + assert.equal(alerts.length, 1, 'the context-refusal alert went out when the hold started'); + assert.match(String(alerts[0]!.message), /the turn is held until \S+ while compression catches up/); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); assert.deepEqual(t.of('inference:exhausted'), [], 'the turn never failed'); } finally { From 9c4677ce434f8330c1e34b69eafe024616933dd3 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:30:14 -0400 Subject: [PATCH 13/31] test(inference): pin what the description says hosts and agents see of a budget hold, and declare the kick's log tag once The hold file, now 46 tests, asserts: - a held turn's traces in order: inference:started, inference:failed, inference:held, inference:hold_ended, then inference:started and inference:completed; - no [inference-failed] marker stored for a held turn; - the hold's interval, when no test shortens it, at the maintenance timer's default of 5 s; - the hold's drain kick logging as "[budget-hold] drain kicked for prime (budget hold, 8 ticks)", and the breaker's kick at exhaustion as [inference-failed] (OverBudget breaker); - a held turn's logical turn open while held, and closed by an inference policy's decline at release; - a conversation agent's refused turn held and run, as a resident's is; - a new framework on the same store, after a stop dropped a hold, with no hold, the held turn's message stored, and its next wake's request carrying that message; - no check while a surgery holds the store, and a look at the surgery's release; - an edit that shrinks a message bringing no look until compression's progress moves; - on the real adaptive strategy, the release check previewing at the budget the refused compile planned at; - on the real hierarchical strategy, its dry compile refusing where the context doesn't fit, as a committing compile does. kickOverBudgetDrain declares its log tag and label once, before the drain, so its two catch branches use them rather than repeating the condition. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 12 +- test/budget-refusal-hold.test.ts | 221 +++++++++++++++++++++++++++---- 2 files changed, 200 insertions(+), 33 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 5430c331d..bf8d96c6c 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -3604,6 +3604,10 @@ export class AgentFramework { } if (this.overBudgetDrainInFlight.has(agentName)) return; this.overBudgetDrainInFlight.add(agentName); + // A hold's kicks, every interval with the maintenance timer off, are its + // own work, not failures. + const tag = by === 'hold' ? '[budget-hold]' : '[inference-failed]'; + const label = by === 'hold' ? 'budget hold' : 'OverBudget breaker'; const drain = (async () => { let ticks = 0; try { @@ -3612,19 +3616,15 @@ export class AgentFramework { await cm.tick(); ticks++; } - // A hold's kicks, every interval with the maintenance timer off, are - // its own work, not failures. - const tag = by === 'hold' ? '[budget-hold]' : '[inference-failed]'; - const label = by === 'hold' ? 'budget hold' : 'OverBudget breaker'; console.error( `${tag} drain kicked for ${agentName} (${label}, ${ticks} ticks` + `${ticks < 8 ? ', stopped while a surgery holds the store' : ''})`, ); } catch (err) { if (err instanceof MaintenanceWithdrawnError) { - console.error(`${by === 'hold' ? '[budget-hold]' : '[inference-failed]'} drain kicked for ${agentName} (${by === 'hold' ? 'budget hold' : 'OverBudget breaker'}, ${ticks} ticks, stopped: ${err.message})`); + console.error(`${tag} drain kicked for ${agentName} (${label}, ${ticks} ticks, stopped: ${err.message})`); } else { - console.error(`${by === 'hold' ? '[budget-hold]' : '[inference-failed]'} drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); + console.error(`${tag} drain kick failed for ${agentName} after ${ticks} ticks: ${describeFailureWithFrames(err)}`); } } finally { this.overBudgetDrainInFlight.delete(agentName); diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index d3dee707e..235264bc7 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -56,6 +56,21 @@ function stored(framework: AgentFramework): Array<{ text: string; metadata: Reco } const storedText = (framework: AgentFramework): string => stored(framework).map((m) => m.text).join('\n'); +/** Every console.error line written while `run` runs, still written through. */ +async function errorLines(run: (lines: string[]) => Promise): Promise { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { + lines.push(args.map(String).join(' ')); + original(...args); + }; + try { + return await run(lines); + } finally { + console.error = original; + } +} + /** * The context manager's compression backlog: the context fits once `fitAt` * ticks have run (or `fit` is set), and pending work is reported until then, @@ -229,15 +244,17 @@ const heldRequests = (framework: AgentFramework): number | undefined => { async function residentTurn(options: { maintenanceIntervalMs: number; holdMs?: number; recheckMs?: number; responses?: number; inferencePolicy?: InferencePolicy; + /** A store the test keeps, to open again after a stop; one of its own when left out. */ + storePath?: string; }) { - const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-')); + const tempDir = options.storePath === undefined ? mkdtempSync(join(tmpdir(), 'budget-hold-')) : undefined; const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); for (let i = 1; i < (options.responses ?? 1); i++) { membrane.pushResponse(createMockResponse([{ type: 'text', text: `Answered turn ${i + 1}` }])); } const framework = await AgentFramework.create({ - storePath: join(tempDir, 'store'), + storePath: options.storePath ?? join(tempDir!, 'store'), membrane: membrane.asMembrane(), agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.' }], modules: [new InboxModule()], @@ -261,15 +278,18 @@ async function residentTurn(options: { } as unknown as ProcessEvent); framework.start(); const of = (type: string) => traces.filter((e) => e.type === type) as unknown as Array>; + /** The traces of these types, in the order they came. */ + const kinds = (types: string[]) => traces.filter((e) => types.includes(e.type)).map((e) => e.type); return { framework, internal, membrane, send, of, + kinds, close: async () => { await framework.stop(); - rmSync(tempDir, { recursive: true, force: true }); + if (tempDir) rmSync(tempDir, { recursive: true, force: true }); }, }; } @@ -296,6 +316,13 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.match(String(alerts[0]!.message), /the turn is held until \S+ while compression catches up/); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); assert.deepEqual(t.of('inference:exhausted'), [], 'the turn never failed'); + assert.deepEqual( + t.kinds(['inference:started', 'inference:failed', 'inference:held', 'inference:hold_ended', 'inference:completed', 'inference:exhausted']), + ['inference:started', 'inference:failed', 'inference:held', 'inference:hold_ended', 'inference:started', 'inference:completed'], + 'the refused attempt fails, the hold starts and ends, and the turn starts again and completes', + ); + assert.ok(!storedText(t.framework).includes('[inference-failed]'), 'no failure marker is written for a held turn'); + assert.equal(t.internal.budgetHoldRecheckMs, 5_000, 'unshortened, the hold paces itself at the maintenance timer\'s default interval'); } finally { await t.close(); } @@ -425,16 +452,22 @@ describe('a compile refused for budget holds its turn (#304)', () => { it('keeps today\'s failure when nothing is pending: compression cannot change it', async () => { await withBacklog({ readyAt: 0 }, async () => { - const t = await residentTurn({ maintenanceIntervalMs: 50 }); - try { - t.send('Are you there?'); - assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000), 'the turn fails, as before the hold'); - assert.deepEqual(t.of('inference:held'), []); - assert.equal(t.of('inference:exhausted')[0]!.errorType, 'over_budget'); - assert.ok(storedText(t.framework).includes('[inference-failed]'), 'the agent gets the failure marker'); - } finally { - await t.close(); - } + await errorLines(async (lines) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000), 'the turn fails, as before the hold'); + assert.deepEqual(t.of('inference:held'), []); + assert.equal(t.of('inference:exhausted')[0]!.errorType, 'over_budget'); + assert.ok(storedText(t.framework).includes('[inference-failed]'), 'the agent gets the failure marker'); + assert.ok(await pollUntil(() => lines.some((l) => l.includes('drain kicked for prime')), 3_000)); + const kicks = lines.filter((l) => l.includes('drain kicked for prime')); + assert.ok(kicks.every((l) => l.startsWith('[inference-failed] drain kicked for prime (OverBudget breaker, ')), + `the breaker's kick logs as the breaker's: ${kicks.join(' | ')}`); + } finally { + await t.close(); + } + }); }); }); @@ -535,18 +568,22 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('kicks the drain at once when the hold starts', async () => { + it('kicks the drain at once when the hold starts, and logs the kick as the hold\'s', async () => { // No maintenance timer, and an interval longer than the test: only the // hold's first kick can compress. await withBacklog({ fitAt: 8 }, async () => { - const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 60_000 }); - try { - t.send('Are you there?'); - assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); - assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); - } finally { - await t.close(); - } + await errorLines(async (lines) => { + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 60_000 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + const kicks = lines.filter((l) => l.includes('drain kicked for prime')); + assert.deepEqual(kicks, ['[budget-hold] drain kicked for prime (budget hold, 8 ticks)'], 'one kick, logged as the hold\'s'); + } finally { + await t.close(); + } + }); }); }); @@ -930,6 +967,27 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('sees an edit that shrinks a message at the next compression step, not before', async () => { + await withBacklog({ stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + await new Promise((r) => setTimeout(r, 200)); + const previews = backlog.previews; + const cm = t.framework.getAgent('prime')!.getContextManager(); + const messages = cm.queryMessages({}).messages; + cm.editMessage(messages[messages.length - 1]!.id, [{ type: 'text', text: 'Hi?' }]); + await new Promise((r) => setTimeout(r, 300)); + assert.equal(backlog.previews, previews, 'an edit moves nothing the hold watches, so it brings no look'); + backlog.progress = 1; + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'the next compression step brings one'); + } finally { + await t.close(); + } + }); + }); + it('ends a released turn refused with nothing pending as a hold with nothing left to do', async () => { await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, slowCompile: { nth: 2, ms: 300 } }, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); @@ -1002,11 +1060,14 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + const openLogical = (t.framework as unknown as { openLogical: Map }).openLogical; + assert.ok(openLogical.has('prime'), 'the held turn\'s logical turn stays open'); decline = true; backlog.fit = true; assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1 && t.internal.budgetHolds.size === 0, 3_000), 'released, declined, and its record gone'); assert.equal(t.membrane.calls.length, 0); + assert.ok(!openLogical.has('prime'), 'and the decline closed it'); decline = false; t.send('A later turn'); assert.ok(await pollUntil(() => t.membrane.calls.length === 1, 3_000)); @@ -1017,6 +1078,97 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('holds a conversation agent\'s refused turn as it holds a resident\'s', async () => { + // The fork is spawned while the context fits, since a spawn dry-compiles + // its template, and the backlog then starts for the fork's own turn. + await withBacklog({ fit: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + const fw = t.framework as unknown as { + conversationRouter: unknown; + createConversationAgent: (name: string, channelId: string) => Promise; + }; + const fork = 'prime--chanA'; + try { + // A conversation router whose template is prime, and a fork of it + // for one channel, as a first message in that channel spawns. + fw.conversationRouter = { + templateAgent: 'prime', expired: () => [], channelForAgent: () => 'chanA', touch: () => {}, + getBinding: () => undefined, closurePrompt: () => undefined, + }; + await fw.createConversationAgent(fork, 'chanA'); + backlog.fit = false; + backlog.fitAt = backlog.ticks + 12; + t.internal.pendingRequests.push({ agentName: fork, reason: 'external-message', source: 'test', timestamp: Date.now() } as never); + assert.ok(await pollUntil(() => t.membrane.calls.length === 1, 10_000), 'the fork\'s turn runs once its context fits'); + const held = t.of('inference:held'); + assert.deepEqual(held.map((e) => e.agentName), [fork], 'the fork\'s refused turn was held'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => [e.agentName, e.outcome]), [[fork, 'fit']]); + assert.deepEqual(t.of('inference:exhausted'), []); + } finally { + fw.conversationRouter = undefined; + await t.close(); + } + }); + }); + + it('leaves nothing of a hold after a restart: its messages stay, and the next wake compiles afresh', async () => { + const dir = mkdtempSync(join(tmpdir(), 'budget-hold-restart-')); + try { + await withBacklog({}, async (backlog) => { + const first = await residentTurn({ maintenanceIntervalMs: 50, storePath: join(dir, 'store') }); + first.send('Are you there?'); + assert.ok(await pollUntil(() => first.of('inference:held').length === 1, 5_000)); + await first.close(); + assert.deepEqual(first.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); + // The same store, opened by a new framework, as a process restart does. + backlog.fit = true; + const second = await residentTurn({ maintenanceIntervalMs: 50, storePath: join(dir, 'store') }); + try { + assert.equal(heldRequests(second.framework), undefined, 'no hold after the restart'); + assert.ok(storedText(second.framework).includes('Are you there?'), 'the held turn\'s message is in the store'); + second.send('Still there?'); + assert.ok(await pollUntil(() => second.membrane.calls.length === 1, 5_000)); + const request = JSON.stringify(second.membrane.calls[0]); + assert.ok(request.includes('Are you there?') && request.includes('Still there?'), + 'the next wake compiles afresh, from the store'); + assert.deepEqual(second.of('inference:held'), []); + } finally { + await second.close(); + } + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('runs no check while a surgery holds the store, and looks again at its release', async () => { + // The maintenance timer off and the provider gate closed: the hold's own + // interval is what looks, and a surgery stops nothing else from looking. + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 0, recheckMs: 50 }); + const open = t.internal.providerGateBlocked.bind(t.framework); + let blocked = true; + t.internal.providerGateBlocked = (name: string) => (name === 'prime' && blocked) || open(name); + const fw = t.framework as unknown as { reserveStoreForSurgery: (verb: string, agentName: string) => () => void }; + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + const release = fw.reserveStoreForSurgery('test surgery', 'prime'); + const previews = backlog.previews; + backlog.fit = true; + await new Promise((r) => setTimeout(r, 400)); + assert.deepEqual(t.of('inference:hold_ended'), [], 'no check, through eight intervals, while the surgery holds the store'); + assert.equal(backlog.previews, previews); + release(); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the release looked again, and the turn fits'); + blocked = false; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + } finally { + await t.close(); + } + }); + }); + it('leaves an ephemeral run to fail as before: its caller is waiting', async () => { await withBacklog({}, async () => { const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-eph-')); @@ -1255,16 +1407,25 @@ describe('the release check takes context-manager\'s verdict on the real strateg * strategy's own, unpatched. */ async function realResident(adaptiveResolution: boolean) { - const proto = ContextManager.prototype as unknown as { tick: () => Promise; previewContext: (...args: unknown[]) => unknown }; + const proto = ContextManager.prototype as unknown as { + tick: () => Promise; previewContext: (...args: unknown[]) => unknown; compile: (...args: unknown[]) => unknown; + }; const tick = proto.tick; proto.tick = async () => {}; - // A spy that counts previews and calls through, to see that a check ran. + // Spies that call through: the budget of each preview, to see that a + // check ran and at what budget, and of each committing compile. const previewContext = proto.previewContext; - let previews = 0; + const previewBudgets: unknown[] = []; proto.previewContext = function (this: unknown, ...args: unknown[]) { - previews++; + previewBudgets.push(args[0]); return previewContext.apply(this, args); }; + const compile = proto.compile; + const compileBudgets: unknown[] = []; + proto.compile = function (this: unknown, ...args: unknown[]) { + if ((args[2] as { dryRun?: boolean } | undefined)?.dryRun !== true) compileBudgets.push(args[0]); + return compile.apply(this, args); + }; const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-real-')); const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); @@ -1299,11 +1460,14 @@ describe('the release check takes context-manager\'s verdict on the real strateg wake: () => (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ agentName: 'prime', reason: 'external-message', source: 'test', timestamp: Date.now(), }), - previews: () => previews, + previews: () => previewBudgets.length, + previewBudgets, + compileBudgets, close: async () => { await framework.stop(); proto.tick = tick; proto.previewContext = previewContext; + proto.compile = compile; rmSync(tempDir, { recursive: true, force: true }); }, }; @@ -1328,6 +1492,7 @@ describe('the release check takes context-manager\'s verdict on the real strateg t.wake(); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000), 'the turn\'s compile is refused there'); assert.ok(await pollUntil(() => t.previews() > before, 3_000), 'and the release check has looked'); + assert.deepEqual(t.previewBudgets.at(-1), t.compileBudgets.at(-1), 'at the budget the refused compile planned at'); await new Promise((r) => setTimeout(r, 1_000)); assert.equal(t.of('inference:held').length, 1, 'held once, never released into the same refusal'); assert.deepEqual(t.of('inference:hold_ended'), []); @@ -1349,6 +1514,8 @@ describe('the release check takes context-manager\'s verdict on the real strateg let fitsFrom = 10_000; while (!(await t.dryCompileResolves(fitsFrom)) && fitsFrom < 400_000) fitsFrom += 1_000; assert.ok(fitsFrom < 400_000); + assert.equal(await t.dryCompileResolves(Math.round(fitsFrom * 0.8)), false, + 'its dry compile refuses where the context does not fit, as a committing compile does'); assert.throws(() => t.preview(fitsFrom), /requires adaptiveResolution/); t.setBudget(Math.round(fitsFrom * 0.8)); t.framework.start(); From 08303adf0b5033f7469e9e21c051dcb858f62274 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:47:22 -0400 Subject: [PATCH 14/31] test(inference): render a long wait in minutes, see the gate stay open, and release at a budget with no room for the note - A new test renders the description's two examples exactly: a turn held 7 minutes, by its record, gets "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min. It fits now.", and a released turn refused past the default 30-minute bound fails with "(the turn was first held 30 min ago; the hold's 30 min bound passed and it still didn't fit)". - The first test fits at 40 ticks rather than 12, past the 16 that the pass at start and the hold's first kick give, so the maintenance loop must tick the held agent, and it checks that the hold leaves the agent's provider gate open. - On the real adaptive strategy, the band test now releases the held turn at the smallest budget whose preview fits, found by bisection, where the plan has under 10 tokens to spare against the preview's rejection budget. The released compile carries the note and runs, so the note, merged after the selection, isn't counted against the budget. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 63 ++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 4 deletions(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 235264bc7..d2947fede 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -296,10 +296,14 @@ async function residentTurn(options: { describe('a compile refused for budget holds its turn (#304)', () => { it('runs the refused turn once compression makes the context fit, with no new wake', async () => { - await withBacklog({ fitAt: 12 }, async (backlog) => { + // 40 ticks: the pass at start and the hold's first kick give 16, so the + // maintenance loop must tick the held agent for the context to fit. + await withBacklog({ fitAt: 40 }, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.equal(t.internal.providerGateBlocked('prime'), false, 'the hold leaves the provider gate open, so compression runs'); assert.ok(await pollUntil(() => fits(backlog), 10_000), 'compression should catch up'); const answered = await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000); assert.ok(answered, 'the refused turn should run once the context fits, with no new message to wake it'); @@ -1053,6 +1057,43 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('says a wait of minutes in minutes, in the note and in the failure', async () => { + type Times = { state: string; startedAt: number; until: number }; + const times = (t: { internal: Internal }) => (t.internal.budgetHolds as unknown as Map).get('prime')!; + // The note: a turn held seven minutes, by its record, then released. + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + times(t).startedAt = Date.now() - 7 * 60_000; + backlog.fit = true; + assert.ok(await pollUntil(() => t.membrane.calls.length === 1, 5_000)); + assert.ok(JSON.stringify(t.membrane.calls[0]).includes( + "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min. It fits now.")); + } finally { + await t.close(); + } + }); + // The failure: the default 30-minute bound, a turn first held 30 minutes + // ago by its record, and its released compile refused past the bound. + await withBacklog({ fit: true, alwaysPending: true, refuseCompiles: Infinity, slowCompile: { nth: 2, ms: 300 } }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => times(t)?.state === 'released' && backlog.compiles === 2, 5_000), 'released, and its compile has begun'); + times(t).startedAt = Date.now() - 30 * 60_000; + times(t).until = Date.now() - 1; + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.ok(String(t.of('inference:exhausted')[0]!.error).endsWith( + "(the turn was first held 30 min ago; the hold's 30 min bound passed and it still didn't fit)"), + String(t.of('inference:exhausted')[0]!.error)); + } finally { + await t.close(); + } + }); + }); + it('ends a released turn the inference policy declines: nothing of its hold reaches a later turn', async () => { await withBacklog({}, async (backlog) => { let decline = false; @@ -1453,7 +1494,7 @@ describe('the release check takes context-manager\'s verdict on the real strateg membrane, of: (type: string) => traces.filter((e) => e.type === type) as unknown as Array>, preview: (maxTokens: number) => (cm as unknown as { - previewContext: (b: { maxTokens: number; reserveForResponse: number }) => { fits: boolean; finalTokens: number }; + previewContext: (b: { maxTokens: number; reserveForResponse: number }) => { fits: boolean; finalTokens: number; budgetTokens: number }; }).previewContext(budget(maxTokens)), dryCompileResolves: (maxTokens: number) => cm.compile(budget(maxTokens), undefined, { dryRun: true } as never).then(() => true, () => false), setBudget: (tokens: number) => framework.updateAgentRuntimeSettings('prime', { contextBudgetTokens: tokens, immediate: true }, { allowInfeasible: true }), @@ -1497,9 +1538,23 @@ describe('the release check takes context-manager\'s verdict on the real strateg assert.equal(t.of('inference:held').length, 1, 'held once, never released into the same refusal'); assert.deepEqual(t.of('inference:hold_ended'), []); assert.equal(t.membrane.calls.length, 0); - t.setBudget(400_000); - assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), 'a budget it fits releases it'); + // The smallest budget the plan fits, by the preview. Its room to spare + // is less than the note's size, so the released compile, which carries + // the note, fits only because the note is merged after the selection. + let lo = band; + let hi = 400_000; + while (hi - lo > 1) { + const mid = Math.floor((lo + hi) / 2); + if (t.preview(mid).fits) hi = mid; else lo = mid; + } + const atHi = t.preview(hi); + const spare = atHi.budgetTokens - atHi.finalTokens; + assert.ok(spare >= 0 && spare < 10, + `${spare} tokens to spare at ${hi} (a plan of ${atHi.finalTokens} against ${atHi.budgetTokens}), well under the note's size`); + t.setBudget(hi); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), 'that budget releases it'); assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + assert.ok(JSON.stringify(t.membrane.calls[0]).includes('[budget-hold]'), 'and its compile carries the note'); } finally { await t.close(); } From 26d5ff0215b1e830df05270e474cb5c4c356e666 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:47:36 -0400 Subject: [PATCH 15/31] fix(inference): a refusal the preview rethrows, such as a context refusal, refuses as a dry compile's does context-manager 0.14.1's previewContext answers an over-budget plan with fits: false and rethrows any other error, so a coverage refusal (UncoveredDropError) comes out of it thrown. budgetHoldPreview counted every throw but "requires adaptiveResolution" as check_failed, so the hold released the turn into the same refusal at every look whose signature had changed. Beth-2389 found it reading 65359f3 and showed it with a probe on the real strategy: on the preview path the turn was held 6 times, the hold ended check_failed 5 times, and 6 context-refusal alerts went out; classified as the dry compile classifies, it was held once. budgetHoldThrownVerdict now classifies a throw for both verdicts, the preview's and the dry compile's: a refusal for budget or coverage refuses, and anything else fails the check. Tests, 48 in budget-refusal-hold: - Beth-2389's probe joins the real-strategy tests: with a coverage defect in the strategy's emission, the preview rethrows UncoveredDropError, the turn is held once through four looks with one context-refusal alert, and it runs once the defect clears. With the old classification it was held 6 times. - The harness's preview follows context-manager's contract, rethrowing a refusal other than OverBudgetError rather than answering fits: false. The context-refusal test now fits at 40 ticks, so the check looks while it doesn't fit; with the old classification that turn was held 3 times. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 25 +++++++++----- test/budget-refusal-hold.test.ts | 59 +++++++++++++++++++++++++++++--- 2 files changed, 72 insertions(+), 12 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index bf8d96c6c..7c179710d 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -3404,11 +3404,12 @@ export class AgentFramework { /** * Context-manager's verdict on a held turn's next compile, at the budget it * plans at (Agent.nextCompileBudget): its preview's, which reports an - * over-budget plan as not fitting. A dry compile can't give that verdict, - * since it doesn't throw an over-budget plan at the plan check - * (SelectOptions.dryRun in context-manager). Undefined for a strategy with - * no preview, and for one whose preview needs adaptive resolution: the - * dry compile decides there (budgetHoldDryRun). + * over-budget plan as not fitting and rethrows any other refusal, such as a + * context refusal (UncoveredDropError), a refusal here too. A dry compile + * can't give that verdict, since it doesn't throw an over-budget plan at + * the plan check (SelectOptions.dryRun in context-manager). Undefined for a + * strategy with no preview, and for one whose preview needs adaptive + * resolution: the dry compile decides there (budgetHoldDryRun). */ private budgetHoldPreview(agent: Agent): 'fit' | 'refused' | 'check_failed' | undefined { const cm = agent.getContextManager() as unknown as { @@ -3420,7 +3421,8 @@ export class AgentFramework { if (!preview) return undefined; return preview.fits === true ? 'fit' : 'refused'; } catch (error) { - return error instanceof Error && error.message.includes('requires adaptiveResolution') ? undefined : 'check_failed'; + if (error instanceof Error && error.message.includes('requires adaptiveResolution')) return undefined; + return this.budgetHoldThrownVerdict(error); } } @@ -3431,11 +3433,18 @@ export class AgentFramework { await agent.compileWithInjections(undefined, undefined, { dryRun: true }); return 'fit'; } catch (error) { - const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); - return errorType === 'over_budget' || errorType === 'context_refusal' ? 'refused' : 'check_failed'; + return this.budgetHoldThrownVerdict(error); } } + /** What a verdict's throw says, from the preview or a dry compile alike: a + * refusal for budget or for coverage refuses, as the turn's own compile + * would; anything else fails the check, and the turn's compile meets it. */ + private budgetHoldThrownVerdict(error: unknown): 'refused' | 'check_failed' { + const { errorType } = this.classifyInferenceError(error instanceof Error ? error : new Error(String(error))); + return errorType === 'over_budget' || errorType === 'context_refusal' ? 'refused' : 'check_failed'; + } + /** * What can make a held turn's context fit: compression's progress (for a * strategy that reports none, whether it still has compression pending), diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index d2947fede..b59bacb98 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -158,7 +158,12 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr b.previewError = undefined; throw error; } - return b.previewless ? null : { fits: fits(b) }; + if (b.previewless) return null; + // As context-manager's preview does: an over-budget plan answers + // fits: false, and any other refusal, a context refusal among them, is + // rethrown. + if (!fits(b) && refusalName !== 'OverBudgetError') throw overBudget(); + return { fits: fits(b) }; }; proto.tick = async function () { b.ticks++; }; proto.isReady = function () { return !pending(b); }; @@ -1023,7 +1028,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); it('holds a context refusal as it holds an over-budget one', async () => { - await withBacklog({ fitAt: 12, refusalName: 'UncoveredDropError' }, async () => { + // 40 ticks, past the pass at start and the hold's first kick: the release + // check looks while the context still doesn't fit. + await withBacklog({ fitAt: 40, refusalName: 'UncoveredDropError' }, async () => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { t.send('Are you there?'); @@ -1447,7 +1454,7 @@ describe('the release check takes context-manager\'s verdict on the real strateg * model). The verdict's path, the preview and both compiles, is the * strategy's own, unpatched. */ - async function realResident(adaptiveResolution: boolean) { + async function realResident(adaptiveResolution: boolean, withStrategy?: (strategy: AutobiographicalStrategy) => void) { const proto = ContextManager.prototype as unknown as { tick: () => Promise; previewContext: (...args: unknown[]) => unknown; compile: (...args: unknown[]) => unknown; }; @@ -1470,12 +1477,14 @@ describe('the release check takes context-manager\'s verdict on the real strateg const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-real-')); const membrane = new SequentialStreamMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Answered once it fit' }])); + const strategy = new AutobiographicalStrategy({ adaptiveResolution, targetChunkTokens: 300, recentWindowTokens: 400 }); + withStrategy?.(strategy); const framework = await AgentFramework.create({ storePath: join(tempDir, 'store'), membrane: membrane.asMembrane(), agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.', contextBudgetTokens: 400_000, maxTokens: 1_000, - strategy: new AutobiographicalStrategy({ adaptiveResolution, targetChunkTokens: 300, recentWindowTokens: 400 }), + strategy, }], modules: [], syncIntervalMs: 0, @@ -1496,6 +1505,7 @@ describe('the release check takes context-manager\'s verdict on the real strateg preview: (maxTokens: number) => (cm as unknown as { previewContext: (b: { maxTokens: number; reserveForResponse: number }) => { fits: boolean; finalTokens: number; budgetTokens: number }; }).previewContext(budget(maxTokens)), + addMessage: (text: string) => cm.addMessage('User', [{ type: 'text', text }]), dryCompileResolves: (maxTokens: number) => cm.compile(budget(maxTokens), undefined, { dryRun: true } as never).then(() => true, () => false), setBudget: (tokens: number) => framework.updateAgentRuntimeSettings('prime', { contextBudgetTokens: tokens, immediate: true }, { allowInfeasible: true }), wake: () => (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ @@ -1560,6 +1570,47 @@ describe('the release check takes context-manager\'s verdict on the real strateg } }); + it('holds a context refusal once on the preview path, and runs the turn once the refusal clears', async () => { + // Beth-2389's probe, kept as a test. A coverage defect in the strategy's + // emission, one emitted entry left out while `drop` is set, makes its + // coverage check refuse every select with UncoveredDropError: the + // committing compile's and the preview's alike. Context-manager's preview + // rethrows that refusal rather than answering fits: false. + let drop = true; + const t = await realResident(true, (strategy) => { + const s = strategy as unknown as { mergeAdjacentBodyGroupRaw: (...args: unknown[]) => unknown[] }; + const merge = s.mergeAdjacentBodyGroupRaw.bind(strategy); + s.mergeAdjacentBodyGroupRaw = (...args: unknown[]) => { + const out = merge(...args); + if (drop && out.length > 2) out.splice(1, 1); + return out; + }; + }); + try { + assert.throws(() => t.preview(400_000), (e: unknown) => (e as { name?: string }).name === 'UncoveredDropError', + 'the preview rethrows the context refusal'); + t.framework.start(); + t.wake(); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.equal(t.of('inference:held')[0]!.errorType, 'context_refusal'); + // Four changes the hold looks again for, a message each. + for (let i = 0; i < 4; i++) { + t.addMessage(`later ${i}`); + await new Promise((r) => setTimeout(r, 300)); + } + assert.equal(t.of('inference:held').length, 1, 'held once while the refusal stands'); + assert.deepEqual(t.of('inference:hold_ended'), []); + assert.equal(t.of('ops:alert').filter((e) => e.kind === 'context-refusal').length, 1); + assert.equal(t.membrane.calls.length, 0); + drop = false; + t.addMessage('after the defect'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000), 'the turn runs once the refusal clears'); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + it('falls back to a dry compile where the preview needs adaptive resolution', async () => { const t = await realResident(false); try { From ace70724c51b5c06c7ba840badd0f0e82821de27 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 21:56:06 -0400 Subject: [PATCH 16/31] docs(inference): the bound's comment and the changelog claim only what the tests show BUDGET_HOLD_MS's comment now says that, short of the bound, a hold ends without a fit only when compression has nothing pending or at a stop, rather than that it "ends sooner only when compression has nothing pending", since a fit and a failed check end it too. It credits its examples of a compression that is pending but can't move to R1-3 of #361's review, and gives the reason for a fixed bound and no stall rule, rather than "no host has needed another". The changelog fragment drops what no test of this change shows: the comparisons with a provider hold and with a turn in flight, what a module that brackets typing does, the mechanism behind a released turn refused again, and a compression model that keeps failing as its example. It names #381 for an undo past a pending wake, says a refusal the preview rethrows counts as a refusal and that the hierarchical strategy gets the dry compile, and says the turn's terminal event comes when the held turn ends. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 59 +++++++++---------- src/framework.ts | 20 ++++--- 2 files changed, 39 insertions(+), 40 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index c531c31f8..51dc94a6a 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -5,44 +5,41 @@ compression had made the context fit, nothing ran it again, so the agent stayed silent until its next wake. Now a resident's or a conversation agent's turn is held while its context manager has compression pending. - Later wakes join it. The hold never closes the provider gate, so - compression keeps running. An ephemeral run, and a refusal with nothing - pending, fail as before. + Later wakes join it. The hold leaves the provider gate open, so compression + keeps running. An ephemeral run, and a refusal with nothing pending, fail + as before. - **When it runs:** as one fresh compile, once context-manager's preview of that compile says it fits. A dry compile can't say so, since it doesn't - throw an over-budget plan at its plan check. A strategy with no preview - gets the dry compile. The hold looks after every maintenance pass, every - drain kick it starts, and every surgery, undo and redo. It looks again - only once something that could make the context fit has changed: - compression's progress (for a strategy that reports none, whether it - still has compression pending), the agent's message count, its runtime - settings, or the store, by a surgery. An edit that shrinks a message is - seen at the next compression step or surgery. A turn released and refused - again, as one is when content reaches its compile after the check, is - held again with its first deadline. It isn't released again until one of - those changes. + throw an over-budget plan at its plan check. A refusal the preview + rethrows, such as a context refusal, counts as a refusal. A strategy with + no preview, or the hierarchical strategy, gets the dry compile. The hold + looks after every maintenance pass, every drain kick it starts, and every + surgery, undo and redo. It looks again only once something that could + make the context fit has changed: compression's progress (for a strategy + that reports none, whether it still has compression pending), the agent's + message count, its runtime settings, or the store, by a surgery. An edit + that shrinks a message is seen at the next compression step or surgery. A + turn released and refused again is held again with its first deadline, + and isn't released again until one of those changes. - **When it ends without running:** 30 minutes after the refusal that started it, or at once when compression has nothing pending, and the turn then fails as a refused compile did, its reason saying when it was first - held. The 30 minutes also bound a compression that is pending but can't - move, such as one whose model keeps failing. The hold lives in memory, as - a turn in flight does: a restart drops it, and the agent's next wake - compiles afresh. + held. The 30 minutes also bound a compression that is pending but doesn't + move. The hold lives in memory: a restart drops it, and the agent's next + wake compiles afresh. - **What the agent sees:** the held turn's compile carries a note saying why it waited, for that compile alone: "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min. It fits now." It's never stored. One held turn is one undo step, which keeps the messages that arrived while it waited. An undo while a - turn is held rewinds past that turn's messages, as it does past any queued - wake's, and the held turn then runs on what the undo left. - - **What hosts see:** the refused attempt still emits `inference:failed`, as - a provider hold's does, so a module that brackets typing with it stops - typing while the turn waits, and the turn's next start emits - `inference:started` again. The logical turn stays open, so it settles when - the held turn ends, up to 30 minutes later, rather than as exhausted - within seconds. New traces `inference:held` and `inference:hold_ended`, - and `healthSnapshot()` gives each agent's `budgetHold` (or null). The - `context-refusal` ops alert goes out when the hold starts (its - `ops:alert` trace each time, its webhook post at most once per 15 minutes - for an agent), and says the turn is held until its deadline. A hold's - drain kicks log as `[budget-hold]`. + turn is held rewinds past that turn's messages, and the held turn then + runs on what the undo left (#381 is that for every pending wake). + - **What hosts see:** the refused attempt emits `inference:failed`, and the + turn's next start emits `inference:started` again. The turn's terminal + event comes when the held turn ends, up to 30 minutes after the refusal, + rather than as an exhaustion within seconds. New traces `inference:held` + and `inference:hold_ended`, and `healthSnapshot()` gives each agent's + `budgetHold` (or null). The `context-refusal` ops alert goes out when the + hold starts (its `ops:alert` trace each time, its webhook post at most + once per 15 minutes for an agent), and says the turn is held until its + deadline. A hold's drain kicks log as `[budget-hold]`. diff --git a/src/framework.ts b/src/framework.ts index 7c179710d..ae59eea66 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1090,15 +1090,17 @@ const PROVIDER_ACCELERATION_MAX_COOLDOWN_MS = 10 * 60_000; const PROVIDER_ACCELERATION_JITTER_MS = 5_000; /** How long a turn whose compile was refused for budget waits for compression * to make its context fit (#304). In the incident that asked for the hold, - * compression took 7 minutes from the first refusal to a fit, so 30 minutes - * leaves a large backlog room. A hold ends sooner only when compression has - * nothing pending (checkBudgetHold). So this bounds a compression that keeps - * working and never makes the context fit, and one that is pending but - * can't move: a merge behind a compression hold, a compression model that - * keeps failing and being retried, the speculative cap. The operator hears - * of either from the context-refusal alert at each hold's start, whose - * webhook post goes at most once per 15 minutes for an agent. A fixed bound - * rather than an option: no host has needed another. */ + * compression finished about 7 minutes after the first refusal, so 30 + * minutes leaves a large backlog room. Short of the bound, a hold ends + * without a fit only when compression has nothing pending (checkBudgetHold) + * or at a stop. So this bounds a compression that keeps working and never + * makes the context fit, and one that is pending but can't move, such as a + * merge behind a compression hold, a compression model that keeps failing + * and being retried, or the speculative cap (R1-3 of #361's review). The + * operator hears of either from the context-refusal alert at each hold's + * start, whose webhook post goes at most once per 15 minutes for an agent. + * A fixed bound, with no option and no stall rule: the alert tells the + * operator, and the bound ends the wait. */ const BUDGET_HOLD_MS = 30 * 60_000; /** How a budget hold ended: its context fits now; its check failed for * another reason, which the turn's own compile then meets; its deadline From 99518055e75a2ed828c19b6e150f5a4ff4e70b8c Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Thu, 15 Oct 2026 22:35:51 -0400 Subject: [PATCH 17/31] test(inference): a failing surgery or restart test fails rather than hangs "runs no check while a surgery holds the store" now releases the store in its finally as well, since stop() waits for a surgery's release: with the check's surgery guard removed, its assertion failed and the file then hung until its 180 s bound, with nothing reported (a removal run at ace7072). It now fails in under a second. "leaves nothing of a hold after a restart" closes its first framework in a finally, so a failed poll doesn't leave that framework's timers holding the file open. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index b59bacb98..58092abea 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -1164,9 +1164,12 @@ describe('a compile refused for budget holds its turn (#304)', () => { try { await withBacklog({}, async (backlog) => { const first = await residentTurn({ maintenanceIntervalMs: 50, storePath: join(dir, 'store') }); - first.send('Are you there?'); - assert.ok(await pollUntil(() => first.of('inference:held').length === 1, 5_000)); - await first.close(); + try { + first.send('Are you there?'); + assert.ok(await pollUntil(() => first.of('inference:held').length === 1, 5_000)); + } finally { + await first.close(); + } assert.deepEqual(first.of('inference:hold_ended').map((e) => e.outcome), ['stopped']); // The same store, opened by a new framework, as a process restart does. backlog.fit = true; @@ -1198,10 +1201,13 @@ describe('a compile refused for budget holds its turn (#304)', () => { let blocked = true; t.internal.providerGateBlocked = (name: string) => (name === 'prime' && blocked) || open(name); const fw = t.framework as unknown as { reserveStoreForSurgery: (verb: string, agentName: string) => () => void }; + // Released in finally too: stop() waits for a surgery's release, so a + // failed assertion would otherwise hang the file rather than fail. + let release = () => {}; try { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - const release = fw.reserveStoreForSurgery('test surgery', 'prime'); + release = fw.reserveStoreForSurgery('test surgery', 'prime'); const previews = backlog.previews; backlog.fit = true; await new Promise((r) => setTimeout(r, 400)); @@ -1212,6 +1218,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { blocked = false; assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); } finally { + release(); await t.close(); } }); From a8b789b3065fc2b2f50869a5d557306fd6afa5e9 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 01:10:17 -0400 Subject: [PATCH 18/31] test(inference): the idle-agent test sees the surgery's own look MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "leaves the held agent idle: … a surgery is admitted and looks again" now waits for the hold's first look, which follows its first drain kick, and checks that nothing looks again before the surgery, before it makes the context fit. Before, the kick's look could come after the context fit and release the turn itself: in a removal run at 9951805, with the surgery's ask at its release removed, the test still passed, and a stack trace showed the release coming from the kick's callback. With the ask removed it now fails, "the surgery brought the release check forward", in three runs of three. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 58092abea..83fc0b5c6 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -653,7 +653,15 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(hold, 'the status names the hold'); assert.equal(hold!.errorType, 'over_budget'); assert.equal(hold!.heldRequests, 1); + // The hold's first kick looks once when it ends. Only after that look + // does the context fit, so the look that releases the turn is the + // surgery's. + assert.ok(await pollUntil(() => backlog.previews >= 1, 3_000), 'the hold has looked once, after its first kick'); + await new Promise((r) => setTimeout(r, 100)); + const previews = backlog.previews; backlog.fit = true; + await new Promise((r) => setTimeout(r, 100)); + assert.equal(backlog.previews, previews, 'nothing looks again before the surgery'); await t.framework.switchBranch('main'); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000), 'the surgery brought the release check forward'); From 01fff39a06f3eee813579a150b7fc700df8a96d1 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 14:17:18 -0400 Subject: [PATCH 19/31] test(inference): the real-strategy context refusal asserts the looks it holds through "holds a context refusal once on the preview path" now asserts that the release check looked at least four times while the four messages arrived, each look a preview that rethrows the refusal, before it asserts the turn was held once. Without it, the test would also pass if no look ran. Beth-2389 pointed out the gap between the description's "held once through four looks" and what the test asserted. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 83fc0b5c6..dae5d3ad9 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -1608,12 +1608,15 @@ describe('the release check takes context-manager\'s verdict on the real strateg t.wake(); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); assert.equal(t.of('inference:held')[0]!.errorType, 'context_refusal'); - // Four changes the hold looks again for, a message each. + // Four changes the hold looks again for, a message each, and a look for + // each: the preview rethrows the refusal at every one. + const looksBefore = t.previews(); for (let i = 0; i < 4; i++) { t.addMessage(`later ${i}`); await new Promise((r) => setTimeout(r, 300)); } - assert.equal(t.of('inference:held').length, 1, 'held once while the refusal stands'); + assert.ok(t.previews() >= looksBefore + 4, `${t.previews() - looksBefore} looks while four messages arrived`); + assert.equal(t.of('inference:held').length, 1, 'held once while the refusal stands, through those looks'); assert.deepEqual(t.of('inference:hold_ended'), []); assert.equal(t.of('ops:alert').filter((e) => e.kind === 'context-refusal').length, 1); assert.equal(t.membrane.calls.length, 0); From 3c852758353074efd6318b40c3339b5528820b42 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 14:40:39 -0400 Subject: [PATCH 20/31] docs(inference): say a dry compile can resolve a plan the turn's compile refuses The release check's comment, the changelog fragment and the band test's comment said a dry compile "doesn't throw an over-budget plan at its plan check". In context-manager 0.14.1 that holds only for a calibration multiplier that isn't provisional and above 1. Under one that is, the dry compile's plan check throws, as the committing compile's does, and selectPath drops the multiplier to 1 and selects again, where the dry compile reports and doesn't throw (strategies/autobiographical.ts:8853-8854 and 4962-4971). In both cases a dry compile can resolve a plan the turn's compile refuses, which is what the hold needs from the preview and what the band test shows, so the three texts now say that. Greta-2425 pointed out the unscoped claim. Text only. Co-Authored-By: Claude Opus 5.5 --- changelog.d/budget-refusal-holds-turn.fixed.md | 4 ++-- src/framework.ts | 4 ++-- test/budget-refusal-hold.test.ts | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 51dc94a6a..92e247d77 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -9,8 +9,8 @@ keeps running. An ephemeral run, and a refusal with nothing pending, fail as before. - **When it runs:** as one fresh compile, once context-manager's preview of - that compile says it fits. A dry compile can't say so, since it doesn't - throw an over-budget plan at its plan check. A refusal the preview + that compile says it fits. A dry compile can't say so, since it can + resolve a plan the turn's compile refuses. A refusal the preview rethrows, such as a context refusal, counts as a refusal. A strategy with no preview, or the hierarchical strategy, gets the dry compile. The hold looks after every maintenance pass, every drain kick it starts, and every diff --git a/src/framework.ts b/src/framework.ts index ae59eea66..969b33621 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -3408,8 +3408,8 @@ export class AgentFramework { * plans at (Agent.nextCompileBudget): its preview's, which reports an * over-budget plan as not fitting and rethrows any other refusal, such as a * context refusal (UncoveredDropError), a refusal here too. A dry compile - * can't give that verdict, since it doesn't throw an over-budget plan at - * the plan check (SelectOptions.dryRun in context-manager). Undefined for a + * can't give that verdict, since it can resolve a plan the turn's compile + * refuses (SelectOptions.dryRun in context-manager). Undefined for a * strategy with no preview, and for one whose preview needs adaptive * resolution: the dry compile decides there (budgetHoldDryRun). */ diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index dae5d3ad9..4685fc6b4 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -1542,9 +1542,9 @@ describe('the release check takes context-manager\'s verdict on the real strateg it('keeps holding at a budget where a dry compile resolves but the plan does not fit', async () => { const t = await realResident(true); try { - // Just over the budget, the adaptive strategy's dry compile resolves: - // it doesn't throw an over-budget plan at its plan check. The preview - // says the plan doesn't fit, and the turn's committing compile refuses. + // Just over the budget, the adaptive strategy's dry compile can resolve + // a plan the turn's compile refuses, and the preview says it doesn't + // fit. The loop finds such a budget. assert.equal(t.preview(400_000).fits, true); const plan = t.preview(400_000).finalTokens; let band: number | undefined; From a2274b24613b8f0ff90136d0b1fb159e091c0881 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 18:58:12 -0400 Subject: [PATCH 21/31] fix(inference): the budget hold sees every change to the store, a stale dry-compile refusal decides nothing, and a start that joins a waiting hold gives up its undo step From an internal review of this round's changes (the previously reviewed head with main merged in, against 3c85275): - The release check's signature took the store's length. A removal and an arrival between two looks leave the length as it was, and an edit doesn't change it at all. The signature now counts changes to the agent's store (a message added, edited or removed, by any path) through context-manager's ContextManager.onMessage, kept per context manager from the hold's first look. The count only rises, so no change hides another, and an edit brings a look. - On the fallback path, a dry compile's refusal decided on the context as it stood when the compile began. If compression came back while it ran, or a surgery came whose own look this check turned away, the refusal was stale, and it could end the hold as one with nothing pending. A refusal now decides only if the signature didn't move while the dry compile ran. Otherwise the next look takes a fresh verdict. - A start can reach its compile while a hold waits: one parked for the primary provider's admission resumes without the scheduler's look at the hold. Such a start joined the hold but kept its turn checkpoint, so the held turn took two undo steps. It now gives up its checkpoint, as the start that opened the hold does. - A spent turn's refusals said when the turn was first held as of the moment the hold was spent. The age is now counted at each refusal, and the record keeps only how the hold ended. - Text: - The signature's doc says a strategy whose progress can't be read is treated as one that reports none. - budgetHoldDryRun's comment no longer says every strategy without a preview keeps its plan check on a dry run. - BUDGET_HOLD_MS's comment names a failed check, and an agent that stops or leaves, among the ends without a fit, and drops a review reference. - The changelog says how the hierarchical strategy's preview and dry compile divide the verdict, and what the signature watches. - Tests: - Four new: a removal and an arrival between two looks, a start that joins a waiting hold, a stale refusal on the fallback path, and a spent turn's age at each refusal. - The edit test now asserts a look. - Each window that asserts no verdict now first asserts that the hold was asked to look in it, by a count of checkBudgetHold calls. - The stop test waits for the refusal before asserting no hold, and the context-refusal test waits for each message's look. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 22 +-- src/framework.ts | 84 +++++++---- test/budget-refusal-hold.test.ts | 133 ++++++++++++++++-- 3 files changed, 189 insertions(+), 50 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 92e247d77..19d396c01 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -10,17 +10,17 @@ as before. - **When it runs:** as one fresh compile, once context-manager's preview of that compile says it fits. A dry compile can't say so, since it can - resolve a plan the turn's compile refuses. A refusal the preview - rethrows, such as a context refusal, counts as a refusal. A strategy with - no preview, or the hierarchical strategy, gets the dry compile. The hold - looks after every maintenance pass, every drain kick it starts, and every - surgery, undo and redo. It looks again only once something that could - make the context fit has changed: compression's progress (for a strategy - that reports none, whether it still has compression pending), the agent's - message count, its runtime settings, or the store, by a surgery. An edit - that shrinks a message is seen at the next compression step or surgery. A - turn released and refused again is held again with its first deadline, - and isn't released again until one of those changes. + resolve a plan the turn's compile refuses. A refusal the preview rethrows, + such as a context refusal, counts as a refusal. A strategy with no preview + gets the dry compile. The hierarchical strategy's preview reports a + context that doesn't fit, and for one that does, the dry compile decides. + The hold looks after every maintenance pass, every drain kick it starts, + and every surgery, undo and redo. It looks again only once something that + could make the context fit has changed: compression's progress (for a + strategy that reports none, whether it still has compression pending), a + message added, edited or removed, its runtime settings, or a surgery on + the store. A turn released and refused again is held again with its first + deadline, and isn't released again until one of those changes. - **When it ends without running:** 30 minutes after the refusal that started it, or at once when compression has nothing pending, and the turn then fails as a refused compile did, its reason saying when it was first diff --git a/src/framework.ts b/src/framework.ts index 969b33621..eab6107a4 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1092,11 +1092,12 @@ const PROVIDER_ACCELERATION_JITTER_MS = 5_000; * to make its context fit (#304). In the incident that asked for the hold, * compression finished about 7 minutes after the first refusal, so 30 * minutes leaves a large backlog room. Short of the bound, a hold ends - * without a fit only when compression has nothing pending (checkBudgetHold) - * or at a stop. So this bounds a compression that keeps working and never - * makes the context fit, and one that is pending but can't move, such as a - * merge behind a compression hold, a compression model that keeps failing - * and being retried, or the speculative cap (R1-3 of #361's review). The + * without a fit when compression has nothing pending, when a check fails + * for another reason (the turn's own compile then meets that failure), or + * when its agent stops or leaves. So this bounds a compression that keeps + * working and never makes the context fit, and one that is pending but + * can't move, such as a merge behind a compression hold, a compression model + * that keeps failing and being retried, or the speculative cap. The * operator hears of either from the context-refusal alert at each hold's * start, whose webhook post goes at most once per 15 minutes for an agent. * A fixed bound, with no option and no stall rule: the alert tells the @@ -1147,7 +1148,7 @@ interface BudgetHold { * opens the event gate again (beginAgentTurn). */ gateClosed: boolean; /** Once spent: how it ended. */ - ended?: { outcome: 'deadline' | 'nothing_pending'; waitedMs: number }; + ended?: { outcome: 'deadline' | 'nothing_pending' }; } /** Longest delay one setTimeout can represent (2^31-1 ms, about 24.8 days); * a longer timer fires almost at once, so longer waits re-arm in steps. */ @@ -1717,6 +1718,9 @@ export class AgentFramework { /** Surgeries, undos and redos so far: one can make a held turn's context * fit whatever compression has done (budgetHoldSignature). */ private storeSurgeries = 0; + /** Messages added, edited or removed in each held agent's store, counted + * from the hold's first look (budgetHoldSignature). */ + private readonly storeChanges = new WeakMap(); private providerAccelerationDefaultCooldownMs = PROVIDER_ACCELERATION_DEFAULT_COOLDOWN_MS; private providerAccelerationJitterMs = PROVIDER_ACCELERATION_JITTER_MS; private providerHoldHook: ProviderHoldHook | undefined; @@ -3270,13 +3274,17 @@ export class AgentFramework { const own: InferenceRequest = trigger ? { ...trigger, coalescingSubject: undefined, coalescingBatch: undefined, coalescingBatchSubjects: undefined } : { agentName: agent.name, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now() }; + // One held turn is one undo step: the refused start's checkpoint goes, and + // the released start records the turn's, after the inputs that joined it. + // A start can reach its compile while a hold waits (one parked for the + // primary provider's admission resumes without the scheduler's look at + // the hold), and joins it, its checkpoint going too. if (existing?.state === 'holding') { if (!existing.heldRequests.some((r) => this.sameInferenceRequest(r, own))) existing.heldRequests.push(own); + if (checkpoint !== undefined) this.dropTurnCheckpoint(agent.name, checkpoint); return true; } if (!this.compressionPending(agent)) return false; - // One held turn is one undo step: the refused start's checkpoint goes, and - // the released start records the turn's, after the inputs that joined it. if (checkpoint !== undefined) this.dropTurnCheckpoint(agent.name, checkpoint); const now = Date.now(); const figures = error as Error & { budget?: unknown; actual?: unknown }; @@ -3390,6 +3398,11 @@ export class AgentFramework { } if (this.budgetHolds.get(agentName) !== hold || hold.state !== 'holding') return; if (this.agents.get(agentName) !== agent || this.surgeryHold) return; + // The dry compile planned the context as it stood when it began. A + // refusal decides nothing if something in the signature moved while it + // ran (compression, the store, a surgery whose own look this check + // turned away): the next look takes a fresh verdict. + if (verdict === 'refused' && this.budgetHoldSignature(agent) !== signature) return; } if (verdict === 'refused' && pending) { hold.lookedAt = signature; @@ -3428,8 +3441,12 @@ export class AgentFramework { } } - /** The verdict of a strategy with no preview: a dry compile, whose plan - * check such a strategy doesn't skip (a hierarchical one's doesn't). */ + /** The verdict of a strategy with no preview, or of the hierarchical + * strategy where its preview needs adaptive resolution: a dry compile. The + * hierarchical strategy's refuses where its committing compile does. If a + * strategy's dry compile resolves a plan the turn's compile refuses, the + * released turn is refused and held again on the same evidence (lookedAt), + * until something in the signature changes. */ private async budgetHoldDryRun(agent: Agent): Promise<'fit' | 'refused' | 'check_failed'> { try { await agent.compileWithInjections(undefined, undefined, { dryRun: true }); @@ -3448,26 +3465,39 @@ export class AgentFramework { } /** - * What can make a held turn's context fit: compression's progress (for a - * strategy that reports none, whether it still has compression pending), - * the agent's message count (a message removed by any path), the agent's - * runtime settings (its budget, and the tail and pace that shape the plan), - * and surgeries on the store. A verdict isn't taken again until one of them - * changes. An edit that shrinks a message changes none of them, and is seen - * at the next compression step or surgery. Undefined, so that the check - * runs, when one of them can't be read, as while the strategy waits to - * reinitialize for a new branch. + * What can make a held turn's context fit: compression's progress, the + * agent's store (a message added, edited or removed, by any path), the + * agent's runtime settings (its budget, and the tail and pace that shape + * the plan), and surgeries on the store. A verdict isn't taken again until + * one of them changes. For a strategy that reports no progress, or none it + * can read (as while it waits to reinitialize for a new branch), whether it + * still has compression pending stands in for its progress. Undefined, so + * that the check runs, when anything else can't be read. */ private budgetHoldSignature(agent: Agent): string | undefined { try { const cm = agent.getContextManager(); const progress = this.contextProgress(cm) ?? { pending: this.compressionPending(agent) }; - return JSON.stringify([progress, cm.getMessageCount(), agent.getRuntimeSettings(), this.storeSurgeries]); + return JSON.stringify([progress, this.storeChangeCount(cm), agent.getRuntimeSettings(), this.storeSurgeries]); } catch { return undefined; } } + /** How many times the store has changed since the first look at it: a + * count that only rises, so no change hides another, as a removal would + * hide behind an arrival in the store's length. */ + private storeChangeCount(cm: ContextManager): number { + let changes = this.storeChanges.get(cm); + if (!changes) { + const counted = { count: 0 }; + cm.onMessage(() => { counted.count++; }); + this.storeChanges.set(cm, counted); + changes = counted; + } + return changes.count; + } + /** * Kick the drain for a held turn (kickOverBudgetDrain), then look again. A * provider gate closed to compression would park the kick's calls there, @@ -3516,7 +3546,7 @@ export class AgentFramework { hold.state = 'released'; } else { hold.state = 'spent'; - hold.ended = { outcome, waitedMs }; + hold.ended = { outcome }; } // The refusal closed the turn at the event gate. A held continuation (a // budget restart, a guard retry) skips the gate's start when it resumes, @@ -3570,10 +3600,10 @@ export class AgentFramework { hold.recheck = undefined; } - /** A refusal of a turn whose hold is spent, its reason saying how long the - * turn waited and why it ended. A released turn refused after its hold's - * deadline passed is such a turn too. The error's name is kept, so it - * still classifies as the refusal it is. */ + /** A refusal of a turn whose hold is spent, its reason saying when the turn + * was first held and why its hold ended. A released turn refused after its + * hold's deadline passed is such a turn too. The error's name is kept, so + * it still classifies as the refusal it is. */ private afterBudgetHold(agentName: string, error: Error): Error { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return error; @@ -3586,14 +3616,14 @@ export class AgentFramework { : agent && !this.compressionPending(agent) ? 'nothing_pending' : undefined; if (outcome) { hold.state = 'spent'; - hold.ended = { outcome, waitedMs: Date.now() - hold.startedAt }; + hold.ended = { outcome }; } } if (hold?.state !== 'spent' || !hold.ended) return error; const why = hold.ended.outcome === 'deadline' ? `the hold's ${describeWait(this.budgetHoldMs)} bound passed` : 'compression had nothing left to do'; - const held = new Error(`${error.message} (the turn was first held ${describeWait(hold.ended.waitedMs)} ago; ${why} and it still didn't fit)`); + const held = new Error(`${error.message} (the turn was first held ${describeWait(Date.now() - hold.startedAt)} ago; ${why} and it still didn't fit)`); held.name = error.name; if (error.stack !== undefined) held.stack = error.stack; return held; diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 4685fc6b4..0b32de460 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -274,6 +274,12 @@ async function residentTurn(options: { if (options.recheckMs !== undefined) internal.budgetHoldRecheckMs = options.recheckMs; const traces: TraceEvent[] = []; framework.onTrace((e) => traces.push(e)); + // Every look the hold is asked for (checkBudgetHold), whatever it decides: + // a window that asserts no verdict asserts first that the hold was asked. + let looks = 0; + const hold = framework as unknown as { checkBudgetHold: (agentName: string) => Promise }; + const checkBudgetHold = hold.checkBudgetHold.bind(framework); + hold.checkBudgetHold = (agentName: string) => { looks++; return checkBudgetHold(agentName); }; const send = (text: string) => framework.pushEvent({ type: 'external-message', source: 'test', @@ -292,6 +298,7 @@ async function residentTurn(options: { send, of, kinds, + looks: () => looks, close: async () => { await framework.stop(); if (tempDir) rmSync(tempDir, { recursive: true, force: true }); @@ -766,6 +773,37 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('on the fallback path, sets aside a refusal that compression coming back made stale', async () => { + // A strategy with no progress snapshot: whether compression is pending is + // all the hold knows of it. + await withBacklog({ previewless: true, noProgress: true, alwaysPending: true, readyAt: 0 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.dryRuns >= 1 && backlog.dryRunsRunning === 0, 5_000)); + // Compression goes idle, so the next look's dry compile begins with + // nothing pending, and while it runs, compression has work again. + let release!: () => void; + backlog.dryRunGate = new Promise((resolve) => { release = resolve; }); + const dryRuns = backlog.dryRuns; + backlog.alwaysPending = false; + assert.ok(await pollUntil(() => backlog.dryRuns === dryRuns + 1, 2_000)); + backlog.alwaysPending = true; + backlog.dryRunGate = undefined; + release(); + await new Promise((r) => setTimeout(r, 300)); + assert.deepEqual(t.of('inference:hold_ended'), [], 'the stale refusal did not end the hold as one with nothing pending'); + assert.deepEqual(t.of('inference:exhausted'), []); + backlog.fit = true; + t.send('Still there?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + }); + it('tells a turn that fits after its hold\'s deadline why it waited', async () => { await withBacklog({}, async (backlog) => { // No passes after the first, so nothing looks between the hold's first @@ -861,6 +899,28 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('gives up the undo step of a start that reaches its compile while the hold waits', async () => { + await withBacklog({}, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('First message'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => !t.internal.activeTurnTokens.has('prime'), 3_000)); + // As a start parked for the primary provider's admission does when it + // resumes: it reaches its compile without the scheduler's look at the + // hold, and its refusal joins the waiting hold. + const fw = t.framework as unknown as { startAgentStream: (agent: unknown, trigger: unknown, attempt: number) => Promise }; + await fw.startAgentStream(t.framework.getAgent('prime'), { agentName: 'prime', reason: 'test-start', source: 'test', timestamp: Date.now() }, 0); + assert.equal(t.of('inference:held').length, 1, 'it joined the hold rather than starting another'); + backlog.fit = true; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.equal(t.framework.getUndoRedoState('prime').undoDepth, 1, 'the held turn is still one undo step'); + } finally { + await t.close(); + } + }); + }); + it('holds a released turn again with its first refusal\'s start and deadline when its own compile still refuses', async () => { // The check says it fits at once, but the turn's compile refuses three // times, as it can when content reaches it after the check (the turn @@ -890,7 +950,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 2, 5_000), 'released once, refused, and held again'); const compiles = backlog.compiles; + const asked = t.looks(); await new Promise((r) => setTimeout(r, 600)); + assert.ok(t.looks() > asked, 'passes asked the hold to look'); assert.equal(t.of('inference:held').length, 2, 'no release on the same evidence, pass after pass'); assert.equal(backlog.compiles, compiles, 'and no compile refused for it'); await t.framework.switchBranch('main'); @@ -909,15 +971,19 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); let previews = backlog.previews; + let asked = t.looks(); await new Promise((r) => setTimeout(r, 500)); - assert.equal(backlog.previews, previews, 'ten passes, and no check while nothing changed'); + assert.ok(t.looks() > asked, 'passes asked the hold to look'); + assert.equal(backlog.previews, previews, 'and it took no verdict while nothing changed'); // A budget for the agent changes what its next compile plans at. (The // change previews once itself, as its preflight.) t.framework.updateAgentRuntimeSettings('prime', { contextBudgetTokens: 200_000 }); previews = backlog.previews; assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'it looks once the budget changes'); previews = backlog.previews; + asked = t.looks(); await new Promise((r) => setTimeout(r, 300)); + assert.ok(t.looks() > asked); assert.equal(backlog.previews, previews); backlog.stalled = false; assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'and once compression moves'); @@ -934,7 +1000,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 2, 5_000), 'released once, refused, and held again'); const compiles = backlog.compiles; + const asked = t.looks(); await new Promise((r) => setTimeout(r, 600)); + assert.ok(t.looks() > asked, 'passes asked the hold to look'); assert.equal(t.of('inference:held').length, 2, 'no release pass after pass, with no progress to go on'); assert.equal(backlog.compiles, compiles); // Compression finishes, which the hold sees, so it looks once more. @@ -972,8 +1040,10 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); await new Promise((r) => setTimeout(r, 200)); const previews = backlog.previews; + const asked = t.looks(); await new Promise((r) => setTimeout(r, 300)); - assert.equal(backlog.previews, previews, 'nothing changed, so no look'); + assert.ok(t.looks() > asked, 'passes asked the hold to look'); + assert.equal(backlog.previews, previews, 'and nothing had changed, so it took no verdict'); const cm = t.framework.getAgent('prime')!.getContextManager(); const messages = cm.queryMessages({}).messages; cm.removeMessage(messages[messages.length - 1]!.id); @@ -984,7 +1054,29 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('sees an edit that shrinks a message at the next compression step, not before', async () => { + it('looks again when a message is removed and another arrives between two looks', async () => { + await withBacklog({ stalled: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.previews >= 1, 5_000)); + await new Promise((r) => setTimeout(r, 200)); + const previews = backlog.previews; + // Both in one synchronous step, so no pass comes between them. + const cm = t.framework.getAgent('prime')!.getContextManager(); + const length = cm.getMessageCount(); + const messages = cm.queryMessages({}).messages; + cm.removeMessage(messages[messages.length - 1]!.id); + cm.addMessage('User', [{ type: 'text', text: 'Still there?' }]); + assert.equal(cm.getMessageCount(), length, 'the store is as long as it was'); + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'yet the removal brings a look'); + } finally { + await t.close(); + } + }); + }); + + it('looks again when a message is edited while held', async () => { await withBacklog({ stalled: true }, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { @@ -995,10 +1087,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { const cm = t.framework.getAgent('prime')!.getContextManager(); const messages = cm.queryMessages({}).messages; cm.editMessage(messages[messages.length - 1]!.id, [{ type: 'text', text: 'Hi?' }]); - await new Promise((r) => setTimeout(r, 300)); - assert.equal(backlog.previews, previews, 'an edit moves nothing the hold watches, so it brings no look'); - backlog.progress = 1; - assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'the next compression step brings one'); + assert.ok(await pollUntil(() => backlog.previews > previews, 2_000), 'an edit, which can shrink the context, brings a look'); } finally { await t.close(); } @@ -1030,8 +1119,8 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(await pollUntil(() => backlog.compiles === 1, 5_000), 'the turn\'s compile has begun'); // stop() closes admission while that compile runs, and it is then refused. await t.close(); - await new Promise((r) => setTimeout(r, 400)); - assert.deepEqual(t.of('inference:held'), [], 'no hold begins once stop has closed admission'); + assert.ok(await pollUntil(() => t.of('inference:failed').length === 1, 3_000), 'the compile was refused'); + assert.deepEqual(t.of('inference:held'), [], 'and no hold began, since stop had closed admission'); }); }); @@ -1072,6 +1161,25 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('says when a spent turn was first held as of each refusal it reports', async () => { + // The deadline spends the hold, and the released turn's compile is slow, + // so its refusal comes after the spend. + await withBacklog({ slowCompile: { nth: 2, ms: 500 } }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, holdMs: 200 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['deadline']); + // As though the turn was first held ten minutes before that refusal. + (t.internal.budgetHolds as unknown as Map).get('prime')!.startedAt -= 600_000; + assert.ok(await pollUntil(() => t.of('inference:exhausted').length === 1, 5_000)); + assert.match(String(t.of('inference:exhausted')[0]!.error), /the turn was first held 10 min ago; the hold's/); + } finally { + await t.close(); + } + }); + }); + it('says a wait of minutes in minutes, in the note and in the failure', async () => { type Times = { state: string; startedAt: number; until: number }; const times = (t: { internal: Internal }) => (t.internal.budgetHolds as unknown as Map).get('prime')!; @@ -1217,8 +1325,10 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); release = fw.reserveStoreForSurgery('test surgery', 'prime'); const previews = backlog.previews; + const asked = t.looks(); backlog.fit = true; await new Promise((r) => setTimeout(r, 400)); + assert.ok(t.looks() > asked, 'the hold\'s interval asked it to look'); assert.deepEqual(t.of('inference:hold_ended'), [], 'no check, through eight intervals, while the surgery holds the store'); assert.equal(backlog.previews, previews); release(); @@ -1610,12 +1720,11 @@ describe('the release check takes context-manager\'s verdict on the real strateg assert.equal(t.of('inference:held')[0]!.errorType, 'context_refusal'); // Four changes the hold looks again for, a message each, and a look for // each: the preview rethrows the refusal at every one. - const looksBefore = t.previews(); for (let i = 0; i < 4; i++) { + const looked = t.previews(); t.addMessage(`later ${i}`); - await new Promise((r) => setTimeout(r, 300)); + assert.ok(await pollUntil(() => t.previews() > looked, 3_000), `a look after message ${i + 1}`); } - assert.ok(t.previews() >= looksBefore + 4, `${t.previews() - looksBefore} looks while four messages arrived`); assert.equal(t.of('inference:held').length, 1, 'held once while the refusal stands, through those looks'); assert.deepEqual(t.of('inference:hold_ended'), []); assert.equal(t.of('ops:alert').filter((e) => e.kind === 'context-refusal').length, 1); From b94fb032ba70cfd34ce864c2d05e968f4cdb5651 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 19:02:10 -0400 Subject: [PATCH 22/31] docs(changelog): give the budget hold's runtime settings their agent The fragment's list of what the release check watches said "its runtime settings" after "a message added, edited or removed", where the "its" had lost the agent it once followed. It now says "the agent's runtime settings". Text only. Co-Authored-By: Claude Opus 5.5 --- changelog.d/budget-refusal-holds-turn.fixed.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 19d396c01..cb95cef94 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -18,9 +18,9 @@ and every surgery, undo and redo. It looks again only once something that could make the context fit has changed: compression's progress (for a strategy that reports none, whether it still has compression pending), a - message added, edited or removed, its runtime settings, or a surgery on - the store. A turn released and refused again is held again with its first - deadline, and isn't released again until one of those changes. + message added, edited or removed, the agent's runtime settings, or a + surgery on the store. A turn released and refused again is held again with + its first deadline, and isn't released again until one of those changes. - **When it ends without running:** 30 minutes after the refusal that started it, or at once when compression has nothing pending, and the turn then fails as a refused compile did, its reason saying when it was first From b16026787ee053d2b5330fd725c82894dd0e6b3f Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 21:08:37 -0400 Subject: [PATCH 23/31] fix(inference): the release check reads whether compression is pending as it applies a verdict checkBudgetHold read whether compression had work before the dry compile, so that a compression ending while the dry compile ran couldn't make its refusal look final. The check a2274b2 added covers that case: a refusal decides only if the signature didn't move while the dry compile ran, and a strategy's progress snapshot (or, for one that reports none, whether it has compression pending) is in the signature. What the early read still decided was the case where a strategy's readiness changes without its progress snapshot showing it. There it read readiness from before the dry compile. So when compression got work again while the dry compile ran, a refusal was taken as one with nothing pending, and the hold ended. The check now reads it as it applies the verdict. A new test freezes the progress snapshot and gives compression work again while a dry compile runs. The hold goes on, and the turn runs once the context fits. On the previous commit's source it fails, with the hold ended as one with nothing left to do. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 10 ++++++---- test/budget-refusal-hold.test.ts | 33 ++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index eab6107a4..874444fbd 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -3372,7 +3372,7 @@ export class AgentFramework { * the agent's next compile (budgetHoldPreview, else budgetHoldDryRun). If * it fits, the hold ends and its turn runs. If the verdict failed for * another reason, the hold ends too, and the turn's own compile meets that - * failure ('check_failed'). If it refuses and compression had nothing + * failure ('check_failed'). If it refuses and compression has nothing * pending, the hold ends without a fit. Otherwise it goes on, and doesn't * look again until something in its signature changes (BudgetHold.lookedAt). * None runs while a surgery holds the store; the surgery's release asks. @@ -3383,9 +3383,6 @@ export class AgentFramework { if (hold?.state !== 'holding' || !agent || this.surgeryHold) return; const signature = this.budgetHoldSignature(agent); if (signature !== undefined && signature === hold.lookedAt) return; - // Read before the verdict: a compression that ends while a dry compile - // runs would otherwise make that compile's refusal look final. - const pending = this.compressionPending(agent); let verdict = this.budgetHoldPreview(agent); if (verdict === undefined) { // Only the dry compile awaits, so only it can overlap another check. @@ -3404,6 +3401,11 @@ export class AgentFramework { // turned away): the next look takes a fresh verdict. if (verdict === 'refused' && this.budgetHoldSignature(agent) !== signature) return; } + // Whether compression has work is read as the verdict is applied, not + // before the dry compile: a strategy's readiness can change without its + // progress snapshot showing it, and a refusal is final only if nothing is + // pending now. + const pending = this.compressionPending(agent); if (verdict === 'refused' && pending) { hold.lookedAt = signature; } else if (verdict === 'refused') { diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 0b32de460..52b6e6b84 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -804,6 +804,39 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('on the fallback path, holds on when compression comes back during a dry compile that its progress does not show', async () => { + // Progress frozen: only whether compression is pending changes, and the + // signature doesn't see it. + await withBacklog({ previewless: true, stalled: true, alwaysPending: true, readyAt: 0 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.dryRuns >= 1 && backlog.dryRunsRunning === 0, 5_000)); + // Compression goes idle, and a message brings a look whose dry + // compile begins with nothing pending. While it runs, compression has + // work again. + let release!: () => void; + backlog.dryRunGate = new Promise((resolve) => { release = resolve; }); + const dryRuns = backlog.dryRuns; + backlog.alwaysPending = false; + t.send('Still there?'); + assert.ok(await pollUntil(() => backlog.dryRuns === dryRuns + 1, 2_000)); + backlog.alwaysPending = true; + backlog.dryRunGate = undefined; + release(); + await new Promise((r) => setTimeout(r, 300)); + assert.deepEqual(t.of('inference:hold_ended'), [], 'the refusal met compression with work again, so the hold goes on'); + assert.deepEqual(t.of('inference:exhausted'), []); + backlog.fit = true; + t.send('And now?'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 3_000)); + assert.deepEqual(t.of('inference:hold_ended').map((e) => e.outcome), ['fit']); + } finally { + await t.close(); + } + }); + }); + it('tells a turn that fits after its hold\'s deadline why it waited', async () => { await withBacklog({}, async (backlog) => { // No passes after the first, so nothing looks between the hold's first From 10ceb2444d216918f63b14c95fab2fb945b191ac Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Fri, 16 Oct 2026 21:55:52 -0400 Subject: [PATCH 24/31] test(inference): the hold file's look counter runs against main's source too residentTurn wrapped the framework's checkBudgetHold to count looks. On main, which has no hold, the wrap threw before any test could run, so the file's run against main's source failed every test that uses residentTurn at setup rather than at what each test asserts. The counter now wraps checkBudgetHold only where it exists, so on main the tests fail, or pass, on their own assertions. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 52b6e6b84..7c37498ea 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -276,10 +276,11 @@ async function residentTurn(options: { framework.onTrace((e) => traces.push(e)); // Every look the hold is asked for (checkBudgetHold), whatever it decides: // a window that asserts no verdict asserts first that the hold was asked. + // Counted only where there is a hold, so the file runs against main too. let looks = 0; - const hold = framework as unknown as { checkBudgetHold: (agentName: string) => Promise }; - const checkBudgetHold = hold.checkBudgetHold.bind(framework); - hold.checkBudgetHold = (agentName: string) => { looks++; return checkBudgetHold(agentName); }; + const hold = framework as unknown as { checkBudgetHold?: (agentName: string) => Promise }; + const checkBudgetHold = hold.checkBudgetHold?.bind(framework); + if (checkBudgetHold) hold.checkBudgetHold = (agentName: string) => { looks++; return checkBudgetHold(agentName); }; const send = (text: string) => framework.pushEvent({ type: 'external-message', source: 'test', From 845fd49b96f0db6e3cbabc94966b9aae0a9b20b2 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Sat, 17 Oct 2026 00:58:09 -0400 Subject: [PATCH 25/31] docs(changelog): a hold that ends without a fit still lets a turn that fits run; the terminal event follows the hold The fragment said that when the hold ends without a fit, at its deadline or with nothing pending, the turn then fails as a refused compile did. The turn compiles again first: one whose context fits by then runs, with the note, as the test of a turn that fits after its hold's deadline shows, and only a refusal then takes the error policy's path. It also said the turn's terminal event comes up to 30 minutes after the refusal. The hold lasts up to 30 minutes, but a released turn can still run, or be refused and fail, after that, as the test of a released turn refused after its deadline shows. Text only. Co-Authored-By: Claude Opus 5.5 --- changelog.d/budget-refusal-holds-turn.fixed.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index cb95cef94..876c0b2f9 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -21,12 +21,12 @@ message added, edited or removed, the agent's runtime settings, or a surgery on the store. A turn released and refused again is held again with its first deadline, and isn't released again until one of those changes. - - **When it ends without running:** 30 minutes after the refusal that - started it, or at once when compression has nothing pending, and the turn - then fails as a refused compile did, its reason saying when it was first - held. The 30 minutes also bound a compression that is pending but doesn't - move. The hold lives in memory: a restart drops it, and the agent's next - wake compiles afresh. + - **When it ends without a fit:** 30 minutes after the refusal that + started it, or at once when compression has nothing pending. The turn + then runs if its context fits by then, and otherwise fails as a refused + compile did, its reason saying when it was first held. The 30 minutes + also bound a compression that is pending but doesn't move. The hold lives + in memory: a restart drops it, and the agent's next wake compiles afresh. - **What the agent sees:** the held turn's compile carries a note saying why it waited, for that compile alone: "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited @@ -36,7 +36,7 @@ runs on what the undo left (#381 is that for every pending wake). - **What hosts see:** the refused attempt emits `inference:failed`, and the turn's next start emits `inference:started` again. The turn's terminal - event comes when the held turn ends, up to 30 minutes after the refusal, + event comes when the held turn ends, after a hold of up to 30 minutes, rather than as an exhaustion within seconds. New traces `inference:held` and `inference:hold_ended`, and `healthSnapshot()` gives each agent's `budgetHold` (or null). The `context-refusal` ops alert goes out when the From 2cf69a09c1dfabedc2f44011569f517a150a5d3a Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Sat, 17 Oct 2026 08:08:03 -0400 Subject: [PATCH 26/31] test(inference): a held turn's drain kick once stop() has begun, and one asked for while a kick runs The two cases the merge after #377 promised: - a hold's kick in its first tick when stop() begins: stop() waits for the tick in flight, a kick asked for meanwhile is declined as "not kicked: the framework is stopping" and keeps no callback, and the kick ticks no more, logging "stopped as the framework stops"; - a hold's kick asked for while another runs for the agent: it's declined as "already draining", no second kick starts, its callback runs with the hold's own when the running kick ends, and that kick's eight ticks are all there are. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 87 ++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 7c37498ea..b89b6f139 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -723,6 +723,93 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('stops the hold\'s drain kick once stop() begins, and starts no kick after it', async () => { + await withBacklog({}, async (backlog) => { + await errorLines(async (lines) => { + // One maintenance pass at start, then none for an hour. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + const fw = t.framework as unknown as { + maintenancePass: unknown; + overBudgetDrainAfter: Map; + kickOverBudgetDrain: (name: string, agent: unknown, after?: () => void, by?: 'breaker' | 'hold') => string; + }; + const proto = ContextManager.prototype as unknown as Patchable; + // Let go in the finally too, so a failed assertion fails the test + // rather than leaving the tick, and the stop that waits for it, hanging. + let letGo = () => {}; + let stopping: Promise | undefined; + try { + assert.ok(await pollUntil(() => fw.maintenancePass === null, 3_000), 'the start pass is done'); + // From here a tick waits to be let go, as a compression call does. + const gate = new Promise((resolve) => { letGo = resolve; }); + proto.tick = async function () { backlog.ticks++; await gate; }; + const before = backlog.ticks; + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.ticks === before + 1, 5_000), + 'the hold\'s kick is in its first tick'); + let stopped = false; + stopping = t.close().then(() => { stopped = true; }); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(stopped, false, 'stop() waits for the tick in flight'); + // A kick asked for once stop() has begun starts nothing and keeps + // no callback. + const waiting = fw.overBudgetDrainAfter.get('prime')?.length ?? 0; + assert.equal(fw.kickOverBudgetDrain('prime', t.framework.getAgent('prime'), () => {}, 'hold'), + 'not kicked: the framework is stopping'); + assert.equal(fw.overBudgetDrainAfter.get('prime')?.length ?? 0, waiting, 'and keeps no callback'); + letGo(); + await stopping; + assert.equal(backlog.ticks, before + 1, 'the kick ticked no more once stop() began'); + const kicks = lines.filter((l) => l.includes('drain kicked for prime')); + assert.deepEqual(kicks, ['[budget-hold] drain kicked for prime (budget hold, 1 ticks, stopped as the framework stops)']); + } finally { + letGo(); + await (stopping ?? t.close()); + } + }); + }); + }); + + it('joins a drain kick already in flight: no second kick, and the hold looks when it ends', async () => { + await withBacklog({}, async (backlog) => { + // One maintenance pass at start, then none for an hour. + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + const fw = t.framework as unknown as { + maintenancePass: unknown; + overBudgetDrains: Set>; + overBudgetDrainAfter: Map; + kickOverBudgetDrain: (name: string, agent: unknown, after?: () => void, by?: 'breaker' | 'hold') => string; + }; + const proto = ContextManager.prototype as unknown as Patchable; + let letGo = () => {}; + try { + assert.ok(await pollUntil(() => fw.maintenancePass === null, 3_000), 'the start pass is done'); + const gate = new Promise((resolve) => { letGo = resolve; }); + proto.tick = async function () { backlog.ticks++; await gate; }; + const before = backlog.ticks; + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1 && backlog.ticks === before + 1, 5_000), + 'the hold\'s kick is in its first tick'); + assert.equal(fw.overBudgetDrains.size, 1); + let joined = 0; + assert.equal(fw.kickOverBudgetDrain('prime', t.framework.getAgent('prime'), () => { joined++; }, 'hold'), + 'already draining'); + assert.equal(fw.overBudgetDrains.size, 1, 'no second kick starts'); + assert.equal(fw.overBudgetDrainAfter.get('prime')?.length, 2, 'its callback waits beside the hold\'s own'); + const asked = t.looks(); + letGo(); + assert.ok(await pollUntil(() => joined === 1 && !fw.overBudgetDrainAfter.has('prime'), 3_000), + 'both callbacks ran when the kick ended'); + assert.ok(t.looks() > asked, 'the hold looked when the kick ended'); + assert.equal(backlog.ticks, before + 8, 'one kick\'s eight ticks'); + assert.ok(await pollUntil(() => fw.overBudgetDrains.size === 0, 1_000), 'and the one kick has left the drains'); + } finally { + letGo(); + await t.close(); + } + }); + }); + it('falls back to a dry compile for a strategy with no preview, one at a time, and lands a fit', async () => { // Beth-2389's probe: dry compiles of 300 ms, a pass every 50 ms, each // asking. Compression finishes, and makes the context fit, while one runs. From a1b459cc55a558560dc4b9905f3be005e321cea6 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Sat, 17 Oct 2026 13:35:48 -0400 Subject: [PATCH 27/31] fix(inference): a held turn under host quiesce, a batch's wake beside a resumed turn, an unstarted framework, pending in the signature, and the undo history a held start leaves From a second automated review of the budget hold at 2cf69a09. Each finding has a test, and at 2cf69a09 those tests fail while the file's other 55 pass. - Host quiesce. A held turn has given back its turn token, so quiesce() reports the host drained, and maintenance keeps running while quiesced, so compression could let the hold release. The released context_budget_restart then passed quiesce's gate, as every continuation does, and started a provider call after "drained". A request a budget hold releases is now marked (InferenceRequest.releasedByBudgetHold), and quiesce's checks pass a continuation only unmarked (passesQuiesce): the scheduler's gate, runUntilIdle's idle test and the status's gatedRequests. So it parks as a wake does and runs at resume. getHostModeStatus() gains heldTurns, the turns still held. The admission recheck keeps its own rule: a request there has passed the gate already and counts as a parked admission, which keeps the host undrained until it settles. - A deferred batch's wake beside a turn that resumes a logical turn. Such a turn (a budget restart, or a guard retry a hold released) skips batch assembly, so a batch's wake it consumed never rendered. The scheduler now leaves deferred-batch wakes queued when the trigger resumes a logical turn, for a fresh turn of their own. This fixes main's case too: a batch's wake that waited behind a turn which then restarted was dequeued with the restart and consumed. The hold made it likelier, since a wake that joins a held continuation is released beside it. - A framework that isn't running. With only runUntilIdle(), nothing would run a released turn, so the hold now needs start(): the refusal takes the error policy's path, as an ephemeral run's does. Since stop() clears running in the same step that closes provider admission, the entry's admission check folds into this one, and once started, the maintenance timer runs exactly when its interval is positive, which the hold's own interval relies on. - Whether compression is pending, in the signature beside the progress snapshot, which needn't cover what the strategy's readiness reads. Pending is read once, with the signature: on the preview path nothing awaits, and on the fallback path a refusal counts only if the signature, pending included, is unchanged when the dry compile ends, so the later read of pending that stood in for this is gone. - The undo history. Recording the refused start's checkpoint at the limit of 20 evicted the oldest, and dropping it didn't put that back; the start's new work also cleared the redo stack. recordTurnCheckpoint now returns what it evicted, the start keeps the redo it cleared, and dropTurnCheckpoint puts both back. Tests: a held continuation under quiesce (no provider call after "drained", parked among gatedRequests, heldTurns, runUntilIdle returns, runs at resume); a batch's wake that joins a held continuation, and one that waited behind a turn which then restarted, each getting a fresh turn; a never-started framework's refusal exhausting before runUntilIdle returns; a progress snapshot that stands still while pending changes; the twenty checkpoints at the limit; the redo depth after an undo. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 35 ++- src/framework.ts | 159 ++++++++---- src/types/agent.ts | 4 + test/budget-refusal-hold.test.ts | 230 +++++++++++++++++- 4 files changed, 362 insertions(+), 66 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 876c0b2f9..a491115fc 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -6,8 +6,9 @@ stayed silent until its next wake. Now a resident's or a conversation agent's turn is held while its context manager has compression pending. Later wakes join it. The hold leaves the provider gate open, so compression - keeps running. An ephemeral run, and a refusal with nothing pending, fail - as before. + keeps running. An ephemeral run, a turn in a framework that `start()` + hasn't started (only `runUntilIdle()` runs its turns, so nothing would run + a released one), and a refusal with nothing pending fail as before. - **When it runs:** as one fresh compile, once context-manager's preview of that compile says it fits. A dry compile can't say so, since it can resolve a plan the turn's compile refuses. A refusal the preview rethrows, @@ -16,11 +17,11 @@ context that doesn't fit, and for one that does, the dry compile decides. The hold looks after every maintenance pass, every drain kick it starts, and every surgery, undo and redo. It looks again only once something that - could make the context fit has changed: compression's progress (for a - strategy that reports none, whether it still has compression pending), a - message added, edited or removed, the agent's runtime settings, or a - surgery on the store. A turn released and refused again is held again with - its first deadline, and isn't released again until one of those changes. + could make the context fit has changed: compression's progress or whether + it still has compression pending, a message added, edited or removed, the + agent's runtime settings, or a surgery on the store. A turn released and + refused again is held again with its first deadline, and isn't released + again until one of those changes. - **When it ends without a fit:** 30 minutes after the refusal that started it, or at once when compression has nothing pending. The turn then runs if its context fits by then, and otherwise fails as a refused @@ -31,9 +32,11 @@ why it waited, for that compile alone: "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min. It fits now." It's never stored. One held turn is one undo step, - which keeps the messages that arrived while it waited. An undo while a - turn is held rewinds past that turn's messages, and the held turn then - runs on what the undo left (#381 is that for every pending wake). + which keeps the messages that arrived while it waited, and the refused + start leaves undo and redo as they were, the oldest checkpoint at the + limit of 20 included. An undo while a turn is held rewinds past that + turn's messages, and the held turn then runs on what the undo left (#381 + is that for every pending wake). - **What hosts see:** the refused attempt emits `inference:failed`, and the turn's next start emits `inference:started` again. The turn's terminal event comes when the held turn ends, after a hold of up to 30 minutes, @@ -42,4 +45,14 @@ `budgetHold` (or null). The `context-refusal` ops alert goes out when the hold starts (its `ops:alert` trace each time, its webhook post at most once per 15 minutes for an agent), and says the turn is held until its - deadline. A hold's drain kicks log as `[budget-hold]`. + deadline. A hold's drain kicks log as `[budget-hold]`. While the host is + quiesced, a turn a hold releases waits for resume as a parked wake does, + a continuation included, so no turn starts after `quiesce()` reported the + host drained. `getHostModeStatus()` counts it among `gatedRequests`, and + its new `heldTurns` counts the turns still held. +- A deferred batch's wake that waited behind a turn which then restarted for + its context budget was consumed by the restarted turn, which continues the + same logical turn and skips batch assembly, so the batch never rendered + until some later turn. It now stays queued and wakes a fresh turn of its + own once the restarted turn ends. A wake that joins a held continuation + (above) is kept out of it the same way. diff --git a/src/framework.ts b/src/framework.ts index e2fd9ecb1..1d9949e99 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -648,6 +648,15 @@ function isTurnContinuation(reason: string): boolean { return reason === 'context_budget_restart' || reason === 'tool_results_ready' || reason === 'tool_result_guard_retry'; } + +/** Whether host quiesce lets a queued request through rather than parking + * it: a continuation of a turn whose token is held, which quiesce's drain + * must let finish. A continuation a budget hold released continues a turn + * that gave its token back when it was held, so it parks, as a wake does, + * and runs at resume (#304). */ +function passesQuiesce(request: InferenceRequest): boolean { + return isTurnContinuation(request.reason) && request.releasedByBudgetHold !== true; +} const CONVERSATION_ROUTER_STATE_ID = 'framework/conversation-router'; const INFERENCE_LOG_ID = 'framework/inference-log'; const PROCESS_LOG_ID = 'framework/process-log'; @@ -668,6 +677,16 @@ interface TurnCheckpoint { timestamp: number; } +/** A turn checkpoint as its start recorded it: its turn index, the oldest + * checkpoints the record evicted at the depth limit, and the redo entries + * the start's new work invalidated, all of which dropping it puts back + * (dropTurnCheckpoint). */ +interface RecordedTurnCheckpoint { + turnIndex: number; + evicted: TurnCheckpoint[]; + redo?: RedoEntry[]; +} + type DiscordAwarenessDrainOutcome = | { status: 'delivered'; delivered: number; failed: number } | { status: 'unavailable'; accounted: number }; @@ -892,9 +911,14 @@ export interface HostModeStatus { parkedAdmissions: number; /** Inference requests parked by the quiesce wake gate (while quiesced), or * simply queued (while serving — non-zero right after a resume until the - * scheduler's next pass consumes them). Continuations of a held turn - * (budget restart, tool results) are never counted. */ + * scheduler's next pass consumes them). Continuations of a turn whose token + * is held (budget restart, tool results) are never counted; one a budget + * hold released is, since it parks until resume (#304). */ gatedRequests: number; + /** Turns held for budget (#304), each waiting for compression to make its + * context fit. One a hold releases while the host is quiesced parks with + * the gated requests and runs at resume. */ + heldTurns: number; /** Background code-execution scripts still running. Quiesce does NOT stop * them — they hold no turn token, so drain doesn't wait for them. Reported * so the operator sees what is still acting during the window. */ @@ -3286,10 +3310,12 @@ export class AgentFramework { * * Only a turn that comes through the scheduler holds: a resident's or a * conversation agent's. An ephemeral run's caller is waiting on it, so its - * refusal keeps the error policy's path. So does a refusal while the - * context manager has nothing pending, since compression can't change it; - * one of a turn whose hold is spent; and one once stop() has closed - * provider admission. + * refusal keeps the error policy's path. So does a refusal in a framework + * that isn't running: one start() hasn't started, where only runUntilIdle() + * runs turns and nothing would run the released one, and one stop() has + * begun to stop. So does one while the context manager has nothing + * pending, since compression can't change it, and one of a turn whose hold + * is spent. * * The agent's one record of its held turn (BudgetHold) lasts from the first * refusal until that turn's stream exists or the turn ends. A refusal of @@ -3302,12 +3328,12 @@ export class AgentFramework { agent: Agent, error: Error, trigger: InferenceRequest | undefined, - /** The turn checkpoint the refused start recorded, by turn index. */ - checkpoint: number | undefined, + /** The turn checkpoint the refused start recorded. */ + checkpoint: RecordedTurnCheckpoint | undefined, ): boolean { const { errorType } = this.classifyInferenceError(error); if (errorType !== 'over_budget' && errorType !== 'context_refusal') return false; - if (!this.budgetHolds || this.providerAdmissionClosed) return false; + if (!this.budgetHolds || !this.running) return false; if (this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; const existing = this.budgetHolds.get(agent.name); if (existing?.state === 'spent') return false; @@ -3320,11 +3346,12 @@ export class AgentFramework { const own: InferenceRequest = trigger ? { ...trigger, coalescingSubject: undefined, coalescingBatch: undefined, coalescingBatchSubjects: undefined } : { agentName: agent.name, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now() }; - // One held turn is one undo step: the refused start's checkpoint goes, and - // the released start records the turn's, after the inputs that joined it. - // A start can reach its compile while a hold waits (one parked for the - // primary provider's admission resumes without the scheduler's look at - // the hold), and joins it, its checkpoint going too. + // One held turn is one undo step: the refused start's checkpoint goes, + // with what recording it took back (the checkpoints it evicted, the redo + // it invalidated), and the released start records the turn's, after the + // inputs that joined it. A start can reach its compile while a hold waits + // (one parked for the primary provider's admission resumes without the + // scheduler's look at the hold), and joins it, its checkpoint going too. if (existing?.state === 'holding') { if (!existing.heldRequests.some((r) => this.sameInferenceRequest(r, own))) existing.heldRequests.push(own); if (checkpoint !== undefined) this.dropTurnCheckpoint(agent.name, checkpoint); @@ -3427,7 +3454,8 @@ export class AgentFramework { const hold = this.budgetHolds?.get(agentName); const agent = this.agents.get(agentName); if (hold?.state !== 'holding' || !agent || this.surgeryHold) return; - const signature = this.budgetHoldSignature(agent); + const pending = this.compressionPending(agent); + const signature = this.budgetHoldSignature(agent, pending); if (signature !== undefined && signature === hold.lookedAt) return; let verdict = this.budgetHoldPreview(agent); if (verdict === undefined) { @@ -3443,15 +3471,12 @@ export class AgentFramework { if (this.agents.get(agentName) !== agent || this.surgeryHold) return; // The dry compile planned the context as it stood when it began. A // refusal decides nothing if something in the signature moved while it - // ran (compression, the store, a surgery whose own look this check - // turned away): the next look takes a fresh verdict. + // ran (compression's progress or whether it has work, the store, a + // surgery whose own look this check turned away): the next look takes a + // fresh verdict. So `pending` is still current below. if (verdict === 'refused' && this.budgetHoldSignature(agent) !== signature) return; } - // Whether compression has work is read as the verdict is applied, not - // before the dry compile: a strategy's readiness can change without its - // progress snapshot showing it, and a refusal is final only if nothing is - // pending now. - const pending = this.compressionPending(agent); + // A refusal is final only if compression has nothing pending. if (verdict === 'refused' && pending) { hold.lookedAt = signature; } else if (verdict === 'refused') { @@ -3513,20 +3538,26 @@ export class AgentFramework { } /** - * What can make a held turn's context fit: compression's progress, the - * agent's store (a message added, edited or removed, by any path), the - * agent's runtime settings (its budget, and the tail and pace that shape - * the plan), and surgeries on the store. A verdict isn't taken again until - * one of them changes. For a strategy that reports no progress, or none it - * can read (as while it waits to reinitialize for a new branch), whether it - * still has compression pending stands in for its progress. Undefined, so - * that the check runs, when anything else can't be read. + * What can make a held turn's context fit: compression's progress and + * whether it still has compression pending, the agent's store (a message + * added, edited or removed, by any path), the agent's runtime settings (its + * budget, and the tail and pace that shape the plan), and surgeries on the + * store. A verdict isn't taken again until one of them changes. Pending sits + * beside the strategy's progress snapshot, which needn't cover what its + * readiness reads, and alone for a strategy that reports no progress, or + * none it can read (as while it waits to reinitialize for a new branch). + * Undefined, so that the check runs, when anything else can't be read. */ - private budgetHoldSignature(agent: Agent): string | undefined { + private budgetHoldSignature(agent: Agent, pending = this.compressionPending(agent)): string | undefined { try { const cm = agent.getContextManager(); - const progress = this.contextProgress(cm) ?? { pending: this.compressionPending(agent) }; - return JSON.stringify([progress, this.storeChangeCount(cm), agent.getRuntimeSettings(), this.storeSurgeries]); + return JSON.stringify([ + this.contextProgress(cm) ?? null, + pending, + this.storeChangeCount(cm), + agent.getRuntimeSettings(), + this.storeSurgeries, + ]); } catch { return undefined; } @@ -3601,7 +3632,9 @@ export class AgentFramework { // since its predecessor would have kept the gate open: this one didn't, // so its start opens it again (beginAgentTurn). hold.gateClosed = hold.heldRequests.some((r) => resumesLogicalTurn(r.reason)); - this.pendingRequests.push(...hold.heldRequests); + // Marked, so that host quiesce parks them until resume, continuations + // included: the turn gave its token back when it was held (passesQuiesce). + this.pendingRequests.push(...hold.heldRequests.map((r) => ({ ...r, releasedByBudgetHold: true }))); hold.heldRequests = []; } @@ -5148,7 +5181,8 @@ export class AgentFramework { drained: this.activeTurnTokens.size === 0 && parkedAdmissions === 0, activeTurns: this.activeTurnTokens.size, parkedAdmissions, - gatedRequests: this.pendingRequests.filter((r) => !isTurnContinuation(r.reason)).length, + gatedRequests: this.pendingRequests.filter((r) => !passesQuiesce(r)).length, + heldTurns: [...(this.budgetHolds?.values() ?? [])].filter((hold) => hold.state === 'holding').length, backgroundScripts: [...this.backgroundScripts.values()] .filter((record) => record.status === 'running').length, deferredWrites: this.deferredMessages.length, @@ -9073,7 +9107,7 @@ export class AgentFramework { // Turn checkpoint internals // ========================================================================== - private recordTurnCheckpoint(agentName: string): number { + private recordTurnCheckpoint(agentName: string): RecordedTurnCheckpoint { const turnIndex = this.turnCounters.get(agentName) ?? 0; this.turnCounters.set(agentName, turnIndex + 1); @@ -9089,21 +9123,25 @@ export class AgentFramework { checkpoints.push(checkpoint); // Trim to max depth - if (checkpoints.length > MAX_TURN_CHECKPOINTS) { - checkpoints.splice(0, checkpoints.length - MAX_TURN_CHECKPOINTS); - } + const evicted = checkpoints.length > MAX_TURN_CHECKPOINTS + ? checkpoints.splice(0, checkpoints.length - MAX_TURN_CHECKPOINTS) + : []; this.saveTurnCheckpoints(agentName, checkpoints); - return turnIndex; + return { turnIndex, evicted }; } - /** Drop the agent's newest turn checkpoint, if it is the one recorded for - * this turn index (holdForBudgetRefusal). */ - private dropTurnCheckpoint(agentName: string, turnIndex: number): void { + /** Drop the agent's newest turn checkpoint, if it is the one this start + * recorded, and put back what recording it took, the checkpoints it + * evicted and the redo it invalidated, so that undo and redo are as they + * were before the start (holdForBudgetRefusal). */ + private dropTurnCheckpoint(agentName: string, recorded: RecordedTurnCheckpoint): void { const checkpoints = this.getTurnCheckpoints(agentName); - if (checkpoints.at(-1)?.turnIndex !== turnIndex) return; + if (checkpoints.at(-1)?.turnIndex !== recorded.turnIndex) return; checkpoints.pop(); + checkpoints.unshift(...recorded.evicted); this.saveTurnCheckpoints(agentName, checkpoints); + if (recorded.redo) this.redoStacks.set(agentName, recorded.redo); } /** @@ -9203,10 +9241,11 @@ export class AgentFramework { // Direct inference requests (e.g. runEphemeralToCompletion) bypass the // event queue — without this the loop can exit before they're drained. // While quiesced, gate-parked requests are NOT progress and must not - // block idle (only a budget restart can still run); identity-equivalent - // to `.length > 0` when not quiesced. + // block idle (only a continuation of a turn whose token is held can + // still run, passesQuiesce); identity-equivalent to `.length > 0` when + // not quiesced. this.pendingRequests.some( - (r) => !this.quiesced || isTurnContinuation(r.reason), + (r) => !this.quiesced || passesQuiesce(r), ) || this.activeStreams.size > 0 || Array.from(this.agents.values()).some((a) => a.state.status !== 'idle') @@ -11261,10 +11300,11 @@ export class AgentFramework { // kept): heartbeat/gate wakes arrive once per tick and nothing else // bounds a multi-day maintenance window's accumulation. if (this.quiesced) { - // Continuations of a held turn (budget restart, non-streaming tool - // results) pass; everything else parks. - const passThrough = requests.filter((r) => isTurnContinuation(r.reason)); - const parked = requests.filter((r) => !isTurnContinuation(r.reason)); + // Continuations of a turn whose token is held (budget restart, + // non-streaming tool results) pass; everything else parks, a + // continuation a budget hold released included (passesQuiesce). + const passThrough = requests.filter((r) => passesQuiesce(r)); + const parked = requests.filter((r) => !passesQuiesce(r)); // Coalesce per (reason, addressed): every channel wake shares one // reason, so keying on reason alone would let an ambient message // parked later evict a DM/mention parked earlier — and the @@ -11379,6 +11419,19 @@ export class AgentFramework { // requests[0] in that mixed batch bypassed the turn lock because a // restart existed, then treated the continuation as a fresh turn. const trigger = budgetRestart ?? requests[0]; + // A turn that resumes a logical turn skips batch assembly + // (beginAgentTurn), so a deferred batch's wake can't be one of its + // causes: its batch would never render. It stays queued for a fresh + // turn of its own, as it would behind any running turn. A wake that + // joined a held continuation is released beside it (#304), and one that + // waited behind a turn that then restarted is dequeued with the restart. + if (resumesLogicalTurn(trigger?.reason)) { + const batches = requests.filter((r) => r !== trigger && r.coalescingBatch === true); + if (batches.length > 0) { + this.pendingRequests.push(...batches); + requests = requests.filter((r) => !batches.includes(r)); + } + } // Route this turn's auto-published speech to the channel that triggered // it (item-3 redux). A batched wake may carry several triggering channels // (messages arrived in >1 channel while the agent was busy/idle): @@ -12423,9 +12476,9 @@ export class AgentFramework { } // Record turn checkpoint before inference (only on first attempt, not retries) - let checkpoint: number | undefined; + let checkpoint: RecordedTurnCheckpoint | undefined; if (attempt === 0 && trigger?.reason !== 'tool_result_guard_retry') { - checkpoint = this.recordTurnCheckpoint(agent.name); + checkpoint = { ...this.recordTurnCheckpoint(agent.name), redo: this.redoStacks.get(agent.name) }; this.redoStacks.delete(agent.name); // new work invalidates redo } diff --git a/src/types/agent.ts b/src/types/agent.ts index 708c7002f..ff5197f1f 100644 --- a/src/types/agent.ts +++ b/src/types/agent.ts @@ -351,6 +351,10 @@ export interface InferenceRequest { * all of them without materializing content, the turn has no cause left * and does not run. */ coalescingBatchSubjects?: string[]; + /** Framework-owned: a request a budget hold released (#304). Its turn gave + * back its token when it was held, so while the host is quiesced it parks + * as a wake does, a continuation included, and runs at resume. */ + releasedByBudgetHold?: boolean; agentName: string; reason: string; source: string; diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index b89b6f139..8afe99aca 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -203,12 +203,15 @@ class InboxModule implements Module { * through a fresh compile (physicalWindowTokens, #92). */ class BlobToolModule implements Module { readonly name = 'canned'; + /** Runs while the tool runs, with the turn alive. */ + constructor(private readonly duringCall?: () => void) {} async start(): Promise {} async stop(): Promise {} getTools(): ToolDefinition[] { return [{ name: 'fetch', description: 'blob', inputSchema: { type: 'object', properties: {} } }]; } async handleToolCall(): Promise { + this.duringCall?.(); return { success: true, data: { blob: 'x'.repeat(30_000) } }; } async onProcess(event: ProcessEvent): Promise { @@ -228,6 +231,73 @@ function nearCapToolResponse() { return r; } +/** + * A turn whose continuation is held: its first compile fits, its tool round's + * usage near a 200k window restarts it through a fresh compile + * (nearCapToolResponse), and that compile is refused for budget, as is every + * dry compile the release check falls back to (the strategy has no preview), + * until `fit()`. Compression is pending until then. Each response text is + * answered by one provider call after the first round's. + */ +async function heldRestart(texts: string[], options: { + /** The restart's compile fits from the start, so nothing is held. */ + fits?: boolean; + /** Runs while the first round's tool runs, with the turn alive. */ + duringTool?: (internal: Internal) => void; +} = {}) { + const proto = ContextManager.prototype as unknown as Patchable; + const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; + let fit = options.fits === true; + let committing = 0; + proto.compile = async function (this: unknown, ...args: unknown[]) { + const dryRun = (args[2] as { dryRun?: boolean } | undefined)?.dryRun === true; + const refused = dryRun ? !fit : ++committing >= 2 && !fit; + if (refused) throw overBudget(49_000); + return original.compile.apply(this, args); + }; + proto.tick = async function () {}; + proto.isReady = function () { return fit; }; + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-restart-')); + const membrane = new SequentialStreamMembrane(); + membrane.pushResponse(nearCapToolResponse()); + for (const text of texts) membrane.pushResponse(createMockResponse([{ type: 'text', text }])); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.', allowedTools: 'all', maxTokens: 4_000, physicalWindowTokens: 200_000 }], + modules: [new BlobToolModule(() => options.duringTool?.(internal))], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + errorPolicy: { maxRetries: 3, onInferenceError: (_error: Error, _agent: string, attempt: number) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const internal = framework as unknown as Internal; + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + let looks = 0; + const hold = framework as unknown as { checkBudgetHold?: (agentName: string) => Promise }; + const checkBudgetHold = hold.checkBudgetHold?.bind(framework); + if (checkBudgetHold) hold.checkBudgetHold = (agentName: string) => { looks++; return checkBudgetHold(agentName); }; + framework.start(); + framework.getAgent('prime')!.getContextManager().addMessage('user', [{ type: 'text', text: 'fetch it' }]); + internal.pendingRequests.push({ agentName: 'prime', reason: 'external-message', source: 'test', timestamp: Date.now() } as unknown as { agentName: string }); + return { + framework, + internal, + membrane, + of: (type: string) => traces.filter((e) => e.type === type) as unknown as Array>, + looks: () => looks, + fit: () => { fit = true; }, + close: async () => { + try { + await framework.stop(); + } finally { + Object.assign(proto, original); + rmSync(tempDir, { recursive: true, force: true }); + } + }, + }; +} + /** The framework's internals these tests read or shorten. */ interface Internal { budgetHoldMs: number; @@ -893,8 +963,8 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); it('on the fallback path, holds on when compression comes back during a dry compile that its progress does not show', async () => { - // Progress frozen: only whether compression is pending changes, and the - // signature doesn't see it. + // Progress frozen: only whether compression is pending changes, which + // the signature sees beside the frozen progress. await withBacklog({ previewless: true, stalled: true, alwaysPending: true, readyAt: 0 }, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 50 }); try { @@ -1690,6 +1760,162 @@ describe('a compile refused for budget holds its turn (#304)', () => { } }); }); + it('keeps a held continuation from running while the host is quiesced: it runs at resume', async () => { + const t = await heldRestart(['Answered after the restart']); + try { + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 10_000), 'the restart is held'); + const status = await t.framework.quiesce({ reason: 'test', timeoutMs: 1_000 }); + assert.equal(status.drained, true, 'the held turn holds no turn token'); + // Asserted below, once the provider calls are. + const heldAtQuiesce = status.heldTurns; + assert.equal(t.membrane.calls.length, 1); + t.fit(); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the context fits, and the hold ends'); + // The released continuation reaches the scheduler, which either starts + // it or parks it as it parks a wake. + await pollUntil(() => t.membrane.calls.length > 1 || t.framework.getHostModeStatus().gatedRequests === 1, 3_000); + assert.equal(t.membrane.calls.length, 1, 'no provider call after quiesce reported the host drained'); + assert.equal(t.framework.getHostModeStatus().gatedRequests, 1, 'the continuation is parked, as a wake is'); + assert.equal(heldAtQuiesce, 1, 'the status quiesce returned showed the held turn'); + // A parked request isn't work for runUntilIdle while the host is quiesced. + const idle = await Promise.race([ + t.framework.runUntilIdle().then(() => true), + new Promise((r) => setTimeout(() => r(false), 2_000)), + ]); + assert.equal(idle, true, 'runUntilIdle returns while the continuation waits for resume'); + await t.framework.resume(); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 5_000), 'the continuation runs at resume'); + assert.equal(t.membrane.calls.length, 2); + } finally { + await t.close(); + } + }); + + it('gives a deferred batch\'s wake that joins a held continuation a fresh turn of its own', async () => { + const t = await heldRestart(['Answered after the restart', 'Answered the batch']); + try { + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 10_000), 'the restart is held'); + t.internal.pendingRequests.push({ + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), + coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'], + } as unknown as { agentName: string }); + assert.ok(await pollUntil(() => holdOf(t)?.heldRequests.length === 2, 3_000), 'the batch\'s wake joins the hold'); + t.fit(); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 5_000), 'the continuation runs'); + // The continuation skips batch assembly, so the batch's wake can't be + // one of its causes: it starts a turn of its own once that one ends. + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered the batch'), 5_000), 'the batch\'s wake gets a fresh turn'); + assert.equal(t.membrane.calls.length, 3); + } finally { + await t.close(); + } + }); + + it('gives a deferred batch\'s wake that waited behind a turn which then restarted a fresh turn of its own', async () => { + // Nothing is held: the restart's compile fits. The batch's wake arrives + // while the turn's tool runs, so it waits, queued, until the restart is + // queued beside it. + const t = await heldRestart(['Answered after the restart', 'Answered the batch'], { + fits: true, + duringTool: (internal) => internal.pendingRequests.push({ + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), + coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'], + } as unknown as { agentName: string }), + }); + try { + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 10_000), 'the restart runs'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered the batch'), 5_000), 'the batch\'s wake gets a fresh turn'); + assert.equal(t.membrane.calls.length, 3); + assert.equal(t.of('inference:held').length, 0); + } finally { + await t.close(); + } + }); + + it('holds nothing in a framework that was never started: the refusal takes the error policy\'s path, as before', async () => { + await withBacklog({}, async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'budget-hold-unstarted-')); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'store'), + membrane: new SequentialStreamMembrane().asMembrane(), + agents: [{ name: 'prime', model: 'test-model', systemPrompt: 'You are prime.' }], + modules: [new InboxModule()], + syncIntervalMs: 0, + maintenanceIntervalMs: 50, + errorPolicy: { maxRetries: 3, onInferenceError: (_error: Error, _agent: string, attempt: number) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + }); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + try { + framework.pushEvent({ + type: 'external-message', source: 'test', content: [{ type: 'text', text: 'Are you there?' }], metadata: {}, triggerInference: true, + } as unknown as ProcessEvent); + await framework.runUntilIdle(); + assert.equal(traces.filter((e) => e.type === 'inference:held').length, 0, 'no run loop would run a released turn'); + assert.equal(traces.filter((e) => e.type === 'inference:exhausted').length, 1, 'the turn fails before runUntilIdle returns'); + } finally { + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + }); + + it('looks again when compression stops being pending though the strategy\'s progress snapshot stands still', async () => { + // A progress snapshot that doesn't move while readiness does. + await withBacklog({ progress: 1, readyAt: 30 }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.ok(await pollUntil(() => backlog.ticks >= 30, 5_000), 'compression stops being pending'); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the hold ends without waiting for its deadline'); + assert.equal(t.of('inference:hold_ended')[0]!.outcome, 'nothing_pending'); + } finally { + await t.close(); + } + }); + }); + + it('keeps the undo history whole at its limit: a held start gives back the checkpoint its record evicted', async () => { + await withBacklog({}, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + const checkpoints = t.framework as unknown as { + getTurnCheckpoints: (agentName: string) => Array<{ turnIndex: number }>; + saveTurnCheckpoints: (agentName: string, checkpoints: unknown[]) => void; + turnCounters: Map; + }; + try { + const full = Array.from({ length: 20 }, (_, i) => ({ agentName: 'prime', turnIndex: i, sequenceBefore: 0, branchName: 'main', timestamp: i })); + checkpoints.saveTurnCheckpoints('prime', full); + checkpoints.turnCounters.set('prime', 20); + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.deepEqual(checkpoints.getTurnCheckpoints('prime').map((c) => c.turnIndex), full.map((c) => c.turnIndex), + 'the twenty checkpoints from before the turn, the oldest included'); + } finally { + await t.close(); + } + }); + }); + + it('keeps the redo history: a held start gives back the redo its new work invalidated', async () => { + await withBacklog({ fit: true }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); + try { + t.send('A turn that fits'); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered once it fit'), 5_000)); + assert.equal(t.framework.undoLastTurn('prime').undone, true); + const before = t.framework.getUndoRedoState('prime'); + assert.equal(before.redoDepth, 1, 'the undone turn can be redone'); + backlog.fit = false; + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + assert.deepEqual(t.framework.getUndoRedoState('prime'), before, 'the held start left undo and redo as they were'); + } finally { + await t.close(); + } + }); + }); }); describe('the release check takes context-manager\'s verdict on the real strategy (#304)', () => { From 53d4609e02eafc8c8ef7e41206a9f03761242e14 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Sat, 17 Oct 2026 14:16:03 -0400 Subject: [PATCH 28/31] fix(inference): a released continuation parks at quiesce's admission recheck too, and the scheduler picks a resumed turn's trigger before the policy sees the batch beside it From an internal review of the previous commit. - The provider admission recheck now asks passesQuiesce, as the gate does. A continuation a budget hold released, which passed the scheduler before quiesce began and waits there on provider admission, holds no turn token, so it is requeued and parked like a wake rather than started while the host is quiesced. - The mark a hold puts on what it releases now ends where the turn starts and takes its token (startAgentStream), so the turn's trigger, and the guard retries and restarts made from it, carry none: they continue a turn that holds its token. - The scheduler now chooses the budget restart, and the turn's trigger, from the requests quiesce lets through, so a restart the gate parked never defines a turn. The trigger is the restart, else any request that resumes a logical turn, else the first: a guard retry a hold released resumes its turn wherever it was queued. The deferred batches beside a resuming trigger are requeued right there, before the busy check and the inference policy, so a policy that declines the resumed turn no longer drops them, and each gets its own decision. - dropTurnCheckpoint also puts back the turn index the dropped record took. Tests: a released restart waiting on provider admission when the host quiesces is parked, counted among gatedRequests, and runs at resume, and the turn it starts carries no mark; a released guard retry queued behind a batch's wake is its turn's trigger, and the wake gets its own turn; a batch's wake beside a restart the policy declines gets a turn of its own; the held start's turn index is put back; heldTurns is 0 once the hold ends; the never-started refusal is over budget; and the quiesce test's bound on runUntilIdle is cleared. At a1b459cc the four tests of the code changes fail and the file's other 61 pass. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 16 +-- src/framework.ts | 97 ++++++++++-------- test/budget-refusal-hold.test.ts | 98 ++++++++++++++++++- 3 files changed, 160 insertions(+), 51 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index a491115fc..17b16cf9f 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -47,12 +47,16 @@ once per 15 minutes for an agent), and says the turn is held until its deadline. A hold's drain kicks log as `[budget-hold]`. While the host is quiesced, a turn a hold releases waits for resume as a parked wake does, - a continuation included, so no turn starts after `quiesce()` reported the - host drained. `getHostModeStatus()` counts it among `gatedRequests`, and - its new `heldTurns` counts the turns still held. + a continuation included, whether it's still queued or already waiting + on provider admission, so no such turn starts while the host is + quiesced. `getHostModeStatus()` counts it among `gatedRequests`, and its + new `heldTurns` counts the turns still held. - A deferred batch's wake that waited behind a turn which then restarted for its context budget was consumed by the restarted turn, which continues the same logical turn and skips batch assembly, so the batch never rendered - until some later turn. It now stays queued and wakes a fresh turn of its - own once the restarted turn ends. A wake that joins a held continuation - (above) is kept out of it the same way. + until some later turn, or was dropped with it when the inference policy + declined the restart. It now stays queued, with its own policy decision, + and wakes a fresh turn of its own once the restarted turn ends. A wake + that joins a held continuation (above) is kept out of it the same way, + and a continuation a hold releases is its turn's trigger wherever it was + queued. diff --git a/src/framework.ts b/src/framework.ts index 1d9949e99..a27fe154a 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -3473,7 +3473,8 @@ export class AgentFramework { // refusal decides nothing if something in the signature moved while it // ran (compression's progress or whether it has work, the store, a // surgery whose own look this check turned away): the next look takes a - // fresh verdict. So `pending` is still current below. + // fresh verdict. So `pending` is still current below, as far as the + // signature can be read. if (verdict === 'refused' && this.budgetHoldSignature(agent) !== signature) return; } // A refusal is final only if compression has nothing pending. @@ -9132,9 +9133,9 @@ export class AgentFramework { } /** Drop the agent's newest turn checkpoint, if it is the one this start - * recorded, and put back what recording it took, the checkpoints it - * evicted and the redo it invalidated, so that undo and redo are as they - * were before the start (holdForBudgetRefusal). */ + * recorded, and put back what recording it took: the checkpoints it + * evicted, the redo it invalidated and its turn index, so that undo and + * redo are as they were before the start (holdForBudgetRefusal). */ private dropTurnCheckpoint(agentName: string, recorded: RecordedTurnCheckpoint): void { const checkpoints = this.getTurnCheckpoints(agentName); if (checkpoints.at(-1)?.turnIndex !== recorded.turnIndex) return; @@ -9142,6 +9143,7 @@ export class AgentFramework { checkpoints.unshift(...recorded.evicted); this.saveTurnCheckpoints(agentName, checkpoints); if (recorded.redo) this.redoStacks.set(agentName, recorded.redo); + this.turnCounters.set(agentName, recorded.turnIndex); } /** @@ -11274,23 +11276,6 @@ export class AgentFramework { const providerWait = this.providerWaits?.active(agentName, agent.model); if (providerWait && !this.ephemeralRuns.has(agentName) && this.holdForProviderWait(agent, providerWait, requests)) continue; - // Skip if agent is busy (inferring, streaming, or waiting for tools) — - // or if a turn is alive at all (activeTurnTokens): the state machine - // goes idle at endTurn's reset while driveStream teardown is still - // pending, and a successor turn dequeued in that gap compiles while the - // predecessor's deferred flush can still write (2026-07-31 Mythos). - // Turn-alive spans dequeue → settled teardown; wakes wait it out. - // - // EXCEPT a context-budget restart: it CONTINUES the turn whose token is - // held — queueing it behind its own turn's teardown is a deadlock when - // that teardown lags (the cancelled stream's `aborted` event is - // asynchronous and, historically, not even guaranteed — see the - // overBudget branch). The restart's startAgentStream overwrites the - // token; the predecessor frame's late finally no-ops via token-match. - // This mirrors how the restart has always overwritten activeStreams - // rather than waiting for the old stream's teardown. - const budgetRestart = requests.find((r) => r.reason === 'context_budget_restart'); - // Host quiesce (issue #122): park wakes instead of starting turns — // the requeue mirrors the busy path below, so requests survive and // fire at resume. A context-budget restart passes through: it @@ -11329,6 +11314,46 @@ export class AgentFramework { requests.push(...passThrough); } + // Skip if agent is busy (inferring, streaming, or waiting for tools) — + // or if a turn is alive at all (activeTurnTokens): the state machine + // goes idle at endTurn's reset while driveStream teardown is still + // pending, and a successor turn dequeued in that gap compiles while the + // predecessor's deferred flush can still write (2026-07-31 Mythos). + // Turn-alive spans dequeue → settled teardown; wakes wait it out. + // + // EXCEPT a context-budget restart: it CONTINUES the turn whose token is + // held — queueing it behind its own turn's teardown is a deadlock when + // that teardown lags (the cancelled stream's `aborted` event is + // asynchronous and, historically, not even guaranteed — see the + // overBudget branch). The restart's startAgentStream overwrites the + // token; the predecessor frame's late finally no-ops via token-match. + // This mirrors how the restart has always overwritten activeStreams + // rather than waiting for the old stream's teardown. Taken from the + // requests quiesce let through, so a restart it parked defines nothing. + const budgetRestart = requests.find((r) => r.reason === 'context_budget_restart'); + // The request that grants the turn-alive exception must also define the + // downstream turn semantics. An ordinary wake can already be pending + // when the tool boundary appends a context-budget restart; selecting + // requests[0] in that mixed batch bypassed the turn lock because a + // restart existed, then treated the continuation as a fresh turn. So + // does any request that resumes a logical turn, such as a guard retry a + // budget hold released, wherever it was queued. + const trigger = budgetRestart ?? requests.find((r) => resumesLogicalTurn(r.reason)) ?? requests[0]; + // A turn that resumes a logical turn skips batch assembly + // (beginAgentTurn), so a deferred batch's wake can't be one of its + // causes: its batch would never render. It stays queued for a fresh + // turn of its own, as it would behind any running turn, and its own + // policy decision. This covers a wake that waited behind a turn which + // then restarted, and one that joined a held continuation and was + // released beside it (#304). + if (resumesLogicalTurn(trigger?.reason)) { + const batches = requests.filter((r) => r !== trigger && r.coalescingBatch === true); + if (batches.length > 0) { + this.pendingRequests.push(...batches); + requests = requests.filter((r) => !batches.includes(r)); + } + } + const turnAlive = !budgetRestart && this.activeTurnTokens.has(agentName); const providerGate = this.providerGates?.get(agentName); // A primary can own provider admission while yielding to an already in-flight @@ -11413,25 +11438,7 @@ export class AgentFramework { } // Start streaming inference (non-blocking — driveStream runs in background) - // The request that grants the turn-alive exception must also define the - // downstream turn semantics. An ordinary wake can already be pending - // when the tool boundary appends a context-budget restart; selecting - // requests[0] in that mixed batch bypassed the turn lock because a - // restart existed, then treated the continuation as a fresh turn. - const trigger = budgetRestart ?? requests[0]; - // A turn that resumes a logical turn skips batch assembly - // (beginAgentTurn), so a deferred batch's wake can't be one of its - // causes: its batch would never render. It stays queued for a fresh - // turn of its own, as it would behind any running turn. A wake that - // joined a held continuation is released beside it (#304), and one that - // waited behind a turn that then restarted is dequeued with the restart. - if (resumesLogicalTurn(trigger?.reason)) { - const batches = requests.filter((r) => r !== trigger && r.coalescingBatch === true); - if (batches.length > 0) { - this.pendingRequests.push(...batches); - requests = requests.filter((r) => !batches.includes(r)); - } - } + // from the trigger chosen above. // Route this turn's auto-published speech to the channel that triggered // it (item-3 redux). A batched wake may carry several triggering channels // (messages arrived in >1 channel while the agent was busy/idle): @@ -12227,8 +12234,9 @@ export class AgentFramework { // park; starting here would bypass it and run a turn against a // context the operator believes frozen. Give admission back and // requeue — the scheduler re-parks it (coalesced) until resume. - // Continuations of a held turn pass, as they do at the gate. - if (this.quiesced && !isTurnContinuation(trigger?.reason ?? '')) { + // A continuation of a turn whose token is held passes, as at the + // gate; one a budget hold released parks, as at the gate. + if (this.quiesced && !(trigger && passesQuiesce(trigger))) { this.releasePrimaryProviderGate(agent.name); this.pendingRequests.push(trigger ?? { agentName: agent.name, reason: 'provider-admission:requeue', source: 'scheduler', timestamp: Date.now(), @@ -12293,6 +12301,11 @@ export class AgentFramework { // clears our own token. Token-matched, so when a retry frame has replaced // the token, or the terminal-failure branch already deleted it, this // no-ops instead of clobbering a successor's marker. + // The turn starts here and holds its token, so a budget hold's mark on + // the request that started it ends here: nothing the turn retries or + // queues from its trigger is parked by quiesce as the release was + // (passesQuiesce). + if (trigger?.releasedByBudgetHold) trigger = { ...trigger, releasedByBudgetHold: undefined }; const turnToken = this.nextTurnToken++; this.activeTurnTokens.set(agent.name, turnToken); this.activeTurnTriggers.set(agent.name, trigger); diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 8afe99aca..1527f8b7e 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -244,6 +244,7 @@ async function heldRestart(texts: string[], options: { fits?: boolean; /** Runs while the first round's tool runs, with the turn alive. */ duringTool?: (internal: Internal) => void; + inferencePolicy?: InferencePolicy; } = {}) { const proto = ContextManager.prototype as unknown as Patchable; const original = { compile: proto.compile, tick: proto.tick, isReady: proto.isReady }; @@ -269,6 +270,7 @@ async function heldRestart(texts: string[], options: { syncIntervalMs: 0, maintenanceIntervalMs: 50, errorPolicy: { maxRetries: 3, onInferenceError: (_error: Error, _agent: string, attempt: number) => (attempt < 3 ? { retry: true, delayMs: 0 } : { retry: false }) }, + ...(options.inferencePolicy ? { inferencePolicy: options.inferencePolicy } : {}), }); const internal = framework as unknown as Internal; const traces: TraceEvent[] = []; @@ -1771,17 +1773,21 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(t.membrane.calls.length, 1); t.fit(); assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the context fits, and the hold ends'); + assert.equal(t.framework.getHostModeStatus().heldTurns, 0, 'no turn is held once the hold ends'); // The released continuation reaches the scheduler, which either starts // it or parks it as it parks a wake. await pollUntil(() => t.membrane.calls.length > 1 || t.framework.getHostModeStatus().gatedRequests === 1, 3_000); assert.equal(t.membrane.calls.length, 1, 'no provider call after quiesce reported the host drained'); assert.equal(t.framework.getHostModeStatus().gatedRequests, 1, 'the continuation is parked, as a wake is'); assert.equal(heldAtQuiesce, 1, 'the status quiesce returned showed the held turn'); - // A parked request isn't work for runUntilIdle while the host is quiesced. + // A parked request isn't work for runUntilIdle while the host is + // quiesced: it returns at once, well inside this bound. + let bound: ReturnType | undefined; const idle = await Promise.race([ t.framework.runUntilIdle().then(() => true), - new Promise((r) => setTimeout(() => r(false), 2_000)), + new Promise((r) => { bound = setTimeout(() => r(false), 10_000); }), ]); + clearTimeout(bound); assert.equal(idle, true, 'runUntilIdle returns while the continuation waits for resume'); await t.framework.resume(); assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 5_000), 'the continuation runs at resume'); @@ -1852,7 +1858,9 @@ describe('a compile refused for budget holds its turn (#304)', () => { } as unknown as ProcessEvent); await framework.runUntilIdle(); assert.equal(traces.filter((e) => e.type === 'inference:held').length, 0, 'no run loop would run a released turn'); - assert.equal(traces.filter((e) => e.type === 'inference:exhausted').length, 1, 'the turn fails before runUntilIdle returns'); + const exhausted = traces.filter((e) => e.type === 'inference:exhausted') as unknown as Array<{ errorType?: string }>; + assert.equal(exhausted.length, 1, 'the turn fails before runUntilIdle returns'); + assert.equal(exhausted[0]!.errorType, 'over_budget', 'from the refusal, by the error policy\'s path'); } finally { await framework.stop(); rmSync(tempDir, { recursive: true, force: true }); @@ -1892,6 +1900,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); assert.deepEqual(checkpoints.getTurnCheckpoints('prime').map((c) => c.turnIndex), full.map((c) => c.turnIndex), 'the twenty checkpoints from before the turn, the oldest included'); + assert.equal(checkpoints.turnCounters.get('prime'), 20, 'and the turn index the held start took'); } finally { await t.close(); } @@ -1916,6 +1925,89 @@ describe('a compile refused for budget holds its turn (#304)', () => { } }); }); + + it('keeps a released continuation that waits on provider admission from starting while the host is quiesced', async () => { + const t = await heldRestart(['Answered after the restart']); + const internals = t.framework as unknown as { + withAuxiliaryAdmission(agentName: string, run: () => Promise): Promise; + providerGates: Map; + activeTurnTriggers: Map; + }; + // What the turn the release starts carries, sampled as it starts. + const started: Array = []; + t.framework.onTrace((e) => { + if (e.type === 'inference:started' && t.of('inference:hold_ended').length === 1) { + started.push(internals.activeTurnTriggers.get('prime')?.releasedByBudgetHold); + } + }); + let releaseAux = () => {}; + try { + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 10_000), 'the restart is held'); + // An auxiliary call holds provider admission, so the released restart + // passes the scheduler and waits behind it, with no turn token. + const auxDone = internals.withAuxiliaryAdmission('prime', () => new Promise((resolve) => { releaseAux = resolve; })); + t.fit(); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1 + && (internals.providerGates.get('prime')?.primaryDepth ?? 0) === 1 + && !t.internal.activeTurnTokens.has('prime'), 5_000), 'released, and waiting on admission'); + const status = await t.framework.quiesce({ reason: 'test', timeoutMs: 1_000 }); + assert.equal(status.parkedAdmissions, 1); + releaseAux(); + await auxDone; + await pollUntil(() => t.membrane.calls.length > 1 || t.framework.getHostModeStatus().gatedRequests === 1, 3_000); + assert.equal(t.membrane.calls.length, 1, 'no turn starts while the host is quiesced'); + assert.equal(t.framework.getHostModeStatus().gatedRequests, 1, 'the continuation is parked, as a wake is'); + await t.framework.resume(); + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 5_000), 'it runs at resume'); + assert.deepEqual(started, [undefined], 'the turn it starts carries no mark, so nothing it retries is parked'); + } finally { + releaseAux(); + await t.close(); + } + }); + + it('starts a released guard retry as its turn\'s trigger wherever it was queued, and a batch\'s wake ahead of it gets a fresh turn', async () => { + await withBacklog({ fit: true }, async () => { + const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, responses: 2 }); + const internals = t.framework as unknown as { activeTurnTriggers: Map }; + const triggers: Array = []; + t.framework.onTrace((e) => { if (e.type === 'inference:started') triggers.push(internals.activeTurnTriggers.get('prime')?.reason); }); + try { + // As a hold releases a held guard retry while a batch's wake is + // already queued ahead of it. + t.internal.pendingRequests.push( + { + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), + coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'], + } as unknown as { agentName: string }, + { agentName: 'prime', reason: 'tool_result_guard_retry', source: 'framework', timestamp: Date.now(), releasedByBudgetHold: true } as unknown as { agentName: string }, + ); + assert.ok(await pollUntil(() => t.membrane.calls.length === 2, 5_000), 'two turns'); + assert.deepEqual(triggers, ['tool_result_guard_retry', 'mcpl:push-event'], 'the guard retry resumes its turn, then the batch\'s wake gets its own'); + } finally { + await t.close(); + } + }); + }); + + it('keeps a deferred batch\'s wake beside a restart the inference policy declines queued for a turn of its own', async () => { + // The policy declines any turn a budget restart would start. + const policy: InferencePolicy = { shouldInfer: (_agentName, requests) => !requests.some((r) => r.reason === 'context_budget_restart') }; + const t = await heldRestart(['Answered the batch'], { + fits: true, + inferencePolicy: policy, + duringTool: (internal) => internal.pendingRequests.push({ + agentName: 'prime', reason: 'mcpl:push-event', source: 'test', timestamp: Date.now(), + coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'], + } as unknown as { agentName: string }), + }); + try { + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered the batch'), 10_000), 'the batch\'s wake gets a turn of its own'); + assert.equal(t.membrane.calls.length, 2, 'the first round, then the batch\'s turn; the declined restart made no call'); + } finally { + await t.close(); + } + }); }); describe('the release check takes context-manager\'s verdict on the real strategy (#304)', () => { From c7ae3d80dbb01995bdbb66fa98f4f40d03cbb8d5 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Sat, 17 Oct 2026 15:04:29 -0400 Subject: [PATCH 29/31] fix(inference): a released restart waits for a token another holds, a held start leaves redo invalidated, and text the review found loose From an internal review of everything since the last reviewed head. - A restart a budget hold released no longer gets the scheduler's turn-lock bypass. That bypass is for a restart that continues a turn still holding its token; a held restart's turn gave its token back when it was held, so a token held when the hold releases it is another holder's, such as a puppeted tool call's reservation, and the restart started over it. It now waits, as a wake does. - A held start no longer puts back the redo its new work invalidated. The held turn's input is stored, and a redo while it waited moved the store from under it; the redo stays invalidated, as at any turn's first start. The start still gives back the checkpoint it recorded, the oldest one it evicted at the limit of 20 and its turn index. - Quiesce's log line names the turns held for budget, as the host status does. - Text: the scheduler's quiesce comment (a continuation passes, one a hold released parks, the coalescing key), a dangling stream-start fragment, kickOverBudgetDrain's sentence on a stop during a tick (it came over from the breaker's comment), compressForBudgetHold's doc on a declined kick, the changelog's "looks again only once" (it takes a fresh verdict only once), its undo and redo sentence, and the scope of a guard retry as its turn's trigger. Tests: a released restart while a puppeted tool call holds the agent waits until the call ends; the redo test is turned round to assert redo invalidated and undo as it was; the quiesce test asserts the log line; the test module's tool awaits what its callback returns. At 53d4609e the three of these that test changed behaviour fail and the file's other 63 pass. Co-Authored-By: Claude Opus 5.5 --- .../budget-refusal-holds-turn.fixed.md | 31 +++++---- src/framework.ts | 69 ++++++++++--------- test/budget-refusal-hold.test.ts | 64 ++++++++++++++--- 3 files changed, 109 insertions(+), 55 deletions(-) diff --git a/changelog.d/budget-refusal-holds-turn.fixed.md b/changelog.d/budget-refusal-holds-turn.fixed.md index 17b16cf9f..e94aa7f69 100644 --- a/changelog.d/budget-refusal-holds-turn.fixed.md +++ b/changelog.d/budget-refusal-holds-turn.fixed.md @@ -16,12 +16,12 @@ gets the dry compile. The hierarchical strategy's preview reports a context that doesn't fit, and for one that does, the dry compile decides. The hold looks after every maintenance pass, every drain kick it starts, - and every surgery, undo and redo. It looks again only once something that - could make the context fit has changed: compression's progress or whether - it still has compression pending, a message added, edited or removed, the - agent's runtime settings, or a surgery on the store. A turn released and - refused again is held again with its first deadline, and isn't released - again until one of those changes. + and every surgery, undo and redo. It takes a fresh verdict only once + something that could make the context fit has changed: compression's + progress or whether it still has compression pending, a message added, + edited or removed, the agent's runtime settings, or a surgery on the + store. A turn released and refused again is held again with its first + deadline, and isn't released again until one of those changes. - **When it ends without a fit:** 30 minutes after the refusal that started it, or at once when compression has nothing pending. The turn then runs if its context fits by then, and otherwise fails as a refused @@ -32,11 +32,12 @@ why it waited, for that compile alone: "[budget-hold] Your context didn't fit its budget (100,201 tokens against 48,568), so this turn waited 7 min. It fits now." It's never stored. One held turn is one undo step, - which keeps the messages that arrived while it waited, and the refused - start leaves undo and redo as they were, the oldest checkpoint at the - limit of 20 included. An undo while a turn is held rewinds past that - turn's messages, and the held turn then runs on what the undo left (#381 - is that for every pending wake). + which keeps the messages that arrived while it waited. The refused start + gives back the checkpoint it recorded, with the oldest one it evicted at + the limit of 20, and the redo its new work invalidated stays invalidated, + as at any turn's first start. An undo while a turn is held rewinds past + that turn's messages, and the held turn then runs on what the undo left + (#381 is that for every pending wake). - **What hosts see:** the refused attempt emits `inference:failed`, and the turn's next start emits `inference:started` again. The turn's terminal event comes when the held turn ends, after a hold of up to 30 minutes, @@ -50,7 +51,9 @@ a continuation included, whether it's still queued or already waiting on provider admission, so no such turn starts while the host is quiesced. `getHostModeStatus()` counts it among `gatedRequests`, and its - new `heldTurns` counts the turns still held. + new `heldTurns` counts the turns still held, as quiesce's log line does. + A budget restart a hold releases waits, as a wake does, for a turn token + another holds, such as a puppeted tool call's. - A deferred batch's wake that waited behind a turn which then restarted for its context budget was consumed by the restarted turn, which continues the same logical turn and skips batch assembly, so the batch never rendered @@ -58,5 +61,5 @@ declined the restart. It now stays queued, with its own policy decision, and wakes a fresh turn of its own once the restarted turn ends. A wake that joins a held continuation (above) is kept out of it the same way, - and a continuation a hold releases is its turn's trigger wherever it was - queued. + and a guard retry a hold releases is its turn's trigger wherever it was + queued, as a budget restart already was. diff --git a/src/framework.ts b/src/framework.ts index a27fe154a..d29361e65 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -677,14 +677,12 @@ interface TurnCheckpoint { timestamp: number; } -/** A turn checkpoint as its start recorded it: its turn index, the oldest - * checkpoints the record evicted at the depth limit, and the redo entries - * the start's new work invalidated, all of which dropping it puts back - * (dropTurnCheckpoint). */ +/** A turn checkpoint as its start recorded it: its turn index and the oldest + * checkpoints the record evicted at the depth limit, both of which dropping + * it puts back (dropTurnCheckpoint). */ interface RecordedTurnCheckpoint { turnIndex: number; evicted: TurnCheckpoint[]; - redo?: RedoEntry[]; } type DiscordAwarenessDrainOutcome = @@ -3347,8 +3345,8 @@ export class AgentFramework { ? { ...trigger, coalescingSubject: undefined, coalescingBatch: undefined, coalescingBatchSubjects: undefined } : { agentName: agent.name, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now() }; // One held turn is one undo step: the refused start's checkpoint goes, - // with what recording it took back (the checkpoints it evicted, the redo - // it invalidated), and the released start records the turn's, after the + // with what recording it took back (the checkpoints it evicted, its turn + // index), and the released start records the turn's, after the // inputs that joined it. A start can reach its compile while a hold waits // (one parked for the primary provider's admission resumes without the // scheduler's look at the hold), and joins it, its checkpoint going too. @@ -3579,10 +3577,12 @@ export class AgentFramework { } /** - * Kick the drain for a held turn (kickOverBudgetDrain), then look again. A - * provider gate closed to compression would park the kick's calls there, - * where a surgery withdraws them (#352): instead it only looks, and the - * next pass or interval kicks. + * Kick the drain for a held turn (kickOverBudgetDrain), and look again when + * the kick ends. A kick declined once stop() has begun, or under a surgery, + * leaves no look here: stop() drops the hold, and the surgery's release + * looks again. A provider gate closed to compression would park the kick's + * calls there, where a surgery withdraws them (#352): instead it only looks, + * and the next pass or interval kicks. */ private compressForBudgetHold(agent: Agent): void { if (this.budgetHolds?.get(agent.name)?.state !== 'holding' || this.agents.get(agent.name) !== agent) return; @@ -3725,9 +3725,10 @@ export class AgentFramework { * The kick joins the drains they wait for before its body runs, and before * its caller goes on (the breaker raises its alert next), so an alert * listener that begins stop() or a surgery waits for it, and its body then - * ticks none and says why. So does a stop() made from inside its first - * tick, before that tick's first await. Returns what it did, for the - * breaker's alert. + * ticks none and says why. A stop() or surgery begun during a tick lets that + * tick finish, unless a compression call it makes after that is withdrawn + * at provider admission, and ends the kick there. Returns what it did, for + * the breaker's alert. */ private kickOverBudgetDrain( agentName: string, @@ -5319,7 +5320,8 @@ export class AgentFramework { }); console.error( status.drained - ? `[host-mode] quiesced — drained, ${status.gatedRequests} wake(s) parked` + ? `[host-mode] quiesced — drained, ${status.gatedRequests} wake(s) parked` + + (status.heldTurns > 0 ? `, ${status.heldTurns} turn(s) held for budget` : '') : `[host-mode] quiesced but drain ${abandoned ? 'needed abandon and' : 'timed out —'} ` + `${status.activeTurns} turn(s) still alive` + (status.parkedAdmissions > 0 @@ -9133,16 +9135,17 @@ export class AgentFramework { } /** Drop the agent's newest turn checkpoint, if it is the one this start - * recorded, and put back what recording it took: the checkpoints it - * evicted, the redo it invalidated and its turn index, so that undo and - * redo are as they were before the start (holdForBudgetRefusal). */ + * recorded, and put back what recording it took, the checkpoints it + * evicted and its turn index, so that undo is as it was before the start + * (holdForBudgetRefusal). The redo the start's new work invalidated stays + * invalidated, as at any turn's first start: the held turn's input is + * stored, and a redo would move the store from under it. */ private dropTurnCheckpoint(agentName: string, recorded: RecordedTurnCheckpoint): void { const checkpoints = this.getTurnCheckpoints(agentName); if (checkpoints.at(-1)?.turnIndex !== recorded.turnIndex) return; checkpoints.pop(); checkpoints.unshift(...recorded.evicted); this.saveTurnCheckpoints(agentName, checkpoints); - if (recorded.redo) this.redoStacks.set(agentName, recorded.redo); this.turnCounters.set(agentName, recorded.turnIndex); } @@ -11278,16 +11281,15 @@ export class AgentFramework { // Host quiesce (issue #122): park wakes instead of starting turns — // the requeue mirrors the busy path below, so requests survive and - // fire at resume. A context-budget restart passes through: it - // CONTINUES a turn whose token is held (see the deadlock note below), - // and quiesce's drain is exactly the phase where such a turn must be - // allowed to finish. Parked requests are coalesced per reason (newest - // kept): heartbeat/gate wakes arrive once per tick and nothing else - // bounds a multi-day maintenance window's accumulation. + // fire at resume. A continuation passes through: it CONTINUES a turn + // whose token is held (see the deadlock note below), and quiesce's + // drain is exactly the phase where such a turn must be allowed to + // finish. One a budget hold released continues a turn that gave its + // token back, so it parks (passesQuiesce). Parked requests are + // coalesced per reason, addressed or not, and coalescing subject + // (newest kept): heartbeat/gate wakes arrive once per tick and nothing + // else bounds a multi-day maintenance window's accumulation. if (this.quiesced) { - // Continuations of a turn whose token is held (budget restart, - // non-streaming tool results) pass; everything else parks, a - // continuation a budget hold released included (passesQuiesce). const passThrough = requests.filter((r) => passesQuiesce(r)); const parked = requests.filter((r) => !passesQuiesce(r)); // Coalesce per (reason, addressed): every channel wake shares one @@ -11354,7 +11356,10 @@ export class AgentFramework { } } - const turnAlive = !budgetRestart && this.activeTurnTokens.has(agentName); + // A restart a budget hold released continues a turn that gave its token + // back when it was held, so a token held now is another holder's, such + // as a puppeted tool call's reservation (#145): it waits, as a wake does. + const turnAlive = !(budgetRestart && !budgetRestart.releasedByBudgetHold) && this.activeTurnTokens.has(agentName); const providerGate = this.providerGates?.get(agentName); // A primary can own provider admission while yielding to an already in-flight // auxiliary call. Keep this agent's later wakes queued, but do not block the @@ -11437,8 +11442,8 @@ export class AgentFramework { continue; } - // Start streaming inference (non-blocking — driveStream runs in background) - // from the trigger chosen above. + // Start streaming inference (non-blocking — driveStream runs in background), + // with the trigger chosen above as its cause. // Route this turn's auto-published speech to the channel that triggered // it (item-3 redux). A batched wake may carry several triggering channels // (messages arrived in >1 channel while the agent was busy/idle): @@ -12491,7 +12496,7 @@ export class AgentFramework { // Record turn checkpoint before inference (only on first attempt, not retries) let checkpoint: RecordedTurnCheckpoint | undefined; if (attempt === 0 && trigger?.reason !== 'tool_result_guard_retry') { - checkpoint = { ...this.recordTurnCheckpoint(agent.name), redo: this.redoStacks.get(agent.name) }; + checkpoint = this.recordTurnCheckpoint(agent.name); this.redoStacks.delete(agent.name); // new work invalidates redo } diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 1527f8b7e..74b5853a6 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -203,15 +203,16 @@ class InboxModule implements Module { * through a fresh compile (physicalWindowTokens, #92). */ class BlobToolModule implements Module { readonly name = 'canned'; - /** Runs while the tool runs, with the turn alive. */ - constructor(private readonly duringCall?: () => void) {} + /** Runs while the tool runs, with the turn alive; the tool waits for what + * it returns. */ + constructor(private readonly duringCall?: () => unknown) {} async start(): Promise {} async stop(): Promise {} getTools(): ToolDefinition[] { return [{ name: 'fetch', description: 'blob', inputSchema: { type: 'object', properties: {} } }]; } async handleToolCall(): Promise { - this.duringCall?.(); + await this.duringCall?.(); return { success: true, data: { blob: 'x'.repeat(30_000) } }; } async onProcess(event: ProcessEvent): Promise { @@ -242,8 +243,9 @@ function nearCapToolResponse() { async function heldRestart(texts: string[], options: { /** The restart's compile fits from the start, so nothing is held. */ fits?: boolean; - /** Runs while the first round's tool runs, with the turn alive. */ - duringTool?: (internal: Internal) => void; + /** Runs while the tool runs, the first round's or a puppeted call's; the + * tool waits for what it returns. */ + duringTool?: (internal: Internal) => unknown; inferencePolicy?: InferencePolicy; } = {}) { const proto = ContextManager.prototype as unknown as Patchable; @@ -1766,10 +1768,14 @@ describe('a compile refused for budget holds its turn (#304)', () => { const t = await heldRestart(['Answered after the restart']); try { assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 10_000), 'the restart is held'); - const status = await t.framework.quiesce({ reason: 'test', timeoutMs: 1_000 }); + const { status, said } = await errorLines(async (lines) => ({ + status: await t.framework.quiesce({ reason: 'test', timeoutMs: 1_000 }), + said: lines.filter((l) => l.startsWith('[host-mode] quiesced')), + })); assert.equal(status.drained, true, 'the held turn holds no turn token'); // Asserted below, once the provider calls are. const heldAtQuiesce = status.heldTurns; + const saidAtQuiesce = said; assert.equal(t.membrane.calls.length, 1); t.fit(); assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the context fits, and the hold ends'); @@ -1780,8 +1786,10 @@ describe('a compile refused for budget holds its turn (#304)', () => { assert.equal(t.membrane.calls.length, 1, 'no provider call after quiesce reported the host drained'); assert.equal(t.framework.getHostModeStatus().gatedRequests, 1, 'the continuation is parked, as a wake is'); assert.equal(heldAtQuiesce, 1, 'the status quiesce returned showed the held turn'); + assert.deepEqual(saidAtQuiesce, ['[host-mode] quiesced — drained, 0 wake(s) parked, 1 turn(s) held for budget'], 'and so did its line'); // A parked request isn't work for runUntilIdle while the host is - // quiesced: it returns at once, well inside this bound. + // quiesced, so it returns rather than waiting for resume; the bound + // only stops a regression from hanging the test. let bound: ReturnType | undefined; const idle = await Promise.race([ t.framework.runUntilIdle().then(() => true), @@ -1907,7 +1915,7 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); - it('keeps the redo history: a held start gives back the redo its new work invalidated', async () => { + it('invalidates redo at a held start, as at any turn\'s first start, and leaves undo as it was', async () => { await withBacklog({ fit: true }, async (backlog) => { const t = await residentTurn({ maintenanceIntervalMs: 3_600_000 }); try { @@ -1919,7 +1927,10 @@ describe('a compile refused for budget holds its turn (#304)', () => { backlog.fit = false; t.send('Are you there?'); assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); - assert.deepEqual(t.framework.getUndoRedoState('prime'), before, 'the held start left undo and redo as they were'); + // The held turn's input is stored, and a redo would move the store + // from under it. + assert.deepEqual(t.framework.getUndoRedoState('prime'), { ...before, canRedo: false, redoDepth: 0 }, + 'the held start invalidated redo and left undo as it was'); } finally { await t.close(); } @@ -1966,6 +1977,41 @@ describe('a compile refused for budget holds its turn (#304)', () => { } }); + it('keeps a released budget restart from starting while a puppeted tool call holds the agent', async () => { + // Armed below: the puppeted call's tool waits for it. + let gate: Promise | undefined; + let letGo = () => {}; + const t = await heldRestart(['Answered after the restart'], { duringTool: () => gate }); + const internals = t.framework as unknown as { processInferenceRequests: () => Promise }; + const pass = internals.processInferenceRequests.bind(t.framework); + let passes = 0; + internals.processInferenceRequests = () => { passes++; return pass(); }; + let puppet: Promise | undefined; + try { + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 10_000), 'the restart is held'); + // The held agent is idle, so the puppeted call is admitted, and it + // holds the agent's turn token while its tool runs (#145). + gate = new Promise((resolve) => { letGo = resolve; }); + puppet = t.framework.puppetToolCall('prime', 'canned--fetch', {}); + assert.ok(await pollUntil(() => t.internal.activeTurnTokens.has('prime'), 3_000), 'the puppeted call holds the agent'); + t.fit(); + assert.ok(await pollUntil(() => t.of('inference:hold_ended').length === 1, 3_000), 'the context fits, and the hold ends'); + // The released restart continued a turn that gave its token back, so + // the scheduler keeps it queued behind the puppeted call, pass after pass. + const ended = passes; + assert.ok(await pollUntil(() => passes >= ended + 5 || t.membrane.calls.length > 1, 3_000)); + assert.equal(t.membrane.calls.length, 1, 'no turn starts while the puppeted call holds the agent'); + letGo(); + await puppet; + assert.ok(await pollUntil(() => storedText(t.framework).includes('Answered after the restart'), 5_000), 'the restart runs once the call ends'); + assert.equal(t.membrane.calls.length, 2); + } finally { + letGo(); + await puppet?.catch(() => {}); + await t.close(); + } + }); + it('starts a released guard retry as its turn\'s trigger wherever it was queued, and a batch\'s wake ahead of it gets a fresh turn', async () => { await withBacklog({ fit: true }, async () => { const t = await residentTurn({ maintenanceIntervalMs: 3_600_000, responses: 2 }); From a6800f967b880b5fd6d610b805dd3691da86b7a5 Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Mon, 19 Oct 2026 05:13:06 -0400 Subject: [PATCH 30/31] test(inference): a held turn's drop as a stop step, and stops that reach a refused or released turn's start Three tests for what the merge of main composed: - stop-step-throws gains a row for a budget hold's drop. Of two holds' records, the first one's drop throws: every later step of stop() still runs, stop() fails with that error, and the second hold's drop still runs, since each hold's drop is a step of its own. - budget-refusal-hold: a stop made while a released turn compiles is reported taken and ends the turn at the check after its compile, with nothing sent. The hold's record goes with it, and the agent's next turn carries no note of the stopped turn's wait. - budget-refusal-hold: a stop made while a turn compiles, whose compile is then refused for budget, ends the turn at its failure rather than holding it. The harness gains compileGate, which makes the nth compile wait until a promise settles. Co-Authored-By: Claude Opus 5.5 --- test/budget-refusal-hold.test.ts | 53 ++++++++++++++++++++++++++++++++ test/stop-step-throws.test.ts | 19 ++++++++++++ 2 files changed, 72 insertions(+) diff --git a/test/budget-refusal-hold.test.ts b/test/budget-refusal-hold.test.ts index 74b5853a6..a46098b9e 100644 --- a/test/budget-refusal-hold.test.ts +++ b/test/budget-refusal-hold.test.ts @@ -104,6 +104,8 @@ interface Backlog { /** The refusal's error name: OverBudgetError unless set. */ refusalName?: string; previewError?: Error; dryRunGate?: Promise; dryRunMs?: number; slowCompile?: { nth: number; ms: number }; + /** The nth compile waits until this settles. */ + compileGate?: { nth: number; until: Promise }; } const fits = (b: Backlog): boolean => b.fit || b.ticks >= b.fitAt; const pending = (b: Backlog): boolean => b.alwaysPending || (!fits(b) && b.ticks < b.readyAt); @@ -144,6 +146,7 @@ async function withBacklog(backlog: Partial, run: (b: Backlog) => Pr } b.compiles++; if (b.slowCompile?.nth === b.compiles) await new Promise((r) => setTimeout(r, b.slowCompile!.ms)); + if (b.compileGate?.nth === b.compiles) await b.compileGate.until; if (!fits(b)) throw overBudget(); if (b.refuseCompiles > 0) { b.refuseCompiles--; @@ -1437,6 +1440,56 @@ describe('a compile refused for budget holds its turn (#304)', () => { }); }); + it('ends a released turn that a stop reaches before it sends anything: nothing of its hold reaches a later turn', async () => { + let open!: () => void; + const until = new Promise((resolve) => { open = resolve; }); + await withBacklog({ compileGate: { nth: 2, until } }, async (backlog) => { + // The released compile makes a stream (the mock counts it among its + // calls) that the stop keeps from being read; the later turn's is the + // second. + const t = await residentTurn({ maintenanceIntervalMs: 50, responses: 2 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => t.of('inference:held').length === 1, 5_000)); + backlog.fit = true; + assert.ok(await pollUntil(() => backlog.compiles === 2, 3_000), 'the hold released the turn into its compile'); + assert.equal(t.framework.abortInference('prime'), true, 'a stop made while the released turn compiles is reported taken'); + open(); + assert.ok(await pollUntil(() => t.of('inference:aborted').length === 1, 3_000), 'the stop ended the turn at the check after its compile'); + assert.equal(t.of('inference:completed').length, 0, 'the stopped turn completed nothing'); + assert.equal(t.internal.budgetHolds.size, 0, 'the hold\'s record went with the turn'); + t.send('A later turn'); + assert.ok(await pollUntil(() => t.of('inference:completed').length === 1, 3_000), 'the later turn ran'); + const later = t.membrane.calls[t.membrane.calls.length - 1]; + assert.ok(!JSON.stringify(later).includes('[budget-hold]'), 'the later turn carries no note of the stopped one\'s wait'); + } finally { + open(); + await t.close(); + } + }); + }); + + it('ends a turn whose compile a stop reached, rather than holding it, when that compile is refused for budget', async () => { + let open!: () => void; + const until = new Promise((resolve) => { open = resolve; }); + await withBacklog({ compileGate: { nth: 1, until } }, async (backlog) => { + const t = await residentTurn({ maintenanceIntervalMs: 50 }); + try { + t.send('Are you there?'); + assert.ok(await pollUntil(() => backlog.compiles === 1, 5_000), 'the turn is in its compile'); + assert.equal(t.framework.abortInference('prime'), true, 'a stop made while the turn compiles is reported taken'); + open(); + assert.ok(await pollUntil(() => t.of('inference:aborted').length === 1, 3_000), 'the stop ended the turn at its failure'); + assert.equal(t.of('inference:held').length, 0, 'the refused turn was not held'); + assert.equal(t.internal.budgetHolds.size, 0); + assert.equal(t.membrane.calls.length, 0, 'nothing was sent'); + } finally { + open(); + await t.close(); + } + }); + }); + it('holds a conversation agent\'s refused turn as it holds a resident\'s', async () => { // The fork is spawned while the context fits, since a spawn dry-compiles // its template, and the backlog then starts for the fork's own turn. diff --git a/test/stop-step-throws.test.ts b/test/stop-step-throws.test.ts index 97a00e13b..c69d5c72f 100644 --- a/test/stop-step-throws.test.ts +++ b/test/stop-step-throws.test.ts @@ -138,6 +138,25 @@ const ROWS: Row[] = [ }, }, { step: "cancelling provider admission", inject: (fw, e) => patchMethod(fw, 'cancelProviderAdmission', e) }, + { + step: "a budget hold's drop", + inject: (fw, e) => { + // Two turns' hold records: the first one's drop throws. + const holds = internals(fw).budgetHolds as Map; + holds.set('first', { state: 'released', heldRequests: [] }); + holds.set('second', { state: 'released', heldRequests: [] }); + const owner = fw as unknown as Record; + const usual = (owner.dropBudgetHold as (name: string) => void).bind(fw); + owner.dropBudgetHold = (name: string) => { + if (name === 'first') throw e; + usual(name); + }; + return () => { delete owner.dropBudgetHold; }; + }, + check: (fw) => { + assert.equal((internals(fw).budgetHolds as Map).has('second'), false, "the other hold's drop still ran"); + }, + }, { step: "the channel registry's stopAll()", inject: (fw, e) => throwIn(internals(fw), 'channelRegistry', 'stopAll', e) }, { step: "the event gate's dispose()", inject: (fw, e) => throwIn(internals(fw), 'eventGate', 'dispose', e) }, { From 54565d8b35d31dba3c7051de4590372b5dfb04fe Mon Sep 17 00:00:00 2001 From: Octavia-2412 Date: Mon, 19 Oct 2026 05:23:30 -0400 Subject: [PATCH 31/31] refactor(inference): the hold takes the refused start's trigger as main's coalescing gate leaves it Since #520, past the coalescing gate beginAgentTurn carries a copy of its trigger without a batch wake's marks, on every start and every attempt, before the try whose catch calls holdForBudgetRefusal. The hold's own clearing of coalescingSubject, coalescingBatch and coalescingBatchSubjects on the refused start's request did nothing after that, so it goes, and the hold keeps a copy of the trigger as it comes. The comment says where the marks go now. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 5ccb13e58..cd0079113 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -4229,14 +4229,15 @@ export class AgentFramework { if (this.ephemeralRuns.has(agent.name) || this.agents.get(agent.name) !== agent) return false; const existing = this.budgetHolds.get(agent.name); if (existing?.state === 'spent') return false; - // The refused start's own request is an ordinary cause now, whatever it - // was: its batches were assembled and their content stored before its - // compile (beginAgentTurn), so the turn is owed. Neither an empty later - // batch of its subject nor a cancellation of the subject withdraws it, as - // neither withdraws a turn already running. A start with no request of - // its own gets one. + // The refused start's own request is an ordinary cause: its batches were + // assembled and their content stored before its compile, and past the + // coalescing gate a start carries its trigger without a batch wake's marks + // (beginAgentTurn), so the turn is owed. Neither an empty later batch of + // its subject nor a cancellation of the subject withdraws it, as neither + // withdraws a turn already running. A start with no request of its own + // gets one. const own: InferenceRequest = trigger - ? { ...trigger, coalescingSubject: undefined, coalescingBatch: undefined, coalescingBatchSubjects: undefined } + ? { ...trigger } : { agentName: agent.name, reason: 'budget-hold-release', source: 'framework', timestamp: Date.now() }; // One held turn is one undo step: the refused start's checkpoint goes, // with what recording it took back (the checkpoints it evicted, its turn