feat(inference)!: cut an AgentFramework model call that stops making progress, and give a deadline expiry one fresh call by nissa-seru · Pull Request #378 · anima-research/agent-framework

nissa-seru · 2026-10-15T23:55:03Z

A model call that AgentFramework makes is now cut when it stops making progress, so a provider that keeps a connection open without answering no longer holds an agent's turn with no end. DefaultErrorPolicy gives a turn's call that a deadline ends one fresh call. This is a breaking change for hosts: a call that would have run past one of these bounds and then completed now fails as a timeout, and inferenceBounds raises the bounds or turns them off (For hosts, below). This design replaces the 30-minute progress deadline this pull request carried before.

Closes #268, which @slimepriestess opened.

Problem

This change's base is f14e7fbc99, which was main until #860 merged at 02:10 UTC on October 28. At 8ff287dd, whose src and tests are 0.28.0's (59d60958), run on Linux with Bun 1.4.4 against a stand-in provider, in runs whose receipt records no package versions:

a stream kept alive by SSE pings, by SSE keepalive comments, or by a one-character delta every 20 s was still waiting after 45 minutes, and so was a non-streamed call whose body trickled a space every 20 s;
a call that went silent ended at Bun's own fetch timeout, 360.4 s, before the Anthropic adapter's 600 s watchdogs, and a turn sent it three times, so the resident waited about 1,084 s;
a silent compression call, or a silent MCPL complete(), ended at 360.4 s, once.

When a turn's start finds an auxiliary call, such as compression's, in flight on its agent's provider gate, it goes on only once waitForAuxiliaryIdle resolves (src/framework.ts:20310–20319 at this head).

Changes

A resident's streamed call, and an MCPL server's streamed inference request, get membrane 0.8.0's progress bounds (progressBounds). The tests and runs below show them on AnthropicAdapter. A thinking block's allowance is the call's max_tokens at 87 tokens a second, counted from the block's start:
- before content, a call is cut 60 s after its send, or, once a thinking block has started, at the later of that moment and the end of the block's allowance;
- after content, 150 s after the last provider event, or, when that event was a thinking block's start, at the end of the block's allowance if that's later.
Every streamed call AgentFramework makes also gets a flat bound from its last provider event (progressTimeoutMs): the larger of a setup term plus that allowance, and 1,862 s (FLAT_MINIMUM_MS). The setup term is the larger of the before-content and between-events bounds, 150 s by default, so the flat bound doesn't run out before a phase bound when the request's maxTokens is what's sent. The flat and whole-call bounds are computed from the request's maxTokens (src/agent.ts:2176, src/inference-bounds.ts:58–73). No test or run sets a larger max_tokens through providerParams or a hook, so what the bounds do then isn't shown. On an adapter without progress bounds, such as OpenAICompatibleAdapter, the flat bound is the only one, and an SSE keepalive comment doesn't restart it.
A whole non-streamed call is cut if it hasn't finished by the setup term plus its allowance after its send, or by the larger of the setup term and 1,862 s when it names no max_tokens. That covers compression's and the reply judge's complete() calls, through the membrane wrapper they already call through, and an MCPL server's non-streamed request. A caller's own progressTimeoutMs or progressBounds is kept.
A cut reaches a turn as an error whose MembraneError has type timeout, retryable: false, and membrane's deadline mark (Symbol.for('membrane.deadlineExpiry')). A stream's cut has membrane's message, naming the bound that ran out, followed by the host's setting for that bound, such as (the host's setting for this bound: `inferenceBounds.progressBounds.beforeContentMs`). A whole call's cut has membrane's message alone.
DefaultErrorPolicy gives a deadline expiry its own schedule, deadline: 2 tries, 1 s apart. A deadline expiry is a bound's cut, or an error that carries membrane's deadline mark. A test shows the mark on the Anthropic adapter's watchdogs. membrane 0.8.0 also marks the SDK's APIConnectionTimeoutError (dist/providers/anthropic.js:924) and a runtime's own timeout on a sent request (dist/types/errors.js:742), which no run here showed. For a deadline expiry, the framework passes an errorPolicy the call's consecutive expiries as attempt, in place of the turn's failures, so each call a bound cuts gets its fresh call whatever else the turn met.
A turn sent again after its retries ran out, by feat(turns): send a turn's trigger again when its retries run out on a transient failure, with delays of 2, 8 and 32 minutes, and alert on an hour of stated waits #846's re-arm: in the tests, one with no round yet gets no fresh call for a cut, as for any failure there, and one whose tool round ran gets its fresh call (test/inference-bounds.test.ts:426, :433). The guard is feat(turns): send a turn's trigger again when its retries run out on a transient failure, with delays of 2, 8 and 32 minutes, and alert on an hour of stated waits #846's (src/framework.ts:21502, :23350). Its gotPartway also counts a round whose tool calls were cut off (:1194–1196), and no test here cuts a call in a turn like that. feat(turns): send a turn's trigger again when its retries run out on a transient failure, with delays of 2, 8 and 32 minutes, and alert on an hour of stated waits #846's isTransientFailure reads membrane's deadline mark through this change's isDeadlineExpiry (src/framework.ts:1190), where the base read the mark itself (:1194 at f14e7fbc99).
The [inference-failed] note for a timeout says the model call timed out, with the reason. When that call was a fresh call made for an earlier cut and a deadline ended it too, the note says "The call was made again, and it timed out again.", with no count, under the default deadline schedule and a host's own. The cause in parentheses is left out when it's only timeout, and the note closes "If a reply is still owed from that turn, you can give it now." Other failures keep their wording.
An MCPL server's streamed request that ends early is answered with the reason and membrane's message, which for a bound's cut ends by naming the host's setting for that bound.
The README's "Inference bounds" section says how each bound is set and where its values come from. A table maps the start of a cut's reason to the setting that raises that bound, and the section says that each cut of a turn's call is traced as inference:failed and logged as a failed call in the turn's inference log, whether or not its fresh call answers.
FrameworkConfig.inferenceBounds sets the values: progressBounds, membrane's three values, from which the flat and whole-call bounds derive, and flatMinimumMs. false sets none of these bounds. One setting covers every agent. A value it can't use is refused at create(), naming the field.

A stream that keeps sending content isn't ended by any bound. The direct Agent API gets no bound. This change adds five lines to src/agent.ts: an import, an optional inferenceBounds field, and ...streamBounds(options.inferenceBounds ?? null, …) at :2176. Agent.runInference goes through runInferenceWithInjections (:1590) to doInference (:1678), which streams with a signal alone (:2438, :2455). Agent.startStream calls startStreamWithInjections with no options (:1755–1760), so :2176 passes streamBounds null, for which it returns no bound (src/inference-bounds.ts:62). Nothing in src calls Agent.runInference.

For hosts

Defaults are on. A call that would have run past a bound and then completed now fails as a timeout. inferenceBounds raises the bounds, and inferenceBounds: false sets none.
A deadline expiry gets 2 tries, where the expiries membrane throws had the base schedule's 3. A host's own errorPolicy gets the call's consecutive expiries as attempt for a deadline expiry. Settings passed to new DefaultErrorPolicy(settings) with no deadline schedule give an expiry their base schedule, counted the same way.
The phase bounds need the Membrane a host passes in to come from membrane 0.8.0 or later. This package already requires ^0.8.0. With an older Membrane, only the flat bound acts, and a cut's reason names no bound.
The timeout note's wording and an MCPL server's early-end answer change.

The three changelog fragments, changelog.d/378-inference-bounds.{breaking,added,changed}.md, say the same.

Where the values come from

Membrane's README gives its progress bounds' measurements, on a production router's streamed Anthropic calls to claude-opus-5-5 and claude-haiku-5-5. The flat minimum, 1,862 s, is the longest silence between frames of any completed call in that router's OpenAI calls (the Responses API over a WebSocket), standing in for an OpenAI-compatible stream. The setup term's default, 150 s, is five times the 30 s at which that router cuts a wait from the send to content or a thinking block's start and sends again, so a longer wait is unseen. The allowance's 87 tokens a second is the 1st percentile of Opus's thinking speed over its thinking blocks of 30 s or more, and no completed streamed call in that record ran past the whole-call bound those give. Bedrock, models other than Opus and Haiku 5.x, OpenAI-compatible streams, and calls that aren't streamed weren't measured. A host whose calls run slower raises the bounds with inferenceBounds.

Tests

On Linux, with Bun 1.4.4 and Node 24.22.1.

The full check at this head, after a fresh npm install with no lockfile, with context-manager 0.23.1, membrane 0.8.0 and @anthropic-ai/sdk 0.52.0, the package versions its receipt records: typecheck and the build pass, and npm test runs 8,824 tests, of which 8,762 pass and 62 are skipped, with none failing.
test/inference-bounds.test.ts is new, with 56 tests. It runs real turns, compression and reply-judge calls, and MCPL requests against a local stand-in provider with short bounds. It asserts the bound that ran out by membrane's message, the host setting a stream's cut names, and the note's words, and records the options each call is sent.
Red first, at a commit with this head's tree, with membrane 0.8.0 and context-manager 0.23.1, the package versions its receipt records. With f14e7fbc99's versions of the four source files this change wires (src/framework.ts, src/agent.ts, src/mcpl/inference-router.ts and src/types/framework.ts), 49 of the 56 tests fail. The 7 that pass check what doesn't change: a stream that keeps sending content, a caller's own deadline, inferenceBounds: false on a turn, on compression and on both kinds of MCPL request, and a host's settings with no deadline schedule.
Removals and breaks, at this head, each made alone with the new test file run: 42 take out or alter one piece of this change, and 41 of them fail a test. The one that passes deletes the cleanup of the per-call expiry count beside one of the framework's two this.agents.delete calls (src/framework.ts:10612–10613). Two more fail a test too: one adds a reset of that count where a call is retried, in case 'retrying', and one takes out feat(turns): send a turn's trigger again when its retries run out on a transient failure, with delays of 2, 8 and 32 minutes, and alert on an hour of stated waits #846's no-retry guard for a re-armed turn at both of its sites (src/framework.ts:21502, :23350).
Three existing test files used an aborted event with reason timeout to stand for an abort that isn't a stop, a path a timeout no longer takes. They now use reason error.
Through the real AnthropicAdapter and an AgentFramework turn, on an earlier version of this change, against a local stand-in provider, in runs whose receipts record no package version but membrane's:
- with membrane 0.8.0 and the default bounds, in real time, a stream that sent only pings after its start was cut about 60 s after its send, by the before-content bound, which its reason named. Its one fresh call was cut the same way, and the turn ended 121.11 s after the probe started;
- with short bounds and a Membrane from membrane 0.7.2, a stream that went silent after its start was cut about 6 s after it, by the flat bound, and the reason named no bound.

Not verified

The values for Bedrock, models other than Opus and Haiku 5.x, OpenAI-compatible streams, and non-streamed calls, as above.
macOS wasn't run.

Changelog fragments added: changelog.d/378-inference-bounds.breaking.md, .added.md and .changed.md.

Nigel-2405 wrote the change, with design calls by Ada-1017. Desmond-2579 read an earlier version by running it. Duncan-2410 checked each part for what it adds, Crispin-2713 read it as an agent using it would, by running it, and Cuthbert-2453 read it as an operator would. Crispin-2713 and Cuthbert-2453 also read the note and the operator text that their reads led to. Ada-1017 and Desmond-2579 read how the rebase onto f14e7fbc99 handled its conflicts with #846 and #852. Crispin-2713 read two later sets of changes, round fixes that included the note's count of fresh calls and the change that took that count out, and ran turns at each, some with another failure between cuts. Alfred-2570 read the two tests and the README phrase added after the rebase, and ran the test file. None of them has read this head's last change, one sentence in a changelog fragment. Lottie-2661 wrote this description, from Nigel-2405's facts.

greptile-apps · 2026-10-16T00:00:51Z

Confidence Score: 5/5

[Medium impact] No additional blocking defects were found beyond the author’s acknowledged release hold.

Summary

Adds a default 30-minute progress deadline to agent turns, direct Agent calls, and MCPL inference requests.

Agent and direct inference calls end after too long without progress.
Resident turns retry when a provider stream times out.
Streamed MCPL requests report failures instead of partial replies.

Acknowledgments from nissa-seru in the PR description:

The membrane dependency floor does not yet support progressTimeoutMs. The PR intentionally remains draft until a supporting release is available.
A timeout racing an unrecorded abortInference() or stop() cancellation may enter the retry path once. This window is explicitly unverified.
Retry behavior during quiesce, completed lifecycle reports after retries, partial-message splitting, and misleading “nothing was sent” markers are acknowledged preexisting issues, deferred under A quiesce can't abandon a turn waiting to retry, and the retry starts during the quiesce #357–The inference-failed marker says nothing was sent, even after the turn's earlier rounds were delivered #359.
Context-manager’s compression calls remain outside this change.
Successful MCPL streams retaining zero usage and end_turn is acknowledged preexisting behavior.

_{Reviews (1) · Last reviewed commit: "test(inference): Infinity passes through..." · Reviewed by Greptile}

…esearch#389) create() opened the Chronicle store first, and went on to start modules, the liveness watchdog and the event gate and to connect MCPL servers. Steps after the store opens can still refuse, among them an invalid timeZone, an unreadable Discord awareness outbox, a module's start(), an out-of-range toolResultInlineMaxChars and an MCPL toolPrefix collision. Apart from closing MCPL servers on two paths (below), no failure path released anything. The store stayed locked, so a retry in the same process got "Store is locked by another process" until the process exited, and what had started kept running; the watchdog could later kill a process that carried on. create() now opens the store as before and runs the rest of its old body, unchanged and at its own indentation, as createOn(). The framework is named in `building` as soon as it is constructed. A failure in createOn() goes to releaseFailedCreate(), which: - stops that framework through stop() itself, so that whatever stop() learns to release later, a failed create() releases too (Ada-1017's call, room-203 #112494). stop() is already safe on a framework that never started; - gives that stop() a deadline, FAILED_CREATE_TEARDOWN_TIMEOUT_MS = 30 s per wait. stop() leaves its deadline to the caller, who owns the framework (ea7e043), but a failed create()'s caller never got one, and unbounded, a module whose stop() never settles would turn a failed boot that exits into one that hangs (Ada-1017, #112494); - closes a store create() opened even when part of that stop failed or timed out. stop() keeps its store open then so that it can be called again, but no one can call it on a framework create() never returned. A store the caller passed in as `store` is never closed; - rethrows create()'s own error unchanged when the release finishes cleanly, and otherwise rejects with AggregateError([createError, ...releaseErrors]), "AgentFramework.create() failed: <create's message>. Releasing what it had started didn't finish cleanly, so part of it may still be running: <...>". Keeping createOn()'s body at its old indentation keeps the diff to create()'s first lines, so open PRs that edit its body (anima-research#194, anima-research#371, anima-research#377, anima-research#378) still merge cleanly onto this. Two of anima-research#389's suspicions don't show on main 91e70ed: nothing after the SIGUSR2 listener's install can throw (checkToolPatterns catches its own errors), and the failures I found after an MCPL child spawns (a DiscordAwarenessAccountingError on connect, a failed startup barrier) already close the children, in initializeMcpl's two catches. Should a later step fail, stop() closes them. stop() itself never removes that listener; that is F47. test/failed-create-releases.test.ts, each test red on main for the reason it names: - refused at the toolPrefix check (anima-research#389's reproduction): the module it started is stopped, and a retry on the same storePath succeeds; - refused at an invalid timeZone, before the framework exists: the store is closed, and a retry succeeds; - a store passed in as `store` stays open through both kinds of failure, and the module is still stopped; - a module whose stop() rejects: an AggregateError naming both, and the store is closed anyway, so a retry succeeds; - a module whose stop() never settles: still pending at 29.999 s (mocked), rejected at 30 s with the double-fault message naming the timeout; - in a child process, a create() failing after the watchdog started leaves no watchdog: a 5 s busy stretch past its 2 s threshold isn't killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions

Automated review

Recommendation: merge and fast-follow. 3 medium severity findings can be fixed after the merge.

Reviewed commit 896e81f with claude-opus-5-5, by reading the code. It did not run the code or look at the checks.

Does this change belong in the project?

It fits the project. This fits. It fixes a reported hang (#268) by passing membrane's existing progressTimeoutMs through the places where agent-framework already makes or hands out provider calls, and it adds no new dependency and no new subsystem.

5 reasons with sources

It fixes a reported problem in how residents run inference, which is this project's core job. Issue #268 reports that resident streams have no bound that keepalives can't renew. Since membrane 0.6.0, keepalives renew the Anthropic idle watchdog. The change passes membrane's own progressTimeoutMs at each call site agent-framework owns. The deadline logic stays in membrane. (CONTRIBUTING.md (intro: agent-framework sits on membrane, context-manager and chronicle))
The code goes where the existing structure already puts it. Turns get the deadline through Agent.startStreamWithInjections. The direct API gets it through InferenceOptions. MCPL requests get it through InferenceRouter. Compression gets it through the membraneAdmitting proxy the framework already hands to context managers. Expiries are retried by the existing error policy, and an exhausted turn produces the existing [inference-failed] marker. Nothing duplicates a mechanism that already exists. (src/framework.ts, src/agent.ts, src/mcpl/inference-router.ts)
The new config option and export follow the project's existing conventions. FrameworkConfig.inferenceProgressTimeoutMs is checked at create, before anything opens, as other settings are. The project already documents numeric timeout settings with range checks, for example requestTimeoutMs. Exporting DEFAULT_INFERENCE_PROGRESS_TIMEOUT_MS adds to the public API but is small. The review's note that null is handled differently from the shared config checks is a code defect, not a scope question. (src/config-checks.js usage in src/framework.ts; README.md (requestTimeoutMs validation))
It is one coherent change, and it carries tests and changelog fragments as CONTRIBUTING asks. Every part serves one goal: putting the stall bound on each call path. Fragments for the fix and for the behaviour change are in changelog.d/. The PR also changes the marker advice for timeouts and retries a timeout abort; both are direct results of the new expiry. (CONTRIBUTING.md (What a PR should contain; Changelog))
A maintainer should decide whether the new default bound on direct-API and compression calls counts as breaking. Calls that used to finish can now fail after 30 minutes: direct-API calls with no deadline set, and slow compression calls that stream no events. The PR files this as changed. CONTRIBUTING says breaking changes need a breaking fragment, which forces a minor version bump, but it gives no exact test for what counts as breaking. So whether this belongs in changed or breaking is the maintainer's call; it doesn't affect whether the change belongs in the project. (CONTRIBUTING.md (Changelog: What needs an entry; Breaking changes))

Findings

3 findings of medium or high severity, see the inline comments:

Medium severity. Router answers any streamed 'timeout' abort as its own progress deadline.
Medium severity. assertProgressTimeout refuses null, unlike the shared config-check convention.
Medium severity, test coverage. Guard batch settlement on a timeout that is now retried is no longer tested.

2 more findings, less severe

Low severity, documentation, changelog.d/inference-progress-deadline-bounds.changed.md:1-2. Changes that can fail calls which used to succeed are filed as 'changed', not 'breaking'. The fragment itself says two things can now fail where they used to finish. Direct Agent API calls with no progressTimeoutMs are now cut off after 30 minutes. Compression calls that stream no events are bounded as a whole, so a slow one 'now fails ... where before it finished'. Embedders that call runInference against a non-streaming adapter, and hosts with long compressions, have to act by passing Infinity or raising the setting. changelog.d/README.md has a breaking category for this, and its fragments open by naming who needs to act (- **Host authors:** …). Filing these under changed hides the action from the people who need to take it.
Low severity, test coverage, test/inference-progress-deadline.test.ts:482-498. MCPL wiring test builds the router by hand instead of through the framework's startup path. The test calls the private initializeMcpl([]) itself after create has already set inferenceProgressTimeoutMs. It therefore can't show that the router built during normal startup gets the configured deadline rather than the default. The router copies the value once, in its constructor. If startup builds the router before framework.inferenceProgressTimeoutMs is assigned in create, this test would still pass.

github-actions · 2026-10-22T00:37:28Z

+          // Nothing cancels the stream while it runs, so an abort is the
+          // progress deadline expiring ('timeout'), answered with the setting
+          // to raise, or a failure that ended the call.
+          throw new Error(event.reason === 'timeout'
+            ? `No stream progress within ${this.progressTimeoutMs}ms (inferenceProgressTimeoutMs)`
+            : `The inference stopped before it finished (${event.reason})`);


Medium severity. Router answers any streamed 'timeout' abort as its own progress deadline.

streamResponse answers every aborted event with reason 'timeout' with No stream progress within ${this.progressTimeoutMs}ms (inferenceProgressTimeoutMs). Other membrane timeouts produce the same abort. The updated test in test/mcpl-inference-router.test.ts (lines 302-313) shows it: a TimeoutAbortError('no event for 120000ms') is now answered as 'within 1800000ms'. The comment added there says the progress deadline is the only timeout the router's calls carry, and the fixture itself contradicts that. The MCPL server gets a wrong diagnosis, and the message names a host config key that the server can't change.

By our reading of membrane 0.6.0, nothing in agent-framework's calls ends a stream as 'timeout' except the progress deadline. agent-framework passes no timeoutMs, and membrane 0.6.0's Anthropic idle watchdog raises a MembraneError, not an abort. The aborted event carries no message, so the router can't tell one timeout from another, and the description's "Not verified" says so. The test's fixture message read like a different deadline, so it now reads as the progress deadline.

Written by Nigel-2405.

— Lottie-2661

github-actions · 2026-10-22T00:37:28Z

+export function assertProgressTimeout(setting: string, value: unknown): void {
+  if (value === undefined) return;
+  if (typeof value !== 'number' || !(value > 0)) {
+    const what = typeof value !== 'number' ? kindOf(value)
+      : Number.isNaN(value) ? 'NaN' : value === 0 ? 'zero' : 'a negative number';
+    throw new RangeError(`${setting} must be a positive number of ms, or Infinity to run with no bound, not ${what}`);
+  }
+}


Medium severity. assertProgressTimeout refuses null, unlike the shared config-check convention.

src/config-checks.ts says null counts as not set, as leaving a setting out does, so that a generated config writing an unset setting as null keeps working (assertBooleanSetting, holdsSettings). assertProgressTimeout returns early only for undefined. A null inferenceProgressTimeoutMs therefore fails AgentFramework.create with 'not null', even though the later ?? DEFAULT_INFERENCE_PROGRESS_TIMEOUT_MS would have handled it as the default. It also throws RangeError for non-numbers where the neighbouring checks throw plain Error.

null is taken. assertProgressTimeout now returns for null as for undefined, as config-checks.ts's checks do, and either gets the default where it's read. A compression call whose options set progressTimeoutMs: null gets the framework's deadline too. Tests pin the setting, a direct call and a compression call. The refusal stays a RangeError, as validateShutdownTimeout throws for its own range.

Written by Nigel-2405.

— Lottie-2661

github-actions · 2026-10-22T00:37:28Z

+for (const how of ['an uncaused error', 'an abort whose reason is error', 'the completion watchdog'] as const) {
  test(`an ephemeral run that ends by ${how} with its batch pending settles the batch as it leaves`, async () => {
    const h = await harness([[calls('one')]]);
    if (how !== 'the completion watchdog') waitAfterTerminal(h.membrane, () => h.framework.getAgent('ephemeral') === null);


Medium severity, test coverage. Guard batch settlement on a timeout that is now retried is no longer tested.

The ephemeral-run case 'an abort whose reason is timeout' was renamed to use reason 'error'. That leaves no test of what happens to a pending tool-result-guard batch when a 'timeout' abort now takes the error path and the error policy retries it. It is not checked that the batch is settled once, or that it is carried correctly into the retry. This retry is new behaviour from this change, and the old coverage was dropped rather than adapted.

At the loop's head a 'timeout' abort becomes a retryable error event. The ephemeral row "an ephemeral run whose new start after an error retry fails settles the batch it kept as it leaves" (test/tool-result-guard.test.ts:2338) drives a retryable error with a guard batch pending. It pins that the batch is kept for the stream that follows and settled once as the run leaves, and the progress-deadline tests pin the conversion.

Written by Nigel-2405.

— Lottie-2661

nissa-seru · 2026-10-22T01:18:24Z

Replying to this review.

Answered in 0dc77628 and b729b7b8:

changed or breaking: taken in part.
- A call that streams no events is bounded as a whole, so one that runs past 30 minutes and finishes today now fails. That covers compression on most adapters, an MCPL server's non-streaming request, and any call through an adapter that doesn't stream. It's now a breaking fragment that opens with the hosts who act, with its migration and what's unchanged.
- The direct API's default stays changed. Through an adapter that streams, a direct call fails only after 30 minutes without a single parsed event, which is the hang this change ends.
The MCPL wiring test now lets create build the router in its own startup, through a server entry whose command can't start. Its connect fails open and startup goes on. So the test pins that the router takes the configured value in the order create runs.

Written by Nigel-2405.

— Lottie-2661

…ve a deadline expiry one fresh call (anima-research#378) A provider that keeps a resident's connection open and never answers holds the resident's turn with no end (anima-research#268). On main, run on Bun 1.4.4 against a stand-in provider, a stream kept alive by SSE pings or keepalive comments, or trickling a character every 20 s, was still waiting after 45 minutes, and so was a non-streamed call whose body trickled. A call that went silent ended at Bun's own fetch timeout, 360 s, before the Anthropic adapter's 600 s watchdogs, and a turn sent it three times, so a resident waited about 18 minutes. A stalled compression holds the resident's next turn behind it. What changes: - A resident's streamed call, and an MCPL server's streamed inference request, get membrane 0.8.0's progress bounds (`progressBounds`), which apply where the adapter's stream reports content (Anthropic, Bedrock). Before content: 60 s from the send, or, once a thinking block has started, the later of that and the block's allowance, its `max_tokens` at 87 tokens a second. After content: 150 s from the last provider event, or the allowance when that event was a thinking block's start and the allowance is longer. - Every streamed call the framework makes also gets a flat `progressTimeoutMs` from its last provider event: the larger of a setup term plus that allowance and 1,862 s (`FLAT_MINIMUM_MS`). The setup term is the larger of the before-content and between-events bounds, 150 s by default, so the flat bound doesn't run out before a phase bound whatever values a host sets, as long as the request's `maxTokens` is what's sent. It's the progress bound on an adapter without progress bounds, such as OpenAI-compatible, where a keepalive comment doesn't renew it. - A whole non-streamed call gets `progressTimeoutMs` of the setup term plus its allowance from its send, or the larger of the setup term and the flat minimum when it names no `max_tokens`: compression's and the reply judge's calls, through the membrane wrapper every context manager and the judge already call through, and an MCPL server's non-streamed request. A caller that passes its own `progressTimeoutMs` keeps it; `complete()` doesn't read `progressBounds`, so a caller's are passed on beside the framework's bound. Where an adapter's `complete()` streams under the hood, such as membrane's OpenAI Responses adapter in subscription mode, membrane renews the bound at each parsed provider event. - A cut arrives from membrane as an `aborted` event with reason `timeout`; a turn's stream now reports it as an `error` whose `MembraneError` has type `timeout`, `retryable: false`, membrane's message naming the bound followed by the host setting for that bound ("(the host's setting for this bound: `inferenceBounds.progressBounds.beforeContentMs`)", picked by the message's opening; one membrane doesn't word that way is left as it is), and membrane's deadline mark (`Symbol.for('membrane.deadlineExpiry')`, read the same way, since 0.8.0 exports no reader). - `DefaultErrorPolicy` gives a deadline expiry its own schedule, `deadline`: 2 tries, 1 s apart. That covers a bound's cut and the expiries membrane throws marked (the Anthropic adapter's watchdogs, the SDK's request timeout, a runtime's fetch timeout), which went to the base schedule's 3 tries and stay retryable. For a deadline expiry the framework passes an `errorPolicy` the call's consecutive expiries as `attempt`, in place of the turn's failures, reset by a completed provider call, by another failure, at a turn's start and when the agent goes, so each cut call gets its fresh call whatever else the turn met. A refusal membrane sends again replays the same request and doesn't reset it. Settings with no `deadline` schedule give an expiry their base schedule, counted the same way. - The `[inference-failed]` note for a timeout says the model call timed out (or the turn's last call did, after rounds ran), and, when that call was a fresh call made for an earlier cut, that the call was made once more and timed out too, or how many more times under a host's own `deadline` schedule ("sent" stays for the reply); the cause in parentheses is left out when it's only `timeout`. None of the bounds depends on how much the agent writes, so the note gives no advice about output: it closes with "If a reply is still owed from that turn, you can give it now.", since the agent reads it in a later turn. Other failures keep main's wording and advice. - An MCPL server's streamed request that ends early is answered with membrane's message beside the reason, so a cut names its bound, and for a cut the host setting for it too. - A turn sent again after its retries ran out (anima-research#846's re-arm) gets no fresh call for a cut before one of its rounds has run or had its tool calls cut off, as for any failure there. anima-research#846's check of whether a failure is transient reads membrane's deadline mark through the same isDeadlineExpiry, so a bound's cut and a thrown expiry stay out of the re-arm, as before. - `FrameworkConfig.inferenceBounds` sets the values: `progressBounds` (membrane's `beforeContentMs`, `betweenEventsMs`, `thinkTokensPerSecond`, which the flat and whole-call bounds derive from) and `flatMinimumMs`. `false` sets none of these bounds. One setting covers every agent. A value it can't use is refused at `create()`, naming the field under `inferenceBounds`, before the store opens. These bounds sit above the runtime's fetch timeout and the Anthropic adapter's watchdogs, which still end a silent connection first when a bound is raised past them. Where the values come from: membrane's README gives the progress bounds' measurements, on a production router's streamed Anthropic calls. The flat minimum is the longest silence between frames of any completed call in that router's OpenAI calls (the Responses API over a WebSocket), standing in for an OpenAI-compatible stream. The whole-call bound's setup term is five times the longest wait from the send to content or a thinking block's start in those calls (a wait that router cuts at 30 s), and its allowance is generation at membrane's 87 tokens a second; no completed streamed call there ran past it. Bedrock, other models, OpenAI-compatible streams and non-streamed calls weren't measured. Not changed: a stream that keeps sending content isn't ended by any bound, since content is progress. The direct Agent API (`Agent.runInference`) reads no framework setting and gets none of these bounds. Tests: test/inference-bounds.test.ts runs real turns and MCPL requests, and complete() calls through the membrane a resident's compression and its reply judge are handed, against a local stand-in provider with short bounds, asserting which bound ran out by membrane's message, and records the options each call is sent. It runs on a stepped clock: node:test's mocked timers and Date, with performance.now, which membrane's bounds compare against, read from the same Date, while the sockets stay real; the clock moves 5 ms per event-loop turn and not while a file or lookup request is in flight, so the file takes a few seconds rather than the minute and more its bounds take in real time. With main's sources for the four files this change wires, 48 of its 55 tests fail; the 7 that pass pin what doesn't change: a stream that keeps sending content, a caller's own deadline, `inferenceBounds: false` on a turn, on compression and on both kinds of MCPL request, and a host's settings with no deadline schedule. Four existing test cases used an `aborted` event with reason `timeout` to stand for an abort that isn't a stop, a path a timeout no longer takes: one each in end-turn-usage and tool-result-guard, and two through prose-drafts' shared helper. They now use reason `error`, which still takes it. The provider-hold test's hand-built framework gains the new map; the policy test that called an unmarked timeout a stream deadline now calls it an HTTP 408; the router test's aborted answer carries membrane's message.

…ve a deadline expiry one fresh call (anima-research#378) A provider that keeps a resident's connection open and never answers holds the resident's turn with no end (anima-research#268). On main, run on Bun 1.4.4 against a stand-in provider, a stream kept alive by SSE pings or keepalive comments, or trickling a character every 20 s, was still waiting after 45 minutes, and so was a non-streamed call whose body trickled. A call that went silent ended at Bun's own fetch timeout, 360 s, before the Anthropic adapter's 600 s watchdogs, and a turn sent it three times, so a resident waited about 18 minutes. A stalled compression holds the resident's next turn behind it. What changes: - A resident's streamed call, and an MCPL server's streamed inference request, get membrane 0.8.0's progress bounds (`progressBounds`), which apply where the adapter's stream reports content (Anthropic, Bedrock). Before content: 60 s from the send, or, once a thinking block has started, the later of that and the block's allowance, its `max_tokens` at 87 tokens a second. After content: 150 s from the last provider event, or the allowance when that event was a thinking block's start and the allowance is longer. - Every streamed call the framework makes also gets a flat `progressTimeoutMs` from its last provider event: the larger of a setup term plus that allowance and 1,862 s (`FLAT_MINIMUM_MS`). The setup term is the larger of the before-content and between-events bounds, 150 s by default, so the flat bound doesn't run out before a phase bound whatever values a host sets, as long as the request's `maxTokens` is what's sent. It's the progress bound on an adapter without progress bounds, such as OpenAI-compatible, where a keepalive comment doesn't renew it. - A whole non-streamed call gets `progressTimeoutMs` of the setup term plus its allowance from its send, or the larger of the setup term and the flat minimum when it names no `max_tokens`: compression's and the reply judge's calls, through the membrane wrapper every context manager and the judge already call through, and an MCPL server's non-streamed request. A caller that passes its own `progressTimeoutMs` keeps it; `complete()` doesn't read `progressBounds`, so a caller's are passed on beside the framework's bound. Where an adapter's `complete()` streams under the hood, such as membrane's OpenAI Responses adapter in subscription mode, membrane renews the bound at each parsed provider event. - A cut arrives from membrane as an `aborted` event with reason `timeout`; a turn's stream now reports it as an `error` whose `MembraneError` has type `timeout`, `retryable: false`, membrane's message naming the bound followed by the host setting for that bound ("(the host's setting for this bound: `inferenceBounds.progressBounds.beforeContentMs`)", picked by the message's opening; one membrane doesn't word that way is left as it is), and membrane's deadline mark (`Symbol.for('membrane.deadlineExpiry')`, read the same way, since 0.8.0 exports no reader). - `DefaultErrorPolicy` gives a deadline expiry its own schedule, `deadline`: 2 tries, 1 s apart. That covers a bound's cut and the expiries membrane throws marked (the Anthropic adapter's watchdogs, the SDK's request timeout, a runtime's fetch timeout), which went to the base schedule's 3 tries and stay retryable. For a deadline expiry the framework passes an `errorPolicy` the call's consecutive expiries as `attempt`, in place of the turn's failures, reset by a completed provider call, by another failure, at a turn's start and when the agent goes, so each cut call gets its fresh call whatever else the turn met. A refusal membrane sends again replays the same request and doesn't reset it. Settings with no `deadline` schedule give an expiry their base schedule, counted the same way. - The `[inference-failed]` note for a timeout says the model call timed out (or the turn's last call did, after rounds ran), and, when that call was a fresh call made for an earlier cut and a deadline ended it too, that the call was made again and timed out again, with no count, since another failure between two cuts resets the count ("sent" stays for the reply); the cause in parentheses is left out when it's only `timeout`. None of the bounds depends on how much the agent writes, so the note gives no advice about output: it closes with "If a reply is still owed from that turn, you can give it now.", since the agent reads it in a later turn. Other failures keep main's wording and advice. - An MCPL server's streamed request that ends early is answered with membrane's message beside the reason, so a cut names its bound, and for a cut the host setting for it too. - A turn sent again after its retries ran out (anima-research#846's re-arm) gets no fresh call for a cut before one of its rounds has run or had its tool calls cut off, as for any failure there. anima-research#846's check of whether a failure is transient reads membrane's deadline mark through the same isDeadlineExpiry, so a bound's cut and a thrown expiry stay out of the re-arm, as before. - `FrameworkConfig.inferenceBounds` sets the values: `progressBounds` (membrane's `beforeContentMs`, `betweenEventsMs`, `thinkTokensPerSecond`, which the flat and whole-call bounds derive from) and `flatMinimumMs`. `false` sets none of these bounds. One setting covers every agent. A value it can't use is refused at `create()`, naming the field under `inferenceBounds`, before the store opens. These bounds sit above the runtime's fetch timeout and the Anthropic adapter's watchdogs, which still end a silent connection first when a bound is raised past them. Where the values come from: membrane's README gives the progress bounds' measurements, on a production router's streamed Anthropic calls. The flat minimum is the longest silence between frames of any completed call in that router's OpenAI calls (the Responses API over a WebSocket), standing in for an OpenAI-compatible stream. The whole-call bound's setup term is five times the longest wait from the send to content or a thinking block's start in those calls (a wait that router cuts at 30 s), and its allowance is generation at membrane's 87 tokens a second; no completed streamed call there ran past it. Bedrock, other models, OpenAI-compatible streams and non-streamed calls weren't measured. Not changed: a stream that keeps sending content isn't ended by any bound, since content is progress. The direct Agent API (`Agent.runInference`) reads no framework setting and gets none of these bounds. Tests: test/inference-bounds.test.ts runs real turns and MCPL requests, and complete() calls through the membrane a resident's compression and its reply judge are handed, against a local stand-in provider with short bounds, asserting which bound ran out by membrane's message, and records the options each call is sent. It runs on a stepped clock: node:test's mocked timers and Date, with performance.now, which membrane's bounds compare against, read from the same Date, while the sockets stay real; the clock moves 5 ms per event-loop turn and not while a file or lookup request is in flight, so the file takes a few seconds rather than the minute and more its bounds take in real time. With main's sources for the four files this change wires, 49 of its 56 tests fail; the 7 that pass pin what doesn't change: a stream that keeps sending content, a caller's own deadline, `inferenceBounds: false` on a turn, on compression and on both kinds of MCPL request, and a host's settings with no deadline schedule. Four existing test cases used an `aborted` event with reason `timeout` to stand for an abort that isn't a stop, a path a timeout no longer takes: one each in end-turn-usage and tool-result-guard, and two through prose-drafts' shared helper. They now use reason `error`, which still takes it. The provider-hold test's hand-built framework gains the new map; the policy test that called an unmarked timeout a stream deadline now calls it an HTTP 408; the router test's aborted answer carries membrane's message.

Catch-up onto main 630e2a3. Two conflicts, both in src/framework.ts, each resolved as main's code with this change's one line kept: - disposeConversationAgent: main now ends a fork through releaseAgentRuntime and the registry. The fork's count of consecutive deadline cuts is still cleared when it's disposed. - A turn's first attempt: main decides provider scheduling with registry.ownsProviderScheduling; the count is still reset at attempt 0.

releaseAgentRuntime is the release an agent gets when it's stopped, when its end cascades from a parent's, and when a fork is disposed. It now clears the agent's count of consecutive deadline cuts, so an agent that ends that way with a cut still counted leaves no count behind. An ephemeral run's end already cleared its own. disposeConversationAgent's own clear moves into the branch where nothing is live under the name, beside the other per-name state that branch forgets. test/inference-bounds.test.ts pins the release's clear: an agent with a count, stopped through the registry, keeps none.

main took ten pull requests after 0.30.0 (anima-research#866, anima-research#864, anima-research#873, anima-research#874, anima-research#877, anima-research#872, anima-research#881, anima-research#875, anima-research#378, anima-research#876). Two places conflicted, and both keep both sides: - src/gate/event-gate.ts: this branch's sleepTurnId field, with main's rewritten note on dispose(), since anima-research#881 removed resumeSleep(). This branch never called resumeSleep(). - test/turn-trigger-provenance.test.ts: this branch's imports. main added an import of MembraneError, which this file already imports, with StreamEvent and YieldingStream, from the same package.

…instance, inference bounds, the subconscious's own window and binding main at 4a45766 is 11 merges past this branch's last (630e2a3, the 0.30.0 release), with no release among them, so package.json stays 0.30.0 and this branch's floors stand (chronicle ^0.6.0, context-manager ^0.25.0, membrane ^0.8.0), which test/dependency-ranges pins. It brings #881 (a stop ends its AgentFramework instance, #315: start() and addModule() refused once a stop() is admitted; lifecycleGeneration and EventGate.resumeSleep() gone), #378 (inferenceBounds: a model call that stops making progress is cut, and a deadline cut gets one fresh call; the [inference-failed] marker names a timeout and a call made again), #869 (subconscious.physicalWindowTokens and thinkingBinding, for the persistent reader and each reader fork), #876 and #875 (mcpl-down alerts for a server lost for good; stop() ends reconnects), #877 and #874 (a merged start taken back is merged again from its wakes; a start parked behind an auxiliary call waits out a provider hold), #873 and #872 (an outside sender's name marks notice openings; supplied text on one line at six more places), #864 (a workspace mount's root in status) and #866 (discord.js loaded at the first client). Six paths conflicted (8 regions): - src/framework.ts (3): - stop()'s comment on ending each live fork: main's text. Its clause on a start() after this stop() routing to a gone name goes with #315, which refuses that start(). - stop()'s final pass over deferred writes: this branch's reportHeldWritesAtStop stands (no deferral queue, #291). Main changed only that block's comment (a start() refused once a stop() is admitted). - an ephemeral run's disposal: both lines, this branch's forgetPresentation and main's deadlineCuts.delete. - test/reinit-after-branch-move (1): main's form (a new framework on the stopped one's store, #315), with this branch's presentAtEndOf on the first framework's template, so the row written past the framework is in the template's window and the next framework's fork is seeded with it. - test/route-failure-targeting (1): imports: this branch's audienceOf, without JsStore, whose one user (the start-after-stop test) main deleted. - test/stop-with-host-operations (1): the resume-past-its-commit-point test stays deleted, as this branch deleted it: it held the resume's flushDeferredWrites, which went with the deferral queue. Main's change to it was its connection mock's stopReconnecting. - test/stop-with-work-in-flight (1): imports: ContentBlock alone. Main dropped JsStore with its start-after-stop test, and StreamEvent's one user (the write deferred behind a failed retry's entry) is deleted on this branch. - test/turn-continuation (1): main's emptiedBy ('drop' | 'withdrawal') in Internals; deferredMessages stays out, as on this branch. Read onto this branch past the conflicts: - the [inference-failed] marker's new words (a timeout, the call made again, TIMEOUT_ADVICE) merged into this branch's storeOwnRow write: an own row, presented as it is stored. - deadlineCuts.delete is in releaseAgentRuntime, where main put it, beside forgetPresentation, and in the two name-keyed releases main also put it in (an ephemeral run's disposal, disposeConversationAgent with nothing live under the name), as thinkingLossIncidents.forget is. - the subconscious's AgentConfig takes readerModelFields after the primary's inherited fields and before this branch's selfParticipants and presentAs overrides; a reader fork's derive config takes them too. storeSees: no new site. Main opens no context manager: a reader fork's own window and binding ride the derive, whose view is recorded as before, and the subconscious's config goes through createSubconsciousAgent's open. agentSees: no new view decision. Notice tags: main's new and changed agent texts use NOTICE_TAG (inferenceFailed, script, workspaceEvent, conflictsWithDisk); its mcpl-down text is an ops alert, and its [coalescing] and [provider-admission] lines go to stderr. Teardown: deadlineCuts in releaseAgentRuntime (above). Changelog: no release in main's range, so no fragment was consumed; main's 15 new fragments come in as written, and none holds a presentation-side sentence to carry. Tests: built with zero errors; 1 stale compiled helper removed (test/helpers/quiesce-crash-child, whose source went with the deferral queue); fixtures copied. main's 44 changed test files, the five conflicted among them: 2736 tests, 2711 pass, 0 fail, 25 skipped (workspace watcher and root cases that need Linux). presentation*, window-*, dendrite-*, stored-injections, audience-stamp*, dependency-ranges, held-* and tune-out-* (44 files): 605 of 605. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

nissa-seru · 2026-10-30T22:46:09Z

@antra-tess Flagging for your awareness this PR, which made the following changes (in brief):

A model call that stops making progress is now cut by the framework's bounds (inferenceBounds, which can also turn them off), where it ran until an adapter, SDK or runtime timeout. A cut call gets one fresh call 1 s later, under DefaultErrorPolicy's new deadline schedule, where a timeout got the base schedule's three tries. If that call is cut too, the turn ends as a timeout.
The timeout's [inference-failed] note now says the call timed out, and its closing advice "If this recurs with the same cause, change approach…" → "If a reply is still owed from that turn, you can give it now." Where the call made again timed out too, it adds "The call was made again, and it timed out again."

— Lottie-2661

nissa-seru mentioned this pull request Oct 16, 2026

fix(mcpl): answer each way a server's inference ends; a failure is the error, a completion its real result #380

Merged

Anarchid mentioned this pull request Oct 17, 2026

stop() neither cancels nor waits for an in-flight MCPL inference/request #418

Closed

nissa-seru marked this pull request as ready for review October 22, 2026 00:33

github-actions Bot reviewed Oct 22, 2026

View reviewed changes

nissa-seru force-pushed the nigel/268-progress-deadline branch from e4b5e37 to eda448f Compare October 28, 2026 02:41

nissa-seru changed the title ~~fix(inference): a progress deadline on resident inference, retried when it expires~~ feat(inference)!: cut an AgentFramework model call that stops making progress, and give a deadline expiry one fresh call Oct 28, 2026

nissa-seru force-pushed the nigel/268-progress-deadline branch from eda448f to 77997a5 Compare October 28, 2026 06:23

Nigel-2405 added 2 commits October 28, 2026 23:05

nissa-seru merged commit 1b596c2 into anima-research:main Oct 29, 2026
12 checks passed

nissa-seru mentioned this pull request Oct 29, 2026

fix(inference): clear an ended agent's failure accounting #897

Merged

1 task done

Navigation Menu

Repository navigation

feat(inference)!: cut an AgentFramework model call that stops making progress, and give a deadline expiry one fresh call - #378

feat(inference)!: cut an AgentFramework model call that stops making progress, and give a deadline expiry one fresh call#378
nissa-seru merged 3 commits into
anima-research:mainfrom
nissa-seru:nigel/268-progress-deadline

nissa-seru commented Oct 15, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Oct 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

github-actions Bot Oct 22, 2026

Uh oh!

nissa-seru Oct 22, 2026

Uh oh!

github-actions Bot Oct 22, 2026

Uh oh!

nissa-seru Oct 22, 2026

Uh oh!

github-actions Bot Oct 22, 2026

Uh oh!

nissa-seru Oct 22, 2026

Uh oh!

nissa-seru commented Oct 22, 2026

Uh oh!

Uh oh!

nissa-seru commented Oct 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

nissa-seru commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

For hosts

Where the values come from

Tests

Not verified

Uh oh!

greptile-apps Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Automated review

Does this change belong in the project?

Findings

Uh oh!

github-actions Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

nissa-seru Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

nissa-seru Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

nissa-seru Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

nissa-seru commented Oct 22, 2026

Uh oh!

Uh oh!

nissa-seru commented Oct 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

nissa-seru commented Oct 15, 2026 •

edited

Loading

greptile-apps Bot commented Oct 16, 2026 •

edited

Loading

github-actions Bot left a comment •

edited

Loading