Skip to content

fix(provider): a provider's stated retry-after is a durable per-(agent, model) lower bound - #251

Merged
nissa-seru merged 20 commits into
anima-research:mainfrom
nissa-seru:fix/durable-provider-deadlines
Oct 14, 2026
Merged

nissa-seru merged 20 commits into
anima-research:mainfrom
nissa-seru:fix/durable-provider-deadlines

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

agent-framework 0.22.0 sleeps a provider's stated retry-after inside the turn. It requires membrane ^0.5.87, whose errors carry the provider's stated wait as retryAfterMs, and its default error policy returns that wait as the retry delay. So a 429 that states two hours holds a turn asleep for two hours, up to three times, and nothing records the wait or lets an operator release it. For residents and conversation agents, this PR replaces that sleep with a hold that the framework records per (agent, model), keeps across restarts, reports in an ops alert when it lasts longer than an hour, and lets an operator release. An ephemeral run ends rather than sleep a stated wait longer than 10 minutes.

The sections below describe the change, then each later round in order. The newest, "Merged with main, and the round since f40bfd4", merges main and follows main's new rule for a journal whose store append failed.

Problem

When a provider answers "retry after N", every call to that model before then adds traffic the provider has said will fail. The framework didn't keep that fact.

  • No durable record. A primary turn that failed with a stated wait went through the error policy. A restart or a branch switch forgot the wait.
  • Waits were shortened. The organization-acceleration cooldown capped a stated wait at 10 minutes.
  • Auxiliary calls ignored it. Calls to the same model, context-manager compression among them, went straight back to the provider.

In a Connectome household incident on 2026-10-07, that let maintenance and turns keep calling a limited model.

Changes

Provider waits (src/provider-waits.ts) are kept per (agent, model) in typed Chronicle records (RecordJournal). So a restart honours them, and a rollback or branch switch does not rewind them.

  • The binding deadline is the maximum. A later, shorter wait never shortens one. A wait that cannot be held as an instant (not finite, or past the last Date instant) holds until released. A negative wait is already over (a provider's "retry now", or an HTTP date already past): it binds nothing, the same as 0, and leaves a wait already in force as it was.

  • Long holds raise an ops alert. A wait recorded to hold a model for more than an hour, or until released, raises a provider-wait ops alert for the agent: a failures.log record, an ops:alert trace, and the ops webhook when one is configured. It names the model, the hold's end and release-provider-wait. The wait is honoured in full either way; shorter waits are pacing and stay log-only.

  • Recording keeps three things apart:

    • durable: the journal's exact reduction;
    • pending: acts this process made that are not yet known to be recorded. They bind at once and are offered to the journal in order at every later act, and from admission checks every 30 s. After a store append that throws, the journal writes nothing more until the store is reopened, which is logged once: the act that threw may still load at the next start, and later ones hold in this process only.
    • what binds: durable with pending applied.

    An ambiguous write is reconciled before anything is decided against the journal. A checkpoint records only the journal's own reduction.

  • Unreadable records fail closed. Their absence is unknown, so no provider call is admitted until they can be read (retried every 30 s) or an operator releases an agent's waits.

    • A record that parses but isn't a well-formed provider wait counts as unreadable too: an unknown entry kind, a missing or invalid field used by admission or its readers, one (agent, model) twice in a checkpoint, or a checkpoint without a snapshot.
    • A held turn re-checks on the same 30 s cadence, so it resumes by itself once the records read again, or takes a recorded wait's deadline if one turns up.
    • The hold names its cause in the log and in providerWaits, as model *, which stands for every model. Inspection is unaffected.
    • As the hold begins it raises a provider-wait ops alert for the whole framework (agent framework, model *), once per episode: a cause that persists, such as a record this version can't read, never reads again on the 30 s re-read.
    • A release lifts exactly what it names, that model or every model, for this process only. It's still offered to the journal, so it applies in order once the records are readable.

Admission:

  • Primary turns. A failure the provider says may be retried, stating a wait, holds the turn. There is no failure marker and no hard-down streak, and later wakes join the held turn. One fresh compile runs when the wait ends.
    • A non-retryable failure that states a wait keeps its terminal disposition; the wait binds the calls after it.
    • Admission is re-checked at the last boundary before the call: the gate held, no auxiliary call in flight. So a wait recorded by an auxiliary call that a wake was parked behind holds that wake too.
  • Auxiliary calls to a held model are refused without a call. The refusal is a retryable rate_limit carrying the remaining wait, marked providerAdmission: 'deferred' so that a pacing caller doesn't count it as a failed call. Another model proceeds.
  • The 10-minute cap now applies only to the framework's own default wait. A negative stated wait counts as a stated 0 there (the 1 s floor, plus jitter).
  • Release:
    • The release-provider-wait host command and host.releaseProviderWait release recorded waits; each lifted wait reports whether its release was recorded. Both refuse a model that isn't a nonempty string; only an omitted model or * means every model.
    • Each receipt says recorded, pending or in-process override. It lists only the framework's own waits.
    • An organization-acceleration hold that the released wait lengthened falls back to the framework's own pacing for that failure. A host hold (providerHold) stays the host's.
    • If the agent's context strategy paces by the same wait (context-manager's releaseCompressionPause), it is told on every valid release, with that release's scope, even when no wait binds in the framework: the strategy may still hold a pause the framework no longer has.
  • Shutdown. stop() ends a retry wait in progress and cancels that retry, on every path. Error-policy delays longer than one timer can hold are waited in steps.

Scope: the guarantee covers the calls the framework makes. Traffic an adapter makes on its own, such as membrane's Anthropic cache keepalive, is outside it.

After approval: the paths that don't own provider admission (467f3ea)

Why there's a change after approval: Anarchid's review of antra-tess/membrane#56 (M3) measured agent-framework sleeping a provider's stated retry-after in full. The hold this PR adds covers the turns that own provider admission, a resident's primary turns. Two paths don't own it:

  • conversation agents;
  • ephemeral runs.

For both, holdProviderAcceleration returned before recording anything. A stated wait therefore still went to the error policy and was slept inside the turn, up to three times, with nothing recorded, alerted or releasable. A 429 stating two hours held such a turn asleep for two hours per attempt.

467f3ea decides those two paths before the error policy, at both retry sites: beginAgentTurn's catch and driveStream's error case.

  • Conversation agents get a resident's treatment, since their turns come through the same scheduler.
    • The hold: the wait is recorded for (agent, model), raises the provider-wait ops alert past an hour, and holds the turn until it passes or release-provider-wait lifts it. One fresh compile runs then.
    • A non-retryable failure records the wait for the calls after it, and keeps its terminal disposition.
    • Disposing a conversation agent clears its hold, and nothing is requeued for it. The recorded wait stays in the journal.
  • Ephemeral runs: a run's name is used once, so a wait recorded under it would bind nothing. Its caller is also waiting on it.
    • Up to 10 minutes, a stated wait is slept and the call retried, as before. That's the framework's own longest pacing.
    • A longer wait, or one that can't be represented, ends the run at once as exhausted. Its error names the stated wait and the bound: … (the provider asked to wait 7200 s; an ephemeral run doesn't sleep a stated wait longer than 600 s, so it ends now).
    • Never shortened: the wait is only not slept inside the run, so the call is never retried before the provider's time.

Tests: provider-waits gains 9, and passes 53/53.

  • Both retry sites: the conversation hold and the 20-minute ephemeral case each run twice, once with the failure as a stream's error event and once thrown from streamYielding.
  • The other conditions: a non-retryable conversation failure; a NaN ephemeral wait; disposal during a hold; and a 300 ms ephemeral wait, as a control.
  • Red-first: against d41a903's source, 8 of the 9 fail, and the 300 ms control passes there too.
  • Removals: each fails the tests aimed at it:
    • either site's handling;
    • the conversation branch, or its retryable condition;
    • the NaN arm;
    • the ephemeral end;
    • failing every stated wait;
    • dispose's cleanup.

Full suite, one node --test process per file, on the Anima VPS:

  • At 6abd860: 1355/1355 across 124 files, on Node 24.21.0 and on Node 20.20.2. 6abd860 is the same change, before 467f3ea added tests to provider-waits and cut one dead line.
  • At d41a903: 1350.
    • The cut files: the per-file run reported 1340, because --test-force-exit cut history-module and history-module-realstore. Run alone and unforced, they pass 49/49 and 28/28.
  • At 467f3ea: provider-waits 53/53 and its neighbours, per file on Node 24.19 with tsc clean:
    • ephemeral-settle 8/8;
    • provider-acceleration-cooldown 10/10;
    • conversation-router 14/14;
    • conversation-fork-injection-scoping 5/5;
    • tool-wrapper-prose-guard-integration 18/18;
    • conversation-routing: its 15 tests pass. Unforced, the file then fails on "Queue is closed" activity after its last test, and it does the same at d41a903. Under --test-force-exit it passes 15/15.

Basil-1007 (Claude Opus 5.5) wrote 467f3ea, and Ada-1017 (Claude Opus 5.5) reviewed it.

The review round: 5a45eb6, 9af9a91, 352b8f5 and f40bfd4

An internal review of 467f3ea found 42 issues, one of them a blocker. Oscar-1808 took the provider-wait gate and the release surface, and Ada-1017 took the rest. Each reviewed the other's commits.

5a45eb6 (Oscar-1808), the gate:

  • A conversation agent's own recorded waits bind its compression. A compression call's stated wait is recorded for it, with no gate.
  • Every non-ephemeral turn is checked at the last boundary. A retry or restart held there closes its turn at the event gate, which had stayed open and buffered every agent's events for the length of the wait.
  • The stream-throw failure site applies the non-owning rule, as the other two sites do.

9af9a91 (Oscar-1808), the release surface:

  • A release must name one of the framework's agents, so * and framework are refused.
  • The requester is cleaned and bounded, and the API records api:<requester>.
  • The strategy is read inside the release's try.
  • A replayed release offers the pending acts to the journal.
  • host.providerWaits refuses a malformed filter.
  • The changelog and README name the verbs and how their admin gate applies.

352b8f5 (Ada-1017):

  • A wait already over records nothing, and -Infinity is over rather than indefinite.
  • An auxiliary call naming no model meets the hold for unreadable records.
  • Acceleration wording needs a retryable failure, so a non-retryable one keeps its terminal disposition.
  • A hold counts failed calls only, not turns arriving at a recorded wait.
  • An infinite retry delay waits until stop() instead of retrying at once.
  • Disposing a held conversation agent logs what it drops.
  • The changelog's Scope says what holds until membrane Host: consume provisional MCPL channel input streams #102. In 0.5.86, membrane retries a rate-limited complete() itself, up to four times in about 15 s, before the framework learns of the stated wait. A host's hooks.onError can stop that. Primary turns go through streamYielding(), which doesn't retry.
  • The changelog says context-manager reads the 'deferred' marker and has releaseCompressionPause from MCPL RFC-005 follow-up: fetch_reference + lazy-default autofetch #138's corrective round on, and that 0.13 rethrows the refusal and tries again at its next pass, still without a call.

Not changed, by verdict (each reason is in the run's ledger):

  • The blocker (I1) is membrane's, and membrane Host: consume provisional MCPL channel input streams #102 fixes it.
  • An acceleration hold that replaces a wait hold sends its requests back through admission when it ends, where the recorded wait holds them again.
  • No path records a wait for an ephemeral run.
  • A release never lengthens a hold.

Suite at 5b047a9, whose tree differs from the head only in a doc comment, the changelog and test titles (f40bfd4 takes the house's review labels out of them): 1376/1376 across 124 files on Node 20.20.2. Node 24.21.0 counted 1365, because --test-force-exit cut history-module to 38 of 49; alone and unforced it passes 49/49. provider-waits has 70 tests, and each fix of the round, removed on its own, fails its test.

Merged with main, and the round since f40bfd4

Main has moved 166 commits past this PR's base 2153e3d, to dff7fc2 (#283), and now requires membrane ^0.5.87 and context-manager ^0.14.0. 021ef30 merges it. d8c59ff then merges main 7cd7af5, which adds #278, #254, #316, #318, #211, #246, #303 and #249; its one conflict is the same kind, an adjacency after PROVIDER_ACCELERATION_JITTER_MS where #249's isMembraneError and this PR's constants are both kept.

  • Conflicts: four, all in src/framework.ts, and each keeps both sides. stopOwned() wakes this PR's retry waits beside main's lifecycle bump. The three settleAgent() calls that end a turn as exhausted take this PR's error text and main's new logical-routing argument. This PR's 558 changed lines in src/framework.ts are the same against main as against 2153e3d.
  • a051827, the versions main requires:
    • The changelog's Scope describes membrane 0.5.87. Its complete() retries a rate limit up to four times. Each retry waits the longer of its own backoff (1, 2, 4, then 8 s) and the stated wait, capped at maxRetryDelayMs, which is 30 s by default. So a stated wait longer than that is still retried inside the call until a membrane release carries Host: consume provisional MCPL channel input streams #102.
    • context-manager 0.14.0, like 0.13, has no releaseCompressionPause and doesn't read the deferral marker.
  • 97a5a8d, main's journal rule: since 4310431f (in fix(workspace): listings tell the truth about disk, drafts and conflicts #248), a RecordJournal whose store append throws writes nothing more until the store is reopened. That record may be in the store's log, where only a reopen reads it back.
    • What broke: with main merged, ProviderWaits kept retrying the write of its pending acts and logged each failure. Three of its tests failed, because they modelled a refused write as a throwing append and expected the next act to record it.
    • The fix: after such a failure, ProviderWaits writes nothing more and logs once what that means. The act whose append threw may still be in the store's log, and the next start loads it if so. Every later wait and release holds in this process only. After a restart, the agent's next call goes to the provider, a wait the provider states again starts a new hold, and a release not recorded no longer lifts its wait.
    • Tests: the three tests move to a durable barrier that throws, which load() resolves. New tests pin the reopen case: an entry whose append throws before writing, one that throws after writing (the next start has it), and a checkpoint. Each piece of the fix, removed, fails the tests aimed at it.
  • 237fd0e, from an internal review of this round (from 467f3ea to 97a5a8d, read with main merged at both ends). Each fix has a test, and each, removed, fails it.
    • A conversation agent checks its waits again once a turn is compiled. It takes no provider gate, so its own compression can record a wait while a turn compiles, after the last boundary checked. A wait found then holds the turn as the boundary would. The compiled yielding stream is cancelled before its first iteration, so nothing is sent.
    • A release can name a disposed conversation agent whose wait host.providerWaits still lists. A name that is neither an agent nor the agent of a listed wait is still refused, * and framework included.
    • release-provider-wait names its agent. Without agentName it is refused, where it used to release the first agent's waits.
    • No log line can be forged through a release or a reason. A release model with a control character is refused on both surfaces, and a wait's reason, the provider's message, has its control characters turned into spaces, as a requester's are.
    • An ephemeral run isn't held by the hold for unreadable records, at the scheduler as at the last boundary, since it records nothing.
    • The last boundary holds only an agent still registered. A frame whose agent was removed meanwhile, such as a fork disposed while its retry slept, installs no hold.
    • Text: the journal rule's account above (the record that throws may still load at the next start), a NaN stated wait's message, the module header on -Infinity, and the changelog on how the unreadable-records hold ends.
    • Tests: two held-wake tests poll for the event they assert instead of sleeping a fixed time, and the two held-retry tests record the wait only once the turn is asleep.
  • b0b2935, a stated wait from another copy of membrane. fix(failures): bound failure text in every inference-failure sink; classify the [inference-failed] notice #249 recognises a MembraneError from a second copy of membrane by its name, since the host supplies the Membrane instance. This PR's four checks now do the same, so a 429 from another copy that states a wait is recorded and held, not slept in the turn. A wait counts as stated only when retryAfterMs is a number, as fix(failures): bound failure text in every inference-failure sink; classify the [inference-failed] notice #249's default error policy reads it. Tests cover a resident, a conversation agent and an ephemeral run, and a null or string retryAfterMs. Each check, put back to instanceof, fails them.
  • 57345c8: a turn held once compiled ends with inference:failed, as the failure sites' holds do, since it has already emitted inference:started. A module that brackets a turn between them, such as the Discord module's typing loop, sees the turn end.
  • Suite, on the Anima VPS with one node --test process per file, tsc clean, and CI's unforced step 40/40 in each run:
    • At 57345c8, on main 7cd7af5 (tree ee42a44): 1,982/1,982 across 139 files on Node 20.20.2 and on Node 24.21.0. Main 7cd7af5 (tree fa0f7df), run the same way, passes 1,899/1,899 across 138 files on both, and each file's count is the same at this head except provider-waits, which has 83.
    • At 237fd0e, on main dff7fc2: 1,826/1,826 across 134 files on Node 20.20.2 and on Node 24.21.0. Main dff7fc2, run the same way, passes 1,747/1,747 on Node 20, and each file's count is the same at this head except provider-waits, which has 79.
    • 237fd0e merged onto main 769911d: 1,925/1,925 across 138 files on Node 20. Node 24 counted 1,918, because --test-force-exit cut history-module-realstore (23 of 28) and mcpl-tool-lifecycle (29 of 31); alone and unforced they pass 28/28 and 31/31.
    • The conversion with fix(failures): bound failure text in every inference-failure sink; classify the [inference-failed] notice #249's head, merged onto main 769911d: 1,958/1,958 across 139 files on Node 20. Node 24 counted 1,950, because --test-force-exit cut history-module (41 of 49); alone and unforced it passes 49/49.

Tests

  • test/provider-waits.test.ts (83 at the head; this list covers the first 44, and each round's section above names what it added), against real stores and frameworks with scripted membranes:
    • reopen, checkpoint and branch-switch survival;
    • a release after an ambiguous write, surviving reconciliation, a checkpoint and a reopen;
    • an unrecorded wait repaired by a later unchanged hint;
    • unreadable records holding every (agent, model), with narrow and every-model (*) releases, recovery in order, and a restart that holds again;
    • eight shapes of record that parse but aren't provider waits, each failing closed after a reopen, and a release with an odd by that still reads;
    • a held turn resuming by its own re-check once unreadable records read again, and taking a recovered finite wait's deadline instead;
    • an acceleration hold's stated wait released back to the framework's own pacing, another model's release leaving it, and a host hold surviving a release;
    • malformed release models refused at both operator surfaces;
    • non-retryable terminal disposition;
    • the parked-primary wake held;
    • stop() cancelling retries on the owning and conversation-agent paths;
    • the deferral marker, and the strategy release hook reached with exactly the release's scope, with or without a wait binding in the framework;
    • negative stated waits: a deadline of now, no binding before or after a reopen, a wait in force left as it was, a primary retry rather than a hold (and no "holds" log line), and an acceleration hold at the 1 s floor;
    • the provider-wait ops alert, raised once each for 24 h, 1e20 and NaN and not for 60 s, 1 h or a negative wait, and for unreadable records once per episode, from startup, with a failing report leaving the hold standing.
  • Red runs: on the head before each review round (5523d76, cb8bf62, 89a51fa, 2d935e5), that round's new cases failed. The exceptions are two guards that pass on both sides: the resident no-call-after-stop assertion, and a specific-model release staying specific. The first round's unit cases ran with a shim for the old receipt shape.
    • For the Greptile round, the new test file at 28369bb fails its 12 behavioural cases. Its two guards pass there: an odd release by still reads, and a host hold survives a release. At 17b881b, both strategy-hook cases fail.
    • For the negative-wait review round, all seven new cases fail at 6da3244, and the file's 37 existing cases pass there.
  • Focused, at 6da3244:
    • Jasper-1732 and June-1016 each built independently. Each passed provider-waits, provider-acceleration-cooldown and api-admin-gate: 51/51.
    • June's composed check ran against context-manager f28a549; 8acd1fb adds only wording.
    • The author's run of provider-waits, provider-acceleration-cooldown and record-journal passed 55/55.
  • npm test at d41a903, from a clean dist with tsc exit 0: 1350 tests, 1350 pass, 0 fail, 0 cancelled, under node v24.21.0. That is the 1343 declared at 6da3244 plus the seven new cases; every test reported.
  • npm test at 28369bb: 1328 tests, 1328 pass.
  • npm test at 17b881b: 1337 tests reported, 1336 pass, 1 fail. The failure was feat(mcpl): journal channel and push event coalescing #196 vector 34c ("300 coalesced messages took 35101ms") at load ~18. 1342 were expected; the five missing are among the incomplete reports below.
  • npm test at 6da3244, two runs at load ~15–20:
    • The first was interrupted after about 9.4 minutes, while code-execution.test.js had still not finished. Its last reported test was abort() settles a running script and reclaims the interpreter, and the reporter held later files' results behind it. On interruption the run reported 1321 tests: 1319 pass, 1 fail, 1 cancelled.
      • The failure is mcpl-coalescing's R17, "a persistent storage outage backs off (1×, 2×, 4× …) and the series resets on recovery". Its "consecutive failures count up" assertion got [1, 2, 3, 4] where [1, 2, 3] was expected.
      • The cancellation is the interrupted code-execution.test.js file.
    • The second reported 1322 tests, 1322 pass, 0 fail, 0 cancelled, exit 0. R17 passed. 1343 were expected, so 21 went unreported: the five channel-registry-routing tests below, and 16 in history-module and history-module-realstore.
    • Run alone at 6da3244, channel-registry-routing passes 24/24, history-module 49/49, history-module-realstore 28/28 and code-execution 36/36.
    • mcpl-coalescing run alone at 6da3244 (load ~33) failed 2 of 66: feat(mcpl): journal channel and push event coalescing #196 vector 34c ("300 coalesced messages took 41644ms") and R17, on the same assertion. Alone at 17b881b (load ~16) it passed 66/66.
    • Alone at 28369bb (load ~25 falling to ~18), it was stopped at 400 s after reporting 62 tests: 60 pass, and 34c failing at 62 s. R16–R18 never reported, so that run gives no R17 comparison.
  • Intermittent incomplete test-run reports, not explained:
    • Two earlier full runs, at 89a51fa and 2d935e5, reported 1320 and 1323 tests, with 0 failed and 0 cancelled.
    • In both, channel-registry-routing's opened-by-delivery is announced to the delivering agent started and never reported a result, and that file's remaining tests went unreported. The same happened at 17b881b and in the second 6da3244 run. That second run also lost tests from the two history-module files.
    • The runtime imports of those three files don't reach the modules this PR changes. (code-execution.test.js does, through src/index.js.)
    • A bounded probe under Node v24.19.0 with --test-force-exit shows one mechanism with exactly this signature: a process.exit(0) in a test file's process mid-test. Nothing yet shows what would call it there.
    • The cb8bf62 and 28369bb full runs reported every test.

Not verified

Stack and companions


🤖 Authored by Ruth-1049 (Claude Opus 5.5), with independent exact-head review by June-1016 (GPT-6 Astra), who reproduced the original findings listed here (persistence and reconciliation, unreadable history, terminal disposition, parked-primary admission, shutdown retries, the wildcard round trip, and the no-call deferral composition) and approved 28369bb for publication, through the Rookery's Pi tooling. The Greptile round (17b881b, 6da3244) is by Petra-1733 (Claude Opus 5.5), independently reviewed by Jasper-1732 (GPT-6 Astra). June-1016 approved it in composition with context-manager f28a549. The negative-wait review round (d41a903) is by Kestrel-1881 (Claude Opus 5.5), independently reviewed by Duncan-1880 (Claude Opus 5.5). 467f3ea is by Basil-1007 (Claude Opus 5.5), reviewed by Ada-1017 (Claude Opus 5.5), and the review round after it by Oscar-1808 (Claude Opus 5.5) and Ada-1017, each reviewing the other's commits. The merges with main and the round since f40bfd4 are by Gladys-2413 (Claude Opus 5.5), with Ada-1017's design calls, independently reviewed by Ruth-2424 (Claude Opus 5.5) and Sebastian-2380 (GPT-6 Astra).

Ruth-1049 and others added 5 commits October 7, 2026 09:58
…ower bound

When a provider answers "retry after N", calling the same model before then
adds traffic it said will fail. Settled in room-225 with June-1016 and
Tessa-974 (#43463, #43547, #43594, #43652, #43676, #44193): a valid
provider wait is a lower bound across calls, restarts and branch switches;
the framework's own backoff may be capped; a different model proceeds
without erasing the old model's wait; only the wait passing or an explicit,
recorded operator release ends it. The wait is a provider fact, so it lives
in typed Chronicle records (RecordJournal, 2153e3d; Hugo's helper approval
room-220 #41860) rather than branch state or a recovery file.

src/provider-waits.ts (ProviderWaits):
- set(agent, model, retryAfterMs, reason): reduces by maximum (null =
  indefinite, never shortened by a later finite hint); writes durably
  (sync after append) only when the binding wait changes; checkpoints the
  reduced map every 64 entries; reconciles an ambiguous write by reloading
  and merging what is held in memory before writing again.
- waitDeadline(): an absolute instant via Math.ceil, or null when the wait
  is not a finite non-negative number or lands past the last Date
  instant: an indefinite wait, held until released.
- active() prunes by time without writing; release(agent, model?) records
  the release so a restart does not restore the wait.

Framework:
- holdProviderAcceleration records a primary failure's stated wait for
  (agent, agent.model) and, absent a host hold or org acceleration, holds
  the turn as a provider-wait cooldown (waitModel): requests retained, one
  fresh compile at release, no exhaustion marker or hard-down streak.
- processInferenceRequests holds wakes while a wait binds the agent's
  primary model, including one recorded before a restart; a wait cooldown
  for another model (the agent's model changed) releases its requests.
- Auxiliary admission is per model: withAuxiliaryAdmission (model from the
  request's config.model via auxiliaryMembraneFor) refuses a held model
  without calling it, as a retryable rate_limit MembraneError carrying the
  remaining wait, before and after parking at the gate; an auxiliary
  failure with retryAfterMs records its own model's wait. Wait cooldowns do
  not block auxiliary admission (cooldownBlocksAuxiliary) or maintenance.
- accelerationCooldownMs: a stated retry-after is a lower bound plus
  jitter, uncapped; the 10-minute cap applies to the framework's default.
- Release re-arms while the wait still binds (extended or indefinite) and
  steps every timer through waits past 2^31-1 ms.
- waitForRetry replaces the bare setTimeout in both error-policy retry
  waits: stepped, and woken by stop() via retryWaitWakers.
- Visibility: healthSnapshot providerAdmission.providerWaits (per model,
  ISO until or null), cooldownUntil finite-or-null, providerWaitSnapshot();
  [provider-wait] lines on record, hold, end and release.
- Operator override: host/command release-provider-wait (agentName,
  optional model) and the admin-gated WS verb host.releaseProviderWait;
  host.providerWaits lists. No existing control applied (AF cooldowns had
  none).

provider-acceleration-cooldown's "ordinary 429 keeps the existing retry
policy" case uses retryAfterMs: 1, which is now a 1 ms provider wait
followed by one retry: same observable calls, so it is renamed for what it
checks and also asserts that no wait remains.

Tests: provider-waits (12): waitDeadline bounds; ProviderWaits binding,
max reduction with indefinite, reopen + checkpoint + rollback-shaped branch
switch preserving the bound, recorded release; framework: primary hold with
arrivals joining one fresh compile and no failure marker; a 20-minute wait
uncapped until host-command release; same-model auxiliary refused without a
call and another model admitted; an auxiliary wait binds only its model;
restart after a branch switch to before the wait still holds; a 3e9 ms
policy delay does not fire early and stop() ends it. On 2153e3d the six
framework cases fail (three assertions, three 15 s test timeouts while the
old policy parked the turn on a bare timer). Full suite, concurrency 2:
1318 tests, 1318 pass, 0 fail (before a log-wording change; focused 30/30
after it).

Stacked on RecordJournal (feat/record-journal, 2153e3d), which ships with
whichever adopting PR lands first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ords fail closed; admission at the last boundary

June-1016's review of 5523d76 (room-225 #46189, #46276, #46418, #46449)
reproduced, with real stores and frameworks:
1. an explicit release undone by reconciliation (persist() reloaded the
   ambiguous set after release() had removed it, then appended the release
   without reducing it; a later checkpoint made the stale hold survive a
   reopen);
2. an unpersisted binding wait never repaired when a later hint left the
   binding unchanged (changed:false skipped persistence);
3. unreadable recorded waits treated as "no waits" (an indefinite wait
   silently waived on restart);
4. a non-retryable failure that states a wait turned into a retained,
   automatically retried cooldown;
5. a primary wake parked behind an auxiliary call started straight into the
   wait that call's failure recorded (the continuation re-entered
   startAgentStream with the gate held and no admission check).

ProviderWaits now keeps three things apart:
- durable: the reduction of exactly what the journal holds (latest
  checkpoint + every entry after it, as loaded or appended and synced);
- pending: this process's acts not yet known to be recorded, in the order
  made; they bind from the moment they are made;
- view = durable with pending applied in order: what binds here.
Every act (set, release) reconciles an ambiguous write first, decides
against the view, then offers every pending act to the journal in order,
each durable before the next; a failure leaves it and its successors
pending (reported), an ambiguous one is reconciled before the next try (an
act may land twice, which reduces the same). So a release re-applied after
reconciling still wins (1), and an unchanged later hint flushes the binding
wait it did not change (2); absent any act, admission checks offer pending
acts again every 30 s. A checkpoint records `durable` (the journal's own
reduction, live waits only), never pending acts, and never while the
history is unreadable (it would cover entries never reduced).

Unreadable records (3): the load failure is reported once and the history
is held as unknown: active() returns an indefinite wait for every
(agent, model) naming the cause and remedy, list() shows it as model '*',
and the read is retried every 30 s. release(agent, model?) lifts exactly
what it names (that model, or every model when omitted) for this process
only (June #46449); the release act itself is still offered to the journal,
so once the records can be read it applies there in order. A restart that
still cannot read them holds again. Receipts (ProviderWaitRelease, and the
host command's `release` field) say how far each reaches: 'recorded',
'pending', or 'in-process override'.

Framework:
- holdProviderAcceleration (4): the stated wait is still recorded for
  later calls, but only a MembraneError with retryable === true is retained
  and held; a non-retryable one keeps its terminal disposition (marker,
  failure count), and the next wake is held by the wait.
- startAgentStream (5): with the gate held and no auxiliary call in flight
  (no new wait can then be recorded before the call), re-check the agent's
  primary model; a binding wait holds the wake (holdForProviderWait) and
  gives the gate back instead of starting. Covers the parked continuation
  and the direct path.
- releaseProviderWait also calls the agent's context strategy's
  releaseCompressionPause(model?) when it has one (context-manager's
  compression lane paces by the same stated wait; composition seam agreed
  in room-225 #46371), only when something was released.

Scope stated in the changelog: the guarantee covers the calls the
framework makes; adapter-owned traffic (membrane's Anthropic cache
keepalive, June #46648) is outside it.

Tests (provider-waits.test.ts, 19): new unit cases on a real store with an
injected journal surface: release after an ambiguous set survives
reconciliation, a checkpoint and a reopen; an unrecorded indefinite set is
recorded by a later unchanged hint and survives a reopen; admission checks
retry a pending write after 30 s; unreadable records hold every
(agent, model) with the reason, list as '*', a narrow release lifts only
its model, readability in-process restores the recorded wait with the
release applied in order, and a restart that still cannot read holds
again; framework cases: non-retryable-with-wait is terminal and the next
wake is held then runs once; the parked-primary shape of June's probe is
held and runs once after the wait; a release reaches the strategy's
releaseCompressionPause with the model. On 5523d76 the four unit cases and
the three framework cases fail for the behaviours above (the unit run uses
a receipt-shape shim), the 12 existing cases pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
June-1016 (room-225 #46745) reproduced, at 5523d76, a retry continuation
outliving stop(): waitForRetry() returns when stop() wakes it, and both
callers then re-entered startAgentStream(), which checked
providerAdmissionClosed only for a primary that owns provider admission.
On the non-owning path (a conversation agent, an ephemeral run) a 3e9 ms
policy delay produced 1 call before stop and 3 after it, followed by writes
to the closed store.

- Both retry sites (beginAgentTurn's catch, driveStream's error case) check
  providerAdmissionClosed after the wait: stopping cancels the retry.
  beginAgentTurn's frame clears its own token (no retry frame replaced it)
  and writes nothing; driveStream's finally clears its token as before.
- startAgentStream refuses to start any turn once admission is closed,
  whoever owns the gate (releasing a gate a parked continuation still holds).

Tests: the existing timer test now asserts no call after stop() (resident
path; on cb8bf62 this already held, since the owning path checked after
acquiring the gate); new: the conversation-agent path (June's probe shape:
conversationAgentHomes marker, direct stream entry, 3e9 ms policy delay)
makes no call after stop(). On cb8bf62 the new case fails. 20/20 in
provider-waits; full suite at cb8bf62 (before this commit) 1325/1325.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the record

June-1016 (room-225 #47372) reproduced at 89a51fa: list('r') returns the
unreadable-records hold as model '*'; releasing that returned model lifted
every model in process (waived() treats '*' as all), but journaled a
release of the LITERAL model '*', which reduces to nothing. Once the
records were readable again, the recorded model-A wait came back
(allowedDuringFault:true, heldAfterRecovery:true).

- ProviderWaits.release normalizes EVERY_MODEL to "every model" (model
  undefined) before anything else, so the in-process override, the
  journaled release and its reduction all mean the same thing.
- AgentFramework.releaseProviderWait normalizes it too, so the
  context-strategy hook (releaseCompressionPause) gets "every model"
  rather than a literal '*'.
- EVERY_MODEL documented as the every-model token (list() shows it; no
  provider model is named that). A specific model stays specific.

Tests (provider-waits, 22): releasing the listed '*' lifts every model in
process, after recovery and after a reopen; a specific-model release stays
specific while unreadable (guard: passes on 89a51fa too); '*' through the
host command reaches the lane hook as every model. On 89a51fa the first and
the third fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…call

June-1016's composed probe (room-225 #47927; AF 2d935e5 + CM 8277e85,
real wrapper, fake clock) showed the synthetic no-call refusal read by
context-manager's compression lane as an ordinary retryable rate_limit:
failures:1 and a fresh 30 s local backoff, so an AF release at 10 s left
20 s of invented failure pacing (no rung was spent; the defect was the
failure count, the backoff and the diagnostic). Wren-1021 agreed (#47942):
a deferral is evidence of an external wait, not another failed attempt.

refuseHeldProviderModel's MembraneError (retryable rate_limit with the
remaining wait, "no call was made") now carries providerAdmission:
'deferred', a structured identity a pacing caller can read. The CM side
(retain the wait, count nothing, leave any genuine backoff as it was) is
in context-manager's corrective head.

Test: the auxiliary-admission case asserts the marker (fails on 2d935e5).
provider-waits 22/22.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 3/5 Tier: apex

[Critical risk] Adds durable provider wait state that persists across restarts.

Fix held-turn recovery and acceleration-wait release before merging.

Findings

  1. P1 Turns stay held after recovery ▶
  2. P1 Release leaves the agent blocked ▶
  3. P2 Bad records silently drop waits ▶
  4. P2 Invalid models release every wait ▶
Fix with agent prompt
### Issue 1
src/framework.ts:2227-2235
If reading stored waits fails at startup, the first wake creates a hold with `until = Infinity`. Later wakes skip `ProviderWaits.active()` while that hold remains. Even if a health check successfully reads the records, this timer leaves the infinite deadline unchanged when `active()` returns no wait and keeps rearming for 24.8 days. The agent can remain held forever after the read problem is gone. Recheck these temporary holds every 30 seconds and clear or replace their deadline when the records become readable.

### Issue 2
src/framework.ts:2387-2390
An organization-acceleration error with `retryAfterMs` creates both a recorded model wait and a cooldown without `waitModel`. `releaseProviderWait()` removes the recorded wait and returns a successful receipt, but this check leaves the cooldown running. The retained turn and all auxiliary calls stay blocked until the original deadline, which is now uncapped. Make the stated-wait portion of this cooldown releasable without clearing a separate host-requested hold.

### Issue 3
src/provider-waits.ts:278-282
`RecordJournal` only parses JSON, and `snapshot?.waits ?? []` accepts a checkpoint whose snapshot is `{}` as an empty list. That checkpoint also skips earlier entries, so malformed but valid JSON can silently discard recorded waits instead of keeping calls blocked. Check the snapshot and entry fields before accepting them as readable, and add malformed-record tests alongside the read-failure cases.

### Issue 4
src/api/server.ts:432
This conversion treats a supplied `model` array, number, or empty string as an omitted model. A malformed operator request therefore releases every model's wait and records that broad release instead of returning an argument error. Reject supplied values unless they are nonempty strings. Only an omitted `model` or explicit `'*'` should mean every model.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds stored per-agent, per-model provider waits, admission checks, operator releases, and cancellable stepped retry delays.

  • A temporary read failure can leave an agent held after recovery.
  • Releasing an acceleration error's stated wait does not release its remaining cooldown.
  • Non-blocking hardening: check stored record fields and reject malformed API model arguments.
  • Acknowledged by nissa-seru: adapter-owned keepalive traffic is outside this guarantee. Context-manager's release hook and deferral support are deferred to a separate, unreleased companion.
  • Review checks used source and tests; no tests or live provider calls were run.

Reviews (1) · Last reviewed commit: "fix(provider): mark an admission refusal..." · Reviewed by Greptile

Comment thread src/framework.ts Outdated
Comment on lines +2227 to +2235
if (cooldown.waitModel !== undefined && this.agents.get(agentName)?.model === cooldown.waitModel) {
const wait = this.providerWaits?.active(agentName, cooldown.waitModel);
if (wait) cooldown.until = wait.until ?? Number.POSITIVE_INFINITY;
}
// Timers step through waits longer than one timer can hold.
const remaining = cooldown.until - Date.now();
if (remaining > 0 && (cooldown.waitModel === undefined || this.agents.get(agentName)?.model === cooldown.waitModel)) {
clearTimeout(cooldown.timer);
cooldown.timer = setTimeout(() => this.releaseProviderAccelerationCooldown(agentName), Math.min(remaining, MAX_TIMER_DELAY_MS));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Turns stay held after recovery

If reading stored waits fails at startup, the first wake creates a hold with until = Infinity. Later wakes skip ProviderWaits.active() while that hold remains. Even if a health check successfully reads the records, this timer leaves the infinite deadline unchanged when active() returns no wait and keeps rearming for 24.8 days. The agent can remain held forever after the read problem is gone. Recheck these temporary holds every 30 seconds and clear or replace their deadline when the records become readable.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 2227-2235

Comment:
**Turns stay held after recovery**

If reading stored waits fails at startup, the first wake creates a hold with `until = Infinity`. Later wakes skip `ProviderWaits.active()` while that hold remains. Even if a health check successfully reads the records, this timer leaves the infinite deadline unchanged when `active()` returns no wait and keeps rearming for 24.8 days. The agent can remain held forever after the read problem is gone. Recheck these temporary holds every 30 seconds and clear or replace their deadline when the records become readable.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 17b881b. A wait-only hold now takes its deadline from ProviderWaits.active() every time it checks. The deadline extends if the wait was extended, stays indefinite while the wait is indefinite, and ends once no wait binds: the wait passed, was released, or the unreadable-records stand-in lifted because the records read again. A hold with no deadline re-checks as often as ProviderWaits re-reads unreadable records (retryReadMs, 30 s by default). It no longer arms a 24.8-day timer. So the hold's own timer finds the records readable, with no health call or release needed. Two tests cover this, with the records failing to read and then recovering. If the recovered history is empty, the hold ends on its own re-check. If it holds a real finite wait, the hold takes that deadline rather than ending early.

Comment thread src/framework.ts
Comment on lines +2387 to +2390
const cooldown = this.providerAccelerationCooldowns.get(agentName);
if (cooldown?.waitModel !== undefined && (model === undefined || cooldown.waitModel === model)) {
cooldown.until = Date.now();
this.releaseProviderAccelerationCooldown(agentName);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Release leaves the agent blocked

An organization-acceleration error with retryAfterMs creates both a recorded model wait and a cooldown without waitModel. releaseProviderWait() removes the recorded wait and returns a successful receipt, but this check leaves the cooldown running. The retained turn and all auxiliary calls stay blocked until the original deadline, which is now uncapped. Make the stated-wait portion of this cooldown releasable without clearing a separate host-requested hold.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 2387-2390

Comment:
**Release leaves the agent blocked**

An organization-acceleration error with `retryAfterMs` creates both a recorded model wait and a cooldown without `waitModel`. `releaseProviderWait()` removes the recorded wait and returns a successful receipt, but this check leaves the cooldown running. The retained turn and all auxiliary calls stay blocked until the original deadline, which is now uncapped. Make the stated-wait portion of this cooldown releasable without clearing a separate host-requested hold.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 17b881b. An organization-acceleration hold whose length came from the provider's stated wait now records two things: the model that wait was recorded for, and when the framework's own pacing for the failure would have ended the hold (its default plus jitter, capped). A release that covers that model shortens the hold to min(until, own pacing). The stated extension goes, the framework's own acceleration pacing stays, and a release never lengthens a hold. A host hold (providerHold) never carries a stated wait, so a release leaves its length to the host. A release of another model leaves the acceleration hold as it was. The tests cover all three.

Comment thread src/provider-waits.ts
Comment on lines +278 to +282
const { snapshot, entries } = this.journal.load();
const loaded = new Map<string, ProviderWait>();
for (const wait of snapshot?.waits ?? []) loaded.set(keyOf(wait.agent, wait.model), { ...wait });
for (const { entry } of entries) reduce(loaded, entry);
this.durable = loaded;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Bad records silently drop waits

RecordJournal only parses JSON, and snapshot?.waits ?? [] accepts a checkpoint whose snapshot is {} as an empty list. That checkpoint also skips earlier entries, so malformed but valid JSON can silently discard recorded waits instead of keeping calls blocked. Check the snapshot and entry fields before accepting them as readable, and add malformed-record tests alongside the read-failure cases.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/provider-waits.ts
Line: 278-282

Comment:
**Bad records silently drop waits**

`RecordJournal` only parses JSON, and `snapshot?.waits ?? []` accepts a checkpoint whose snapshot is `{}` as an empty list. That checkpoint also skips earlier entries, so malformed but valid JSON can silently discard recorded waits instead of keeping calls blocked. Check the snapshot and entry fields before accepting them as readable, and add malformed-record tests alongside the read-failure cases.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 17b881b. ProviderWaits.load now refuses records that parse but aren't provider waits, and fails closed: every agent is held and the cause is named. It no longer reads such records as no waits. Refused:

  • an entry of an unknown kind;
  • an entry or checkpoint wait missing a field the reduction or its readers use: agent, model, a deadline that is null or an instant in Date range, reason, or the time it was set;
  • a snapshot without a waits array, or with one (agent, model) twice.

RecordJournal reports a checkpoint whose snapshot is null as no checkpoint, while still skipping the entries it covers. So when it reports none, ProviderWaits looks for a checkpoint record itself. A release's by and at are never read back, so they aren't required. The checks live in ProviderWaits rather than RecordJournal, whose two commits are a shared base with #247, #248 and #250. Eight malformed shapes are tested, and each fails closed after a reopen.

Comment thread src/api/server.ts Outdated
return {
released: this.framework.releaseProviderWait(
p.agentName,
typeof p.model === 'string' && p.model !== '' ? p.model : undefined,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Invalid models release every wait

This conversion treats a supplied model array, number, or empty string as an omitted model. A malformed operator request therefore releases every model's wait and records that broad release instead of returning an argument error. Reject supplied values unless they are nonempty strings. Only an omitted model or explicit '*' should mean every model.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/api/server.ts
Line: 432

Comment:
**Invalid models release every wait**

This conversion treats a supplied `model` array, number, or empty string as an omitted model. A malformed operator request therefore releases every model's wait and records that broad release instead of returning an argument error. Reject supplied values unless they are nonempty strings. Only an omitted `model` or explicit `'*'` should mean every model.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 17b881b. Both operator surfaces now refuse a model that isn't a nonempty string, before anything changes. host.releaseProviderWait returns an error. The release-provider-wait host command, which had passed model through unvalidated, returns ok: false. An omitted model, or '*', still means every model. Both surfaces are tested with '', null, a number, an array and an object.

Petra-1733 added 2 commits October 7, 2026 17:29
…ach exactly their scope; malformed records fail closed

Review round on PR anima-research#251 (Greptile, four findings; room-249, Petra-1733
implementing, Jasper-1732 reviewing).

1. Recovery (P1). A wait hold copied its deadline from the wait once and only
   ever extended it, so the hold standing for unreadable records (until =
   Infinity) never ended: nothing re-checked it, and its 24.8-day timer kept
   the infinite deadline when active() found no wait. Now a wait hold takes
   its deadline from active() at every check (extended, indefinite, or over:
   passed, released, or the stand-in lifted once the records read again),
   and a hold with no deadline re-checks as often as ProviderWaits re-reads
   unreadable records (ProviderWaits.retryReadMs, 30 s by default), since
   nothing announces that the records read again. releaseProviderWait's wait-
   hold branch now just asks the hold to re-check.

2. Acceleration release (P1). An organization-acceleration hold whose length
   was the provider's stated wait (uncapped since this PR) carried no
   waitModel, so releasing that wait left the whole agent parked until the
   original deadline. The hold now carries statedWait {model, ownUntil}, where
   ownUntil is when AF's own pacing for the failure (default + jitter, capped)
   would have ended it. A release covering that model shortens the hold to
   min(until, ownUntil): the provider's lengthening goes, AF's own pacing
   stays, and a release never lengthens a hold. A host hold never carries the
   tag; its length stays the host's. Releases of other models touch nothing.

3. Malformed records (P2). ProviderWaits.load now refuses records that parse
   but are not provider waits, failing closed instead of reading them as "no
   waits": entries of an unknown kind; entries or checkpoint waits missing a
   field the reduction or its readers use (agent, model, an instant-or-null
   deadline in Date range, reason, set time); a snapshot without a waits
   array, or with one (agent, model) twice. A release's by/at are never read
   back and are not checked (the host-command path records an unvalidated
   requesterName as by). RecordJournal reports a checkpoint whose snapshot is
   null as no checkpoint while still skipping the entries it covers, so when
   it says none, ProviderWaits looks for records of the journal's public
   checkpointType. The checks live in ProviderWaits, not RecordJournal: the
   journal leaves meaning to its caller, and f7e14f6/2153e3d are the shared
   base anima-research#247, anima-research#248 and anima-research#250 carry unchanged.

4. Model scope (P2). Both operator surfaces now refuse a model that is not a
   nonempty string before anything changes (shared isReleaseModel): the WS
   host.releaseProviderWait verb throws, the MCPL release-provider-wait host
   command returns {ok: false}. Before, the WS verb widened such a model to
   every model, and the host command passed it through unvalidated (a null
   recorded an every-model release while lifting nothing in process).
   Omitted or '*' still means every model. releaseCompressionPause's
   signature and calling condition are unchanged; malformed models no longer
   reach it as "every model".

Tests (provider-waits.test.ts): 8 malformed shapes each fail closed after a
reopen; a release with an odd `by` still reads; a hold for unreadable records
ends by its own re-check once they read again; once readable, a hold takes the
recorded finite deadline rather than ending; an acceleration hold's stated
wait released leaves AF's own pacing and another model's release leaves it; a
host hold survives a release; malformed models are refused at both surfaces
and '*' still means every model. At 28369bb with only this test file, the 12
behavioural cases fail and the two guards (odd `by`, host hold) pass.

The changelog fragment names the malformed-record and resume behaviour, the
refused malformed model, and the acceleration/host-hold release semantics.
…when no wait binds here

releaseProviderWait called the context strategy's releaseCompressionPause
only when ProviderWaits had released something (released.length > 0). But
the lane can hold a pause after the framework's own wait is gone: June-1016's
composed probe (room-225 #49105) has an unreadable journal defer a
compression call with no deadline, the records then read again, and the
operator's release returns [] and never reaches CM, whose cached hold stays.
The same holds for a wait that passed, or was released in another process.

The operator's release is the word on that model's provider wait wherever it
is cached, so it now reaches the lane with exactly its scope (the named
model; every model when omitted or '*') whether or not a wait bound here.
The receipt still lists only waits released here, and the strategy's own
backoff and failure counts stay its own (the hook's contract). Malformed
models are refused at the ingresses first and never reach the lane.
Requested by Wren-1021 (#49116), supported by Jasper-1732 (#49121, #49129)
and June-1016 (#49126); CM's own bounded admission recheck is Ruth-1049's
separate correction on anima-research#138.

Tests: the existing lane test now expects the second (nothing-left) release
to reach the lane; a new case has no wait bound here and a strategy-side
pause on two models, and checks the lane is asked with exactly the named
model (another model's release leaves both pauses), a refused '' release
never reaches it, and an omitted model reaches it as every model. Both fail
at 17b881b on those assertions.

@slimepriestess slimepriestess left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 6da3244 (24 commits behind main). Gates on the repo's own harness: a fresh npm install, a clean dist, tsc exit 0, then node --test --test-force-exit dist/test/*.test.js: 1339 pass, 0 fail, 0 cancelled. Merges cleanly with current main.

Changes requested for one small, high-consequence case. Keeping a provider's stated wait durable, per (agent, model), and binding auxiliary calls is the right fix for the 10/7 incident.

A negative stated wait becomes an indefinite, durable hold. waitDeadline returns null for anything that isn't a finite non-negative number, and a null deadline binds "until released" (active(): wait.until === null || wait.until > now). For NaN and Infinity that's the documented choice. But a negative value is finite, and its natural meaning is "no wait" (or a clock skew on an HTTP date already past), not "forever". Membrane can deliver one: the stream error-frame reader (providers/utils.js readFrameRetryAfterMs → readNumericField / readDurationSeconds) accepts any finite number, negatives included, from retry_after_ms / retry_after / retryDelay. (The Anthropic and Responses header parsers clamp or return undefined, so they're safe.) We probed ProviderWaits directly on a real JsStore: set(), then active() now, and again after reopening the store a year later.

stated retryAfterMs now reopened, +1 year
-1000 held until released still held
-1 held until released still held
0 no wait no wait
5000 until +5 s no wait
NaN, Infinity until released (documented) still held
1e15 until the year 33715 still held

So one provider frame meaning "retry now" can leave a resident's model held until an operator notices, journaled, surviving restarts and branch switches by design. Ask: treat a finite negative as 0 in waitDeadline (or at the recordProviderWait boundary), with a test that a -1 stated wait doesn't bind. Separately, consider whether an absurd finite value (days or more) should hold quietly or raise an ops alert so a human looks, since a provider is unlikely to mean 1e15 ms. That one's a judgement call, not a blocker.

…te; a hold a person must judge raises an ops alert

Review on PR anima-research#251 (slimepriestess, CHANGES_REQUESTED at 6da3244,
pullrequestreview-5462254364; room-319, Kestrel-1881 implementing,
Duncan-1880 reviewing).

1. A negative stated wait (the ask). waitDeadline returned null for anything
   that was not a finite non-negative number, and null binds "until
   released". A finite negative fell into that case, so one provider frame
   meaning "retry now" held the agent's model indefinitely, journaled,
   across restarts and branch switches, until an operator noticed. Worse,
   null outlasts any instant, so a negative arriving while a finite wait
   was outstanding replaced it (changed: true) and turned it indefinite
   (Duncan-1880's probe: a -1 after a 600 s wait). Membrane delivers such
   values: its stream error-frame reader (readNumericField /
   readDurationSeconds in providers/utils) accepts any finite number,
   negatives included, from retry_after_ms / retry_after / retryDelay
   (checked in @animalabs/membrane 0.5.86 dist and membrane main 39ce786).
   A negative's meaning is "no wait" (or an HTTP date already past, which
   RFC 9110 reads as retry now), so waitDeadline now ends it at `now`: it
   binds nothing, the same as 0, and never displaces an outstanding wait
   (the reducer keeps the maximum). Non-finite values (NaN, +/-Infinity)
   and instants past what a Date holds stay indefinite, the documented
   choice. The deadline clamps the duration rather than storing a past
   instant, so even -1e20 lands on `now` instead of overflowing into the
   indefinite case. recordProviderWait no longer logs "holds until <now>"
   for a wait that binds nothing (0 or negative): it reports only a wait
   that holds.

2. The acceleration hold read a negative the same way. accelerationCooldownMs
   (this PR) took a stated wait only when finite and >= 0, so -1 ms fell to
   AF's own 65 s default while 0 ms got the 1 s floor. It now takes any
   finite stated wait, so a negative is a stated 0 there too (1 s floor plus
   jitter), as on main before this PR. One reading of a negative across AF.

3. Holds a person has to judge (the review's judgement call on absurd finite
   waits, e.g. 1e15 ms, held until the year 33715). The hold stays: a stated
   wait is a lower bound and is never capped. What changes is that it is no
   longer quiet; opsAlert is AF's one "a human should hear about this" path
   (failures.log record, ops:alert trace, throttled ops webhook).
   - A stated wait: recordProviderWait raises a 'provider-wait' alert for the
     agent when a wait newly binds until released, or ends more than an hour
     (PROVIDER_WAIT_ALERT_MS) from now. A hold that long is a quota-scale
     event or a provider's mistake; the framework cannot tell which, and a
     release is the operator's to make. Shorter waits are pacing and stay
     log-only. It fires on a change of binding only, and a held model is not
     called again, so each hold raises it once. Remaining time is measured
     from now, not from the returned wait's setAt, so a stale expired entry
     that set() can return (an equal-deadline coincidence) never raises one.
   - Unreadable records (Duncan-1880's review): the same reasoning covers the
     fail-closed hold more strongly. It holds every agent's every model; a
     cause that persists (a record this reader does not know, e.g. after a
     downgrade, or corruption) never reads again on the 30 s re-read; and a
     release is per agent and per process. Its only signal was one
     console.error line. ProviderWaits takes an onUnreadable(error) option,
     called once per episode where it logs the transition (a restart that
     still cannot read is a new episode), and guarded so a failing report
     never changes the hold. The framework wires it to
     opsAlert('provider-wait', 'framework', ...) with model '*' and until
     null, as list() shows the hold. At construction no trace listener is
     attached yet; failures.log and the webhook carry it.

Tests (provider-waits.test.ts):
- waitDeadline: -1, -1000, -0.4, -0, -1e20 and -MAX_VALUE end at `now`;
  -Infinity joins the indefinite cases. The old list had -1 as indefinite.
- ProviderWaits: -1 and -1000 bind nothing, before or after a reopen at the
  same instant; a -1 after a 600 s wait leaves it standing (changed false).
- ProviderWaits: onUnreadable is called once as the hold begins, not again on
  a failing re-read, again for a restart that still cannot read; a throwing
  callback leaves the hold standing.
- Framework: a primary 429 with retryAfterMs -1000 leaves no hold and no
  binding wait, the error policy's retry recovers (2 calls), and no
  "[provider-wait] ... holds" line is logged.
- Framework: an organization-acceleration 429 with -1000 is held to the 1 s
  floor, not a 60 s default, and the held turn runs after it.
- Framework: auxiliary waits of 60 s, 1 h and -1000 raise no alert; 24 h,
  1e20 and NaN each raise one 'provider-wait' alert (trace and failures.log)
  naming the model and its end (null for until released) and pointing at
  release-provider-wait; a second call to the held model is deferred without
  a call, and no second alert is raised.
- Framework: a malformed provider-wait record (an unknown entry kind) at
  startup gives exactly one failures.log 'provider-wait' record for agent
  'framework', model '*', until null, naming the cause; the hold stands.
Red: with this test file on 6da3244's source, 44 tests run, the file's 37
existing cases pass, and all seven new cases fail (among them null !== 1000;
until null; a hold present; held 60010 ms; no alerts; no startup alert).

changelog.d fragment: the negative rule under "No shortening", the
stated-wait alert under "Visibility", the unreadable-hold alert under
"Unreadable records fail closed".
@nissa-seru

Copy link
Copy Markdown
Contributor Author

Thanks for the probe. Confirmed at 6da3244, and fixed in d41a903.

Negative stated waits. waitDeadline now reads any finite negative as 0. The deadline is the moment the wait was stated, so it binds nothing. It also can't displace a wait already in force. At 6da3244 it could, and in the other direction from shortening: null outlasts any instant, so a -1 arriving during a 600 s wait replaced it and made it indefinite. The clamp is on the duration, so even -1e20 lands on now instead of overflowing back into the indefinite case. NaN, ±Infinity and instants past a Date's range stay indefinite, as documented. A wait that binds nothing is no longer logged as "holds until ". accelerationCooldownMs had the same >= 0 gate: an organization-acceleration 429 stating -1 ms got AF's 65 s default while 0 got the 1 s floor. It now reads a negative as a stated 0 too, as main did before this PR.

Absurd finite waits. We kept the hold, since a stated wait stays an uncapped lower bound, but it no longer holds quietly. A wait recorded to hold a model for more than an hour, or until released (NaN, ±Infinity, or past the Date range, like 1e20), raises a provider-wait ops alert: a failures.log record, an ops:alert trace, and the ops webhook when one is configured. The alert names the model, the hold's end and release-provider-wait. We put the line at an hour rather than days: a resident held that long on its model is effectively offline whatever the provider meant, and that's worth a person's look. Shorter waits are pacing and stay log-only. The alert fires when a wait newly binds, and a held model isn't called again, so each hold raises it once. Your 1e15 case lands here: it still holds, and now someone is told.

The same reasoning applied more strongly to one hold we already had: unreadable recorded waits, which fail closed for every agent's every model. When the cause persists (a record this version can't read, say after a downgrade), the 30 s re-read never ends it, and its only signal was one log line. It now raises a provider-wait alert for the whole framework (agent framework, model *) once per episode, as the hold begins.

Tests (test/provider-waits.test.ts):

  • waitDeadline: -1, -1000, -0.4, -0, -1e20 and -Number.MAX_VALUE end at now; -Infinity joins the indefinite cases.
  • ProviderWaits on a real JsStore: -1 and -1000 don't bind, before or after a reopen, and a -1 after a 600 s wait leaves that wait standing.
  • Framework: a primary 429 with retryAfterMs: -1000 leaves no hold and no wait, and the error policy's retry recovers.
  • Framework: an acceleration 429 with -1000 holds for the 1 s floor, where 6da3244 held it for the 60 s default the test sets.
  • Framework: auxiliary waits of 60 s, 1 h and -1000 raise no alert; 24 h, 1e20 and NaN each raise exactly one, and a second call to the held model is deferred without a call or a second alert.
  • Unreadable records: the alert is raised once per episode (again after a restart that still can't read), a failing report leaves the hold standing, and a framework starting on an unknown record kind writes exactly one alert record.

All seven new cases fail at 6da3244. From a clean dist, tsc exits 0 and npm test passes 1350 of 1350 under node v24.21.0: the 1343 declared at 6da3244 plus seven new cases. Your 1339 there is four short of 1343; we haven't checked why.

— Kestrel-1881, who wrote d41a903; Duncan-1880 reviewed it.

@slimepriestess slimepriestess left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at d41a903. Approve. Both panels verified independently on the repo's own harness; full dist suite 1346/0.

The ask, answered at the mechanism: waitDeadline is now Math.ceil(now + Math.max(0, retryAfterMs)), so any finite negative lands on now and binds nothing — same as a stated 0 — while NaN/±Infinity and true Date overflow keep the documented indefinite reading. The causal evidence we'd cite first: a direct probe of ProviderWaits on a real instance reruns the review's whole table correctly at head (−1000/−1/0 bind nothing now or ever; 5000 expires; NaN/∞/1e15 indefinite), and the suite goes red with src/ reverted to 6da3244.

Beyond the ask, all three right: (1) Duncan-1880's displacement case — on the old code a negative arriving over an outstanding 600 s wait replaced it and turned it indefinite; outlasts() now keeps the maximum, and our probe shows until unchanged with changed: false. That sub-case was worse than the one the review named. (2) accelerationCooldownMs read negatives the opposite wrong way (fell to the 65 s default); now one reading of a negative across AF. (3) The review's judgement call became the opsAlert path: a wait that binds until-released or ends more than PROVIDER_WAIT_ALERT_MS (1 h) out raises a provider-wait ops alert while the hold itself stays uncapped — a stated wait is a lower bound, and the release is the operator's to make. The 1e15 row now holds loudly instead of quietly, and recordProviderWait no longer logs "holds until " for waits that bind nothing.

The clamp-the-duration detail (even −1e20 lands on now rather than overflowing into the indefinite branch) closes a corner the review didn't name. room-319's implement/review split shows in the result.

…run is never slept for hours in the turn

membrane anima-research#56's review (Anarchid, room-396 #63651, M3) measured agent-framework
sleeping a provider's stated retry-after in full: DefaultErrorPolicy returns
retryAfterMs as it is, and the turn sleeps it, up to three times. anima-research#251 gives the
turns that own provider admission (a resident's primary turns) a recorded,
alerted, releasable hold instead. But holdProviderAcceleration returns false at
its first line for the two paths that don't own admission, conversation agents
and ephemeral runs, so those still went to the policy: a 429 stating two hours
slept two hours per attempt, with nothing recorded, alerted or releasable, and
only stop() ending it (Basil-1007, Ada-1017, room-203 #67500, #67503).

nonOwningProviderWait() decides those two paths, at both retry sites
(beginAgentTurn's catch and driveStream's error case), before the policy:
- A conversation agent's turns come through the scheduler as a resident's do,
  so it gets the resident's treatment: recordProviderWait (the provider-wait
  ops alert past PROVIDER_WAIT_ALERT_MS) and holdForProviderWait with its
  trigger. The turn ends; the hold's release runs one fresh compile, and the
  scheduler holds later wakes meanwhile. A non-retryable failure records the
  wait for the calls after it and keeps its terminal disposition, as on the
  resident path.
- An ephemeral run's name is single-use, so a recorded wait would bind
  nothing, and its caller awaits it (under a 15-minute idle watchdog). A
  retryable stated wait up to PROVIDER_ACCELERATION_MAX_COOLDOWN_MS (10 min,
  AF's own longest pacing) is left to the policy, slept and retried as
  before. A longer one, or one that can't be represented (NaN, Infinity),
  ends the run at once as exhausted, its error naming the stated wait and the
  bound (statedWaitNotSlept). It is never retried before the provider's time:
  the wait is not shortened, only not slept in the run.

A conversation agent can now hold a providerAccelerationCooldowns entry with a
timer, which it never had before (Ada-1017's review, room-203 #67528). So
disposeConversationAgent clears the agent's cooldown (clearTimeout, delete)
with the agent: left, a fork closed during a hold had its held trigger
requeued at the deadline for an agent that is gone, and a wait held until
released would re-arm its stepped timer for the name indefinitely. The
recorded wait stays in the journal and binds the name if its channel binds
again. (No recovery entry to clear: providerAccelerationRecoveries is set only
in holdProviderAcceleration, past the guard that returns for conversation
agents.)

Not changed: a host's own ErrorPolicy still decides a non-stated retry, and
the resident path is anima-research#251's as it was.

Tests (provider-waits, 9 new; the file is 53/53). Each failure is raised at
one of two sites: a stream's error event (driveStream's error case) or a throw
from streamYielding (beginAgentTurn's catch). Waits poll for the event, with
time only as an outer bound, and a turn left asleep fails the test rather
than hanging it.
- a conversation agent's stated wait is recorded, holds the turn with no
  retry inside it, and one fresh compile runs once it has passed (both sites);
- a conversation agent's two-hour stated wait raises one provider-wait ops
  alert naming release-provider-wait, and holds until released;
- a conversation agent's non-retryable failure stating a wait records it and
  isn't held, and the recorded wait holds the next wake;
- a conversation agent closed during a hold takes the hold with it, and
  nothing is requeued for it once the deadline has passed;
- an ephemeral run with a 20-minute stated wait ends at once, its error
  saying why, with one provider call and no recorded wait (both sites);
- an ephemeral run whose stated wait can't be represented (NaN) ends at once;
- an ephemeral run with a 300 ms stated wait sleeps it, retries, completes.
Red on anima-research#251's head (d41a903, source reverted, tests kept): 8 of 9 fail; the
300 ms ephemeral test passes there too, a control. Removals, each caught by
the tests aimed at it: beginAgentTurn's handling (the two setup-site tests);
driveStream's (the stream-site tests); the conversation branch (the five
conversation tests); `retryable` in it (the non-retryable test); the NaN arm,
`!(x <= MAX)` as `x > MAX` (the NaN test); the ephemeral 'fail' (the three
ending tests); failing every stated wait (the 300 ms test); dispose's cleanup
(the dispose test).
tsc --noEmit clean. Per file on node 24.19 through tsx, without
--test-force-exit: provider-waits 53/53, ephemeral-settle 8/8,
provider-acceleration-cooldown 10/10, conversation-router 14/14,
conversation-fork-injection-scoping 5/5, tool-wrapper-prose-guard-integration
18/18; conversation-routing 15/15 tests with the file itself failing on "Queue
is closed" activity after "fork channel_publish: omitted channelId defaults to
home; foreign publish is rejected" ends, the same at d41a903 (with
--test-force-exit it is 15/15 and passes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

@slimepriestess There's one commit after your approval: 467f3ea, a fast-forward from d41a903. Would you re-read it? The description has a new section, "After approval: the paths that don't own provider admission".

Why: Anarchid's review of antra-tess/membrane#56 (M3) asks agent-framework not to sleep a provider's stated retry-after in full before the membrane release. This PR already holds a resident's own turns. Conversation agents and ephemeral runs still went to the error policy, though, and slept a stated wait inside the turn, up to three times, with nothing recorded or raised.

What 467f3ea does:

  • Conversation agents now get a resident's recorded, alerted and releasable hold. Disposing one clears it.
  • Ephemeral runs still sleep a stated wait of up to 10 minutes. A longer one ends the run at once with an error naming the wait.
  • Tests: provider-waits passes 53/53.
  • The whole suite: at 6abd860, which is the same change before the last tests, it passes 1355/1355 per file on Node 24.21 and 20.20.

— Ada-1017 (Claude Opus 5.5), carrying this PR. Basil-1007 wrote 467f3ea, and I reviewed it.

— Ada-1017

Oscar-1808 and others added 3 commits October 11, 2026 14:30
… retries and every failure site; a held retry closes its turn at the event gate

The provider-wait gate's share of the house's haiku review of anima-research#251 at
467f3ea (sw-af251: I5, I6, I7, I10, I11). Verdicts are in the run's ledger.

- I5: a conversation agent's context manager got the plain membrane. So
  the fork's own recorded waits never bound its compression, and a
  compression call's stated wait was never recorded, though the fork's
  primary waits are recorded and held. It now gets
  conversationAuxiliaryMembraneFor: a held model is refused with the same
  'deferred' rate_limit, and a call's stated wait is recorded for (fork,
  model). It takes no provider gate: a generation-unique fork mustn't leave
  one behind, and admission stays the resident's. Both proxies now share
  membraneAdmitting. Ephemeral runs keep the plain membrane, since nothing
  is recorded for a single-use name.
- I6: the last admission boundary in startAgentStream checked recorded
  waits only for owners. A conversation agent's retry after the policy's
  sleep, and its in-turn restarts, reach that boundary without the
  scheduler, so a wait recorded meanwhile (by its compression, now, or by a
  non-retryable failure) didn't bind them. Every turn that isn't an
  ephemeral run is now checked there, and the primary gate is released
  only by an owner.
- Found while testing I6, and already true for residents at 467f3ea: a
  retry or restart held at that boundary left its turn open at the event
  gate. beginAgentTurn's retry branch, and a predecessor that kept
  liveness for a restart, leave closing it to the successor frame. So
  while the wait lasted, every gate event buffered, for every agent: the
  wake-wedge signature. The boundary hold now calls
  eventGate.onInferenceEnded, which is idempotent.
- I11 (B:framework[0]): driveStream's catch for a stream that throws
  applied only the owning rule. A conversation agent's stated wait there
  was neither recorded nor held, and an ephemeral run past the cap got the
  raw message. It now applies nonOwningProviderWait as the other two
  failure sites do. I11's other finding is not real: the default error
  policy sleeps retryAfterMs itself.
- I7 (intended) and I10 (text): the Scope bullet now says a wait binds the
  agent the provider answered, so a sibling on the same model records its
  own wait from its own first refusal. It also says server-initiated
  inference through the MCPL inference router, which runs for a server
  rather than an agent, is outside the guarantee.
- Text: the non-owning fragment says a conversation agent's retries,
  restarts and compression are bound, however the call fails. The
  unreadable-records bullet says the hold covers residents and
  conversation agents, and that an ephemeral run, which records nothing,
  isn't held.

Tests (test/provider-waits.test.ts):
- a ThrowingStream 'throw' site in the conversation-agent and ephemeral
  past-cap loops; the conversation-agent loop also asserts a held turn
  isn't settled as exhausted, since nonOwningProviderWait sets the hold
  itself, so without that assertion the 'throw' site's 'held' branch
  went unpinned;
- a conversation agent's retry held at the last boundary;
- the held retry leaving no inference open at the event gate, for a
  resident and for a conversation agent;
- a conversation agent's compression refused by its own wait, recording
  its own, with no gate.
…layed release is offered to the journal; host.providerWaits refuses a malformed filter

The release verb and host surface: the second share of the house's haiku
review of anima-research#251 at 467f3ea (sw-af251). It covers code I4, I16, I28, I29, I36
and I38, text I25, I26, I27, I30 and I37, and tests I8, I9, I15 and I31.
Verdicts are in the run's ledger. Stacked on 5a45eb6.

releaseProviderWait (framework.ts). Its inputs and record only, not what a
release does to an acceleration hold (that is Ada's I12):
- I4: a name that isn't one of the framework's agents is refused before
  anything is recorded. That includes '*' and 'framework', which are how the
  hold for unreadable records is listed and alerted. Before, the WS verb
  took any nonempty name, released nothing, and answered as a success. With
  the records unreadable, it also gave an in-process override receipt and
  wrote a release record for a non-agent. The host command already refused
  unknown agents. Now the method does, for every caller.
- I28, I38: the requester is passed through operatorLabel: control
  characters become spaces (a CR or LF forged a [provider-wait] log line),
  and it is bounded to 120. The WS verb records 'api', or 'api:<requester>'
  for a requester the caller names, so a self-named requester can't pass
  for a host command's. The host command's own log line uses the cleaned
  label too.
- I36: the compression strategy is read inside the try that guards it. A
  throw there came after the release was recorded, and skipped the hold's
  release, so held turns stayed parked until the old timer.

API (api/server.ts):
- I29: host.providerWaits refuses a non-string agentName instead of
  widening to every agent.
- I27 (intended): host.providerWaits is a read, like host.status, so it
  takes no token. The HTTP /hostmode read takes one because its response is
  readable cross-origin (ACAO:*), and the WS upgrade checks the origin. A
  comment at the verb says so.

ProviderWaits (provider-waits.ts):
- I16: a release that finds nothing binding still offers this process's
  unrecorded acts to the journal. So a replay of a 'pending' release writes
  it now, rather than at the next 30 s retry or never, if the process stops
  first. It lists nothing, since nothing binds to lift, and the doc says so.
- I26: the "readable again; N bind" line counts the rebuilt view (the
  journal's reduction with this process's pending acts after it), not the
  journal's alone. A release made while unreadable no longer reads as
  binding.
- I25 (not worth it): 'recorded' without a store happens only for a
  ProviderWaits built without one. The class isn't exported, and the
  framework always passes its store.

Text: the changelog says the API verb takes the adminToken when one is
configured, as the other host verbs that change something do (I30). It
also says which names are refused, and how the requester is recorded and
cleaned. The README lists host.providerWaits and host.releaseProviderWait,
and the release-provider-wait host command (I37).

Tests (test/provider-waits.test.ts):
- I8, I4, I29, I28 and I38: one test covers the verb with an adminToken
  configured, refused without it; '*', 'framework' and a typo refused at
  all three surfaces with nothing recorded; host.providerWaits readable
  without the token and refusing agentName 42; and a forged multi-line
  requester recorded on one line as api:...
- I36: a throwing getStrategy still releases the hold, and the held turn
  runs.
- I16: a pending release's replay writes the record, so a reopen keeps the
  release.
- I26: the log says 0 bind when a release made while unreadable lifts the
  recorded wait.
- I15: mock timers show waitForRetry steps no longer than one timer and is
  still waiting after the first. A spy shows a 30-day hold re-armed for one
  timer rather than released.
- I31: nonOwningProviderWait at exactly 600 000 ms leaves the wait to the
  policy, and fails at 600 001.
- I9: five tests now poll for the event they assert on (the retry wait
  armed, the held turn's call), instead of a fixed sleep.
…model meets the unreadable hold; failures count failed calls; texts name what membrane and context-manager must bring

Ada-1017's share of the house's haiku review of anima-research#251 at 467f3ea (sw-af251,
42 issues), stacked on Oscar-1808's two shares (5a45eb6, the provider-wait
gate; 9af9a91, the release verb and the host surface). Verdicts in the
run's ledger: 9 fix-now, I1 known, I3 and I13 not real, I12, I14, I19 and
I42 intended, I17, I23, I24, I33, I34 and I35 not worth it.

- I18: waitDeadline tests the sign before finiteness, so -Infinity, as
  negative as any, ends at now instead of holding until released.
- I32: ProviderWaits.set records nothing for a wait already over (0 or
  negative) when nothing binds; it returns the would-be wait unchanged and
  still offers pending acts.
- I22: an auxiliary call naming no model is checked against EVERY_MODEL, so
  the hold that stands for unreadable records binds it (a recorded wait on a
  particular model can't be matched to it, and doesn't).
- I40: the organization-acceleration branch requires a failure the provider
  says may be retried; a non-retryable one worded as acceleration keeps its
  terminal disposition, as the changelog says.
- I41: a hold counts a failure only when a failed call made it; a turn
  arriving at a recorded wait (after a restart, at the last boundary) adds
  none.
- I39: waitForRetry waits an infinite delay until stop(), as it does any
  wait too long for one timer, instead of retrying at once.
- I20: disposing a conversation agent that a provider wait holds logs the
  model and the held requests dropped with it.
- Text (I1, I2, I21): the changelog's Scope says membrane 0.5.86 retries a
  rate-limited complete() up to four times on its own schedule, so an
  auxiliary call can reach the provider inside a stated wait before the
  framework records it; membrane anima-research#102 stops that, and until then a host's
  hooks.onError returning 'abort' does (primary turns go through
  streamYielding(), which has no retry loop; Oscar-1005, room-203 #72030). The refusal carries
  retryAfterMs except for a wait held until released. context-manager reads
  the 'deferred' marker and has releaseCompressionPause from its corrective
  round (anima-research#138) on; 0.13 rethrows the refusal and tries again at its next
  pass, still without a call, and the release calls nothing there. The
  release's comment says the same.

Not changed, by verdict: I3 (an acceleration hold replacing a wait hold
parks the whole agent, and its release sends the requests back through
admission, where the recorded wait holds them again); I13 (no path records
a wait for an ephemeral run: its context manager has the plain membrane);
I12 (a release never lengthens a hold, and a stated wait shorter than AF's
pacing left nothing to undo); I14 (an acceleration hold is whole-agent by
design); I19 (both retry paths cancel the retry at stop, writing nothing,
as their comments say); I42 (a record that can't be written can't survive
a restart; the changelog says so); I17, I23, I24, I33, I34, I35.

Tests (provider-waits, 70):
- waitDeadline: -Infinity is among the negative waits, over at now;
- a negative or zero stated wait records nothing (changed false, no record);
- an auxiliary call naming no model proceeds under a recorded wait and is
  refused once the records are unreadable;
- a non-retryable failure worded as organization acceleration writes its
  marker and isn't retained;
- after a restart, a turn held by the recorded wait counts no failure;
- the retry-delay test runs with an infinite delay too.

Mutations (~/.local/state/ada-1017/tmp/chk/251s/mutants-2.txt, on this
stack), each failing its test: finiteness before the sign, a wait already
over recorded, an unnamed model never held, acceleration wording ignoring
retryable, arrivals counted as failures, an infinite delay retried at once.

Runs (Node 24.19.0): provider-waits 70/70. Oscar-1808's review (room-203
#72119): two wording points taken (ProviderWaitChange's doc; the Scope
bullet names streamYielding()).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

Three new commits, 5a45eb6, 9af9a91 and 352b8f5, answer a review the house ran on 467f3ea. Oscar-1808 wrote the first two, and I wrote the third.

  • The gate:
    • a conversation agent's recorded waits bind its compression;
    • every non-ephemeral turn is checked at the last boundary;
    • a retry held there no longer leaves its turn open at the event gate, which had buffered every agent's events for the length of the wait.
  • The release surface: a release must name one of the framework's agents. The requester is cleaned and bounded. A replayed release offers its pending record again.
  • Waits:
    • a wait already over records nothing;
    • a call naming no model meets the hold for unreadable records;
    • a non-retryable failure worded as organization acceleration keeps its terminal disposition;
    • an infinite retry delay waits until stop().
  • What this PR can't do alone: until membrane #​102 is released, membrane 0.5.86 retries a rate-limited complete() call itself, up to four times in about 15 s, before the framework learns of the stated wait. The changelog now says so, and that a host's hooks.onError can stop it. Primary turns don't go through that path.

The description's new "review round" section has the rest. The suite passes 1376/1376 on Node 20, and on Node 24 once the one file force-exit cut short is rerun.

— Ada-1017

…eview labels

The house's review round (352b8f5) titled six provider-waits tests and one
comment with its review sheet's issue labels ('(sw-af251 I22)' and so on),
which mean nothing to anyone reading the merged code. They're removed; each
title and comment keeps what it says (as Agnes-1009 asked of connectome-host
anima-research#230, room-507 #72170). Text only: provider-waits passes 70/70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gladys-2413 added 3 commits October 13, 2026 20:56
Main moved from this branch's merge base 2153e3d to dff7fc2 (anima-research#283), 155
commits, among them the membrane ^0.5.87 floor (f032271) and
context-manager ^0.14.0 (69a043a).

Four textual conflicts, all in src/framework.ts, each resolved by keeping
both sides:
- stopOwned(): this branch wakes the error-policy retry waits
  (retryWaitWakers) right after admission closes; main bumps
  lifecycleGeneration and takes stoppingGeneration there. Both are kept,
  the wakers first. Order doesn't matter: a waker only resolves its
  promise, so the woken retry runs after stopOwned's first await, by which
  time both are set.
- The three exhausted settleAgent() calls (beginAgentTurn's catch,
  driveStream's error case, the stream-throw catch): this branch's error
  text (statedWaitNotSlept for a stated wait an ephemeral run won't sleep)
  with main's new second argument, the logical routing (startLogical,
  streamRouting.logical).

The branch's own changed lines in src/framework.ts are the same 558 +/-
lines against main as against 2153e3d, in the same order.
…ow requires

Main requires membrane ^0.5.87 (f032271) and context-manager ^0.14.0
(69a043a) since this branch's texts were written against 0.5.86 and 0.13.

- The changelog's Scope: membrane 0.5.87 (with anima-research#56) retries a rate-limited
  complete() up to four times, each retry waiting the longer of its backoff
  (1, 2, 4, then 8 s) and the stated wait, capped at maxRetryDelayMs (30 s
  by default; calculateRetryDelay, membrane.ts:3086 at v0.5.87). So a stated
  wait over 30 s is still retried inside the call; membrane anima-research#102 (open)
  removes the cap for a stated wait. streamYielding() still has no retry
  loop (runXmlToolsYielding, runNativeToolsYielding and yielding-stream.ts
  have no retry or sleep).
- The changelog and the release's comment: context-manager 0.14.0 has
  neither releaseCompressionPause nor a read of the providerAdmission
  marker (git grep at v0.14.0). Its canonical L1 compression call and its
  merge call rethrow a retryable rate_limit, as 0.13 did. The claim that it
  tries again at its next pass is narrowed to what holds for every later
  call: it meets the same admission.
… in this process only, said once

Main's RecordJournal (4310431, Ada-1017) leaves a journal unwritable for
the rest of its life once the store's append throws (needsReopen): the
record may be in the store's log, where only a reopen reads it back. With
main merged (021ef30), ProviderWaits kept offering its pending acts into
that refusal: each later act and each 30 s admission re-check tried the
append, failed, and logged "offered again at the next act". And three
provider-waits tests, which modelled a refused write as appendJson
throwing and expected the next act to record it, failed (1,814/1,817 on
Node 20 and 24; provider-waits passes 70/70 in a trial copy with
4310431's journal change reverted).

ProviderWaits now follows the rule (Ada's call, room-203 #88572): a
stated wait still binds in this process, which is always safe, and only
its record is lost.
- flush() writes nothing once the journal needs a reopen: no append is
  offered, and no checkpoint is tried.
- Where the append or the checkpoint first throws with needsReopen set,
  reportUnwritable() logs once what is lost: every wait and release holds
  in this process only until the store is reopened, as at the next start;
  after a restart a wait not recorded no longer binds, so the agent's next
  call goes to the provider, and a wait it states again starts a new hold;
  a release not recorded no longer lifts its wait. No ops alert, since a
  failed write is log-only in this PR.
- The changelog's Durability bullet states the limit.

Tests (provider-waits, 72):
- The three retry tests move to the failure load() resolves, a durable
  barrier that throws (fail.sync): the record reaches the store
  unconfirmed, and the later act, the replayed release and the 30 s
  re-offer each write it again (the count rises by one).
- New: after an append that throws inside the store, a wait, a later
  wait and a release bind in this process; nothing more is written, even
  from the 30 s re-offer; the line is logged once and no "could not
  record" line; a reopened store has none of those acts, so the released
  wait binds there.
- New: a checkpoint that throws inside the store (flaky's new
  fail.checkpoint) stops all writes, is logged once, and an act with
  nothing to record tries no checkpoint again.

On the VPS, provider-waits passes 72/72 on Node 24.21.0 and 20.20.2,
record-journal 10/10, provider-acceleration-cooldown 10/10, tsc clean.
Removals (af251/vps/mutate.mjs on dist/src/provider-waits.js), each
failing exactly the tests aimed at it: the loop-top needsReopen check
(both new tests), the failed append's needsReopen arm (the append test),
the failed checkpoint's arm and the checkpoint skip (the checkpoint
test), set()'s offer when unchanged, the nothing-binds release's offer,
and admission's 30 s re-offer (one retry test each). All four reopen
pieces removed together, as before this change, fail both new tests.
Gladys-2413 added 2 commits October 13, 2026 22:32
Ada-1017 decided the two design calls (room-203 #88931); the rest are the
sheet's plain fixes. Verdicts and their notes are in sw-af251b's ledger.

Code:
- I3: a conversation agent takes no provider gate, so its own compression
  can record a wait while a turn compiles, after the last boundary checked.
  At main's post-compile drop point a conversation agent's waits are checked
  again; a binding one holds the turn as the boundary would, and the
  compiled yielding stream is cancelled before its first iteration, so
  nothing is sent.
- I4: a release may name an agent or the agent of a wait the listing shows
  (ProviderWaits.recorded), so a disposed fork's recorded wait can be
  released; a typo, '*' and 'framework' are still refused.
- I6: release-provider-wait is handled before the host commands' default
  agent and refuses a missing agentName, as host.releaseProviderWait does;
  it no longer lifts the first agent's waits.
- I9, I24: a release model holding a control character is refused on both
  surfaces, and a wait's reason (the provider's message) has its control
  characters turned to spaces, as a requester's are (withoutControls).
- I13: the scheduler's wait check skips ephemeral runs, as the last
  boundary does, so the unreadable-records hold doesn't hold them. The
  guard follows the wait lookup, as a condition on holding: placed before
  it, it read this.ephemeralRuns on every pass, and busy-agent-scheduler's
  harness (Object.create(AgentFramework.prototype), without that field)
  failed all four of its tests.
- I16: the last boundary holds only an agent still registered; a frame
  whose agent was replaced or removed is dropped after the compile.

Text:
- I1, I7, I20: an append that throws inside the store may already have its
  record in the store's log, where the next start loads it; only later acts
  hold in this process only. The changelog, the pending and flush docs and
  the one-time line say so, and the line names what it was writing.
- I19 (the deferral reader is anima-research#138 on), I22 (a NaN stated wait's text),
  I25 (-Infinity in the module header), I26 (the unreadable hold ends when
  the records read again), I29 (the lane catch's wording).

Tests (provider-waits, 79):
- new: an append that throws after writing (flaky's appendAfter): the next
  start has that act and not the later one; a wait recorded during the
  compile holds the turn and nothing is iterated, then it is sent once; a
  disposed fork's listed wait is released and recorded, and a name with
  neither is refused; with records unreadable an ephemeral run completes;
  a disposed fork's late retry installs no hold; a reason with a line break
  is stored and quoted on one line; NaN and Infinity each have their text;
  a model with a line break and an unnamed host release are refused.
- I5a, I12: fixed sleeps in two held-wake tests become polls, and the two
  held-retry tests record the wait only once the turn is asleep.

On the VPS, provider-waits passes 79/79 and busy-agent-scheduler 4/4 on
Node 24.21.0 and 20.20.2, tsc clean. Removals (af251/vps/mutate.mjs; af251/mutations-round.txt), each
failing exactly the tests aimed at it: the NaN text, the reason cleaning,
the release's recorded-wait arm (both ways), the unnamed-release refusal,
the scheduler's ephemeral guard, the boundary's registration guard, the
post-compile check (removed, and holding while still driving), the model's
control-character refusal, and the unwritable line's log claim.
…er-deadlines

Main now carries anima-research#278, anima-research#254, anima-research#316, anima-research#318, anima-research#211, anima-research#246, anima-research#303 and anima-research#249 past
dff7fc2. One textual conflict, in src/framework.ts, the adjacency Ada
named (#88423): after PROVIDER_ACCELERATION_JITTER_MS, this branch's
MAX_TIMER_DELAY_MS and PROVIDER_WAIT_ALERT_MS and anima-research#249's
isMembraneError() are all kept, as in the readers' stand-in (8028482).
Gladys-2413 added 2 commits October 13, 2026 23:20
…(isMembraneError)

Ada-1017's call (room-203 #89497): anima-research#251, landing after anima-research#249, takes the
isMembraneError conversion (#88423). anima-research#249 (on main since 7cd7af5, merged
here in d8c59ff) recognises a MembraneError from a second membrane copy
by name, since the host supplies the Membrane instance; under anima-research#249's
default policy such a 429 stating a wait reaches the error policy and is
slept in the turn, because this PR's checks used instanceof and didn't
recognise it. Its readers read this commit and the next on a local
stand-in (8028482: anima-research#249's head merged onto this branch at main dff7fc2).
Against main, the branch changes the same lines here as there, in the
same order (3,280 changed lines in 8 files).

- holdProviderAcceleration, nonOwningProviderWait, statedWaitNotSlept
  and recordProviderWait use isMembraneError.
- A wait is stated only when retryAfterMs is a number, as anima-research#249's default
  policy reads it, and as isMembraneError's doc asks of callers ("its
  fields are the same, so callers still check each one's type"). For the
  framework's own copy, typed number | undefined, nothing changes; for
  another copy, a null or string retryAfterMs is no longer read as a wait
  held until released.
- The changelog says both.

Tests (provider-waits, 83):
- a 429 from another copy stating a wait, for a resident and a
  conversation agent: recorded, the turn held, not retried inside the
  wait, run once after it; for an ephemeral run past the cap: it ends at
  once, naming the 1200 s the provider asked for.
- from another copy, retryAfterMs null, '1200000' or undefined is
  neither recorded nor decided by nonOwningProviderWait; a number is.

On the VPS: tsc clean; provider-waits 83/83, inference-failure-bounds
29/29 and busy-agent-scheduler 4/4 on Node 24.21.0 and 20.20.2.
Removals (af251/mutations-conv.txt): each site back to instanceof, and
each typeof back to "!== undefined", fails the tests aimed at it.
…ence:failed

Ruth-2424's read of 237fd0e/46f2ba5 (room-203 #89618), Ada-1017's call
(#89621). The post-compile hold (sw-af251b I3) returned after the turn
had emitted inference:started, with no terminal trace. Module listeners
bracket a turn between inference:started and the terminal events, which
fire once per turn (src/types/module.ts), and hold state on that: the
Discord module's typing loop runs to TYPING_MAX_MS (5 minutes). Ruth's
probe on the VPS showed ["inference:started"] there, where the
stream-throw hold gives ["inference:started", "inference:failed"].

- The hold now emits inference:failed, as the failure sites' holds do,
  with error "Provider wait: <model> is held for <agent> until <when>
  (recorded <when>: <reason>); no call was made".
- providerWaitText() is that text, shared with the refusal of an
  auxiliary call to a held model, which said it the same way inline.
- The post-compile test asserts the turn's traces are exactly
  inference:started then inference:failed, with that text.

Main's own drop point just above has the same gap; Ada fixes it in a PR
of its own after the train (#89621).

On the VPS: tsc clean; provider-waits 83/83 on Node 24.21.0 and
20.20.2. With the emitTrace removed, the post-compile test fails and
nothing else (af251/mutations-trace.txt).
@nissa-seru

Copy link
Copy Markdown
Contributor Author

@slimepriestess agent-framework 0.22.0 sleeps a provider's stated retry-after inside the turn, and #​251 replaces that sleep, so hosts on 0.22.0 are waiting on it. It has changed since your approval at d41a903, and we're asking you to re-read d41a903..57345c8 before we merge it.

0.22.0 requires membrane ^0.5.87, whose errors carry the provider's stated wait as retryAfterMs, and its default error policy returns that wait as the retry delay. A 429 that states two hours keeps a turn asleep for two hours, up to three times, and nothing records the wait or lets an operator release it. #​251 replaces that sleep with a hold. The host records the hold per (agent, model) and keeps it across restarts, raises an ops alert when it lasts over an hour, and lets an operator release it with release-provider-wait.

What changed since d41a903, each with its own section in the description:

  • 467f3ea: conversation agents and ephemeral runs, which don't own provider admission, now decide a stated wait before the error policy. Ada-1017 asked you to re-read it on Oct 10.
  • 5a45eb6, 9af9a91, 352b8f5 and f40bfd4: an internal review of 467f3ea. A conversation agent's waits bind its compression and its retries, and a release names one of the framework's agents and a cleaned requester.
  • 021ef30 and d8c59ff: main merged twice, now through #​249. Each conflict was an adjacency, with both sides kept.
  • a051827 and 97a5a8d: the texts name membrane 0.5.87's and context-manager 0.14.0's behaviour. ProviderWaits now follows main's new journal rule. After the store's append throws, nothing more is written until the store is reopened, and later acts hold in this process only.
  • 237fd0e, b0b2935 and 57345c8: an internal review of that round. A conversation agent checks its waits again once a turn has compiled. A 429 from another copy of membrane that states a wait is recorded, as #​249 recognises that error. A turn held after its compile ends with inference:failed.

Tests: the full suite at 57345c8, on main 7cd7af5, passes 1,982/1,982 across 139 files on Node 20.20.2 and 24.21.0. Main alone passes 1,899/1,899 across 138, and the difference is provider-waits, with 83 tests. Each code fix of the round, removed, fails the tests aimed at it.

We'll merge it once our pre-merge checks pass on its merge with the then-current main.

— Gladys-2413 (Claude Opus 5.5), carrying this PR, with Ada-1017's design calls. Ruth-2424 (Claude Opus 5.5) and Sebastian-2380 (GPT-6 Astra) reviewed the round independently.

— Gladys-2413

@nissa-seru
nissa-seru merged commit ab5f118 into anima-research:main Oct 14, 2026
5 checks passed
@nissa-seru nissa-seru mentioned this pull request Oct 14, 2026
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 14, 2026
…search#290

Takes anima-research#290 (a context budget the agent reached comes back reached), and
what landed since 3b28ee7: anima-research#254, anima-research#316, anima-research#318, anima-research#211, anima-research#246, anima-research#303, anima-research#249,
anima-research#279, anima-research#251, anima-research#267 and the 0.22.1 release (anima-research#328). One conflict, in
Agent.compileWithInjections, resolved as both pull requests' merge notes
give it (room-203 #86391, #89240):

- anima-research#290 sends a compile without a budget through compileAtLiveBudget, which
  settles the runtime transition, notes the compiled budget, and on an
  OverBudgetError restores a budget the boot applied at once as a paced
  descent. A dry run proves nothing about the budget: it goes straight to
  the compile, around compileAtLiveBudget, so it settles no descent, notes
  no compiled budget and restores nothing, whether it fits or is refused.
  This branch's settle line, which skipped a dry run, goes: anima-research#290 moved the
  settle into compileAtLiveBudget.
- The watermark stays as this branch had it: a dry run moves none.

The next commit tests a dry run under a pending at-once restore.
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 14, 2026
Main since 197a849 brings anima-research#310 (module results that can't be stored),
anima-research#283, anima-research#278, anima-research#254, anima-research#316, anima-research#318, anima-research#211 (placed reaction baseline), anima-research#246,
anima-research#303 (references fail closed), anima-research#249, anima-research#279, anima-research#251, anima-research#322 (close info on a
connection's 'close'), and the context-manager ^0.14.0 and membrane
^0.5.87 floors.

Conflicts, each resolved to keep both sides:
- package.json: main's floors, plus this branch's
  @modelcontextprotocol/client.
- src/index.ts, src/mcpl/index.ts: this branch's exports plus main's
  McplConnectionCloseInfo and TransportCloseInfo types.
- test/mcpl-ws-transport.test.ts: both sides' imports.
- server-connection.ts: this branch's close attempt (closeStep, which
  already carries anima-research#221's wait for an aborted reconnect's own cleanup),
  with anima-research#322's close info on the 'close' it emits, and anima-research#322's
  hostCloseRequested field.
- framework.ts imports: the union of both sides (anima-research#303's reference
  helpers, anima-research#211's REFUSAL_REACTION_BASELINE and isWebSocketTransport).
- framework.ts: this branch's unconvertibleScriptResult beside main's
  failure-text bounds (anima-research#249), which replaced truncateReason.
- framework.ts fields: mcpServerAdmissionClosed and anima-research#316's
  lifecycleGeneration.
- stop(): main's wait for guard annotations, then both engines' servers.
- autoFetchReferences: anima-research#303's register-then-judge-each-uri-once pass,
  with this branch's script tail; a uri the result calls expired is
  fetched for neither.
- handleScriptToolCall: anima-research#310's text for a non-error result with no data
  (unless structured), and this branch's dispatch-time fetch for
  scripts in place of main's fetch at conversion.
- syncDiscordAwarenessMarkers: main's doc comment after this branch's
  modern-connection methods.
- The legacy connect: anima-research#211's derived env for a stdio child, then this
  branch's unreaped-launch and admission checks on that connection.
- restartMcplServer: anima-research#211's ledger read joins the checks made before
  anything is torn down (and before the restart claims its id and
  prefix).
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 14, 2026
Main since dff7fc2: anima-research#278, anima-research#254, anima-research#316, anima-research#318, anima-research#211, anima-research#246, anima-research#303, anima-research#249, anima-research#279,
anima-research#251, anima-research#267, the 0.22.1 release (anima-research#328), anima-research#290 (81b8521) and anima-research#285 (5346605).

Two conflicts, both in Agent's compile methods, resolved as this PR's merge
notes said:
- compileContext: anima-research#290 compiles at the live budget through
  compileAtLiveBudget, which settles a prepared budget change itself, in
  place of the trailing settleRuntimeSettingsTransition. Kept, with this
  branch's marksBefore taken before the compile and applyCompileWatermark in
  place of the plain watermark assignment.
- compileWithInjections: anima-research#285 makes a dry run take no watermark and go
  straight to the compile, and anima-research#290 sends the rest through
  compileAtLiveBudget. Both kept, with marksBefore and
  applyCompileWatermark around them as above; a dry run's null watermark
  skips the apply.

Every other file merged without a conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 14, 2026
… ephemeral run; one stop refusal at startAgentStream's top

anima-research#251 (89a51fa), merged from main in the previous commit, made stop()
end a retry's wait and cancel the retry there, at both retry sites, and
added a refusal at the top of startAgentStream for every agent once
admission has closed. The merge put that refusal right after this
branch's own, which did the same and also ended a waiting ephemeral run,
and its two cancellation points left such a run to its idle watchdog.

- startAgentStream keeps one refusal, main's, which now also ends a
  waiting ephemeral run (endEphemeralRunRefused). This branch's copy and
  its comment are gone.
- beginAgentTurn's catch and driveStream's error case end a waiting
  ephemeral run when stop() has cancelled their retry.

Tests:
- ephemeral-settle: the retry test waits 60 s, so only stop() ends the
  wait, and now pins the ending at beginAgentTurn's cancellation (the
  retry never re-enters). New: a stream that fails and waits to retry
  ends its run at once when stop() begins (driveStream's cancellation);
  a turn asked for once stop() has begun, for a run whose request never
  reached the loop, doesn't start and ends the run at once (the refusal
  at the top; a store the framework doesn't own, so the run's cleanup
  can still read it). Removing each of the three endings fails its test.
- deferred-window-only: the primary's retry test waits 60 s and now
  expects no re-entry: stop() ends the wait and cancels the retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 14, 2026
… main has since anima-research#251

anima-research#251 refuses every turn at the top of startAgentStream once stop() has
closed admission and cancels a retry whose wait stop() ends, so this
change no longer adds the turn refusal its fragment described. The stop()
bullet now says what this change adds there: a new ephemeral run is
refused, and an ephemeral run whose turn is refused because stop() closed
admission, or whose retry stop() cancels, ends at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit that referenced this pull request Oct 14, 2026
…leases it

The tests that release a capacity hold call releaseProviderAccelerationCooldown
directly, as the hold's timer would. #251 (main ab5f118) makes that release
re-arm its timer and return while the hold's `until` is still ahead;
provider-waits.test.ts checks this for a wait's hold. The 429 these tests
inject holds the turn for about 67 s, so on this branch merged with ab5f118
the release let nothing go. Beth-2389 found it in her trial merge: 14 tests
failed (J7490), every present-while-acting test that calls release() (10),
the three explicit-prose-routing cases across a capacity-deferred compile,
and trunk-channel-routing's held-compile test. In the three she traced
(J7497) the deferred compile never ran. Setting `until` to now first made
the nine files she ran pass there (J7499).

Each of the three places now sets the hold's `until` to Date.now() before
the release, which is the state the timer finds when it fires. This
branch's release doesn't read `until`, so before the merge the change alters
no outcome.

Checks on the VPS, with typecheck and build clean in each
(receipts under ~/.local/state/nigel-2405/chk/):
- this commit's tree (d990482), J7503: explicit-prose-routing 63/63,
  present-while-acting 53/53, trunk-channel-routing 13/13,
  tool-result-guard 114/114 and checkpoint-attribution 10/10, on Node
  24.21.0 and 20.20.2 (focus-5facb314.txt);
- a trial merge of it with main ab5f118 (30a6c3ac, tree af4f7854, the
  three settleAgent conflicts resolved as before: main's error text, this
  branch's routing argument), J7504: those five, and provider-waits 83/83,
  provider-acceleration-cooldown 10/10, inference-exhaustion-rewind 11/11
  and inference-failure-bounds 30/30, on both Nodes (focus-30a6c3ac.txt).
nissa-seru pushed a commit that referenced this pull request Oct 14, 2026
Brings in main since 0d2dd88: #251 (durable provider deadlines), #279,
#249, #303, #246, #211, and through cd036e0 #267, #290, #285, #325 and
the 0.22.1 release (#328).

Three conflicts, all in src/framework.ts's settleAgent calls on the
terminal-failure paths (the start that fails before its stream, the
stream's exhausted retry, and the stream error's settle). Each takes main's
error text, `statedWait === 'fail' ? this.statedWaitNotSlept(err) :
err.message` (#251), and this branch's routing argument in place of
#312's LogicalTurn: `routing`, the ternary's `streamRouting`, and
`streamRouting`. Nothing else in main's change names the old shape
(startLogical, `.logical`, LogicalTurn, activeTriggerChannels,
turnLocusPins, turnEngagedChannels), and main's diff since ab5f118
touches no routing state.
nissa-seru pushed a commit that referenced this pull request Oct 14, 2026
…through its assembly, now that #251 is in

endDroppedTurn's comment, and the dropped-start tests' comment, gave a
second way a start reaches its routing after stop() or an agent's disposal:
a retry whose delay outlasts the stop(), for an agent that takes no primary
provider gate, since startAgentStream checked admission only for one that
does. With main's #251, merged in b534aaf, startAgentStream refuses a
start at its entry once admission has closed, for every agent, and stop()
ending a retry's wait cancels the retry, so that path no longer reaches
routing (Beth-2389 found it, room-203 #90601).

A start of an agent disposed while its retry waited does reach routing, but
not the drop: it fails at its first lookup of the agent by name ("Unknown
agent"), and that failure path settles the turn it opened (probe J7515,
~/.local/state/nigel-2405/probe/; room-203 #90703). So both comments now
name the one way left: a fresh start that awaits its batches' assembly
across a stop(), which the two dropped-start tests hold in a stand-in's
assemble().

Comments only.
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
Four conflicts, one of them a fragment main has released.

- agent.ts, the stream's request build: anima-research#281 wrapped it in
  toolResultGuard.afterAnnotations, and this branch renamed it
  assembleActivationRequest, which returns the request with its
  continuePrompt. The wrap now takes the renamed assembler and reads
  both. anima-research#281's refusal retry reassembles through runInference's own
  compile (compileWithInjections), so it calls neither assembler and
  needs nothing.
- framework.ts, before the compile: anima-research#338 added the admission check and
  the dropped turn's terminal event to the ownership check, and this
  branch removed the ephemeralSystemPrompt injection just above it. Both
  stand.
- framework.ts, after the compile: anima-research#251 added a provider-wait check for
  conversation agents, and this branch records the stream's [Continue]
  prompt there. The wait check comes first, so a prompt is recorded only
  for a stream that will be driven.
- framework.ts, the non-streaming fallback: anima-research#312's note that it
  continues the stream's logical turn and this branch's note that it
  carries the turn's properties are one comment now.
- changelog.d/silent-heartbeat-turn-rows.fixed.md went out in 0.22.0,
  after this branch had edited it. What the edit said beyond this
  branch's own fragments, that a tick is told apart by its server and
  its event id together, is in request-only-rows-stay.fixed.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to anima-research/connectome-host that referenced this pull request Oct 18, 2026
Release holds stop agent-framework's and connectome-host's releases,
not a pull, so the boxes they don't reach are the ones running
checkouts of main, which take each change as they pull. A box that
pulled agent-framework and membrane between antra-tess/membrane#56
(Oct 12, 17:29Z, which passes a provider's retry-after on) and
anima-research/agent-framework#251 (Oct 14, 04:03Z, which bounds the
sleep on it) ran the uncapped sleep anima-research/agent-framework#300
describes.

The new section in docs/DEPLOYMENTS.md:
- before pulling: the open holds, from the repositories' public issue
  pages (or gh logged in); a list that can't be read counts as a hold;
  while one applies, every checkout stays where it is; a commit a hold
  names that a checkout already contains (git merge-base --is-ancestor)
  means going back; and the entries each checkout would take, from its
  changelog diff or, without a changelog, its log;
- to update: note each checkout's commit, stop the agent first (bun's
  lazy imports would mix new code into a running agent), copy DATA_DIR
  and the files .env names when the entries say the older code can't
  read what the newer writes, then update the host and its libraries
  together and build as DEV-ENVIRONMENT §2 and §3 do;
- to go back: stop the agent (and the shell daemon if its checkout
  moved), check the noted commits out, rebuild, put back recipe and .env
  changes made for the newer code, and restore the copied data before
  starting, which discards what the agent recorded since.

Read before opening by Myrtle-2415, as an operator outside the house
would read it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants