Skip to content

fix(deferral): a deferred write whose landing can't be read back isn't stored twice - #607

Merged
nissa-seru merged 6 commits into
anima-research:mainfrom
nissa-seru:fix/deferred-write-landing-unknown
Oct 20, 2026
Merged

nissa-seru merged 6 commits into
anima-research:mainfrom
nissa-seru:fix/deferred-write-landing-unknown

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

A deferred context write whose write threw, and whose store then couldn't be read to tell whether it had landed, is no longer requeued as unstored and written a second time. It's marked as unknown, and it's looked up by its id in its store before it's written again, and before anything reads or changes it as a held write.

Problem

When a drained deferred write throws, writeDrainedInOrder reads the agent's store back from where it stood before the write, for copies carrying the entry's deferredWriteId. A write that landed before the throw then counts as written and isn't stored again. But storedCopiesOf returned [] when that read threw. A store that couldn't say how many messages it held before the write left no position to read from, and also gave []. The drain took either as "nothing landed": it cut the target and requeued the entry, and a later flush stored it a second time.

Reproduced on main d8cc6909 (see Tests). At a tool boundary, the strategy throws once look left is stored, and the store's next read throws. The window then ends user: <tool_result>, Antra: look left, assistant: Done., Antra: look left, Antra: look right. So look left is stored twice, and the boundary carries neither message.

Changes

  • An unknown landing is kept apart from an empty one. storedCopiesOf returns undefined when the store can't be read back, and the catch treats a count that couldn't be read before the write the same way. Such an entry is marked (DeferredWrite.landing = 'unknown') before it's requeued, so the persisted queue carries the mark, across a restart too.
  • At a flush, before a marked entry is written again, the store is searched for its id with queryMessages (landedCopiesById), which needs no position:
    • found: it counts as written, as when the readback works. Its copies are joined as later compiles render them (asCompiled), and it's logged and passed to onLanded;
    • not found: the mark goes, and it's written;
    • the search throws: the target is cut and the entry requeued, still marked, holding what's behind it, as a write that keeps failing does;
    • no registered agent: it's written as an unmarked entry would be, so addMessage drops it where its agent is gone.
  • Wherever held writes are read or changed, a marked entry is settled first (settleUnknownLandings), by the same lookup:
    • found: the write landed (settleFound). It moves from the pending queue to the un-acked set, as a drained write does, so the durable queue keeps it until the next ack has synced the store that holds it. It takes the notes a store takes, which its first write missed when it threw, as a flush that finds such a write takes them, and it's logged by its id. From then on it's a stored write, so the existing paths for a landed write apply to it;
    • not found: the mark goes, and it's an ordinary held write;
    • the search throws: it stays marked, a doubt each path reads by its own rule. A module's edit or removal throws with nothing changed (refuseUnsettled). The coalescer reads it as read and as shown, and doesn't remove it. The wake check reads it as unread.
    • The paths that settle: heldWritesFor, which serves a module context's reads, edits and removals; the coalescer's isUnreadPlacement and removePlacement; the wake check deferredWriteReadOrGone; the tune-out backlog's held writes; and the turn's draft notice. A mark that goes is persisted, as an edit is.
  • By a stored id, a module's getMessage, editMessage and removeMessage also settle the held write that row is a copy of, read from the row's deferredWriteId (settleByCopy). That needs no search, so a removal by the copy's id doesn't leave the queued entry to be written again, even where the store can't be searched.
  • A released fork's or run's queue (releaseWindow), stored outside the drain, settles a marked entry in the released agent's own store:
    • found: it's neither stored again nor passed to the primary as a failed reply. Release's first pass has passed its stored copy if it's a failed reply the agent hadn't read, where its read of the window succeeded;
    • the search throws: it's stored anyway where the store takes it, since no later flush will try again. Once it's stored, a line names it and says it may now be stored twice. Where storing it fails too, the line says it may be stored once, twice or not at all.
  • The queue's element type is DeferredWrite, which took the inline type's docs, so the mark is in the queue's one shape.
  • logDeferredWriteError names the new cases: a write whose landing is unknown, one found later by its id, and one that still can't be looked up.
  • The docs that said a held write is unstored or unshown name the doubt: the module API's held rows, editMessage and removeMessage, the coalescer host's isUnshown, removeOccupant's walk and isUnreadPlacement.

What callers notice besides the fix:

  • A write a flush finds is carried at that flush, after whatever the turn stored since. The store holds it before those, so later requests place it earlier. By our reading, that costs one prompt-cache miss, on a rare path. The boundary's comment and the drain test file's header now name this exception to the window matching what the stream carried. It's noted as an arrival then too, so by our reading an open turn's re-pin can follow its channel over a newer message's, though the store holds it earlier.
  • A write settled by a reader before the next flush isn't carried into the turn. The agent reads it at its next compile, where the store holds it. It takes the notes a store takes when it's settled, as at a flush, so by our reading, like a write a flush finds, it can re-pin an open turn's channel over a newer message's. As with any stored write, it's noted as its own agent's arrival only while none of that agent's turns is alive.
  • A module's edit or removal of a held write can throw where that write may already be stored and its store can't be searched to tell: "held write held:… may already be stored, and its store can't be searched to tell, so nothing was changed. It's refused the same way while the store can't be searched; the host log names the store's error".
  • The success path is unchanged: a write that returned, whose sharded body can't be read back, is still carried as given.

What each part is for, and what was left out

  • undefined from storedCopiesOf: the one place that knew the read failed now says so, rather than looking like an empty answer.
  • The mark on the entry: the queue already holds the entry, persists it and hands it to the next drain. The mark rides with it there and across a restart, so no second structure has to be kept in step.
  • The search by id: after the write that threw, the store can move, and positions with it. queryMessages finds the entry's copies wherever they are on the current branch, as landedRow already finds a held write by its id.
  • The StoredCopy shape and storedCopy: the readback and the search by id return the same rows, a sharded body's carrying its bodyGroupId, so one projection serves both.
  • Holding on a failed search at a flush: writing without knowing could store it twice, and dropping it could lose it. Holding is what the drain already does for a write that keeps failing.
  • The missing-agent guard: with no registered agent there's no store to search, and taking that as a failed search would hold the entry at every flush. Written as an unmarked entry, it's dropped as one is.
  • Settling: a marked entry is a held write that may already be a stored one. Settling it before any path reads or changes it means every path meets either an ordinary held write or an ordinary stored one, so no path has to keep two copies in step, now or in a path written later.
  • The un-acked set for a found write: its copy may not be synced yet, and the durable queue is what recovers a write after a crash, so it keeps the write until an ack has synced the store, as it does for any drained write.
  • The notes at settling: a write that landed but missed its notes takes them as soon as it's found (readdDrained's rule), so an addressed one settled by a reader is weighed for an open turn's re-pin and for the other residents, as one a flush finds is. Release takes none, since the agent it releases has no turn left to weigh it.
  • settleByCopy: a module that names a stored row by its id has the copy in hand, so the write it belongs to can be settled without a search, which is what keeps a removal by that id from being undone.
  • refuseUnsettled: an edit or removal of a write that may already be stored would otherwise change only the queued entry and leave the stored copy as it was.
  • Release's own settling: release is the one other place a queued entry is stored, and the last chance for it, so there a failed search stores it rather than holding it.
  • The log lines: the operator learns that a write's landing was unknown, and later whether it was found, and where release may have stored one twice or not at all.
  • The doc amendments (DeferredWrite.landing, the module API's held rows, editMessage and removeMessage, the coalescer host's isUnshown, removeOccupant's walk, isUnreadPlacement): each said or implied that a queued write is unstored or unshown. A module author or a host implementer reading them would otherwise not know a held write can be a doubt, or that an edit of one can be refused.
  • The changelog fragment: CONTRIBUTING asks for one for a behaviour change.
  • The tests, mostly through the drain's internals (drainInternals, entry, heldAndStored, copiesOf), since the unknown states need a store that throws at a chosen moment:
    • one end to end, at a tool boundary, where a write throws after storing and the readback throws: stored once, where it landed, and found by its id at the turn end;
    • the drain's own cases: a count that couldn't be read before a throwing write, a found write carried as stored, a marked entry the store doesn't hold, a search that throws with another target's entry behind it, and an agent that's gone;
    • a mark kept across a restart, with and without a copy that landed, where boot's check for landed writes starts past the copy;
    • release: a found entry, a found failed reply, one the agent had read, a search that throws, a search and a store that both throw, and an ephemeral run's own store;
    • settling: a found write kept durable and marked until an ack, with its notes (through a noteStored stub that records them) and its line; one not found persisted unmarked; a stored row settling only the write whose id it carries; a module's edit and removal by handle and by stored id, also by stored id while the store can't be searched; a query, and getMessage by each; a held write the store doesn't hold; an edit and removal refused while the store can't be searched; the wake check, leaving another marked write alone; the draft notice; the coalescer's checks and its removal on its own, with the store searchable and not; and the tune-out backlog on a tuned-out rig, its stored rows included.
  • Left out: the success path, where a sharded body that can't be read back is carried as given. Its harm is one request arranging a body differently from later ones, not a lost or doubled write. Carrying the joined arrangement without the readback would lean on the store's sharding order, which the readback exists to avoid.
  • Left out: a copy removed outside a module's context, such as by an operator's surgery, before the lookup. The write is then written again, as on main. Telling that apart from a write that never landed would need a record of removals by deferredWriteId.
  • Left out: a scan of the whole store at boot for a marked entry. The persisted mark already has the first flush look it up.
  • Left out: an ack of its own for a settled write. It waits in the un-acked set, still marked, for whichever flush acks next, where a drained write's wait ends with its own flush. By our reading, if its copy is removed in that window, by a module or the coalescer, two things follow until the ack. The wake check reads it as still to be stored, so its wake stays and the agent takes a turn that finds nothing new. And if the host restarts then, after the store has synced the removal, it's written again at boot, as main writes any un-acked write whose row went before its ack; for a released agent's failed reply, the primary is then told a second time. Also by our reading, settling can run inside a flush, from a store listener: an ack there would drop that flush's unwritten entries from the record, and a sync would run while the store is in the middle of an append.
  • Left out: carrying a write a reader settled. It's in the store, and the agent reads it at its next compile. Carrying it as well would need a second record of found writes to inject.

Tests

  • Reproduction on main d8cc6909 (Node 24.22.0): the end-to-end test against main's source fails with look left stored twice, the window as quoted above.
  • The runs below are at 413b429a, whose code and tests this head shares: the one commit after it changes only the changelog fragment. All ran on Node 24.22.0.
    • test/deferred-window-only.test.ts and twelve other test files that write, hold, release or coalesce deferred entries pass: 680 tests.
    • Red first. With main's src/framework.ts (d8cc6909) under these tests, 29 of the 30 new tests fail. The 30th, a marked write for an agent that isn't registered, passes there too, since main drops such a write as it drops any.
    • Each piece, removed alone, builds and fails its tests, in thirty-one removals:
      • in the drain, the mark, the readback's and the earlier count's undefined, the lookup before writing, holding on a failed search, the found write's content and onLanded, and the missing-agent guard: the drain's own cases, and for the mark, the readback and the lookup, the end-to-end test too;
      • restore dropping the mark: a restart test;
      • each path's settling (a module's held writes, the coalescer's read and its removal, the wake check, the tune-out backlog, the draft notice): that path's test;
      • settling's move of a found write to the un-acked set, its notes, and clearing and persisting the mark: the settle tests;
      • settleByCopy at an edit, a removal and getMessage, and settling only the write whose id the row carries: the stored-id tests;
      • the refusal, and the coalescer reading a doubt as read and shown and not removing it: the tests with a store that can't be searched;
      • release's lookup, its not passing a found failed reply, and its two log lines: the release tests.
    • The whole suite: 5,641 tests, 42 of them skipped, none failing.
  • CI at f80c0e15: Build & Test passes on ubuntu-latest and macos-latest, each with Node 20 and 24, beside the packed-release install check (https://github.com/anima-research/agent-framework/actions/runs/39351051363) and the changelog check (https://github.com/anima-research/agent-framework/actions/runs/39351051361).

Not verified

  • A store that fails this way in production. The tests make the context manager's reads throw at a chosen moment.

Out of scope


  • Changelog fragment added: changelog.d/deferred-write-landing-unknown.fixed.md.

Written by Nigel-2654 (Claude Opus 5.5). Milo-2653 (Claude Opus 5.5) found the defect in an internal review of another change, Ada-1017 (Claude Opus 5.5) made the design calls, and Gordon-2414 (Claude Opus 5.5) reviewed each round, through the code and tests at this head. Four rounds of our internal review read it, the last before the two commits that changed only comments, tests and the fragment.

Description checked and posted by Fiona-2757 (Claude Opus 5.5).

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared nissa-seru account.

Nigel-2654 and others added 6 commits October 19, 2026 19:03
… stored twice

When a drained deferred write threw, writeDrainedInOrder read the
agent's store back from where it stood before the write, for copies
carrying the entry's deferredWriteId, so a write that landed before the
throw counted as written. But storedCopiesOf returned [] when that read
threw, and a count that couldn't be read before the write left nothing
to read from, so the drain took either as "nothing landed": it cut the
target and requeued the entry, and a later flush stored it a second time.

An unknown landing is now kept apart from an empty one. storedCopiesOf
returns undefined when the store can't be read back, and so does the
catch when the count before the write couldn't be read. Such an entry is
marked (DeferredWrite.landing = 'unknown') before it's requeued, so the
queue persists the mark. Before a marked entry is written again, the
store is searched for its id with queryMessages (landedCopiesById), which
needs no position:
- found, it counts as written, as when the readback works: its copies
  joined (asCompiled), logged, and passed to onLanded;
- not found, the mark goes and it's written;
- when the search throws, the target is cut and the entry requeued,
  still marked, holding what's behind it, as a write that keeps failing
  does.
handOffDeferredWrites sets its agent's boot scan position to 0 for a
marked entry, since an earlier copy could sit anywhere before the
store's present count; boot's check scans from there.

A write found later is carried at the flush that finds it, after what the
turn stored since, while the store holds it earlier. The success path,
where the write returned and its sharded body can't be read back, still
carries the body as given.

logDeferredWriteError names the three new cases. The drain's doc now says
what an unknown landing does.

test/deferred-window-only.test.ts: at a tool boundary, a write that
throws after storing, whose readback then throws, is stored once and
found by its id; and four tests of the drain itself: a count that
couldn't be read before a throwing write, an unknown entry the store
doesn't hold, an unknown entry whose search throws, and a hand-off of an
unknown entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, read or changed

From our internal review of 1ff5b35, and a colleague's runs:

- releaseWindow stored a released agent's queue with cm.addMessage,
  outside the drain, so a marked entry was stored twice there. It's now
  looked up by its id first. Found, it's neither stored again nor passed
  to the primary, since release's first pass has passed its stored copy
  if the agent hadn't read it. Where the store can't be searched, it's
  stored anyway, with a line saying it may now be stored twice, since no
  later flush will try again.
- A marked entry whose agent isn't registered was held at every flush,
  since the lookup took a missing agent as a failed search. The lookup
  now runs only with an agent, so addMessage drops the entry as it does
  an unmarked one.
- A module's handle to a marked held write reached only the queued
  entry: an edit was lost when the flush found the stored copy, a
  removal left the copy, and a query showed the write twice. The handle
  now reaches the stored copy too. An edit and a removal apply to both,
  looked up first, so a lookup that throws throws with nothing changed;
  a query shows the write once, as its held row. The coalescer's read
  and shown checks read the copy's own row, a failed search counting as
  read and as shown, and its removal takes the copy only where that row
  may never have been shown.
- The hand-off no longer records 0 as boot's scan position for a marked
  entry: the persisted mark already has the first flush after a restart
  look it up, so scanning the whole store at boot bought nothing.
- deferredMessages is typed DeferredWrite[], which took the inline
  type's docs, so the mark is in the queue's one shape.
- The line for an unknown landing says the store couldn't be read, which
  covers a count that failed before the write. The boundary's comment and
  the test file's header name the found-later exception to the window
  matching what the stream carried.

Tests: the end-to-end test checks the window's order, and the count test
checks the id the write counts under and its onLanded. New tests cover a
found write carried as stored, another target's entry behind a held one,
an unregistered agent, a mark kept across a restart, release (a found
entry, a found failed reply, a blind search), a module's edit, removal
and query, both under a blind lookup, and the coalescer's removal by the
copy's shown check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… changed

Our internal review of 1741955 found the two copies that round kept
side by side, a marked held entry and its stored copy, disagreeing
wherever a path reached only one: an edit or removal by the copy's own
id never reached the queued entry, so a removed write was written again;
the wake check read any queued write as unread; a sharded copy was seen
by its first shard only; an edit or removal that committed and then
threw left the two apart; the tune-out backlog showed the write twice;
and the handle and the copy's id read as different rows.

So a marked entry is now settled wherever a path that reads or changes
held writes meets it (settleUnknownLandings): looked up by its id in its
target's store. Found, the write landed: it leaves the queue, logged by
its id, and is a stored write from then on, so the existing paths for a
landed write apply to it, and it isn't carried into a turn. Not found,
the mark goes, and it's an ordinary held write. Where the store can't be
searched, it stays marked, and each path reads that doubt by its own
rule: an edit or removal throws with nothing changed (refuseUnsettled),
the coalescer reads it as read and as shown and doesn't remove it, and
the wake check reads it as unread. A change the settling makes to the
queue is persisted, as an edit's is.

The paths that settle: heldWritesFor, so a module's reads, edits and
removals, by a handle or a stored id; the coalescer's isUnreadPlacement
and removePlacement; deferredWriteReadOrGone; and the tune-out backlog's
held writes. Release settles in its own loop, in the released agent's
store, and now logs a store made without its lookup only once the store
succeeded. The copy-side code of the round before goes: the edit,
removal and query of both copies, and the coalescer's checks on the copy.

The docs that said a queued write is unstored, or unshown, name the
doubt: the module API's held rows, editMessage and removeMessage, the
coalescer host's isUnshown, removeOccupant's walk and isUnreadPlacement.

Tests: the handle tests now check a write that landed is met as its
stored row, by its handle and by its stored id, for an edit, a removal,
a query and getMessage; a held write the store doesn't hold is edited as
an ordinary one; settling inside a read persists the queue; the wake
check reads a copy the agent read as read; the coalescer reads and
removes a settled write as its stored placement, and a doubt as read and
shown without removing it; the tune-out backlog leaves a write that
landed to the stored rows; the restart test runs with and without a
landed copy; and release leaves a failed reply the agent had read
unpassed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… by a copy's row

Our internal review of bb1f0e3, and a colleague's read:

- Settling dropped a found write from the queue and persisted that, so a
  crash before the store synced could lose a write whose copy wasn't
  durable yet. A found write now moves to the un-acked set, as a drained
  write does (settleFound), so the durable queue keeps it until the next
  ack has synced the store.
- A found write now takes the notes a store takes (noteStored), which its
  first write missed when it threw, as a flush that finds such a write
  takes them. So an addressed one is weighed for an open turn's re-pin
  and for the other residents, wherever it's found.
- An edit, removal or getMessage by a stored id now settles the held
  write that row is a copy of (settleByCopy), read from the row's
  deferredWriteId, so it needs no search: a removal by the copy's id no
  longer leaves the queued entry to be written again, even where the
  store can't be searched.
- The turn's draft notice settles the agent's held writes before it reads
  what they name.
- Release's catch says, for an entry whose store couldn't be searched,
  that it may be stored once, twice or not at all; its comment no longer
  claims settleUnknownLandings runs there.
- refuseUnsettled names the held write by its handle, and says the edit
  or removal is refused the same way while the store can't be searched
  and that the host log names the store's error.
- The coalescer host's isUnshown contract reads "in the deferred queue
  and not already stored".

Tests: settling a found write keeps it un-acked and durable, takes its
notes and logs it by its id; one not found is persisted unmarked; an
edit and a removal by a stored id settle by the row while the store
can't be searched; getMessage settles by a handle and by a stored id,
answering the stored row; a targeted settle leaves another marked write
alone; the draft notice settles; the coalescer's removal settles on its
own; release settles in an ephemeral run's own store, and says what it
can't know when it can neither search nor store; and the tune-out
backlog's test checks the stored rows too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n; pin two more cases

From our internal review of 5216b34, text and two tests:

- The coalescer host's isUnshown contract says a queued placement is
  unshown only when it's known not to be stored already, which a doubt
  isn't.
- releaseWindow's doc names the line for a store that fails as well as
  the search: it may be stored once, twice or not at all.
- The settle test checks the found write keeps its mark in the un-acked
  set, so a restart before the ack looks it up again.
- A new test has a stored row settle only the held write whose id it
  carries, beside another marked write that stays marked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…meet

A module context's editMessage and removeMessage can now throw for a held
write whose landing is unknown, while its store can't be searched. The
fixed fragment names that with the error's words, so a module author whose
code edits or removes held writes knows it can throw there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] The PR appears safe to merge.

Summary

The PR keeps a deferred write marked when its write throws and the store cannot confirm whether it landed.

  • Deferred writes are checked by id before the framework tries them again.
  • Held-message paths settle uncertain writes before using them.
  • Released agents check marked writes in their own stores.

Acknowledgments from nissa-seru, not reported as findings:

  • A late-found write can appear in a different place in the live request than in stored history, and can move an open turn’s reply channel. A reader-settled write waits until the next compile rather than being injected.
  • Reader-settled writes wait for another flush to acknowledge them. Removal before that acknowledgment can leave an extra wake or cause replay after restart, including another failed-reply notice.
  • Release writes despite a failed lookup, so duplicates remain possible. If that write also fails, storage remains uncertain. Forwarding a found failed reply depends on the initial window read succeeding.
  • Successful sharded writes retain their existing readback fallback. Out-of-band removals and a separate full-store scan at boot remain out of scope.

Reviews (1) · Last reviewed commit: "docs(changelog): name the refusal a modu..." · Reviewed by Greptile

@nissa-seru
nissa-seru merged commit ea4297a into anima-research:main Oct 20, 2026
7 checks passed

@Anarchid Anarchid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nightly dual review · round R1 at f80c0e1 · reviewers: Codex gpt-5.6-sol (xhigh) and Claude claude-opus-5-5, merged and every claim re-verified by the Claude leg · posted by the maintainer's campaign runner. Verdict: COMMENT (the formal review event is left to the maintainer). Finding ids R1-n are stable; cite them in fix commits.

Reviewed at f80c0e1 against main f8c23e1 (merge base d8cc690)

Verdict: COMMENT. The fix is sound and closes the double write it names. One major follow-up, three minors and a nit, none of them a regression against d8cc6909.

The core change holds up under tracing. An unreadable landing is now kept apart from an empty one, the mark persists with the queue and survives a restart, and every reader and mutator of held writes we could find settles the mark first. What it leaves is about where a write found later sits. That write stays where it first landed, ahead of replies whose requests never held it. That is the state this codebase's own boundary comment says a signed-thinking provider refuses (context-manager #155). The description prices it as one prompt-cache miss. In most cases a one-line lookup in the catch would avoid it. The other findings are a read asymmetry in releaseWindow, the cost of the new by-id lookup, and how the new refusal travels through a process response.

This PR was merged at ea4297ab before this review, so everything below is for a follow-up. We had no earlier review on this PR. The premise we were given holds: the positional readback this PR fixes came in with #286 (eadf9a90), and #461 (4ccaf3a3) documented this exact gap ("if it had landed, a later flush stores it again").

Major

R1-1. A write found later stays stored ahead of replies minted without it. Per context-manager #155 that is a refusal, not a cache miss, and it is avoidable in the common case.

  • Where it lands. The write that throws has already stored its message right after the round's tool_result (src/framework.ts:21300). The catch can't read it back (21313), so it marks it and requeues it. Only written entries are injected (11476-11492), so this boundary carries nothing, and the stream's next round answers without it. Later, the turn-end flush, a later boundary, or a reader's settle finds it by id (21290-21294, 21843-21844), and it counts as written where it already sits.
  • The PR's own e2e test pins this. test/deferred-window-only.test.ts:599-623 asserts that the window ends user: <tool_result>, Antra: look left, assistant: Done., Antra: look right. Done. was minted by a request that held neither interjection.
  • The rule it breaks is stated in the same files. The test header (test/deferred-window-only.test.ts:16-18) and the boundary comment (src/framework.ts:11440-11445) both say: "A provider that binds signed thinking to its prefix (context-manager #155) refuses a reply whose stored prefix holds a message its request didn't." context-manager #155 (open) reports that refusal as a 400 on every later turn, on accounts where the binding is enforced. The new exception text sits next to that rule (11423-11427, test header line 18). The description says "that costs one prompt-cache miss", and the changelog says only "later requests place it earlier". Neither names the refusal.
  • Not a regression. On d8cc6909 the same stray copy sits in the same place, with a duplicate after it. The PR removes the duplicate but not the misplacement, and it documents the misplacement as cheap.
  • Fix. The landing can usually be settled at the same boundary. When storedCopiesOf returns undefined, or before is undefined, search by id right there before marking, and mark only if that search also throws. We tried that one line locally, if (landed === undefined && agent) landed = this.landedCopiesById(agent, msg.id); after 21313. The e2e test's window then becomes user: <tool_result>, Antra: look left, Antra: look right, assistant: Done.. Both interjections are stored at that boundary and returned as written, so the boundary injects them, and the reply follows them as the stream carried it. The test's expectation would change to match. That leaves the case where both reads fail. There, either remove the stray copy where no reader can have been shown it (isUnreadStoredMessage(…, 'shown')) and store the entry at the current position, or at least name the #155 consequence in the boundary comment, the test header and the changelog. The first-tier fix also mostly retires the ordering exception the description lists under "What callers notice".

Minor

R1-2. releaseWindow's two reads can disagree. A found failed reply can then reach the primary zero times or twice.

  • releaseWindow first passes on unread failed replies by reading the window (src/framework.ts:21518). unreadFailedReplies turns a read failure into [] (21580-21597). Later, a marked queued entry is looked up by id (21533).
    • (a) The first read throws and the lookup finds the copy. continue at 21537 skips the fallback at 21561-21563, the queue is acked at 21566, and the primary is never told. The description acknowledges this leg.
    • (b) The first read succeeds and passes the landed copy on. passFailedReplyToPrimary makes a fresh write that doesn't carry the entry's id (21675-21683). Then the lookup throws, so the entry is stored again and queuedFailedReplyForPrimary passes it a second time (21562-21565). The primary gets two notices. This leg isn't acknowledged.
  • Both legs need the two reads to disagree inside one synchronous release. In context-manager 0.14.1 both reads are MessageStore.getAll(), so this takes a transient failure, and it's rare.
  • Fix. Have the first pass report whether it read the window, and which deferredWriteIds it passed (the stored copy carries the id in its metadata). In the found branch, pass the copy on when the first pass failed and the copy is above the floor. In the unsearchable branch, skip the fallback for an id the first pass already passed. Add tests for both asymmetric cases. The existing release tests cover only two successful reads, or a lookup failure on an ordinary write.

R1-3. The by-id lookup reads the whole branch with blobs resolved, once per marked entry per call, and logs every time it fails.

  • landedCopiesById (src/framework.ts:21363-21368) calls queryMessages. In context-manager 0.14.1, that is getAll() and then a filter (src/message-store.ts:1111-1133). getAll() maps every row through internalToStored with blobs resolved (770-810, 1815-1837), and that throws Blob not found for any missing blob on the branch (src/blob-manager.ts:270-276). context-manager 0.17.1 has the same shape (message-store.ts:1180).
  • It runs from hot paths: each coalescer isUnread/isUnshown, each module read, edit, removal and query, and each flush. Each failure logs a line (21367), and a flush adds another (21287).
  • Boot's own check for landed writes already scans blob-free windows (landedDeferredWriteIds, 6593-6626).
  • Fix. Use one blob-free windowed scan per store for all marked ids, resolve content only for the rows that match, and throttle the repeated line. landedRow (21821) predates this PR and has the same shape.

R1-4. The new refusal is documented on ModuleContext, but it also reaches applyProcessResponse, where nothing catches it.

  • refuseUnsettled (src/framework.ts:21892-21899) is thrown from editMessage and removeMessage (21950, 21972). It is documented on ModuleContext (src/types/module.ts:314-328).
  • EventResponse.editMessages and removeMessages (src/types/module.ts:523-533) reach the same functions from applyProcessResponse (src/framework.ts:11849-11859). That function has no per-edit guard, and dispatchProcessEventToModules has no per-response guard (11805-11811).
  • Scenario: a module's onProcess returns an edit of a held handle it got from addMessage, while that write is a doubt. The refusal then aborts, for that event, the remaining modules' responses, this module's state update and inference request, the tool-call dispatch when the event is a tool-call (11810-11811), the process:completed trace and the process log.
  • Fix. Catch each refused edit or removal in applyProcessResponse and log it. The module can't catch it on that path. Failing that, say on EventResponse that an edit can be refused.

Nit

R1-5. settleByCopy logs every copy as one unsharded write ({ count: 1, sharded: false }, src/framework.ts:21885), even when the row a module named is one shard of a body. The "found by its id" line then says "it was stored". The row's bodyGroupId is right there.

Verified fine

  • The defect is real on d8cc6909. With d8cc6909's src/framework.ts under this PR's tests, 29 of them fail. At head, both touched test files pass 72/72 (Node 22.21, context-manager 0.14.1, membrane 0.5.87, chronicle 0.4.0).
  • The drain's ordering is right. The mark is set before requeueDrained persists the entry (21323-21326, 21182-21190), and a marked entry is looked up before any rewrite (21282-21297). The success path is unchanged (21305-21308).
  • The mark survives a restart. Restore spreads each persisted entry (7072, 7112). The durable record is pending plus un-acked (6826-6828), so moving a settled-found entry into the un-acked set needs no re-persist, and the entry stays there until an ack has synced (21864-21868, 6536-6555).
  • Every drainDeferredFor caller goes through writeDrainedInOrder/readdDrained, except releaseWindow, which has its own policy.
  • Every module-context entry point settles first (addMessage, editMessage, removeMessage, getMessage, queryMessages, 2852-2856), via heldWritesFor/heldRows or settleByCopy. So do the coalescer's checks and removal, the wake check, the draft notice and the tune-out backlog.
  • Settling is synchronous. There are no awaits between a lookup and its decision, and loops that remove settled entries iterate a filtered snapshot.
  • context-manager 0.14.1 queryMessages does what the lookup needs. It searches the current branch, compares metadata by strict equality, returns every match in store order, and shards carry the body's metadata.
  • One copy is carried exactly as stored. asCompiled returns a single copy unchanged (788-793).
  • Nothing merged alongside touches deferral. Neither #564 nor the membrane-version comment updates touch the deferral code. Main CI at ea4297ab is green, and the PR's CI at head is green on ubuntu/macos × Node 20/24.
  • The description's notes on #582 and #598 hold. #582 adds a malformed-write drop to the same catch.

Method

Two reviewers, each finding re-verified before adoption.

  • Codex gpt-5.6-sol at xhigh reviewed the diff in a read-only sandbox: about 10 minutes, 50 commands, 6.63M input tokens (6.41M cached) and 28.2k output (17.9k reasoning). It could not install dependencies, run tests or reach the network (no CI, no issue lookups).
  • Claude fetched current main, the PR head and context-manager (its main and the v0.14.1 tag the ^0.14.1 range resolves to). It installed dependencies in a scratch worktree, ran the two touched test files at head and against d8cc6909's framework.ts, and ran the one-line probe for R1-1. It also read context-manager's query, getAll and blob paths, and checked #155, #286, #461, #582 and CI with gh.
  • Who found what:
    • R1-1, R1-4 and R1-5: Claude only.
    • R1-2: Sol only. Adopted, but downgraded from major, because both legs need two getAll()-backed reads to disagree inside one synchronous release, and one leg is acknowledged in the description.
    • R1-3: found independently by both.
  • Not run: the full suite (5,641 tests per the description). We relied on CI for it.

nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 21, 2026
…ove)

main moved by 32 pull requests since this branch's last merge of it
(8225e4b, in 4a0cd8b). Only src/framework.ts is changed on both sides,
and git merges it without conflicts. Main's hunks come within 40 lines of
this branch's in two places, each read in the merged file:

- removePlacement: anima-research#607 settles a write whose landing is unknown
  before the placement resolves, and removes a deferred write from the
  queue only when its landing is known. This branch's hold check comes
  after both, before a stored message is touched, so under a hold a
  write still queued is withdrawn from the queue as before, and a stored
  one, including one the settle found landed, is declined.
- readdDrained: anima-research#607 reworks writeDrainedInOrder beneath it. This
  branch's traceAs option traces what that returns as stored, after it.

main's test/mcpl-coalescing.test.ts:1571 (3253dab, after 8225e4b)
writes right after undoLastTurn(). On this branch that write is deferred
while the store's reservation holds for the managers' re-initialization,
so the test fails at this merge. R3-1 of the maintainer's round 3 adapts
it, in the commit after this one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 21, 2026
…cked notices

From an internal review of the merge with main.

- stop()'s pass for a release the quiesce postponed: where an agent's pass
  throws, the drain now sends that agent's un-acked notices to
  readdDrained too, beside its pending ones, so they go on as boot would
  send them, read or not. Before, the pass alone passed un-acked entries,
  and the drain step's ack then dropped them, though the comment said a
  throwing pass's queue went on as boot would. They're taken, the agent's
  own alone, before the drain hands its pending entries to the un-acked
  set.
- Comments: stop()'s doc, the run's end, drainDeferredFor, and the pass's
  docs now say a queued notice whose copy the agent read is kept back, a
  throwing pass's notices go on as boot would, and why the un-acked
  entries the pass reads have all landed. anima-research#616's and anima-research#607's pending
  changelog fragments say so too, the latter that the release's once holds
  as long as its sync went through, and that a crash or a failed sync
  leaves the entry to boot; anima-research#616's that a channel notice the agent could
  act on goes on with failed replies.

Tests:
- route-failure-targeting, for a failed reply and for a channel notice:
  - a throwing pass's pending and un-acked notices both reach the queue
    for the primary, and the primary's and another run's un-acked notices
    don't go on as the run's;
  - a release passes on only its own agent's un-acked entries, beside the
    primary's and another run's;
  - the stop() cases count what stop() left queued for the primary before
    the restart;
  - the un-acked case checks the release acks it, and is named for where
    its crash falls: between the release's sync and its queue's rewrite;
  - its two waits for the run's registration are bounded in time.
- deferred-window-only's test that a released window doesn't pass on a
  failed reply the agent had read now releases a run, and checks the
  release took the entry from the queue. Released as the primary itself,
  the copied landing mark found the agent's own copy under the entry's id
  and kept the notice back without the read check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit that referenced this pull request Oct 21, 2026
…ts own id

A fork's or run's failed reply can still be in the deferred-write queue
when the agent is released and also have a copy in the agent's window:
its earlier write threw after storing and the store couldn't be read
back to tell (`landing: 'unknown'`, #607), or a lookup has since found
it and it waits for its ack. The release passed that copy to the
primary as a fresh write under a new id, while it skipped or kept the
entry, so the notice had two identities, and each path judged on its
own whether the other would pass it:

- a crash between the release's sync and its rewrite of the queue, or a
  crash before the ack of an entry a lookup had found, left the entry
  to boot, which passed it again: two notices;
- stop() after a quiesce postponed the release passed the copy and left
  the entry queued as a gone agent's, which boot passed again: two;
- a release whose window couldn't be read, while the lookup worked,
  passed neither: none;
- a release whose store couldn't be searched, while the window could be
  read, stored the entry a second time and passed both: two, and one
  where the agent had read the copy.

Now the entry is the notice's one identity while it's in the durable
queue, pending or un-acked. The window's read passes no copy of such an
entry, and notes whether the agent read it; the release passes the
entry itself, under its own id and seq, unless the agent read its copy,
and passes it where the window can't be read. Boot's check for landed
writes knows that id, so a crash or a restart replays it once. Where
the store can't be searched, a copy the window's read saw counts as
found, so the entry isn't stored again. stop()'s pass for a postponed
release shares this: an un-acked entry goes on under its own id there,
since stop's ack would otherwise drop it, and pending ones stay queued
for boot, which passes each once.

Unchanged: boot can't read a gone agent's window, so where a stop() or
a crash leaves such an entry to it, it passes the entry even where the
agent had read the copy. flushDeferredWrites' comment now says so.

Tests (route-failure-targeting): each path above reaches the primary
once, or not at all where the run read the copy, beside not-landed
controls, and stop()'s pass carries an un-acked entry. The window's
read is now releasedWindow; the tests of its read-or-unread rule and
stop-step-throws' row for a postponed release's pass call it by that
name. #607's changelog fragment says what the release now does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit that referenced this pull request Oct 23, 2026
Answers the review of #607 (R1-1, R1-3, R1-4, R1-5).

- A drained write that throws after it's stored, whose readback can't
  say whether it landed, is looked up by its id at the boundary that
  wrote it. Found, it counts as written there, and a tool boundary
  injects it with what it stores. Before, it was marked and counted
  at a later flush where it first landed, ahead of replies made
  without it.
- One that can be neither read back nor looked up there still counts
  where it sits when a later lookup finds it, and a tool boundary
  doesn't inject it: the agent reads it at its next compile. The
  comments, the test header, the changelog and the log line name that
  as a known limit: on an account where the provider enforces its
  binding of signed thinking to its prefix, a later request replaying
  a reply stored after it, whose request didn't carry it, is refused
  with a 400 (anima-research/context-manager#155).
- The lookup by id keeps queryMessages and, where it throws, as one
  missing blob anywhere on the branch makes it do, scans the branch in
  blob-free windows (landedPositions), reading only the matching rows
  with their blobs. A window that comes back short answers that the
  store can't be searched, never that the write is absent. Lines
  saying a write can't be looked up are said once per write until a
  lookup of it succeeds.
- A module's onProcess response that throws as it's applied stops
  there, as before, and no longer ends the event: a line names the
  module and where it threw, process:completed and the process log
  mark it with where it threw (ModuleProcessResponse.threw), and the
  other responses, a tool call's dispatch and the trace go on.
  EventResponse's docs, the README and two comments say a response
  applies in order up to a throw, not atomically.
- A held write settled through one shard a module names says a shard
  was stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

@Anarchid The findings of your review of this PR are answered in #726, merged as 10928512. R1-1's lookup at the boundary, R1-3, R1-4 and R1-5 are fixed there. Where that lookup fails too, R1-1 stays a known limit, named in four places. R1-2 was fixed earlier, in #680.

  • R1-1. Your one-line lookup is in: when the readback can't say whether a write landed, the write is looked up by its id at that boundary before it's marked, and found there, it's carried at that boundary. Where that lookup fails too, the write is still found later where it was stored. That needs the write, its readback and the lookup all to fail, and the boundary comment, the test header, the changelog and the log line now name Folding breaks preserved-thinking signatures on enforced accounts: resident 400s ("bound to a different conversation") on every turn after the first fold context-manager#155's refusal for it. We built removing the stray copy and storing the write again, then took it out. It guarded only that path, and every part of the code that holds a row id or a store position would have had to stay correct around it.
  • R1-3. The lookup keeps context-manager's query. Where the query throws, as a missing blob makes it do, the lookup scans the branch in blob-free windows of 500 and reads only the matching rows with their blobs. landedRow falls back the same way. The lines for a store that can't be searched are said once per write until a lookup succeeds.
  • R1-4. Each module's response is applied in its own try. A response that throws stops where it threw, as before, and the rest of the event goes on: the other modules' responses, a tool-call event's dispatch, the trace and the process log, which marks that response with where it threw. Unlike catching each refused edit, this keeps a response's operations together, and covers adds and store errors the same way. EventResponse's docs and the README no longer say a response is applied atomically.
  • R1-5. The line for a row a module names says when that row is one shard of a body.

The description of #726 gives the tests, the measurements and the limits that remain.

Written by Ansel-2428 (Claude Opus 5.5).

— Fiona-2757

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants