Repository navigation
feat(mcpl): RFC-006 event coalescing, kept in context - #197
Conversation
Implement mcpl RFC-006 revision 7 (replace-if-unread, atomic retraction, `initial`, deferred push/render) for both delivery lanes without a staging area, journal or subject cap. A plain occurrence is delivered on arrival through the ordinary channel/push path; a later occurrence of the same subject removes it while it is still unread — above the agent's consumed watermark, on the current branch, not sharded, not folded by compression — and delivers the replacement as a fresh message; once read, it appends. Deferred batches alone wait outside context and render at the turn's assembly boundary with the connection's 5 s bound. On any doubt (restart, branch switch, eviction, unknown history) the host appends, as today. Channel-scoped pushes are delivered as messages of their channel, so conversation forks, tune-out and the reply locus follow. Channel registration lifetime is unchanged. Receipts (1 h retry window) and subject history live in the `mcpl/coalescing` snapshot, written throttled and without a sync per message. Tests: 34 wire-level cases named by RFC-006 §14 vector, plus regressions for the five findings on the earlier journal implementation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
| if (message.coalesce !== undefined && this.handleCoalescedIncoming) { | ||
| // RFC-006 §14.3: a coalesced message is admitted like any other and | ||
| // then handed, with the event the ordinary path would have queued, to | ||
| // the coalescer, which replaces, appends or withdraws. Malformed | ||
| // `coalesce` is a per-message failure; siblings are unaffected. | ||
| try { | ||
| results.push(await this.handleCoalescedIncoming(serverId, message, event)); |
There was a problem hiding this comment.
Rejected items change reply routing. Channel admission changes the default publish channel before this hook validates a coalesced item or recognizes a retry. An item that returns
coalesce_invalid, or a duplicate that delivers nothing, can therefore redirect a subsequent turn's fallback reply to its channel.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/mcpl/channel-registry.ts
Line: 971-977
Comment:
**Rejected items change reply routing.** Channel admission changes the default publish channel before this hook validates a coalesced item or recognizes a retry. An item that returns `coalesce_invalid`, or a duplicate that delivers nothing, can therefore redirect a subsequent turn's fallback reply to its channel.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
Additional review of 1. Channel-scoped render results survive revocation of
|
… recovery Address the first review of #197 (15 Greptile findings, 3 Codex findings): - Channel scope always checks the CURRENT channels.incoming grant, on both lanes, at admission and at render response (C1, §5.4 rule 4). - Subject and receipt keys carry a host-owned binding identity (server id bound to its endpoint/command): a reconnect keeps it, reassigning the id to another endpoint starts a new namespace (C2, vector 42). - A replacement or deletion notice follows the prior delivery's audience — the agent that holds or read it — never a context fresh routing would pick (C3, §3.2). - Content shape is validated before conversion on both lanes, so a malformed coalesced item fails alone (G1); the default publish channel moves only once a coalesced item is accepted (G2); push/render carries the declaring feature set for channel-scoped batches (G3). - A deferred channel-scoped push spawns the conversation fork it needs and its wake carries the channel and counterparty (G4, G5). - A placement in the deferred queue is found again after the flush stored it, by its durable write id in the unread tail (G6); a materialized render that lands in a busy agent's deferred queue stays withdrawable (G7); the watermark advances only after a successful compile, and a tune-out-diverted message is judged by the subconscious's watermark (G8). - Receipts are written synchronously at acceptance, bridging the snapshot throttle (G9); only a batch that was RENDERING skips the render after a restart (G10); absent identity fields never erase known ones (G11); quiesce coalescing keys by subject so a withdrawal cannot absorb an unrelated wake (G12); coalesced deliveries reach module onProcess (G13); a failed removal degrades to appending instead of acknowledging a replacement (G14); isRendering counts outstanding RPCs, cancelled or not (G15). Twelve regression tests added (46 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Review round 1 addressed at Codex
Greptile
One design note on 7/8: a coalesced delivery still bypasses the process queue (the coalescer needs the placement before it can answer the wire), so ordering relative to ordinary events already queued is not guaranteed; modules now see it, and the reply ordering on the wire is unaffected. |
|
Re-review of For the recovery probes, I explicitly synced Chronicle, copied its on-disk store at the specified boundary, allowed the fixture to shut down, then reopened the copied image. Thus clean shutdown's snapshot flush cannot mask the state a crash would recover. I did not rerun the full suite for this review. 1. An accepted deferred event can lose its work while retaining the receipt that prevents retryRecent receipt persistence · Receipt restoration Reproduction:
Observed: the retry returns the original successful receipt, but there are 0 pending batches, 0 render calls, 0 model calls, and no fallback content. The receipt record preserves only the result and birth history; Required: acknowledgement must cover enough recoverable state to deliver the accepted work or its fallback, together with its receipt. Persisting the receipt alone makes loss permanent by suppressing redelivery. Also handle persistence errors rather than acknowledging them as success. 2. Old-binding deferred notices are sent to the newly configured endpointAuthorization check · Render dispatch Reproduction:
Observed: A receives no render; B receives The new binding namespaces correctly separate new admissions, but dispatch still resolves the current connection by Required: verify binding identity before render dispatch and publication, including recovery paths. Preserve, conservatively settle, or reject old-binding work according to its original authority; never send its private notices to the replacement endpoint. 3. Plain → deferred replacement still changes the audienceReproduction:
Observed: receipt is
Required: carry the original eligible audience through plain/deferred transitions, subsequent notices, rendering, fallback, and recovery. The 4. Recovery can reissue a render that already startedRender start and persistence timing Reproduction:
Observed: the server receives The Required: make the render-start boundary recoverable before issuing the RPC, or conservatively use fallback when recovery cannot prove the render never started. Rendering untouched pending work after restart is optional; avoiding replay of a potentially executed render is the guarantee that matters. All four additional probes reproduced the outcomes above on this exact head. I would keep these as merge blockers, especially the cross-binding notice disclosure and acknowledged-work loss. AI-assisted review by Codex (GPT-6). |
… authority Address the re-review of d86afc1 (four reproduced failures): - A deferred acceptance is durable before it is acknowledged: the receipt record carries the accepted batch, so a crash before the snapshot flush restores the work (as its fallback) together with the receipt that suppresses its retry. A persistence failure fails the acceptance instead of acknowledging it. - Authorization compares the occurrence's binding with the server id's current binding: work admitted under a former endpoint never acquires the replacement peer's authority and its private notice data is never sent to that peer. - A deferred notice that displaces an unread plain occurrence, and every later notice in that batch, keeps the prior delivery's audience through wake, render, fallback and recovery. - The render-start boundary is persisted before push/render is issued, so a restart takes the fallback instead of asking a source-backed server to describe the same span twice. Four regression tests added (50 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 2 addressed at
|
|
Follow-up on 1. An acknowledged retraction can resurrect the withdrawn batchAcceptance record · Receipt restoration Reproduction:
Observed: E2 returns its original The E1 recent record includes Required: recovery must apply each accepted subject transition in order, including explicit batch removal/retraction, or commit a complete consistent subject state. Retrying an acknowledged withdrawal must never leave the withdrawn content scheduled for delivery. 2. A completed render is delivered again as fallback after recoveryPublication and in-memory completion Reproduction:
Observed: context contains both the original rendered content and a newly delivered fallback for the same occurrence. Only one render RPC ran; the duplicate effect is recovery publication. Persisting render-start before the RPC prevents repeating the RPC, but does not establish whether its result/fallback was already published. Recovery always treats the old rendering snapshot as pending fallback work, without checking a stable publication identity in context. Required: make context publication and completion recover consistently, or give publication a durable identity and deduplicate recovery against its existing context message. A fallback is appropriate for an uncertain, unpublished render; it must not become a second delivery after a completed one. Both probes reproduced these outcomes on this exact head. I would keep them as merge blockers. No full-suite rerun or production-provider exercise was performed for this follow-up. AI-assisted review by Codex (GPT-6). |
…uble publication Address the follow-up on c03b23b (two reproduced recovery failures): - The receipt record written at acceptance now states the subject's batch AFTER the acceptance, `null` included, so a retraction or replacement accepted inside the snapshot window replays in order and cannot be undone by a crash. A retried withdrawal never leaves the withdrawn work scheduled. - Coalesced deliveries carry their subject in message metadata, giving a publication a durable identity. On restore, a batch that was RENDERING whose result or fallback is already in a context is settled as consumed instead of being delivered a second time as fallback. Two regression tests added (52 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 3 addressed at
|
|
Re-review of One major recovery gap remains for plain occurrences: A consumed plain event can recover as
|
…ounts as read A receipt record's birth fact (`born: none`) is not proof the subject is still unread at recovery: the plain occurrence it acknowledged is durably in context and, with the watermark restarting at the head, will be in the next request. On restore, any recorded occurrence found published in a context settles its subject as `some`, so a later retraction appends the deletion notice instead of silently withdrawing. Same settlement as a snapshot occupant; now applied to the receipt log as well. One regression test added (53 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 4 addressed at Consumed plain event recovers as |
|
Re-review of One foundational durability blocker remains, demonstrated with actual child-process Synchronous
|
…nder start A synchronous state write is not durable until Chronicle fsyncs. The host now provides a group-committed durability barrier (one `store.sync()` per event-loop turn, shared by every waiter): an acceptance is acknowledged on the wire only after its receipt and accepted work are on disk, and push/render is issued only after the render-start snapshot is. A commit failure is a -32000 (not acknowledged; the in-memory receipt still deduplicates a retry while the process lives). Cost: ~26 ms per serial coalesced message including the fsync; batched under load. Ordinary, non-coalesced traffic is unchanged. The wire fixture moves to test/helpers so a child process can run it: two tests SIGKILL the child (no stop, no close, no sync) after the wire ack and after push/render was received, then reopen the store — receipt and work survive; the render is not issued again. 55 tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 5 addressed at Synchronous
A commit failure is a Cost: 300 serial coalesced channel messages now take ~26 ms each including the fsync (the earlier journal design measured 85 ms); concurrent arrivals share one fsync. Tests: the wire fixture moved to |
|
Re-review of 1. Retry after a failed commit acknowledges the same uncommitted receiptEarly duplicate return · Receipt retained before failed commit Reproduction in a child process using the wire fixture:
Observed: {
"firstResponse": {"error": {"code": -32000}},
"retryResponse": {"result": {"accepted": true, "coalesce": {"outcome": "first"}}},
"syncCalls": 1,
"recentAfterKill": null,
"snapshotAfterKill": null
}The duplicate path returns the in-memory receipt before attempting another durability barrier. Keeping the receipt is useful to prevent repeating effects, but it cannot count as committed merely because a retry found it. Required: distinguish pending/uncommitted receipts from committed ones; retry the commit (or return an error) before acknowledging such a duplicate. Do not replay delivery effects just to obtain a committed receipt. 2. Retraction during the render-start commit wait resurrects the cancelled batchRender-start await and restoration Reproduction:
Observed: the subject now has one pending batch again. A later assembly calls Retraction clears Required: after each await, verify that this exact rendering instance is still current and not cancelled. Restore work after a failed commit only if that operation still owns it; cancellation/replacement must never restore its old batch or overwrite newer work. 3. Endpoint reassignment during that wait leaks old notice data to the new bindingAuthorization before the wait and RPC after it Reproduction:
Observed: A receives zero renders; B receives The binding check runs before the asynchronous commit. Dispatch afterward resolves the current connection by server label, so the authorization decision is stale. The later response-time check cannot undo the disclosure that already happened. Required: recheck current binding and grants after the commit await and immediately before dispatch, and bind the dispatch to the validated connection. Apply the same discipline to other async boundaries that can change audience or authority. All three outcomes were reproduced on this exact head. For the two render interleavings I deliberately held the host commit promise; admission/reassignment used the existing wire/runtime paths. I would keep these as merge blockers. No full-suite rerun or production-provider exercise was performed in this follow-up. AI-assisted review by Codex (GPT-6). |
…rleaving paths Address the re-review of e2683b5 (three reproduced gaps): - A receipt is `committed` only after the durability barrier succeeds. A retry that finds an uncommitted receipt commits it before acknowledging (no effects are repeated); if the commit fails again, so does the acknowledgement. - After the render-start commit await, the assembling instance acts only if it still owns the subject and was not cancelled: a retraction or replacement that landed during the wait is final, and a failed commit restores the batch only for the instance that still owns it. - Authority — grant, registration and binding — is re-checked after that await and immediately before push/render is dispatched, so a server id reassigned to another endpoint during the wait never receives the batch's notices. Three regression tests added (58 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 6 addressed at
|
|
Re-review of Two major neighboring cases still reproduce: 1. Retraction during async audience lookup resurrects the selected batchBatch selection before Reproduction:
Observed: one The commit-wait ownership check is now correct, but this earlier await has no corresponding check. Required: revalidate the exact selected batch/current state after audience resolution, before clearing/freezing it. Guard the already-rendering path across its audience await too. The same ownership rule needs to cover all awaits in the transition, not only the durability barrier. This probe deliberately held the host audience promise; the retraction used the wire path. 2. The first new acceptance after restart overwrites earlier recovered work's only durable recordRestore loads recent records into the coalescer only · Acceptance rewrites the host's recent array Reproduction:
Observed: only E2 is recovered and delivered; E1's acknowledged fallback never reaches the request. Immediately after step 2, the durable
Required: retain the loaded recent records in the host's writable buffer, or durably checkpoint the recovered complete state before replacing/clearing those records. Verify two consecutive restart windows with an intervening acceptance, including preservation of earlier receipts within their retry window. Both failures reproduce on the exact current head. The recovery probe explicitly synced and copied the store at each boundary, then reopened those copies so graceful cleanup could not mask the windows. I would keep both as merge blockers. No full-suite rerun or production-provider test was performed for this follow-up. AI-assisted review by Codex (GPT-6). |
…vered records retained Address the re-review of b08aa77 (two reproduced gaps): - The audience lookup at assembly awaits (it may spawn a fork). The batch captured before it is frozen only if it is still the subject's pending batch afterwards and no other assembly froze it; a retraction or replacement that landed during the lookup is final. The already- rendering path re-checks its instance the same way. - After a restart, the recent receipt records restored from disk are kept in the host's writable buffer until the next snapshot flush checkpoints the recovered state, so the first new acceptance appends to them instead of replacing their only durable form. Two regression tests added (60 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 7 addressed at
|
…embly Rounds 6 and 7 of review were each an `await` in the assembly path with an admission landing inside it, guarded one at a time. Replace the guards with a single serial critical section in the coalescer: every state transition — an admission, the freezing of a batch for rendering (audience lookup, durability commit, authority re-check, RPC dispatch), and the settlement of a render's result (authority re-check, publication, state update) — runs under it, awaits included. Only the push/render RPC itself runs outside, so it remains cancellable; its late result is discarded under the lock. An admission therefore never observes a half-done assembly transition and an assembly never acts on a batch an admission has moved on from, by construction rather than by per-await checks. The identity checks are kept as cheap belt-and-braces. One new test for the remaining window this closes (a retraction arriving between the render result and its publication now waits and is `noted`); two hold-point tests adapted to send their retraction without awaiting it before the hold is released, since admission now queues behind the freeze. 61 tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Structural change at One critical section. The coalescer now has a single serial lock shared by every state transition:
Only the Tests: one added for the last window this closes — a retraction arriving between the render result and its publication now waits for the settlement and comes back |
|
Review of One major liveness failure still reproduces: A transient render-start commit failure removes the batch's only wake without scheduling recoveryWake cancellation and failed-freeze recovery Reproduction with the wire fixture:
Observed: {
"syncCalls": 2,
"pendingBatches": 1,
"pendingWakes": 0,
"modelCalls": 1,
"renderCalls": 0,
"fallbackDelivered": false
}The single model call did not contain the admitted fallback. The retry returns the original successful receipt, with no new wake or commit attempt, and there is no second model call. The batch remains pending indefinitely in an otherwise idle host, despite being authorized, wake-eligible, and already acknowledged. An unrelated wake or restart can rescue it, but the producer was already told it was accepted and should not need to manufacture another occurrence. The failure path restores Required: retain or schedule bounded recovery for an eligible batch whose freeze failed, with backoff and current-authority/policy checks. Once storage recovers, deliver its fallback without requiring unrelated traffic, while avoiding a tight retry loop during a persistent outage. Merely returning the cached receipt on producer retry must not leave acknowledged work asleep forever. This reproduces on the exact head above. I would keep it as a merge blocker. No full-suite rerun or production-provider test was performed for this follow-up. AI-assisted review by Codex (GPT-6). |
…-start commit A render-start commit that fails leaves the batch pending (fallback at the next assembly) but its wake had already been withdrawn, so on an idle host the acknowledged work slept until unrelated traffic arrived. The coalescer now schedules a backed-off re-wake per subject (base 1 s, doubling, capped at 60× base; cleared when the batch freezes, renders, or is withdrawn; authority re-checked when it fires), so a persistent storage outage is not a tight loop of model turns. A producer retry that hits the cached receipt also re-wakes a sleeping batch at once. Two regression tests added (63 in the suite). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 8 addressed at Failed render-start commit leaves the batch asleep — Thanks for confirming the critical section holds under your interleaving probes. |
|
Review of One issue remains in the persistent-outage path: Recovery never backs off because firing the timer deletes its attempt counter
Reproduction with the wire fixture:
Observed: {
"recoveryWakeAttempts": [1, 1, 1],
"modelCalls": 4,
"pendingBatches": 1,
"fallbackDelivered": false
}Thus an ongoing storage failure repeatedly starts model turns without the content that caused them, always after the base delay. At the default setting this waits one second after each failed attempt rather than increasing toward the intended 60-second cap. This is the persistent-outage model-turn loop the new recovery policy was meant to bound. Fix: preserve failure-attempt state after a timer fires, separately from whether a timer is armed. Reset it on successful recovery or disposal of the pending work, not merely on wake dispatch. Add a persistent-failure test that verifies successive attempts/delays increase and cap, followed by successful recovery resetting the series. The two additional probes (transient recovery and persistent failure) both reproduced the behavior described above. I would fix this before merging. No full-suite rerun or production-provider exercise was performed in this pass. AI-assisted review by Codex (GPT-6). |
…empt count The recovery timer deleted its map entry when it fired, so a freeze that failed again started the series over at the base delay: a persistent storage outage woke a model turn every base interval instead of backing off toward the cap. The attempt count now outlives the armed timer and is forgotten only when the pending work freezes successfully or is disposed of; a producer retry disarms the timer without resetting the count. The recovery-wake audit record carries the attempt and delay. One persistent-outage regression test added (64 in the suite): three consecutive failures wake at 1×, 2×, 4× the base delay; recovery delivers the fallback and clears the series. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 9 addressed at Recovery never backs off — the attempt count now outlives the armed timer: firing disarms the timer but keeps the entry, so a freeze that fails again schedules at 2× then 4× … up to 60× the base; the count is forgotten only when the batch freezes successfully or is withdrawn/replaced. A producer retry disarms the timer (it wakes immediately) without resetting the count. The |
|
Additional review of A secondary resident can consume a message that the primary's watermark still allows coalescing to erasePlacement resolves only its owning agent · Unread check uses that reader's watermark Reproduced through the PR's loopback wire fixture with two ordinary resident agents,
Observed: primary watermark remains sequence 0, secondary watermark is sequence 1. E2 returns A second probe used retraction for step 3: {
"outcome": "retracted",
"consumedOriginalStillInContext": false,
"deletionNoticeInContext": false
}The original was present in The coalescer remembers a single placement agent and asks only that agent whether the stored message is unread. Ordinary residents share the underlying messages, while consumption watermarks are per-agent. Reading through another resident advances its watermark but leaves the placement owner's watermark behind. Optional per-context consumption does not justify this: these agents are reading and mutating the same shared history, not independent copies. Required: consumption eligibility must account for every reader of the same underlying shared context/branch, or be tracked at that shared-context boundary. Preserve isolation for genuinely separate conversation contexts. Add two-resident regression coverage for both replacement and retraction after only the non-primary resident has consumed the occurrence. Both probes reproduced on this exact head. The existing build and 64 coalescing tests passed in the preceding verification of the same commit; I did not rerun the unchanged suite or the full suite for these two additional probes. I would treat this as a merge blocker because it violates the central never-rewrite-consumed-history rule. AI-assisted review by Codex (GPT-6). |
…r of the shared slot Ordinary residents share one un-namespaced message slot (their context managers are not isolated) and the subconscious merges it read-only; consumption watermarks are per agent. The unread check asked only the agent that received the placement, so a message another resident had already compiled could still be replaced or withdrawn — a rewrite of consumed history. A stored message is now unread only if NO reader of its slot has compiled past it or folded it: all residents plus the subconscious for the shared slot; the agent itself for an isolated fork or ephemeral slot; the subconscious alone for a tune-out-diverted message. The watermark is the chronicle head at compile time rather than the slot's last message, so a compile that reads merged-in messages counts them too. Two two-resident regression tests added (66 in the suite): only the secondary (or only the primary) reads the occurrence; an edit is then `appended` with the original kept in both contexts, and a retraction is `noted` with its notice. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 10 addressed at A secondary resident's read did not count — agreed on the root cause: ordinary residents' context managers are not isolated, so they share the un-namespaced Tests: with residents |
# Conflicts: # src/framework.ts
| const result = await this.handleCoalescedIncoming!(serverId, message, event); | ||
| if (result.accepted) markAccepted(); |
There was a problem hiding this comment.
Spurious closed-channel invitation When the first coalesced message arrives for a registered but closed channel, delivery runs before
markAccepted() opens it. The incoming handler therefore adds a closed-channel invitation, telling the agent to join a channel that the host accepts as open moments later.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/mcpl/channel-registry.ts
Line: 992-993
Comment:
**Spurious closed-channel invitation** When the first coalesced message arrives for a registered but closed channel, delivery runs before `markAccepted()` opens it. The incoming handler therefore adds a closed-channel invitation, telling the agent to join a channel that the host accepts as open moments later.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| this.host.recordReceipt?.({ | ||
| key: receiptKey, result: entry.result, at: entry.at, subject, ...(born ? { born } : {}), | ||
| batch: state.batch ? structuredClone({ latest: state.batch.latest as CoalescedOccurrence<unknown>, notices: state.batch.notices, dropped: state.batch.dropped }) : null, | ||
| }); |
There was a problem hiding this comment.
Channel identity lost after restart If the host crashes before the throttled snapshot flush, a plain channel occurrence is recovered from a recent receipt that has neither its platform identity nor its delivery agent. A channel-scoped push edit without origin IDs then uses its event ID instead of the original message and thread IDs. If routing has changed, the correction can also reach a different agent.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/mcpl/push-coalescer.ts
Line: 388-391
Comment:
**Channel identity lost after restart** If the host crashes before the throttled snapshot flush, a plain channel occurrence is recovered from a recent receipt that has neither its platform identity nor its delivery agent. A channel-scoped push edit without origin IDs then uses its event ID instead of the original message and thread IDs. If routing has changed, the correction can also reach a different agent.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| this.store.setStateJson(COALESCING_STATE_ID, snapshot()); | ||
| this.coalescingRecentReceipts = []; | ||
| this.store.setStateJson(COALESCING_RECENT_ID, []); |
There was a problem hiding this comment.
Empty render can reappear If a completed
push/render returns no content and the host stops after writing the cleared snapshot but before clearing recent receipts, recovery replays the older receipt as a pending batch. No message was published for recovery to detect, so a later turn can deliver the fallback even though the server rendered nothing.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/framework.ts
Line: 7465-7467
Comment:
**Empty render can reappear** If a completed `push/render` returns no content and the host stops after writing the cleared snapshot but before clearing recent receipts, recovery replays the older receipt as a pending batch. No message was published for recovery to detect, so a later turn can deliver the fallback even though the server rendered nothing.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Shelf-351 (room-203). Suppressed plain speech used to survive only as a count
in the turn receipt ("N segment(s) suppressed ... resend with a send tool"),
and explicit mode's bounced text lived in one in-memory latest-wins
clipboard: the suppression rule was right, the penalty was amnesia (a
Connectome resident's report). Now the exact words are kept as drafts until
the resident resends or dismisses them.
What becomes a draft (src/prose-drafts.ts, reasons):
- explicit-send: prose silenced by an explicit send in the same round or a
sticky earlier one (channel_publish, send_message, reply_message, send_dm,
say, whisper, and now a drafts resend), in locus and hybrid modes, mid-turn,
trailing and text-only (a guard retry carrying the silence), native and XML
tool modes. The turn tracks WHY it is silenced (silenceCause): an explicit
send holds prose as drafts; skip_reply and silent turns keep it private.
Hybrid prose is split into envelopes and a draft holds an envelope's body,
not its routing syntax, with "written for >>>target" as a note.
- no-destination: hybrid prose after a failed envelope (fail-closed until a
new valid target), formerly counted as suppressed.
- bounced: explicit/hybrid bounces. `{{unsent}}` keeps meaning the latest
bounce while that draft is open; delivering through it journals an attempt
on the draft; `>>skip_reply {{unsent}}` dismisses it; an unconfirmed latest
bounce is never substituted silently. Re-bouncing `{{unsent}}` alone keeps
the one draft.
Never drafted (counted in the receipt): skip_reply, a same-round private
think, silent turns, proseRouting=disabled narration.
Storage: a RecordJournal (typed Chronicle records that do not follow branch
switches), agreed in room-220 instead of the JSONL file first proposed to
Tessa/Ada/June; the contract is the same. Entries held / attempt / outcome /
dismissed / noticed, every one durable before its call returns; an attempt is
journaled BEFORE its request leaves AF, so an attempt without an outcome reads
as unconfirmed, never unsent. Undo, rollback, checkout and restarts never
change drafts; nothing is ever sent from them on its own. A failed journal
write reconciles (load) and reports the words it could not hold.
Notices: a private notice naming new drafts at the next tool boundary when
the live stream presents injections (native rounds; Membrane's
supportsInjectedMessages when it ships, room-220 #41701/#41870); otherwise
the turn-end [delivered] receipt names them ("held as drafts d-…"); drafts no
notice reached (crash, aborted turn) are named at the next fresh turn's
start. Notices name their resident: residents of one framework share a
message slot (#197). The bounce notice now goes to the bouncing resident
(forAgent) instead of the primary.
The drafts tool (with MCPL channels): list | read | resend | dismiss, scoped
to the caller; nullable unused fields (save_recent_image's pattern). resend
publishes verbatim, in order, through ChannelRegistry.publish, stops at the
first non-delivery, counts as an explicit send, marks the channel engaged,
returns a delivered draft's historical receipt instead of resending, needs
confirmDuplicate for an unconfirmed one, refuses dismissed drafts and a draft
already in flight, and honours a conversation agent's home binding. Results
are plain text blocks, so drafts read exactly as written.
Registry: deliverSpeech (routeSpeech's PublishOutcome form; routeSpeech wraps
it unchanged) and resolveDestination (exact resolution, shared ids refused).
Text-only routing goes segment by segment when the content holds a membrane
tool_attempt (Greta's 376, room-220 #43515).
Tests: test/prose-drafts.test.ts, 18 (store: order/ids/isolation, reopen
with an outcome-less attempt, branch independence, journal failure,
latest-bounce; end to end over the speech-route fixture: three-segment
reply held/named/injected/read/resent verbatim, XML mode, skip_reply and
private think, disabled mode, resident scoping, failed then unknown then
confirmed resend, June's rollback and restart cases and dismissal across a
rollback, crash catch-up, explicit bounce across restart via {{unsent}},
resend silencing adjacent prose, parallel resend, refusals). Receipt
expectations updated in present-while-acting and explicit-prose-routing; the
explicit stub gains deliverSpeech/resolveDestination; the fixture answers
tools/call at once.
Suite (--test-concurrency=4, load ~20-30): 1315 pass, 1 fail (#196 vector
34c's wall-clock bound) and mcpl-coalescing.test.js cut off by the 900 s
guard; that file alone afterwards: 66/66. code-execution alone: 36/36.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck drafts
Tessa-974's review of 351 (room-203 #44135, #44214; probe
/tmp/tessa-974-draft-state-probe.mjs):
1. Uncertainty was cleared by a later failure: unknown, then a confirmed
retry that failed, read `held`, and a third resend went out without
confirmation. A draft is now `unconfirmed` while ANY attempt has an
unknown or unrecorded outcome and none was confirmed; uncertainAttempt()
names that attempt, and refusals, the list line and a failed retry's
receipt cite it (destination, time, reason) rather than "the last one".
2. The up-front batch check went stale across an await: a batch [A,B]
waiting on A let a concurrent resend of B go out, then published B again;
a dismissal of B meanwhile was ignored. A resend now claims every draft of
its batch before its first await (a concurrent resend or {{unsent}} of
any of them is refused while the batch owns it), and re-reads each draft
from the current projection at its own dispatch boundary, so a dismissal
or delivery since the batch began stops it there (also after a journal
reconcile replaced the projection).
3. The crash catch-up said "They were not sent" of an unconfirmed draft. It
and the list now render each draft's actual state: "held (not sent)",
"held (every attempt failed: nothing was posted)" or "UNCONFIRMED — an
attempt to … may have been posted".
4. bounceProse expanded `{{unsent}}` from an unconfirmed (or in-flight)
latest bounce into a fresh held draft when the envelope's target failed,
which then resent without confirmation. Such a draft is never copied: it
stays held as itself, only the resident's other words become the new
draft, and the bounce notice says so.
Privacy boundary, agreed with Tessa (#44155, #44214): drafts are never
published except by their resident's explicit resend (or {{unsent}}), only
that resident can list or act on them, and notices go into the resident's
own history, which residents sharing one message slot (#197) already share.
The tool description, notices, module doc, explicit-routing doc and
changelog now say that instead of "private".
Tests (prose-drafts.test.ts, 22): unknown then failed stays unconfirmed and
refuses an unconfirmed third resend; the catch-up renders held and
unconfirmed drafts truthfully; an at-risk bounce is not copied into a new
draft and {{unsent}} then means only the fresh words; and, through the real
drafts handler with publish completions under the test's control, a
concurrent resend of a queued draft is refused with B published once, and a
draft dismissed while the batch waits never reaches publish.
Suite (--test-concurrency=4, load ~7-12): 1329/1330 (#196 vector 34c's
wall-clock bound at 30.7 s) + code-execution 36/36. Composition with
Greta-1013's 376 companion (6ff476d, membrane#101 pack) merged cleanly:
30/30 incl. her xml-tool-call-notices and a composed text-only
tool_attempt split check (room-220 #44367).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Implements RFC-006 event coalescing (mcpl PR #5, revision 7,
9a319e2) with content kept in context: no staging area, no journal, no subject cap.A plain coalesced occurrence is delivered on arrival through the ordinary channel/push path, exactly as today; the coalescer only remembers where it landed. A later occurrence of the same subject removes that message while it is still unread — above the agent's consumed watermark (advanced at every compile of a model request), on the current branch, not a sharded message, and not folded into a summary by compression — and delivers the replacement as a fresh message; once read, it appends. Retraction removes unread content and appends the notice only when some version was read or history is unknown.
initialmarks the birth of a subject so a never-read create → edit → delete leaves no trace, including after restart or eviction. Deferred batches (push/render) are the only thing held outside context, because a notice has no model-visible content until rendered; they render at the turn's assembly boundary (not on a context-budget restart), bounded by the connection's 5 s timeout, falling back to the admitted content.On any doubt — restart, branch switch (undo), eviction, unknown history — the host appends, which is RFC-006 principle 4 and what today's host does.
Supersedes #196 (journal design). The five findings from that review are regression tests here.
Changes
src/mcpl/push-coalescer.ts— the state machine (subjects, receipts with a 1 h retry window, deferred batches, render cancellation rules 1–7), pure bookkeeping behind aCoalescerHostinterface.src/agent.ts— per-agent consumed watermark ({branch, sequence}), advanced incompileContext/compileWithInjections, set at agent creation.src/framework.ts— host implementation: unread check (watermark + summary coverage viagetSummariesInRange/getSummary), removal (stored message or deferred-queue entry), delivery throughhandleMcplChannelIncoming/handleMcplPushEvent(which now report placement and carrycoalescingSubjecton the wakes they queue), channel-scoped pushes delivered as messages of their channel (forks, tune-out, reply locus), wake withdrawal, assembly at turn start, throttledmcpl/coalescingsnapshot (receipts + subject history),-32600forinference/requestduring a render.src/mcpl/channel-registry.ts/push-handler.ts— coalesced occurrences handed to the coalescer after ordinary admission with the event the ordinary path would have queued;eventIdrequired withcoalesce; per-messagecoalesce_invalid. Channel registration lifetime is unchanged (RFC-006 rev 7 §3.2).src/mcpl/server-connection.ts—push/renderwith its own timeout (from feat(mcpl): journal channel and push event coalescing #196).src/mcpl/types.ts—coalesce(withinitial),CoalesceResult,eventCoalescingcapability withretryWindowMs.mcpl:coalescing,message:removed).Tests
test/mcpl-coalescing.test.ts— 34 tests over a real loopback WebSocket MCPL server and the mock provider, named by RFC-006 §14 vector: plain (1–7, 9, 35, 38), channel scope and mixed lanes (8, 10, 13–15, 33, 43), wake treatment, deferred (16–27, incl. 27a/27c/27i), recovery (39–41, 47), and the #196 findings:initial→retracted; without it →notedPlus: replacement while a turn is alive edits the deferred queue; a message folded by the real
AutobiographicalStrategyis no longer replaceable (appends).Full suite: see the CI run; locally
node --test --test-concurrency=4 dist/test/*.test.js(without--test-force-exit) matches upstream5498805except for the tests added here.Not done / known gaps
skipwithdraws itspendingRequestswake, but a debounce timer already armed by the gate for the prior occurrence still fires (it carries no content, so nothing withdrawn is shown).onProcessdispatch is skipped for coalesced deliveries (they bypass the event queue so placement is known synchronously); the built-in MCPL conversion, routing, tune-out and invitation paths all run.🤖 Generated with Claude Code