From 8454df05a0cdd87c563e9cad0b262f101cf493cd Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Wed, 14 Oct 2026 10:26:57 -0400 Subject: [PATCH 01/19] fix(streaming): the outgoing stream carries only what delivery publishes MCPL Spec 14.3 says a chunk stream must not contain text the host's delivery path would refuse to deliver, and that `complete` goes out on every exit path. ProseStreamRouter was fed every text delta as it arrived, while delivery decides a round's fate only at its tool-calls event. So a streaming server received, and discord-mcpl's voice spoke, a round that called skip_reply, a silencing send or a private-policy think; a round its stream never finished; hybrid text that a later stream of the turn's hold kept back or its pin sent elsewhere; and text after a mid-turn re-pin, streamed to the old locus. A stream that handed its turn to a budget restart or a guard retry never sent `complete`. Delivery now feeds the stream: - Every prose delivery a stream queues hands what it published to that stream once the publish is confirmed (SpeechPublished): locus mode's mid-turn, text-only and trailing publishes, hybrid's two in deliverHybridProse, and explicit's in deliverProseEnvelope. A publish that throws, times out or returns delivered: false streams nothing. routeSpeech keeps its signature. - streamSpeech sends the chunk and emits an `inference:speech` trace. A later publish to the same channel within the stream opens with a paragraph break, so a channel's chunks joined in index order are its text. - Each stream sends `complete` (and `inference:speech_complete`) for every channel it streamed to, once the deliveries it queued have run, on every exit path, a stream that hands its turn to a successor included. When the framework stops, each stream still waiting completes at once, with what it has sent, and streams nothing after (pendingOutgoingCompletes). stop() does this once every stream's finally has run and before the channel registry forgets its channels (stopAll) and the MCPL connections close; after either, a complete finds no server. Closing a connection marks it closed before it rejects a pending publish, so a complete that waited for that publish went nowhere. What only the speculative stream needed goes: - ProseStreamRouter and its test file. - TurnRouting.destination and roundDestination, which only seeded and retargeted the router. - The new inference id and router reset after a membrane refusal retry: nothing of a refused attempt is delivered, so a frame keeps one inference id, and its lifecycle `started` and terminal match. - The text half of the tool-result guard's hold (it holds only `inference:tokens` traces now), and the tool-wrapper guard's exclusion from streaming: a contained wrapper is never published. - docs/hybrid-prose-routing and #314's changelog fragment no longer say a hybrid stream's live deltas start at the turn's locus: the hold and pin govern the stream too. - The texts say what the stream doesn't cover: inference:tokens is still the model's text as written (held while a guard batch is pending); mcpl:speech-routed also reports each speech publish, with no stream id, index or end; a wrapper the tool-wrapper guard contains never streams, while wrapper-shaped text it doesn't match, as in a turn with a real tool call, is ordinary prose and now streams; deliverProse, the direct entry only tests call, has no stream; the prose grammar is the parsers' definition, not the envelope splitters'. Tests: - outgoing-stream (new): each delivery path streams what it published and only once confirmed; skip_reply, a private think and a send stream nothing in every mode; a resumed hybrid stream's first round stays held under, or streams to the pin of, an earlier stream; a failed stream's unfinished round streams nothing and its retry streams what it delivers; a re-pin streams to the new locus; complete waits for the stream's deliveries; an unconfirmed or throwing publish streams nothing, on every delivery path, while a confirmed one beside it does; a stop() with a publish still out completes the stream before the registry forgets its channels and the teardown, whether teardown rejects that publish or confirms it too late, which then doesn't stream (the harness's registry stub forgets its channels on stopAll, as the real one does), and the same holds through a real ChannelRegistry and a server it owns, from a reader's probe; the speech traces mirror the stream with or without a streaming server. - explicit-prose-routing, tool-result-guard and tool-wrapper-prose-guard-integration: their stream checks follow delivery. #314's continuing-turn tests (an error retry, a capacity- deferred compile, a re-inference) check in every mode that the later stream streams what delivery publishes, where it publishes it, rather than that an explicit stream starts at the turn's destination; the retried-turn tests check every chunk. A budget restart's resumed reply streams only once its own publish is confirmed, after the earlier posts, and the test checks its premise: the restart writes while the first post is held and the second still queued. Co-Authored-By: Claude Opus 5.5 --- .../continuing-starts-keep-routing.fixed.md | 3 - changelog.d/hold-round-speech.added.md | 12 + changelog.d/hold-round-speech.changed.md | 19 + changelog.d/hold-round-speech.fixed.md | 36 ++ docs/hybrid-prose-routing.md | 4 +- docs/tool-result-guard.md | 13 +- docs/tool-wrapper-prose-guard.md | 2 +- src/framework.ts | 320 +++++----- src/mcpl/channel-registry.ts | 13 +- src/mcpl/prose-grammar.ts | 9 +- src/mcpl/prose-stream-router.ts | 268 -------- src/types/trace.ts | 42 ++ test/explicit-prose-routing.test.ts | 125 ++-- test/outgoing-stream.test.ts | 596 ++++++++++++++++++ test/present-while-acting.test.ts | 10 +- test/prose-stream-router.test.ts | 197 ------ test/tool-result-guard.test.ts | 100 +-- ...ol-wrapper-prose-guard-integration.test.ts | 62 +- 18 files changed, 1020 insertions(+), 811 deletions(-) create mode 100644 changelog.d/hold-round-speech.added.md create mode 100644 changelog.d/hold-round-speech.changed.md create mode 100644 changelog.d/hold-round-speech.fixed.md delete mode 100644 src/mcpl/prose-stream-router.ts create mode 100644 test/outgoing-stream.test.ts delete mode 100644 test/prose-stream-router.test.ts diff --git a/changelog.d/continuing-starts-keep-routing.fixed.md b/changelog.d/continuing-starts-keep-routing.fixed.md index e0b995ab8..14114d75b 100644 --- a/changelog.d/continuing-starts-keep-routing.fixed.md +++ b/changelog.d/continuing-starts-keep-routing.fixed.md @@ -11,9 +11,6 @@ - the turn's locus was resolved again: a retry lost a mid-turn addressed re-pin, a re-inference took the locus of whichever wake started it, and a held turn's later compile let a newer ambient message move it. - - A hybrid stream's live outgoing deltas still start at the turn's locus, - as before. - A message that no tool boundary weighed now moves the turn's locus at the turn's next start, by the rule a mid-turn message follows: a conversational message that addressed the agent, or that follows up in a diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md new file mode 100644 index 000000000..63f8eda45 --- /dev/null +++ b/changelog.d/hold-round-speech.added.md @@ -0,0 +1,12 @@ +- `inference:speech` and `inference:speech_complete` traces carry the + outgoing stream to in-process consumers: each chunk as it is sent, with + its `channelId`, `inferenceId`, `index` and `text`, and each channel's + `complete`. They are emitted whether or not the channel's server takes + the stream, and carry only what delivery published, so a host relay that + voices them, such as a TTS bridge, never speaks text that `skip_reply`, a + send or a private `think` withheld, or a publish that failed. + `inference:tokens` is unchanged: it carries the model's text as it is + written, held, as before, while a tool-result guard's batch is pending. + `mcpl:speech-routed` also reports each speech publish, the `sleep` tool's + announcement among them, with no stream id, index or end, so a relay + that voices speech takes one of the two. diff --git a/changelog.d/hold-round-speech.changed.md b/changelog.d/hold-round-speech.changed.md new file mode 100644 index 000000000..d1dc82a94 --- /dev/null +++ b/changelog.d/hold-round-speech.changed.md @@ -0,0 +1,19 @@ +- **MCPL servers that take the outgoing stream (`channels.streaming`):** + chunks now arrive once each publish is confirmed, one per published + message, rather than token by token while the model writes. A round + reaches a voice server once the model has finished writing it, its + routing is decided and its publish is confirmed, after any earlier post + of the agent's still in flight. A later message to the same channel + within one stream opens with a paragraph break (`\n\n`), so the + channel's chunks joined in `index` order are its text, which `complete` + carries, with its messages kept apart. A server that read chunks as a + sign the agent is composing, such as eidoverse-worlds' typing dots, now + sees them only as a reply is published; `channels/typing`, which the + host sends for the whole turn to a server granted `channels.typing`, is + that signal. +- Agents with `toolWrapperProseGuard` now stream their delivered prose. The + stream was off for them because it ran ahead of the guard's + classification; it now carries only prose that delivery published, and a + wrapper the guard contains is never published. Wrapper-shaped text the + guard doesn't match, as in a turn that has made a real tool call, was + already posted as ordinary prose, and now streams too. diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md new file mode 100644 index 000000000..a23a5e5a4 --- /dev/null +++ b/changelog.d/hold-round-speech.fixed.md @@ -0,0 +1,36 @@ +- The MCPL outgoing stream (`channels/outgoing/chunk` and `complete`) no + longer carries text that delivery withholds or sends elsewhere. It + streamed each round's text as the model wrote it, but whether a round is + delivered is decided only once its tool calls are known. So a streaming + server received, and discord-mcpl's voice spoke: + - a round that called `skip_reply`, a silencing send such as + `send_message` or `channel_publish`, or a `think` under + `same_round_think_text_policy: private`; + - a round its stream never finished, such as one cut off by a stream + error; + - in hybrid mode, in a later stream of the turn, such as a budget + restart's, a framework retry's or a re-inference's, text the turn's + hold kept back or its pin sent to another channel; + - in locus and hybrid mode, after a mid-turn re-pin, text bound for the + new locus, which streamed to the old one. + + The stream is now the prose that delivery published: each publish's + text, to its channel, sent once the publish is confirmed, a later one to + the same channel opening with a paragraph break. Tool publications, such + as `send_message` or the `sleep` tool's announcement, aren't part of it, + as before. A publish that fails, times out or isn't confirmed is never + streamed. A `{{unsent}}` resend streams the retained text it sends, + rather than the token. +- Every stream now sends `channels/outgoing/complete` for each channel it + streamed to, on every exit path, as MCPL Spec 14.3 asks: once its + deliveries have run, or, when the host stops, at once, with what it has + sent. A stream that handed its turn to a budget restart or a + tool-result-guard retry never sent it, so a voice server never learned + that stream's utterance had ended. +- A membrane refusal retry no longer moves the outgoing stream to a new + inference id. Nothing of the refused attempt was delivered, so none of it + streamed; the rounds delivered before the refusal stay in the stream, and + its `complete` carries them. The stream's `inference/lifecycle` `started` + and terminal now carry that one id too. The retry re-minted the id + between them, so a server matching them by `inferenceId` saw the first + inference never end, and a terminal for an id that never started. diff --git a/docs/hybrid-prose-routing.md b/docs/hybrid-prose-routing.md index 14696c14d..7d0defaff 100644 --- a/docs/hybrid-prose-routing.md +++ b/docs/hybrid-prose-routing.md @@ -30,8 +30,8 @@ names a destination that resolves; if that envelope's body then isn't delivered, the hold returns. The sticky destination and this hold last for the whole logical turn, across its later streams, such as a framework retry's, a budget restart's or the compile a provider-capacity hold defers -the turn to. They govern publication: a stream's live outgoing deltas still -start at the turn's locus. +the turn to. They govern publication, and with it the outgoing stream, which +carries only what delivery publishes, where delivery publishes it. This is publication routing only. It creates no privacy meaning for `` or other prose tags, and grants no channel authority beyond the existing diff --git a/docs/tool-result-guard.md b/docs/tool-result-guard.md index f463c7ba9..112748625 100644 --- a/docs/tool-result-guard.md +++ b/docs/tool-result-guard.md @@ -64,10 +64,11 @@ wire cost (chars/4 plus a flat per-image cost) in `reserveForResponse`, so substituting them for the placeholders cannot push a restart's request past its budget. -While a submitted batch is pending (or a recovery is running), streamed text -and its `inference:tokens` trace events are held at one publication +While a submitted batch is pending (or a recovery is running), the +round's `inference:tokens` trace events are held at one publication boundary; they are released in order on a clean round and discarded on a -guard refusal. +guard refusal. The outgoing stream needs no hold: it carries only what +delivery publishes, and a refused attempt publishes nothing. Normal successful rounds admit the preceding results to memory. This works for framework yielding streams (including ephemeral agents and context-budget @@ -265,9 +266,9 @@ retroactive and does not rewrite results already accepted into memory. - **Token stats.** The context manager's token-stats cache is not write-through on edit, so a stats read taken while a batch was pending can keep pricing that message as the placeholder. -- **Held stream text on abort.** Text held while a submitted batch is - pending is dropped (not previewed) if the stream aborts or errors before - the round settles. +- **Held token traces on abort.** `inference:tokens` traces held while a + submitted batch is pending are dropped (not previewed) if the stream + aborts or errors before the round settles. - **Withheld originals in the workspace.** Every reader of the mount can read a withheld original's file, and nothing removes it; delete it as any workspace file. diff --git a/docs/tool-wrapper-prose-guard.md b/docs/tool-wrapper-prose-guard.md index 65d395839..00d601eee 100644 --- a/docs/tool-wrapper-prose-guard.md +++ b/docs/tool-wrapper-prose-guard.md @@ -4,7 +4,7 @@ The guard matches only when the entire visible prose is one wrapper naming a tool registered on that exact inference. Signed thinking may precede the visible wrapper. It does not parse or execute the wrapper body. -Guarded turns buffer prose until completion so outgoing preview/voice surfaces cannot expose a wrapper before classification. Ordinary non-wrapper prose is then delivered normally at turn completion. +The outgoing preview/voice stream carries only prose that delivery has published, so a wrapper the guard contains never reaches it. Prose the guard doesn't match, wrapper-shaped text in a turn that has made a real tool call among it, is ordinary prose: delivery posts it, and it streams once its publish is confirmed. On a match, Agent Framework: diff --git a/src/framework.ts b/src/framework.ts index 013277a59..574f9d7c3 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -83,7 +83,6 @@ import { import type { ChannelIncomingMessage, ChannelIncomingMessageResult, McplContentBlock, PushEventResult } from './mcpl/types.js'; import { isSilentHeartbeatMarker, isVisiblyEmptyContent } from './mcpl/visible-content.js'; import { parseProsePrefix, parseHybridProsePrefix } from './mcpl/prose-grammar.js'; -import { ProseStreamRouter } from './mcpl/prose-stream-router.js'; import { detectKnownToolWrapperProse } from './tool-wrapper-prose-guard.js'; import { InferenceRouter } from './mcpl/inference-router.js'; import { ChannelRegistry, type ChannelToolOrigin } from './mcpl/channel-registry.js'; @@ -276,9 +275,9 @@ const isAddressedMessage = ( _metadata?: Record, ): boolean => Array.isArray(tags) && tags.includes('chat:addressed'); -// parseProsePrefix moved to ./mcpl/prose-grammar.ts — ONE grammar shared with -// the outgoing-stream router (Spec 14.3), so streamed chunks and delivered -// envelopes can never disagree on what is a prefix. +// parseProsePrefix lives in ./mcpl/prose-grammar.ts. The outgoing stream +// (Spec 14.3) needs no grammar of its own: it carries what delivery +// published. /** One-time primer appended when an agent's proseRouting mode changes. */ function proseModePrimer(mode: 'explicit' | 'hybrid' | 'locus' | 'disabled'): string { @@ -576,12 +575,6 @@ interface TurnRouting { engaged: Set; /** The sticky destination, set by each `>>` (hybrid: `>>>`) prefix that resolves. */ pin?: string; - /** - * Explicit mode: the destination the turn's delivered rounds have set, - * decided at each round boundary (roundDestination), where each of the - * turn's streams starts. - */ - destination: string | null; /** * Hybrid: held back after `>>>skip_reply`, or an envelope that is malformed, * doesn't resolve or isn't delivered, until a later `>>>` names a target. @@ -624,16 +617,27 @@ interface Arrival { function freshTurnRouting(): TurnRouting { return { - engaged: new Set(), destination: null, hybridHold: false, + engaged: new Set(), hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; } +/** + * What a prose delivery tells its stream once a publish is confirmed: the + * channel and the text it published. The stream that queued the delivery + * sends them on as its outgoing stream (MCPL Spec 14.3), so the stream + * carries only what delivery published: nothing it withholds, and nothing + * whose publish failed or wasn't confirmed. + */ +type SpeechPublished = (channelId: string, text: string) => void; + /** How a round's prose is delivered: kept private, and under which turn's routing. */ interface ProseRound { keepPrivate?: boolean; /** The routing state of the turn that queued the round. */ routing: TurnRouting; + /** The outgoing stream of the stream that queued the round. */ + onPublished?: SpeechPublished; } function isTurnContinuation(reason: string): boolean { @@ -1681,13 +1685,13 @@ export class AgentFramework { /** * Each agent's routing state for its current logical turn (TurnRouting): * its trigger channel, locus and engaged channels, the sticky destination, - * the turn's destination, hybrid's hold, a ` !` continuation and receipt - * tallies. The start that opens a turn makes a - * new object instead of clearing the old one, and every later stream of - * the turn shares it (openLogical). A queued delivery holds the object of - * the turn that queued it, so prose a turn leaves queued when it ends (a - * skip_reply ends it before its posts have run) still follows that turn's - * destination, and its late writes can't reach the next turn. + * hybrid's hold, a ` !` continuation and receipt tallies. The start that + * opens a turn makes a new object instead of clearing the old one, and + * every later stream of the turn shares it (openLogical). A queued delivery + * holds the object of the turn that queued it, so prose a turn leaves + * queued when it ends (a skip_reply ends it before its posts have run) + * still follows that turn's routing, and its late writes can't reach the + * next turn. */ private turnRouting: Map = new Map(); /** @@ -1696,6 +1700,12 @@ export class AgentFramework { * posts go out in the order they were written, across streams and turns. */ private proseSpeechChains: Map> = new Map(); + /** + * Each ended stream whose `channels/outgoing/complete` still waits for the + * deliveries it queued (driveStream's finally). stop() sends them at once, + * before it clears the channel registry and closes the MCPL connections. + */ + private pendingOutgoingCompletes: Set<() => void> = new Set(); /** * Each agent's open logical turn: its routing state and receipt tallies * (TurnRouting). It stays open from the start that opens it until the @@ -2775,7 +2785,16 @@ export class AgentFramework { operatorOperationsTimedOut = error; } - // Stop typing indicators + // A stream's `complete` waits for the deliveries it queued, and the + // teardown below would leave it nowhere to go: the registry forgets its + // channels, and closing an MCPL connection marks it closed before it + // rejects a publish still out. Every stream has ended by now (its finally + // ran before activeStreams settled), so each still waiting sends its + // `complete` now, with what it has streamed, and streams nothing after + // (pendingOutgoingCompletes). + for (const complete of [...this.pendingOutgoingCompletes]) complete(); + + // Stop typing indicators, and forget the registry's channels this.channelRegistry?.stopAll(); // Dispose EventGate (clear debounce timers) @@ -10838,12 +10857,13 @@ export class AgentFramework { } /** - * Explicit-prose-routing delivery gateway (docs/explicit-prose-routing.md). - * Every prose segment of an `explicit`-mode agent flows through here: - * parse the `>>` prefix, resolve the target, deliver — or bounce to the - * clipboard with a resend notice. Misdelivery is structurally impossible: - * nothing is ever sent to a destination the model did not name (directly - * or via the turn's sticky target). + * Explicit-prose-routing delivery gateway (docs/explicit-prose-routing.md), + * entered directly: parse the `>>` prefix, resolve the target, deliver, or + * bounce to the clipboard with a resend notice. Misdelivery is structurally + * impossible: nothing is ever sent to a destination the model did not name + * (directly or via the turn's sticky target). A stream's own deliveries go + * through deliverAdmittedProse with its outgoing stream (onPublished); + * this entry has no stream, and only tests call it. */ private async deliverProse(agent: Agent, rawText: string, round?: { keepPrivate?: boolean }): Promise { // The direct entry delivers under the live turn's routing. @@ -10855,9 +10875,9 @@ export class AgentFramework { /** * Explicit prose's envelopes, each `>>` target resolved now, once. A live - * round is admitted at its boundary: its turn's destination is decided from - * these resolutions (roundDestination), and its queued delivery posts to - * them, however the registry's names change before it runs. + * round is admitted as it ends, at its tool-calls event, and its queued + * delivery posts to these resolutions, however the registry's names change + * before it runs. */ private admitProse(rawText: string): AdmittedEnvelope[] { return splitExplicitEnvelopes(rawText).map((text) => { @@ -10885,20 +10905,6 @@ export class AgentFramework { } } - /** - * Where a delivered round leaves its turn's sticky destination, as the - * explicit gateway sets it once the round's envelopes run: each `>>` - * envelope whose target resolved becomes the destination. One that didn't - * resolve bounces and leaves it, as do `>>skip_reply` and unprefixed text. - */ - private roundDestination(current: string | null, envelopes: AdmittedEnvelope[]): string | null { - let destination = current; - for (const envelope of envelopes) { - if (envelope.resolved && 'channelId' in envelope.resolved) destination = envelope.resolved.channelId; - } - return destination; - } - /** Locus-preserving explicit publication. Source remains byte-identical in Chronicle. */ private async deliverHybridProse( agent: Agent, @@ -10906,6 +10912,8 @@ export class AgentFramework { locus: string | null, allowLocus: boolean, turnRouting: TurnRouting, + /** The outgoing stream of the stream that queued this delivery. */ + onPublished?: SpeechPublished, ): Promise { const routing = turnRouting; const lines = rawText.split('\n'); @@ -10955,6 +10963,7 @@ export class AgentFramework { if (outcome?.delivered) { this.proseBounceStreaks.delete(agent.name); if (usedClipboard) this.proseClipboards.delete(agent.name); + onPublished?.(outcome.channelId, body); } else { delete routing.pin; routing.hybridHold = true; @@ -10986,6 +10995,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry!.routeSpeech(agent.name, envelope, target); this.recordProseDelivery(routing, outcome); + if (outcome?.delivered) onPublished?.(outcome.channelId, envelope); } catch (err) { console.error('hybrid locus delivery failed:', err); } @@ -11065,6 +11075,7 @@ export class AgentFramework { if (result?.delivered) { this.proseBounceStreaks.delete(name); if (usedClipboard) this.proseClipboards.delete(name); + round.onPublished?.(result.channelId, body); } } catch (err) { console.error(`[prose] ${name}: delivery to ${targetChannel} failed:`, err); @@ -11462,9 +11473,9 @@ export class AgentFramework { // results the non-streaming fallback left (whichever wake the // scheduler starts it with), the later compile a provider-capacity // hold defers a turn to. It takes the turn's object: the sticky - // destination, the destination and the hold, the ` !` and receipt - // tallies, the trigger channel, the locus with any re-pin, and the - // engaged channels. A delivery still queued from an earlier stream + // destination and the hold, the ` !` and receipt tallies, the + // trigger channel, the locus with any re-pin, and the engaged + // channels. A delivery still queued from an earlier stream // runs ahead of this one's on the agent's chain, so what it sets // applies to this stream's prose, in the order the turn wrote them. // With no turn open, the start opens one. @@ -11891,10 +11902,51 @@ export class AgentFramework { // A stream another has replaced doesn't move the turn's latest link. if (agent.streamId === myStreamId) this.proseSpeechChains.set(agent.name, turnSpeechChain); }; + + // MCPL Spec 14.3 outgoing streaming: the prose this stream's deliveries + // publish, sent to the channel's server once each publish is confirmed + // (voice synthesis), and to in-process consumers as an `inference:speech` + // trace. Delivery feeds it, never the text as the model writes it, so it + // carries only what delivery published, where it published it. A + // round's text reaches it once the round's routing is decided and its + // publish is confirmed. Text that delivery withholds never does: a round + // a silencing tool, `skip_reply` or a private `think` keeps back, a hold, + // a bounce, or a round its stream never finished. Nor does a publish that + // fails or isn't confirmed. Emission to a server is gated in the registry + // (`channels.streaming`), and `channels/publish` stays the authoritative + // send. + const outgoingInferenceId = `inf_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; + let outgoingIndex = 0; + // Per channel, everything this stream has sent there, which its + // `complete` carries. + const outgoingText = new Map(); + // Set once the stream's `complete` has gone: no chunk follows it. + let outgoingCompleted = false; + const streamSpeech: SpeechPublished = (channelId, text) => { + if (outgoingCompleted) return; + // A later publish to the same channel is another message: a paragraph + // break keeps the two apart for a consumer that joins the chunks, so a + // voice doesn't run "Let me look.Found it." together. + const sent = outgoingText.get(channelId); + const delta = sent === undefined ? text : `\n\n${text}`; + outgoingText.set(channelId, (sent ?? '') + delta); + const index = outgoingIndex++; + this.channelRegistry?.sendOutgoingChunk(channelId, agent.name, outgoingInferenceId, index, delta); + this.emitTrace({ + type: 'inference:speech', + agentName: agent.name, + channelId, + inferenceId: outgoingInferenceId, + index, + text: delta, + }); + }; + const enqueueSpeech = (text: string, locus: string | null): void => { chainSpeech(async () => { const outcome = await this.channelRegistry!.routeSpeech(agent.name, text, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, text); }, 'mid-turn speech routing failed:'); }; @@ -11934,24 +11986,6 @@ export class AgentFramework { : resolveTurnLocus(); if (typingChannel) this.channelRegistry!.startTyping(typingChannel); - // MCPL Spec 14.3 outgoing streaming: route text deltas to their - // destination server AS THEY GENERATE (voice synthesis, live message - // rendering). Pure observer surface — suppression is fail-closed (nothing - // streams that delivery would refuse), emission is capability-gated in - // the registry (`channels.streaming`), and delivery via deliverProse - // remains the sole authoritative send. - const newOutgoingInferenceId = (): string => - `inf_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; - // NOT const: a membrane refusal retry abandons everything streamed so - // far, and the surface must not concatenate two attempts. Re-minting the - // id makes the discarded attempt's chunk buffer orphaned (never - // finalized) instead of being appended to — see the 'retrying' case. - let outgoingInferenceId = newOutgoingInferenceId(); - // RFC-007: every inference id THIS stream mints, so the stream's end - // aborts only its own open tool calls — never a successor's. - const lifecycleInferenceIds = new Set([outgoingInferenceId]); - let outgoingIndex = 0; - // §10.5 inference/lifecycle: `started` now, exactly one terminal in the // finally below (the one block every exit path shares). Which terminal // is decided by the paths themselves: default completed, catch sets @@ -11965,55 +11999,17 @@ export class AgentFramework { turnIndex: 0, phase: 'started', }); - // The wrapper guard needs the complete response before it can distinguish - // an exact invocation-shaped body from ordinary XML/prose. Buffer guarded - // turns until completion: otherwise speculative outgoing chunks could - // expose the wrapper before the fail-closed classifier runs. - const proseStream = this.channelRegistry && - turnProseRouting !== 'disabled' && - !agent.toolWrapperProseGuard - ? new ProseStreamRouter({ - mode: turnProseRouting === 'explicit' ? 'explicit' : turnProseRouting === 'hybrid' ? 'hybrid' : 'locus', - // Explicit text has no destination until a `>>` prefix names one, - // so a fresh turn's stream starts suppressed. A later stream of the - // turn (a resume, a retry, a re-inference, a capacity-deferred - // compile) starts at the destination the turn's delivered rounds - // have set: a private round's prefixes don't count, and the - // gateway's queued deliveries needn't have run. The trigger channel - // is only where explicit mode's typing indicator shows. - initialTarget: turnProseRouting === 'explicit' - ? streamRouting.destination - : typingChannel, - resolve: (spec) => { - const r = this.channelRegistry!.resolveProseTarget(spec); - return 'channelId' in r ? r.channelId : null; - }, - }) - : null; - const emitOutgoing = (deltas: { channelId: string; delta: string }[]): void => { - for (const rd of deltas) { - this.channelRegistry!.sendOutgoingChunk( - rd.channelId, agent.name, outgoingInferenceId, outgoingIndex++, rd.delta, - ); - } - }; - // Tool-result-guard publication boundary. While a SUBMITTED batch is // pending (or a guard recovery is running), a refusal can still abandon - // this physical round, so its streamed text must not reach any surface - // yet — neither the prose router nor the inference:tokens trace (TTS and - // other trace consumers voice it). Hold both; release in order on a clean - // round (tool-calls / non-guard completion), discard on a guard refusal. - let guardHeld: Array<{ trace: Parameters[0]; text?: string }> = []; - // The text block the stream's last text came from, within this round. - let lastTextBlock: number | undefined; + // this physical round, so its `inference:tokens` traces must not reach + // trace consumers yet. Hold them; release in order on a clean round + // (tool-calls / non-guard completion), discard on a guard refusal. The + // outgoing stream needs no hold: it only carries what delivery publishes. + let guardHeld: Array[0]> = []; const releaseGuardHeld = (): void => { const held = guardHeld; guardHeld = []; - for (const item of held) { - this.emitTrace(item.trace); - if (proseStream && item.text !== undefined) emitOutgoing(proseStream.feed(item.text)); - } + for (const trace of held) this.emitTrace(trace); }; const discardGuardHeld = (): void => { guardHeld = []; }; @@ -12051,22 +12047,10 @@ export class AgentFramework { // turn's prose is bound for, without re-deriving routing. channelId: typingChannel ?? undefined, }; - // Delivery joins a round's text blocks with a newline - // (splitProseSegments), so a later block starts a new line for - // the stream too. - let text = event.meta.type === 'text' ? event.content : undefined; - if (text !== undefined) { - const block = event.meta.blockIndex; - if (block !== undefined) { - if (lastTextBlock !== undefined && block !== lastTextBlock) text = '\n' + text; - lastTextBlock = block; - } - } if (agent.toolResultGuard.hasSubmittedPending || agent.toolResultGuard.recovering) { - guardHeld.push({ trace, text }); + guardHeld.push(trace); } else { this.emitTrace(trace); - if (proseStream && text !== undefined) emitOutgoing(proseStream.feed(text)); } break; } @@ -12074,31 +12058,18 @@ export class AgentFramework { case 'retrying': { // Membrane is re-issuing after a content-policy refusal. Per the // RetryingEvent contract we must DISCARD everything this call has - // emitted: the tokens above were already streamed to the surface - // as outgoing chunks, and appending a second attempt to them - // would show the human two half-answers spliced together. - // - // Re-mint the outgoing id so the partial buffer is orphaned - // rather than continued (delivery is authoritative via - // deliverProse and has not happened yet — nothing has been - // *sent*, only previewed), and reset the prose router so its - // destination bookkeeping starts clean for the new attempt. + // emitted. Nothing of the attempt has been delivered, so the + // outgoing stream holds none of it: the stream only carries what + // delivery publishes, and rounds this stream delivered before the + // refusal stand, under the same inference id. A guarded round's + // held token traces go. console.error( `[refusal-retry] agent=${agent.name} membrane retry ` + `${event.attempt}/${event.maxAttempts}` + (event.category ? ` category=${event.category}` : '') + ' — discarding the refused attempt', ); - outgoingInferenceId = newOutgoingInferenceId(); - lifecycleInferenceIds.add(outgoingInferenceId); - outgoingIndex = 0; discardGuardHeld(); - proseStream?.reset(); - lastTextBlock = undefined; - // Explicit mode's retried attempt starts where the turn's - // delivered rounds have set the destination, not at the stream's - // seed: rounds this stream delivered before the refusal stand. - if (turnProseRouting === 'explicit') proseStream?.retarget(streamRouting.destination); this.emitTrace({ type: 'inference:tokens', agentName: agent.name, @@ -12128,10 +12099,6 @@ export class AgentFramework { // preceding results before storing/dispatching this new round. agent.toolResultGuard.accept(); releaseGuardHeld(); - // The round's text ends here: delivery parses each round's text - // on its own, so the next round's text starts a new line. - if (proseStream) emitOutgoing(proseStream.endSegment()); - lastTextBlock = undefined; adoptInjectedRound(); hadToolCalls = true; this.recordLogicalTurnToolCalls(agent, myTurnToken ?? -1, event.calls.length); @@ -12249,7 +12216,7 @@ export class AgentFramework { // Queued under this turn's routing, whenever it posts. const routing = streamRouting; for (const seg of roundSegments) { - chainSpeech(() => this.deliverHybridProse(agent, seg, locus, true, routing), 'mid-turn hybrid prose delivery failed:'); + chainSpeech(() => this.deliverHybridProse(agent, seg, locus, true, routing, streamSpeech), 'mid-turn hybrid prose delivery failed:'); } } } else if (turnProseRouting === 'explicit') { @@ -12264,27 +12231,17 @@ export class AgentFramework { if (!hasSameRoundPrivateThink) { const keepPrivate = roundToolNames.some((n) => bareToolName(n) === 'skip_reply'); // The round is admitted now: its targets are resolved - // once, for the turn's destination and for its queued - // delivery alike. + // once, for its queued delivery. const admitted = roundSegments.map((seg) => this.admitProse(seg)); // Queued under this turn's routing, whenever it posts. const routing = streamRouting; - if (!keepPrivate) { - routing.destination = this.roundDestination(routing.destination, admitted.flat()); - } console.error( `[prose] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> ${roundSegments.length} segment(s) via prose gateway${keepPrivate ? ', kept private (skip_reply)' : ''}`, ); for (const envelopes of admitted) { - chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { keepPrivate, routing }), 'mid-turn prose delivery failed:'); + chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { keepPrivate, routing, onPublished: streamSpeech }), 'mid-turn prose delivery failed:'); } } - // The stream's next round starts where delivery sends - // unprefixed text: the turn's destination. The stream - // followed this round's prefixes as it was written, but a - // private round's, a >>skip_reply envelope's or a target - // that didn't resolve leave the destination as it was. - proseStream?.retarget(streamRouting.destination); } else if (turnSilenced) { console.error( `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> prose NOT routed (turn silenced)`, @@ -12331,9 +12288,8 @@ export class AgentFramework { const withheld = agent.toolResultGuard.withhold(category); // Nothing from the abandoned physical attempt is published or // persisted as assistant speech, including a terminal refusal - // after the single recovery retry. - proseStream?.reset(); - if (turnProseRouting === 'explicit') proseStream?.retarget(streamRouting.destination); + // after the single recovery retry, so the outgoing stream + // carries none of it either. if (withheld) { const usage = response.details?.usage ?? response.usage; const tokenUsage = usage ? { @@ -12761,14 +12717,14 @@ export class AgentFramework { const locus = resolveTurnLocus(); console.error(`[prose] ${agent.name}: text-only turn -> hybrid prose gateway`); try { - await this.deliverHybridProse(agent, speechText, locus, true, finalRouting); + await this.deliverHybridProse(agent, speechText, locus, true, finalRouting, streamSpeech); } catch (err) { console.error('text-only hybrid prose delivery failed:', err); } } else if (turnProseRouting === 'explicit') { console.error(`[prose] ${agent.name}: text-only turn -> prose gateway`); try { - await this.deliverAdmittedProse(agent, admittedSpeech!, { routing: finalRouting }); + await this.deliverAdmittedProse(agent, admittedSpeech!, { routing: finalRouting, onPublished: streamSpeech }); } catch (err) { console.error('text-only prose delivery failed:', err); } @@ -12786,6 +12742,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, speechText, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, speechText); } catch (err) { console.error('speech routing failed:', err); } @@ -12835,7 +12792,7 @@ export class AgentFramework { const locus = resolveTurnLocus(); for (const seg of segments) { try { - await this.deliverHybridProse(agent, seg, locus, true, finalRouting); + await this.deliverHybridProse(agent, seg, locus, true, finalRouting, streamSpeech); } catch (err) { console.error('trailing hybrid prose delivery failed:', err); } @@ -12848,7 +12805,7 @@ export class AgentFramework { ); for (const envelopes of admittedSegments) { try { - await this.deliverAdmittedProse(agent, envelopes, { routing: finalRouting }); + await this.deliverAdmittedProse(agent, envelopes, { routing: finalRouting, onPublished: streamSpeech }); } catch (err) { console.error('trailing prose delivery failed:', err); } @@ -12877,6 +12834,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, seg, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, seg); } catch (err) { console.error('speech routing failed:', err); } @@ -13334,9 +13292,9 @@ export class AgentFramework { // RFC-007 §7.3: a tool call still open when its turn's stream ends was // cancelled before a result — its terminal is `aborted`. (A normal turn // has none: the stream waits for every result before continuing.) - // Scoped to this stream's own inference ids: a budget-restart successor + // Scoped to this stream's own inference id: a budget-restart successor // may already be running for the same agent. - this.toolLifecycleEmitter?.abortOpen(agent.name, lifecycleInferenceIds); + this.toolLifecycleEmitter?.abortOpen(agent.name, [outgoingInferenceId]); this.hookOrchestrator?.emitLifecycle({ inferenceId: outgoingInferenceId, conversationId: agent.name, @@ -13378,15 +13336,33 @@ export class AgentFramework { // exhausted, abort) so it never sticks after the turn ends. if (frameReachedTerminal) this.channelRegistry?.stopTyping(); - // Spec 14.3: flush any held line-start text, then close each streamed - // channel with its final moderated content — the consumer's signal to - // finalize (end the TTS utterance, settle the rendered message). - if (frameReachedTerminal && proseStream) { - emitOutgoing(proseStream.finish()); - for (const [channelId, text] of proseStream.byChannel()) { - this.channelRegistry!.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); + // Spec 14.3: once the deliveries this stream queued have run, close + // each channel it streamed to with everything it sent there, the + // consumer's signal to finalize (end the TTS utterance, settle the + // rendered message). Every exit path does, a stream that hands its turn + // to a successor (a budget restart, a guard retry) included: the + // successor streams under its own inference id. Its deliveries can + // still be queued here, as when a tool ends an explicit-mode turn, so + // `complete` waits for them and no chunk follows it. A stop() sends it + // at once instead, with what the stream has sent, since stopping closes + // the connections a queued delivery waits on (pendingOutgoingCompletes). + const completeOutgoing = (): void => { + if (outgoingCompleted) return; + outgoingCompleted = true; + this.pendingOutgoingCompletes.delete(completeOutgoing); + for (const [channelId, text] of outgoingText) { + this.channelRegistry?.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); + this.emitTrace({ + type: 'inference:speech_complete', + agentName: agent.name, + channelId, + inferenceId: outgoingInferenceId, + text, + }); } - } + }; + this.pendingOutgoingCompletes.add(completeOutgoing); + void turnSpeechChain.then(completeOutgoing); this.frameworkCancelledStreams.delete(`${agent.name}:${myStreamId}`); if (ownsPhysicalStream) { this.activeStreams.delete(agent.name); diff --git a/src/mcpl/channel-registry.ts b/src/mcpl/channel-registry.ts index 082e6ed9b..cb7363211 100644 --- a/src/mcpl/channel-registry.ts +++ b/src/mcpl/channel-registry.ts @@ -2994,12 +2994,13 @@ export class ChannelRegistry { * process-global default). Public so a multi-segment caller can snapshot it * ONCE and pin every segment to it via routeSpeech's `overrideChannelId`. */ /** - * MCPL Spec 14.3 outgoing streaming: forward a moderated text delta to the - * server owning the channel, AS THE MODEL GENERATES. Emitted only when that - * server declared `channels.streaming` in its initialize capabilities — - * servers that never opted in (the whole existing fleet) receive nothing. - * Fire-and-forget and never throws: streaming is an observer surface; the - * authoritative delivery is the eventual channels/publish. + * MCPL Spec 14.3 outgoing streaming: forward a text delta to the server + * owning the channel. The framework sends each piece of prose once its + * publish is confirmed, so a stream carries only what was published. + * Emitted only when that server's grant has `channels.streaming` — + * servers that never opted in receive nothing. Fire-and-forget and never + * throws: streaming is an observer surface; the authoritative delivery is + * channels/publish. */ sendOutgoingChunk( channelId: string, diff --git a/src/mcpl/prose-grammar.ts b/src/mcpl/prose-grammar.ts index 557a5b2ce..9bac23ef3 100644 --- a/src/mcpl/prose-grammar.ts +++ b/src/mcpl/prose-grammar.ts @@ -1,9 +1,8 @@ /** - * Explicit-prose routing grammar — the SINGLE definition shared by the - * delivery path (framework.deliverProse*) and the outgoing-stream router - * (prose-stream-router.ts). Extracted from framework.ts verbatim so the two - * parsers cannot drift: a streamed chunk and its eventual delivery must agree - * on what is a prefix, what is a modifier, and what is body. + * Explicit-prose routing grammar: the prefix parsers the delivery path uses + * (parseProsePrefix, parseHybridProsePrefix). The envelope splitters in + * framework.ts test a line's `>>` or `>>>` start on their own. The outgoing + * stream needs no parser of its own: it carries what delivery publishes. * * Grammar (docs/explicit-prose-routing.md): * `>>target [!] [body…]` — target = channel spec or `skip_reply` diff --git a/src/mcpl/prose-stream-router.ts b/src/mcpl/prose-stream-router.ts deleted file mode 100644 index 308f554ff..000000000 --- a/src/mcpl/prose-stream-router.ts +++ /dev/null @@ -1,268 +0,0 @@ -/** - * Incremental prose router: turns a stream of text deltas into a stream of - * (channelId, delta) pairs AS THE MODEL GENERATES, so downstream surfaces - * (voice synthesis, streamed message rendering, future platforms) can act on - * speech before the turn completes. This is the emission half of MCPL Spec - * 14.3 (`channels/outgoing/chunk`) — the notification has existed in the - * wire types since the spec landed; this class is what finally produces it. - * - * Correctness contract: routing decisions here MUST agree with the delivery - * path (deliverProse/parseProsePrefix). Both use prose-grammar.ts; this - * router additionally holds back exactly as much text as needed to make each - * line's decision — the start of a line is buffered only until it either - * cannot be a `>>` prefix (flush as body) or its prefix token (+ optional - * ` !` modifier) completes (switch target, swallow the prefix). Body text - * after the decision point streams through with zero added latency. - * - * Suppression is fail-closed: text whose destination is unknown (explicit - * mode before the first `>>`, unresolvable/ambiguous targets, `>>skip_reply`) - * emits NOTHING — the delivery path will bounce/retain it; a stream consumer - * must never see text that delivery would refuse to send. - * - * Three modes, mirroring proseRouting: - * - explicit: parse `>>` prefixes. A fresh turn's stream starts suppressed; - * a later stream of the turn starts at the turn's destination, - * and each round's end returns the stream to it (retarget). - * - hybrid: parse `>>>` envelopes, with the turn's locus as the - * fallback. An envelope with no target, or `>>>skip_reply`, - * holds back what follows until a later `>>>` names a target. - * - locus: NO prefix parsing (locus mode's `>>` lines are plain text); - * everything streams to the turn's locus. - */ - -export interface RoutedDelta { - channelId: string; - delta: string; -} - -type State = 'line-start' | 'indent' | 'maybe-prefix' | 'prefix-gap' | 'prefix-token' | 'after-token' | 'body'; - -/** Whitespace as the delivery path reads it (`trimStart()`, `\s`, `\S`). */ -const isSpace = (ch: string): boolean => /\s/.test(ch); - -export class ProseStreamRouter { - private target: string | null; - private state: State = 'line-start'; - private held = ''; // text held while a line's decision is pending - private token = ''; // the >>spec token being accumulated - private sawBang = false; - private accumulated = new Map(); - - constructor( - private opts: { - /** explicit parses `>>`; hybrid parses `>>>` while retaining locus fallback. */ - mode: 'explicit' | 'hybrid' | 'locus'; - /** Initial destination: the turn's locus (locus and hybrid mode); in - * explicit mode, null on a fresh turn, or the turn's destination on a - * later stream of the turn. */ - initialTarget: string | null; - /** Resolve a `>>spec` to a channelId, or null (unresolved/ambiguous). */ - resolve: (spec: string) => string | null; - }, - ) { - this.target = opts.initialTarget; - if (opts.mode === 'locus') this.state = 'body'; - } - - /** Per-channel full text emitted so far (for outgoing/complete). */ - byChannel(): ReadonlyMap { - return this.accumulated; - } - - /** - * Point the stream at a destination decided outside it. Explicit mode does - * this at each round's end, with the turn's destination: a private round's - * prefixes, a `>>skip_reply` envelope and a target that didn't resolve all - * leave that destination as it was, though the stream followed them while - * the round was written. - */ - retarget(target: string | null): void { - this.target = target; - } - - /** - * Drop everything routed so far and return to the constructed state. - * - * For a membrane refusal retry (RetryingEvent): the abandoned attempt's - * prose must not survive into the new one, and `accumulated` feeds - * outgoing/complete — leaving it would finalize text the model never - * actually said in the attempt that stands. - */ - reset(): void { - this.target = this.opts.initialTarget; - this.state = this.opts.mode === 'locus' ? 'body' : 'line-start'; - this.held = ''; - this.token = ''; - this.sawBang = false; - this.accumulated = new Map(); - } - - feed(delta: string): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (const ch of delta) this.step(ch, out); - return coalesce(out); - } - - /** - * The end of a segment: one round's text, which delivery parses on its - * own. The pending line's decision is made now. Held line-start text can - * no longer become a prefix, so it is body. A prefix whose target is - * written but has no body takes effect, as delivery reads `>>#chan` with - * nothing after it. The next segment starts a new line. - */ - endSegment(): RoutedDelta[] { - const out: RoutedDelta[] = []; - if (this.state === 'indent') this.emit(this.held, out); - if (this.state === 'maybe-prefix') { - // Hybrid's `>>>` with nothing after it is an envelope with no target: - // delivery bounces it and holds back what follows. Anything else held - // is text to delivery, explicit's bare `>>` included. - const arrows = this.held.trimStart(); - if (this.opts.mode === 'hybrid' && arrows === '>>>') this.target = null; - else this.emit(this.held, out); - } - // Hybrid's `>>>` and a gap with nothing after it: no target either. - if (this.state === 'prefix-gap') this.target = null; - if (this.state === 'prefix-token') this.applyTarget(); - this.held = ''; - this.token = ''; - if (this.opts.mode !== 'locus') this.state = 'line-start'; - return coalesce(out); - } - - /** End of stream: the last segment ends. */ - finish(): RoutedDelta[] { - return this.endSegment(); - } - - private step(ch: string, out: RoutedDelta[]): void { - switch (this.state) { - case 'body': - if (ch === '\n' && this.opts.mode !== 'locus') { - this.emit('\n', out); - this.state = 'line-start'; - } else { - this.emit(ch, out); - } - return; - - case 'line-start': - // Indentation before a prefix is held until the line's decision: the - // delivery path reads an indented `>>` (explicit) or `>>>` (hybrid) - // line as a prefix too, after any leading whitespace. - if (ch !== '\n' && isSpace(ch)) { - this.state = 'indent'; - this.held = ch; - return; - } - if (ch === '>') { this.state = 'maybe-prefix'; this.held = '>'; return; } - this.state = 'body'; - this.step(ch, out); - return; - - case 'indent': - if (ch !== '\n' && isSpace(ch)) { this.held += ch; return; } - if (ch === '>') { this.held += ch; this.state = 'maybe-prefix'; return; } - this.emit(this.held, out); - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - - case 'maybe-prefix': { - const prefix = this.opts.mode === 'hybrid' ? '>>>' : '>>'; - const arrows = this.held.trimStart(); - if (ch === '>' && arrows.length < prefix.length) { - this.held += '>'; - return; - } - if (arrows === prefix) { - if (this.opts.mode === 'hybrid' && (ch === ' ' || ch === '\t')) { - this.held = ''; - this.state = 'prefix-gap'; - return; - } - if (isSpace(ch)) { - if (this.opts.mode === 'explicit') { - // `>>` and then whitespace is a quoted arrow: body text, since - // delivery's prefix needs a target right after the arrows. - this.emit(this.held, out); - } else { - // `>>>` and then whitespace other than a space or tab (a - // newline included) is an envelope with no target. Delivery - // bounces it and holds back what follows until a later `>>>` - // names a target. - this.target = null; - } - // The whitespace itself is body, a newline included. - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - } - this.state = 'prefix-token'; - this.token = ch; - this.held = ''; - return; - } - this.emit(this.held, out); - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - } - - case 'prefix-gap': - if (ch === ' ' || ch === '\t') return; - if (ch === '\n') { this.target = null; this.state = 'line-start'; return; } - // Other whitespace can't start hybrid's target (`[ \t]*\S+`): an - // envelope with no target, as above. - if (isSpace(ch)) { this.target = null; this.state = 'body'; return; } - this.state = 'prefix-token'; - this.token = ch; - return; - - case 'prefix-token': - if (ch === ' ' || ch === '\t') { this.applyTarget(); this.state = 'after-token'; this.sawBang = false; return; } - if (ch === '\n') { this.applyTarget(); this.state = 'line-start'; return; } - // Any other whitespace ends the token too (`\S+`), and starts the - // body: the grammar's gap after a token is spaces and tabs only. - if (isSpace(ch)) { this.applyTarget(); this.state = 'body'; this.step(ch, out); return; } - this.token += ch; - return; - - case 'after-token': - // Grammar: `([ \t]+!)?[ \t]*` — one optional `!` immediately after the - // first whitespace run is the continue-turn modifier, not body. - if (ch === ' ' || ch === '\t') return; - if (ch === '\n') { this.state = 'line-start'; return; } - if (ch === '!' && !this.sawBang) { this.sawBang = true; return; } - this.state = 'body'; - this.step(ch, out); - return; - } - } - - private applyTarget(): void { - const spec = this.token; - this.token = ''; - if (spec === 'skip_reply') { this.target = null; return; } - this.target = this.opts.resolve(spec); - } - - private emit(text: string, out: RoutedDelta[]): void { - if (this.target === null || !text) return; - out.push({ channelId: this.target, delta: text }); - this.accumulated.set(this.target, (this.accumulated.get(this.target) ?? '') + text); - } -} - -function coalesce(deltas: RoutedDelta[]): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (const d of deltas) { - const last = out[out.length - 1]; - if (last && last.channelId === d.channelId) last.delta += d.delta; - else out.push({ ...d }); - } - return out; -} diff --git a/src/types/trace.ts b/src/types/trace.ts index b1526bbcb..c8811d270 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -156,6 +156,48 @@ export type TraceEvent = type: 'inference:turn_ended'; agentName: string; }) + | (TraceEventBase & { + /** + * Prose that delivery published, sent once the publish is confirmed: + * the outgoing stream's chunk (MCPL Spec 14.3 + * `channels/outgoing/chunk`), emitted whether or not the channel's + * server takes the stream. It carries only what was published, so a + * consumer that speaks or shows it, such as a voice relay, never gets + * text that delivery withholds (a `skip_reply`, a silencing send, a + * private `think`, a hold, a bounce) or a publish that failed or + * wasn't confirmed. `inference:tokens` stays the model's text as it is + * written (held, as before, while a tool-result guard's batch is + * pending). A later publish to the same channel by the same stream + * opens with a paragraph break (`\n\n`), so a channel's chunks joined + * in `index` order are its text, its messages kept apart. + * `mcpl:speech-routed` also reports each speech publish, the `sleep` + * tool's announcement among them, with no stream id, index or end, so + * a relay that voices speech takes one of the two. + */ + type: 'inference:speech'; + agentName: string; + /** Where the prose is published. */ + channelId: string; + /** The stream's id, shared with its `channels/outgoing/*` notifications. */ + inferenceId: string; + /** Increases by one per chunk of the stream, across its channels. */ + index: number; + text: string; + }) + | (TraceEventBase & { + /** + * A stream's end for one channel it published to (MCPL Spec 14.3 + * `channels/outgoing/complete`), sent on every exit path once the + * stream has ended and its deliveries have run, or, when the host + * stops, at once: `text` is everything its `inference:speech` chunks + * carried there, and none follows it. + */ + type: 'inference:speech_complete'; + agentName: string; + channelId: string; + inferenceId: string; + text: string; + }) // Tool lifecycle | (TraceEventBase & { diff --git a/test/explicit-prose-routing.test.ts b/test/explicit-prose-routing.test.ts index 5159458e4..e00c33903 100644 --- a/test/explicit-prose-routing.test.ts +++ b/test/explicit-prose-routing.test.ts @@ -26,7 +26,6 @@ import type { } from '../src/index.js'; import { AgentFramework } from '../src/index.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; -import { ProseStreamRouter } from '../src/mcpl/prose-stream-router.js'; import { MembraneError } from '@animalabs/membrane'; import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; @@ -355,13 +354,13 @@ describe('explicit prose routing', () => { type RoutingState = { - pin?: string; destination: string | null; hybridHold: boolean; + pin?: string; hybridHold: boolean; continuation: boolean; deliveries: string[]; suppressed: number; triggerChannel?: string; locus?: string; engaged: Set; arrivals: Array<{ messageId: string; channelId: string }>; }; /** A turn's routing state, as the framework makes a fresh one. */ const routingState = (fields: Partial = {}): RoutingState => ({ - destination: null, hybridHold: false, continuation: false, deliveries: [], suppressed: 0, + hybridHold: false, continuation: false, deliveries: [], suppressed: 0, engaged: new Set(), arrivals: [], ...fields, }); @@ -1201,7 +1200,7 @@ describe('explicit prose routing', () => { await framework.stop(); }); - it('the stream reads leading whitespace and a token\'s end as delivery does', async () => { + it("delivery reads leading whitespace and a token's end before a `>>` prefix", async () => { const framework = await createFramework(); const routed = stubRegistry(framework); const gateway = framework as unknown as { deliverProse(agent: unknown, text: string): Promise }; @@ -1211,61 +1210,23 @@ describe('explicit prose routing', () => { for (const text of cases) { routed.length = 0; await gateway.deliverProse(agent, text); - const router = new ProseStreamRouter({ - mode: 'explicit', initialTarget: null, resolve: (spec) => (spec === '#alpha' ? 'chan-alpha' : null), - }); - const streamed = [...router.feed(text), ...router.finish()]; - assert.deepEqual(routed.map((r) => r.locus), ['chan-alpha'], `delivery sends ${JSON.stringify(text)} to #alpha`); - assert.ok(streamed.length > 0 && streamed.every((d) => d.channelId === 'chan-alpha'), - `the stream sends ${JSON.stringify(text)} there too`); + assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), [{ text: 'hello', locus: 'chan-alpha' }], + `delivery sends the body of ${JSON.stringify(text)} to #alpha`); } } finally { await framework.stop(); } }); - it("a round's destination, decided at its boundary, is the sticky destination its delivery leaves", async () => { - const framework = await createFramework(); - stubRegistry(framework); - const internals = framework as unknown as { - deliverProse(agent: unknown, text: string): Promise; - admitProse(text: string): unknown[]; - roundDestination(current: string | null, envelopes: unknown[]): string | null; - turnRouting: Map; - }; - const agent = framework.getAgent('assistant')!; - const rounds: string[][] = [ - ['>>#alpha first\n>>#alpha'], - [' >>#alpha indented'], - ['>>#nowhere bounced'], - ['>>skip_reply kept\nand unprefixed'], - ['unprefixed', '>>#alpha second segment'], - ]; - try { - for (const segments of rounds) { - for (const start of [null, 'chan-alpha']) { - internals.turnRouting.set('assistant', routingState({ pin: start ?? undefined })); - for (const segment of segments) await internals.deliverProse(agent, segment); - const admitted = segments.flatMap((segment) => internals.admitProse(segment)); - assert.equal(internals.roundDestination(start, admitted), internals.turnRouting.get('assistant')?.pin ?? null, - `${JSON.stringify(segments)} from ${start}`); - } - } - } finally { - await framework.stop(); - } - }); - - it('hybrid: a `>>>` envelope with no target streams nothing, as delivery bounces it', async () => { + it('hybrid: a `>>>` envelope with no target publishes nothing, so nothing streams, and holds the turn that queued it', async () => { const framework = await createFramework('hybrid'); const routed = stubRegistry(framework); const gateway = framework as unknown as { - deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, routing: RoutingState): Promise; + deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, routing: RoutingState, + onPublished?: (channelId: string, text: string) => void): Promise; turnRouting: Map; }; const agent = framework.getAgent('assistant')!; - // The registry's resolver trims its argument, as the router's must here. - const resolve = (spec: string) => (spec.trim() === '#alpha' ? 'chan-alpha' : null); const cases = ['>>>\r\nsecret', '>>>\nsecret', '>>>\u00a0#alpha secret', '>>> \u00a0#alpha secret']; try { for (const text of cases) { @@ -1273,50 +1234,45 @@ describe('explicit prose routing', () => { const queued: RoutingState = routingState(); const live: RoutingState = routingState(); gateway.turnRouting.set('assistant', live); - await gateway.deliverHybridProse(agent, text, 'chan-locus', true, queued); + const streamed: string[] = []; + await gateway.deliverHybridProse(agent, text, 'chan-locus', true, queued, (_channel, body) => streamed.push(body)); assert.equal(queued.hybridHold, true, 'the supplied turn receives the hold'); assert.equal(live.hybridHold, false, 'the live turn stays unheld'); - const router = new ProseStreamRouter({ mode: 'hybrid', initialTarget: 'chan-locus', resolve }); - const streamed = [...router.feed(text), ...router.finish()]; assert.equal(routed.length, 0, `delivery bounces ${JSON.stringify(text)}`); - assert.deepEqual(streamed, [], `and the stream sends none of ${JSON.stringify(text)}`); + assert.deepEqual(streamed, [], `and the stream hears none of ${JSON.stringify(text)}`); } } finally { await framework.stop(); } }); - it('hybrid: a segment that is only `>>>`, or `>>>` and a gap, holds back the next segment, as delivery does', async () => { + it('hybrid: a segment that is only `>>>`, or `>>>` and a gap, holds back the next segment, from delivery and the stream alike', async () => { const framework = await createFramework('hybrid'); const routed = stubRegistry(framework); const gateway = framework as unknown as { - deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, routing: RoutingState): Promise; + deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, routing: RoutingState, + onPublished?: (channelId: string, text: string) => void): Promise; turnRouting: Map; }; const agent = framework.getAgent('assistant')!; - const resolve = (spec: string) => (spec.trim() === '#alpha' ? 'chan-alpha' : null); try { for (const prefix of ['>>>', '>>> ', '>>>\t']) { routed.length = 0; const queued: RoutingState = routingState(); const live: RoutingState = routingState(); gateway.turnRouting.set('assistant', live); - await gateway.deliverHybridProse(agent, '>>>#alpha first', 'chan-locus', true, queued); - await gateway.deliverHybridProse(agent, prefix, 'chan-locus', true, queued); - await gateway.deliverHybridProse(agent, 'secret', 'chan-locus', true, queued); + const streamed: Array<{ channelId: string; text: string }> = []; + const onPublished = (channelId: string, text: string) => { streamed.push({ channelId, text }); }; + await gateway.deliverHybridProse(agent, '>>>#alpha first', 'chan-locus', true, queued, onPublished); + await gateway.deliverHybridProse(agent, prefix, 'chan-locus', true, queued, onPublished); + await gateway.deliverHybridProse(agent, 'secret', 'chan-locus', true, queued, onPublished); assert.deepEqual(queued.deliveries, ['chan-alpha'], 'the supplied turn records its delivery'); assert.equal(queued.suppressed, 1, 'the supplied turn records its suppression'); assert.deepEqual(live.deliveries, [], 'the live turn receives no delivery'); assert.equal(live.suppressed, 0, 'the live turn receives no suppression'); - const router = new ProseStreamRouter({ mode: 'hybrid', initialTarget: 'chan-locus', resolve }); - const streamed = [ - ...router.feed('>>>#alpha first'), ...router.endSegment(), - ...router.feed(prefix), ...router.endSegment(), - ...router.feed('secret'), ...router.finish(), - ]; assert.deepEqual(routed.map((r) => r.text), ['first'], `delivery sends only first, after ${JSON.stringify(prefix)}`); - assert.equal(streamed.filter((d) => d.delta.includes('secret')).length, 0, - `and the stream holds secret back after ${JSON.stringify(prefix)}`); + assert.deepEqual(streamed, [{ channelId: 'chan-alpha', text: 'first' }], + `and the stream hears first, where it was published, and holds secret back after ${JSON.stringify(prefix)}`); } } finally { await framework.stop(); @@ -1624,8 +1580,8 @@ describe('explicit prose routing', () => { // ---- A start that continues a logical turn keeps the turn's routing ---- // A framework retry after a stream error, the compile a provider-capacity // hold defers a turn to, and the re-inference that consumes unstreamed tool - // results each continue the turn: the sticky destination, explicit mode's - // turn destination and hybrid's hold carry into the new stream. + // results each continue the turn: the sticky destination and hybrid's hold + // carry into the new stream, which streams what its deliveries publish. /** An organization acceleration 429, which holds the turn for a later compile. */ const accelerationLimit = (request: NormalizedRequest): MembraneError => new MembraneError({ @@ -1706,7 +1662,7 @@ describe('explicit prose routing', () => { for (const [label, mode, first, expected] of [ ['hybrid >>>skip_reply keeps later unprefixed prose private', 'hybrid', '>>>skip_reply an aside', []], ['hybrid >>>#alpha is sticky', 'hybrid', '>>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], - ['explicit >>#alpha is sticky, and the stream starts at the turn\'s destination', 'explicit', '>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], + ['explicit >>#alpha is sticky, and the later stream streams where delivery publishes', 'explicit', '>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], ] as const) { it(`across ${start}, ${label}`, async () => { const { framework, routed, bounces, laterChunks } = await continueTurn(how, mode, first); @@ -1714,11 +1670,11 @@ describe('explicit prose routing', () => { // Publication as the registry stub saw it; bounce notices by their kind. assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), expected); assert.equal(bounces, 0, 'nothing bounced'); - if (mode === 'explicit') { - // The later stream streams its unprefixed text where delivery sends it. - assert.deepEqual(laterChunks.map((c) => c.channelId), laterChunks.map(() => 'chan-alpha')); - assert.equal(laterChunks.map((c) => c.delta).join(''), 'carrying on'); - } + // In every mode, the later stream streams what it published, where + // it published it: nothing under a hold, and its unprefixed text at + // the sticky destination. + assert.deepEqual(laterChunks.map((c) => ({ text: c.delta, locus: c.channelId })), + expected.filter((post) => post.text === 'carrying on')); } finally { await framework.stop(); } @@ -1729,7 +1685,7 @@ describe('explicit prose routing', () => { for (const [label, mode, left, expected] of [ ['hybrid: the hold keeps later unprefixed prose private', 'hybrid', { hybridHold: true, locus: 'world:commons' }, []], ['hybrid: the sticky destination', 'hybrid', { pin: 'chan-alpha', locus: 'world:commons' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], - ['explicit: the sticky destination, and the stream starts at the turn\'s destination', 'explicit', { pin: 'chan-alpha', destination: 'chan-alpha' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], + ['explicit: the sticky destination, and the stream streams where delivery publishes', 'explicit', { pin: 'chan-alpha' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], ] as const) { it(`a re-inference on unstreamed tool results keeps the turn's routing: ${label}`, async () => { membrane.pushResponse(createMockResponse([{ type: 'text', text: 'carrying on' }] as ContentBlock[])); @@ -1752,10 +1708,9 @@ describe('explicit prose routing', () => { await within(framework.runUntilIdle(), 'the re-inference', 10000); assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), expected); assert.equal(context(framework).bounces, 0, 'nothing bounced'); - if (mode === 'explicit') { - assert.deepEqual(chunks.map((c) => c.channelId), chunks.map(() => 'chan-alpha')); - assert.equal(chunks.map((c) => c.delta).join(''), 'carrying on'); - } + // In every mode, the re-inference streams what it published, where it + // published it. + assert.deepEqual(chunks.map((c) => ({ text: c.delta, locus: c.channelId })), expected); } finally { await framework.stop(); } @@ -1779,7 +1734,7 @@ describe('explicit prose routing', () => { const cm = framework.getAgent('assistant')!.getContextManager(); const getMessage = cm.getMessage.bind(cm); try { - const earlier = routingState({ pin: 'chan-alpha', destination: 'chan-alpha' }); + const earlier = routingState({ pin: 'chan-alpha' }); internals.turnRouting.set('assistant', earlier); internals.openLogical.set('assistant', earlier); // Someone addresses the agent from another channel while the turn waits. @@ -1839,12 +1794,10 @@ describe('explicit prose routing', () => { return outcome; }; let starts = 0; - let retryChunks = 0; framework.onTrace((event) => { if (event.type !== 'inference:started') return; starts++; if (starts === 2) { - retryChunks = chunks.length; membrane.pushResponse(createMockResponse([{ type: 'text', text: 'carrying on' }])); } if (starts >= 3) membrane.pushResponse(createMockResponse([{ type: 'text', text: `${p}skip_reply` }])); @@ -1881,7 +1834,6 @@ describe('explicit prose routing', () => { const routing = internals.turnRouting.get('assistant')!; if (mode === 'explicit') { assert.equal(routing.pin, 'discord:g:cafe', "round two's delivery hasn't run yet"); - assert.equal(routing.destination, 'chan-alpha', 'though round two set the destination as it ended'); } else { assert.equal(routing.hybridHold, false, "round two's delivery hasn't run yet"); } @@ -1894,11 +1846,16 @@ describe('explicit prose routing', () => { { text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }, ], "the replaced stream's posts go first, and the retry's text follows the destination they set"); - assert.deepEqual(chunks.slice(retryChunks).map((c) => c.channelId), chunks.slice(retryChunks).map(() => 'chan-alpha'), - "the retry's stream starts at the destination round two set"); + assert.deepEqual(chunks.map((c) => ({ text: c.delta.trim(), locus: c.channelId })), [ + { text: 'zero', locus: 'discord:g:cafe' }, + { text: 'first', locus: 'chan-alpha' }, + { text: 'carrying on', locus: 'chan-alpha' }, + ], "every chunk follows its publish: the replaced stream's late posts, then the retry's text at the destination they set"); } else { assert.deepEqual(posts, [{ text: 'zero', locus: 'discord:g:cafe' }], "the retry's unprefixed text stays private under the hold the late delivery set"); + assert.deepEqual(chunks.map((c) => ({ text: c.delta.trim(), locus: c.channelId })), [{ text: 'zero', locus: 'discord:g:cafe' }], + "only the late post streams: the retry's held text doesn't"); const receipt = context(framework).texts.filter((t) => t.startsWith('[delivered]')); assert.equal(receipt.length, 1, `one receipt for the logical turn: ${receipt.join(' | ')}`); assert.match(receipt[0]!, /discord:g:cafe/, "it reports the replaced stream's late post"); diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts new file mode 100644 index 000000000..7e88f9b29 --- /dev/null +++ b/test/outgoing-stream.test.ts @@ -0,0 +1,596 @@ +/** + * The outgoing stream (MCPL Spec 14.3 `channels/outgoing/chunk` and + * `complete`, mirrored as the `inference:speech` traces) carries what + * delivery published, sent once each publish is confirmed. Text that + * delivery withholds never reaches it: a round that a silencing tool, + * `skip_reply` or a private `think` keeps back, text under a hybrid hold, a + * bounce, or a round the stream never finished. Nor does a publish that fails + * or isn't confirmed. A voice server speaks what it gets, and words already + * spoken can't be taken back, so the stream can't run ahead of delivery. + */ + +import { afterEach, test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, Membrane, NormalizedRequest, NormalizedResponse, StreamEvent, YieldingStreamOptions } from '@animalabs/membrane'; +import { AgentFramework, type AgentConfig, type Module, type ModuleContext, type ToolCall, type ToolResult, + type ProcessEvent, type ProcessState } from '../src/index.js'; +import { MockYieldingStream, createMockResponse } from './helpers/mock-membrane.js'; +import { ChannelRegistry } from '../src/mcpl/channel-registry.js'; +import { ALL_CAPABILITY_PATHS, CapabilityGrant } from '../src/mcpl/capability-grant.js'; + +const dirs: string[] = []; +afterEach(() => { while (dirs.length) rmSync(dirs.pop()!, { recursive: true, force: true }); }); + +const text = (value: string): ContentBlock => ({ type: 'text', text: value }); +const call = (id: string, name: string, input: Record = {}): ContentBlock => + ({ type: 'tool_use', id, name, input }) as ContentBlock; +/** A round that ends in tool calls. */ +const round = (...blocks: ContentBlock[]) => createMockResponse(blocks, 'tool_use'); +/** A round that ends the stream. */ +const final = (...blocks: ContentBlock[]) => createMockResponse(blocks); + +/** Each stream the framework opens takes the next list of responses. */ +class ScriptMembrane { + streams = 0; + /** Wraps a stream's events, by its 1-based number. */ + wrap?: (index: number, events: AsyncIterable) => AsyncIterable; + constructor(readonly scripts: NormalizedResponse[][]) {} + streamYielding(_request: NormalizedRequest, _options: YieldingStreamOptions = {}) { + const script = this.scripts.shift(); + assert.ok(script, 'unexpected extra inference'); + const stream = new MockYieldingStream(script); + const index = ++this.streams; + const wrap = this.wrap; + if (wrap) { + const iterate = stream[Symbol.asyncIterator].bind(stream); + (stream as unknown as { [Symbol.asyncIterator]: () => AsyncIterator })[Symbol.asyncIterator] = + () => wrap(index, { [Symbol.asyncIterator]: iterate })[Symbol.asyncIterator](); + } + return stream; + } + asMembrane() { return this as unknown as Membrane; } +} + +class ToolModule implements Module { + readonly name = 'test'; + /** Runs inside a tool call, before its result: lets a test act mid-round. */ + during?: (call: ToolCall) => Promise; + async start(_ctx: ModuleContext) {} + async stop() {} + getTools() { + return [ + { name: 'read', description: 'Read', inputSchema: { type: 'object' as const, properties: {} } }, + { name: 'send_message', description: 'Explicit send', inputSchema: { type: 'object' as const, properties: {} } }, + ]; + } + async handleToolCall(call: ToolCall): Promise { + await this.during?.(call); + return { success: true, data: `payload-${call.id}` }; + } + /** Called with each external message's text as it is processed. */ + processed?: (content: string) => void; + async onProcess(event: ProcessEvent, _state: ProcessState) { + if (event.type === 'external-message') this.processed?.(String(event.content)); + return event.type === 'external-message' + ? { + addMessages: [{ + participant: 'user', + content: [{ type: 'text' as const, text: String(event.content) }], + ...((event as { metadata?: Record }).metadata ? { metadata: (event as { metadata?: Record }).metadata } : {}), + }], + requestInference: String(event.content) === 'go', + } + : {}; + } +} + +type Logged = + | { kind: 'chunk'; channelId: string; text: string; inferenceId: string; index: number } + | { kind: 'publish'; channelId: string; text: string } + | { kind: 'complete'; channelId: string; text: string; inferenceId: string } + | { kind: 'mark'; channelId: string; text: string }; + +/** + * The registry stub: #alpha, #cafe and @laria resolve, the turn's locus is + * world:test, and every chunk, publish and complete lands in one ordered + * log. A publish whose text is in `fails` returns unconfirmed. + */ +function registry(framework: AgentFramework, options: { streaming?: boolean } = {}) { + const log: Logged[] = []; + /** A publish whose text is held here doesn't return until the test lets it go. */ + const holds = new Map>(); + /** A publish of one of these texts isn't confirmed: `delivered: false`, or a throw. */ + const fails = new Map(); + const known: Record = { '#alpha': 'chan-alpha', '#cafe': 'chan-cafe', '@laria': 'discord:dm:99' }; + let stopped = false; + const target: Record = { + resolveLocus: () => 'world:test', + resolveProseTarget: (spec: string) => (known[spec] ? { channelId: known[spec] } : { error: `no channel ${spec}` }), + routeSpeech: async (_agent: string, body: string, channelId: string | null) => { + if (!channelId) return null; + log.push({ kind: 'publish', channelId, text: body }); + await holds.get(body); + const failure = fails.get(body); + if (failure === 'throw') throw new Error('synthetic publish failure'); + return { delivered: failure === undefined, channelId }; + }, + sendOutgoingChunk: (channelId: string, _agent: string, inferenceId: string, index: number, delta: string) => { + if (stopped) return; + log.push({ kind: 'chunk', channelId, text: delta, inferenceId, index }); + }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, body: string) => { + if (stopped) return; + log.push({ kind: 'complete', channelId, text: body, inferenceId }); + }, + // As the real registry's stopAll forgets its channels, a chunk or + // complete sent after it finds no server and goes nowhere. + stopAll: () => { + stopped = true; + log.push({ kind: 'mark', channelId: '', text: 'registry stopped' }); + }, + // skip_reply ends the turn, as the real registry's does; think is noted. + handleChannelToolCall: async (name: string) => (name === 'skip_reply' + ? { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } } + : { success: true, data: { noted: true } }), + getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, + getChannelTools: () => [], + }; + // A registry whose server doesn't take the stream sends no chunk or complete. + if (options.streaming === false) { + delete target.sendOutgoingChunk; + delete target.sendOutgoingComplete; + } + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(target, { + get: (t, key: string) => (key in t ? t[key] : () => undefined), + }); + return { + log, + holds, + fails, + chunks: () => log.filter((e): e is Extract => e.kind === 'chunk'), + publishes: () => log.filter((e) => e.kind === 'publish').map((e) => ({ channelId: e.channelId, text: e.text })), + completes: () => log.filter((e): e is Extract => e.kind === 'complete'), + }; +} + +async function harness(scripts: NormalizedResponse[][], config: Partial = {}, options: { streaming?: boolean } = {}) { + const dir = mkdtempSync(join(tmpdir(), 'af-outgoing-stream-')); dirs.push(dir); + const membrane = new ScriptMembrane(scripts); + const module = new ToolModule(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test', systemPrompt: 'system', ...config }], modules: [module], syncIntervalMs: 0, + }); + const stub = registry(framework, options); + const traces: Array<{ type: string; channelId: string; text: string; inferenceId: string; index?: number }> = []; + framework.onTrace((event) => { + if (event.type === 'inference:speech' || event.type === 'inference:speech_complete') { + const { type, channelId, text: body, inferenceId } = event; + traces.push({ type, channelId, text: body, inferenceId, ...(event.type === 'inference:speech' ? { index: event.index } : {}) }); + } + }); + /** + * Once the agent's queued deliveries have run. A stream's `complete` is + * chained on its deliveries in its teardown, ahead of this wait. + */ + const settled = async () => { + await (framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + }; + const run = async () => { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + await framework.runUntilIdle(); + await settled(); + }; + return { framework, membrane, module, run, settled, traces, ...stub }; +} + +test('a delivered round streams its text once its publish is confirmed, and the speech traces mirror the stream', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + try { + await h.run(); + const [id] = new Set(h.chunks().map((c) => c.inferenceId)); + assert.deepEqual(h.log, [ + { kind: 'publish', channelId: 'world:test', text: 'first' }, + { kind: 'chunk', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, + { kind: 'publish', channelId: 'world:test', text: 'second' }, + { kind: 'chunk', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, + { kind: 'complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, + ], 'each publish is streamed once it is confirmed, and a later one to the same channel opens with a paragraph break'); + assert.deepEqual(h.traces, [ + { type: 'inference:speech', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, + { type: 'inference:speech', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, + { type: 'inference:speech_complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, + ]); + } finally { await h.framework.stop(); } +}); + +for (const mode of ['locus', 'hybrid', 'explicit'] as const) { + const aside = mode === 'explicit' ? '>>#alpha an aside I decided against' : 'an aside I decided against'; + + test(`${mode}: a round that calls skip_reply streams nothing`, async () => { + const h = await harness([[round(text(aside), call('skip', 'skip_reply'))]], { proseRouting: mode }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [], 'delivery keeps the round'); + assert.deepEqual(h.log, [], 'and the stream gets none of it'); + assert.deepEqual(h.traces, []); + } finally { await h.framework.stop(); } + }); + + test(`${mode}: a round beside a private think streams nothing, and the next round streams as it is published`, async () => { + const answer = mode === 'explicit' ? '>>#alpha the answer' : 'the answer'; + const h = await harness([[round(text(aside), call('plan', 'think', { content: 'planning' })), final(text(answer))]], + { proseRouting: mode, sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + const where = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + assert.deepEqual(h.publishes(), [{ channelId: where, text: 'the answer' }]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: where, text: 'the answer' }]); + } finally { await h.framework.stop(); } + }); +} + +for (const mode of ['locus', 'hybrid'] as const) { + test(`${mode}: a round with a send streams nothing, and neither does the turn's later prose`, async () => { + const h = await harness([[round(text('sending it directly'), call('send', 'test--send_message')), final(text('and done'))]], + { proseRouting: mode }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [], 'the send silences the turn from its round on'); + assert.deepEqual(h.log, [], 'and the stream carries none of it'); + } finally { await h.framework.stop(); } + }); +} + +test('explicit: a round starts where the sticky destination is, and its >> lines move it, as delivery does', async () => { + const h = await harness([[round(text('>>#alpha first'), call('one', 'test--read')), final(text('middle\n>>#cafe later'))]], + { proseRouting: 'explicit' }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'middle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: '\n\nmiddle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + assert.deepEqual(h.completes().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first\n\nmiddle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + } finally { await h.framework.stop(); } +}); + +test('explicit: a {{unsent}} resend streams the retained text it publishes', async () => { + const original = 'the reply I meant for laria'; + // The unprefixed reply bounces and wakes a second turn, which resends it. + const h = await harness([[final(text(original))], [final(text('>>@laria {{unsent}}'))]], { proseRouting: 'explicit' }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [{ channelId: 'discord:dm:99', text: original }]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'discord:dm:99', text: original }], + 'the stream carries the substituted text, not the token'); + } finally { await h.framework.stop(); } +}); + +/** Two streams of one turn: the first round crosses the stream budget, so the turn restarts. */ +function budgetRestart(first: string, continued: string) { + const overBudget = round(text(first), call('one', 'test--read')); + overBudget.usage = { inputTokens: 100, outputTokens: 5 }; + return [[overBudget], [final(text(continued))]]; +} + +test("hybrid: a resumed stream's first round stays held under the hold an earlier stream left", async () => { + const h = await harness(budgetRestart('>>>skip_reply an aside', 'carrying on'), { proseRouting: 'hybrid', maxStreamTokens: 20 }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the turn restarted for its budget'); + assert.deepEqual(h.publishes(), [], 'delivery keeps the hold across the restart'); + assert.deepEqual(h.log, [], 'and the resumed stream sends nothing to the locus'); + } finally { await h.framework.stop(); } +}); + +test("hybrid: a resumed stream's first round streams to the pin an earlier stream left, and each stream completes", async () => { + const h = await harness(budgetRestart('>>>#alpha first', 'carrying on'), { proseRouting: 'hybrid', maxStreamTokens: 20 }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the turn restarted for its budget'); + assert.deepEqual(h.publishes(), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'carrying on' }, + ]); + const chunks = h.chunks(); + assert.deepEqual(chunks.map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'carrying on' }, + ], 'the resumed stream follows the pin, not the locus'); + assert.notEqual(chunks[0]!.inferenceId, chunks[1]!.inferenceId, 'each stream has its own id'); + assert.deepEqual(h.completes().map((c) => ({ text: c.text, inferenceId: c.inferenceId })), [ + { text: 'first', inferenceId: chunks[0]!.inferenceId }, + { text: 'carrying on', inferenceId: chunks[1]!.inferenceId }, + ], 'the stream that handed its turn to the restart completes too'); + } finally { await h.framework.stop(); } +}); + +test('a stream that fails partway streams none of its unfinished round, and its retry streams what it delivers', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('half a thought'))], [final(text('carrying on'))]]); + // The first stream errors where its last round would complete. + h.membrane.wrap = (index, events) => ({ + async *[Symbol.asyncIterator]() { + for await (const event of events) { + if (index === 1 && event.type === 'complete') { + yield { type: 'error', error: new Error('synthetic transient stream error') } as StreamEvent; + return; + } + yield event; + } + }, + }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the error was retried'); + assert.deepEqual(h.publishes().map((p) => p.text), ['first', 'carrying on']); + assert.equal(h.chunks().some((c) => c.text.includes('half a thought')), false, 'the unfinished round never streams'); + assert.deepEqual(h.completes().map((c) => c.text), ['first', 'carrying on'], 'each stream completes with what it delivered'); + } finally { await h.framework.stop(); } +}); + +test('locus: after a mid-turn addressed re-pin, the next round streams to the new locus, where it is published', async () => { + const h = await harness([[round(text('station four, proceeding'), call('one', 'test--read')), final(text('answering you'))]]); + // While the tool runs, someone addresses the agent from a DM. The tool + // returns once the message has been processed, so the round's boundary + // weighs it. + h.module.during = async () => { + const seen = new Promise((resolve) => { + h.module.processed = (content) => { if (content === 'quick question over here?') resolve(); }; + }); + h.framework.pushEvent({ + type: 'external-message', source: 'test', content: 'quick question over here?', + metadata: { channelId: 'discord:dm:antra', tags: ['chat:addressed'] }, + } as unknown as ProcessEvent); + await seen; + }; + try { + await h.run(); + assert.deepEqual(h.publishes(), [ + { channelId: 'world:test', text: 'station four, proceeding' }, + { channelId: 'discord:dm:antra', text: 'answering you' }, + ]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'world:test', text: 'station four, proceeding' }, + { channelId: 'discord:dm:antra', text: 'answering you' }, + ], 'the stream follows the re-pin'); + } finally { await h.framework.stop(); } +}); + +test("explicit: a stream's complete waits for the deliveries it queued, so no chunk follows it", async () => { + // skip_reply ends the turn while round one's post is still out: explicit + // mode doesn't hold the turn for its queued deliveries. + const h = await harness([[round(text('>>#alpha first'), call('one', 'test--read')), round(text('an aside'), call('skip', 'skip_reply'))]], + { proseRouting: 'explicit' }); + let release!: () => void; + h.holds.set('first', new Promise((resolve) => { release = resolve; })); + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + // The turn has ended and its stream torn down, with 'first' still out. + await h.framework.runUntilIdle(); + h.log.push({ kind: 'mark', channelId: '', text: 'post released' }); + release(); + await h.settled(); + assert.deepEqual(h.log.map((e) => `${e.kind}:${e.text}`), [ + 'publish:first', + 'mark:post released', + 'chunk:first', + 'complete:first', + ], 'the chunk follows the confirmed post, the complete follows the chunk, and the private round streams nothing'); + } finally { + release(); + await h.framework.stop(); + } +}); + +for (const failure of ['unconfirmed', 'throw'] as const) { + test(`a publish that ${failure === 'throw' ? 'throws' : "isn't confirmed"} never streams, and the stream goes on with what is`, async () => { + const h = await harness([[round(text('lost'), call('one', 'test--read')), final(text('kept'))]]); + h.fails.set('lost', failure); + try { + await h.run(); + assert.deepEqual(h.publishes().map((p) => p.text), ['lost', 'kept'], 'both publishes were tried'); + assert.deepEqual(h.chunks().map((c) => c.text), ['kept'], 'only the confirmed one streams'); + assert.deepEqual(h.completes().map((c) => c.text), ['kept'], 'and the complete carries only it'); + assert.equal(h.traces.some((t) => t.text.includes('lost')), false, 'nor do the speech traces'); + } finally { await h.framework.stop(); } + }); +} + +test('a stream stopped mid-turn still completes what it published', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('never written'))]]); + // The tool doesn't return until the test is done with it. + let release!: () => void; + const held = new Promise((resolve) => { release = resolve; }); + let published!: () => void; + const firstOut = new Promise((resolve) => { published = resolve; }); + h.module.during = async () => { await held; }; + const unlog = h.framework.onTrace((event) => { if (event.type === 'inference:speech') published(); }); + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + let timer: ReturnType | undefined; + await Promise.race([firstOut, new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error('the first round never streamed')), 10000); + })]).finally(() => clearTimeout(timer)); + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.completes().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'world:test', text: 'first' }], + 'the stopped stream ends its utterance with what it carried'); + } finally { + unlog?.(); + release(); + await h.framework.stop(); + } +}); + +// A stop() that finds a publish still out ends the stream before its teardown +// closes the connections the publish waits on, whatever becomes of the +// publish then: the close rejects it, or confirms it too late to stream. +for (const late of ['rejected', 'confirmed'] as const) { + test(`a stop() with a publish still out completes what the stream sent before teardown, and the publish ${late} then never streams`, async () => { + const h = await harness([[ + round(text('first'), call('one', 'test--read')), + round(text('second'), call('two', 'test--read')), + final(text('never written')), + ]]); + // 'second' is still out when stop() begins; teardown settles it, as + // closing an MCPL connection settles a request it still holds. + let settleSecond!: () => void; + const secondHeld = new Promise((resolve, reject) => { + settleSecond = () => (late === 'rejected' ? reject(new Error('connection closed')) : resolve()); + }); + secondHeld.catch(() => {}); + h.holds.set('second', secondHeld); + // The second round's tool doesn't return until the test is done with it. + let releaseTwo!: () => void; + const twoHeld = new Promise((resolve) => { releaseTwo = resolve; }); + h.module.during = async (c) => { if (c.id === 'two') await twoHeld; }; + h.module.stop = async () => { + h.log.push({ kind: 'mark', channelId: '', text: 'teardown' }); + settleSecond(); + }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !h.log.some((e) => e.kind === 'publish' && e.text === 'second');) { + assert.ok(Date.now() < end, 'the second publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.log.filter((e) => e.kind !== 'publish').map((e) => `${e.kind}:${e.text}`), + ['chunk:first', 'complete:first', 'mark:registry stopped', 'mark:teardown'], + 'the stream completes before the registry forgets its channels and the teardown, with what it sent, and nothing streams after its complete'); + assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), ['inference:speech:first', 'inference:speech_complete:first'], + 'and the speech traces say the same'); + } finally { + releaseTwo(); + await h.framework.stop(); + } + }); +} + +test("with the real channel registry, a stop() with a publish still out sends the stream's complete to the server before the registry forgets its channels", async () => { + // The stop tests above use the harness's stub registry. Here the publishes, + // the chunks, the complete and stopAll go through a real ChannelRegistry + // and a server it owns, so the registry's own teardown keeps the order honest. + const dir = mkdtempSync(join(tmpdir(), 'af-outgoing-real-')); dirs.push(dir); + const membrane = new ScriptMembrane([[ + round(text('first'), call('one', 'test--read')), + round(text('second'), call('two', 'test--read')), + final(text('never written')), + ]]); + const module = new ToolModule(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test', systemPrompt: 'system' }], modules: [module], syncIntervalMs: 0, + }); + // What reaches the server, with the registry's stopAll and the teardown marked. + const wire: string[] = []; + // 'second' is still out when stop() begins; teardown rejects it, as closing + // an MCPL connection rejects a request it still holds. + let rejectSecond!: () => void; + const secondHeld = new Promise((_, reject) => { rejectSecond = () => reject(new Error('connection closed')); }); + secondHeld.catch(() => {}); + const server = { + grant: new CapabilityGrant(new Set(ALL_CAPABILITY_PATHS), []), + sendChannelsPublish: async (params: { content: Array<{ text: string }> }) => { + const body = params.content[0]!.text; + wire.push(`publish:${body}`); + if (body === 'second') await secondHeld; + return { delivered: true }; + }, + sendChannelsOutgoingChunk: (params: { delta: string }) => { wire.push(`chunk:${params.delta}`); }, + sendChannelsOutgoingComplete: (params: { content: Array<{ text: string }> }) => { wire.push(`complete:${params.content[0]!.text}`); }, + sendChannelsTyping: () => {}, + }; + const registry = new ChannelRegistry({ getServer: () => server } as never, {} as never, () => {}, () => {}, {}); + (registry as unknown as { channels: Map }).channels.set('srv:world:test', + { serverId: 'srv', descriptor: { id: 'world:test', type: 'srv', label: 'world:test' }, open: true }); + const stopAll = registry.stopAll.bind(registry); + (registry as unknown as { stopAll: () => void }).stopAll = () => { wire.push('registry stopped'); stopAll(); }; + // The turn's locus and channel tools, as the harness gives them. + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(registry, { + get: (target, key: string) => { + if (key === 'resolveLocus') return () => 'world:test'; + if (key === 'handleChannelToolCall') return async () => ({ success: true, data: {} }); + if (key === 'getChannelTools') return () => []; + const value = (target as unknown as Record)[key]; + return typeof value === 'function' ? (value as (...args: unknown[]) => unknown).bind(target) : value; + }, + }); + // The second round's tool doesn't return until the test is done with it. + let releaseTwo!: () => void; + const twoHeld = new Promise((resolve) => { releaseTwo = resolve; }); + module.during = async (c) => { if (c.id === 'two') await twoHeld; }; + module.stop = async () => { wire.push('teardown'); rejectSecond(); }; + try { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !wire.includes('publish:second');) { + assert.ok(Date.now() < end, 'the second publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + await framework.stop(); + await (framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + assert.deepEqual(wire, ['publish:first', 'chunk:first', 'publish:second', 'complete:first', 'registry stopped', 'teardown'], + "the server gets the stream's complete before the registry forgets its channels, and nothing streams after it"); + } finally { + releaseTwo(); + await framework.stop(); + } +}); + +test("the speech traces are emitted when the channel's server takes no stream", async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]], {}, { streaming: false }); + try { + await h.run(); + assert.deepEqual(h.chunks(), [], 'the server is sent no chunk'); + assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), [ + 'inference:speech:first', + 'inference:speech:\n\nsecond', + 'inference:speech_complete:first\n\nsecond', + ], 'in-process consumers still hear what was published'); + } finally { await h.framework.stop(); } +}); + +// A publish that isn't confirmed streams nothing, on every delivery path. +const unconfirmedCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'explicit'; script: () => NormalizedResponse[]; lost: string; kept?: string }> = [ + { name: 'locus text-only', mode: 'locus', script: () => [final(text('lost'))], lost: 'lost' }, + { name: 'locus trailing', mode: 'locus', script: () => [round(text('kept'), call('one', 'test--read')), final(text('lost'))], lost: 'lost', kept: 'kept' }, + { name: 'hybrid unprefixed', mode: 'hybrid', script: () => [round(text('lost'), call('one', 'test--read')), final(text('kept'))], lost: 'lost', kept: 'kept' }, + { name: 'hybrid >>> target', mode: 'hybrid', script: () => [final(text('>>>#alpha lost'))], lost: 'lost' }, + { name: 'explicit mid-turn', mode: 'explicit', script: () => [round(text('>>#alpha lost'), call('one', 'test--read')), final(text('>>#cafe kept'))], lost: 'lost', kept: 'kept' }, + { name: 'explicit text-only', mode: 'explicit', script: () => [final(text('>>#alpha lost'))], lost: 'lost' }, +]; +for (const c of unconfirmedCases) { + for (const failure of ['unconfirmed', 'throw'] as const) { + test(`${c.name}: a publish that ${failure === 'throw' ? 'throws' : "isn't confirmed"} streams nothing`, async () => { + const h = await harness([c.script()], { proseRouting: c.mode }); + h.fails.set(c.lost, failure); + try { + await h.run(); + assert.ok(h.publishes().some((p) => p.text === c.lost), 'the publish was tried'); + assert.equal(h.chunks().some((ch) => ch.text.includes(c.lost)), false, 'no chunk carries it'); + assert.equal(h.completes().some((co) => co.text.includes(c.lost)), false, 'nor does a complete'); + assert.equal(h.traces.some((t) => t.text.includes(c.lost)), false, 'nor a speech trace'); + if (c.kept) { + assert.deepEqual(h.chunks().map((ch) => ch.text), [c.kept], 'the confirmed publish beside it still streams'); + } + } finally { await h.framework.stop(); } + }); + } +} diff --git a/test/present-while-acting.test.ts b/test/present-while-acting.test.ts index 0d965d9e6..cc980b5c0 100644 --- a/test/present-while-acting.test.ts +++ b/test/present-while-acting.test.ts @@ -1011,7 +1011,7 @@ describe('present while acting', () => { // fallback's re-inference. Messages arrive before that starts. const open: Routing = { locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(['discord:guild:portables']), - destination: null, hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], + hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; internals.turnRouting.set('assistant', open); internals.openLogical.set('assistant', open); @@ -1040,7 +1040,7 @@ describe('present while acting', () => { }; const open: Routing = { locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), - destination: null, hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], + hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; internals.turnRouting.set('assistant', open); internals.openLogical.set('assistant', open); @@ -1191,7 +1191,7 @@ describe('present while acting', () => { recordTurnCheckpoint(agentName: string): void; }; const open: Routing = { - locus: 'chan-turn', engaged: new Set(), destination: null, hybridHold: false, + locus: 'chan-turn', engaged: new Set(), hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; internals.openLogical.set('assistant', open); @@ -1220,7 +1220,7 @@ describe('present while acting', () => { // live, and stop() ended every open turn. Its budget restart now runs, // and its stream fails and is retried. const turn: Routing = { - locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), destination: null, hybridHold: false, + locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; internals.turnRouting.set('assistant', turn); @@ -1267,7 +1267,7 @@ describe('present while acting', () => { // Built by hand, as #312's silent-start test does: production runs no // silent turn while a prose turn is open. const open: Routing = { - locus: 'chan-turn', engaged: new Set(), destination: null, hybridHold: false, + locus: 'chan-turn', engaged: new Set(), hybridHold: false, continuation: false, deliveries: [], suppressed: 0, arrivals: [], }; internals.openLogical.set('assistant', open); diff --git a/test/prose-stream-router.test.ts b/test/prose-stream-router.test.ts deleted file mode 100644 index 8875ee237..000000000 --- a/test/prose-stream-router.test.ts +++ /dev/null @@ -1,197 +0,0 @@ -// ProseStreamRouter: streamed routing must agree with the delivery path's -// grammar (prose-grammar.parseProsePrefix + the envelope splitter) regardless -// of how the token stream is chopped. The last test is the property that -// matters most: char-by-char feeding ≡ one-shot feeding. -import { test } from 'node:test'; -import assert from 'node:assert/strict'; -import { ProseStreamRouter, type RoutedDelta } from '../src/mcpl/prose-stream-router.js'; - -const RESOLVE: Record = { - '#general': 'discord:g:100', - '#ops': 'discord:g:200', - '@sol': 'discord:dm:300', -}; - -function explicit(): ProseStreamRouter { - return new ProseStreamRouter({ - mode: 'explicit', - initialTarget: null, - resolve: (spec) => RESOLVE[spec] ?? null, - }); -} - -function run(r: ProseStreamRouter, text: string, chunk = text.length): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (let i = 0; i < text.length; i += chunk) out.push(...r.feed(text.slice(i, i + chunk))); - out.push(...r.finish()); - return out; -} - -function textFor(deltas: RoutedDelta[], ch: string): string { - return deltas.filter((d) => d.channelId === ch).map((d) => d.delta).join(''); -} - -test('locus mode: pure passthrough, >> lines are plain text', () => { - const r = new ProseStreamRouter({ mode: 'locus', initialTarget: 'discord:g:100', resolve: () => null }); - const out = run(r, 'hello\n>>#ops not a prefix here\nbye'); - assert.equal(textFor(out, 'discord:g:100'), 'hello\n>>#ops not a prefix here\nbye'); -}); - -test('explicit: unprefixed leading text is suppressed (delivery would bounce it)', () => { - const out = run(explicit(), 'no destination yet\n>>#general now routed'); - assert.equal(textFor(out, 'discord:g:100'), 'now routed'); - assert.equal(out.every((d) => !d.delta.includes('no destination')), true); -}); - -test('explicit: multi-envelope switching, prefixes swallowed', () => { - const out = run(explicit(), '>>#general hello there\nstill general\n>>#ops aside for ops\n>>@sol dm line'); - assert.equal(textFor(out, 'discord:g:100'), 'hello there\nstill general\n'); - assert.equal(textFor(out, 'discord:g:200'), 'aside for ops\n'); - assert.equal(textFor(out, 'discord:dm:300'), 'dm line'); -}); - -test('quoted `>> arrow` and lone `>` stay body text', () => { - const out = run(explicit(), '>>#general look:\n>> quoted arrow\n> markdown quote\ndone'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n>> quoted arrow\n> markdown quote\ndone'); -}); - -test('skip_reply suppresses; later envelope resumes', () => { - const out = run(explicit(), '>>skip_reply private thought\n>>#ops public line'); - assert.equal(out.every((d) => !d.delta.includes('private')), true); - assert.equal(textFor(out, 'discord:g:200'), 'public line'); -}); - -test('unresolvable target suppresses that envelope only', () => { - const out = run(explicit(), '>>#nonexistent to nowhere\n>>#general to somewhere'); - assert.equal(out.every((d) => !d.delta.includes('nowhere')), true); - assert.equal(textFor(out, 'discord:g:100'), 'to somewhere'); -}); - -test('explicit: an indented prefix routes, as delivery reads it', () => { - const out = run(explicit(), ' >>#general indented reply\n\t>>#ops tabbed aside\n plain indented line'); - assert.equal(textFor(out, 'discord:g:100'), 'indented reply\n'); - assert.equal(textFor(out, 'discord:g:200'), 'tabbed aside\n plain indented line'); -}); - -test('explicit: indentation before a quoted arrow or a lone `>` stays body', () => { - const out = run(explicit(), '>>#general look:\n >> quoted arrow\n > quote'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n >> quoted arrow\n > quote'); -}); - -test('explicit: any leading whitespace before a prefix, as delivery trims it', () => { - const out = run(explicit(), '\u00a0>>#general nbsp\n\u2003>>#ops em space\n\r>>@sol after a CR'); - assert.equal(textFor(out, 'discord:g:100'), 'nbsp\n'); - assert.equal(textFor(out, 'discord:g:200'), 'em space\n'); - assert.equal(textFor(out, 'discord:dm:300'), 'after a CR'); -}); - -test('explicit: whitespace other than a space or tab ends the token and starts the body', () => { - const out = run(explicit(), '\t>>#ops\r\nhello'); - assert.equal(textFor(out, 'discord:g:200'), '\r\nhello'); -}); - -test('explicit: `>>` and then any whitespace is a quoted arrow', () => { - const out = run(explicit(), '>>#general look:\n>>\u00a0quoted'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n>>\u00a0quoted'); -}); - -test('explicit: a bare `>>` line is text, its newline included, as delivery sends it', () => { - const r = new ProseStreamRouter({ mode: 'explicit', initialTarget: 'discord:g:100', resolve: (spec) => RESOLVE[spec] ?? null }); - const out = [...r.feed('>>\nhello'), ...r.finish()]; - assert.equal(textFor(out, 'discord:g:100'), '>>\nhello'); -}); - -test('explicit: each segment starts a new line, and a prefix with no body takes effect at its end', () => { - // Delivery parses each round's text on its own: a round may begin with a - // prefix, and a round that is only a prefix sets the sticky destination. - const r = explicit(); - const out: RoutedDelta[] = []; - out.push(...r.feed('>>#general first'), ...r.endSegment()); - out.push(...r.feed('>>#ops second'), ...r.endSegment()); - out.push(...r.feed('>>@sol'), ...r.endSegment()); - out.push(...r.feed('to sol'), ...r.finish()); - assert.equal(textFor(out, 'discord:g:100'), 'first'); - assert.equal(textFor(out, 'discord:g:200'), 'second'); - assert.equal(textFor(out, 'discord:dm:300'), 'to sol'); -}); - -test('` !` continuation modifier is swallowed; `!` as body survives', () => { - const out = run(explicit(), '>>#general ! next turn continues\n>>#ops !bang body'); - assert.equal(textFor(out, 'discord:g:100'), 'next turn continues\n'); - // grammar: `([ \t]+!)?` — the first ! after whitespace is the modifier, - // remainder is body (matches parseProsePrefix on ">>#ops !bang body"). - assert.equal(textFor(out, 'discord:g:200'), 'bang body'); -}); - -test('prefix on its own line routes the following lines', () => { - const out = run(explicit(), '>>#general\nbody on next line\nand more'); - assert.equal(textFor(out, 'discord:g:100'), 'body on next line\nand more'); -}); - -test('byChannel accumulates exactly the emitted text', () => { - const r = explicit(); - run(r, '>>#general one\n>>#ops two'); - assert.equal(r.byChannel().get('discord:g:100'), 'one\n'); - assert.equal(r.byChannel().get('discord:g:200'), 'two'); -}); - -test('PROPERTY: char-by-char ≡ one-shot, across every chunk size', () => { - const text = 'lead-in ignored\n>>#general alpha\nbeta > gamma\n>> not a prefix\n>>#ops ! delta\n>>skip_reply hidden\n>>#general {{unsent}} tail'; - const oneShot = run(explicit(), text); - for (const size of [1, 2, 3, 5, 7, 11]) { - const chunked = run(explicit(), text, size); - for (const ch of Object.values(RESOLVE)) { - assert.equal(textFor(chunked, ch), textFor(oneShot, ch)); - } - } -}); - -function hybrid(): ProseStreamRouter { - return new ProseStreamRouter({ - mode: 'hybrid', - initialTarget: 'world:commons', - resolve: (spec) => RESOLVE[spec] ?? (spec === 'world:commons' ? 'world:commons' : null), - }); -} - -test('hybrid: unprefixed prose stays on the frozen locus', () => { - const out = run(hybrid(), 'ordinary world speech'); - assert.equal(textFor(out, 'world:commons'), 'ordinary world speech'); -}); - -test('hybrid: triple-arrow target switches publication and consumes the envelope', () => { - const out = run(hybrid(), '>>>#general cafe body'); - assert.equal(textFor(out, 'discord:g:100'), 'cafe body'); - assert.equal(out.every(d => !d.delta.includes('>>>')), true); - assert.equal(textFor(out, 'world:commons'), ''); -}); - -test('hybrid: permissive whitespace and cross-surface canonical ids', () => { - const out = run(hybrid(), ' >>> #ops spaced\n\t>>> world:commons home'); - assert.equal(textFor(out, 'discord:g:200'), 'spaced\n'); - assert.equal(textFor(out, 'world:commons'), 'home'); -}); - -test('hybrid: double arrows and mid-line triple arrows remain ordinary locus prose', () => { - const text = '>>#ops old syntax is prose\nlook >>>#ops quoted'; - const out = run(hybrid(), text); - assert.equal(textFor(out, 'world:commons'), text); - assert.equal(textFor(out, 'discord:g:200'), ''); -}); - -test('hybrid: missing target suppresses that envelope; a later valid envelope resumes', () => { - const out = run(hybrid(), '>>>#missing nowhere\nplain remains on failed target\n>>>world:commons restored'); - assert.equal(out.every(d => !d.delta.includes('nowhere') && !d.delta.includes('plain remains')), true); - assert.equal(textFor(out, 'world:commons'), 'restored'); -}); - -test('hybrid PROPERTY: char-by-char equals one-shot', () => { - const text = 'locus first\n>>>#general alpha\nbeta\n>>> #ops delta\n>>>world:commons home'; - const one = run(hybrid(), text); - for (const size of [1, 2, 3, 4, 7, 11]) { - const chunked = run(hybrid(), text, size); - for (const ch of ['world:commons', ...Object.values(RESOLVE)]) { - assert.equal(textFor(chunked, ch), textFor(one, ch)); - } - } -}); diff --git a/test/tool-result-guard.test.ts b/test/tool-result-guard.test.ts index 8cc2886ca..7b6822a6f 100644 --- a/test/tool-result-guard.test.ts +++ b/test/tool-result-guard.test.ts @@ -332,7 +332,7 @@ test('native Membrane observes the first refusal even when guard is enabled by a const outgoing: string[] = []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, speech: string) => { + routeSpeech: async (_agent: string, speech: string, channelId: string) => { routed.push(speech); return { delivered: true, channelId: 'world:test' }; }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, @@ -395,7 +395,7 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery } finally { await h.framework.stop(); } }); -test('explicit mode: a budget restart streams on to the destination the turn last addressed, before delivery catches up', { timeout: 15000 }, async () => { +test('explicit mode: after a budget restart, the resumed reply streams where delivery sends it, once the earlier posts have gone', { timeout: 15000 }, async () => { const h = await harness([ [ createMockResponse([ @@ -411,6 +411,10 @@ test('explicit mode: a budget restart streams on to the destination the turn las ], { proseRouting: 'explicit', maxStreamTokens: 1000 }); const routed: Array<{ text: string; channelId: string }> = []; const streamed: Array<{ channelId: string; delta: string }> = []; + // Each confirmed publish and each chunk, in the order they happened. + const order: string[] = []; + // What delivery had started to publish when the restarted stream wrote. + let atRestart: string[] | undefined; // Delivery of 'first' is held until the resumed stream has written // 'continued', so '>>#cafe second' is still queued, and delivery's sticky // destination still #alpha, when the restart begins. @@ -434,6 +438,9 @@ test('explicit mode: a budget restart streams on to the destination the turn las for await (const event of events) { yield event; if (event.type === 'tokens' && event.content === 'continued') { + // The premise: the restarted stream is writing while 'first' is + // still held and '>>#cafe second' is still queued behind it. + atRestart = routed.map((r) => r.text); releaseFirst(); await secondDone; } @@ -453,10 +460,12 @@ test('explicit mode: a budget restart streams on to the destination the turn las routed.push({ text, channelId }); if (text === 'first') await firstHeld; if (text === 'second') secondDelivered(); + order.push(`published:${text}`); return { delivered: true, channelId }; }, sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { streamed.push({ channelId, delta }); + order.push(`chunk:${delta.trim()}`); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -465,6 +474,7 @@ test('explicit mode: a budget restart streams on to the destination the turn las await h.run(); assert.equal(h.membrane.requests.length, 2); assert.ok(restarts.includes('context_budget'), 'the turn restarted for its budget'); + assert.deepEqual(atRestart, ['first'], "the restarted stream wrote while 'first' was held and 'second' still queued"); assert.deepEqual(routed, [ { text: 'first', channelId: 'chan-alpha' }, { text: 'second', channelId: 'chan-cafe' }, @@ -475,6 +485,9 @@ test('explicit mode: a budget restart streams on to the destination the turn las assert.equal(streamed.some((d) => d.delta.includes('>>')), false, 'no prefix is streamed as text'); assert.deepEqual(streamed.filter((d) => d.delta.includes('continued')).map((d) => d.channelId), ['chan-cafe'], 'the resumed stream follows the destination the turn last addressed'); + assert.ok(order.indexOf('chunk:continued') > order.indexOf('published:continued') + && order.indexOf('published:continued') > order.indexOf('published:first'), + `the resumed reply streams once its own publish is confirmed, after the earlier posts: ${order.join(', ')}`); } finally { releaseFirst(); secondDelivered(); @@ -652,11 +665,13 @@ test("locus mode: a resumed stream's deliveries queue behind the turn's earlier /** * The registry stub for one-stream explicit turns: #alpha and #cafe resolve, - * deliveries and stream chunks are recorded, and a `think` tool is offered. + * deliveries, stream chunks and stream completes are recorded, and a `think` + * tool is offered. */ function explicitRegistry(h: Awaited>) { const routed: Array<{ text: string; channelId: string }> = []; - const streamed: Array<{ channelId: string; delta: string }> = []; + const streamed: Array<{ channelId: string; delta: string; inferenceId: string }> = []; + const completed: Array<{ channelId: string; text: string; inferenceId: string }> = []; (h.framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveProseTarget: (spec: string) => spec === '#alpha' ? { channelId: 'chan-alpha' } : spec === '#cafe' ? { channelId: 'chan-cafe' } : { error: `no channel ${spec}` }, @@ -664,20 +679,23 @@ function explicitRegistry(h: Awaited>) { routed.push({ text: text.trim(), channelId }); return { delivered: true, channelId }; }, - sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { - streamed.push({ channelId, delta }); + sendOutgoingChunk: (channelId: string, _agent: string, inferenceId: string, _index: number, delta: string) => { + streamed.push({ channelId, delta, inferenceId }); + }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, text: string) => { + completed.push({ channelId, text, inferenceId }); }, handleChannelToolCall: async () => ({ success: true, data: { noted: true } }), getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [{ name: 'think', description: 'Reason privately', inputSchema: { type: 'object', properties: { content: { type: 'string' } } } }], }, { get: (target, key: string) => key in target ? (target as Record)[key] : () => undefined }); - return { routed, streamed }; + return { routed, streamed, completed }; } test("explicit mode: within one stream, each round after a private round or >>skip_reply streams where delivery sends it", { timeout: 15000 }, async () => { - // The stream follows each prefix as it's written, a private round's too. - // At each round's end it returns to the turn's destination, which a private - // think's >>#cafe and a >>skip_reply envelope leave at #alpha. + // The stream carries what delivery publishes, where it publishes it. A + // private think's >>#cafe and a >>skip_reply envelope publish nothing and + // leave the sticky destination at #alpha. const h = await harness([[ createMockResponse([ { type: 'text', text: '>>#alpha first\n' }, @@ -714,10 +732,13 @@ test("explicit mode: within one stream, each round after a private round or >>sk } }); -test("explicit mode: a membrane retry within a stream starts again at the turn's destination", { timeout: 15000 }, async () => { +test("explicit mode: a membrane retry within a stream streams none of the refused attempt, and the stream goes on under its id", { timeout: 15000 }, async () => { // Round one set #alpha. The membrane then refuses the next attempt and // retries it; the retried attempt's unprefixed reply is delivered to - // #alpha, and streams there too. + // #alpha, and streams there too. The refused attempt was never delivered, + // so it never streamed, and the stream's one `complete` carries both + // delivered rounds. The stream's inference/lifecycle `started` and its + // terminal share that one id too. const h = await harness([[ createMockResponse([ { type: 'text', text: '>>#alpha first\n' }, @@ -741,22 +762,37 @@ test("explicit mode: a membrane retry within a stream starts again at the turn's }; return stream; }; - const { routed, streamed } = explicitRegistry(h); + const { routed, streamed, completed } = explicitRegistry(h); + const lifecycle: Array<{ inferenceId: string; phase: string }> = []; + (h.framework as unknown as { hookOrchestrator: unknown }).hookOrchestrator = { + beforeInference: async () => [], + emitLifecycle: (event: { inferenceId: string; phase: string }) => { lifecycle.push({ inferenceId: event.inferenceId, phase: event.phase }); }, + }; try { await h.run(); + // The stream's `complete` is chained on its deliveries, ahead of this wait. + await (h.framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); assert.deepEqual(routed, [ { text: 'first', channelId: 'chan-alpha' }, { text: 'continued', channelId: 'chan-alpha' }, ]); - assert.deepEqual(streamed.filter((d) => d.delta.includes('continued')).map((d) => d.channelId), ['chan-alpha']); - assert.equal(streamed.some((d) => d.delta.includes('refused partial')), true, - 'the refused attempt had streamed, as text streams before a refusal is known'); + assert.deepEqual(streamed.map((d) => ({ channelId: d.channelId, delta: d.delta })), [ + { channelId: 'chan-alpha', delta: 'first' }, + { channelId: 'chan-alpha', delta: '\n\ncontinued' }, + ], 'the stream carries what was published, and none of the refused attempt'); + assert.equal(new Set(streamed.map((d) => d.inferenceId)).size, 1, 'the retry keeps the stream id'); + assert.deepEqual(completed, [{ channelId: 'chan-alpha', text: 'first\n\ncontinued', inferenceId: streamed[0]!.inferenceId }], + 'one complete closes the stream with everything it carried'); + assert.deepEqual(lifecycle, [ + { inferenceId: streamed[0]!.inferenceId, phase: 'started' }, + { inferenceId: streamed[0]!.inferenceId, phase: 'completed' }, + ], 'the lifecycle notifications start and end the same inference'); } finally { await h.framework.stop(); } }); -test("explicit mode: a round's later text block starts a new line for the stream, as delivery joins blocks", { timeout: 15000 }, async () => { +test("explicit mode: a round's later text block starts a new envelope, for delivery and the stream alike", { timeout: 15000 }, async () => { // Two text blocks in one round, the second opening with a prefix, the first // not ending with a newline. Delivery reads them joined by a newline. const h = await harness([[ @@ -767,27 +803,6 @@ test("explicit mode: a round's later text block starts a new line for the stream ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'continued' }] as ContentBlock[]), ]], { proseRouting: 'explicit' }); - const open = h.membrane.streamYielding.bind(h.membrane); - h.membrane.streamYielding = (request: NormalizedRequest, options?: YieldingStreamOptions) => { - const stream = open(request, options); - const iterate = stream[Symbol.asyncIterator].bind(stream); - (stream as unknown as { [Symbol.asyncIterator]: () => AsyncIterator })[Symbol.asyncIterator] = async function* () { - const events = { [Symbol.asyncIterator]: iterate } as AsyncIterable<{ type?: string; content?: unknown }>; - let split = false; - for await (const event of events) { - if (!split && event.type === 'tokens' && typeof event.content === 'string' && event.content.includes('>>#cafe')) { - // The mock streams a round's text as one token: stream it as the - // two blocks the round holds. - split = true; - yield { type: 'tokens', content: '>>#alpha first part', meta: { type: 'text', visible: true, blockIndex: 0 } }; - yield { type: 'tokens', content: '>>#cafe second', meta: { type: 'text', visible: true, blockIndex: 1 } }; - continue; - } - yield event; - } - }; - return stream; - }; const { routed, streamed } = explicitRegistry(h); try { await h.run(); @@ -806,8 +821,8 @@ test("explicit mode: a round's targets are resolved once, when it ends, for its // From a review's probe. Both rounds that name #cafe have ended, and their // deliveries wait behind 'first', when the registry's #cafe starts naming // another channel. Each goes to the channel #cafe named when its round - // ended, where the stream sent it: the queued '>>#cafe second', and the - // resumed '>>#cafe continued', whose text-only round ended before the wait. + // ended, and streams there: the queued '>>#cafe second', and the resumed + // '>>#cafe continued', whose text-only round ended before the wait. const t = await heldRestart('explicit', { first: '>>#alpha first\n', second: '>>#cafe second', continued: '>>#cafe continued' }); try { await t.held; @@ -996,11 +1011,11 @@ test("hybrid: a restarted turn whose last response is empty writes its receipt a test("a fresh turn starts new routing state in every mode, and leaves the earlier turn's as it was", async () => { for (const mode of ['explicit', 'hybrid', 'locus'] as const) { const h = await harness([[answer()]], { proseRouting: mode }); - type Routing = { pin?: string; destination: string | null; hybridHold: boolean; + type Routing = { pin?: string; hybridHold: boolean; continuation: boolean; deliveries: string[]; suppressed: number; triggerChannel?: string; locus?: string; engaged: Set; arrivals: Array<{ messageId: string; channelId: string }> }; const internals = h.framework as unknown as { turnRouting: Map }; - const earlierFields = (): Routing => ({ pin: 'chan-earlier', destination: 'chan-earlier', hybridHold: true, + const earlierFields = (): Routing => ({ pin: 'chan-earlier', hybridHold: true, continuation: true, deliveries: ['chan-earlier'], suppressed: 2, triggerChannel: 'chan-earlier', locus: 'chan-earlier', engaged: new Set(['chan-earlier']), arrivals: [{ messageId: 'earlier-message', channelId: 'chan-earlier' }] }); const earlier = earlierFields(); @@ -1010,7 +1025,6 @@ test("a fresh turn starts new routing state in every mode, and leaves the earlie const now = internals.turnRouting.get('assistant'); assert.notEqual(now, earlier, `${mode}: the new turn has its own routing state`); assert.equal(now?.pin, undefined, `${mode}: no sticky destination`); - assert.equal(now?.destination, null, `${mode}: no turn destination`); assert.equal(now?.hybridHold, false, `${mode}: no hybrid hold`); assert.equal(now?.continuation, false, `${mode}: no continuation`); assert.deepEqual(now?.deliveries, [], `${mode}: no deliveries carried into the new turn`); diff --git a/test/tool-wrapper-prose-guard-integration.test.ts b/test/tool-wrapper-prose-guard-integration.test.ts index 5d8010f0b..d58d1c69a 100644 --- a/test/tool-wrapper-prose-guard-integration.test.ts +++ b/test/tool-wrapper-prose-guard-integration.test.ts @@ -121,7 +121,9 @@ async function run(enabled?: boolean, allowedTools?: string[]) { const outgoing: string[] = []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, speech: string) => { routed.push(speech); return { delivered: true, channelId: 'world:test' }; }, + routeSpeech: async (_agent: string, speech: string, channelId: string) => { + routed.push(speech); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -172,7 +174,9 @@ describe('tool wrapper prose guard integration', () => { agents: [{ name: 'assistant', model: 'test', systemPrompt: 'sys', toolWrapperProseGuard: true }], modules: [module], }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ - resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string, channelId: string) => { + routed.push(text); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_c: string, _a: string, _id: string, _i: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -219,7 +223,7 @@ describe('tool wrapper prose guard integration', () => { assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); - it('buffers then delivers ordinary prose when enabled', async () => { + it("a guarded turn's ordinary prose streams what delivery publishes", async () => { const dir = mkdtempSync(join(tmpdir(), 'af-wrapper-guard-ordinary-')); dirs.push(dir); const membrane = new MockMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Ordinary answer.' }] as ContentBlock[])); @@ -230,7 +234,9 @@ describe('tool wrapper prose guard integration', () => { }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + routeSpeech: async (_agent: string, text: string, channelId: string) => { + routed.push(text); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -240,7 +246,8 @@ describe('tool wrapper prose guard integration', () => { const all = framework.getAgent('assistant')!.getContextManager().getAllMessages() as Array<{ content: ContentBlock[]; metadata?: Record }>; await framework.stop(); assert.deepEqual(routed, ['Ordinary answer.']); - assert.deepEqual(outgoing, [], 'guarded turns buffer ordinary prose until completion'); + assert.deepEqual(outgoing, ['Ordinary answer.'], + 'the guard classifies before delivery, and the stream carries what delivery publishes'); assert.ok(all.some((m) => m.content.some((b) => b.type === 'text' && b.text === 'Ordinary answer.'))); assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); @@ -413,7 +420,9 @@ describe('tool wrapper prose guard integration', () => { resolveLocus: () => 'world:test', startTyping: (channel: string) => typing.push(`start:${channel}`), stopTyping: (channel?: string) => typing.push(`stop:${channel ?? '(all)'}`), - routeSpeech: async () => ({ delivered: true, channelId: 'world:test' }), + routeSpeech: async (_agent: string, text: string, channelId: string) => { + return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => outgoing.push(['chunk', delta]), sendOutgoingComplete: (_channel: string, _agent: string, _id: string, text: string) => outgoing.push(['complete', text]), getDefaultPublishChannel: () => 'world:test', isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -423,11 +432,18 @@ describe('tool wrapper prose guard integration', () => { created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); const run = framework.runEphemeralToCompletion(created.agent, created.contextManager); framework.start(); - assert.deepEqual(await run, { speech: 'Ordinary ephemeral answer.', toolCallsCount: 0 }); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.ok(typing.includes('stop:(all)'), 'terminal physical frame stops typing after ephemeral caller disposal'); - assert.ok(outgoing.some(([kind, text]) => kind === 'complete' && text === 'Ordinary ephemeral answer.'), 'terminal physical frame finalizes outgoing stream'); - await framework.stop(); + try { + assert.deepEqual(await run, { speech: 'Ordinary ephemeral answer.', toolCallsCount: 0 }); + // The terminal frame's teardown stops typing and sends the complete. + const until = Date.now() + 5000; + while (Date.now() < until && !(typing.includes('stop:(all)') && outgoing.some(([kind]) => kind === 'complete'))) { + await new Promise((resolve) => setTimeout(resolve, 5)); + } + assert.ok(typing.includes('stop:(all)'), 'terminal physical frame stops typing after ephemeral caller disposal'); + assert.ok(outgoing.some(([kind, text]) => kind === 'complete' && text === 'Ordinary ephemeral answer.'), 'terminal physical frame finalizes outgoing stream'); + } finally { + await framework.stop(); + } }); it('releases a conversation name reservation after transient creation failure', async () => { @@ -452,7 +468,7 @@ describe('tool wrapper prose guard integration', () => { }); - it('idle-disposed ephemeral stops typing and finalizes every started outgoing stream', async () => { + it("idle-disposed ephemeral stops typing, and text it never delivered doesn't stream", async () => { const dir = mkdtempSync(join(tmpdir(), 'af-wrapper-guard-idle-terminal-')); dirs.push(dir); const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: new LateCompletionAfterIdleMembrane().asMembrane(), agents: [], modules: [] }); const events: string[] = []; @@ -460,7 +476,9 @@ describe('tool wrapper prose guard integration', () => { resolveLocus: () => 'world:test', startTyping: (channel: string) => events.push(`start:${channel}`), stopTyping: (channel?: string) => events.push(`stop:${channel ?? '(all)'}`), - routeSpeech: async () => ({ delivered: true, channelId: 'world:test' }), + routeSpeech: async (_agent: string, text: string, channelId: string) => { + return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => events.push(`chunk:${delta}`), sendOutgoingComplete: (_channel: string, _agent: string, _id: string, text: string) => events.push(`complete:${text}`), getDefaultPublishChannel: () => 'world:test', isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -470,12 +488,18 @@ describe('tool wrapper prose guard integration', () => { created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); const run = framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 500, idleTimeoutMs: 20, idlePollMs: 5 }); framework.start(); - await assert.rejects(run, /stalled/); - await new Promise((resolve) => setTimeout(resolve, 180)); - assert.ok(events.includes('stop:(all)'), 'disposer-owned frame stops typing'); - assert.ok(events.some((event) => event.startsWith('chunk:')), 'probe actually opened an outgoing stream'); - assert.ok(events.some((event) => event.startsWith('complete:')), 'every started outgoing stream receives a terminal complete'); - await framework.stop(); + try { + await assert.rejects(run, /stalled/); + // The disposed frame's typing stops in its teardown, which runs once + // its late answer has come in. + const until = Date.now() + 5000; + while (Date.now() < until && !events.includes('stop:(all)')) await new Promise((resolve) => setTimeout(resolve, 5)); + assert.ok(events.includes('stop:(all)'), 'disposer-owned frame stops typing'); + assert.deepEqual(events.filter((event) => event.startsWith('chunk:') || event.startsWith('complete:')), [], + "the partial text and the disposed frame's late answer were never delivered, so neither streams"); + } finally { + await framework.stop(); + } }); }); From 16ee15c37b8246f7ff5eb35dbe20d93206de35e0 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Wed, 14 Oct 2026 10:27:47 -0400 Subject: [PATCH 02/19] fix(routing): a code_execution script's calls count as its round's (#306) A round's tool names decide whether its prose goes out: skip_reply keeps it, a silencing send suppresses it and the turn's later prose, and a think under the private same-round policy holds it back. The same calls made inside a code_execution script didn't count, so a round that skipped or sent from a script was still published, and streamed. - A round that calls code_execution is routed once those calls have returned (ScriptRound, AgentFramework.scriptRounds). A foreground script records each tool it calls, and settleScriptRound routes the round with its own calls and its scripts' together, from the tool-result handler, before anything there can end or continue the turn. - Its segments, the turn's locus and an explicit round's admitted `>>` targets are taken at its tool-calls event, so a re-pin, a channel_open made by the script or a label renamed while it runs doesn't move prose already written. Only whether its calls silence it waits. Hybrid's `>>>` targets, sticky pin and hold apply as its delivery runs, as for any queued hybrid delivery. - The round waits only for its code_execution calls: a direct call's effect is known from its name, so a slow tool beside the script doesn't delay it. A background script's call already returns as soon as the script has started, so its round isn't delayed, and its later calls aren't the round's. - A round that can't be decided is withheld, since whether a script would have silenced it is unknown: its stream ends first (a stopped turn, a disposed ephemeral), its stream no longer owns it when its last result arrives (route()'s guard: the agent reset, deregistered, or on a newer stream), or a newer round of the same agent name replaces it. A log line names it, and its segments count as suppressed in its turn's routing state: since #314's fix, a turn that a later stream continues writes its delivery receipt after the withhold. - The tool-calls handler is restructured for this: the round's routing is one closure (routeRound), called at once for a round without code_execution, as before, after its calls are dispatched. Tests (code-execution): an inner skip_reply keeps its round's prose from delivery and the stream in each mode; an inner send silences the round and the turn's later prose in locus mode, and in explicit mode silences neither, as a direct send doesn't; an inner private think holds the round back; the round goes out once its script returns, not after a slow tool beside it; a background script's round is routed at once, and its calls aren't the round's, even beside a foreground script; with two code_execution calls the round waits for both; a script round's prose goes to the locus and `>>` target of its end, though the script opens another channel or the label is renamed; a disposed ephemeral's round, a stopped stream's round, a round the guard finds unowned (three ways, one per clause, each also counted in the turn's receipt tally) and a displaced round are each withheld; a stream's teardown leaves alone a round of another agent object, or of the same agent object's later stream. The tests that simulate a departure clear the bookkeeping a real one does, so the file exits without --test-force-exit. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.fixed.md | 17 + src/framework.ts | 274 ++++++++++---- test/code-execution.test.ts | 500 +++++++++++++++++++++++++ 3 files changed, 709 insertions(+), 82 deletions(-) diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index a23a5e5a4..a96e4f265 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -21,6 +21,23 @@ as before. A publish that fails, times out or isn't confirmed is never streamed. A `{{unsent}}` resend streams the retained text it sends, rather than the token. +- A call made inside a foreground `code_execution` script now counts + toward its round's routing, as the same call made directly does (#306). + In explicit mode an inner `skip_reply` keeps the round's envelopes in the + context. In locus and hybrid mode an inner `skip_reply` suppresses the + round's prose, and an inner silencing send suppresses it and the turn's + later prose. An inner `think` under the private same-round policy holds + the round back in every mode. A round that calls `code_execution` is + routed once those calls have returned, so its prose goes out when the + script finishes rather than when it starts. The turn's locus and an + explicit round's `>>` targets are still taken when the round ends, at its + tool-calls event; hybrid's `>>>` targets apply as its delivery runs, as + for any queued hybrid delivery. A background script's call returns as + soon as the script has started, and its later calls aren't the round's. A + round whose turn ends or stops waiting on it, or that a newer round of + the same agent replaces, before its scripts return, as when the turn is + stopped, is withheld: a log line names it, and the turn's delivery + receipt counts it as suppressed. - Every stream now sends `channels/outgoing/complete` for each channel it streamed to, on every exit path, as MCPL Spec 14.3 asks: once its deliveries have run, or, when the host stops, at once, with what it has diff --git a/src/framework.ts b/src/framework.ts index 574f9d7c3..3a14207e6 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -640,6 +640,33 @@ interface ProseRound { onPublished?: SpeechPublished; } +/** + * A round that called `code_execution`, waiting for those calls to return + * before its prose is routed: a foreground script's own calls count as the + * round's, so a `skip_reply` or a send inside the script silences the round + * as a direct call does (AgentFramework.scriptRounds). + */ +interface ScriptRound { + /** The agent and stream that dispatched the round. */ + agent: Agent; + streamId: number; + /** Each of the round's code_execution calls, with the tools its foreground script has called so far, in order. */ + scripts: Map; + /** The round's code_execution calls whose results haven't arrived. */ + pending: Set; + /** Routes the round's prose; called once, when its last code_execution result arrives. */ + route: () => void; + /** + * Withholds the round's prose when its routing can't be decided: before + * its scripts returned, its stream ended or no longer waits on it, or a + * newer round of the same agent name replaced it. Whether a script would + * have silenced the round is unknown. A log line names it, and the turn's + * delivery receipt counts its segments as suppressed: a turn that a later + * stream continues writes its receipt after the withhold. + */ + withhold: (why: string) => void; +} + function isTurnContinuation(reason: string): boolean { return reason === 'context_budget_restart' || reason === 'tool_results_ready' || reason === 'tool_result_guard_retry'; @@ -1966,8 +1993,12 @@ export class AgentFramework { * result marks the script that made the call, and is applied to that * script's code_execution result instead of cancelling the stream * mid-script (which would wedge the turn). A stopped script's late result - * marks only that script, never the agent's next one. */ - private scriptDeferredEndTurn: Map = new Map(); + * marks only that script, never the agent's next one. `calls` records + * each tool the script calls, for its round's routing (ScriptRound). */ + private scriptDeferredEndTurn: Map = new Map(); + /** Per agent, a round that called code_execution and waits for those calls' + * results before its prose is routed (ScriptRound). */ + private scriptRounds: Map = new Map(); /** RFC-007: per agent, the origin of the foreground script its runner is * executing, handed to that script's inner calls (scripts run one at a * time per runner). Background runners capture theirs in the closure. */ @@ -8495,6 +8526,10 @@ export class AgentFramework { // itself is never sent, only whether it was an error. this.toolLifecycleEmitter?.onResult(event.agentName, event.callId, event.result); + // A code_execution result: once its round has them all, the round's + // prose is routed, before anything here can end or continue the turn. + this.settleScriptRound(event.agentName, event.callId); + const agent = this.agents.get(event.agentName); if (agent) { this.touchEphemeralRun(event.agentName, true); @@ -12169,17 +12204,6 @@ export class AgentFramework { agent.enterWaitingForTools(event.calls, stream); - for (const call of event.calls) { - // RFC-007: register at the one dispatch point for model-issued - // calls, keyed by (agent, call id) with this inference's id; - // `started` goes out once dispatch returns. Refusal sites inside - // dispatch run synchronously and call refuse() first, so a call - // the host refuses produces no events. - this.toolLifecycleEmitter?.register(agent.name, outgoingInferenceId, call); - this.dispatchToolCall(agent.name, call); - this.toolLifecycleEmitter?.open(agent.name, call.id); - } - // Speak-while-acting: route THIS round's prose to the locus NOW // instead of batching every segment to the end of the turn. Long // tool-using turns (robot control, live sessions) otherwise dump @@ -12191,80 +12215,140 @@ export class AgentFramework { // from the round it occurs (see SILENCING_TOOLS). Only rounds // with verbatim roundContent are live-routed (see liveProseRouting // note above — the fallback preamble is cumulative in XML mode). - if (this.channelRegistry) { - const roundToolNames = event.calls.map((c) => c.name); + // + // The round's tool names decide whether its prose goes out, so a + // round that calls code_execution is routed once those calls have + // returned: a foreground script's own calls count as the round's, + // and a skip_reply or a send inside the script silences the round + // as a direct call does (ScriptRound). Its prose then goes out + // when the script finishes. A background script's call returns as + // soon as the script has started. The turn's locus and an + // explicit round's admitted targets are taken now, at the round's + // tool-calls event, so a re-pin or a renamed label while a script + // runs doesn't move them. Hybrid's `>>>` targets, sticky pin and + // hold apply as its delivery runs, as for any queued hybrid + // delivery. + const roundToolNames = event.calls.map((c) => c.name); + const liveRound = !!this.channelRegistry && !!roundContent && roundContent.length > 0; + if (liveRound) liveProseRouting = true; + const roundSegments = liveRound ? splitProseSegments(assistantBlocks) : []; + const roundLocus = resolveTurnLocus(); + // An explicit round is admitted as it ends: its targets are + // resolved once, for its queued delivery. + const roundAdmitted = turnProseRouting === 'explicit' + ? roundSegments.map((seg) => this.admitProse(seg)) + : []; + const routeRound = (toolNames: string[]): void => { + if (!this.channelRegistry) return; const hasSameRoundPrivateThink = - roundToolNames.includes('think') && + toolNames.includes('think') && requestSnapshot.sameRoundThinkTextPolicy === 'private'; - if (roundToolNames.some(isSilencingTool)) { + if (toolNames.some(isSilencingTool)) { turnSilenced = true; } - if (roundContent && roundContent.length > 0) { - liveProseRouting = true; - const roundSegments = splitProseSegments(assistantBlocks); - if (roundSegments.length > 0) { - if (turnProseRouting === 'disabled') { - console.error( - `[routing] ${agent.name}: mid-turn prose NOT routed (proseRouting=disabled)`, - ); - this.recordProseSuppression(streamRouting, roundSegments.length); - } else if (turnProseRouting === 'hybrid') { - if (turnSilenced) { - this.recordProseSuppression(streamRouting, roundSegments.length); - } else if (!hasSameRoundPrivateThink) { - const locus = resolveTurnLocus(); - // Queued under this turn's routing, whenever it posts. - const routing = streamRouting; - for (const seg of roundSegments) { - chainSpeech(() => this.deliverHybridProse(agent, seg, locus, true, routing, streamSpeech), 'mid-turn hybrid prose delivery failed:'); - } - } - } else if (turnProseRouting === 'explicit') { - // Explicit mode: the segments go through the prose - // gateway, unless a private think in the round holds them - // back. Send silencing does not apply: unprefixed prose - // bounces, and prefixed prose is deliberate. skip_reply - // does: its round's envelopes all stay in the context - // (deliverProseEnvelope's keepPrivate), as the tool - // promises, and skip_reply then ends the turn at this - // round's tool results. - if (!hasSameRoundPrivateThink) { - const keepPrivate = roundToolNames.some((n) => bareToolName(n) === 'skip_reply'); - // The round is admitted now: its targets are resolved - // once, for its queued delivery. - const admitted = roundSegments.map((seg) => this.admitProse(seg)); - // Queued under this turn's routing, whenever it posts. - const routing = streamRouting; - console.error( - `[prose] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> ${roundSegments.length} segment(s) via prose gateway${keepPrivate ? ', kept private (skip_reply)' : ''}`, - ); - for (const envelopes of admitted) { - chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { keepPrivate, routing, onPublished: streamSpeech }), 'mid-turn prose delivery failed:'); - } - } - } else if (turnSilenced) { - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> prose NOT routed (turn silenced)`, - ); - // Visible in the turn-end receipt — silencing must never - // be a silent black hole (n=8: the flying-scene reply). - this.recordProseSuppression(streamRouting, roundSegments.length); - } else if (hasSameRoundPrivateThink) { - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> prose NOT routed (same_round_think_text_policy=private)`, - ); - } else { - const locus = resolveTurnLocus(); - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> routing ${roundSegments.length} prose segment(s) live -> ${locus ?? '(default)'}`, - ); - for (const seg of roundSegments) { - enqueueSpeech(seg, locus); - } + if (roundSegments.length === 0) return; + if (turnProseRouting === 'disabled') { + console.error( + `[routing] ${agent.name}: mid-turn prose NOT routed (proseRouting=disabled)`, + ); + this.recordProseSuppression(streamRouting, roundSegments.length); + } else if (turnProseRouting === 'hybrid') { + if (turnSilenced) { + this.recordProseSuppression(streamRouting, roundSegments.length); + } else if (!hasSameRoundPrivateThink) { + // Queued under this turn's routing, whenever it posts. + const routing = streamRouting; + for (const seg of roundSegments) { + chainSpeech(() => this.deliverHybridProse(agent, seg, roundLocus, true, routing, streamSpeech), 'mid-turn hybrid prose delivery failed:'); + } + } + } else if (turnProseRouting === 'explicit') { + // Explicit mode: the segments go through the prose gateway, + // unless a private think in the round holds them back. Send + // silencing does not apply: unprefixed prose bounces, and + // prefixed prose is deliberate. skip_reply does: its round's + // envelopes all stay in the context (deliverProseEnvelope's + // keepPrivate), as the tool promises, and skip_reply then + // ends the turn at this round's tool results. + if (!hasSameRoundPrivateThink) { + const keepPrivate = toolNames.some((n) => bareToolName(n) === 'skip_reply'); + // Queued under this turn's routing, whenever it posts. + const routing = streamRouting; + console.error( + `[prose] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> ${roundSegments.length} segment(s) via prose gateway${keepPrivate ? ', kept private (skip_reply)' : ''}`, + ); + for (const envelopes of roundAdmitted) { + chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { keepPrivate, routing, onPublished: streamSpeech }), 'mid-turn prose delivery failed:'); } } + } else if (turnSilenced) { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> prose NOT routed (turn silenced)`, + ); + // Visible in the turn-end receipt — silencing must never + // be a silent black hole (n=8: the flying-scene reply). + this.recordProseSuppression(streamRouting, roundSegments.length); + } else if (hasSameRoundPrivateThink) { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> prose NOT routed (same_round_think_text_policy=private)`, + ); + } else { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> routing ${roundSegments.length} prose segment(s) live -> ${roundLocus ?? '(default)'}`, + ); + for (const seg of roundSegments) { + enqueueSpeech(seg, roundLocus); + } } + }; + const scriptCallIds = event.calls + .filter((c) => c.name === CODE_EXECUTION_TOOL_NAME) + .map((c) => c.id); + if (scriptCallIds.length > 0) { + // In place before dispatch, so a foreground script records + // its calls here from its first one. + const round: ScriptRound = { + agent, + streamId: myStreamId, + scripts: new Map(scriptCallIds.map((id) => [id, []])), + pending: new Set(scriptCallIds), + route: () => { + // Only while this stream still owns the agent and waits on + // the round: a turn replaced, reset or cancelled meanwhile + // can't finish deciding it. + if (this.agents.get(agent.name) !== agent || agent.streamId !== myStreamId + || agent.state.status !== 'waiting_for_tools') { + round.withhold('its turn no longer waits on it (replaced, reset or stopped) while its scripts ran'); + return; + } + routeRound([...roundToolNames, ...[...round.scripts.values()].flat()]); + }, + withhold: (why) => { + if (roundSegments.length === 0) return; + console.error( + `[routing] ${agent.name}: round [${roundToolNames.join(', ')}] -> ${roundSegments.length} prose segment(s) withheld: ${why}`, + ); + this.recordProseSuppression(streamRouting, roundSegments.length); + }, + }; + // A round still waiting under this name can't be decided now: + // a newer round displaces it. + this.scriptRounds.get(agent.name)?.withhold('a newer round replaced it before its scripts returned'); + this.scriptRounds.set(agent.name, round); } + + for (const call of event.calls) { + // RFC-007: register at the one dispatch point for model-issued + // calls, keyed by (agent, call id) with this inference's id; + // `started` goes out once dispatch returns. Refusal sites inside + // dispatch run synchronously and call refuse() first, so a call + // the host refuses produces no events. + this.toolLifecycleEmitter?.register(agent.name, outgoingInferenceId, call); + this.dispatchToolCall(agent.name, call); + this.toolLifecycleEmitter?.open(agent.name, call.id); + } + + if (scriptCallIds.length === 0) routeRound(roundToolNames); // Stream's async iterator blocks on next() until provideToolResults() is called break; } @@ -13336,6 +13420,16 @@ export class AgentFramework { // exhausted, abort) so it never sticks after the turn ends. if (frameReachedTerminal) this.channelRegistry?.stopTyping(); + // A round of this stream still waiting for its code_execution calls + // can't be routed now: whether its scripts would have silenced it is + // unknown, so its prose is withheld, as text whose routing is + // undecided is. + const unsettled = this.scriptRounds.get(agent.name); + if (unsettled?.agent === agent && unsettled.streamId === myStreamId) { + this.scriptRounds.delete(agent.name); + unsettled.withhold('its stream ended before its scripts returned'); + } + // Spec 14.3: once the deliveries this stream queued have run, close // each channel it streamed to with everything it sent there, the // consumer's signal to finalize (end the TTS utterance, settle the @@ -13734,6 +13828,18 @@ export class AgentFramework { }); } + /** + * A model-issued tool's result has arrived. When it answers the last of + * the code_execution calls a round waits on (ScriptRound), the round's + * prose is routed now, its scripts' calls counted. + */ + private settleScriptRound(agentName: string, callId: string): void { + const round = this.scriptRounds.get(agentName); + if (!round || !round.pending.delete(callId) || round.pending.size > 0) return; + this.scriptRounds.delete(agentName); + round.route(); + } + /** * Run one code_execution script for an agent. Never rejects. * @@ -13834,9 +13940,10 @@ export class AgentFramework { const runner = this.getOrCreateScriptRunner(agentName); // A busy runner refuses this exec; the running script keeps its mark and - // its origin. + // its origin. The script records the tools it calls where its round + // reads them (ScriptRound), when a round dispatched it. const ownsRunner = !runner.busy; - const script = { endTurn: false }; + const script = { endTurn: false, calls: this.scriptRounds.get(agentName)?.scripts.get(call.id) ?? [] }; if (ownsRunner) this.scriptDeferredEndTurn.set(agentName, script); const ownsOrigin = origin !== undefined && ownsRunner; if (ownsOrigin) this.foregroundScriptOrigins.set(agentName, origin); @@ -14390,11 +14497,14 @@ export class AgentFramework { args: Record, origin?: ScriptCallOrigin, /** The foreground script that made the call; none for a background script. */ - script?: { endTurn: boolean }, + script?: { endTurn: boolean; calls: string[] }, ): Promise { if (toolName === CODE_EXECUTION_TOOL_NAME) { return 'Error: code_execution cannot be called from within a script'; } + // The call counts toward its round's routing however it turns out, as a + // direct call does. + script?.calls.push(toolName); const result = await this.dispatchScriptToolCall(agentName, toolName, args, origin); if (result.endTurn && script) { // Deferred: applied to the final code_execution result (see diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 5deec59aa..201c279e1 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -26,6 +26,7 @@ import type { } from '../src/index.js'; import { AgentFramework, PYTHON_RUNTIME_SOURCE, PyRunner, buildInjectedTools } from '../src/index.js'; import { createMockResponse, MockMembrane } from './helpers/mock-membrane.js'; +import type { ContentBlock } from '@animalabs/membrane'; import { PYTC_STOP_ROWS_DRIVER } from './helpers/pytc-stop-rows.js'; const ECHO_TOOLS: { pyName: string; toolName: string }[] = [ @@ -647,6 +648,505 @@ describe('framework code_execution integration (real python3)', () => { }); }); +// --------------------------------------------------------------------------- +// A round that runs code_execution: its script's calls count as the round's +// --------------------------------------------------------------------------- + +describe("a round that runs code_execution is routed once its scripts return (real python3)", () => { + /** echo, send_message and a `slow` tool held until the test lets it go; each call is logged. */ + class RoundToolModule implements Module { + readonly name = 'test'; + releaseSlow: () => void = () => {}; + private slowReleased = new Promise((resolve) => { this.releaseSlow = resolve; }); + constructor(private readonly log: string[]) {} + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + getTools(): ToolDefinition[] { + const tool = (name: string): ToolDefinition => ({ name, description: name, inputSchema: { type: 'object', properties: {} } }); + return [tool('echo'), tool('send_message'), tool('slow')]; + } + async handleToolCall(call: ToolCall): Promise { + const name = call.name.includes('--') ? call.name.slice(call.name.lastIndexOf('--') + 2) : call.name; + this.log.push(`tool:${name}`); + if (name === 'slow') { + await this.slowReleased; + this.log.push('slow:returned'); + } + return { success: true, data: { ok: true } }; + } + async onProcess(event: ProcessEvent, _state: ProcessState): Promise { + return event.type === 'external-message' + ? { addMessages: [{ participant: 'User', content: [{ type: 'text', text: 'go' }] }], requestInference: true } + : {}; + } + } + + const CHANNEL_TOOLS = [ + { name: 'skip_reply', description: 'End your turn without sending anything.', inputSchema: { type: 'object' as const, properties: {} } }, + { name: 'think', description: 'Reason privately.', inputSchema: { type: 'object' as const, properties: { content: { type: 'string' } } } }, + { name: 'channel_open', description: 'Open a channel.', inputSchema: { type: 'object' as const, properties: { channelId: { type: 'string' } } } }, + ]; + + async function scriptRound(mode: 'locus' | 'hybrid' | 'explicit', responses: ReturnType[], + config: { sameRoundThinkTextPolicy?: 'private' } = {}) { + const { tempDir, storePath } = tempStorePath('pytc-round-'); + const membrane = new MockMembrane(); + for (const response of responses) membrane.pushResponse(response); + const log: string[] = []; + const module = new RoundToolModule(log); + const framework = await AgentFramework.create({ + storePath, + membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test-model', systemPrompt: 'Act.', proseRouting: mode, ...config }], + modules: [module], + syncIntervalMs: 0, + codeExecution: { enabled: true }, + }); + // The registry stub: skip_reply and think are on the agent's surface, so + // its scripts get them too, and skip_reply ends the turn as the real one + // does. + // What each `>>` label names; a test can rename one while a script runs. + const directory: Record = { '#alpha': 'chan-alpha' }; + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ + resolveLocus: () => 'world:test', + resolveProseTarget: (spec: string) => (directory[spec] ? { channelId: directory[spec] } : { error: `no channel ${spec}` }), + routeSpeech: async (_agent: string, text: string, channelId: string | null) => { + if (!channelId) return null; + log.push(`publish:${text}@${channelId}`); + return { delivered: true, channelId }; + }, + sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { + log.push(`chunk:${delta}@${channelId}`); + }, + handleChannelToolCall: async (name: string, input: Record | undefined) => { + log.push(`tool:${name}`); + if (name === 'skip_reply') return { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }; + // A successful channel_open moves the turn's locus, as the real one does. + if (name === 'channel_open') return { success: true, data: { channelId: input?.channelId } }; + return { success: true, data: {} }; + }, + getChannelTools: () => CHANNEL_TOOLS, + getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, + }, { get: (target, key: string) => (key in target ? (target as Record)[key] : () => undefined) }); + const run = async () => { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + await framework.runUntilIdle(); + }; + const done = async () => { + module.releaseSlow(); + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + }; + return { framework, module, log, run, done, directory }; + } + + const scriptCall = (input: Record, id = 'ce1') => + ({ type: 'tool_use', id, name: 'code_execution', input }) as ContentBlock; + + for (const mode of ['locus', 'hybrid', 'explicit'] as const) { + it(`${mode}: a skip_reply inside the script keeps its round's prose from delivery and the stream`, async () => { + const prose = mode === 'explicit' ? '>>#alpha checking quietly' : 'checking quietly'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: prose }, + scriptCall({ code: 'await skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:skip_reply'), 'the script called skip_reply'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round's prose is kept, as a direct skip_reply keeps it"); + } finally { await h.done(); } + }); + } + + it("locus: a send inside the script silences its round and the turn's later prose", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await test__send_message({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the script sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + 'the send silences the round and what follows, as a direct send does'); + } finally { await h.done(); } + }); + + it("explicit: a send inside the script silences neither its round nor the turn's later prose, as a direct send doesn't", async () => { + const h = await scriptRound('explicit', [ + createMockResponse([ + { type: 'text', text: '>>#alpha posting the summary' }, + scriptCall({ code: 'await test__send_message({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: '>>#alpha done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the script sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [ + 'publish:posting the summary@chan-alpha', 'chunk:posting the summary@chan-alpha', + 'publish:done@chan-alpha', 'chunk:\n\ndone@chan-alpha', + ], "explicit mode's prefixed prose is deliberate, so a send doesn't silence it"); + } finally { await h.done(); } + }); + + it('locus: a think inside the script holds its round back under the private same-round policy', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'working it through' }, + scriptCall({ code: 'await think({"content": "notes"})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'the answer' }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the script called think'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:the answer@world:test'], + 'the round beside the think stays private, as with a direct think, and the next round is delivered'); + } finally { await h.done(); } + }); + + it("the round's prose goes out once its script returns, not after a slow tool beside it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'on it' }, + scriptCall({ code: 'print(await test__echo({}))' }), + { type: 'tool_use', id: 'slow1', name: 'test--slow', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Let the slow tool go once the round's prose has gone out, or after a + // bound, so a round that waited for it still ends and fails the order. + const watch = setInterval(() => { if (h.log.includes('publish:on it@world:test')) h.module.releaseSlow(); }, 10); + // The bound only matters when the code is wrong: a round that waited + // for the slow tool still ends, and fails the order. + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:on it@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('tool:echo') < at('publish:on it@world:test'), "it waited for the script's own call"); + assert.ok(at('publish:on it@world:test') < at('slow:returned'), 'and not for the slow tool beside it'); + assert.ok(at('publish:on it@world:test') < at('chunk:on it@world:test'), 'and its chunk followed the confirmed publish'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a background script's round is routed as soon as the script starts", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'starting a watcher' }, + scriptCall({ code: 'import asyncio\nawait asyncio.sleep(30)', background: true }), + { type: 'tool_use', id: 'slow1', name: 'test--slow', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:starting a watcher@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:starting a watcher@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:starting a watcher@world:test') < at('slow:returned'), 'without waiting for the slow tool'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a round with two code_execution calls waits for both: the second is refused at once, and the first's later skip_reply still counts", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'two scripts at once' }, + scriptCall({ code: 'await test__slow({})\nawait skip_reply({})' }, 'ce1'), + scriptCall({ code: 'print("second")' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + ]); + // Let the first script go once the second's refusal has come back. + const results: string[] = []; + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') { + results.push(event.callId); + if (event.callId === 'ce2') h.module.releaseSlow(); + } + }); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(results, ['ce2', 'ce1'], "the second script's refusal came back while the first ran"); + assert.ok(h.log.includes('tool:skip_reply'), "the first script's skip_reply ran"); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round waited for the first script, so its skip_reply kept the round's prose"); + } finally { + clearTimeout(bound); + await h.done(); + } + }); + + it("a background script's calls aren't its round's, even while a foreground script in the round runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'still mine to say' }, + scriptCall({ code: 'await test__send_message({})', background: true }, 'bg'), + scriptCall({ code: 'await test__slow({})' }, 'fg'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The foreground script holds the round open until the background send has run. + const watch = setInterval(() => { if (h.log.includes('tool:send_message')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('tool:send_message') >= 0 && at('tool:send_message') < at('slow:returned'), + 'the background send ran while the round waited on its foreground script'); + assert.ok(h.log.includes('publish:still mine to say@world:test'), "the background send didn't silence the round"); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("locus: a script round's prose goes to the locus it was written for, though the script opens another channel", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'for the room I was in' }, + scriptCall({ code: 'await channel_open({"channelId": "chan-other"})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'now over here' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [ + 'publish:for the room I was in@world:test', + 'publish:now over here@chan-other', + ], 'the open moves later prose, not the round written before it'); + } finally { await h.done(); } + }); + + it("an ephemeral disposed while its round's script runs doesn't publish the round when the script returns", async () => { + const h = await scriptRound('locus', []); + const membrane = (h.framework as unknown as { membrane: MockMembrane }).membrane; + membrane.pushResponse(createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})' }), + ] as ContentBlock[], 'tool_use')); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + try { + const created = await h.framework.createEphemeralAgent({ name: 'eph-script', model: 'test-model', systemPrompt: 'Act.', allowedTools: 'all' }); + created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); + // The idle watchdog disposes the ephemeral while its script waits in the slow call. + const run = h.framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 15000, idleTimeoutMs: 300, idlePollMs: 20 }); + h.framework.start(); + await assert.rejects(run, /stalled/); + assert.ok(h.log.includes('tool:slow'), 'the script was running when the ephemeral was disposed'); + // The script now returns, and its result arrives with the framework still running. + let resulted!: () => void; + const result = new Promise((resolve) => { resulted = resolve; }); + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') resulted(); + }); + h.module.releaseSlow(); + await result; + await h.framework.runUntilIdle(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round's routing was never decided, so the late result publishes nothing"); + assert.ok(lines.some((line) => line.includes('eph-script: round [code_execution] -> 1 prose segment(s) withheld: its stream ended before its scripts returned')), + "the disposed ephemeral's stream withheld the round as it ended"); + } finally { + console.error = error; + await h.done(); + } + }); + + // Each way a stream can stop owning its round while the round's script runs: + // the agent stops waiting on it, is deregistered, or moves to a newer stream. + const leaveRound: Record void> = { + 'its agent is reset': (_framework, agent) => agent.reset(), + 'its agent is deregistered': (framework) => { + (framework as unknown as { agents: Map }).agents.delete('assistant'); + }, + 'its agent moves to a newer stream': (_framework, agent) => { + (agent as unknown as { _streamId: number })._streamId++; + }, + }; + for (const [how, leave] of Object.entries(leaveRound)) it(`a round is withheld when its script returns if, meanwhile, ${how}`, async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + let stream: { cancel(): void } | undefined; + let agent: { state: { stream?: { cancel(): void } }; reset(): void } | undefined; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + void h.framework.runUntilIdle().catch(() => {}); + const started = Date.now(); + while (!h.log.includes('tool:slow') && Date.now() - started < 15000) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached its slow call'); + // The turn's routing state, whose receipt counts what the turn suppressed. + const routing = (h.framework as unknown as { turnRouting: Map }).turnRouting.get('assistant')!; + const suppressedBefore = routing.suppressed; + // The stream no longer owns the round, while it lives on. + agent = h.framework.getAgent('assistant')! as unknown as { state: { stream?: { cancel(): void } }; reset(): void }; + stream = agent.state.stream; + leave(h.framework, agent); + let resulted!: () => void; + const result = new Promise((resolve) => { resulted = resolve; }); + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') resulted(); + }); + h.module.releaseSlow(); + await result; + const until = Date.now() + 15000; + while (Date.now() < until && !lines.some((line) => line.includes('prose segment(s) withheld'))) await new Promise((r) => setTimeout(r, 10)); + assert.ok(lines.some((line) => line.includes('assistant: round [code_execution] -> 1 prose segment(s) withheld: its turn no longer waits on it')), + 'the script result found the round undecidable and withheld it'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], 'and published nothing'); + assert.equal(routing.suppressed, suppressedBefore + 1, "and the turn's receipt counts the withheld segment as suppressed"); + } finally { + console.error = error; + stream?.cancel(); + // The departures are simulated, without the bookkeeping a real one does: + // a real deregistration also clears the name's active stream, and a real + // newer stream takes that entry over and clears it when it ends. Clear + // it and settle the agent, so the loop this test started finds the + // framework idle and the file can exit. + (h.framework as unknown as { activeStreams: Map }).activeStreams.delete('assistant'); + agent?.reset(); + await h.done(); + } + }); + + it("a round whose stream is stopped before its script returns is withheld", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})\nawait skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + void h.framework.runUntilIdle().catch(() => {}); + // The script is running, held in its slow call. + const started = Date.now(); + while (!h.log.includes('tool:slow') && Date.now() - started < 15000) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached its slow call'); + await h.framework.stop(); + h.module.releaseSlow(); + await new Promise((r) => setTimeout(r, 50)); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "whether the script would have silenced the round is unknown, so its prose isn't sent"); + assert.ok(lines.some((line) => line.includes('1 prose segment(s) withheld: its stream ended before its scripts returned')), + 'and the withheld round is logged, not dropped silently'); + } finally { + console.error = error; + await h.done(); + } + }); + + /** A pending round of another stream, as scriptRounds holds one: it records what it was asked to do. */ + const pendingRound = (agent: unknown) => { + const asked: string[] = []; + return { + asked, + round: { + agent, streamId: -1, scripts: new Map(), pending: new Set(['elsewhere']), + route: () => { asked.push('route'); }, + withhold: (why: string) => { asked.push(`withhold: ${why}`); }, + }, + }; + }; + + it('a round registered while another waits under the same agent name withholds the older one', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the newer round' }, + scriptCall({ code: 'print("quick")' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const older = pendingRound({}); + (h.framework as unknown as { scriptRounds: Map }).scriptRounds.set('assistant', older.round); + try { + await h.run(); + assert.deepStrictEqual(older.asked, ['withhold: a newer round replaced it before its scripts returned'], + 'the older round is withheld, not overwritten silently'); + assert.ok(h.log.includes('publish:the newer round@world:test'), 'and the newer round is routed as usual'); + } finally { await h.done(); } + }); + + // A successor's round comes to wait under the agent's name while this stream + // runs: one of another agent object of the name, or one of the same agent + // object's later stream (a restart overlapping this one). + for (const whose of ['another agent object', "the same agent object's later stream"] as const) { + it(`a stream's teardown leaves alone a round that belongs to ${whose}`, async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'working' }, + { type: 'tool_use', id: 'r1', name: 'test--echo', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // pendingRound's round has stream id -1, which no stream has. + const successor = pendingRound(whose === 'another agent object' ? {} : h.framework.getAgent('assistant')); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'r1') rounds.set('assistant', successor.round); + }); + try { + await h.run(); + assert.ok(h.log.includes('publish:done@world:test'), 'the stream ran to its end'); + assert.deepStrictEqual(successor.asked, [], "this stream's teardown doesn't withhold the successor's round"); + assert.equal(rounds.get('assistant'), successor.round, 'nor remove it'); + } finally { await h.done(); } + }); + } + + it("explicit: a script round goes to the target its `>>` named when the round ended, though the label is renamed while the script runs", async () => { + const h = await scriptRound('explicit', [ + createMockResponse([ + { type: 'text', text: '>>#alpha written for alpha' }, + scriptCall({ code: 'await test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: '>>skip_reply done' }] as ContentBlock[]), + ]); + // While the script waits in its slow call, #alpha comes to name another channel. + const watch = setInterval(() => { + if (h.log.includes('tool:slow')) { h.directory['#alpha'] = 'chan-renamed'; h.module.releaseSlow(); } + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:written for alpha@chan-alpha'], + 'the round was admitted when it ended'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); +}); + // --------------------------------------------------------------------------- // Background scripts (wake_agent) + spill-to-file // --------------------------------------------------------------------------- From 9b0b2304e99a3cc327e6485b7fbb2426f6209778 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Wed, 14 Oct 2026 14:29:27 -0400 Subject: [PATCH 03/19] fix(streaming): a stop() completes a stream the framework no longer tracks stop() sends each waiting stream's `complete` before the channel registry forgets its channels (pendingOutgoingCompletes), but a stream registered its `complete` there only in driveStream's finally. stop() waits for the streams activeStreams tracks, so their finallys have run by the flush. A stream it no longer tracks hasn't: an idle ephemeral's, whose disposal removed its entry while the stream was still awaiting its last publish, or a predecessor a restart's stream replaced there. Its finally ran after the registry had forgotten its channels, so its `complete` went nowhere and its chunks never got an end. Each stream now registers its `complete` as it starts, just before the try whose finally sends it, so stop() reaches every stream that hasn't sent it. One still running completes with what it has streamed, and streams nothing after, as a stream whose deliveries are still queued already does on a stop. Test (outgoing-stream): a stream whose last publish is still out, and whose activeStreams entry is gone as disposal leaves it, completes before the registry forgets its channels when the host stops. A removal probe that registers the `complete` only in the finally fails it. Found by Greptile's review of agent-framework#360. An internal review had raised it too, and I misjudged it. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 76 +++++++++++++++++++++--------------- test/outgoing-stream.test.ts | 32 +++++++++++++++ 2 files changed, 76 insertions(+), 32 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 3a14207e6..4ac9a95cd 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1728,9 +1728,12 @@ export class AgentFramework { */ private proseSpeechChains: Map> = new Map(); /** - * Each ended stream whose `channels/outgoing/complete` still waits for the - * deliveries it queued (driveStream's finally). stop() sends them at once, - * before it clears the channel registry and closes the MCPL connections. + * Each stream whose `channels/outgoing/complete` hasn't gone yet, from + * its start: an ended one's waits for the deliveries it queued + * (driveStream's finally), and one the framework no longer tracks, such + * as a disposed ephemeral's, may still be running. stop() sends them at + * once, before it clears the channel registry and closes the MCPL + * connections. */ private pendingOutgoingCompletes: Set<() => void> = new Set(); /** @@ -2819,10 +2822,11 @@ export class AgentFramework { // A stream's `complete` waits for the deliveries it queued, and the // teardown below would leave it nowhere to go: the registry forgets its // channels, and closing an MCPL connection marks it closed before it - // rejects a publish still out. Every stream has ended by now (its finally - // ran before activeStreams settled), so each still waiting sends its - // `complete` now, with what it has streamed, and streams nothing after - // (pendingOutgoingCompletes). + // rejects a publish still out. So each stream that hasn't sent it sends + // it now, with what it has streamed, and streams nothing after + // (pendingOutgoingCompletes): every stream activeStreams tracked has + // ended by now, and one it no longer tracks, such as a disposed + // ephemeral's, may still be finishing. for (const complete of [...this.pendingOutgoingCompletes]) complete(); // Stop typing indicators, and forget the registry's channels @@ -11976,6 +11980,29 @@ export class AgentFramework { text: delta, }); }; + // Spec 14.3: the stream's end for each channel it streamed to, with + // everything it sent there, the consumer's signal to finalize (end the + // TTS utterance, settle the rendered message). Sent once, and no chunk + // follows it. The finally below sends it once the stream's deliveries + // have run. It is registered as the stream starts, so a stop() reaches + // every stream that hasn't sent it, one the framework no longer tracks + // included, such as a disposed ephemeral's still finishing its last + // publish (pendingOutgoingCompletes). + const completeOutgoing = (): void => { + if (outgoingCompleted) return; + outgoingCompleted = true; + this.pendingOutgoingCompletes.delete(completeOutgoing); + for (const [channelId, text] of outgoingText) { + this.channelRegistry?.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); + this.emitTrace({ + type: 'inference:speech_complete', + agentName: agent.name, + channelId, + inferenceId: outgoingInferenceId, + text, + }); + } + }; const enqueueSpeech = (text: string, locus: string | null): void => { chainSpeech(async () => { @@ -12058,6 +12085,9 @@ export class AgentFramework { turnSilenced = trigger?.suppressProse === true; }; + // The stream's `complete` is pending from here: every exit reaches the + // finally, which sends it, and a stop() can send it sooner. + this.pendingOutgoingCompletes.add(completeOutgoing); try { for await (const event of stream) { // Ignore every late event from an Agent generation that no longer owns @@ -13431,31 +13461,13 @@ export class AgentFramework { } // Spec 14.3: once the deliveries this stream queued have run, close - // each channel it streamed to with everything it sent there, the - // consumer's signal to finalize (end the TTS utterance, settle the - // rendered message). Every exit path does, a stream that hands its turn - // to a successor (a budget restart, a guard retry) included: the - // successor streams under its own inference id. Its deliveries can - // still be queued here, as when a tool ends an explicit-mode turn, so - // `complete` waits for them and no chunk follows it. A stop() sends it - // at once instead, with what the stream has sent, since stopping closes - // the connections a queued delivery waits on (pendingOutgoingCompletes). - const completeOutgoing = (): void => { - if (outgoingCompleted) return; - outgoingCompleted = true; - this.pendingOutgoingCompletes.delete(completeOutgoing); - for (const [channelId, text] of outgoingText) { - this.channelRegistry?.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); - this.emitTrace({ - type: 'inference:speech_complete', - agentName: agent.name, - channelId, - inferenceId: outgoingInferenceId, - text, - }); - } - }; - this.pendingOutgoingCompletes.add(completeOutgoing); + // each channel it streamed to (completeOutgoing). Every exit path does, + // a stream that hands its turn to a successor (a budget restart, a + // guard retry) included: the successor streams under its own inference + // id. Its deliveries can still be queued here, as when a tool ends an + // explicit-mode turn, so `complete` waits for them and no chunk follows + // it. A stop() sends it at once instead, with what the stream has sent, + // since stopping closes the connections a queued delivery waits on. void turnSpeechChain.then(completeOutgoing); this.frameworkCancelledStreams.delete(`${agent.name}:${myStreamId}`); if (ownsPhysicalStream) { diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 7e88f9b29..abc5dfd27 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -483,6 +483,38 @@ for (const late of ['rejected', 'confirmed'] as const) { }); } +test('a stop() completes a stream the framework no longer tracks, as a disposed ephemeral\'s, before the registry forgets its channels', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + // The turn's last publish is still out when stop() begins; teardown rejects + // it, as closing an MCPL connection rejects a request it still holds. + let rejectSecond!: () => void; + const secondHeld = new Promise((_, reject) => { rejectSecond = () => reject(new Error('connection closed')); }); + secondHeld.catch(() => {}); + h.holds.set('second', secondHeld); + h.module.stop = async () => { + h.log.push({ kind: 'mark', channelId: '', text: 'teardown' }); + rejectSecond(); + }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !h.log.some((e) => e.kind === 'publish' && e.text === 'second');) { + assert.ok(Date.now() < end, 'the last publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + // The stream, still awaiting that publish, is no longer tracked, as when + // an idle ephemeral is disposed, so stop() doesn't wait for it. + (h.framework as unknown as { activeStreams: Map }).activeStreams.delete('assistant'); + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.log.filter((e) => e.kind !== 'publish').map((e) => `${e.kind}:${e.text}`), + ['chunk:first', 'complete:first', 'mark:registry stopped', 'mark:teardown'], + 'the stream completes with what it sent before the registry forgets its channels, and nothing streams after'); + } finally { + await h.framework.stop(); + } +}); + test("with the real channel registry, a stop() with a publish still out sends the stream's complete to the server before the registry forgets its channels", async () => { // The stop tests above use the harness's stub registry. Here the publishes, // the chunks, the complete and stopAll go through a real ChannelRegistry From fd882c794f3db75666c7a7b2d1f665ccbfed9be9 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Thu, 15 Oct 2026 05:46:59 -0400 Subject: [PATCH 04/19] fix(streaming): name the outgoing stream's traces inference:published connectome-host's headless server forwards every framework trace onto its fleet wire (headless.ts:177), where its own synthetic `inference:speech` already means something else: `{ agentName, content }`, once per primary round that ended without tool calls, which fleet--relay reads and docs/fleet-protocol.md documents. A host that took this change would put two shapes on the wire under one name. Neither trace has been released, so they take names of their own: `inference:published` for each chunk and `inference:published_complete` for each channel's end. "Published" names what a voice relay relies on: only text that delivery published, set against `inference:tokens`, the text as written. The rename covers the trace types, the emits and the comments that name them, the outgoing-stream tests and their titles, and the changelog fragment. Nothing else changes. Found in an internal review of connectome-host's TTS relay change. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.added.md | 2 +- src/framework.ts | 6 ++--- src/types/trace.ts | 6 ++--- test/outgoing-stream.test.ts | 32 +++++++++++++------------- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md index 63f8eda45..f549eb0b2 100644 --- a/changelog.d/hold-round-speech.added.md +++ b/changelog.d/hold-round-speech.added.md @@ -1,4 +1,4 @@ -- `inference:speech` and `inference:speech_complete` traces carry the +- `inference:published` and `inference:published_complete` traces carry the outgoing stream to in-process consumers: each chunk as it is sent, with its `channelId`, `inferenceId`, `index` and `text`, and each channel's `complete`. They are emitted whether or not the channel's server takes diff --git a/src/framework.ts b/src/framework.ts index 4ac9a95cd..4df34f442 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -11944,7 +11944,7 @@ export class AgentFramework { // MCPL Spec 14.3 outgoing streaming: the prose this stream's deliveries // publish, sent to the channel's server once each publish is confirmed - // (voice synthesis), and to in-process consumers as an `inference:speech` + // (voice synthesis), and to in-process consumers as an `inference:published` // trace. Delivery feeds it, never the text as the model writes it, so it // carries only what delivery published, where it published it. A // round's text reaches it once the round's routing is decided and its @@ -11972,7 +11972,7 @@ export class AgentFramework { const index = outgoingIndex++; this.channelRegistry?.sendOutgoingChunk(channelId, agent.name, outgoingInferenceId, index, delta); this.emitTrace({ - type: 'inference:speech', + type: 'inference:published', agentName: agent.name, channelId, inferenceId: outgoingInferenceId, @@ -11995,7 +11995,7 @@ export class AgentFramework { for (const [channelId, text] of outgoingText) { this.channelRegistry?.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); this.emitTrace({ - type: 'inference:speech_complete', + type: 'inference:published_complete', agentName: agent.name, channelId, inferenceId: outgoingInferenceId, diff --git a/src/types/trace.ts b/src/types/trace.ts index c8811d270..d55c0d53d 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -174,7 +174,7 @@ export type TraceEvent = * tool's announcement among them, with no stream id, index or end, so * a relay that voices speech takes one of the two. */ - type: 'inference:speech'; + type: 'inference:published'; agentName: string; /** Where the prose is published. */ channelId: string; @@ -189,10 +189,10 @@ export type TraceEvent = * A stream's end for one channel it published to (MCPL Spec 14.3 * `channels/outgoing/complete`), sent on every exit path once the * stream has ended and its deliveries have run, or, when the host - * stops, at once: `text` is everything its `inference:speech` chunks + * stops, at once: `text` is everything its `inference:published` chunks * carried there, and none follows it. */ - type: 'inference:speech_complete'; + type: 'inference:published_complete'; agentName: string; channelId: string; inferenceId: string; diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index abc5dfd27..3e2a59be7 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -1,6 +1,6 @@ /** * The outgoing stream (MCPL Spec 14.3 `channels/outgoing/chunk` and - * `complete`, mirrored as the `inference:speech` traces) carries what + * `complete`, mirrored as the `inference:published` traces) carries what * delivery published, sent once each publish is confirmed. Text that * delivery withholds never reaches it: a round that a silencing tool, * `skip_reply` or a private `think` keeps back, text under a hybrid hold, a @@ -167,9 +167,9 @@ async function harness(scripts: NormalizedResponse[][], config: Partial = []; framework.onTrace((event) => { - if (event.type === 'inference:speech' || event.type === 'inference:speech_complete') { + if (event.type === 'inference:published' || event.type === 'inference:published_complete') { const { type, channelId, text: body, inferenceId } = event; - traces.push({ type, channelId, text: body, inferenceId, ...(event.type === 'inference:speech' ? { index: event.index } : {}) }); + traces.push({ type, channelId, text: body, inferenceId, ...(event.type === 'inference:published' ? { index: event.index } : {}) }); } }); /** @@ -187,7 +187,7 @@ async function harness(scripts: NormalizedResponse[][], config: Partial { +test('a delivered round streams its text once its publish is confirmed, and the published traces mirror the stream', async () => { const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); try { await h.run(); @@ -200,9 +200,9 @@ test('a delivered round streams its text once its publish is confirmed, and the { kind: 'complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, ], 'each publish is streamed once it is confirmed, and a later one to the same channel opens with a paragraph break'); assert.deepEqual(h.traces, [ - { type: 'inference:speech', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, - { type: 'inference:speech', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, - { type: 'inference:speech_complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, + { type: 'inference:published', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, + { type: 'inference:published', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, + { type: 'inference:published_complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, ]); } finally { await h.framework.stop(); } }); @@ -404,7 +404,7 @@ for (const failure of ['unconfirmed', 'throw'] as const) { assert.deepEqual(h.publishes().map((p) => p.text), ['lost', 'kept'], 'both publishes were tried'); assert.deepEqual(h.chunks().map((c) => c.text), ['kept'], 'only the confirmed one streams'); assert.deepEqual(h.completes().map((c) => c.text), ['kept'], 'and the complete carries only it'); - assert.equal(h.traces.some((t) => t.text.includes('lost')), false, 'nor do the speech traces'); + assert.equal(h.traces.some((t) => t.text.includes('lost')), false, 'nor do the published traces'); } finally { await h.framework.stop(); } }); } @@ -417,7 +417,7 @@ test('a stream stopped mid-turn still completes what it published', async () => let published!: () => void; const firstOut = new Promise((resolve) => { published = resolve; }); h.module.during = async () => { await held; }; - const unlog = h.framework.onTrace((event) => { if (event.type === 'inference:speech') published(); }); + const unlog = h.framework.onTrace((event) => { if (event.type === 'inference:published') published(); }); try { h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); void h.framework.runUntilIdle().catch(() => {}); @@ -474,8 +474,8 @@ for (const late of ['rejected', 'confirmed'] as const) { assert.deepEqual(h.log.filter((e) => e.kind !== 'publish').map((e) => `${e.kind}:${e.text}`), ['chunk:first', 'complete:first', 'mark:registry stopped', 'mark:teardown'], 'the stream completes before the registry forgets its channels and the teardown, with what it sent, and nothing streams after its complete'); - assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), ['inference:speech:first', 'inference:speech_complete:first'], - 'and the speech traces say the same'); + assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), ['inference:published:first', 'inference:published_complete:first'], + 'and the published traces say the same'); } finally { releaseTwo(); await h.framework.stop(); @@ -586,15 +586,15 @@ test("with the real channel registry, a stop() with a publish still out sends th } }); -test("the speech traces are emitted when the channel's server takes no stream", async () => { +test("the published traces are emitted when the channel's server takes no stream", async () => { const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]], {}, { streaming: false }); try { await h.run(); assert.deepEqual(h.chunks(), [], 'the server is sent no chunk'); assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), [ - 'inference:speech:first', - 'inference:speech:\n\nsecond', - 'inference:speech_complete:first\n\nsecond', + 'inference:published:first', + 'inference:published:\n\nsecond', + 'inference:published_complete:first\n\nsecond', ], 'in-process consumers still hear what was published'); } finally { await h.framework.stop(); } }); @@ -618,7 +618,7 @@ for (const c of unconfirmedCases) { assert.ok(h.publishes().some((p) => p.text === c.lost), 'the publish was tried'); assert.equal(h.chunks().some((ch) => ch.text.includes(c.lost)), false, 'no chunk carries it'); assert.equal(h.completes().some((co) => co.text.includes(c.lost)), false, 'nor does a complete'); - assert.equal(h.traces.some((t) => t.text.includes(c.lost)), false, 'nor a speech trace'); + assert.equal(h.traces.some((t) => t.text.includes(c.lost)), false, 'nor a published trace'); if (c.kept) { assert.deepEqual(h.chunks().map((ch) => ch.text), [c.kept], 'the confirmed publish beside it still streams'); } From b5aab9e605535e64412ca4c703e5a61ed11d5329 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Thu, 15 Oct 2026 08:28:11 -0400 Subject: [PATCH 05/19] fix(routing): a script round waits only when its script can change its routing Answers Anarchid's review of agent-framework#360 (R1-1 to R1-5). R1-1: every round that called code_execution waited for its scripts before its prose was routed, so narration beside a script, in text channels as in voice, was held for the script's whole runtime, up to the foreground limit (600 s by default), to protect the rare round whose script silences it. - A round now waits only for a script that can change its routing (scriptMayRoute): one whose code names a tool whose call changes the round's routing in the turn's mode (scriptToolNames, changesRouting). In locus and hybrid mode that's the silencing set, `say` and `whisper`; in explicit mode, where sends don't silence, only `skip_reply`; in either, `think` under the round's private same-round policy. Disabled mode routes nothing, so nothing holds. - scriptToolNames (src/code-execution/script-tool-names.ts) reads the script as Python reads it, past its comments and string literals (quotes, triple quotes, prefixes, backslashes). A script reaches a tool through a function named for it (after any prefix joined with `__`) or a `tools[...]` key, so a name counts in code where Python could call it: after a non-identifier character or `__`, and before a non-identifier character. `say_hello` and `my_say` don't name `say`. A comment names nothing. A name anywhere in a string counts for the script's own round, since a string can be a key or code to run. - A round also waits when the interpreter its script will run in has run a script that left such a tool reachable: one that defined a function, lambda or class, named the tool other than in a direct call (an alias), or kept a string that is the tool's whole name, as a key is. A direct call finishes inside its own script and leaves no mark, and a string that only mentions a tool, as a printed sentence or an f-string does, leaves none either: a mark from "I think" in a comment would put R1-1's cost back on most agents that script. The mark is per agent (scriptRoutingInterpreters), keyed by the foreground runner's live interpreter (PyRunner.interpreter: a number no other interpreter in the process has had, from any runner, null while none runs), and it keeps the names, so a later round decides by its own mode and policy. stop() clears it with the runners. - Without code execution, a code_execution call holds nothing: it is answered as an unknown tool, whose result settles no round. - Any other round is routed at its tool-calls event, as before #306. A background script and a list or cancel action return at once and aren't waited for. What the gate can't see is published as on main, and the changelog names it: a name the code doesn't spell out (built at runtime, or written with escapes), code loaded from a file or module, an enumeration of the tools, a call made in one script but awaited in a later one, and code a later script runs from a string an earlier one kept. R1-2: a code_execution result settled whichever round waited under its agent's name with that call id, so a stopped round's late result could settle a newer round that reused the id. The dispatch now records which round its call belongs to (scriptRoundOfResult, keyed by the result), and a result settles only that round. Matching the owed entry instead, as the review suggested, can't tell the two apart: once ids repeat, the late result arrives while the newer round's entry is the one owed. Every newer tool-calls event of the agent also withholds and removes an older round still waiting under its name before it dispatches, whether or not the newer round waits itself, so a newer call that reuses a pending id can't settle the older round either. R1-3: "confirmed" claimed more than holds: routeSpeech counts a reply that leaves `delivered` out as delivered. The texts now say a publish succeeds unless it throws or times out or the server reports it undelivered, and that the stream follows delivery's outcome, and the tests' wording follows: a publish that throws, or that the server reports undelivered. The stream is not made stricter than delivery, which would put it out of step with the receipt. R1-4: #306's fragment is scoped to native tool calls (`roundContent`); on the fallback an inner call counts only as a direct one does there (#307). R1-5: an outgoing-stream test pins that disabled routing streams nothing from a mid-turn round, trailing text or a text-only turn. Tests (code-execution): a script whose code names no such tool doesn't hold its round, which goes out while the script runs; nor does one whose identifiers merely contain a name (say_hello, my_say), or whose comment names one; in explicit mode, one that names only a send; in disabled mode, one that names skip_reply; one that names a world utterance such as say holds it; a tool name a script builds at runtime goes unseen, and its round is published as before; under the public policy, a script that calls think doesn't hold its round; without code execution, a code_execution call holds nothing; a helper an earlier script defined holds a later script's round, and its send silences the round and the turn's later prose, as do an alias bound to the tool and a key kept for it; a script that only calls a routing tool directly, by name or by key, leaves no mark; a helper that calls think holds a later script's round under the private policy and not under the public one; once the interpreter is reclaimed, a script naming no such tool doesn't hold its round, and a mark from it doesn't hold a round once a newer interpreter runs; no two runners' interpreters share a number, and one a signal the runner didn't send ended has none; a script's result settles only the round that dispatched it, though a newer round waits on the same call id; a newer round that doesn't wait displaces an older waiting one, so a call of its own with the same id can't settle it. The test of a send inside a script now runs in hybrid mode as well as locus. Tests whose point is a round that waits for its script give that script a line naming skip_reply that never runs. script-tool-names: the reading itself, a triple-quoted string, a raw string, an escaped quote, an f-string's call, a # inside a string and an unclosed quote among them, and what goes unseen. Readers found the no-code-execution case, the stale mark across a restart and the private-policy `think` in the gate, and that a comment or a printed sentence marked the interpreter. An internal review of this round found the identifier match, the mode, the over-wide mark and the undisplaced older round. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.changed.md | 8 +- changelog.d/hold-round-speech.fixed.md | 82 ++-- docs/tool-wrapper-prose-guard.md | 2 +- src/code-execution/py-runner.ts | 16 + src/code-execution/script-tool-names.ts | 97 +++++ src/framework.ts | 162 +++++-- src/mcpl/channel-registry.ts | 2 +- src/types/trace.ts | 34 +- test/code-execution.test.ts | 511 ++++++++++++++++++++++- test/outgoing-stream.test.ts | 61 ++- test/script-tool-names.test.ts | 84 ++++ test/tool-result-guard.test.ts | 4 +- 12 files changed, 939 insertions(+), 124 deletions(-) create mode 100644 src/code-execution/script-tool-names.ts create mode 100644 test/script-tool-names.test.ts diff --git a/changelog.d/hold-round-speech.changed.md b/changelog.d/hold-round-speech.changed.md index d1dc82a94..40f959206 100644 --- a/changelog.d/hold-round-speech.changed.md +++ b/changelog.d/hold-round-speech.changed.md @@ -1,16 +1,16 @@ - **MCPL servers that take the outgoing stream (`channels.streaming`):** - chunks now arrive once each publish is confirmed, one per published + chunks now arrive once each publish succeeds, one per published message, rather than token by token while the model writes. A round reaches a voice server once the model has finished writing it, its - routing is decided and its publish is confirmed, after any earlier post + routing is decided and its publish succeeds, after any earlier post of the agent's still in flight. A later message to the same channel within one stream opens with a paragraph break (`\n\n`), so the channel's chunks joined in `index` order are its text, which `complete` carries, with its messages kept apart. A server that read chunks as a sign the agent is composing, such as eidoverse-worlds' typing dots, now sees them only as a reply is published; `channels/typing`, which the - host sends for the whole turn to a server granted `channels.typing`, is - that signal. + host sends to a server granted `channels.typing` while a turn runs on + one of its channels, is that signal. - Agents with `toolWrapperProseGuard` now stream their delivered prose. The stream was off for them because it ran ahead of the guard's classification; it now carries only prose that delivery published, and a diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index a96e4f265..03b4c19e3 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -15,35 +15,65 @@ new locus, which streamed to the old one. The stream is now the prose that delivery published: each publish's - text, to its channel, sent once the publish is confirmed, a later one to - the same channel opening with a paragraph break. Tool publications, such - as `send_message` or the `sleep` tool's announcement, aren't part of it, - as before. A publish that fails, times out or isn't confirmed is never - streamed. A `{{unsent}}` resend streams the retained text it sends, - rather than the token. -- A call made inside a foreground `code_execution` script now counts - toward its round's routing, as the same call made directly does (#306). - In explicit mode an inner `skip_reply` keeps the round's envelopes in the - context. In locus and hybrid mode an inner `skip_reply` suppresses the - round's prose, and an inner silencing send suppresses it and the turn's - later prose. An inner `think` under the private same-round policy holds - the round back in every mode. A round that calls `code_execution` is - routed once those calls have returned, so its prose goes out when the - script finishes rather than when it starts. The turn's locus and an - explicit round's `>>` targets are still taken when the round ends, at its - tool-calls event; hybrid's `>>>` targets apply as its delivery runs, as - for any queued hybrid delivery. A background script's call returns as - soon as the script has started, and its later calls aren't the round's. A - round whose turn ends or stops waiting on it, or that a newer round of - the same agent replaces, before its scripts return, as when the turn is - stopped, is withheld: a log line names it, and the turn's delivery - receipt counts it as suppressed. + text, to its channel, sent once the publish succeeds, a later one to the + same channel opening with a paragraph break. Tool publications, such as + `send_message` or the `sleep` tool's announcement, aren't part of it, as + before. A publish that fails is never streamed: one refused before + sending, one that throws or times out, or one the server reports + undelivered (`delivered: false`). A reply that leaves `delivered` out + counts as delivered, as delivery counts it. A `{{unsent}}` resend + streams the retained text it sends, rather than the token. +- With native tool calls (`roundContent`), a call made inside a foreground + `code_execution` script now counts toward its round's routing, as the + same call made directly does (#306). In explicit mode an inner + `skip_reply` keeps the round's envelopes in the context. In locus and + hybrid mode an inner `skip_reply` suppresses the round's prose, and an + inner silencing send suppresses it and the turn's later prose. An inner + `think` under the private same-round policy holds the round back in every + mode. On the fallback without `roundContent`, as in XML tool mode, an + inner call counts only as a direct one does there: a silencing call + silences the turn's later prose, while the private `think` hold and an + explicit round's `skip_reply` don't apply (#307). +- A round whose script can change its routing waits for the script, and its + prose goes out when the script finishes. A script can when its code names + a tool whose call changes the round's routing in the turn's mode: in locus + and hybrid mode `skip_reply`, a silencing send, `say` or `whisper`; in + explicit mode only `skip_reply`; and in either, `think` under the private + same-round policy. Disabled mode holds nothing. A name counts where the + script could call it, as a function or a `tools[...]` key: `say_hello` + doesn't name `say`. A name in a comment doesn't count, since Python never + runs one, while a name anywhere in a string does, since a string can be a + key or code to run. A script can also change its round's routing when the + interpreter it runs in has run a script that left such a tool reachable, + by defining a function, lambda or class, by naming the tool other than in + a direct call, or by keeping a string that is the tool's whole name, as a + key is. That holds until the interpreter is reclaimed or restarted. A + string that only mentions a tool, as a printed sentence does, leaves + nothing reachable. Without code execution, a `code_execution` call holds + nothing. Any other round is routed at its tool-calls event, as before, so + narration beside a script isn't held for the script's runtime. What goes + unseen is routed as it was before scripts counted, such as a tool name the + code doesn't spell out (built at runtime, or written with escapes), code + loaded from a file or module, an enumeration of the tools, a call a script + makes but awaits only in a later one, and code a later script runs from a + string an earlier one kept. The turn's locus and an explicit round's `>>` + targets are taken at the round's tool-calls event, when the model has + finished writing it, not when its scripts return; hybrid's `>>>` targets + apply as its delivery runs, as for any queued hybrid delivery. A + background script's call returns as soon as the script has started, and + its later calls aren't the round's. A script's result settles only the + round that dispatched it, even when a newer round reuses its call id. A + round whose turn ends or stops waiting on it, or that a newer round of the + same agent replaces, before its scripts return, as when the turn is + stopped, is withheld: a log line names it, and the turn's delivery receipt + counts it as suppressed. - Every stream now sends `channels/outgoing/complete` for each channel it streamed to, on every exit path, as MCPL Spec 14.3 asks: once its deliveries have run, or, when the host stops, at once, with what it has - sent. A stream that handed its turn to a budget restart or a - tool-result-guard retry never sent it, so a voice server never learned - that stream's utterance had ended. + sent, so a publish that succeeds after the stop began isn't streamed. A + stream that handed its turn to a budget restart or a tool-result-guard + retry never sent it, so a voice server never learned that stream's + utterance had ended. - A membrane refusal retry no longer moves the outgoing stream to a new inference id. Nothing of the refused attempt was delivered, so none of it streamed; the rounds delivered before the refusal stay in the stream, and diff --git a/docs/tool-wrapper-prose-guard.md b/docs/tool-wrapper-prose-guard.md index 00d601eee..18be1e28b 100644 --- a/docs/tool-wrapper-prose-guard.md +++ b/docs/tool-wrapper-prose-guard.md @@ -4,7 +4,7 @@ The guard matches only when the entire visible prose is one wrapper naming a tool registered on that exact inference. Signed thinking may precede the visible wrapper. It does not parse or execute the wrapper body. -The outgoing preview/voice stream carries only prose that delivery has published, so a wrapper the guard contains never reaches it. Prose the guard doesn't match, wrapper-shaped text in a turn that has made a real tool call among it, is ordinary prose: delivery posts it, and it streams once its publish is confirmed. +The outgoing preview/voice stream carries only prose that delivery has published, so a wrapper the guard contains never reaches it. Prose the guard doesn't match, wrapper-shaped text in a turn that has made a real tool call among it, is ordinary prose: delivery posts it, and it streams once its publish succeeds. On a match, Agent Framework: diff --git a/src/code-execution/py-runner.ts b/src/code-execution/py-runner.ts index 817e38343..8fb641a44 100644 --- a/src/code-execution/py-runner.ts +++ b/src/code-execution/py-runner.ts @@ -103,6 +103,9 @@ interface PendingExec { deadlineMs: number | null; } +/** Interpreters started in this process, by every runner: each one's number. */ +let interpretersStarted = 0; + export class PyRunner { private readonly pythonPath: string; private readonly toolCallTimeoutMs: number; @@ -120,6 +123,8 @@ export class PyRunner { private pending: PendingExec | null = null; private idleTimer: ReturnType | null = null; private execCounter = 0; + /** The number of this runner's latest interpreter (interpretersStarted). */ + private interpreterNumber = 0; private disposed = false; /** An exec has passed the busy check but is still awaiting the interpreter. */ private starting = false; @@ -138,6 +143,16 @@ export class PyRunner { return this.pending !== null || this.starting; } + /** + * The live interpreter, whose state (variables, imports, functions) the + * scripts run in it share: a number no other interpreter in this process + * has had, from any runner, or null while none is running, before the + * first script or once it was reclaimed, killed or has exited. + */ + get interpreter(): number | null { + return this.child && this.child.exitCode === null && !this.child.killed ? this.interpreterNumber : null; + } + /** * Run one script. Tools are (re-)injected before every exec so the * interpreter always reflects the current tool surface (list_changed etc.). @@ -294,6 +309,7 @@ export class PyRunner { env: { ...process.env, PYTHONUNBUFFERED: '1' }, }); this.child = child; + this.interpreterNumber = ++interpretersStarted; child.stderr.setEncoding('utf8'); child.stderr.on('data', (chunk: string) => { diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts new file mode 100644 index 000000000..3677a11a7 --- /dev/null +++ b/src/code-execution/script-tool-names.ts @@ -0,0 +1,97 @@ +/** + * Which of some tools a code_execution script names, read as Python reads + * it, and which of them it leaves reachable by a later script in the same + * interpreter. The framework holds a script's round for the names it + * returns, and marks the agent's interpreter with the reachable ones + * (AgentFramework.scriptMayRoute). + * + * A script reaches a tool through a function named for it (the tool's name, + * after any prefix joined with `__`) or a `tools[...]` key. So a name counts + * in code where Python could call it: after a character that can't continue + * an identifier, or after `__`, and before a character that can't continue + * one. `say_hello` and `my_say` don't name `say`; `mcpl__world__say(` does. + * A comment names nothing, since Python never runs one. A name anywhere in a + * string counts, since a string can be a key or code passed to eval or exec. + * + * A script leaves a tool reachable when its code names the tool other than + * in a direct call (an alias, a reference) or defines a function, lambda or + * class, or when one of its strings is the tool's whole name, as a key is, + * and isn't called at once, as `tools["world--say"](…)` is. A direct call + * finishes inside its own script. A string that only mentions a tool, as a + * printed sentence does, leaves nothing reachable. + * + * Some reaches go unseen, such as a tool name the code doesn't spell out + * (built at runtime, or written with escapes), code loaded from a file or + * module, an enumeration of the tools, a call a script makes but awaits only + * in a later one, and code a later script runs from a string an earlier one + * kept. + * + * @param tools plain tool names, of identifier characters + */ +export function scriptToolNames(script: string, tools: readonly string[]): { names: Set; reachable: Set } { + const named = new RegExp(`(?:(?(); + // Named in code, or by a string that is the whole name. + const callable = new Set(); + let referenced = false; + for (const match of code.matchAll(named)) { + callable.add(match[1]); + CALLED.lastIndex = match.index + match[0].length; + if (!CALLED.test(code)) referenced = true; + } + for (const { text, end } of strings) { + const whole = key.exec(text); + if (whole) { + callable.add(whole[1]); + KEY_CALLED.lastIndex = end; + if (!KEY_CALLED.test(code)) referenced = true; + } + for (const match of text.matchAll(named)) names.add(match[1]); + } + for (const name of callable) names.add(name); + const leavesReachable = callable.size > 0 && (referenced || /\b(?:def|lambda|class)\b/.test(code)); + return { names, reachable: leavesReachable ? callable : new Set() }; +} + +/** A direct call right after a name in code. */ +const CALLED = /\s*\(/y; +/** A direct call right after a key, a `tools[...]` subscript's close and then `(`. */ +const KEY_CALLED = /\s*\]\s*\(/y; + +/** + * A script's code, with each comment and string literal blanked to spaces + * (its newlines kept, so offsets hold), and each literal's text and the + * offset just past it. A literal may be triple-quoted and take any prefix + * (r, b, f, u), and a backslash keeps the character after it inside the + * literal, raw or not, as Python reads it. An f-string's replacement fields + * are read as its text. A literal in single quotes that reaches the end of + * its line ends there, so a stray quote can't take the rest of the script. + */ +function readScript(script: string): { code: string; strings: { text: string; end: number }[] } { + const strings: { text: string; end: number }[] = []; + let code = ''; + let at = 0; + const special = /[#'"]/g; + for (let found = special.exec(script); found; found = special.exec(script)) { + const start = found.index; + let end: number; + if (found[0] === '#') { + const lineEnd = script.indexOf('\n', start); + end = lineEnd === -1 ? script.length : lineEnd; + } else { + const quote = script.startsWith(found[0].repeat(3), start) ? found[0].repeat(3) : found[0]; + let close = start + quote.length; + while (close < script.length && !script.startsWith(quote, close) && (quote.length === 3 || script[close] !== '\n')) { + close += script[close] === '\\' ? 2 : 1; + } + close = Math.min(close, script.length); + end = script.startsWith(quote, close) ? close + quote.length : close; + strings.push({ text: script.slice(start + quote.length, close), end }); + } + code += script.slice(at, start) + script.slice(start, end).replace(/[^\n]/g, ' '); + at = special.lastIndex = end; + } + return { code: code + script.slice(at), strings }; +} diff --git a/src/framework.ts b/src/framework.ts index 4df34f442..5c086cc3c 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -108,6 +108,7 @@ import { } from './tool-image-ledger.js'; import { randomUUID, createHash } from 'node:crypto'; import { PyRunner, buildInjectedTools, formatLimit } from './code-execution/py-runner.js'; +import { scriptToolNames } from './code-execution/script-tool-names.js'; import { buildCodeExecutionToolDefinition, CODE_EXECUTION_TOOL_NAME, @@ -181,6 +182,28 @@ const isSilencingTool = (name: string): boolean => { return SILENCING_TOOLS.has(bare) || WORLD_PUBLICATION_TOOLS.has(bare); }; +/** + * Whether a call of one of these tools can change a round's routing + * (routeRound): in locus and hybrid mode a silencing or world-publication + * tool; in explicit mode, where sends don't silence, only `skip_reply`, which + * keeps the round private; in either, `think` under the private same-round + * policy, which holds the round back. Disabled mode routes nothing. + */ +const changesRouting = ( + named: ReadonlySet, + mode: NonNullable, + privateThink: boolean, +): boolean => { + if (mode === 'disabled') return false; + for (const name of named) { + if (name === 'think' ? privateThink : mode !== 'explicit' || name === 'skip_reply') return true; + } + return false; +}; + +/** The tools whose call can change a round's routing in some mode or policy (changesRouting). */ +const ROUTING_TOOLS = [...SILENCING_TOOLS, ...WORLD_PUBLICATION_TOOLS, 'think']; + /** * True when an injected message is real conversational input — something the * agent might actually be replying to — rather than ambient machinery. @@ -623,11 +646,16 @@ function freshTurnRouting(): TurnRouting { } /** - * What a prose delivery tells its stream once a publish is confirmed: the + * What a prose delivery tells its stream once a publish succeeds: the * channel and the text it published. The stream that queued the delivery * sends them on as its outgoing stream (MCPL Spec 14.3), so the stream * carries only what delivery published: nothing it withholds, and nothing - * whose publish failed or wasn't confirmed. + * whose publish failed. A publish succeeds when routeSpeech returns an + * outcome that says delivered; a reply that leaves `delivered` out counts + * as delivered, as routeSpeech counts it. It fails when routeSpeech refuses + * it before sending (such as for no channel, a closed channel that won't + * open, or a server without `channels.publish`), when it throws or times + * out, or when the server reports it undelivered (`delivered: false`). */ type SpeechPublished = (channelId: string, text: string) => void; @@ -641,18 +669,19 @@ interface ProseRound { } /** - * A round that called `code_execution`, waiting for those calls to return - * before its prose is routed: a foreground script's own calls count as the - * round's, so a `skip_reply` or a send inside the script silences the round - * as a direct call does (AgentFramework.scriptRounds). + * A round whose `code_execution` scripts can change its routing + * (AgentFramework.scriptMayRoute), waiting for those calls to return before + * its prose is routed: a foreground script's own calls count as the round's, + * so a `skip_reply` or a send inside the script silences the round as a + * direct call does (AgentFramework.scriptRounds). */ interface ScriptRound { /** The agent and stream that dispatched the round. */ agent: Agent; streamId: number; - /** Each of the round's code_execution calls, with the tools its foreground script has called so far, in order. */ + /** Each code_execution call the round waits on, with the tools its foreground script has called so far, in order. */ scripts: Map; - /** The round's code_execution calls whose results haven't arrived. */ + /** Those calls whose results haven't arrived. */ pending: Set; /** Routes the round's prose; called once, when its last code_execution result arrives. */ route: () => void; @@ -2002,6 +2031,15 @@ export class AgentFramework { /** Per agent, a round that called code_execution and waits for those calls' * results before its prose is routed (ScriptRound). */ private scriptRounds: Map = new Map(); + /** Each code_execution result, by the round that dispatched its call: it + * settles only that round, even if a newer round reuses its call id. */ + private scriptRoundOfResult: WeakMap = new WeakMap(); + /** Per agent, the foreground interpreter (PyRunner.interpreter) that ran a + * script leaving a tool that can change routing reachable, and the tools + * its scripts named: a later script there can call them, so the agent's + * script rounds wait while that interpreter lives, when one of them can + * change that round's routing (scriptMayRoute). */ + private scriptRoutingInterpreters: Map }> = new Map(); /** RFC-007: per agent, the origin of the foreground script its runner is * executing, handed to that script's inner calls (scripts run one at a * time per runner). Background runners capture theirs in the closure. */ @@ -2751,6 +2789,7 @@ export class AgentFramework { runner.dispose(); } this.codeExecutionRunners.clear(); + this.scriptRoutingInterpreters.clear(); // Background daemons die with the process (documented limitation) — // mark cancelled FIRST so their settle path stays silent (no crash-wake // into a framework that is shutting down). @@ -8532,7 +8571,7 @@ export class AgentFramework { // A code_execution result: once its round has them all, the round's // prose is routed, before anything here can end or continue the turn. - this.settleScriptRound(event.agentName, event.callId); + this.settleScriptRound(event.agentName, event.callId, event.result); const agent = this.agents.get(event.agentName); if (agent) { @@ -11943,15 +11982,15 @@ export class AgentFramework { }; // MCPL Spec 14.3 outgoing streaming: the prose this stream's deliveries - // publish, sent to the channel's server once each publish is confirmed - // (voice synthesis), and to in-process consumers as an `inference:published` - // trace. Delivery feeds it, never the text as the model writes it, so it - // carries only what delivery published, where it published it. A - // round's text reaches it once the round's routing is decided and its - // publish is confirmed. Text that delivery withholds never does: a round - // a silencing tool, `skip_reply` or a private `think` keeps back, a hold, - // a bounce, or a round its stream never finished. Nor does a publish that - // fails or isn't confirmed. Emission to a server is gated in the registry + // publish, sent to the channel's server once each publish succeeds + // (SpeechPublished; voice synthesis), and to in-process consumers as an + // `inference:published` trace. Delivery feeds it, never the text as the + // model writes it, so it carries only what delivery published, where it + // published it. A round's text reaches it once the round's routing is + // decided and its publish succeeds. Text that delivery withholds never + // does: a round a silencing tool, `skip_reply` or a private `think` keeps + // back, a hold, a bounce, or a round its stream never finished. Nor does + // a publish that fails. Emission to a server is gated in the registry // (`channels.streaming`), and `channels/publish` stays the authoritative // send. const outgoingInferenceId = `inf_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; @@ -12246,18 +12285,19 @@ export class AgentFramework { // with verbatim roundContent are live-routed (see liveProseRouting // note above — the fallback preamble is cumulative in XML mode). // - // The round's tool names decide whether its prose goes out, so a - // round that calls code_execution is routed once those calls have - // returned: a foreground script's own calls count as the round's, - // and a skip_reply or a send inside the script silences the round - // as a direct call does (ScriptRound). Its prose then goes out - // when the script finishes. A background script's call returns as - // soon as the script has started. The turn's locus and an - // explicit round's admitted targets are taken now, at the round's - // tool-calls event, so a re-pin or a renamed label while a script - // runs doesn't move them. Hybrid's `>>>` targets, sticky pin and - // hold apply as its delivery runs, as for any queued hybrid - // delivery. + // The round's tool names decide whether its prose goes out, and a + // foreground script's own calls count as the round's: a skip_reply + // or a send inside the script silences the round as a direct call + // does. So a round whose script can call such a tool is routed + // once its scripts have returned (ScriptRound, scriptMayRoute), + // and its prose goes out when they finish. Any other round is + // routed now, a round with a script that can't call one included, + // so narration beside a script isn't held for the script's + // runtime. The turn's locus and an explicit round's admitted + // targets are taken now, at the round's tool-calls event, so a + // re-pin or a renamed label while a script runs doesn't move them. + // Hybrid's `>>>` targets, sticky pin and hold apply as its delivery + // runs, as for any queued hybrid delivery. const roundToolNames = event.calls.map((c) => c.name); const liveRound = !!this.channelRegistry && !!roundContent && roundContent.length > 0; if (liveRound) liveProseRouting = true; @@ -12331,8 +12371,17 @@ export class AgentFramework { } } }; + // A round still waiting under this name can't be decided now: a + // newer round displaces it, whether or not this one waits, so no + // call of this round can settle it. + const displaced = this.scriptRounds.get(agent.name); + if (displaced) { + this.scriptRounds.delete(agent.name); + displaced.withhold('a newer round replaced it before its scripts returned'); + } const scriptCallIds = event.calls - .filter((c) => c.name === CODE_EXECUTION_TOOL_NAME) + .filter((c) => c.name === CODE_EXECUTION_TOOL_NAME + && this.scriptMayRoute(agent.name, c, turnProseRouting, requestSnapshot.sameRoundThinkTextPolicy === 'private')) .map((c) => c.id); if (scriptCallIds.length > 0) { // In place before dispatch, so a foreground script records @@ -12361,9 +12410,6 @@ export class AgentFramework { this.recordProseSuppression(streamRouting, roundSegments.length); }, }; - // A round still waiting under this name can't be decided now: - // a newer round displaces it. - this.scriptRounds.get(agent.name)?.withhold('a newer round replaced it before its scripts returned'); this.scriptRounds.set(agent.name, round); } @@ -13803,6 +13849,9 @@ export class AgentFramework { input: call.input, }); const startTime = Date.now(); + // The round that waits on this call, registered just before dispatch. + const waitingRound = this.scriptRounds.get(agentName); + const round = waitingRound?.pending.has(call.id) ? waitingRound : undefined; this.runCodeExecution(agentName, call) .catch((error): ToolResult => { this.toolLifecycleEmitter?.markDispatchFailure(agentName, call.id); @@ -13829,6 +13878,7 @@ export class AgentFramework { durationMs: Date.now() - startTime, }); } + if (round) this.scriptRoundOfResult.set(result, round); this.pushEvent({ type: 'tool-result', callId: call.id, @@ -13845,13 +13895,42 @@ export class AgentFramework { * the code_execution calls a round waits on (ScriptRound), the round's * prose is routed now, its scripts' calls counted. */ - private settleScriptRound(agentName: string, callId: string): void { + private settleScriptRound(agentName: string, callId: string, result: ToolResult): void { const round = this.scriptRounds.get(agentName); - if (!round || !round.pending.delete(callId) || round.pending.size > 0) return; + if (!round || this.scriptRoundOfResult.get(result) !== round) return; + if (!round.pending.delete(callId) || round.pending.size > 0) return; this.scriptRounds.delete(agentName); round.route(); } + /** + * Whether a code_execution call's script can change its round's routing, + * so the round waits for it (ScriptRound). Its code must name a tool whose + * call can change the round's routing in the turn's mode, under its + * same-round policy (scriptToolNames, changesRouting), or the interpreter + * it will run in must have run a script that left such a tool reachable + * (scriptRoutingInterpreters), until that interpreter is reclaimed or + * restarted. What neither sees is routed at once, as a round was before + * scripts counted (scriptToolNames says what goes unseen). A background + * script, or a list or cancel action, returns at once, and its calls + * aren't the round's. Without code execution no script runs: the call is + * answered as an unknown tool, a result that settles no round. + */ + private scriptMayRoute( + agentName: string, + call: ToolCall, + mode: NonNullable, + privateThink: boolean, + ): boolean { + if (!this.codeExecutionConfig) return false; + const input = (call.input ?? {}) as { code?: unknown; background?: unknown; action?: unknown }; + if (input.background === true || (input.action !== undefined && input.action !== 'run')) return false; + if (typeof input.code === 'string' && changesRouting(scriptToolNames(input.code, ROUTING_TOOLS).names, mode, privateThink)) return true; + const interpreter = this.codeExecutionRunners.get(agentName)?.interpreter ?? null; + const mark = this.scriptRoutingInterpreters.get(agentName); + return interpreter !== null && mark?.interpreter === interpreter && changesRouting(mark.names, mode, privateThink); + } + /** * Run one code_execution script for an agent. Never rejects. * @@ -13969,6 +14048,17 @@ export class AgentFramework { } } const endTurn = script.endTurn; + // A script that left a tool reachable, such as a helper it defined, may + // call it from a later script in the same interpreter (scriptMayRoute). + // Whether that changes the later round's routing is its mode's and + // policy's to say, so the mark keeps the names. + const interpreter = runner.interpreter; + const { reachable } = scriptToolNames(input.code, ROUTING_TOOLS); + if (ownsRunner && interpreter !== null && reachable.size > 0) { + const mark = this.scriptRoutingInterpreters.get(agentName); + const earlier = mark?.interpreter === interpreter ? mark.names : []; + this.scriptRoutingInterpreters.set(agentName, { interpreter, names: new Set([...earlier, ...reachable]) }); + } return { success: true, diff --git a/src/mcpl/channel-registry.ts b/src/mcpl/channel-registry.ts index cb7363211..0c0259935 100644 --- a/src/mcpl/channel-registry.ts +++ b/src/mcpl/channel-registry.ts @@ -2996,7 +2996,7 @@ export class ChannelRegistry { /** * MCPL Spec 14.3 outgoing streaming: forward a text delta to the server * owning the channel. The framework sends each piece of prose once its - * publish is confirmed, so a stream carries only what was published. + * publish succeeds, so a stream carries only what was published. * Emitted only when that server's grant has `channels.streaming` — * servers that never opted in receive nothing. Fire-and-forget and never * throws: streaming is an observer surface; the authoritative delivery is diff --git a/src/types/trace.ts b/src/types/trace.ts index d55c0d53d..54ad0c5c6 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -158,21 +158,25 @@ export type TraceEvent = }) | (TraceEventBase & { /** - * Prose that delivery published, sent once the publish is confirmed: - * the outgoing stream's chunk (MCPL Spec 14.3 - * `channels/outgoing/chunk`), emitted whether or not the channel's - * server takes the stream. It carries only what was published, so a - * consumer that speaks or shows it, such as a voice relay, never gets - * text that delivery withholds (a `skip_reply`, a silencing send, a - * private `think`, a hold, a bounce) or a publish that failed or - * wasn't confirmed. `inference:tokens` stays the model's text as it is - * written (held, as before, while a tool-result guard's batch is - * pending). A later publish to the same channel by the same stream - * opens with a paragraph break (`\n\n`), so a channel's chunks joined - * in `index` order are its text, its messages kept apart. - * `mcpl:speech-routed` also reports each speech publish, the `sleep` - * tool's announcement among them, with no stream id, index or end, so - * a relay that voices speech takes one of the two. + * Prose that delivery published, sent once the publish succeeds: the + * outgoing stream's chunk (MCPL Spec 14.3 `channels/outgoing/chunk`), + * emitted whether or not the channel's server takes the stream. It + * carries only what was published, so a consumer that speaks or shows + * it, such as a voice relay, never gets text that delivery withholds (a + * `skip_reply`, a silencing send, a private `think`, a hold, a bounce) + * or a publish that failed: one refused before sending, one that threw + * or timed out, or one the server reported undelivered + * (`delivered: false`). A reply that leaves `delivered` out counts as + * delivered, as delivery counts it. `inference:tokens` stays the + * model's text as it is written (held, as before, while a tool-result + * guard's batch is pending). A later publish to the same channel by the + * same stream opens with a paragraph break (`\n\n`), so a channel's + * chunks joined in `index` order are its text, its messages kept apart. + * `mcpl:speech-routed` also reports speech publishes, with no stream + * id, index or end: routeSpeech's once delivered, the `sleep` tool's + * announcement among them, and a subconscious's `speak_in_channel` as + * it is sent, before its outcome is known. A relay that voices speech + * takes one of the two, not both. */ type: 'inference:published'; agentName: string; diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 201c279e1..aed8ede99 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -72,6 +72,39 @@ describe('buildInjectedTools', () => { }); describe('PyRunner (real python3)', () => { + it("no two runners' interpreters share a number, so a mark keyed by one can't match another's", async () => { + const first = new PyRunner({ onToolCall: async () => '' }); + const second = new PyRunner({ onToolCall: async () => '' }); + try { + assert.strictEqual(first.interpreter, null, 'no interpreter before the first script'); + await first.exec('x = 1', []); + await second.exec('x = 1', []); + assert.notStrictEqual(first.interpreter, null); + assert.notStrictEqual(second.interpreter, null); + assert.notStrictEqual(first.interpreter, second.interpreter, 'each interpreter has its own number'); + } finally { + first.dispose(); + second.dispose(); + } + assert.strictEqual(first.interpreter, null, 'and none once the runner is disposed'); + }); + + it('an interpreter killed by a signal the runner didn\'t send has no number: it is gone', async () => { + const runner = new PyRunner({ onToolCall: async () => '' }); + try { + await runner.exec('x = 1', []); + const child = (runner as unknown as { child: import('node:child_process').ChildProcess }).child; + assert.notStrictEqual(runner.interpreter, null, 'live while it runs'); + const exited = new Promise((resolve) => child.once('exit', resolve)); + process.kill(child.pid!, 'SIGKILL'); + await exited; + assert.equal(child.killed, false, 'the runner sent no signal'); + assert.strictEqual(runner.interpreter, null, 'and once the signal ended it, none is live'); + } finally { + runner.dispose(); + } + }); + it('runs a script and returns stdout', async () => { const runner = new PyRunner({ onToolCall: async () => '' }); try { @@ -687,8 +720,8 @@ describe("a round that runs code_execution is routed once its scripts return (re { name: 'channel_open', description: 'Open a channel.', inputSchema: { type: 'object' as const, properties: { channelId: { type: 'string' } } } }, ]; - async function scriptRound(mode: 'locus' | 'hybrid' | 'explicit', responses: ReturnType[], - config: { sameRoundThinkTextPolicy?: 'private' } = {}) { + async function scriptRound(mode: 'locus' | 'hybrid' | 'explicit' | 'disabled', responses: ReturnType[], + config: { sameRoundThinkTextPolicy?: 'private' } = {}, options: { codeExecution?: boolean } = {}) { const { tempDir, storePath } = tempStorePath('pytc-round-'); const membrane = new MockMembrane(); for (const response of responses) membrane.pushResponse(response); @@ -700,7 +733,7 @@ describe("a round that runs code_execution is routed once its scripts return (re agents: [{ name: 'assistant', model: 'test-model', systemPrompt: 'Act.', proseRouting: mode, ...config }], modules: [module], syncIntervalMs: 0, - codeExecution: { enabled: true }, + ...(options.codeExecution === false ? {} : { codeExecution: { enabled: true } }), }); // The registry stub: skip_reply and think are on the agent's surface, so // its scripts get them too, and skip_reply ends the turn as the real one @@ -742,6 +775,10 @@ describe("a round that runs code_execution is routed once its scripts return (re const scriptCall = (input: Record, id = 'ce1') => ({ type: 'tool_use', id, name: 'code_execution', input }) as ContentBlock; + // A script holds its round only when it can change the round's routing: + // its code names such a tool (scriptMayRoute). A script given this suffix + // names skip_reply on a line that never runs, so its round waits for it. + const HOLDS = '\nif False: await skip_reply({})'; for (const mode of ['locus', 'hybrid', 'explicit'] as const) { it(`${mode}: a skip_reply inside the script keeps its round's prose from delivery and the stream`, async () => { @@ -761,8 +798,8 @@ describe("a round that runs code_execution is routed once its scripts return (re }); } - it("locus: a send inside the script silences its round and the turn's later prose", async () => { - const h = await scriptRound('locus', [ + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a send inside the script silences its round and the turn's later prose`, async () => { + const h = await scriptRound(mode, [ createMockResponse([ { type: 'text', text: 'posting the summary' }, scriptCall({ code: 'await test__send_message({})' }), @@ -815,7 +852,7 @@ describe("a round that runs code_execution is routed once its scripts return (re const h = await scriptRound('locus', [ createMockResponse([ { type: 'text', text: 'on it' }, - scriptCall({ code: 'print(await test__echo({}))' }), + scriptCall({ code: 'print(await test__echo({}))' + HOLDS }), { type: 'tool_use', id: 'slow1', name: 'test--slow', input: {} }, ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), @@ -832,7 +869,395 @@ describe("a round that runs code_execution is routed once its scripts return (re assert.ok(at('publish:on it@world:test') >= 0, 'the prose was delivered'); assert.ok(at('tool:echo') < at('publish:on it@world:test'), "it waited for the script's own call"); assert.ok(at('publish:on it@world:test') < at('slow:returned'), 'and not for the slow tool beside it'); - assert.ok(at('publish:on it@world:test') < at('chunk:on it@world:test'), 'and its chunk followed the confirmed publish'); + assert.ok(at('publish:on it@world:test') < at('chunk:on it@world:test'), 'and its chunk followed its publish'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a script whose code names no tool that changes routing doesn't hold its round: its prose goes out while the script runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The script waits in its slow call until the round's prose has gone out, + // or until a bound, so a round that waited for the script still ends. + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + /** Runs a one-script round whose prose should go out while its script waits in a slow call. */ + const publishedWhileScriptRuns = async (mode: 'locus' | 'explicit', code: string) => { + const prefix = mode === 'explicit' ? '>>#alpha ' : ''; + const target = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: `${prefix}running the analysis` }, + scriptCall({ code }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: `${prefix}done` }] as ContentBlock[]), + ]); + const published = `publish:running the analysis@${target}`; + const watch = setInterval(() => { if (h.log.includes(published)) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at(published) >= 0, 'the prose was delivered'); + assert.ok(at(published) < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }; + + it("a name inside a longer identifier, as say_hello or my_say, doesn't hold its round", async () => { + await publishedWhileScriptRuns('locus', 'say_hello = 1\nmy_say = 2\nsend_message_log = []\nawait test__slow({})'); + }); + + it("a comment that names a routing tool doesn't hold its round, since Python never runs it", async () => { + await publishedWhileScriptRuns('locus', 'await test__slow({}) # then skip_reply, or say so'); + }); + + it("explicit: a script that names only a send doesn't hold its round, since a send can't change an explicit round's routing", async () => { + await publishedWhileScriptRuns('explicit', 'await test__slow({})\nif False: await test__send_message({})'); + }); + + it("disabled: a script that names skip_reply doesn't hold its round, since nothing is routed", async () => { + const h = await scriptRound('disabled', [ + createMockResponse([ + { type: 'text', text: 'never routed' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + let waitedOn: boolean | undefined; + const watch = setInterval(() => { + if (!h.log.includes('tool:slow')) return; + clearInterval(watch); + waitedOn = rounds.has('assistant'); + h.module.releaseSlow(); + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.equal(waitedOn, false, 'no round waited on the script'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [], 'and nothing was published'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a script whose code names a world utterance, as say, holds its round until it returns", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'about to speak' }, + scriptCall({ code: 'await test__slow({})\nif False: await mcpl__world__say({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Once the script is in its slow call, a round routed at its tool-calls + // event has had its delivery queued long before; with the agent's queued + // deliveries run, the round must still be unpublished. Then the call goes. + let publishedWhileRunning: boolean | undefined; + let checking = false; + const watch = setInterval(() => { + if (checking || !h.log.includes('tool:slow')) return; + checking = true; + const queued = (h.framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + void Promise.resolve(queued).then(() => {}, () => {}).then(() => { + publishedWhileRunning = h.log.includes('publish:about to speak@world:test'); + h.module.releaseSlow(); + }); + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.equal(publishedWhileRunning, false, "the round wasn't published while its script ran"); + assert.ok(at('slow:returned') < at('publish:about to speak@world:test'), 'it went out once the script had returned'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a tool name a script builds at runtime goes unseen: its round is routed at its tool-calls event, as before scripts counted", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'not held' }, + scriptCall({ code: 'name = "skip" + "_reply"\nawait tools[name]({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('tool:skip_reply') >= 0, 'the script called skip_reply by a name it built'); + assert.ok(at('publish:not held@world:test') >= 0, + "the round was published anyway: its code names no tool that changes routing, so it didn't wait for the skip_reply that would have kept it back"); + } finally { await h.done(); } + }); + + it("under the public same-round policy, a script that calls think doesn't hold its round", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'thinking it over' }, + scriptCall({ code: 'await think({"content": "notes"})\nawait test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:thinking it over@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:thinking it over@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:thinking it over@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("without code execution, a code_execution call holds nothing: its round is routed at its tool-calls event", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'not a script after all' }, + scriptCall({ code: 'await skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ], {}, { codeExecution: false }); + try { + await h.run(); + assert.ok(h.log.includes('publish:not a script after all@world:test'), + "the call is answered as an unknown tool, whose result settles no round, so the round doesn't wait for it"); + } finally { await h.done(); } + }); + + it("a helper an earlier script defined holds a later script's round: the interpreter ran a script that names a tool that changes routing", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await post()' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the helper sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:defining a helper@world:test'], + "the send through the helper silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + it("a script that only calls a routing tool directly, by name or by key, leaves no mark: a later script's round isn't held for it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'thinking first' }, + scriptCall({ code: 'await think({"content": "notes"})\nawait tools["think"]({"content": "more"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(!h.log.includes('publish:thinking first@world:test'), 'the round whose script thought stays private'); + assert.ok(at('publish:running the analysis@world:test') >= 0, "the later round's prose was delivered"); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while its script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a key an earlier script kept holds a later script's round, as an alias does", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'keeping a key' }, + scriptCall({ code: 'name = "test--send_message"' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await tools[name]({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the kept key sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:keeping a key@world:test'], + "the send through the kept key silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + it("an alias an earlier script bound to a routing tool holds a later script's round, as a helper does", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'keeping a handle' }, + scriptCall({ code: 'post = test__send_message' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await post({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the alias sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:keeping a handle@world:test'], + "the send through the alias silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + it("a mark left in an earlier interpreter doesn't hold a round once a newer interpreter runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'starting fresh' }, + scriptCall({ code: 'x = 1' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce3'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The helper's interpreter is reclaimed once its script returns; the next + // script starts a newer one, which is live when the third round is decided. + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'ce1') { + const runner = (h.framework as unknown as { codeExecutionRunners: Map }) + .codeExecutionRunners.get('assistant')!; + runner.reclaim('test'); + } + }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the third round was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while its script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("under the public same-round policy, a helper that calls think doesn't hold a later script's round", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def note():\n await think({"content": "notes"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await note()\nawait test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(h.log.includes('tool:think'), 'the helper thought'); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("under the private same-round policy, a helper that calls think holds a later script's round back", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def note():\n await think({"content": "notes"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'working it through' }, + scriptCall({ code: 'await note()' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'the answer' }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the helper thought'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), + ['publish:defining a helper@world:test', 'publish:the answer@world:test'], + 'the round whose helper thought stays private, as with a direct think, and the rounds around it are delivered'); + } finally { await h.done(); } + }); + + it("once the interpreter is reclaimed, a script that names no such tool doesn't hold its round, since the helpers went with it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The interpreter that ran the helper's script is reclaimed once that + // script returns, as an idle reclaim or a restart would. + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'ce1') { + const runner = (h.framework as unknown as { codeExecutionRunners: Map }) + .codeExecutionRunners.get('assistant')!; + runner.reclaim('test'); + } + }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); } finally { clearInterval(watch); clearTimeout(bound); @@ -868,7 +1293,7 @@ describe("a round that runs code_execution is routed once its scripts return (re createMockResponse([ { type: 'text', text: 'two scripts at once' }, scriptCall({ code: 'await test__slow({})\nawait skip_reply({})' }, 'ce1'), - scriptCall({ code: 'print("second")' }, 'ce2'), + scriptCall({ code: 'print("second")' + HOLDS }, 'ce2'), ] as ContentBlock[], 'tool_use'), ]); // Let the first script go once the second's refusal has come back. @@ -897,7 +1322,7 @@ describe("a round that runs code_execution is routed once its scripts return (re createMockResponse([ { type: 'text', text: 'still mine to say' }, scriptCall({ code: 'await test__send_message({})', background: true }, 'bg'), - scriptCall({ code: 'await test__slow({})' }, 'fg'), + scriptCall({ code: 'await test__slow({})' + HOLDS }, 'fg'), ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), ]); @@ -921,7 +1346,7 @@ describe("a round that runs code_execution is routed once its scripts return (re const h = await scriptRound('locus', [ createMockResponse([ { type: 'text', text: 'for the room I was in' }, - scriptCall({ code: 'await channel_open({"channelId": "chan-other"})' }), + scriptCall({ code: 'await channel_open({"channelId": "chan-other"})' + HOLDS }), ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'now over here' }] as ContentBlock[]), ]); @@ -939,7 +1364,7 @@ describe("a round that runs code_execution is routed once its scripts return (re const membrane = (h.framework as unknown as { membrane: MockMembrane }).membrane; membrane.pushResponse(createMockResponse([ { type: 'text', text: 'never decided' }, - scriptCall({ code: 'await test__slow({})' }), + scriptCall({ code: 'await test__slow({})' + HOLDS }), ] as ContentBlock[], 'tool_use')); const lines: string[] = []; const error = console.error; @@ -986,7 +1411,7 @@ describe("a round that runs code_execution is routed once its scripts return (re const h = await scriptRound('locus', [ createMockResponse([ { type: 'text', text: 'never decided' }, - scriptCall({ code: 'await test__slow({})' }), + scriptCall({ code: 'await test__slow({})' + HOLDS }), ] as ContentBlock[], 'tool_use'), ]); const lines: string[] = []; @@ -1065,23 +1490,77 @@ describe("a round that runs code_execution is routed once its scripts return (re }); /** A pending round of another stream, as scriptRounds holds one: it records what it was asked to do. */ - const pendingRound = (agent: unknown) => { + const pendingRound = (agent: unknown, pending: string[] = ['elsewhere']) => { const asked: string[] = []; return { asked, round: { - agent, streamId: -1, scripts: new Map(), pending: new Set(['elsewhere']), + agent, streamId: -1, scripts: new Map(), pending: new Set(pending), route: () => { asked.push('route'); }, withhold: (why: string) => { asked.push(`withhold: ${why}`); }, }, }; }; + it("a script's result settles only the round that dispatched it, though a newer round waits on the same call id", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the first round' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + // While the script runs, a newer round comes to wait under the name on a + // call with the same id, as a provider that reuses ids could make one. + const newer = pendingRound(h.framework.getAgent('assistant'), ['ce1']); + let swapped = false; + const watch = setInterval(() => { + if (!h.log.includes('tool:slow')) return; + clearInterval(watch); + rounds.set('assistant', newer.round); + swapped = true; + h.module.releaseSlow(); + }, 10); + // A script that never reaches its slow call still ends, and fails below. + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.ok(swapped, 'the newer round came to wait while the script ran'); + assert.deepStrictEqual(newer.asked, [], "the first round's result doesn't settle the newer round"); + assert.equal(rounds.get('assistant'), newer.round, 'which still waits'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a newer round that doesn't wait displaces an older waiting round, so a call of its own with the same id can't settle it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the newer round' }, + scriptCall({ code: 'print("quick")' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // An older round of another stream still waits on a call with the same + // id, as a provider that reuses ids could leave one. + const older = pendingRound({}, ['ce1']); + (h.framework as unknown as { scriptRounds: Map }).scriptRounds.set('assistant', older.round); + try { + await h.run(); + assert.deepStrictEqual(older.asked, ['withhold: a newer round replaced it before its scripts returned'], + "the older round is withheld, and the newer round's result settles nothing of it"); + assert.ok(h.log.includes('publish:the newer round@world:test'), 'and the newer round is routed at once'); + } finally { await h.done(); } + }); + it('a round registered while another waits under the same agent name withholds the older one', async () => { const h = await scriptRound('locus', [ createMockResponse([ { type: 'text', text: 'the newer round' }, - scriptCall({ code: 'print("quick")' }), + scriptCall({ code: 'print("quick")' + HOLDS }), ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), ]); @@ -1126,7 +1605,7 @@ describe("a round that runs code_execution is routed once its scripts return (re const h = await scriptRound('explicit', [ createMockResponse([ { type: 'text', text: '>>#alpha written for alpha' }, - scriptCall({ code: 'await test__slow({})' }), + scriptCall({ code: 'await test__slow({})' + HOLDS }), ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: '>>skip_reply done' }] as ContentBlock[]), ]); diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 3e2a59be7..8e27f0c21 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -1,12 +1,13 @@ /** * The outgoing stream (MCPL Spec 14.3 `channels/outgoing/chunk` and * `complete`, mirrored as the `inference:published` traces) carries what - * delivery published, sent once each publish is confirmed. Text that + * delivery published, sent once each publish succeeds. Text that * delivery withholds never reaches it: a round that a silencing tool, * `skip_reply` or a private `think` keeps back, text under a hybrid hold, a - * bounce, or a round the stream never finished. Nor does a publish that fails - * or isn't confirmed. A voice server speaks what it gets, and words already - * spoken can't be taken back, so the stream can't run ahead of delivery. + * bounce, or a round the stream never finished. Nor does a publish that + * throws or that the server reports undelivered. A voice server speaks what + * it gets, and words already spoken can't be taken back, so the stream can't + * run ahead of delivery. */ import { afterEach, test } from 'node:test'; @@ -96,14 +97,14 @@ type Logged = /** * The registry stub: #alpha, #cafe and @laria resolve, the turn's locus is * world:test, and every chunk, publish and complete lands in one ordered - * log. A publish whose text is in `fails` returns unconfirmed. + * log. A publish whose text is in `fails` fails. */ function registry(framework: AgentFramework, options: { streaming?: boolean } = {}) { const log: Logged[] = []; /** A publish whose text is held here doesn't return until the test lets it go. */ const holds = new Map>(); - /** A publish of one of these texts isn't confirmed: `delivered: false`, or a throw. */ - const fails = new Map(); + /** A publish of one of these texts fails: the server reports it undelivered (`delivered: false`), or it throws. */ + const fails = new Map(); const known: Record = { '#alpha': 'chan-alpha', '#cafe': 'chan-cafe', '@laria': 'discord:dm:99' }; let stopped = false; const target: Record = { @@ -187,7 +188,7 @@ async function harness(scripts: NormalizedResponse[][], config: Partial { +test('a delivered round streams its text once its publish succeeds, and the published traces mirror the stream', async () => { const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); try { await h.run(); @@ -198,7 +199,7 @@ test('a delivered round streams its text once its publish is confirmed, and the { kind: 'publish', channelId: 'world:test', text: 'second' }, { kind: 'chunk', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, { kind: 'complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, - ], 'each publish is streamed once it is confirmed, and a later one to the same channel opens with a paragraph break'); + ], 'each publish is streamed once it succeeds, and a later one to the same channel opens with a paragraph break'); assert.deepEqual(h.traces, [ { type: 'inference:published', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, { type: 'inference:published', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, @@ -388,21 +389,21 @@ test("explicit: a stream's complete waits for the deliveries it queued, so no ch 'mark:post released', 'chunk:first', 'complete:first', - ], 'the chunk follows the confirmed post, the complete follows the chunk, and the private round streams nothing'); + ], 'the chunk follows the post once it succeeds, the complete follows the chunk, and the private round streams nothing'); } finally { release(); await h.framework.stop(); } }); -for (const failure of ['unconfirmed', 'throw'] as const) { - test(`a publish that ${failure === 'throw' ? 'throws' : "isn't confirmed"} never streams, and the stream goes on with what is`, async () => { +for (const failure of ['undelivered', 'throw'] as const) { + test(`a publish that ${failure === 'throw' ? 'throws' : 'the server reports undelivered'} never streams, and the stream goes on with what is`, async () => { const h = await harness([[round(text('lost'), call('one', 'test--read')), final(text('kept'))]]); h.fails.set('lost', failure); try { await h.run(); assert.deepEqual(h.publishes().map((p) => p.text), ['lost', 'kept'], 'both publishes were tried'); - assert.deepEqual(h.chunks().map((c) => c.text), ['kept'], 'only the confirmed one streams'); + assert.deepEqual(h.chunks().map((c) => c.text), ['kept'], 'only the one that succeeded streams'); assert.deepEqual(h.completes().map((c) => c.text), ['kept'], 'and the complete carries only it'); assert.equal(h.traces.some((t) => t.text.includes('lost')), false, 'nor do the published traces'); } finally { await h.framework.stop(); } @@ -438,9 +439,9 @@ test('a stream stopped mid-turn still completes what it published', async () => // A stop() that finds a publish still out ends the stream before its teardown // closes the connections the publish waits on, whatever becomes of the -// publish then: the close rejects it, or confirms it too late to stream. -for (const late of ['rejected', 'confirmed'] as const) { - test(`a stop() with a publish still out completes what the stream sent before teardown, and the publish ${late} then never streams`, async () => { +// publish then: the close rejects it, or it succeeds too late to stream. +for (const late of ['rejects', 'succeeds'] as const) { + test(`a stop() with a publish still out completes what the stream sent before teardown, and a publish that ${late === 'rejects' ? 'teardown rejects' : 'succeeds after teardown'} never streams`, async () => { const h = await harness([[ round(text('first'), call('one', 'test--read')), round(text('second'), call('two', 'test--read')), @@ -450,7 +451,7 @@ for (const late of ['rejected', 'confirmed'] as const) { // closing an MCPL connection settles a request it still holds. let settleSecond!: () => void; const secondHeld = new Promise((resolve, reject) => { - settleSecond = () => (late === 'rejected' ? reject(new Error('connection closed')) : resolve()); + settleSecond = () => (late === 'rejects' ? reject(new Error('connection closed')) : resolve()); }); secondHeld.catch(() => {}); h.holds.set('second', secondHeld); @@ -586,6 +587,20 @@ test("with the real channel registry, a stop() with a publish still out sends th } }); +test('disabled: nothing streams, from a mid-turn round, trailing text or a text-only turn', async () => { + for (const script of [ + [round(text('mid-turn'), call('one', 'test--read')), final(text('trailing'))], + [final(text('text only'))], + ]) { + const h = await harness([script], { proseRouting: 'disabled' }); + try { + await h.run(); + assert.deepEqual(h.log, [], 'nothing is published, streamed or completed'); + assert.deepEqual(h.traces, [], 'and no published trace is emitted'); + } finally { await h.framework.stop(); } + } +}); + test("the published traces are emitted when the channel's server takes no stream", async () => { const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]], {}, { streaming: false }); try { @@ -599,8 +614,8 @@ test("the published traces are emitted when the channel's server takes no stream } finally { await h.framework.stop(); } }); -// A publish that isn't confirmed streams nothing, on every delivery path. -const unconfirmedCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'explicit'; script: () => NormalizedResponse[]; lost: string; kept?: string }> = [ +// A publish that fails streams nothing, on every delivery path. +const failedPublishCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'explicit'; script: () => NormalizedResponse[]; lost: string; kept?: string }> = [ { name: 'locus text-only', mode: 'locus', script: () => [final(text('lost'))], lost: 'lost' }, { name: 'locus trailing', mode: 'locus', script: () => [round(text('kept'), call('one', 'test--read')), final(text('lost'))], lost: 'lost', kept: 'kept' }, { name: 'hybrid unprefixed', mode: 'hybrid', script: () => [round(text('lost'), call('one', 'test--read')), final(text('kept'))], lost: 'lost', kept: 'kept' }, @@ -608,9 +623,9 @@ const unconfirmedCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'explic { name: 'explicit mid-turn', mode: 'explicit', script: () => [round(text('>>#alpha lost'), call('one', 'test--read')), final(text('>>#cafe kept'))], lost: 'lost', kept: 'kept' }, { name: 'explicit text-only', mode: 'explicit', script: () => [final(text('>>#alpha lost'))], lost: 'lost' }, ]; -for (const c of unconfirmedCases) { - for (const failure of ['unconfirmed', 'throw'] as const) { - test(`${c.name}: a publish that ${failure === 'throw' ? 'throws' : "isn't confirmed"} streams nothing`, async () => { +for (const c of failedPublishCases) { + for (const failure of ['undelivered', 'throw'] as const) { + test(`${c.name}: a publish that ${failure === 'throw' ? 'throws' : 'the server reports undelivered'} streams nothing`, async () => { const h = await harness([c.script()], { proseRouting: c.mode }); h.fails.set(c.lost, failure); try { @@ -620,7 +635,7 @@ for (const c of unconfirmedCases) { assert.equal(h.completes().some((co) => co.text.includes(c.lost)), false, 'nor does a complete'); assert.equal(h.traces.some((t) => t.text.includes(c.lost)), false, 'nor a published trace'); if (c.kept) { - assert.deepEqual(h.chunks().map((ch) => ch.text), [c.kept], 'the confirmed publish beside it still streams'); + assert.deepEqual(h.chunks().map((ch) => ch.text), [c.kept], 'the publish beside it that succeeds still streams'); } } finally { await h.framework.stop(); } }); diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts new file mode 100644 index 000000000..d48a581ac --- /dev/null +++ b/test/script-tool-names.test.ts @@ -0,0 +1,84 @@ +/** + * scriptToolNames reads a code_execution script as Python reads it: which + * routing tools it names, which hold its own round, and which it leaves + * reachable by a later script, which mark the interpreter. code-execution's + * tests run the gate and the mark through real scripts; these pin the + * reading itself. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { scriptToolNames } from '../src/code-execution/script-tool-names.js'; + +const TOOLS = ['skip_reply', 'send_message', 'say', 'think']; +const read = (script: string) => { + const { names, reachable } = scriptToolNames(script, TOOLS); + return { names: [...names].sort(), reachable: [...reachable].sort() }; +}; + +test('a name counts where Python could call it, not inside a longer identifier', () => { + assert.deepEqual(read('say_hello = 1\nmy_say = 2\nsend_message_log = []'), { names: [], reachable: [] }); + assert.deepEqual(read('await mcpl__world__say({})'), { names: ['say'], reachable: [] }); +}); + +test('a direct call leaves nothing reachable, by name or by key', () => { + assert.deepEqual(read('await think({"content": "notes"})\nawait tools["think"]({"content": "more"})'), + { names: ['think'], reachable: [] }); + assert.deepEqual(read('await tools[ "world--say" ] ({})'), { names: ['say'], reachable: [] }); +}); + +test('an alias, a definition or a kept key leaves the tool reachable', () => { + assert.deepEqual(read('post = test__send_message'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('async def post():\n await test__send_message({})'), + { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('quiet = lambda: skip_reply({})'), { names: ['skip_reply'], reachable: ['skip_reply'] }); + assert.deepEqual(read('name = "test--send_message"'), { names: ['send_message'], reachable: ['send_message'] }); +}); + +test('a comment names nothing, and its words are not code', () => { + assert.deepEqual(read('await test__slow({}) # then skip_reply, or say so'), { names: [], reachable: [] }); + assert.deepEqual(read('say("hi") # first class result'), { names: ['say'], reachable: [] }); +}); + +test('a sentence in a string counts for its own round but leaves nothing reachable', () => { + assert.deepEqual(read('# I think this works\ndef fmt(x):\n return x\nprint("They say the build is green")'), + { names: ['say'], reachable: [] }); + assert.deepEqual(read('def fmt(result):\n print(f"I would say {result}")'), { names: ['say'], reachable: [] }); +}); + +test('a triple-quoted string runs across lines, with quotes and # inside it', () => { + assert.deepEqual(read('note = """They said "skip_reply" # not a comment\nI think"""\npost = test__send_message'), + { names: ['send_message', 'skip_reply', 'think'], reachable: ['send_message'] }); +}); + +test('in a raw string, a backslash still keeps the quote after it inside the string', () => { + assert.deepEqual(read('pattern = r"\\" # say"\npost = test__send_message'), + { names: ['say', 'send_message'], reachable: ['send_message'] }); +}); + +test('an escaped quote stays inside its string', () => { + assert.deepEqual(read('msg = "she said \\"skip_reply\\" # done"'), { names: ['skip_reply'], reachable: [] }); +}); + +test("an f-string's call inside its braces counts for its own round but leaves nothing reachable", () => { + assert.deepEqual(read('print(f"result: {say({\'text\': \'hi\'})}")'), { names: ['say'], reachable: [] }); +}); + +test('a # inside a string starts no comment', () => { + assert.deepEqual(read('label = "say #3"; post = test__send_message'), + { names: ['say', 'send_message'], reachable: ['send_message'] }); +}); + +test("a string in single quotes ends at its line's end, so a stray quote can't take the rest of the script", () => { + assert.deepEqual(read('x = "unclosed\npost = test__send_message'), { names: ['send_message'], reachable: ['send_message'] }); +}); + +test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call awaited later, code kept in a string", () => { + assert.deepEqual(read('name = "skip" + "_reply"\nawait tools[name]({})'), { names: [], reachable: [] }, 'built at runtime'); + assert.deepEqual(read('await tools["s\\x61y"]({})'), { names: [], reachable: [] }, 'written with escapes'); + assert.deepEqual(read('fns = list(tools.values())'), { names: [], reachable: [] }, 'an enumeration'); + assert.deepEqual(read('exec(open("helper.py").read())'), { names: [], reachable: [] }, 'code from a file'); + assert.deepEqual(read('pending = say({"text": "hi"})'), { names: ['say'], reachable: [] }, 'a call made here...'); + assert.deepEqual(read('await pending'), { names: [], reachable: [] }, '...and awaited later'); + assert.deepEqual(read('later = "say({\'text\': \'hi\'})"'), { names: ['say'], reachable: [] }, 'code kept in a string...'); + assert.deepEqual(read('await eval(later)'), { names: [], reachable: [] }, '...and run later'); +}); diff --git a/test/tool-result-guard.test.ts b/test/tool-result-guard.test.ts index 7b6822a6f..da90935fd 100644 --- a/test/tool-result-guard.test.ts +++ b/test/tool-result-guard.test.ts @@ -411,7 +411,7 @@ test('explicit mode: after a budget restart, the resumed reply streams where del ], { proseRouting: 'explicit', maxStreamTokens: 1000 }); const routed: Array<{ text: string; channelId: string }> = []; const streamed: Array<{ channelId: string; delta: string }> = []; - // Each confirmed publish and each chunk, in the order they happened. + // Each publish that succeeds and each chunk, in the order they happened. const order: string[] = []; // What delivery had started to publish when the restarted stream wrote. let atRestart: string[] | undefined; @@ -487,7 +487,7 @@ test('explicit mode: after a budget restart, the resumed reply streams where del 'the resumed stream follows the destination the turn last addressed'); assert.ok(order.indexOf('chunk:continued') > order.indexOf('published:continued') && order.indexOf('published:continued') > order.indexOf('published:first'), - `the resumed reply streams once its own publish is confirmed, after the earlier posts: ${order.join(', ')}`); + `the resumed reply streams once its own publish succeeds, after the earlier posts: ${order.join(', ')}`); } finally { releaseFirst(); secondDelivered(); From 00ce087436056704b25f7e29c9cafedb167efedd Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Thu, 15 Oct 2026 19:55:39 -0400 Subject: [PATCH 06/19] test(streaming): pin what the PR description says the stream does Each of these was a sentence in agent-framework#360's description with no test of its own: - A publish that fails streams nothing, and the {{unsent}} resend of its retained text streams once, so a voice server speaks the reply once (outgoing-stream, hybrid). - The sleep tool's announcement publishes through routeSpeech, as prose does, but isn't part of the stream (outgoing-stream). The test ends the sleep before it stops the framework: EventGate.dispose leaves a pending sleep's timer running, which kept the file alive for the whole sleep in an unforced run. - A guard retry: the stream that handed its turn to the retry completes what it streamed, and the retry completes under its own inference id (tool-result-guard). - Wrapper-shaped text the guard doesn't match, after a real tool call, is posted as ordinary prose and streams as delivery publishes it (tool-wrapper-prose-guard-integration). - routeSpeech reports each delivered publish as mcpl:speech-routed, with no stream id or index and no trace that ends an utterance (channel-registry-routing). - routeSpeech counts a publish reply that leaves delivered out as delivered, which the stream follows (channel-registry-routing). Co-Authored-By: Claude Opus 5.5 --- test/channel-registry-routing.test.ts | 31 ++++++++++++++++ test/outgoing-stream.test.ts | 35 ++++++++++++++++++- test/tool-result-guard.test.ts | 11 +++++- ...ol-wrapper-prose-guard-integration.test.ts | 5 ++- 4 files changed, 79 insertions(+), 3 deletions(-) diff --git a/test/channel-registry-routing.test.ts b/test/channel-registry-routing.test.ts index 80a825658..2a3c1e0ed 100644 --- a/test/channel-registry-routing.test.ts +++ b/test/channel-registry-routing.test.ts @@ -136,6 +136,37 @@ test('routeSpeech surfaces a failure when the server reports delivered:false', a assert.ok(traces.some(t => t.type === 'mcpl:speech-route-failed')); }); +test('routeSpeech counts a publish reply that leaves delivered out as delivered', async () => { + const { registry, failures } = makeRegistry({}); + seedRegistered(registry, 'discord', 'ch-x'); + registry.handleIncoming('discord', incoming('ch-x', 'hi')); + + const res = await registry.routeSpeech('cairn', 'a reply', registry.resolveLocus('cairn')); + + assert.deepEqual(res, { delivered: true, channelId: 'ch-x' }, 'the outcome says delivered, so the stream follows it'); + assert.equal(failures.length, 0, 'and no route failure is reported'); +}); + +test('routeSpeech reports each delivered publish as mcpl:speech-routed, with nothing that ties it to a stream or ends one', async () => { + const { registry, traces } = makeRegistry({ delivered: true }); + seedRegistered(registry, 'discord', 'ch-x'); + registry.handleIncoming('discord', incoming('ch-x', 'hi')); + + const before = traces.length; + await registry.routeSpeech('cairn', 'first reply', registry.resolveLocus('cairn')); + await registry.routeSpeech('cairn', 'second reply', registry.resolveLocus('cairn')); + + const routed = traces.slice(before); + assert.deepEqual(routed.map((t) => ({ type: t.type, channelId: t.channelId, text: t.text, delivered: t.delivered })), [ + { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'first reply', delivered: true }, + { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'second reply', delivered: true }, + ], 'one trace per publish, with its whole text, and none that ends an utterance'); + for (const t of routed) { + assert.deepEqual(['inferenceId', 'streamId', 'index'].filter((key) => key in t), [], + 'no stream id or index, unlike inference:published'); + } +}); + test('routeSpeech routes a conversation fork to its HOME channel, not the global last-inbound (item 3)', async () => { // Two channels are live. chanA registered first; then a message arrives on // chanB, flipping the process-global defaultPublishChannel to chanB. A fork diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 8e27f0c21..7bd33d118 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -157,13 +157,15 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = }; } -async function harness(scripts: NormalizedResponse[][], config: Partial = {}, options: { streaming?: boolean } = {}) { +async function harness(scripts: NormalizedResponse[][], config: Partial = {}, options: { streaming?: boolean; gate?: boolean } = {}) { const dir = mkdtempSync(join(tmpdir(), 'af-outgoing-stream-')); dirs.push(dir); const membrane = new ScriptMembrane(scripts); const module = new ToolModule(); const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: membrane.asMembrane(), agents: [{ name: 'assistant', model: 'test', systemPrompt: 'system', ...config }], modules: [module], syncIntervalMs: 0, + // An EventGate gives the agent the sleep tool. + ...(options.gate ? { gate: { configPath: join(dir, 'gate.json') } } : {}), }); const stub = registry(framework, options); const traces: Array<{ type: string; channelId: string; text: string; inferenceId: string; index?: number }> = []; @@ -280,6 +282,23 @@ test('explicit: a {{unsent}} resend streams the retained text it publishes', asy } finally { await h.framework.stop(); } }); +test('hybrid: a publish that fails streams nothing, and the {{unsent}} resend of its retained text streams once', async () => { + const reply = 'the reply I meant for alpha'; + // The >>> post fails, so delivery retains its text and wakes a second turn, which resends it. + const h = await harness([[final(text(`>>>#alpha ${reply}`))], [final(text('>>>#alpha {{unsent}}'))]], { proseRouting: 'hybrid' }); + h.fails.set(reply, 'undelivered'); + // The failure applies to the first post alone, so the resend succeeds. + const failureOf = h.fails.get.bind(h.fails); + h.fails.get = (body: string) => { const failure = failureOf(body); h.fails.delete(body); return failure; }; + try { + await h.run(); + assert.deepEqual(h.publishes(), [{ channelId: 'chan-alpha', text: reply }, { channelId: 'chan-alpha', text: reply }], + 'the post failed, and the resend posted the retained text'); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'chan-alpha', text: reply }], + 'only the resend streams, once, so a voice server speaks the reply once'); + } finally { await h.framework.stop(); } +}); + /** Two streams of one turn: the first round crosses the stream budget, so the turn restarts. */ function budgetRestart(first: string, continued: string) { const overBudget = round(text(first), call('one', 'test--read')); @@ -601,6 +620,20 @@ test('disabled: nothing streams, from a mid-turn round, trailing text or a text- } }); +test("the sleep tool's announcement publishes through routeSpeech, as prose does, but isn't part of the stream", async () => { + const h = await harness([[round(text('going quiet now'), call('s1', 'sleep', { seconds: 600 }))]], {}, { gate: true }); + try { + await h.run(); + assert.ok(h.publishes().some((p) => p.channelId === 'world:test' && p.text.startsWith('\u{1F4A4}')), 'the sleep tool announced at the locus'); + assert.deepEqual(h.chunks().map((c) => c.text), ['going quiet now'], "the round's prose streams, and the announcement doesn't"); + } finally { + // EventGate.dispose leaves a pending sleep's timer running, which would + // keep this file alive for the whole sleep, so the test ends the sleep. + (h.framework as unknown as { eventGate: { clearSleep(): boolean } }).eventGate.clearSleep(); + await h.framework.stop(); + } +}); + test("the published traces are emitted when the channel's server takes no stream", async () => { const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]], {}, { streaming: false }); try { diff --git a/test/tool-result-guard.test.ts b/test/tool-result-guard.test.ts index da90935fd..55249caca 100644 --- a/test/tool-result-guard.test.ts +++ b/test/tool-result-guard.test.ts @@ -361,7 +361,7 @@ test('native Membrane observes the first refusal even when guard is enabled by a } finally { await framework.stop(); } }); -test('explicit mode: a guard retry streams on to the sticky destination delivery uses', async () => { +test('explicit mode: a guard retry streams on to the sticky destination delivery uses, and each stream completes under its own id', async () => { const h = await harness([ [createMockResponse([ { type: 'text', text: '>>#alpha first' }, @@ -371,6 +371,7 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery ], { toolResultGuard: true, proseRouting: 'explicit', refusalHandling: { retries: 3 } }); const routed: Array<{ text: string; channelId: string }> = []; const streamed: Array<{ channelId: string; delta: string }> = []; + const completed: Array<{ channelId: string; text: string; inferenceId: string }> = []; (h.framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveProseTarget: (spec: string) => spec === '#alpha' ? { channelId: 'chan-alpha' } : { error: `no channel ${spec}` }, routeSpeech: async (_agent: string, speech: string, channelId: string) => { @@ -379,6 +380,9 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { streamed.push({ channelId, delta }); }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, text: string) => { + completed.push({ channelId, text, inferenceId }); + }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, key: string) => key in target ? (target as Record)[key] : () => undefined }); @@ -392,6 +396,11 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery const toAlpha = streamed.filter((d) => d.channelId === 'chan-alpha').map((d) => d.delta).join(''); assert.match(toAlpha, /continued/, 'the resumed stream follows delivery there'); assert.doesNotMatch(streamed.map((d) => d.delta).join(''), /discard-this-partial-output/); + assert.deepEqual(completed.map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'continued' }, + ], 'the stream that handed its turn to the retry completes what it streamed, and so does the retry'); + assert.notEqual(completed[0].inferenceId, completed[1].inferenceId, 'the retry streams under its own inference id'); } finally { await h.framework.stop(); } }); diff --git a/test/tool-wrapper-prose-guard-integration.test.ts b/test/tool-wrapper-prose-guard-integration.test.ts index d58d1c69a..6cf9b8844 100644 --- a/test/tool-wrapper-prose-guard-integration.test.ts +++ b/test/tool-wrapper-prose-guard-integration.test.ts @@ -261,7 +261,7 @@ describe('tool wrapper prose guard integration', () => { membrane.pushResponse(createMockResponse([{ type: 'text', text: '\n', }] as ContentBlock[])); - const module = new HeartbeatModule(); const routed: string[] = []; + const module = new HeartbeatModule(); const routed: string[] = []; const outgoing: string[] = []; const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: membrane.asMembrane(), agents: [{ name: 'assistant', model: 'test', systemPrompt: 'sys', toolWrapperProseGuard: true }], @@ -269,6 +269,7 @@ describe('tool wrapper prose guard integration', () => { }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); @@ -277,6 +278,8 @@ describe('tool wrapper prose guard integration', () => { await framework.stop(); assert.equal(module.calls, 1, 'genuine structured tool executes exactly once'); assert.deepEqual(routed, ['\n']); + assert.deepEqual(outgoing, ['\n'], + 'posted as ordinary prose, it streams as delivery publishes it'); assert.ok(all.some((m) => m.content.some((b) => b.type === 'text' && b.text.includes('')))); assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); From 2aa99a4fb464255edde3935ebd21d34045d3c0d7 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Fri, 16 Oct 2026 14:28:13 -0400 Subject: [PATCH 07/19] feat(streaming): inference:published carries its publish's messageId A voice relay that edits what it voiced, cutting an interrupted message down to the words that were heard, needs the id of the message each chunk was published as. Only mcpl:speech-routed carried it, and matching that trace to what was voiced goes wrong: the sleep announcement, a subconscious's speak_in_channel and a message published while the relay was away all reach it, though none was voiced. inference:published is this PR's own trace and isn't on main yet, so its shape is still ours to get right. Each delivery site already holds routeSpeech's outcome, which carries the messageId the channel's server returned (ChannelsPublishResult.messageId) when it returned one. SpeechPublished takes it as a third argument, which every site passes, and streamSpeech puts it on the trace when it's there. Each inference:published is one publish, so each carries its own id; inference:published_complete carries none, since its text can span several messages. channels/outgoing/chunk and the MCPL wire are unchanged. The trace doc, the added fragment and the PR description name the field, and say a relay that edits what it voiced takes the id from inference:published. Tests (outgoing-stream): on each of the six delivery paths, each published trace carries the id its server returned for that publish; a published trace carries no messageId when the server returned none. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.added.md | 23 ++++++++------- src/framework.ts | 17 ++++++----- src/types/trace.ts | 7 ++++- test/outgoing-stream.test.ts | 41 ++++++++++++++++++++++++-- 4 files changed, 66 insertions(+), 22 deletions(-) diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md index f549eb0b2..69d4494c0 100644 --- a/changelog.d/hold-round-speech.added.md +++ b/changelog.d/hold-round-speech.added.md @@ -1,12 +1,15 @@ - `inference:published` and `inference:published_complete` traces carry the outgoing stream to in-process consumers: each chunk as it is sent, with - its `channelId`, `inferenceId`, `index` and `text`, and each channel's - `complete`. They are emitted whether or not the channel's server takes - the stream, and carry only what delivery published, so a host relay that - voices them, such as a TTS bridge, never speaks text that `skip_reply`, a - send or a private `think` withheld, or a publish that failed. - `inference:tokens` is unchanged: it carries the model's text as it is - written, held, as before, while a tool-result guard's batch is pending. - `mcpl:speech-routed` also reports each speech publish, the `sleep` tool's - announcement among them, with no stream id, index or end, so a relay - that voices speech takes one of the two. + its `channelId`, `inferenceId`, `index` and `text`, and the `messageId` + the channel's server returned for that publish, when it returned one; + and each channel's `complete`. They are emitted whether or not the + channel's server takes the stream, and carry only what delivery + published, so a host relay that voices them, such as a TTS bridge, never + speaks text that `skip_reply`, a send or a private `think` withheld, or a + publish that failed. `inference:tokens` is unchanged: it carries the + model's text as it is written, held, as before, while a tool-result + guard's batch is pending. `mcpl:speech-routed` also reports speech + publishes, the `sleep` tool's announcement among them, with no stream + id, index or end, so a relay that voices speech takes one of the two, + not both, and a relay that edits what it voiced takes the message's id + from `inference:published`. diff --git a/src/framework.ts b/src/framework.ts index 79cc0b2cc..789d43e2c 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -665,7 +665,7 @@ function freshTurnRouting(): TurnRouting { * open, or a server without `channels.publish`), when it throws or times * out, or when the server reports it undelivered (`delivered: false`). */ -type SpeechPublished = (channelId: string, text: string) => void; +type SpeechPublished = (channelId: string, text: string, messageId: string | undefined) => void; /** How a round's prose is delivered: kept private, and under which turn's routing. */ interface ProseRound { @@ -11319,7 +11319,7 @@ export class AgentFramework { if (outcome?.delivered) { this.proseBounceStreaks.delete(agent.name); if (usedClipboard) this.proseClipboards.delete(agent.name); - onPublished?.(outcome.channelId, body); + onPublished?.(outcome.channelId, body, outcome.messageId); } else { delete routing.pin; routing.hybridHold = true; @@ -11351,7 +11351,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry!.routeSpeech(agent.name, envelope, target); this.recordProseDelivery(routing, outcome); - if (outcome?.delivered) onPublished?.(outcome.channelId, envelope); + if (outcome?.delivered) onPublished?.(outcome.channelId, envelope, outcome.messageId); } catch (err) { console.error('hybrid locus delivery failed:', err); } @@ -11431,7 +11431,7 @@ export class AgentFramework { if (result?.delivered) { this.proseBounceStreaks.delete(name); if (usedClipboard) this.proseClipboards.delete(name); - round.onPublished?.(result.channelId, body); + round.onPublished?.(result.channelId, body, result.messageId); } } catch (err) { console.error(`[prose] ${name}: delivery to ${targetChannel} failed:`, err); @@ -12307,7 +12307,7 @@ export class AgentFramework { const outgoingText = new Map(); // Set once the stream's `complete` has gone: no chunk follows it. let outgoingCompleted = false; - const streamSpeech: SpeechPublished = (channelId, text) => { + const streamSpeech: SpeechPublished = (channelId, text, messageId) => { if (outgoingCompleted) return; // A later publish to the same channel is another message: a paragraph // break keeps the two apart for a consumer that joins the chunks, so a @@ -12324,6 +12324,7 @@ export class AgentFramework { inferenceId: outgoingInferenceId, index, text: delta, + ...(messageId !== undefined ? { messageId } : {}), }); }; // Spec 14.3: the stream's end for each channel it streamed to, with @@ -12354,7 +12355,7 @@ export class AgentFramework { chainSpeech(async () => { const outcome = await this.channelRegistry!.routeSpeech(agent.name, text, locus); this.recordProseDelivery(streamRouting, outcome); - if (outcome?.delivered) streamSpeech(outcome.channelId, text); + if (outcome?.delivered) streamSpeech(outcome.channelId, text, outcome.messageId); }, 'mid-turn speech routing failed:'); }; @@ -13209,7 +13210,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, speechText, locus); this.recordProseDelivery(streamRouting, outcome); - if (outcome?.delivered) streamSpeech(outcome.channelId, speechText); + if (outcome?.delivered) streamSpeech(outcome.channelId, speechText, outcome.messageId); } catch (err) { console.error('speech routing failed:', err); } @@ -13301,7 +13302,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, seg, locus); this.recordProseDelivery(streamRouting, outcome); - if (outcome?.delivered) streamSpeech(outcome.channelId, seg); + if (outcome?.delivered) streamSpeech(outcome.channelId, seg, outcome.messageId); } catch (err) { console.error('speech routing failed:', err); } diff --git a/src/types/trace.ts b/src/types/trace.ts index b05af0ee6..6726208a4 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -184,7 +184,8 @@ export type TraceEvent = * id, index or end: routeSpeech's once delivered, the `sleep` tool's * announcement among them, and a subconscious's `speak_in_channel` as * it is sent, before its outcome is known. A relay that voices speech - * takes one of the two, not both. + * takes one of the two, not both, and a relay that edits what it voiced + * takes the message's id from this trace's `messageId`. */ type: 'inference:published'; agentName: string; @@ -195,6 +196,10 @@ export type TraceEvent = /** Increases by one per chunk of the stream, across its channels. */ index: number; text: string; + /** The id the channel's server returned for this chunk's publish + * (`ChannelsPublishResult.messageId`); absent when it returned none. + * Each chunk is one publish, so each carries its own. */ + messageId?: string; }) | (TraceEventBase & { /** diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 7bd33d118..8ce344a84 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -105,6 +105,8 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = const holds = new Map>(); /** A publish of one of these texts fails: the server reports it undelivered (`delivered: false`), or it throws. */ const fails = new Map(); + /** A publish of one of these texts is answered with this message id, as a server that returns one. */ + const messageIds = new Map(); const known: Record = { '#alpha': 'chan-alpha', '#cafe': 'chan-cafe', '@laria': 'discord:dm:99' }; let stopped = false; const target: Record = { @@ -116,7 +118,8 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = await holds.get(body); const failure = fails.get(body); if (failure === 'throw') throw new Error('synthetic publish failure'); - return { delivered: failure === undefined, channelId }; + const messageId = messageIds.get(body); + return { delivered: failure === undefined, channelId, ...(messageId !== undefined ? { messageId } : {}) }; }, sendOutgoingChunk: (channelId: string, _agent: string, inferenceId: string, index: number, delta: string) => { if (stopped) return; @@ -151,6 +154,7 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = log, holds, fails, + messageIds, chunks: () => log.filter((e): e is Extract => e.kind === 'chunk'), publishes: () => log.filter((e) => e.kind === 'publish').map((e) => ({ channelId: e.channelId, text: e.text })), completes: () => log.filter((e): e is Extract => e.kind === 'complete'), @@ -168,11 +172,16 @@ async function harness(scripts: NormalizedResponse[][], config: Partial = []; + const traces: Array<{ type: string; channelId: string; text: string; inferenceId: string; index?: number; messageId?: string }> = []; framework.onTrace((event) => { if (event.type === 'inference:published' || event.type === 'inference:published_complete') { const { type, channelId, text: body, inferenceId } = event; - traces.push({ type, channelId, text: body, inferenceId, ...(event.type === 'inference:published' ? { index: event.index } : {}) }); + traces.push({ + type, channelId, text: body, inferenceId, + ...(event.type === 'inference:published' ? { index: event.index } : {}), + // Kept whenever the key is there, so a trace that carries it as undefined shows. + ...('messageId' in event ? { messageId: event.messageId } : {}), + }); } }); /** @@ -656,6 +665,32 @@ const failedPublishCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'expl { name: 'explicit mid-turn', mode: 'explicit', script: () => [round(text('>>#alpha lost'), call('one', 'test--read')), final(text('>>#cafe kept'))], lost: 'lost', kept: 'kept' }, { name: 'explicit text-only', mode: 'explicit', script: () => [final(text('>>#alpha lost'))], lost: 'lost' }, ]; +// The id a server returns for a publish rides on that publish's trace, on every delivery path. +for (const c of failedPublishCases) { + test(`${c.name}: each published trace carries the message id its server returned for that publish`, async () => { + const h = await harness([c.script()], { proseRouting: c.mode }); + const ids = new Map([[c.lost, 'msg-1'], ...(c.kept ? [[c.kept, 'msg-2'] as [string, string]] : [])]); + for (const [body, id] of ids) h.messageIds.set(body, id); + try { + await h.run(); + const published = h.traces.filter((t) => t.type === 'inference:published'); + const bodyOf = (t: { text: string }) => t.text.replace(/^\n\n/, ''); + assert.deepEqual(published.map(bodyOf).sort(), [...ids.keys()].sort(), 'each publish streamed once'); + for (const t of published) assert.equal(t.messageId, ids.get(bodyOf(t)), `"${bodyOf(t)}" carries its own publish's id`); + } finally { await h.framework.stop(); } + }); +} + +test('a published trace carries no message id when the server returned none', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + try { + await h.run(); + const published = h.traces.filter((t) => t.type === 'inference:published'); + assert.equal(published.length, 2, 'both publishes streamed'); + assert.ok(published.every((t) => !('messageId' in t)), 'no id came back, so the traces carry none'); + } finally { await h.framework.stop(); } +}); + for (const c of failedPublishCases) { for (const failure of ['undelivered', 'throw'] as const) { test(`${c.name}: a publish that ${failure === 'throw' ? 'throws' : 'the server reports undelivered'} streams nothing`, async () => { From a11381374d780760016b87aaf71bb988d3999dc5 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Fri, 16 Oct 2026 15:03:45 -0400 Subject: [PATCH 08/19] fix(routing): a script's mark holds only what a later script can reach Answers an internal review of the scanner round and the messageId commit. - reachable now holds only what a later script can still call: a name referenced other than in a direct call (an alias), a name inside a definition, and a kept key. Before, once a script referenced any name or held any def, lambda or class, every name it called directly was marked too, so a script with an unrelated helper and a top-level say() held every later round in that interpreter. A definition is a def's header and indented block, a class's methods among them, or a lambda to the end of its logical line (definitions()). A line continued inside brackets, after a backslash or inside a string literal an earlier line opened stays in it, whatever its indentation, and a blank line ending in a carriage return doesn't end it. A class's own body runs as the class statement does, so a direct call there leaves nothing reachable. A key called at once inside a definition counts, since the call runs when the definition does. - A direct call reads as one only on the same line: CALLED and KEY_CALLED match spaces and tabs, so a name at a line's end before a line opening with `(` is a reference, as Python reads it outside brackets. - Names in a string are matched with its backslash escapes read as breaks, so exec of "x = 1\nsay({})" counts say. A key's name is counted by that same match, so the key branch no longer adds it. - Texts: the module doc says the boundary is an ASCII letter, digit or underscore; the unseen lists name non-ASCII spellings, a definition or alias made inside a string the script runs or an f-string's field, and a call that outlives its script (awaited only later, scheduled as a task, or left in a generator expression); a name counts wherever it stands as a whole identifier. The added fragment, the trace doc and the description say mcpl:speech-routed also reports the sleep announcement and a subconscious's speak_in_channel, the latter before its outcome is known, so a relay voicing the agent's prose takes inference:published alone, and each message's id from it. Tests: script-tool-names covers a direct call beside an unrelated definition, definitions over several lines with dedented continuation lines, a line a string literal carries over and a blank \r\n line, class methods and a class's own body, a called key inside a definition, a name or key at a line's end before a bracketed line, an escape before a name, both one-quote forms of an unclosed literal, and the newly listed misses. code-execution: a script with an unrelated helper and a direct call leaves no mark; a later mark adds its names to an earlier one in the same interpreter; a script the busy runner refuses leaves no mark; the private-policy think holds its round in locus, hybrid and explicit mode. A reader found the two lines that ended a definition early, a line a string literal carries over and a blank \r\n line, and that a class's own body runs at once, not later. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.added.md | 25 ++--- changelog.d/hold-round-speech.fixed.md | 61 ++++++----- src/code-execution/script-tool-names.ts | 138 ++++++++++++++++-------- src/types/trace.ts | 6 +- test/code-execution.test.ts | 80 +++++++++++++- test/script-tool-names.test.ts | 43 +++++++- 6 files changed, 258 insertions(+), 95 deletions(-) diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md index 69d4494c0..04fcb1955 100644 --- a/changelog.d/hold-round-speech.added.md +++ b/changelog.d/hold-round-speech.added.md @@ -1,15 +1,16 @@ - `inference:published` and `inference:published_complete` traces carry the outgoing stream to in-process consumers: each chunk as it is sent, with its `channelId`, `inferenceId`, `index` and `text`, and the `messageId` - the channel's server returned for that publish, when it returned one; - and each channel's `complete`. They are emitted whether or not the - channel's server takes the stream, and carry only what delivery - published, so a host relay that voices them, such as a TTS bridge, never - speaks text that `skip_reply`, a send or a private `think` withheld, or a - publish that failed. `inference:tokens` is unchanged: it carries the - model's text as it is written, held, as before, while a tool-result - guard's batch is pending. `mcpl:speech-routed` also reports speech - publishes, the `sleep` tool's announcement among them, with no stream - id, index or end, so a relay that voices speech takes one of the two, - not both, and a relay that edits what it voiced takes the message's id - from `inference:published`. + the channel's server returned for that publish, when it returned one; and + each channel's `complete`. They are emitted whether or not the channel's + server takes the stream, and carry only what delivery published, so a host + relay that voices them, such as a TTS bridge, never speaks text that + `skip_reply`, a send or a private `think` withheld, or a publish that + failed. `inference:tokens` is unchanged: it carries the model's text as it + is written, held, as before, while a tool-result guard's batch is pending. + `mcpl:speech-routed` also reports speech publishes, with no stream id, + index or end: among them the `sleep` tool's announcement, and a + subconscious's `speak_in_channel` as it is sent, before its outcome is + known. A relay that voices the agent's prose takes `inference:published` + alone, not both, and takes each message's id from it to edit what it + voiced. diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index 03b4c19e3..fb5b8003f 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -39,34 +39,39 @@ a tool whose call changes the round's routing in the turn's mode: in locus and hybrid mode `skip_reply`, a silencing send, `say` or `whisper`; in explicit mode only `skip_reply`; and in either, `think` under the private - same-round policy. Disabled mode holds nothing. A name counts where the - script could call it, as a function or a `tools[...]` key: `say_hello` - doesn't name `say`. A name in a comment doesn't count, since Python never - runs one, while a name anywhere in a string does, since a string can be a - key or code to run. A script can also change its round's routing when the - interpreter it runs in has run a script that left such a tool reachable, - by defining a function, lambda or class, by naming the tool other than in - a direct call, or by keeping a string that is the tool's whole name, as a - key is. That holds until the interpreter is reclaimed or restarted. A - string that only mentions a tool, as a printed sentence does, leaves - nothing reachable. Without code execution, a `code_execution` call holds - nothing. Any other round is routed at its tool-calls event, as before, so - narration beside a script isn't held for the script's runtime. What goes - unseen is routed as it was before scripts counted, such as a tool name the - code doesn't spell out (built at runtime, or written with escapes), code - loaded from a file or module, an enumeration of the tools, a call a script - makes but awaits only in a later one, and code a later script runs from a - string an earlier one kept. The turn's locus and an explicit round's `>>` - targets are taken at the round's tool-calls event, when the model has - finished writing it, not when its scripts return; hybrid's `>>>` targets - apply as its delivery runs, as for any queued hybrid delivery. A - background script's call returns as soon as the script has started, and - its later calls aren't the round's. A script's result settles only the - round that dispatched it, even when a newer round reuses its call id. A - round whose turn ends or stops waiting on it, or that a newer round of the - same agent replaces, before its scripts return, as when the turn is - stopped, is withheld: a log line names it, and the turn's delivery receipt - counts it as suppressed. + same-round policy. Disabled mode holds nothing. A name counts in code + wherever it stands as a whole identifier, as a function is called or a + `tools[...]` key is written: `say_hello` doesn't name `say`. A name in a + comment doesn't count, since Python never runs one, while a name anywhere + in a string does, since a string can be a key or code to run. A script can + also change its round's routing when the interpreter it runs in has run a + script that left such a tool reachable, by naming the tool inside a + definition (a function or lambda, a class's methods among them) or other + than in a direct call, or by keeping a string that is the tool's whole + name, as a key is. That holds until the interpreter is reclaimed or + restarted. A direct call outside a definition, a class's own body + included, or a string that only mentions a tool, as a printed sentence + does, leaves nothing reachable. Without code execution, a `code_execution` + call holds nothing. Any other round is routed at its tool-calls event, as + before, so narration beside a script isn't held for the script's runtime. + What goes unseen is routed as it was before scripts counted, such as a + tool name the code doesn't spell out (built at runtime, written with + escapes, or in non-ASCII spellings), code loaded from a file or module, an + enumeration of the tools, a definition or alias made inside a string the + script runs or an f-string's field, a call that outlives its script (one + it awaits only in a later script, schedules as a task, or leaves in a + generator expression), and code a later script runs from a string an + earlier one kept. The turn's locus and an explicit round's `>>` targets + are taken at the round's tool-calls event, when the model has finished + writing it, not when its scripts return; hybrid's `>>>` targets apply as + its delivery runs, as for any queued hybrid delivery. A background + script's call returns as soon as the script has started, and its later + calls aren't the round's. A script's result settles only the round that + dispatched it, even when a newer round reuses its call id. A round whose + turn ends or stops waiting on it, or that a newer round of the same agent + replaces, before its scripts return, as when the turn is stopped, is + withheld: a log line names it, and the turn's delivery receipt counts it + as suppressed. - Every stream now sends `channels/outgoing/complete` for each channel it streamed to, on every exit path, as MCPL Spec 14.3 asks: once its deliveries have run, or, when the host stops, at once, with what it has diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts index 3677a11a7..3002c26a2 100644 --- a/src/code-execution/script-tool-names.ts +++ b/src/code-execution/script-tool-names.ts @@ -1,30 +1,36 @@ /** - * Which of some tools a code_execution script names, read as Python reads - * it, and which of them it leaves reachable by a later script in the same - * interpreter. The framework holds a script's round for the names it - * returns, and marks the agent's interpreter with the reachable ones - * (AgentFramework.scriptMayRoute). + * Which of some tools a code_execution script names, and which of them it + * leaves reachable by a later script in the same interpreter, read past the + * script's comments and string literals as Python reads them. The framework + * holds a script's round for the names it returns, and marks the agent's + * interpreter with the reachable ones (AgentFramework.scriptMayRoute). * * A script reaches a tool through a function named for it (the tool's name, * after any prefix joined with `__`) or a `tools[...]` key. So a name counts - * in code where Python could call it: after a character that can't continue - * an identifier, or after `__`, and before a character that can't continue - * one. `say_hello` and `my_say` don't name `say`; `mcpl__world__say(` does. - * A comment names nothing, since Python never runs one. A name anywhere in a - * string counts, since a string can be a key or code passed to eval or exec. + * in code wherever it stands as a whole identifier: after a character other + * than an ASCII letter, digit or underscore, or after `__`, and before such a + * character. `say_hello` and `my_say` don't name `say`; `mcpl__world__say(` + * does. A comment names nothing, since Python never runs one. A name anywhere + * in a string counts, an escape before it read as a break, since a string + * can be a key or code passed to eval or exec. * - * A script leaves a tool reachable when its code names the tool other than - * in a direct call (an alias, a reference) or defines a function, lambda or - * class, or when one of its strings is the tool's whole name, as a key is, - * and isn't called at once, as `tools["world--say"](…)` is. A direct call - * finishes inside its own script. A string that only mentions a tool, as a - * printed sentence does, leaves nothing reachable. + * A script leaves a tool reachable when it names the tool other than in a + * direct call (an alias, a reference), names it inside a definition, whose + * body runs later, when it's called (a def's header and indented block, a + * class's methods among them, or a lambda), or keeps a string that is the + * tool's whole name, as a key is, without calling it at once, as + * `tools["world--say"](…)` does. A direct call anywhere else finishes inside + * its own script, a class's own body included, since it runs as the class + * statement does; and a string that only mentions a tool, as a printed + * sentence does, leaves nothing reachable. * * Some reaches go unseen, such as a tool name the code doesn't spell out - * (built at runtime, or written with escapes), code loaded from a file or - * module, an enumeration of the tools, a call a script makes but awaits only - * in a later one, and code a later script runs from a string an earlier one - * kept. + * (built at runtime, written with escapes, or in non-ASCII spellings), code + * loaded from a file or module, an enumeration of the tools, a definition or + * alias made inside a string the script runs or inside an f-string's field, + * a call that outlives its script (one it awaits only in a later script, + * schedules as a task, or leaves in a generator expression), and code a + * later script runs from a string an earlier one kept. * * @param tools plain tool names, of identifier characters */ @@ -32,45 +38,87 @@ export function scriptToolNames(script: string, tools: readonly string[]): { nam const named = new RegExp(`(?:(?(); - // Named in code, or by a string that is the whole name. - const callable = new Set(); - let referenced = false; + const reachable = new Set(); for (const match of code.matchAll(named)) { - callable.add(match[1]); + names.add(match[1]); CALLED.lastIndex = match.index + match[0].length; - if (!CALLED.test(code)) referenced = true; + if (!CALLED.test(code) || inDefinition(match.index)) reachable.add(match[1]); } - for (const { text, end } of strings) { + for (const { text, start, end } of strings) { + // The name itself counts as any name in a string does, just below. const whole = key.exec(text); if (whole) { - callable.add(whole[1]); KEY_CALLED.lastIndex = end; - if (!KEY_CALLED.test(code)) referenced = true; + if (!KEY_CALLED.test(code) || inDefinition(start)) reachable.add(whole[1]); } - for (const match of text.matchAll(named)) names.add(match[1]); + for (const match of text.replace(ESCAPE, ' ').matchAll(named)) names.add(match[1]); } - for (const name of callable) names.add(name); - const leavesReachable = callable.size > 0 && (referenced || /\b(?:def|lambda|class)\b/.test(code)); - return { names, reachable: leavesReachable ? callable : new Set() }; + return { names, reachable }; } -/** A direct call right after a name in code. */ -const CALLED = /\s*\(/y; -/** A direct call right after a key, a `tools[...]` subscript's close and then `(`. */ -const KEY_CALLED = /\s*\]\s*\(/y; +/** A direct call right after a name in code: `(`, on the same line. */ +const CALLED = /[ \t]*\(/y; +/** A direct call right after a key: a `tools[...]` subscript's close and then `(`, on the same line. */ +const KEY_CALLED = /[ \t]*\][ \t]*\(/y; +/** A backslash escape in a string's text, which a name after it follows as it would a break. */ +const ESCAPE = /\\(?:x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8}|N\{[^}]*\}|[0-7]{1,3}|[\s\S])/g; + +/** + * Whether an offset of a script's code falls inside a definition, whose body + * runs later, when it's called: a def's header and the indented block after + * it, or a lambda from its keyword to the end of its line. A line inside + * brackets, after a backslash, or inside a string literal that an earlier + * line opened continues the line before it, whatever its indentation; and a + * line of only whitespace, a carriage return included, doesn't end a block. + */ +function definitions(code: string, strings: readonly { start: number; end: number }[]): (offset: number) => boolean { + const lines: { start: number; text: string; indent: number; blank: boolean; continued: boolean }[] = []; + let depth = 0; + let continued = false; + let start = 0; + for (const text of code.split('\n')) { + const indent = /^[ \t]*/.exec(text)![0].length; + const inString = strings.some((s) => s.start < start && start < s.end); + lines.push({ start, text, indent, blank: text.trim() === '', continued: continued || depth > 0 || inString }); + for (const c of text) { + if (c === '(' || c === '[' || c === '{') depth++; + else if ((c === ')' || c === ']' || c === '}') && depth > 0) depth--; + } + continued = text.trimEnd().endsWith('\\'); + start += text.length + 1; + } + const startOf = (j: number) => (j < lines.length ? lines[j].start : code.length); + const spans: [number, number][] = []; + lines.forEach((line, i) => { + if (line.continued || !/^[ \t]*(?:async[ \t]+)?def\b/.test(line.text)) return; + // The header's own continued lines, then the block indented under it. + let j = i + 1; + while (j < lines.length && (lines[j].blank || lines[j].continued || lines[j].indent > line.indent)) j++; + spans.push([line.start, startOf(j)]); + }); + for (const lambda of code.matchAll(/\blambda\b/g)) { + let j = 0; + while (j < lines.length && lines[j].start <= lambda.index) j++; + while (j < lines.length && lines[j].continued) j++; + spans.push([lambda.index, startOf(j)]); + } + return (offset) => spans.some(([from, to]) => offset >= from && offset < to); +} /** * A script's code, with each comment and string literal blanked to spaces - * (its newlines kept, so offsets hold), and each literal's text and the - * offset just past it. A literal may be triple-quoted and take any prefix - * (r, b, f, u), and a backslash keeps the character after it inside the - * literal, raw or not, as Python reads it. An f-string's replacement fields - * are read as its text. A literal in single quotes that reaches the end of - * its line ends there, so a stray quote can't take the rest of the script. + * (its newlines kept, so offsets hold), and each literal's text with the + * offsets of its opening quote and just past its close. A literal may be + * triple-quoted and take any prefix (r, b, f, u), and a backslash keeps the + * character after it inside the literal, raw or not, as Python reads it. An + * f-string's replacement fields are read as its text. A literal in single + * quotes that reaches the end of its line ends there, so a stray quote can't + * take the rest of the script. */ -function readScript(script: string): { code: string; strings: { text: string; end: number }[] } { - const strings: { text: string; end: number }[] = []; +function readScript(script: string): { code: string; strings: { text: string; start: number; end: number }[] } { + const strings: { text: string; start: number; end: number }[] = []; let code = ''; let at = 0; const special = /[#'"]/g; @@ -88,7 +136,7 @@ function readScript(script: string): { code: string; strings: { text: string; en } close = Math.min(close, script.length); end = script.startsWith(quote, close) ? close + quote.length : close; - strings.push({ text: script.slice(start + quote.length, close), end }); + strings.push({ text: script.slice(start + quote.length, close), start, end }); } code += script.slice(at, start) + script.slice(start, end).replace(/[^\n]/g, ' '); at = special.lastIndex = end; diff --git a/src/types/trace.ts b/src/types/trace.ts index 6726208a4..5704dc8ee 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -183,9 +183,9 @@ export type TraceEvent = * `mcpl:speech-routed` also reports speech publishes, with no stream * id, index or end: routeSpeech's once delivered, the `sleep` tool's * announcement among them, and a subconscious's `speak_in_channel` as - * it is sent, before its outcome is known. A relay that voices speech - * takes one of the two, not both, and a relay that edits what it voiced - * takes the message's id from this trace's `messageId`. + * it is sent, before its outcome is known. A relay that voices the + * agent's prose takes this trace alone, not both, and takes each + * message's id from its `messageId` to edit what it voiced. */ type: 'inference:published'; agentName: string; diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index aed8ede99..621d6efc9 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -832,18 +832,20 @@ describe("a round that runs code_execution is routed once its scripts return (re } finally { await h.done(); } }); - it('locus: a think inside the script holds its round back under the private same-round policy', async () => { - const h = await scriptRound('locus', [ + for (const mode of ['locus', 'hybrid', 'explicit'] as const) it(`${mode}: a think inside the script holds its round back under the private same-round policy`, async () => { + const prefix = mode === 'explicit' ? '>>#alpha ' : ''; + const target = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ createMockResponse([ - { type: 'text', text: 'working it through' }, + { type: 'text', text: `${prefix}working it through` }, scriptCall({ code: 'await think({"content": "notes"})' }), ] as ContentBlock[], 'tool_use'), - createMockResponse([{ type: 'text', text: 'the answer' }] as ContentBlock[]), + createMockResponse([{ type: 'text', text: `${prefix}the answer` }] as ContentBlock[]), ], { sameRoundThinkTextPolicy: 'private' }); try { await h.run(); assert.ok(h.log.includes('tool:think'), 'the script called think'); - assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:the answer@world:test'], + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [`publish:the answer@${target}`], 'the round beside the think stays private, as with a direct think, and the next round is delivered'); } finally { await h.done(); } }); @@ -1143,6 +1145,74 @@ describe("a round that runs code_execution is routed once its scripts return (re } finally { await h.done(); } }); + /** The agent's interpreter mark: the names it holds, or undefined when there is none. */ + const markOf = (framework: AgentFramework) => { + const mark = (framework as unknown as { scriptRoutingInterpreters: Map }> }) + .scriptRoutingInterpreters.get('assistant'); + return mark && [...mark.names].sort(); + }; + + it("a script that calls a routing tool directly, beside a definition that doesn't use it, leaves no mark", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'formatting notes' }, + scriptCall({ code: 'def fmt(x):\n return x.upper()\nawait think({"content": fmt("notes")})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the script thought'); + assert.equal(markOf(h.framework), undefined, 'fmt can still be called, but it reaches no routing tool'); + } finally { await h.done(); } + }); + + it('a later mark in the same interpreter adds its names to the earlier ones', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'keeping a handle' }, + scriptCall({ code: 't = think' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.deepStrictEqual(markOf(h.framework), ['send_message', 'think'], 'the helper the first script left is still there'); + } finally { await h.done(); } + }); + + it('a script the busy runner refuses leaves no mark, though its code would have', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'two scripts at once' }, + scriptCall({ code: 'await test__slow({})' }, 'ce1'), + scriptCall({ code: 'post = test__send_message' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Let the first script go once the second's refusal has come back. + const results: string[] = []; + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') { + results.push(event.callId); + if (event.callId === 'ce2') h.module.releaseSlow(); + } + }); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(results, ['ce2', 'ce1'], "the second script was refused while the first ran"); + assert.equal(markOf(h.framework), undefined, 'the refused alias never ran, so it marks nothing'); + } finally { + clearTimeout(bound); + await h.done(); + } + }); + it("a mark left in an earlier interpreter doesn't hold a round once a newer interpreter runs", async () => { const h = await scriptRound('locus', [ createMockResponse([ diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts index d48a581ac..8eac8d648 100644 --- a/test/script-tool-names.test.ts +++ b/test/script-tool-names.test.ts @@ -34,6 +34,36 @@ test('an alias, a definition or a kept key leaves the tool reachable', () => { assert.deepEqual(read('name = "test--send_message"'), { names: ['send_message'], reachable: ['send_message'] }); }); +test("a direct call outside a definition leaves nothing reachable, beside a definition that doesn't use the tool", () => { + assert.deepEqual(read('def fmt(x):\n return x\nawait say({"text": fmt("hi")})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('await say(sorted(xs, key=lambda x: x))'), { names: ['say'], reachable: [] }, 'a lambda after the call'); +}); + +test('a name inside a definition is reachable, a called key included, whatever lines the definition spans', () => { + assert.deepEqual(read('def go():\n return tools["say"]({})'), { names: ['say'], reachable: ['say'] }); + // A header over several lines, and a body line continued inside brackets at + // no indentation, as Python allows; the call after the definition is direct. + assert.deepEqual(read('async def post(\n text,\n):\n x = [\n1,\n]\n await say({"text": text})\nawait skip_reply({})'), + { names: ['say', 'skip_reply'], reachable: ['say'] }); + assert.deepEqual(read('class Poster:\n async def go(self):\n await send_message({})\n\nawait think({})'), + { names: ['send_message', 'think'], reachable: ['send_message'] }); + // A line a string literal carries over continues its statement, whatever its + // indentation (Ramona-2416); so does a blank line written with \r\n. + assert.deepEqual(read('if True:\n def f():\n q = """\n"""; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a triple-quoted string'); + assert.deepEqual(read('def f():\n q = "a\\\n"; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a backslash-continued string'); + assert.deepEqual(read('def f():\r\n x = 1\r\n\r\n await say({})\r\n'), { names: ['say'], reachable: ['say'] }, 'a blank \\r\\n line'); +}); + +test("a class's own body runs as the class statement does, so a direct call there leaves nothing reachable", () => { + assert.deepEqual(read('class A:\n say({})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('class A:\n post = say'), { names: ['say'], reachable: ['say'] }, 'an alias kept on the class'); +}); + +test("a name at a line's end, before a line that opens with a bracket, is a reference, not a call", () => { + assert.deepEqual(read('alias = say\n(print("x"))'), { names: ['say'], reachable: ['say'] }); + assert.deepEqual(read('post = tools["say"]\n(print("x"))'), { names: ['say'], reachable: ['say'] }, 'a key, likewise'); +}); + test('a comment names nothing, and its words are not code', () => { assert.deepEqual(read('await test__slow({}) # then skip_reply, or say so'), { names: [], reachable: [] }); assert.deepEqual(read('say("hi") # first class result'), { names: ['say'], reachable: [] }); @@ -59,6 +89,10 @@ test('an escaped quote stays inside its string', () => { assert.deepEqual(read('msg = "she said \\"skip_reply\\" # done"'), { names: ['skip_reply'], reachable: [] }); }); +test('a name right after an escape in a string counts, as the string run as code would call it', () => { + assert.deepEqual(read('exec("x = 1\\nsay({})")'), { names: ['say'], reachable: [] }); +}); + test("an f-string's call inside its braces counts for its own round but leaves nothing reachable", () => { assert.deepEqual(read('print(f"result: {say({\'text\': \'hi\'})}")'), { names: ['say'], reachable: [] }); }); @@ -68,17 +102,22 @@ test('a # inside a string starts no comment', () => { { names: ['say', 'send_message'], reachable: ['send_message'] }); }); -test("a string in single quotes ends at its line's end, so a stray quote can't take the rest of the script", () => { +test("a string in one quote mark, not three, ends at its line's end, so a stray quote can't take the rest of the script", () => { assert.deepEqual(read('x = "unclosed\npost = test__send_message'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read("x = 'unclosed\npost = test__send_message"), { names: ['send_message'], reachable: ['send_message'] }); }); -test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call awaited later, code kept in a string", () => { +test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call that outlives its script, an alias made inside a string or an f-string's field, code kept in a string", () => { assert.deepEqual(read('name = "skip" + "_reply"\nawait tools[name]({})'), { names: [], reachable: [] }, 'built at runtime'); assert.deepEqual(read('await tools["s\\x61y"]({})'), { names: [], reachable: [] }, 'written with escapes'); assert.deepEqual(read('fns = list(tools.values())'), { names: [], reachable: [] }, 'an enumeration'); assert.deepEqual(read('exec(open("helper.py").read())'), { names: [], reachable: [] }, 'code from a file'); assert.deepEqual(read('pending = say({"text": "hi"})'), { names: ['say'], reachable: [] }, 'a call made here...'); assert.deepEqual(read('await pending'), { names: [], reachable: [] }, '...and awaited later'); + assert.deepEqual(read('asyncio.create_task(say({"text": "hi"}))'), { names: ['say'], reachable: [] }, 'a call scheduled as a task'); + assert.deepEqual(read('gen = (say({"text": m}) for m in msgs)'), { names: ['say'], reachable: [] }, 'a call left in a generator expression'); + assert.deepEqual(read('exec("post = say")'), { names: ['say'], reachable: [] }, 'an alias made inside a string the script runs'); + assert.deepEqual(read('print(f"{(post := say)}")'), { names: ['say'], reachable: [] }, "an alias made inside an f-string's field"); assert.deepEqual(read('later = "say({\'text\': \'hi\'})"'), { names: ['say'], reachable: [] }, 'code kept in a string...'); assert.deepEqual(read('await eval(later)'), { names: [], reachable: [] }, '...and run later'); }); From 473ab4c6ddb31c935b579e21ea8ef20324d504c5 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Fri, 16 Oct 2026 17:19:18 -0400 Subject: [PATCH 09/19] fix(routing): a kept key is a whole name, and a line stands where it's written Answers an internal review of the previous fix round. - A kept key is a whole tool name, optionally namespaced: its prefix is a run of tool-name characters before `--` or `__`, so a printed sentence ending in a qualified name ("to speak, call world--say") no longer marks the interpreter. - definitions() reads each line's indent from the script itself, not the code view, where a string that opens a line was blanked to spaces, so a line at column 0 that opens with a string ends the def before it, as Python reads it, and a call on it is direct. A line counts as blank only when it holds whitespace or a comment, so a string-only line at column 0 ends a def too. - scriptToolNames returns empty sets for an empty tools list, where the patterns matched the empty name. - Texts: the module doc's kept-key sentence no longer reads as if tools["world--say"](...) were a kept key; a direct call outside a definition "leaves nothing reachable", not "finishes inside its own script"; a lambda's span runs to the end of its logical line; the unseen lists name a call written in a string's text, inside a definition too (intended, since the mark counts a string only when it's a whole name); the fragment's name rule names ASCII and `__` and its "A script can when" reads "can change it when"; a voicing relay takes the inference:published stream with its inference:published_complete, not mcpl:speech-routed as well. Tests (script-tool-names): a space before a call; no tools; a bare whole-name string marks; a sentence ending in a namespaced name is no key, a namespaced key is; a lambda's span ends with its logical line and takes its continued line; a call on a column-0 line that opens with a string, a string-only line after a def, a comment at column 0 inside a def; a bracket or a lambda inside a string changes no line; the backslash-continued literal nests its def, so only the string rule keeps it; a backslash continuation in code; every escape form before a name; a call in a string's text inside a def goes unseen. code-execution: the busy-refusal test's control runs the same alias alone and checks it marks. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.added.md | 7 ++- changelog.d/hold-round-speech.fixed.md | 76 +++++++++++++------------ src/code-execution/script-tool-names.ts | 48 +++++++++------- src/types/trace.ts | 5 +- test/code-execution.test.ts | 15 ++++- test/script-tool-names.test.ts | 34 ++++++++++- 6 files changed, 118 insertions(+), 67 deletions(-) diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md index 04fcb1955..6d19df4cb 100644 --- a/changelog.d/hold-round-speech.added.md +++ b/changelog.d/hold-round-speech.added.md @@ -11,6 +11,7 @@ `mcpl:speech-routed` also reports speech publishes, with no stream id, index or end: among them the `sleep` tool's announcement, and a subconscious's `speak_in_channel` as it is sent, before its outcome is - known. A relay that voices the agent's prose takes `inference:published` - alone, not both, and takes each message's id from it to edit what it - voiced. + known. A relay that voices the agent's prose takes the + `inference:published` stream, with its `inference:published_complete`, and + not `mcpl:speech-routed` as well, and takes each message's id from + `inference:published` to edit what it voiced. diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index fb5b8003f..733b9f9dd 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -35,43 +35,45 @@ silences the turn's later prose, while the private `think` hold and an explicit round's `skip_reply` don't apply (#307). - A round whose script can change its routing waits for the script, and its - prose goes out when the script finishes. A script can when its code names - a tool whose call changes the round's routing in the turn's mode: in locus - and hybrid mode `skip_reply`, a silencing send, `say` or `whisper`; in - explicit mode only `skip_reply`; and in either, `think` under the private - same-round policy. Disabled mode holds nothing. A name counts in code - wherever it stands as a whole identifier, as a function is called or a - `tools[...]` key is written: `say_hello` doesn't name `say`. A name in a - comment doesn't count, since Python never runs one, while a name anywhere - in a string does, since a string can be a key or code to run. A script can - also change its round's routing when the interpreter it runs in has run a - script that left such a tool reachable, by naming the tool inside a - definition (a function or lambda, a class's methods among them) or other - than in a direct call, or by keeping a string that is the tool's whole - name, as a key is. That holds until the interpreter is reclaimed or - restarted. A direct call outside a definition, a class's own body - included, or a string that only mentions a tool, as a printed sentence - does, leaves nothing reachable. Without code execution, a `code_execution` - call holds nothing. Any other round is routed at its tool-calls event, as - before, so narration beside a script isn't held for the script's runtime. - What goes unseen is routed as it was before scripts counted, such as a - tool name the code doesn't spell out (built at runtime, written with - escapes, or in non-ASCII spellings), code loaded from a file or module, an - enumeration of the tools, a definition or alias made inside a string the - script runs or an f-string's field, a call that outlives its script (one - it awaits only in a later script, schedules as a task, or leaves in a - generator expression), and code a later script runs from a string an - earlier one kept. The turn's locus and an explicit round's `>>` targets - are taken at the round's tool-calls event, when the model has finished - writing it, not when its scripts return; hybrid's `>>>` targets apply as - its delivery runs, as for any queued hybrid delivery. A background - script's call returns as soon as the script has started, and its later - calls aren't the round's. A script's result settles only the round that - dispatched it, even when a newer round reuses its call id. A round whose - turn ends or stops waiting on it, or that a newer round of the same agent - replaces, before its scripts return, as when the turn is stopped, is - withheld: a log line names it, and the turn's delivery receipt counts it - as suppressed. + prose goes out when the script finishes. A script can change it when its + code names a tool whose call changes the round's routing in the turn's + mode: in locus and hybrid mode `skip_reply`, a silencing send, `say` or + `whisper`; in explicit mode only `skip_reply`; and in either, `think` + under the private same-round policy. Disabled mode holds nothing. A name + counts in code wherever it stands as a whole word of ASCII letters, digits + and underscores, or after `__` as in `mcpl__world__say`, as a function is + called or a `tools[...]` key is written: `say_hello` doesn't name `say`. A + name in a comment doesn't count, since Python never runs one, while a name + anywhere in a string does, since a string can be a key or code to run. A + script can also change its round's routing when the interpreter it runs in + has run a script that left such a tool reachable, by naming the tool + inside a definition (a function or lambda, a class's methods among them) + or other than in a direct call, or by keeping a string that is the tool's + whole name, optionally namespaced, as a key kept for later is. That holds + until the interpreter is reclaimed or restarted. A direct call outside a + definition, a class's own body included, or a string that only mentions a + tool, as a printed sentence does, leaves nothing reachable. Without code + execution, a `code_execution` call holds nothing. Any other round is + routed at its tool-calls event, as before, so narration beside a script + isn't held for the script's runtime. What goes unseen is routed as it was + before scripts counted, such as a tool name the code doesn't spell out + (built at runtime, written with escapes, or in non-ASCII spellings), code + loaded from a file or module, an enumeration of the tools, a call written + in a string's text (code run with `exec` or `eval`, or an f-string's + field), inside a definition too, a definition or alias made inside such a + string, a call that outlives its script (one it awaits only in a later + script, schedules as a task, or leaves in a generator expression), and + code a later script runs from a string an earlier one kept. The turn's + locus and an explicit round's `>>` targets are taken at the round's + tool-calls event, when the model has finished writing it, not when its + scripts return; hybrid's `>>>` targets apply as its delivery runs, as for + any queued hybrid delivery. A background script's call returns as soon as + the script has started, and its later calls aren't the round's. A script's + result settles only the round that dispatched it, even when a newer round + reuses its call id. A round whose turn ends or stops waiting on it, or + that a newer round of the same agent replaces, before its scripts return, + as when the turn is stopped, is withheld: a log line names it, and the + turn's delivery receipt counts it as suppressed. - Every stream now sends `channels/outgoing/complete` for each channel it streamed to, on every exit path, as MCPL Spec 14.3 asks: once its deliveries have run, or, when the host stops, at once, with what it has diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts index 3002c26a2..205d21b20 100644 --- a/src/code-execution/script-tool-names.ts +++ b/src/code-execution/script-tool-names.ts @@ -18,27 +18,29 @@ * direct call (an alias, a reference), names it inside a definition, whose * body runs later, when it's called (a def's header and indented block, a * class's methods among them, or a lambda), or keeps a string that is the - * tool's whole name, as a key is, without calling it at once, as - * `tools["world--say"](…)` does. A direct call anywhere else finishes inside - * its own script, a class's own body included, since it runs as the class - * statement does; and a string that only mentions a tool, as a printed - * sentence does, leaves nothing reachable. + * tool's whole name, optionally namespaced, as a key kept for later is. A + * key called at once, as in `tools["world--say"](…)`, is a direct call. A + * direct call outside a definition leaves nothing reachable, a class's own + * body included, since it runs as the class statement does, and so does a + * string that only mentions a tool, as a printed sentence does. * * Some reaches go unseen, such as a tool name the code doesn't spell out * (built at runtime, written with escapes, or in non-ASCII spellings), code - * loaded from a file or module, an enumeration of the tools, a definition or - * alias made inside a string the script runs or inside an f-string's field, - * a call that outlives its script (one it awaits only in a later script, - * schedules as a task, or leaves in a generator expression), and code a - * later script runs from a string an earlier one kept. + * loaded from a file or module, an enumeration of the tools, a call written + * in a string's text (code a script runs with exec or eval, or an f-string's + * field), inside a definition too, a definition or alias made inside such a + * string, a call that outlives its script (one it awaits only in a later + * script, schedules as a task, or leaves in a generator expression), and + * code a later script runs from a string an earlier one kept. * * @param tools plain tool names, of identifier characters */ export function scriptToolNames(script: string, tools: readonly string[]): { names: Set; reachable: Set } { + if (tools.length === 0) return { names: new Set(), reachable: new Set() }; const named = new RegExp(`(?:(?(); const reachable = new Set(); for (const match of code.matchAll(named)) { @@ -68,20 +70,28 @@ const ESCAPE = /\\(?:x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8}|N\{[^}]*\}| /** * Whether an offset of a script's code falls inside a definition, whose body * runs later, when it's called: a def's header and the indented block after - * it, or a lambda from its keyword to the end of its line. A line inside - * brackets, after a backslash, or inside a string literal that an earlier - * line opened continues the line before it, whatever its indentation; and a - * line of only whitespace, a carriage return included, doesn't end a block. + * it, or a lambda from its keyword to the end of its logical line. A line's + * indent is read from the script itself, so a line that opens with a string + * stands where it's written. A line inside brackets, after a backslash, or + * inside a string literal that an earlier line opened continues the line + * before it, whatever its indentation; and a line holding only whitespace (a + * carriage return included) or a comment doesn't end a block. */ -function definitions(code: string, strings: readonly { start: number; end: number }[]): (offset: number) => boolean { +function definitions( + script: string, + code: string, + strings: readonly { start: number; end: number }[], +): (offset: number) => boolean { const lines: { start: number; text: string; indent: number; blank: boolean; continued: boolean }[] = []; let depth = 0; let continued = false; let start = 0; for (const text of code.split('\n')) { - const indent = /^[ \t]*/.exec(text)![0].length; + const written = script.slice(start, start + text.length); + const indent = /^[ \t]*/.exec(written)![0].length; + const blank = /^\s*(?:#[^\n]*)?$/.test(written); const inString = strings.some((s) => s.start < start && start < s.end); - lines.push({ start, text, indent, blank: text.trim() === '', continued: continued || depth > 0 || inString }); + lines.push({ start, text, indent, blank, continued: continued || depth > 0 || inString }); for (const c of text) { if (c === '(' || c === '[' || c === '{') depth++; else if ((c === ')' || c === ']' || c === '}') && depth > 0) depth--; diff --git a/src/types/trace.ts b/src/types/trace.ts index 5704dc8ee..874bd2531 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -184,8 +184,9 @@ export type TraceEvent = * id, index or end: routeSpeech's once delivered, the `sleep` tool's * announcement among them, and a subconscious's `speak_in_channel` as * it is sent, before its outcome is known. A relay that voices the - * agent's prose takes this trace alone, not both, and takes each - * message's id from its `messageId` to edit what it voiced. + * agent's prose takes this trace, with `inference:published_complete`, + * and not `mcpl:speech-routed` as well, and takes each message's id + * from its `messageId` to edit what it voiced. */ type: 'inference:published'; agentName: string; diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 621d6efc9..76ab1355e 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -1192,21 +1192,30 @@ describe("a round that runs code_execution is routed once its scripts return (re scriptCall({ code: 'await test__slow({})' }, 'ce1'), scriptCall({ code: 'post = test__send_message' }, 'ce2'), ] as ContentBlock[], 'tool_use'), + // The control: the same script, run alone, marks. + createMockResponse([ + { type: 'text', text: 'now alone' }, + scriptCall({ code: 'post = test__send_message' }, 'ce3'), + ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), ]); - // Let the first script go once the second's refusal has come back. + // Let the first script go once the second's refusal has come back, and + // read the mark once the first has finished, before the control runs. const results: string[] = []; + let markAfterRefusal: string[] | undefined = ['unread']; h.framework.onTrace((event) => { if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') { results.push(event.callId); if (event.callId === 'ce2') h.module.releaseSlow(); + if (event.callId === 'ce1') markAfterRefusal = markOf(h.framework); } }); const bound = setTimeout(() => h.module.releaseSlow(), 15000); try { await h.run(); - assert.deepStrictEqual(results, ['ce2', 'ce1'], "the second script was refused while the first ran"); - assert.equal(markOf(h.framework), undefined, 'the refused alias never ran, so it marks nothing'); + assert.deepStrictEqual(results, ['ce2', 'ce1', 'ce3'], "the second script was refused while the first ran"); + assert.equal(markAfterRefusal, undefined, 'the refused alias never ran, so it marked nothing'); + assert.deepStrictEqual(markOf(h.framework), ['send_message'], 'the same script, run alone, marks'); } finally { clearTimeout(bound); await h.done(); diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts index 8eac8d648..dcbf07968 100644 --- a/test/script-tool-names.test.ts +++ b/test/script-tool-names.test.ts @@ -24,6 +24,12 @@ test('a direct call leaves nothing reachable, by name or by key', () => { assert.deepEqual(read('await think({"content": "notes"})\nawait tools["think"]({"content": "more"})'), { names: ['think'], reachable: [] }); assert.deepEqual(read('await tools[ "world--say" ] ({})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('await say ({})'), { names: ['say'], reachable: [] }, 'a space before the call'); +}); + +test('no tools names nothing', () => { + assert.deepEqual([...scriptToolNames('await say({})', []).names], []); + assert.deepEqual([...scriptToolNames('await say({})', []).reachable], []); }); test('an alias, a definition or a kept key leaves the tool reachable', () => { @@ -32,11 +38,26 @@ test('an alias, a definition or a kept key leaves the tool reachable', () => { { names: ['send_message'], reachable: ['send_message'] }); assert.deepEqual(read('quiet = lambda: skip_reply({})'), { names: ['skip_reply'], reachable: ['skip_reply'] }); assert.deepEqual(read('name = "test--send_message"'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('name = "say"'), { names: ['say'], reachable: ['say'] }, 'a bare whole name, as a key kept in a variable'); +}); + +test('a key is a whole tool name, optionally namespaced, not a sentence that ends in one', () => { + assert.deepEqual(read('print("to speak, call world--say")'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('print("use mcpl__world__say")'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('k = "mcpl__world__say"'), { names: ['say'], reachable: ['say'] }); }); test("a direct call outside a definition leaves nothing reachable, beside a definition that doesn't use the tool", () => { assert.deepEqual(read('def fmt(x):\n return x\nawait say({"text": fmt("hi")})'), { names: ['say'], reachable: [] }); assert.deepEqual(read('await say(sorted(xs, key=lambda x: x))'), { names: ['say'], reachable: [] }, 'a lambda after the call'); + assert.deepEqual(read('k = lambda x: x\nawait say({})'), { names: ['say'], reachable: [] }, "a lambda's span ends with its logical line"); + // A line that opens with a string stands where it's written: at column 0 it + // ends the def before it, as Python reads it. + assert.deepEqual(read('def f():\n return 1\n"%s" % say({})'), { names: ['say'], reachable: [] }, 'a call on a line that opens with a string'); + assert.deepEqual(read('def f():\n return 1\n"""note"""\nawait say({})'), { names: ['say'], reachable: [] }, 'a string-only line after a def'); + assert.deepEqual(read('def f():\n# a comment at column 0\n await say({})'), { names: ['say'], reachable: ['say'] }, "a comment doesn't end a def"); + assert.deepEqual(read('def f():\n return "("\nawait say({})'), { names: ['say'], reachable: [] }, "a bracket in a string continues no line"); + assert.deepEqual(read('print("a lambda"); await say({})'), { names: ['say'], reachable: [] }, "a lambda in a string isn't one"); }); test('a name inside a definition is reachable, a called key included, whatever lines the definition spans', () => { @@ -50,8 +71,11 @@ test('a name inside a definition is reachable, a called key included, whatever l // A line a string literal carries over continues its statement, whatever its // indentation (Ramona-2416); so does a blank line written with \r\n. assert.deepEqual(read('if True:\n def f():\n q = """\n"""; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a triple-quoted string'); - assert.deepEqual(read('def f():\n q = "a\\\n"; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a backslash-continued string'); - assert.deepEqual(read('def f():\r\n x = 1\r\n\r\n await say({})\r\n'), { names: ['say'], reachable: ['say'] }, 'a blank \\r\\n line'); + assert.deepEqual(read('if True:\n def f():\n q = "a\\\n"; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a backslash-continued string'); + assert.deepEqual(read('def f():\r\n x = 1\r\n\r\n # a note\r\n await say({})\r\n'), { names: ['say'], reachable: ['say'] }, 'a blank \\r\\n line and a comment'); + // A backslash at a line's end continues it in code too. + assert.deepEqual(read('def f():\n x = 1 + \\\n2; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line continued by a backslash in code'); + assert.deepEqual(read('f = (lambda x:\n say(x))'), { names: ['say'], reachable: ['say'] }, "a lambda's body on its continued line"); }); test("a class's own body runs as the class statement does, so a direct call there leaves nothing reachable", () => { @@ -91,6 +115,9 @@ test('an escaped quote stays inside its string', () => { test('a name right after an escape in a string counts, as the string run as code would call it', () => { assert.deepEqual(read('exec("x = 1\\nsay({})")'), { names: ['say'], reachable: [] }); + for (const escape of ['\\x0a', '\\u000a', '\\U0000000a', '\\N{LINE FEED}', '\\012']) { + assert.deepEqual(read(`exec("x = 1${escape}say({})")`), { names: ['say'], reachable: [] }, escape); + } }); test("an f-string's call inside its braces counts for its own round but leaves nothing reachable", () => { @@ -107,11 +134,12 @@ test("a string in one quote mark, not three, ends at its line's end, so a stray assert.deepEqual(read("x = 'unclosed\npost = test__send_message"), { names: ['send_message'], reachable: ['send_message'] }); }); -test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call that outlives its script, an alias made inside a string or an f-string's field, code kept in a string", () => { +test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call in a string's text inside a definition, a call that outlives its script, an alias made inside a string or an f-string's field, code kept in a string", () => { assert.deepEqual(read('name = "skip" + "_reply"\nawait tools[name]({})'), { names: [], reachable: [] }, 'built at runtime'); assert.deepEqual(read('await tools["s\\x61y"]({})'), { names: [], reachable: [] }, 'written with escapes'); assert.deepEqual(read('fns = list(tools.values())'), { names: [], reachable: [] }, 'an enumeration'); assert.deepEqual(read('exec(open("helper.py").read())'), { names: [], reachable: [] }, 'code from a file'); + assert.deepEqual(read('def f():\n exec("say({})")'), { names: ['say'], reachable: [] }, "a call in a string's text inside a definition"); assert.deepEqual(read('pending = say({"text": "hi"})'), { names: ['say'], reachable: [] }, 'a call made here...'); assert.deepEqual(read('await pending'), { names: [], reachable: [] }, '...and awaited later'); assert.deepEqual(read('asyncio.create_task(say({"text": "hi"}))'), { names: ['say'], reachable: [] }, 'a call scheduled as a task'); From a115ab1b439ef2069a54e3bfc94986d6a8c3e68b Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Fri, 16 Oct 2026 23:28:06 -0400 Subject: [PATCH 10/19] fix(routing): a kept key's prefix may hold any punctuation, and the disabled row checks it ran From the round-3 review: - R3-1: a kept tools[...] key marked no interpreter when the tool's prefix held punctuation other than `-` or `_`, such as `mcpl--zulip.work--send_message` or `discord:v1--send_message`, so a later script's call through it went unseen. The key's prefix is now any text without whitespace. A sentence that ends in a tool's name, which has spaces, still counts only for its own round. - R3-2: the disabled row asserted only absences. It now also checks that each branch it covers ran, by the line each logs when it routes nothing (mid-turn, trailing and text-only prose), and that the tool round's call was made. Tests: script-tool-names gains a `.` and a `:` prefix row beside the sentence row; outgoing-stream's disabled row gains its positive control. Co-Authored-By: Claude Opus 5.5 --- src/code-execution/script-tool-names.ts | 4 +++- test/outgoing-stream.test.ts | 22 ++++++++++++++++------ test/script-tool-names.test.ts | 3 +++ 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts index 205d21b20..719c52e6c 100644 --- a/src/code-execution/script-tool-names.ts +++ b/src/code-execution/script-tool-names.ts @@ -38,7 +38,9 @@ export function scriptToolNames(script: string, tools: readonly string[]): { names: Set; reachable: Set } { if (tools.length === 0) return { names: new Set(), reachable: new Set() }; const named = new RegExp(`(?:(?(); diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 8ce344a84..1f2f03f2a 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -616,13 +616,23 @@ test("with the real channel registry, a stop() with a publish still out sends th }); test('disabled: nothing streams, from a mid-turn round, trailing text or a text-only turn', async () => { - for (const script of [ - [round(text('mid-turn'), call('one', 'test--read')), final(text('trailing'))], - [final(text('text only'))], - ]) { - const h = await harness([script], { proseRouting: 'disabled' }); + for (const [script, branches] of [ + [[round(text('mid-turn'), call('one', 'test--read')), final(text('trailing'))], ['mid-turn prose', 'tool-call trailing prose']], + [[final(text('text only'))], ['text-only prose']], + ] as const) { + const h = await harness([[...script]], { proseRouting: 'disabled' }); + const called: string[] = []; + h.module.during = async (call) => { called.push(call.name); }; + // Each branch the turn reaches says it routed nothing; the test checks it reached them. + const logged: string[] = []; + const error = console.error; try { - await h.run(); + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + try { await h.run(); } finally { console.error = error; } + for (const branch of branches) { + assert.ok(logged.includes(`[routing] assistant: ${branch} NOT routed (proseRouting=disabled)`), `the ${branch} branch ran`); + } + assert.deepEqual(called, script.length > 1 ? ['read'] : [], "the tool round's call was made"); assert.deepEqual(h.log, [], 'nothing is published, streamed or completed'); assert.deepEqual(h.traces, [], 'and no published trace is emitted'); } finally { await h.framework.stop(); } diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts index dcbf07968..53368ffa7 100644 --- a/test/script-tool-names.test.ts +++ b/test/script-tool-names.test.ts @@ -45,6 +45,9 @@ test('a key is a whole tool name, optionally namespaced, not a sentence that end assert.deepEqual(read('print("to speak, call world--say")'), { names: ['say'], reachable: [] }); assert.deepEqual(read('print("use mcpl__world__say")'), { names: ['say'], reachable: [] }); assert.deepEqual(read('k = "mcpl__world__say"'), { names: ['say'], reachable: ['say'] }); + // A prefix can hold punctuation besides `-` and `_`. + assert.deepEqual(read('post = tools["mcpl--zulip.work--say"]'), { names: ['say'], reachable: ['say'] }, 'a `.` in the prefix'); + assert.deepEqual(read('post = tools["discord:v1--say"]'), { names: ['say'], reachable: ['say'] }, 'a `:` in the prefix'); }); test("a direct call outside a definition leaves nothing reachable, beside a definition that doesn't use the tool", () => { From fd964be072efb0873a444970ef087e2f216c47e7 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sat, 17 Oct 2026 00:19:28 -0400 Subject: [PATCH 11/19] test(code-execution): the disposed-ephemeral test no longer races python3's start "an ephemeral disposed while its round's script runs doesn't publish the round when the script returns" disposed the ephemeral through a 300 ms idle watchdog, which counts from the round's dispatch. The script's python3 had to start and reach the slow call inside those 300 ms, so on a loaded machine the watchdog could dispose the ephemeral before the script was running, and the test failed on "the script was running when the ephemeral was disposed". With python3 delayed 0.6 s it fails every time. The watchdog's threshold is now 60 s, and the test makes the run look idle only once the script is in the slow call, so the next poll disposes it then. Co-Authored-By: Claude Opus 5.5 --- test/code-execution.test.ts | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 76ab1355e..6831d6502 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -1451,11 +1451,19 @@ describe("a round that runs code_execution is routed once its scripts return (re try { const created = await h.framework.createEphemeralAgent({ name: 'eph-script', model: 'test-model', systemPrompt: 'Act.', allowedTools: 'all' }); created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); - // The idle watchdog disposes the ephemeral while its script waits in the slow call. - const run = h.framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 15000, idleTimeoutMs: 300, idlePollMs: 20 }); + // The idle watchdog disposes the ephemeral while its script waits in the + // slow call. Its threshold is too long for python3's start to trip it, so + // the run is made to look idle only once the script is in the slow call, + // and the next poll disposes it. + const run = h.framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 15000, idleTimeoutMs: 60_000, idlePollMs: 20 }); + const stalled = assert.rejects(run, /stalled/); h.framework.start(); - await assert.rejects(run, /stalled/); - assert.ok(h.log.includes('tool:slow'), 'the script was running when the ephemeral was disposed'); + const deadline = Date.now() + 15000; + while (Date.now() < deadline && !h.log.includes('tool:slow')) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached the slow call'); + const runs = (h.framework as unknown as { ephemeralRuns: Map }).ephemeralRuns; + runs.get(created.agent.name)!.lastActivity = 0; + await stalled; // The script now returns, and its result arrives with the framework still running. let resulted!: () => void; const result = new Promise((resolve) => { resulted = resolve; }); From c9d92a2a29e69e9e3f0bffde6339a54d0c4687a9 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sat, 17 Oct 2026 13:12:09 -0400 Subject: [PATCH 12/19] fix(code-execution): read a def header across backslash joins definitions() never took a continued line as a def header. So a helper written as `async \` with `def quiet():` on the next line, which Python reads as one `async def`, got no span: its skip_reply counted as a direct call, and no mark was left on the interpreter. A later script calling the helper then published its round's prose as main does, which the helper's skip_reply was there to keep private. The header is now read as its logical line, joining the lines a trailing backslash continues, so `async \` over any number of joins before `def` is a header too. Greptile's second review found it (discussion_r4303632390). Co-Authored-By: Claude Opus 5.5 --- src/code-execution/script-tool-names.ts | 9 ++++++++- test/script-tool-names.test.ts | 3 +++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts index 719c52e6c..3ff2e0d9d 100644 --- a/src/code-execution/script-tool-names.ts +++ b/src/code-execution/script-tool-names.ts @@ -104,7 +104,14 @@ function definitions( const startOf = (j: number) => (j < lines.length ? lines[j].start : code.length); const spans: [number, number][] = []; lines.forEach((line, i) => { - if (line.continued || !/^[ \t]*(?:async[ \t]+)?def\b/.test(line.text)) return; + if (line.continued) return; + // A header read as Python reads its logical line: `async \` and `def` on + // the next line are one `async def`. + let header = line.text; + for (let k = i + 1; k < lines.length && header.trimEnd().endsWith('\\'); k++) { + header = `${header.trimEnd().slice(0, -1)} ${lines[k].text}`; + } + if (!/^[ \t]*(?:async[ \t]+)?def\b/.test(header)) return; // The header's own continued lines, then the block indented under it. let j = i + 1; while (j < lines.length && (lines[j].blank || lines[j].continued || lines[j].indent > line.indent)) j++; diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts index 53368ffa7..379b71809 100644 --- a/test/script-tool-names.test.ts +++ b/test/script-tool-names.test.ts @@ -78,6 +78,9 @@ test('a name inside a definition is reachable, a called key included, whatever l assert.deepEqual(read('def f():\r\n x = 1\r\n\r\n # a note\r\n await say({})\r\n'), { names: ['say'], reachable: ['say'] }, 'a blank \\r\\n line and a comment'); // A backslash at a line's end continues it in code too. assert.deepEqual(read('def f():\n x = 1 + \\\n2; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line continued by a backslash in code'); + // ...a header's too: `async \` with `def` on the next line is one `async def`. + assert.deepEqual(read('async \\\ndef quiet():\n await skip_reply({})\nawait say({})'), { names: ['say', 'skip_reply'], reachable: ['skip_reply'] }, 'async joined to def by a backslash'); + assert.deepEqual(read('async \\\n \\\n def quiet():\n await skip_reply({})'), { names: ['skip_reply'], reachable: ['skip_reply'] }, 'a header over two backslash joins'); assert.deepEqual(read('f = (lambda x:\n say(x))'), { names: ['say'], reachable: ['say'] }, "a lambda's body on its continued line"); }); From 9fd8b7f066b90bade8e99fbb5306b390bb018915 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sat, 17 Oct 2026 16:08:47 -0400 Subject: [PATCH 13/19] test(skip_reply): a wake refused inside a script, in XML tool mode In XML tool mode nothing routes a round live, so what keeps a round private when its script's skip_reply is refused is the round's place in refusedWakeRounds, which the turn-end delivery drops. No test reached that. Two cases in skip-reply-wake-xml-mode.test.ts now do: a script round whose skip_reply is refused, then a reply; and the same round followed by a public one. For them the harness turns code execution on and offers skip_reply as a channel tool a script can call. Both cases come from a reader's probe of the merge. The comment on silenceBeforeRefusal said the silence is saved as its round is dispatched. It's saved as the round is routed, which for a round that waits on scripts is when they return. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 7 +++-- test/skip-reply-wake-xml-mode.test.ts | 42 +++++++++++++++++++++++---- 2 files changed, 40 insertions(+), 9 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index b619e176f..26e98f474 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -12879,9 +12879,10 @@ export class AgentFramework { const refusedWakeRounds = new Set(); let toolRounds = 0; // A refused skip_reply silences only its own round: the silence before it, - // saved as its round is dispatched, comes back at the next round or at - // 'complete'. It's this stream's: a budget restart starts from its - // trigger, and a tool-result-guard retry takes the turn's silence as + // saved as its round is routed (once its calls are dispatched, or, for a + // round that waits on scripts, once they return), comes back at the next + // round or at 'complete'. It's this stream's: a budget restart starts from + // its trigger, and a tool-result-guard retry takes the turn's silence as // 'complete' restored it, through guardRetryTurnSilenced. let silenceBeforeRefusal: boolean | undefined; diff --git a/test/skip-reply-wake-xml-mode.test.ts b/test/skip-reply-wake-xml-mode.test.ts index 518f3659f..1f86ff6b5 100644 --- a/test/skip-reply-wake-xml-mode.test.ts +++ b/test/skip-reply-wake-xml-mode.test.ts @@ -2,7 +2,9 @@ * skip_reply's refused wake in XML tool mode: without an EventGate, a round * that calls skip_reply with wake_in_seconds is refused and the turn goes on. * That round's prose stays private, and every other round's prose is - * delivered as it would be without the refusal, in order. + * delivered as it would be without the refusal, in order. A skip_reply + * refused inside a round's code_execution script keeps its round private the + * same way. * * In XML tool mode, tool-calls events carry no roundContent, so no round is * routed live: the turn's prose is delivered at its end, from the turn-end @@ -41,6 +43,9 @@ const echo = (id: string, text: string, ...more: string[]): Round => ({ results: `${[id, ...more].map(() => '{"echoed":"x"}').join('')}`, }); const refusedWake = (id: string, text: string): Round => ({ text, calls: [{ id, name: 'skip_reply', input: { reason: 'later', wake_in_seconds: 60 } }], results: 'refused' }); +/** A round whose one call is a foreground code_execution script. */ +const script = (id: string, text: string, code: string): Round => ({ text, calls: [{ id, name: 'code_execution', input: { code } }], results: '' }); +const REFUSED_IN_SCRIPT = 'await skip_reply({"wake_in_seconds": 60})'; class XmlStream implements YieldingStream { private events: StreamEvent[] = []; @@ -118,8 +123,13 @@ class EchoModule implements Module { async onProcess(_event: ProcessEvent, _state: ProcessState): Promise { return {}; } } -/** One XML-mode turn with no EventGate: what reached the channel, and whether each round's result was an error. */ -async function xmlTurn(rounds: Round[], finalText: string, thinking = false) { +/** + * One XML-mode turn with no EventGate: what reached the channel, whether each + * round's result was an error, and which channel tools the registry handled. + * With scripts, code execution is on and skip_reply is a channel tool a script + * can call. + */ +async function xmlTurn(rounds: Round[], finalText: string, thinking = false, opts: { scripts?: boolean } = {}) { let stream: XmlStream | undefined; const membrane = { async complete(_r: NormalizedRequest): Promise { throw new Error('not used'); }, @@ -132,12 +142,20 @@ async function xmlTurn(rounds: Round[], finalText: string, thinking = false) { const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: membrane as unknown as import('@animalabs/membrane').Membrane, agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], modules: [new EchoModule()], + ...(opts.scripts ? { codeExecution: { enabled: true } } : {}), }); const routed: string[] = []; + const registrySaw: string[] = []; + const channelTools = opts.scripts + ? [{ name: 'skip_reply', description: 'End your turn without sending anything.', inputSchema: { type: 'object' as const, properties: {} } }] + : []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ - getChannelTools: () => [], resolveLocus: () => 'channel:test', + getChannelTools: () => channelTools, resolveLocus: () => 'channel:test', routeSpeech: async (_agent: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'channel:test' }; }, - handleChannelToolCall: async () => ({ success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }), + handleChannelToolCall: async (name: string) => { + registrySaw.push(name); + return { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }; + }, }, { get: (target, prop: string) => prop in target ? target[prop as keyof typeof target] : () => undefined }); try { const agent = framework.getAgent('agent')!; @@ -145,7 +163,7 @@ async function xmlTurn(rounds: Round[], finalText: string, thinking = false) { .startAgentStream(agent, { agentName: 'agent', reason: 'test', source: 'test', timestamp: Date.now() }); await framework.runUntilIdle(); const errors = (stream?.received ?? []).map((r) => (r[0] as { isError?: boolean } | undefined)?.isError === true); - return { routed, errors }; + return { routed, errors, registrySaw }; } finally { await framework.stop(); rmSync(dir, { recursive: true, force: true }); @@ -213,4 +231,16 @@ describe('skip_reply refused in XML tool mode: its round stays private, and the assert.deepEqual(r.errors, [false, true, false]); assert.deepEqual(r.routed, ['ONE', 'TWO', 'THIRD']); }); + + it('a skip_reply refused inside a code_execution script keeps its round private too', async () => { + const r = await xmlTurn([script('ce1', 'SCRIPT_PRIVATE', REFUSED_IN_SCRIPT)], '\nFINAL_REPLY', false, { scripts: true }); + assert.ok(!r.registrySaw.includes('skip_reply'), 'the host refused the call before the registry saw it'); + assert.deepEqual(r.routed, ['FINAL_REPLY']); + }); + + it('after a script round whose skip_reply was refused, the next round is delivered', async () => { + const r = await xmlTurn([script('ce1', 'SCRIPT_PRIVATE', REFUSED_IN_SCRIPT), echo('e1', 'TWO')], '\nTHIRD', false, { scripts: true }); + assert.ok(!r.registrySaw.includes('skip_reply'), 'the host refused the call before the registry saw it'); + assert.deepEqual(r.routed, ['TWO', 'THIRD']); + }); }); From ce34f8f2b6a7654e5d3b0e647c444c33ca536265 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sun, 18 Oct 2026 07:04:46 -0400 Subject: [PATCH 14/19] docs(changelog): list a kept key with whitespace in its prefix among what goes unseen The scanner marks a kept key whose prefix, as the script writes it, holds no whitespace (`\S*`). A real space, tab or no-break space in a key's prefix leaves the key unseen, while an escaped `\t` is seen, since the scanner reads the script's source text. The fixed fragment's list of what goes unseen now names it. Raised in review of agent-framework#360 (R4-1). Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.fixed.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index 0ea18a675..b55ccf496 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -65,7 +65,8 @@ isn't held for the script's runtime. What goes unseen is routed as it was before scripts counted, such as a tool name the code doesn't spell out (built at runtime, written with escapes, or in non-ASCII spellings), code - loaded from a file or module, an enumeration of the tools, a call written + loaded from a file or module, an enumeration of the tools, a kept key + whose prefix, as the script writes it, contains whitespace, a call written in a string's text (code run with `exec` or `eval`, or an f-string's field), inside a definition too, a definition or alias made inside such a string, a call that outlives its script (one it awaits only in a later From 802af12de98e01b98c0ace7d428d6484a0fcb3b5 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sun, 18 Oct 2026 07:20:14 -0400 Subject: [PATCH 15/19] test(streaming): an unconfirmed publish streams nothing on every path, and a script's skip_reply outranks its send The outgoing-stream stand-in's publish can now come back unknown, as the real registry reads a reply that leaves `delivered` out, an error answer or a timeout: it may or may not have been posted. Each delivery path's failure test, and the test of a stream going on beside a lost publish, run with it too. At a `>>` or `>>>` envelope, streaming on any outcome but `failed` now fails them; nothing caught that before. A script that both sends and calls skip_reply keeps its round's prose private and holds no draft, in locus and hybrid mode, as the same calls made directly do: a deliberate silence outranks a send's hold. Ranking the send first in roundSilenceCause now fails it; nothing caught that before. Co-Authored-By: Claude Opus 5.5 --- test/code-execution.test.ts | 17 +++++++++++++++++ test/outgoing-stream.test.ts | 30 +++++++++++++++++++++--------- 2 files changed, 38 insertions(+), 9 deletions(-) diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 1376440b4..341a44ab9 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -1309,6 +1309,23 @@ describe("a round that runs code_execution is routed once its scripts return (re } finally { await h.done(); } }); + // skip_reply's privacy outranks a send's hold, as for direct calls: a + // round whose script calls both keeps its prose private, and holds no draft. + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a script that sends and calls skip_reply keeps its round's prose private, with no draft`, async () => { + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: 'posting the summary, then going quiet' }, + scriptCall({ code: 'await test__send_message({})\nawait skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message') && h.log.includes('tool:skip_reply'), 'the script sent and skipped'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], 'nothing is published'); + assert.deepStrictEqual(heldDrafts(h.framework), [], 'and the round is kept private, not held as a draft'); + } finally { await h.done(); } + }); + it("explicit: a send inside the script silences neither its round nor the turn's later prose, as a direct send doesn't", async () => { const h = await scriptRound('explicit', [ createMockResponse([ diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts index 66652d5d4..3d94d2e63 100644 --- a/test/outgoing-stream.test.ts +++ b/test/outgoing-stream.test.ts @@ -5,9 +5,9 @@ * delivery withholds never reaches it: a round that a silencing tool, * `skip_reply` or a private `think` keeps back, text under a hybrid hold, a * bounce, or a round the stream never finished. Nor does a publish that - * throws or that the server reports undelivered. A voice server speaks what - * it gets, and words already spoken can't be taken back, so the stream can't - * run ahead of delivery. + * throws, that the server reports undelivered, or that it doesn't confirm. + * A voice server speaks what it gets, and words already spoken can't be + * taken back, so the stream can't run ahead of delivery. */ import { afterEach, test } from 'node:test'; @@ -103,8 +103,13 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = const log: Logged[] = []; /** A publish whose text is held here doesn't return until the test lets it go. */ const holds = new Map>(); - /** A publish of one of these texts fails: the server reports it undelivered (`delivered: false`), or it throws. */ - const fails = new Map(); + /** + * A publish of one of these texts fails: the server reports it undelivered + * (`delivered: false`); its reply confirms nothing, so it may or may not + * have been posted (`unknown`, as the real registry reads a reply that + * leaves `delivered` out, an error answer or a timeout); or it throws. + */ + const fails = new Map(); /** A publish of one of these texts is answered with this message id, as a server that returns one. */ const messageIds = new Map(); const known: Record = { '#alpha': 'chan-alpha', '#cafe': 'chan-cafe', '@laria': 'discord:dm:99' }; @@ -118,6 +123,9 @@ function registry(framework: AgentFramework, options: { streaming?: boolean } = if (failure === 'throw') throw new Error('synthetic publish failure'); const destination = { serverId: 'stub', channelId }; const messageId = messageIds.get(body); + if (failure === 'unknown') { + return { status: 'unknown' as const, destination, reason: 'the stub returned no delivery receipt (delivery uncertain)', at: Date.now() }; + } return failure === undefined ? { status: 'delivered' as const, destination, ...(messageId !== undefined ? { messageId } : {}), at: Date.now() } : { status: 'failed' as const, destination, reason: 'the stub reported delivered: false', at: Date.now() }; @@ -438,8 +446,12 @@ test("explicit: a stream's complete waits for the deliveries it queued, so no ch } }); -for (const failure of ['undelivered', 'throw'] as const) { - test(`a publish that ${failure === 'throw' ? 'throws' : 'the server reports undelivered'} never streams, and the stream goes on with what is`, async () => { +/** How a test title says a publish failed. */ +const failedHow = (failure: 'undelivered' | 'unknown' | 'throw'): string => + failure === 'throw' ? 'throws' : failure === 'unknown' ? "the server doesn't confirm" : 'the server reports undelivered'; + +for (const failure of ['undelivered', 'unknown', 'throw'] as const) { + test(`a publish that ${failedHow(failure)} never streams, and the stream goes on with what is`, async () => { const h = await harness([[round(text('lost'), call('one', 'test--read')), final(text('kept'))]]); h.fails.set('lost', failure); try { @@ -716,8 +728,8 @@ test('a published trace carries no message id when the server returned none', as }); for (const c of failedPublishCases) { - for (const failure of ['undelivered', 'throw'] as const) { - test(`${c.name}: a publish that ${failure === 'throw' ? 'throws' : 'the server reports undelivered'} streams nothing`, async () => { + for (const failure of ['undelivered', 'unknown', 'throw'] as const) { + test(`${c.name}: a publish that ${failedHow(failure)} streams nothing`, async () => { const h = await harness([c.script()], { proseRouting: c.mode }); h.fails.set(c.lost, failure); try { From 21d718188e67bac198254f9ea9dc4b0680d9e1a2 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sun, 18 Oct 2026 07:24:42 -0400 Subject: [PATCH 16/19] docs(streaming): the published trace's doc and the fragment follow the confirmed-publish rule inference:published's doc still said a reply that leaves `delivered` out counts as delivered. Since the merge of main, a publish streams only once the server confirms it, so the doc now lists each publish that isn't streamed: one refused before sending, one that threw, timed out or got an error answer, one reported undelivered, and one whose reply leaves `delivered` out. It also names drafts among what delivery withholds, and says speech publishes reach mcpl:speech-routed through deliverSpeech. The fixed fragment said that in XML tool mode an inner silencing call silences the turn's later prose. Nothing is routed there until the turn ends, so a silencing send holds the whole turn's prose as drafts, the rounds before it included, made directly or inside a script. A comment's lines are reflowed. Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.fixed.md | 7 ++++--- src/framework.ts | 8 ++++---- src/types/trace.ts | 21 +++++++++++---------- 3 files changed, 19 insertions(+), 17 deletions(-) diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index b55ccf496..0bd3d94ea 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -37,9 +37,10 @@ a host without an EventGate, counts as a refused direct one: it keeps only its own round private, and the turn goes on. On the fallback without `roundContent`, as in XML tool mode, an inner call counts only - as a direct one does there: a silencing call silences the turn's later - prose, while the private `think` hold and an explicit round's - `skip_reply` don't apply (#307). + as a direct one does there. Nothing is routed until the turn ends, so a + silencing send holds the whole turn's prose as drafts, the rounds before + it included. The private `think` hold and an explicit round's + `skip_reply` don't apply there (#307). - A round whose script can change its routing waits for the script, and its prose goes out when the script finishes. A script can change it when its code names a tool whose call changes the round's routing in the turn's diff --git a/src/framework.ts b/src/framework.ts index cf9d8db22..06839b4d0 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -15522,10 +15522,10 @@ export class AgentFramework { // A refused skip_reply silences only its own round: the silence before it, // saved with its cause as its round is routed (once its calls are // dispatched, or, for a round that waits on scripts, once they return), - // comes back at the next - // round or at 'complete'. It's this stream's: a budget restart starts from - // its trigger, and a tool-result-guard retry takes the turn's silence as - // 'complete' restored it, through guardRetryTurnSilenced. + // comes back at the next round or at 'complete'. It's this stream's: a + // budget restart starts from its trigger, and a tool-result-guard retry + // takes the turn's silence as 'complete' restored it, through + // guardRetryTurnSilenced. let silenceBeforeRefusal: 'send' | 'private' | null | undefined; // Typing indicator: show " is typing…" for the whole duration of diff --git a/src/types/trace.ts b/src/types/trace.ts index dc997dfef..6afc449af 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -180,17 +180,18 @@ export type TraceEvent = * emitted whether or not the channel's server takes the stream. It * carries only what was published, so a consumer that speaks or shows * it, such as a voice relay, never gets text that delivery withholds (a - * `skip_reply`, a silencing send, a private `think`, a hold, a bounce) - * or a publish that failed: one refused before sending, one that threw - * or timed out, or one the server reported undelivered - * (`delivered: false`). A reply that leaves `delivered` out counts as - * delivered, as delivery counts it. `inference:tokens` stays the - * model's text as it is written (held, as before, while a tool-result - * guard's batch is pending). A later publish to the same channel by the - * same stream opens with a paragraph break (`\n\n`), so a channel's - * chunks joined in `index` order are its text, its messages kept apart. + * `skip_reply`, a silencing send, a private `think`, a hold, a bounce, + * a draft) or a publish the server didn't confirm (`delivered: true`): + * one refused before sending, one that threw, timed out or got an + * error answer, one the server reported undelivered + * (`delivered: false`), and one whose reply leaves `delivered` out. + * `inference:tokens` stays the model's text as it is written (held, as + * before, while a tool-result guard's batch is pending). A later + * publish to the same channel by the same stream opens with a paragraph + * break (`\n\n`), so a channel's chunks joined in `index` order are its + * text, its messages kept apart. * `mcpl:speech-routed` also reports speech publishes, with no stream - * id, index or end: routeSpeech's once delivered, the `sleep` tool's + * id, index or end: deliverSpeech's once delivered, the `sleep` tool's * announcement among them, and a subconscious's `speak_in_channel` as * it is sent, before its outcome is known. A relay that voices the * agent's prose takes this trace, with `inference:published_complete`, From 7674c9f6346086d7f47c2438b7df33d7b2defb41 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Sun, 18 Oct 2026 11:05:42 -0400 Subject: [PATCH 17/19] docs(changelog): a refused inner send's sentence says its one reading Co-Authored-By: Claude Opus 5.5 --- changelog.d/hold-round-speech.fixed.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md index 0f14cb217..7453271a8 100644 --- a/changelog.d/hold-round-speech.fixed.md +++ b/changelog.d/hold-round-speech.fixed.md @@ -37,8 +37,8 @@ allowedTools doesn't name it or because this host can't set the `wake_in_seconds` it asks for, counts as a refused direct one: it keeps only its own round private, and the turn goes on. An inner send the - allowedTools refuses silences nothing, as a refused direct one doesn't. - On the fallback without `roundContent`, as in XML tool mode, an inner + allowedTools refuses, like a refused direct one, silences nothing. On + the fallback without `roundContent`, as in XML tool mode, an inner call counts only as a direct one does there. Nothing is routed until the turn ends, so a silencing send holds the whole turn's prose as drafts, the rounds before it included. The private `think` hold and an explicit From d30c8fc51471e9c49efb46d0ece9222d4a420a54 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Mon, 19 Oct 2026 06:17:57 -0400 Subject: [PATCH 18/19] test(streaming): the speech-routed check leaves out the open a first publish now announces Since main's channel lifecycle, sending to a channel seeded closed opens it, and the first publish's mcpl:channel-opened-by-send trace comes before its mcpl:speech-routed. The test leaves that trace out and checks every other: one speech trace per publish, with its whole text and no stream id or index, and none that ends an utterance. Co-Authored-By: Claude Opus 5.5 --- test/channel-registry-routing.test.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test/channel-registry-routing.test.ts b/test/channel-registry-routing.test.ts index 9dc22cd57..874046a5a 100644 --- a/test/channel-registry-routing.test.ts +++ b/test/channel-registry-routing.test.ts @@ -187,7 +187,9 @@ test('routeSpeech reports each delivered publish as mcpl:speech-routed, with not await registry.routeSpeech('cairn', 'first reply', registry.resolveLocus('cairn')); await registry.routeSpeech('cairn', 'second reply', registry.resolveLocus('cairn')); - const routed = traces.slice(before); + // The first publish also opens the channel, and says so in a trace of its + // own; every other trace the two sends give is checked. + const routed = traces.slice(before).filter((t) => t.type !== 'mcpl:channel-opened-by-send'); assert.deepEqual(routed.map((t) => ({ type: t.type, channelId: t.channelId, text: t.text, delivered: t.delivered })), [ { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'first reply', delivered: true }, { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'second reply', delivered: true }, From 8fe9a1ddf2073454babce20962224ab9ff5ef0e2 Mon Sep 17 00:00:00 2001 From: Humphrey-2407 Date: Mon, 19 Oct 2026 06:28:19 -0400 Subject: [PATCH 19/19] docs(changelog): leave the allowlist fragment as 0.23.0 published it 0.23.0 folded the allowlist fragment into its release notes, and the release's merge back into main deletes it. This branch's own fixed fragment says what changes from 0.23.0: a skip_reply the allowedTools refuses inside a foreground script keeps only its own round private, and a refused inner send silences nothing, as the same direct calls do. Co-Authored-By: Claude Opus 5.5 --- changelog.d/agent-allowed-tools.fixed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/agent-allowed-tools.fixed.md b/changelog.d/agent-allowed-tools.fixed.md index be9b0f84e..c6c0114eb 100644 --- a/changelog.d/agent-allowed-tools.fixed.md +++ b/changelog.d/agent-allowed-tools.fixed.md @@ -1 +1 @@ -- Enforce each registered agent's allowedTools before model, script and public agent-origin dispatch, including utility names inside utils run. Filter utility list/describe to the same executable names. Refused calls retain paired error results without inner lifecycle state. A refusal withholds a call's effect, never the model's choice about its own words: a refused send changes nothing about prose, so its round and what follows go out; a `skip_reply` that a round calls, directly or from a foreground `code_execution` script, keeps that round's prose private and silences nothing after it, as a refused wake already did, when the agent's allowedTools doesn't name it, another module's namespaced `skip_reply` included, or when it's the registry's own `skip_reply` asking for a wake this host can't set; and a refused `think` is read as a permitted one is, so the private same-round policy keeps its round private wherever it applies, which is where rounds are routed live. Framework-created identities (ephemerals and conversation forks) have no tool authority while unregistered, including after disposal; genuine unregistered legacy callers and trusted module closures retain their behavior. Prose primers and bounce notices advertise prose_help only when permitted, and `think`'s and `journal`'s descriptions tell an agent to call `skip_reply` only when it may. `think`'s result and `prose_help`'s reference no longer point at the `skip_reply` tool. +- Enforce each registered agent's allowedTools before model, script and public agent-origin dispatch, including utility names inside utils run. Filter utility list/describe to the same executable names. Refused calls retain paired error results without inner lifecycle state. A refusal withholds a call's effect, never the model's choice about its own words: a refused send changes nothing about prose, so its round and what follows go out; a `skip_reply` that a round calls itself (not one a `code_execution` script makes) keeps that round's prose private and silences nothing after it, as a refused wake already did, when the agent's allowedTools doesn't name it, another module's namespaced `skip_reply` included, or when it's the registry's own `skip_reply` asking for a wake this host can't set; and a refused `think` is read as a permitted one is, so the private same-round policy keeps its round private wherever it applies, which is where rounds are routed live. Framework-created identities (ephemerals and conversation forks) have no tool authority while unregistered, including after disposal; genuine unregistered legacy callers and trusted module closures retain their behavior. Prose primers and bounce notices advertise prose_help only when permitted, and `think`'s and `journal`'s descriptions tell an agent to call `skip_reply` only when it may. `think`'s result and `prose_help`'s reference no longer point at the `skip_reply` tool.