diff --git a/changelog.d/hold-round-speech.added.md b/changelog.d/hold-round-speech.added.md new file mode 100644 index 000000000..1f382f298 --- /dev/null +++ b/changelog.d/hold-round-speech.added.md @@ -0,0 +1,17 @@ +- `inference:published` and `inference:published_complete` traces carry the + outgoing stream to in-process consumers: each chunk as it is sent, with + its `channelId`, `inferenceId`, `index` and `text`, and the `messageId` + the channel's server returned for that publish, when it returned one; and + each channel's `complete`. They are emitted whether or not the channel's + server takes the stream, and carry only what delivery published, so a host + relay that voices them, such as a TTS bridge, never speaks text that + `skip_reply`, a send or a private `think` withheld, or a publish the + server didn't confirm. `inference:tokens` is unchanged: it carries the + model's text as it is written, held, as before, while a tool-result + guard's batch is pending. + `mcpl:speech-routed` also reports speech publishes, with no stream id, + index or end: among them the `sleep` tool's announcement and a + subconscious's `speak_in_channel`. A relay that voices the agent's prose + takes the `inference:published` stream, with its + `inference:published_complete`, and not `mcpl:speech-routed` as well, and + takes each message's id from `inference:published` to edit what it voiced. diff --git a/changelog.d/hold-round-speech.changed.md b/changelog.d/hold-round-speech.changed.md new file mode 100644 index 000000000..40f959206 --- /dev/null +++ b/changelog.d/hold-round-speech.changed.md @@ -0,0 +1,19 @@ +- **MCPL servers that take the outgoing stream (`channels.streaming`):** + chunks now arrive once each publish succeeds, one per published + message, rather than token by token while the model writes. A round + reaches a voice server once the model has finished writing it, its + routing is decided and its publish succeeds, after any earlier post + of the agent's still in flight. A later message to the same channel + within one stream opens with a paragraph break (`\n\n`), so the + channel's chunks joined in `index` order are its text, which `complete` + carries, with its messages kept apart. A server that read chunks as a + sign the agent is composing, such as eidoverse-worlds' typing dots, now + sees them only as a reply is published; `channels/typing`, which the + host sends to a server granted `channels.typing` while a turn runs on + one of its channels, is that signal. +- Agents with `toolWrapperProseGuard` now stream their delivered prose. The + stream was off for them because it ran ahead of the guard's + classification; it now carries only prose that delivery published, and a + wrapper the guard contains is never published. Wrapper-shaped text the + guard doesn't match, as in a turn that has made a real tool call, was + already posted as ordinary prose, and now streams too. diff --git a/changelog.d/hold-round-speech.fixed.md b/changelog.d/hold-round-speech.fixed.md new file mode 100644 index 000000000..7453271a8 --- /dev/null +++ b/changelog.d/hold-round-speech.fixed.md @@ -0,0 +1,102 @@ +- The MCPL outgoing stream (`channels/outgoing/chunk` and `complete`) no + longer carries text that delivery withholds or sends elsewhere. It + streamed each round's text as the model wrote it, but whether a round is + delivered is decided only once its tool calls are known. So a streaming + server received, and discord-mcpl's voice spoke: + - a round that called `skip_reply`, a silencing send such as + `send_message` or `channel_publish`, or a `think` under + `same_round_think_text_policy: private`; + - a round its stream never finished, such as one cut off by a stream + error; + - in hybrid mode, in a later stream of the turn, such as a budget + restart's, a framework retry's or a re-inference's, text the turn's + hold kept back or its pin sent to another channel; + - in locus and hybrid mode, after a mid-turn re-pin, text bound for the + new locus, which streamed to the old one. + + The stream is now the prose that delivery published: each publish's + text, to its channel, sent once the publish succeeds, a later one to the + same channel opening with a paragraph break. Tool publications, such as + `send_message`, a `drafts` resend or the `sleep` tool's announcement, + aren't part of it, as before. A publish succeeds only when the server + confirms it (`delivered: true`), so one refused before sending, one that + throws, times out or gets an error response, one the server reports + undelivered (`delivered: false`) and one whose reply leaves `delivered` + out are never streamed, and neither is text held as a draft. A + `{{unsent}}` resend streams the draft's words it sends, rather than the + token. +- With native tool calls (`roundContent`), a call made inside a foreground + `code_execution` script now counts toward its round's routing, as the + same call made directly does (#306). In explicit mode an inner + `skip_reply` keeps the round's envelopes in the context. In locus and + hybrid mode an inner `skip_reply` keeps the round's prose private, and an + inner silencing send, a `drafts` resend among them, holds it and the + turn's later prose as drafts, as a direct one does. An inner + `think` under the private same-round policy holds the round back in + every mode. An inner `skip_reply` that's refused, because the agent's + allowedTools doesn't name it or because this host can't set the + `wake_in_seconds` it asks for, counts as a refused direct one: it keeps + only its own round private, and the turn goes on. An inner send the + allowedTools refuses, like a refused direct one, silences nothing. On + the fallback without `roundContent`, as in XML tool mode, an inner + call counts only as a direct one does there. Nothing is routed until the + turn ends, so a silencing send holds the whole turn's prose as drafts, + the rounds before it included. The private `think` hold and an explicit + round's `skip_reply` don't apply there (#307). +- A round whose script can change its routing waits for the script, and its + prose goes out when the script finishes. A script can change it when its + code names a tool whose call changes the round's routing in the turn's + mode: in locus and hybrid mode `skip_reply`, a silencing send, `drafts` + (whose resend is one), `say` or `whisper`; in explicit mode only + `skip_reply`; and in either, `think` under the private same-round + policy. Disabled mode holds nothing. A name + counts in code wherever it stands as a whole word of ASCII letters, digits + and underscores, or after `__` as in `mcpl__world__say`, as a function is + called or a `tools[...]` key is written: `say_hello` doesn't name `say`. A + name in a comment doesn't count, since Python never runs one, while a name + anywhere in a string does, since a string can be a key or code to run. A + script can also change its round's routing when the interpreter it runs in + has run a script that left such a tool reachable, by naming the tool + inside a definition (a function or lambda, a class's methods among them) + or other than in a direct call, or by keeping a string that is the tool's + whole name, optionally namespaced, as a key kept for later is. That holds + until the interpreter is reclaimed or restarted. A direct call outside a + definition, a class's own body included, or a string that only mentions a + tool, as a printed sentence does, leaves nothing reachable. Without code + execution, a `code_execution` call holds nothing. Any other round is + routed at its tool-calls event, as before, so narration beside a script + isn't held for the script's runtime. What goes unseen is routed as it was + before scripts counted, such as a tool name the code doesn't spell out + (built at runtime, written with escapes, or in non-ASCII spellings), code + loaded from a file or module, an enumeration of the tools, a kept key + whose prefix, as the script writes it, contains whitespace, a call written + in a string's text (code run with `exec` or `eval`, or an f-string's + field), inside a definition too, a definition or alias made inside such a + string, a call that outlives its script (one it awaits only in a later + script, schedules as a task, or leaves in a generator expression), and + code a later script runs from a string an earlier one kept. The turn's + locus and an explicit round's `>>` targets are taken at the round's + tool-calls event, when the model has finished writing it, not when its + scripts return; hybrid's `>>>` targets apply as its delivery runs, as for + any queued hybrid delivery. A background script's call returns as soon as + the script has started, and its later calls aren't the round's. A script's + result settles only the round that dispatched it, even when a newer round + reuses its call id. A round whose turn ends or stops waiting on it, or + that a newer round of the same agent replaces, before its scripts return, + as when the turn is stopped, is withheld: it is neither sent nor held as + a draft, a log line names it, and in locus and hybrid mode the turn's + delivery receipt counts it as suppressed. +- Every stream now sends `channels/outgoing/complete` for each channel it + streamed to, on every exit path, as MCPL Spec 14.3 asks: once its + deliveries have run, or, when the host stops, at once, with what it has + sent, so a publish that succeeds after the stop began isn't streamed. A + stream that handed its turn to a budget restart or a tool-result-guard + retry never sent it, so a voice server never learned that stream's + utterance had ended. +- A membrane refusal retry no longer moves the outgoing stream to a new + inference id. Nothing of the refused attempt was delivered, so none of it + streamed; the rounds delivered before the refusal stay in the stream, and + its `complete` carries them. The stream's `inference/lifecycle` `started` + and terminal now carry that one id too. The retry re-minted the id + between them, so a server matching them by `inferenceId` saw the first + inference never end, and a terminal for an id that never started. diff --git a/docs/hybrid-prose-routing.md b/docs/hybrid-prose-routing.md index 14696c14d..7d0defaff 100644 --- a/docs/hybrid-prose-routing.md +++ b/docs/hybrid-prose-routing.md @@ -30,8 +30,8 @@ names a destination that resolves; if that envelope's body then isn't delivered, the hold returns. The sticky destination and this hold last for the whole logical turn, across its later streams, such as a framework retry's, a budget restart's or the compile a provider-capacity hold defers -the turn to. They govern publication: a stream's live outgoing deltas still -start at the turn's locus. +the turn to. They govern publication, and with it the outgoing stream, which +carries only what delivery publishes, where delivery publishes it. This is publication routing only. It creates no privacy meaning for `` or other prose tags, and grants no channel authority beyond the existing diff --git a/docs/tool-result-guard.md b/docs/tool-result-guard.md index eb4aa9ae7..b89d65774 100644 --- a/docs/tool-result-guard.md +++ b/docs/tool-result-guard.md @@ -64,10 +64,11 @@ wire cost (chars/4 plus a flat per-image cost) in `reserveForResponse`, so substituting them for the placeholders cannot push a restart's request past its budget. -While a submitted batch is pending (or a recovery is running), streamed text -and its `inference:tokens` trace events are held at one publication +While a submitted batch is pending (or a recovery is running), the +round's `inference:tokens` trace events are held at one publication boundary; they are released in order on a clean round and discarded on a -guard refusal. +guard refusal. The outgoing stream needs no hold: it carries only what +delivery publishes, and a refused attempt publishes nothing. Normal successful rounds admit the preceding results to memory. This works for framework yielding streams (including ephemeral agents and context-budget @@ -271,9 +272,9 @@ retroactive and does not rewrite results already accepted into memory. - **Token stats.** The context manager's token-stats cache is not write-through on edit, so a stats read taken while a batch was pending can keep pricing that message as the placeholder. -- **Held stream text on abort.** Text held while a submitted batch is - pending is dropped (not previewed) if the stream aborts or errors before - the round settles. +- **Held token traces on abort.** `inference:tokens` traces held while a + submitted batch is pending are dropped (not previewed) if the stream + aborts or errors before the round settles. - **Withheld originals in the workspace.** Every reader of the mount can read a withheld original's file, and nothing removes it; delete it as any workspace file. diff --git a/docs/tool-wrapper-prose-guard.md b/docs/tool-wrapper-prose-guard.md index 65d395839..18be1e28b 100644 --- a/docs/tool-wrapper-prose-guard.md +++ b/docs/tool-wrapper-prose-guard.md @@ -4,7 +4,7 @@ The guard matches only when the entire visible prose is one wrapper naming a tool registered on that exact inference. Signed thinking may precede the visible wrapper. It does not parse or execute the wrapper body. -Guarded turns buffer prose until completion so outgoing preview/voice surfaces cannot expose a wrapper before classification. Ordinary non-wrapper prose is then delivered normally at turn completion. +The outgoing preview/voice stream carries only prose that delivery has published, so a wrapper the guard contains never reaches it. Prose the guard doesn't match, wrapper-shaped text in a turn that has made a real tool call among it, is ordinary prose: delivery posts it, and it streams once its publish succeeds. On a match, Agent Framework: diff --git a/src/code-execution/py-runner.ts b/src/code-execution/py-runner.ts index 397be0054..5da51ff1c 100644 --- a/src/code-execution/py-runner.ts +++ b/src/code-execution/py-runner.ts @@ -197,6 +197,9 @@ interface PendingExec { sent: boolean; } +/** Interpreters started in this process, by every runner: each one's number. */ +let interpretersStarted = 0; + /** * The interpreter to spawn. A bare name is looked up on PATH. A path with a * separator is resolved against the host's working directory, where it @@ -226,6 +229,8 @@ export class PyRunner { private pending: PendingExec | null = null; private idleTimer: ReturnType | null = null; private execCounter = 0; + /** The number of this runner's latest interpreter (interpretersStarted). */ + private interpreterNumber = 0; private disposed = false; constructor(options: PyRunnerOptions) { @@ -248,6 +253,16 @@ export class PyRunner { return this.pending !== null; } + /** + * The live interpreter, whose state (variables, imports, functions) the + * scripts run in it share: a number no other interpreter in this process + * has had, from any runner, or null while none is running, before the + * first script or once it was reclaimed, killed or has exited. + */ + get interpreter(): number | null { + return this.child && this.child.exitCode === null && !this.child.killed ? this.interpreterNumber : null; + } + /** * Run one script. Tools are (re-)injected before every exec so the * interpreter always reflects the current tool surface (list_changed etc.). @@ -438,6 +453,7 @@ export class PyRunner { env: { ...buildChildEnv(this.envOptions), PYTHONUNBUFFERED: '1', PYTHONIOENCODING: 'utf-8' }, }); this.child = child; + this.interpreterNumber = ++interpretersStarted; child.stderr.setEncoding('utf8'); child.stderr.on('data', (chunk: string) => { diff --git a/src/code-execution/script-tool-names.ts b/src/code-execution/script-tool-names.ts new file mode 100644 index 000000000..3ff2e0d9d --- /dev/null +++ b/src/code-execution/script-tool-names.ts @@ -0,0 +1,164 @@ +/** + * Which of some tools a code_execution script names, and which of them it + * leaves reachable by a later script in the same interpreter, read past the + * script's comments and string literals as Python reads them. The framework + * holds a script's round for the names it returns, and marks the agent's + * interpreter with the reachable ones (AgentFramework.scriptMayRoute). + * + * A script reaches a tool through a function named for it (the tool's name, + * after any prefix joined with `__`) or a `tools[...]` key. So a name counts + * in code wherever it stands as a whole identifier: after a character other + * than an ASCII letter, digit or underscore, or after `__`, and before such a + * character. `say_hello` and `my_say` don't name `say`; `mcpl__world__say(` + * does. A comment names nothing, since Python never runs one. A name anywhere + * in a string counts, an escape before it read as a break, since a string + * can be a key or code passed to eval or exec. + * + * A script leaves a tool reachable when it names the tool other than in a + * direct call (an alias, a reference), names it inside a definition, whose + * body runs later, when it's called (a def's header and indented block, a + * class's methods among them, or a lambda), or keeps a string that is the + * tool's whole name, optionally namespaced, as a key kept for later is. A + * key called at once, as in `tools["world--say"](…)`, is a direct call. A + * direct call outside a definition leaves nothing reachable, a class's own + * body included, since it runs as the class statement does, and so does a + * string that only mentions a tool, as a printed sentence does. + * + * Some reaches go unseen, such as a tool name the code doesn't spell out + * (built at runtime, written with escapes, or in non-ASCII spellings), code + * loaded from a file or module, an enumeration of the tools, a call written + * in a string's text (code a script runs with exec or eval, or an f-string's + * field), inside a definition too, a definition or alias made inside such a + * string, a call that outlives its script (one it awaits only in a later + * script, schedules as a task, or leaves in a generator expression), and + * code a later script runs from a string an earlier one kept. + * + * @param tools plain tool names, of identifier characters + */ +export function scriptToolNames(script: string, tools: readonly string[]): { names: Set; reachable: Set } { + if (tools.length === 0) return { names: new Set(), reachable: new Set() }; + const named = new RegExp(`(?:(?(); + const reachable = new Set(); + for (const match of code.matchAll(named)) { + names.add(match[1]); + CALLED.lastIndex = match.index + match[0].length; + if (!CALLED.test(code) || inDefinition(match.index)) reachable.add(match[1]); + } + for (const { text, start, end } of strings) { + // The name itself counts as any name in a string does, just below. + const whole = key.exec(text); + if (whole) { + KEY_CALLED.lastIndex = end; + if (!KEY_CALLED.test(code) || inDefinition(start)) reachable.add(whole[1]); + } + for (const match of text.replace(ESCAPE, ' ').matchAll(named)) names.add(match[1]); + } + return { names, reachable }; +} + +/** A direct call right after a name in code: `(`, on the same line. */ +const CALLED = /[ \t]*\(/y; +/** A direct call right after a key: a `tools[...]` subscript's close and then `(`, on the same line. */ +const KEY_CALLED = /[ \t]*\][ \t]*\(/y; +/** A backslash escape in a string's text, which a name after it follows as it would a break. */ +const ESCAPE = /\\(?:x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8}|N\{[^}]*\}|[0-7]{1,3}|[\s\S])/g; + +/** + * Whether an offset of a script's code falls inside a definition, whose body + * runs later, when it's called: a def's header and the indented block after + * it, or a lambda from its keyword to the end of its logical line. A line's + * indent is read from the script itself, so a line that opens with a string + * stands where it's written. A line inside brackets, after a backslash, or + * inside a string literal that an earlier line opened continues the line + * before it, whatever its indentation; and a line holding only whitespace (a + * carriage return included) or a comment doesn't end a block. + */ +function definitions( + script: string, + code: string, + strings: readonly { start: number; end: number }[], +): (offset: number) => boolean { + const lines: { start: number; text: string; indent: number; blank: boolean; continued: boolean }[] = []; + let depth = 0; + let continued = false; + let start = 0; + for (const text of code.split('\n')) { + const written = script.slice(start, start + text.length); + const indent = /^[ \t]*/.exec(written)![0].length; + const blank = /^\s*(?:#[^\n]*)?$/.test(written); + const inString = strings.some((s) => s.start < start && start < s.end); + lines.push({ start, text, indent, blank, continued: continued || depth > 0 || inString }); + for (const c of text) { + if (c === '(' || c === '[' || c === '{') depth++; + else if ((c === ')' || c === ']' || c === '}') && depth > 0) depth--; + } + continued = text.trimEnd().endsWith('\\'); + start += text.length + 1; + } + const startOf = (j: number) => (j < lines.length ? lines[j].start : code.length); + const spans: [number, number][] = []; + lines.forEach((line, i) => { + if (line.continued) return; + // A header read as Python reads its logical line: `async \` and `def` on + // the next line are one `async def`. + let header = line.text; + for (let k = i + 1; k < lines.length && header.trimEnd().endsWith('\\'); k++) { + header = `${header.trimEnd().slice(0, -1)} ${lines[k].text}`; + } + if (!/^[ \t]*(?:async[ \t]+)?def\b/.test(header)) return; + // The header's own continued lines, then the block indented under it. + let j = i + 1; + while (j < lines.length && (lines[j].blank || lines[j].continued || lines[j].indent > line.indent)) j++; + spans.push([line.start, startOf(j)]); + }); + for (const lambda of code.matchAll(/\blambda\b/g)) { + let j = 0; + while (j < lines.length && lines[j].start <= lambda.index) j++; + while (j < lines.length && lines[j].continued) j++; + spans.push([lambda.index, startOf(j)]); + } + return (offset) => spans.some(([from, to]) => offset >= from && offset < to); +} + +/** + * A script's code, with each comment and string literal blanked to spaces + * (its newlines kept, so offsets hold), and each literal's text with the + * offsets of its opening quote and just past its close. A literal may be + * triple-quoted and take any prefix (r, b, f, u), and a backslash keeps the + * character after it inside the literal, raw or not, as Python reads it. An + * f-string's replacement fields are read as its text. A literal in single + * quotes that reaches the end of its line ends there, so a stray quote can't + * take the rest of the script. + */ +function readScript(script: string): { code: string; strings: { text: string; start: number; end: number }[] } { + const strings: { text: string; start: number; end: number }[] = []; + let code = ''; + let at = 0; + const special = /[#'"]/g; + for (let found = special.exec(script); found; found = special.exec(script)) { + const start = found.index; + let end: number; + if (found[0] === '#') { + const lineEnd = script.indexOf('\n', start); + end = lineEnd === -1 ? script.length : lineEnd; + } else { + const quote = script.startsWith(found[0].repeat(3), start) ? found[0].repeat(3) : found[0]; + let close = start + quote.length; + while (close < script.length && !script.startsWith(quote, close) && (quote.length === 3 || script[close] !== '\n')) { + close += script[close] === '\\' ? 2 : 1; + } + close = Math.min(close, script.length); + end = script.startsWith(quote, close) ? close + quote.length : close; + strings.push({ text: script.slice(start + quote.length, close), start, end }); + } + code += script.slice(at, start) + script.slice(start, end).replace(/[^\n]/g, ' '); + at = special.lastIndex = end; + } + return { code: code + script.slice(at), strings }; +} diff --git a/src/framework.ts b/src/framework.ts index bc909dfa1..174ca4058 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -99,7 +99,6 @@ import { import type { ChannelIncomingMessage, ChannelIncomingMessageResult, McplContentBlock, McpToolResultContent, PushEventResult } from './mcpl/types.js'; import { isSilentHeartbeatMarker, isVisiblyEmptyContent, NON_STRING_TEXT_PLACEHOLDER } from './mcpl/visible-content.js'; import { parseProsePrefix, parseHybridProsePrefix } from './mcpl/prose-grammar.js'; -import { ProseStreamRouter } from './mcpl/prose-stream-router.js'; import { detectKnownToolWrapperProse } from './tool-wrapper-prose-guard.js'; import { InferenceRouter } from './mcpl/inference-router.js'; import { ChannelRegistry, connectorText, journalToolDescription, type ChannelToolOrigin, type PublishDestination, type PublishOutcome } from './mcpl/channel-registry.js'; @@ -133,6 +132,7 @@ import { } from './tool-image-ledger.js'; import { randomUUID, createHash } from 'node:crypto'; import { PyRunner, buildInjectedTools, formatLimit, type ScriptToolResult } from './code-execution/py-runner.js'; +import { scriptToolNames } from './code-execution/script-tool-names.js'; import { buildCodeExecutionToolDefinition, CODE_EXECUTION_TOOL_NAME, @@ -282,6 +282,29 @@ const silenceCauseOf = (calls: Array<{ name: string; input?: unknown }>): 'send' return calls.some(isSilencingCall) ? 'send' : null; }; +/** + * Whether a call of one of these tools can change a round's routing + * (routeRound): in locus and hybrid mode a silencing or world-publication + * tool, or `drafts`, whose resend is an explicit send (isSilencingCall); in + * explicit mode, where sends don't silence, only `skip_reply`, which keeps + * the round private; in either, `think` under the private same-round policy, + * which holds the round back. Disabled mode routes nothing. + */ +const changesRouting = ( + named: ReadonlySet, + mode: NonNullable, + privateThink: boolean, +): boolean => { + if (mode === 'disabled') return false; + for (const name of named) { + if (name === 'think' ? privateThink : mode !== 'explicit' || name === 'skip_reply') return true; + } + return false; +}; + +/** The tools whose call can change a round's routing in some mode or policy (changesRouting). */ +const ROUTING_TOOLS = [...SILENCING_TOOLS, ...WORLD_PUBLICATION_TOOLS, 'drafts', 'think']; + /** * True when an injected message is real conversational input — something the * agent might actually be replying to — rather than ambient machinery. @@ -426,9 +449,9 @@ const isAddressedMessage = ( _metadata?: Record, ): boolean => Array.isArray(tags) && tags.includes('chat:addressed'); -// parseProsePrefix moved to ./mcpl/prose-grammar.ts — ONE grammar shared with -// the outgoing-stream router (Spec 14.3), so streamed chunks and delivered -// envelopes can never disagree on what is a prefix. +// parseProsePrefix lives in ./mcpl/prose-grammar.ts. The outgoing stream +// (Spec 14.3) needs no grammar of its own: it carries what delivery +// published. /** One-time primer appended when an agent's proseRouting mode changes. */ function proseModePrimer(mode: 'explicit' | 'hybrid' | 'locus' | 'disabled', helpAvailable: boolean): string { @@ -816,18 +839,6 @@ interface TurnRouting { engaged: Set; /** The sticky destination, set by each `>>` (hybrid: `>>>`) prefix that resolves. */ pin?: string; - /** - * Explicit mode: where each of the turn's streams starts. It changes as a - * round is admitted, before its delivery (roundDestination): a `>>` target - * that resolves sets it, and one refused for speech clears it. A target - * that matches no channel, or several, leaves it, and so does a round that - * names none, or one kept private by skip_reply. A failed delivery to a channel that is - * inbound-only by then clears it too, while it still names that channel - * (deliverProseEnvelope). The turn's final text doesn't set it: the stream - * settles the turn before delivering that text, so no later stream of the - * turn starts after it, and a later start opens a turn of its own. - */ - destination: string | null; /** * Hybrid: the router is fail-closed until a later `>>>` names a target that * resolves. After a `>>>skip_reply` envelope ('private') the rest stays @@ -839,7 +850,7 @@ interface TurnRouting { continuation: boolean; /** Successful plain-prose destinations in delivery order; deduped by the receipt. */ deliveries: string[]; - /** Plain-prose segments the turn suppressed without drafts (proseRouting=disabled), reported by its receipt. */ + /** Plain-prose segments the turn suppressed without drafts (proseRouting=disabled, or a round withheld before its scripts returned: ScriptRound.withhold), reported by its receipt. */ suppressed: number; /** * Drafts held during the turn (src/prose-drafts.ts), which its receipt @@ -886,11 +897,27 @@ let lastTurnId = 0; function freshTurnRouting(): TurnRouting { return { turnId: ++lastTurnId, - engaged: new Set(), destination: null, + engaged: new Set(), continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; } +/** + * What a prose delivery tells its stream once a publish succeeds: the + * channel and the text it published. The stream that queued the delivery + * sends them on as its outgoing stream (MCPL Spec 14.3), so the stream + * carries only what delivery published: nothing it withholds or holds as a + * draft, and nothing whose publish failed or may not have gone out. A + * publish succeeds when its outcome is `delivered`, which takes the + * server's `delivered: true` (ChannelRegistry.publish). Nothing else is + * streamed: a send refused before it goes out (such as for no channel, a + * closed channel that won't open, or a server without `channels.publish`), + * one that throws, and one sent with no receipt saying delivered (an error + * response, a timeout, a lost connection, a reply that leaves `delivered` + * out, or `delivered: false`). + */ +type SpeechPublished = (channelId: string, text: string, messageId: string | undefined) => void; + /** * How a round's prose is delivered: under which turn's routing, and how what * it doesn't send is held. `notice` says when the notice for an envelope that @@ -904,6 +931,57 @@ interface ProseRound { routing: TurnRouting; notice: 'now' | 'later'; keepPrivate?: boolean; + /** The outgoing stream of the stream that queued the round. */ + onPublished?: SpeechPublished; +} + +/** + * A call that counts toward a round's routing, made directly or by the + * round's foreground script (AgentFramework.roundCallOf): why it silences the + * round's prose, if it does (silenceCauseOf), and whether it's a refused + * skip_reply, which silences only its own round. + */ +interface RoundCall { + name: string; + cause: 'send' | 'private' | null; + refused: boolean; +} + +/** Why a round's calls silence its prose, as silenceCauseOf reads them: a + * deliberate silence outranks a send. */ +const roundSilenceCause = (calls: readonly RoundCall[]): 'send' | 'private' | null => { + if (calls.some((c) => c.cause === 'private')) return 'private'; + return calls.some((c) => c.cause === 'send') ? 'send' : null; +}; + +/** + * A round whose `code_execution` scripts can change its routing + * (AgentFramework.scriptMayRoute), waiting for those calls to return before + * its prose is routed: a foreground script's own calls count as the round's, + * so a `skip_reply` or a send inside the script silences the round as a + * direct call does (AgentFramework.scriptRounds). + */ +interface ScriptRound { + /** The agent and stream that dispatched the round. */ + agent: Agent; + streamId: number; + /** Each code_execution call the round waits on, with the tools its foreground script has called so far, in order. */ + scripts: Map; + /** Those calls whose results haven't arrived. */ + pending: Set; + /** Routes the round's prose; called once, when its last code_execution result arrives. */ + route: () => void; + /** + * Withholds the round's prose when its routing can't be decided: before + * its scripts returned, its stream ended or no longer waits on it, or a + * newer round of the same agent name replaced it. Whether a script would + * have silenced the round is unknown, and a skip_reply's silence would + * keep its prose private, so it is neither sent nor held as a draft. A log + * line names it, and in locus and hybrid mode the turn's delivery receipt + * counts its segments as suppressed: a turn that a later stream continues + * writes its receipt after the withhold. + */ + withhold: (why: string) => void; } /** What an ephemeral run ends with once stop() has begun, whatever ends it: @@ -2130,13 +2208,13 @@ export class AgentFramework { /** * Each agent's routing state for its current logical turn (TurnRouting): * its trigger channel, locus and engaged channels, the sticky destination, - * the turn's destination, hybrid's hold, a ` !` continuation and receipt - * tallies. The start that opens a turn makes a - * new object instead of clearing the old one, and every later stream of - * the turn shares it (openLogical). A queued delivery holds the object of - * the turn that queued it, so prose a turn leaves queued when it ends (a - * skip_reply ends it before its posts have run) still follows that turn's - * destination, and its late writes can't reach the next turn. + * hybrid's hold, a ` !` continuation and receipt tallies. The start that + * opens a turn makes a new object instead of clearing the old one, and + * every later stream of the turn shares it (openLogical). A queued delivery + * holds the object of the turn that queued it, so prose a turn leaves + * queued when it ends (a skip_reply ends it before its posts have run) + * still follows that turn's routing, and its late writes can't reach the + * next turn. */ private turnRouting: Map = new Map(); /** @@ -2145,6 +2223,15 @@ export class AgentFramework { * posts go out in the order they were written, across streams and turns. */ private proseSpeechChains: Map> = new Map(); + /** + * Each stream whose `channels/outgoing/complete` hasn't gone yet, from + * its start: an ended one's waits for the deliveries it queued + * (driveStream's finally), and one the framework no longer tracks, such + * as a disposed ephemeral's, may still be running. stop() sends them at + * once, before it clears the channel registry and closes the MCPL + * connections. + */ + private pendingOutgoingCompletes: Set<() => void> = new Set(); /** * Each agent's open logical turn: its routing state and receipt tallies * (TurnRouting). It stays open from the start that opens it until the @@ -2473,8 +2560,22 @@ export class AgentFramework { * script's code_execution result instead of cancelling the stream * mid-script (which would wedge the turn). A stopped script's late result * marks only that script, never the agent's next one. Its `turnId` is the - * turn of the code_execution call running it, which its calls carry. */ - private scriptDeferredEndTurn: Map = new Map(); + * turn of the code_execution call running it, which its calls carry. + * `calls` records each tool the script calls, for its round's routing + * (ScriptRound). */ + private scriptDeferredEndTurn: Map = new Map(); + /** Per agent, a round that called code_execution and waits for those calls' + * results before its prose is routed (ScriptRound). */ + private scriptRounds: Map = new Map(); + /** Each code_execution result, by the round that dispatched its call: it + * settles only that round, even if a newer round reuses its call id. */ + private scriptRoundOfResult: WeakMap = new WeakMap(); + /** Per agent, the foreground interpreter (PyRunner.interpreter) that ran a + * script leaving a tool that can change routing reachable, and the tools + * its scripts named: a later script there can call them, so the agent's + * script rounds wait while that interpreter lives, when one of them can + * change that round's routing (scriptMayRoute). */ + private scriptRoutingInterpreters: Map }> = new Map(); /** RFC-007: per agent, the origin of the foreground script its runner is * executing, handed to that script's inner calls (scripts run one at a * time per runner). Background runners capture theirs in the closure. */ @@ -3469,6 +3570,7 @@ export class AgentFramework { step(() => runner.dispose()); } this.codeExecutionRunners.clear(); + this.scriptRoutingInterpreters.clear(); // Background daemons die with the process (documented limitation) — // mark cancelled FIRST so their settle path stays silent (no crash-wake // into a framework that is shutting down). @@ -3617,7 +3719,17 @@ export class AgentFramework { shutdownErrors.push(error); } - // Stop typing indicators + // A stream's `complete` waits for the deliveries it queued, and the + // teardown below would leave it nowhere to go: the registry forgets its + // channels, and closing an MCPL connection marks it closed before it + // rejects a publish still out. So each stream that hasn't sent it sends + // it now, with what it has streamed, and streams nothing after + // (pendingOutgoingCompletes): every stream activeStreams tracked has + // ended by now, and one it no longer tracks, such as a disposed + // ephemeral's, may still be finishing. + for (const complete of [...this.pendingOutgoingCompletes]) step(complete); + + // Stop typing indicators, and forget the registry's channels step(() => this.channelRegistry?.stopAll()); // Dispose EventGate (clear debounce timers) @@ -10726,6 +10838,10 @@ export class AgentFramework { // itself is never sent, only whether it was an error. this.toolLifecycleEmitter?.onResult(event.agentName, event.callId, event.result); + // A code_execution result: once its round has them all, the round's + // prose is routed, before anything here can end or continue the turn. + this.settleScriptRound(event.agentName, event.callId, event.result); + const agent = this.agents.get(event.agentName); if (agent) { this.touchEphemeralRun(event.agentName, true); @@ -13327,7 +13443,7 @@ export class AgentFramework { return stamps; } - /** Record plain-prose segments this turn suppressed without drafts (proseRouting=disabled). */ + /** Record plain-prose segments this turn suppressed without drafts (proseRouting=disabled, or a round withheld before its scripts returned). */ private recordProseSuppression(routing: TurnRouting, count: number): void { if (count <= 0) return; routing.suppressed += count; @@ -14484,12 +14600,13 @@ export class AgentFramework { } /** - * Explicit-prose-routing delivery gateway (docs/explicit-prose-routing.md). - * Every prose segment of an `explicit`-mode agent flows through here: - * parse the `>>` prefix, resolve the target, deliver — or bounce to the - * held bounce draft with a resend notice. Misdelivery is structurally impossible: - * nothing is ever sent to a destination the model did not name (directly - * or via the turn's sticky target). + * Explicit-prose-routing delivery gateway (docs/explicit-prose-routing.md), + * entered directly: parse the `>>` prefix, resolve the target, deliver, or + * bounce to the held bounce draft with a resend notice. Misdelivery is + * structurally impossible: nothing is ever sent to a destination the model + * did not name (directly or via the turn's sticky target). A stream's own + * deliveries go through deliverAdmittedProse with its outgoing stream + * (onPublished); this entry has no stream, and only tests call it. */ private async deliverProse( agent: Agent, @@ -14507,9 +14624,9 @@ export class AgentFramework { /** * Explicit prose's envelopes, each `>>` target resolved now, once. A live - * round is admitted at its boundary: its turn's destination is decided from - * these resolutions (roundDestination), and its queued delivery posts to - * them, however the registry's names change before it runs. + * round is admitted as it ends, at its tool-calls event, and its queued + * delivery posts to these resolutions, however the registry's names change + * before it runs. */ private admitProse(rawText: string): AdmittedEnvelope[] { return splitExplicitEnvelopes(rawText).map((text) => { @@ -14704,26 +14821,6 @@ export class AgentFramework { return draft ? draftState(draft) : 'let go'; } - /** - * Where an admitted round leaves its turn's sticky destination, set as - * the round is admitted, before its delivery: each `>>` envelope whose - * target resolved becomes the destination. One that didn't - * resolve bounces and leaves it, as do `>>skip_reply` and unprefixed text. - * One refused for speech (its server declared the channel inbound-only) - * bounces and clears it: the text after it was meant for that channel, so - * it bounces too rather than going to the turn's earlier destination. - */ - private roundDestination(current: string | null, envelopes: AdmittedEnvelope[]): string | null { - let destination = current; - for (const envelope of envelopes) { - const resolved = envelope.resolved; - if (!resolved) continue; - if ('channelId' in resolved) destination = resolved.channelId; - else if (resolved.refusedChannelId) destination = null; - } - return destination; - } - /** Locus-preserving explicit publication. Source remains byte-identical in Chronicle. */ private async deliverHybridProse( agent: Agent, @@ -14775,6 +14872,7 @@ export class AgentFramework { this.recordProseDelivery(routing, outcome.status === 'delivered' ? { delivered: true, channelId: outcome.destination!.channelId } : null); if (outcome.status === 'delivered') { this.proseBounceStreaks.delete(agent.name); + round.onPublished?.(outcome.destination!.channelId, body, outcome.messageId); } else { delete routing.pin; routing.hybridHold = 'failed'; @@ -14815,6 +14913,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry!.routeSpeech(agent.name, envelope, target); this.recordProseDelivery(routing, outcome); + if (outcome?.delivered) round.onPublished?.(outcome.channelId, envelope, outcome.messageId); } catch (err) { console.error('hybrid locus delivery failed:', err); } @@ -14905,11 +15004,8 @@ export class AgentFramework { const res = envelope.resolved; if (!res) throw new Error(`[prose] ${name}: a >> envelope reached delivery without the target admitted with it`); if ('error' in res) { - // A target refused for speech ends the turn's destination. This drops - // the pin, which later text follows. roundDestination cleared the - // destination later streams start at when it admitted the round, for - // any round but the turn's final text, after which nothing of the turn - // streams. The text after the target was meant for that channel, not + // A target refused for speech drops the pin, which later text + // follows: the text after the target was meant for that channel, not // the earlier one, so it bounces too. if (res.refusedChannelId) delete routing.pin; this.bounceProse(agent, parsed.body, res.error, res.candidates, '>>', round); @@ -14948,6 +15044,7 @@ export class AgentFramework { delivered = outcome.status === 'delivered'; if (delivered) { this.proseBounceStreaks.delete(name); + round.onPublished?.(outcome.destination!.channelId, body, outcome.messageId); } else if (outcome.status === 'failed') { // Unknown deliveries are marker-only: offering a fresh draft would // invite a duplicate. Keep the authored body for the re-bounce rule. @@ -14960,13 +15057,10 @@ export class AgentFramework { // The channel is inbound-only by now: its server declared it after the // round was admitted, or in its answer to opening it. Whatever made this // delivery fail, a refusal or a publish whose outcome is unknown, later - // text there would be refused, so what still names the channel ends, as - // a refusal at admission ends it: the pin, unless a later delivery has - // moved it, and the destination later streams start at, unless a later - // round has named another. A failed delivery to a channel that isn't - // inbound-only keeps them, as before. + // text there would be refused, so the pin ends, as a refusal at + // admission ends it, unless a later delivery has moved it. A failed + // delivery to a channel that isn't inbound-only keeps it, as before. if (routing.pin === targetChannel) delete routing.pin; - if (routing.destination === targetChannel) routing.destination = null; } } @@ -15618,9 +15712,9 @@ export class AgentFramework { // results the non-streaming fallback left (whichever wake the // scheduler starts it with), the later compile a provider-capacity // hold defers a turn to. It takes the turn's object: the sticky - // destination, the destination and the hold, the ` !` and receipt - // tallies, the trigger channel, the locus with any re-pin, and the - // engaged channels. A delivery still queued from an earlier stream + // destination and the hold, the ` !` and receipt tallies, the + // trigger channel, the locus with any re-pin, and the engaged + // channels. A delivery still queued from an earlier stream // runs ahead of this one's on the agent's chain, so what it sets // applies to this stream's prose, in the order the turn wrote them. // With no turn open, the start opens one. @@ -16252,10 +16346,75 @@ export class AgentFramework { // A stream another has replaced doesn't move the turn's latest link. if (agent.streamId === myStreamId) this.proseSpeechChains.set(agent.name, turnSpeechChain); }; + + // MCPL Spec 14.3 outgoing streaming: the prose this stream's deliveries + // publish, sent to the channel's server once each publish succeeds + // (SpeechPublished; voice synthesis), and to in-process consumers as an + // `inference:published` trace. Delivery feeds it, never the text as the + // model writes it, so it carries only what delivery published, where it + // published it. A round's text reaches it once the round's routing is + // decided and its publish succeeds. Text that delivery withholds never + // does: a round a silencing tool, `skip_reply` or a private `think` keeps + // back, a hold, a bounce, a draft, or a round its stream never finished. + // Nor does a publish the server doesn't confirm (SpeechPublished). + // Emission to a server is gated in the registry (`channels.streaming`), + // and `channels/publish` stays the authoritative send. + const outgoingInferenceId = `inf_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; + let outgoingIndex = 0; + // Per channel, everything this stream has sent there, which its + // `complete` carries. + const outgoingText = new Map(); + // Set once the stream's `complete` has gone: no chunk follows it. + let outgoingCompleted = false; + const streamSpeech: SpeechPublished = (channelId, text, messageId) => { + if (outgoingCompleted) return; + // A later publish to the same channel is another message: a paragraph + // break keeps the two apart for a consumer that joins the chunks, so a + // voice doesn't run "Let me look.Found it." together. + const sent = outgoingText.get(channelId); + const delta = sent === undefined ? text : `\n\n${text}`; + outgoingText.set(channelId, (sent ?? '') + delta); + const index = outgoingIndex++; + this.channelRegistry?.sendOutgoingChunk(channelId, agent.name, outgoingInferenceId, index, delta); + this.emitTrace({ + type: 'inference:published', + agentName: agent.name, + channelId, + inferenceId: outgoingInferenceId, + index, + text: delta, + ...(messageId !== undefined ? { messageId } : {}), + }); + }; + // Spec 14.3: the stream's end for each channel it streamed to, with + // everything it sent there, the consumer's signal to finalize (end the + // TTS utterance, settle the rendered message). Sent once, and no chunk + // follows it. The finally below sends it once the stream's deliveries + // have run. It is registered as the stream starts, so a stop() reaches + // every stream that hasn't sent it, one the framework no longer tracks + // included, such as a disposed ephemeral's still finishing its last + // publish (pendingOutgoingCompletes). + const completeOutgoing = (): void => { + if (outgoingCompleted) return; + outgoingCompleted = true; + this.pendingOutgoingCompletes.delete(completeOutgoing); + for (const [channelId, text] of outgoingText) { + this.channelRegistry?.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); + this.emitTrace({ + type: 'inference:published_complete', + agentName: agent.name, + channelId, + inferenceId: outgoingInferenceId, + text, + }); + } + }; + const enqueueSpeech = (text: string, locus: string | null): void => { chainSpeech(async () => { const outcome = await this.channelRegistry!.routeSpeech(agent.name, text, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, text, outcome.messageId); }, 'mid-turn speech routing failed:'); }; @@ -16287,10 +16446,12 @@ export class AgentFramework { const refusedWakeRounds = new Set(); let toolRounds = 0; // A refused skip_reply silences only its own round: the silence before it, - // saved with its cause as its round is dispatched, comes back at the next - // round or at 'complete'. It's this stream's: a budget restart starts from - // its trigger, and a tool-result-guard retry takes the turn's silence as - // 'complete' restored it, through guardRetryTurnSilenced. + // saved with its cause as its round is routed (once its calls are + // dispatched, or, for a round that waits on scripts, once they return), + // comes back at the next round or at 'complete'. It's this stream's: a + // budget restart starts from its trigger, and a tool-result-guard retry + // takes the turn's silence as 'complete' restored it, through + // guardRetryTurnSilenced. let silenceBeforeRefusal: 'send' | 'private' | null | undefined; // Typing indicator: show " is typing…" for the whole duration of @@ -16311,26 +16472,6 @@ export class AgentFramework { : resolveTurnLocus(); if (typingChannel) this.channelRegistry!.startTyping(typingChannel); - // MCPL Spec 14.3 outgoing streaming: route text deltas to their - // destination server AS THEY GENERATE (voice synthesis, live message - // rendering). Pure observer surface — suppression is fail-closed (nothing - // streams whose destination delivery wouldn't choose, or to a channel - // the router's `speakable` refuses), emission is capability-gated in the - // registry (`channels.streaming`), and delivery via deliverProse remains - // the sole authoritative send. Delivery can still refuse text that has - // streamed, for what it learns later (ProseStreamRouter's header). - const newOutgoingInferenceId = (): string => - `inf_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; - // NOT const: a membrane refusal retry abandons everything streamed so - // far, and the surface must not concatenate two attempts. Re-minting the - // id makes the discarded attempt's chunk buffer orphaned (never - // finalized) instead of being appended to — see the 'retrying' case. - let outgoingInferenceId = newOutgoingInferenceId(); - // RFC-007: every inference id THIS stream mints, so the stream's end - // aborts only its own open tool calls — never a successor's. - const lifecycleInferenceIds = new Set([outgoingInferenceId]); - let outgoingIndex = 0; - // §10.5 inference/lifecycle: `started` now, exactly one terminal in the // finally below (the one block every exit path shares). Which terminal // is decided by the paths themselves: `completed` by default; `aborted` @@ -16350,59 +16491,17 @@ export class AgentFramework { turnIndex: 0, phase: 'started', }); - // The wrapper guard needs the complete response before it can distinguish - // an exact invocation-shaped body from ordinary XML/prose. Buffer guarded - // turns until completion: otherwise speculative outgoing chunks could - // expose the wrapper before the fail-closed classifier runs. - const proseStream = this.channelRegistry && - turnProseRouting !== 'disabled' && - !agent.toolWrapperProseGuard - ? new ProseStreamRouter({ - mode: turnProseRouting === 'explicit' ? 'explicit' : turnProseRouting === 'hybrid' ? 'hybrid' : 'locus', - // Explicit text has no destination until a `>>` prefix names one, - // so a fresh turn's stream starts suppressed. A later stream of the - // turn (a resume, a retry, a re-inference, a capacity-deferred - // compile) starts at the destination the turn's admitted rounds - // have set: a private round's prefixes don't count, and the - // gateway's queued deliveries needn't have run. The trigger channel - // is only where explicit mode's typing indicator shows. - initialTarget: turnProseRouting === 'explicit' - ? streamRouting.destination - : typingChannel, - resolve: (spec) => { - const r = this.channelRegistry!.resolveProseTarget(spec); - return 'channelId' in r ? r.channelId : null; - }, - // Nothing is published into a channel its server declared - // inbound-only, so text routed there while it is one is never - // streamed or kept for the stream's end. - speakable: (channelId) => !this.channelRegistry!.inboundOnlyServer(channelId), - }) - : null; - const emitOutgoing = (deltas: { channelId: string; delta: string }[]): void => { - for (const rd of deltas) { - this.channelRegistry!.sendOutgoingChunk( - rd.channelId, agent.name, outgoingInferenceId, outgoingIndex++, rd.delta, - ); - } - }; - // Tool-result-guard publication boundary. While a SUBMITTED batch is // pending (or a guard recovery is running), a refusal can still abandon - // this physical round, so its streamed text must not reach any surface - // yet — neither the prose router nor the inference:tokens trace (TTS and - // other trace consumers voice it). Hold both; release in order on a clean - // round (tool-calls / non-guard completion), discard on a guard refusal. - let guardHeld: Array<{ trace: Parameters[0]; text?: string }> = []; - // The text block the stream's last text came from, within this round. - let lastTextBlock: number | undefined; + // this physical round, so its `inference:tokens` traces must not reach + // trace consumers yet. Hold them; release in order on a clean round + // (tool-calls / non-guard completion), discard on a guard refusal. The + // outgoing stream needs no hold: it only carries what delivery publishes. + let guardHeld: Array[0]> = []; const releaseGuardHeld = (): void => { const held = guardHeld; guardHeld = []; - for (const item of held) { - this.emitTrace(item.trace); - if (proseStream && item.text !== undefined) emitOutgoing(proseStream.feed(item.text)); - } + for (const trace of held) this.emitTrace(trace); }; const discardGuardHeld = (): void => { guardHeld = []; }; @@ -16490,6 +16589,9 @@ export class AgentFramework { silenceCause = turnSilenced ? 'private' : null; }; + // The stream's `complete` is pending from here: every exit reaches the + // finally, which sends it, and a stop() can send it sooner. + this.pendingOutgoingCompletes.add(completeOutgoing); try { for await (const event of stream) { // Ignore every late event from an Agent generation that no longer owns @@ -16514,22 +16616,10 @@ export class AgentFramework { // turn's prose is bound for, without re-deriving routing. channelId: typingChannel ?? undefined, }; - // Delivery joins a round's text blocks with a newline - // (splitProseRuns), so a later block starts a new line for - // the stream too. - let text = event.meta.type === 'text' ? event.content : undefined; - if (text !== undefined) { - const block = event.meta.blockIndex; - if (block !== undefined) { - if (lastTextBlock !== undefined && block !== lastTextBlock) text = '\n' + text; - lastTextBlock = block; - } - } if (agent.toolResultGuard.hasSubmittedPending || agent.toolResultGuard.recovering) { - guardHeld.push({ trace, text }); + guardHeld.push(trace); } else { this.emitTrace(trace); - if (proseStream && text !== undefined) emitOutgoing(proseStream.feed(text)); } break; } @@ -16537,31 +16627,18 @@ export class AgentFramework { case 'retrying': { // Membrane is re-issuing after a content-policy refusal. Per the // RetryingEvent contract we must DISCARD everything this call has - // emitted: the tokens above were already streamed to the surface - // as outgoing chunks, and appending a second attempt to them - // would show the human two half-answers spliced together. - // - // Re-mint the outgoing id so the partial buffer is orphaned - // rather than continued (delivery is authoritative via - // deliverProse and has not happened yet — nothing has been - // *sent*, only previewed), and reset the prose router so its - // destination bookkeeping starts clean for the new attempt. + // emitted. Nothing of the attempt has been delivered, so the + // outgoing stream holds none of it: the stream only carries what + // delivery publishes, and rounds this stream delivered before the + // refusal stand, under the same inference id. A guarded round's + // held token traces go. console.error( `[refusal-retry] agent=${agent.name} membrane retry ` + `${event.attempt}/${event.maxAttempts}` + (event.category ? ` category=${event.category}` : '') + ' — discarding the refused attempt', ); - outgoingInferenceId = newOutgoingInferenceId(); - lifecycleInferenceIds.add(outgoingInferenceId); - outgoingIndex = 0; discardGuardHeld(); - proseStream?.reset(); - lastTextBlock = undefined; - // Explicit mode's retried attempt starts where the turn's - // admitted rounds have set the destination, not at the stream's - // seed: rounds this stream admitted before the refusal stand. - if (turnProseRouting === 'explicit') proseStream?.retarget(streamRouting.destination); this.emitTrace({ type: 'inference:tokens', agentName: agent.name, @@ -16591,10 +16668,6 @@ export class AgentFramework { // preceding results before storing/dispatching this new round. agent.toolResultGuard.accept(); releaseGuardHeld(); - // The round's text ends here: delivery parses each round's text - // on its own, so the next round's text starts a new line. - if (proseStream) emitOutgoing(proseStream.endSegment()); - lastTextBlock = undefined; adoptInjectedRound(); hadToolCalls = true; const toolRound = toolRounds++; @@ -16674,35 +16747,6 @@ export class AgentFramework { (event.context as { supportsInjectedMessages?: boolean }).supportsInjectedMessages, ); - // A refused skip_reply, whether the agent's allowedTools doesn't - // name it or this host can't set the wake it asks for, keeps its - // own round private and silences nothing after it: the silence - // before it, and its round's other permitted silencing calls, - // come back at the next round, each with why it silences. A - // refusal withholds the call's effect, not the model's choice to - // keep its words private. - const refusedSkipReply = (call: ToolCall): boolean => bareToolName(call.name) === 'skip_reply' - && (!this.canAgentUseTool(agent.name, call.name) || this.refusesSelfWake(call)); - if (event.calls.some(refusedSkipReply)) { - const otherCause = silenceCauseOf(event.calls.filter((call) => - this.canAgentUseTool(agent.name, call.name) && !refusedSkipReply(call))); - silenceBeforeRefusal = silenceCause === 'private' || otherCause === 'private' - ? 'private' - : (silenceCause ?? otherCause); - refusedWakeRounds.add(toolRound); - } - - for (const call of event.calls) { - // RFC-007: register at the one dispatch point for model-issued - // calls, keyed by (agent, call id) with this inference's id; - // `started` goes out once dispatch returns. Refusal sites inside - // dispatch run synchronously and call refuse() first, so a call - // the host refuses produces no events. - this.toolLifecycleEmitter?.register(agent.name, outgoingInferenceId, call); - this.dispatchToolCall(agent.name, call, streamRouting.turnId); - this.toolLifecycleEmitter?.open(agent.name, call.id); - } - // Speak-while-acting: route THIS round's prose to the locus NOW // instead of batching every segment to the end of the turn. Long // tool-using turns (robot control, live sessions) otherwise dump @@ -16714,114 +16758,185 @@ export class AgentFramework { // from the round it occurs (see SILENCING_TOOLS). Only rounds // with verbatim roundContent are live-routed (see liveProseRouting // note above — the fallback preamble is cumulative in XML mode). - if (this.channelRegistry) { - // Every call the model made in the round, refused or not: a - // skip_reply or think refused still asks for the round's words - // to stay private. A refused send posted nothing, so it - // silences nothing; a refused skip_reply silences its own - // round, and the silence before it comes back at the next. - const roundToolNames = event.calls.map((c) => c.name); + // + // The round's tool calls decide whether its prose goes out, and a + // foreground script's own calls count as the round's: a skip_reply + // or a send inside the script silences the round as a direct call + // does. So a round whose script can call such a tool is routed + // once its scripts have returned (ScriptRound, scriptMayRoute), + // and its prose goes out when they finish. Any other round is + // routed now, a round with a script that can't call one included, + // so narration beside a script isn't held for the script's + // runtime. The turn's locus and an explicit round's admitted + // targets are taken now, at the round's tool-calls event, so a + // re-pin or a renamed label while a script runs doesn't move them. + // Hybrid's `>>>` targets, sticky pin and hold apply as its delivery + // runs, as for any queued hybrid delivery. + const roundToolNames = event.calls.map((c) => c.name); + const roundCalls = event.calls.map((c) => this.roundCallOf(agent.name, c)); + const liveRound = !!this.channelRegistry && !!roundContent && roundContent.length > 0; + if (liveRound) liveProseRouting = true; + // Whether this round's tool-result batch (and anything queued + // with it) will be presented to the model before it speaks + // again: the producer says so when it can; otherwise native + // rounds (verbatim roundContent) carry injections. + const presentsInjections = + (event.context as { supportsInjectedMessages?: boolean }).supportsInjectedMessages ?? true; + // Queued under this turn's routing, whenever it posts, and + // streamed by this stream once published. + const holdOpts: ProseRound = { + routing: streamRouting, + notice: presentsInjections ? 'now' : 'later', + onPublished: streamSpeech, + }; + // Routing publishes trimmed segments; held drafts keep the runs + // exactly as written. + const roundRuns = liveRound ? splitProseRuns(assistantBlocks) : []; + const roundSegments = roundRuns.map((run) => run.trim()); + const roundLocus = resolveTurnLocus(); + // An explicit round is admitted as it ends: its targets are + // resolved once, for its queued delivery. + const roundAdmitted = turnProseRouting === 'explicit' + ? roundSegments.map((seg) => this.admitProse(seg)) + : []; + const routeRound = (calls: readonly RoundCall[]): void => { + if (!this.channelRegistry) return; + const toolNames = calls.map((c) => c.name); const hasSameRoundPrivateThink = - roundToolNames.includes('think') && + toolNames.includes('think') && requestSnapshot.sameRoundThinkTextPolicy === 'private'; - const roundCause = silenceCauseOf(event.calls.filter((c) => - this.canAgentUseTool(agent.name, c.name) || refusedSkipReply(c))); + // A refused skip_reply, called directly or from the round's + // script, keeps its own round private and silences nothing + // after it (roundCallOf): the silence before it, and the + // round's other silencing calls, come back at the next round, + // each with why it silences, and the turn-end delivery drops + // the round's prose by its place where nothing routed it live + // (XML mode). A refused send posted nothing, so it counts for + // nothing. + if (calls.some((c) => c.refused)) { + const otherCause = roundSilenceCause(calls.filter((c) => !c.refused)); + silenceBeforeRefusal = silenceCause === 'private' || otherCause === 'private' + ? 'private' + : (silenceCause ?? otherCause); + refusedWakeRounds.add(toolRound); + } + const roundCause = roundSilenceCause(calls); if (roundCause) { turnSilenced = true; silenceCause = roundCause === 'private' || silenceCause === 'private' ? 'private' : 'send'; } - if (roundContent && roundContent.length > 0) { - liveProseRouting = true; - // Whether this round's tool-result batch (and anything queued - // with it) will be presented to the model before it speaks - // again: the producer says so when it can; otherwise native - // rounds (verbatim roundContent) carry injections. - const presentsInjections = - (event.context as { supportsInjectedMessages?: boolean }).supportsInjectedMessages ?? true; - // Queued under this turn's routing, whenever it posts. - const holdOpts: ProseRound = { routing: streamRouting, notice: presentsInjections ? 'now' : 'later' }; - // Routing publishes trimmed segments; held drafts keep the - // runs exactly as written. - const roundRuns = splitProseRuns(assistantBlocks); - const roundSegments = roundRuns.map((run) => run.trim()); - if (roundSegments.length > 0) { - if (turnProseRouting === 'disabled') { - console.error( - `[routing] ${agent.name}: mid-turn prose NOT routed (proseRouting=disabled)`, - ); - this.recordProseSuppression(streamRouting, roundSegments.length); - } else if (turnProseRouting === 'hybrid') { - if (turnSilenced) { - // In order behind earlier rounds' envelopes: the router - // state they leave is where this round's starts. - const cause = silenceCause; - chainSpeech( - async () => this.holdSilencedProse(agent, roundRuns, cause, hasSameRoundPrivateThink, holdOpts), - 'mid-turn hybrid prose hold failed:', - ); - } else if (!hasSameRoundPrivateThink) { - const locus = resolveTurnLocus(); - for (const seg of roundSegments) { - chainSpeech(() => this.deliverHybridProse(agent, seg, locus, true, holdOpts), 'mid-turn hybrid prose delivery failed:'); - } - } - } else if (turnProseRouting === 'explicit') { - // Explicit mode: the segments go through the prose - // gateway, unless a private think in the round holds them - // back. Send silencing does not apply: unprefixed prose - // bounces, and prefixed prose is deliberate. skip_reply - // does: its round's envelopes all stay in the context - // (deliverProseEnvelope's keepPrivate), as the tool - // promises, and skip_reply then ends the turn at this - // round's tool results, unless it was refused, when the - // turn goes on to its next round. - if (!hasSameRoundPrivateThink) { - const keepPrivate = roundToolNames.some((n) => bareToolName(n) === 'skip_reply'); - // The round is admitted now: its targets are resolved - // once, for the turn's destination and for its queued - // delivery alike. - const admitted = roundSegments.map((seg) => this.admitProse(seg)); - const routing = holdOpts.routing; - if (!keepPrivate) { - routing.destination = this.roundDestination(routing.destination, admitted.flat()); - } - console.error( - `[prose] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> ${roundSegments.length} segment(s) via prose gateway${keepPrivate ? ', kept private (skip_reply)' : ''}`, - ); - for (const envelopes of admitted) { - chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { ...holdOpts, keepPrivate }), 'mid-turn prose delivery failed:'); - } - } - // The stream's next round starts where delivery sends - // unprefixed text: the turn's destination. The stream - // followed this round's prefixes as it was written, but a - // private round's, a >>skip_reply envelope's or a target - // that didn't resolve leave the destination as it was, - // and a target refused for speech clears it. - proseStream?.retarget(streamRouting.destination); - } else if (turnSilenced) { - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> prose NOT routed (turn silenced: ${silenceCause})`, - ); - // Never a silent black hole (n=8: the flying-scene reply): - // held as drafts the resident can resend, and named. - this.holdSilencedProse(agent, roundRuns, silenceCause, hasSameRoundPrivateThink, holdOpts); - } else if (hasSameRoundPrivateThink) { - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> prose NOT routed (same_round_think_text_policy=private)`, - ); - } else { - const locus = resolveTurnLocus(); - console.error( - `[routing] ${agent.name}: mid-turn round [${roundToolNames.join(', ')}] -> routing ${roundSegments.length} prose segment(s) live -> ${locus ?? '(default)'}`, - ); - for (const seg of roundSegments) { - enqueueSpeech(seg, locus); - } + if (roundSegments.length === 0) return; + if (turnProseRouting === 'disabled') { + console.error( + `[routing] ${agent.name}: mid-turn prose NOT routed (proseRouting=disabled)`, + ); + this.recordProseSuppression(streamRouting, roundSegments.length); + } else if (turnProseRouting === 'hybrid') { + if (turnSilenced) { + // In order behind earlier rounds' envelopes: the router + // state they leave is where this round's starts. + const cause = silenceCause; + chainSpeech( + async () => this.holdSilencedProse(agent, roundRuns, cause, hasSameRoundPrivateThink, holdOpts), + 'mid-turn hybrid prose hold failed:', + ); + } else if (!hasSameRoundPrivateThink) { + for (const seg of roundSegments) { + chainSpeech(() => this.deliverHybridProse(agent, seg, roundLocus, true, holdOpts), 'mid-turn hybrid prose delivery failed:'); } } + } else if (turnProseRouting === 'explicit') { + // Explicit mode: the segments go through the prose gateway, + // unless a private think in the round holds them back. Send + // silencing does not apply: unprefixed prose bounces, and + // prefixed prose is deliberate. skip_reply does: its round's + // envelopes all stay in the context (deliverProseEnvelope's + // keepPrivate), as the tool promises, and skip_reply then + // ends the turn at this round's tool results, unless it was + // refused, when the turn goes on to its next round. + if (!hasSameRoundPrivateThink) { + const keepPrivate = toolNames.some((n) => bareToolName(n) === 'skip_reply'); + console.error( + `[prose] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> ${roundSegments.length} segment(s) via prose gateway${keepPrivate ? ', kept private (skip_reply)' : ''}`, + ); + for (const envelopes of roundAdmitted) { + chainSpeech(() => this.deliverAdmittedProse(agent, envelopes, { ...holdOpts, keepPrivate }), 'mid-turn prose delivery failed:'); + } + } + } else if (turnSilenced) { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> prose NOT routed (turn silenced: ${silenceCause})`, + ); + // Never a silent black hole (n=8: the flying-scene reply): + // held as drafts the resident can resend, and named. + this.holdSilencedProse(agent, roundRuns, silenceCause, hasSameRoundPrivateThink, holdOpts); + } else if (hasSameRoundPrivateThink) { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> prose NOT routed (same_round_think_text_policy=private)`, + ); + } else { + console.error( + `[routing] ${agent.name}: mid-turn round [${toolNames.join(', ')}] -> routing ${roundSegments.length} prose segment(s) live -> ${roundLocus ?? '(default)'}`, + ); + for (const seg of roundSegments) { + enqueueSpeech(seg, roundLocus); + } } + }; + // A round still waiting under this name can't be decided now: a + // newer round displaces it, whether or not this one waits, so no + // call of this round can settle it. + const displaced = this.scriptRounds.get(agent.name); + if (displaced) { + this.scriptRounds.delete(agent.name); + displaced.withhold('a newer round replaced it before its scripts returned'); } + const scriptCallIds = event.calls + .filter((c) => c.name === CODE_EXECUTION_TOOL_NAME + && this.scriptMayRoute(agent.name, c, turnProseRouting, requestSnapshot.sameRoundThinkTextPolicy === 'private')) + .map((c) => c.id); + if (scriptCallIds.length > 0) { + // In place before dispatch, so a foreground script records + // its calls here from its first one. + const round: ScriptRound = { + agent, + streamId: myStreamId, + scripts: new Map(scriptCallIds.map((id) => [id, []])), + pending: new Set(scriptCallIds), + route: () => { + // Only while this stream still owns the agent and waits on + // the round: a turn replaced, reset or cancelled meanwhile + // can't finish deciding it. + if (this.agents.get(agent.name) !== agent || agent.streamId !== myStreamId + || agent.state.status !== 'waiting_for_tools') { + round.withhold('its turn no longer waits on it (replaced, reset or stopped) while its scripts ran'); + return; + } + routeRound([...roundCalls, ...[...round.scripts.values()].flat()]); + }, + withhold: (why) => { + if (roundSegments.length === 0) return; + console.error( + `[routing] ${agent.name}: round [${roundToolNames.join(', ')}] -> ${roundSegments.length} prose segment(s) withheld: ${why}`, + ); + this.recordProseSuppression(streamRouting, roundSegments.length); + }, + }; + this.scriptRounds.set(agent.name, round); + } + + for (const call of event.calls) { + // RFC-007: register at the one dispatch point for model-issued + // calls, keyed by (agent, call id) with this inference's id; + // `started` goes out once dispatch returns. Refusal sites inside + // dispatch run synchronously and call refuse() first, so a call + // the host refuses produces no events. + this.toolLifecycleEmitter?.register(agent.name, outgoingInferenceId, call); + this.dispatchToolCall(agent.name, call, streamRouting.turnId); + this.toolLifecycleEmitter?.open(agent.name, call.id); + } + + if (scriptCallIds.length === 0) routeRound(roundCalls); // Stream's async iterator blocks on next() until provideToolResults() is called break; } @@ -16846,9 +16961,8 @@ export class AgentFramework { const withheld = agent.toolResultGuard.withhold(category); // Nothing from the abandoned physical attempt is published or // persisted as assistant speech, including a terminal refusal - // after the single recovery retry. - proseStream?.reset(); - if (turnProseRouting === 'explicit') proseStream?.retarget(streamRouting.destination); + // after the single recovery retry, so the outgoing stream + // carries none of it either. if (withheld) { // The abandoned stream was billed (membrane usage here is // cumulative across this physical tool loop): count it before @@ -17262,7 +17376,7 @@ export class AgentFramework { if (speechContent.length > 0 && this.channelRegistry) { const speechRuns = finalSpeechRuns; const speechSegments = finalSpeechSegments; - const textOnlyHold: ProseRound = { routing: finalRouting, notice: 'later' }; + const textOnlyHold: ProseRound = { routing: finalRouting, notice: 'later', onPublished: streamSpeech }; if (speechSegments.length > 0) { if (turnProseRouting === 'disabled') { console.error(`[routing] ${agent.name}: text-only prose NOT routed (proseRouting=disabled)`); @@ -17308,6 +17422,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, segment, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, segment, outcome.messageId); } catch (err) { console.error('speech routing failed:', err); } @@ -17344,7 +17459,7 @@ export class AgentFramework { : turnSilenced || fallbackCause !== null; const trailingCause: 'send' | 'private' | null = silenceCause === 'private' || fallbackCause === 'private' ? 'private' : (silenceCause ?? fallbackCause); - const trailingHold: ProseRound = { routing: finalRouting, notice: 'later' }; + const trailingHold: ProseRound = { routing: finalRouting, notice: 'later', onPublished: streamSpeech }; const runs = finalRuns; const segments = finalSegments; @@ -17405,6 +17520,7 @@ export class AgentFramework { try { const outcome = await this.channelRegistry.routeSpeech(agent.name, seg, locus); this.recordProseDelivery(streamRouting, outcome); + if (outcome?.delivered) streamSpeech(outcome.channelId, seg, outcome.messageId); } catch (err) { console.error('speech routing failed:', err); } @@ -18011,9 +18127,9 @@ export class AgentFramework { // RFC-007 §7.3: a tool call still open when its turn's stream ends was // cancelled before a result — its terminal is `aborted`. (A normal turn // has none: the stream waits for every result before continuing.) - // Scoped to this stream's own inference ids: a budget-restart successor + // Scoped to this stream's own inference id: a budget-restart successor // may already be running for the same agent. - this.toolLifecycleEmitter?.abortOpen(agent.name, lifecycleInferenceIds); + this.toolLifecycleEmitter?.abortOpen(agent.name, [outgoingInferenceId]); this.hookOrchestrator?.emitLifecycle({ inferenceId: outgoingInferenceId, conversationId: agent.name, @@ -18055,15 +18171,25 @@ export class AgentFramework { // exhausted, abort) so it never sticks after the turn ends. if (frameReachedTerminal) this.channelRegistry?.stopTyping(); - // Spec 14.3: flush any held line-start text, then close each streamed - // channel with its final moderated content — the consumer's signal to - // finalize (end the TTS utterance, settle the rendered message). - if (frameReachedTerminal && proseStream) { - emitOutgoing(proseStream.finish()); - for (const [channelId, text] of proseStream.byChannel()) { - this.channelRegistry!.sendOutgoingComplete(channelId, agent.name, outgoingInferenceId, text); - } - } + // A round of this stream still waiting for its code_execution calls + // can't be routed now: whether its scripts would have silenced it is + // unknown, so its prose is withheld, as text whose routing is + // undecided is. + const unsettled = this.scriptRounds.get(agent.name); + if (unsettled?.agent === agent && unsettled.streamId === myStreamId) { + this.scriptRounds.delete(agent.name); + unsettled.withhold('its stream ended before its scripts returned'); + } + + // Spec 14.3: once the deliveries this stream queued have run, close + // each channel it streamed to (completeOutgoing). Every exit path does, + // a stream that hands its turn to a successor (a budget restart, a + // guard retry) included: the successor streams under its own inference + // id. Its deliveries can still be queued here, as when a tool ends an + // explicit-mode turn, so `complete` waits for them and no chunk follows + // it. A stop() sends it at once instead, with what the stream has sent, + // since stopping closes the connections a queued delivery waits on. + void turnSpeechChain.then(completeOutgoing); this.frameworkCancelledStreams.delete(`${agent.name}:${myStreamId}`); if (ownsPhysicalStream) { this.activeStreams.delete(agent.name); @@ -18444,6 +18570,9 @@ export class AgentFramework { input: call.input, }); const startTime = Date.now(); + // The round that waits on this call, registered just before dispatch. + const waitingRound = this.scriptRounds.get(agentName); + const round = waitingRound?.pending.has(call.id) ? waitingRound : undefined; this.runCodeExecution(agentName, call) .catch((error): ToolResult => { this.toolLifecycleEmitter?.markDispatchFailure(agentName, call.id); @@ -18470,6 +18599,7 @@ export class AgentFramework { durationMs: Date.now() - startTime, }); } + if (round) this.scriptRoundOfResult.set(result, round); this.pushEvent({ type: 'tool-result', callId: call.id, @@ -18481,6 +18611,47 @@ export class AgentFramework { }); } + /** + * A model-issued tool's result has arrived. When it answers the last of + * the code_execution calls a round waits on (ScriptRound), the round's + * prose is routed now, its scripts' calls counted. + */ + private settleScriptRound(agentName: string, callId: string, result: ToolResult): void { + const round = this.scriptRounds.get(agentName); + if (!round || this.scriptRoundOfResult.get(result) !== round) return; + if (!round.pending.delete(callId) || round.pending.size > 0) return; + this.scriptRounds.delete(agentName); + round.route(); + } + + /** + * Whether a code_execution call's script can change its round's routing, + * so the round waits for it (ScriptRound). Its code must name a tool whose + * call can change the round's routing in the turn's mode, under its + * same-round policy (scriptToolNames, changesRouting), or the interpreter + * it will run in must have run a script that left such a tool reachable + * (scriptRoutingInterpreters), until that interpreter is reclaimed or + * restarted. What neither sees is routed at once, as a round was before + * scripts counted (scriptToolNames says what goes unseen). A background + * script, or a list or cancel action, returns at once, and its calls + * aren't the round's. Without code execution no script runs: the call is + * answered as an unknown tool, a result that settles no round. + */ + private scriptMayRoute( + agentName: string, + call: ToolCall, + mode: NonNullable, + privateThink: boolean, + ): boolean { + if (!this.codeExecutionConfig) return false; + const input = (call.input ?? {}) as { code?: unknown; background?: unknown; action?: unknown }; + if (input.background === true || (input.action !== undefined && input.action !== 'run')) return false; + if (typeof input.code === 'string' && changesRouting(scriptToolNames(input.code, ROUTING_TOOLS).names, mode, privateThink)) return true; + const interpreter = this.codeExecutionRunners.get(agentName)?.interpreter ?? null; + const mark = this.scriptRoutingInterpreters.get(agentName); + return interpreter !== null && mark?.interpreter === interpreter && changesRouting(mark.names, mode, privateThink); + } + /** * Run one code_execution script for an agent. Never rejects. * @@ -18581,9 +18752,14 @@ export class AgentFramework { const runner = this.getOrCreateScriptRunner(agentName); // A busy runner refuses this exec; the running script keeps its mark and - // its origin. + // its origin. The script records the tools it calls where its round + // reads them (ScriptRound), when a round dispatched it. const ownsRunner = !runner.busy; - const script = { endTurn: false, turnId: call.callerTurnId }; + const script = { + endTurn: false, + turnId: call.callerTurnId, + calls: this.scriptRounds.get(agentName)?.scripts.get(call.id) ?? [], + }; if (ownsRunner) this.scriptDeferredEndTurn.set(agentName, script); const ownsOrigin = origin !== undefined && ownsRunner; if (ownsOrigin) this.foregroundScriptOrigins.set(agentName, origin); @@ -18602,6 +18778,17 @@ export class AgentFramework { } } const endTurn = script.endTurn; + // A script that left a tool reachable, such as a helper it defined, may + // call it from a later script in the same interpreter (scriptMayRoute). + // Whether that changes the later round's routing is its mode's and + // policy's to say, so the mark keeps the names. + const interpreter = runner.interpreter; + const { reachable } = scriptToolNames(input.code, ROUTING_TOOLS); + if (ownsRunner && interpreter !== null && reachable.size > 0) { + const mark = this.scriptRoutingInterpreters.get(agentName); + const earlier = mark?.interpreter === interpreter ? mark.names : []; + this.scriptRoutingInterpreters.set(agentName, { interpreter, names: new Set([...earlier, ...reachable]) }); + } return { success: true, @@ -19203,11 +19390,15 @@ export class AgentFramework { args: Record, origin?: ScriptCallOrigin, /** The foreground script that made the call; none for a background script. */ - script?: { endTurn: boolean; turnId?: number }, + script?: { endTurn: boolean; turnId?: number; calls: RoundCall[] }, ): Promise { if (toolName === CODE_EXECUTION_TOOL_NAME) { return { text: 'Error: code_execution cannot be called from within a script', isError: true }; } + // The call counts toward its round's routing however it turns out, as a + // direct call does: a refused skip_reply as refused, and another call the + // agent's allowedTools refuses for nothing. + script?.calls.push(this.roundCallOf(agentName, { name: toolName, input: args })); // A foreground script's call was made in the turn of the code_execution // call whose script is running when it arrives. A background script runs // apart from any turn, so its calls carry none. @@ -19776,10 +19967,10 @@ export class AgentFramework { * any wake_in_seconds; with one, a wake_in_seconds that isn't a number of * seconds. Only when the channel registry handles skip_reply. The dispatch * that refuses the call reads it here, and so does the turn's bookkeeping of - * a refused round, for the model's own calls: the silence before it, which - * comes back after its round, and in XML mode which round's prose stays - * private. A skip_reply round's own prose stays private by the tool's name, - * refused or not. + * a refused round (roundCallOf), for the model's calls and its foreground + * script's: the silence before it, which comes back after its round, and in + * XML mode which round's prose stays private. A skip_reply round's own + * prose stays private by the tool's name, refused or not. */ private skipReplyRefusal(call: { name?: string; input?: unknown }): string | undefined { if (call.name !== 'skip_reply' || !this.channelRegistry) return undefined; @@ -19790,6 +19981,21 @@ export class AgentFramework { return this.skipReplyRefusal(call) !== undefined; } + /** + * A call as its round's routing counts it, made directly or by the round's + * foreground script. A skip_reply refused, because the agent's allowedTools + * doesn't name it or because this host can't set the wake it asks for, is + * marked refused: it keeps its own round private and silences nothing + * after it. A refusal withholds the call's effect, not the model's choice + * to keep its words private. Any other call the list refuses posted + * nothing, so it gives no cause. + */ + private roundCallOf(agentName: string, call: { name: string; input?: unknown }): RoundCall { + const permitted = this.canAgentUseTool(agentName, call.name); + const refused = bareToolName(call.name) === 'skip_reply' && (!permitted || this.refusesSelfWake(call)); + return { name: call.name, cause: permitted || refused ? silenceCauseOf([call]) : null, refused }; + } + /** * Dispatch a call the agent made. `turnId` is the turn it was made in * (TurnRouting.turnId), as its caller knows it: a model's call, the turn diff --git a/src/mcpl/channel-registry.ts b/src/mcpl/channel-registry.ts index 4e558591f..0d48590f3 100644 --- a/src/mcpl/channel-registry.ts +++ b/src/mcpl/channel-registry.ts @@ -3583,12 +3583,13 @@ export class ChannelRegistry { } /** - * MCPL Spec 14.3 outgoing streaming: forward a moderated text delta to the - * server owning the channel, AS THE MODEL GENERATES. Emitted only when that - * server declared `channels.streaming` in its initialize capabilities — - * servers that never opted in (the whole existing fleet) receive nothing. - * Fire-and-forget and never throws: streaming is an observer surface; the - * authoritative delivery is the eventual channels/publish. + * MCPL Spec 14.3 outgoing streaming: forward a text delta to the server + * owning the channel. The framework sends each piece of prose once its + * publish succeeds, so a stream carries only what was published. + * Emitted only when that server's grant has `channels.streaming` — + * servers that never opted in receive nothing. Fire-and-forget and never + * throws: streaming is an observer surface; the authoritative delivery is + * channels/publish. */ sendOutgoingChunk( channelId: string, diff --git a/src/mcpl/prose-grammar.ts b/src/mcpl/prose-grammar.ts index 557a5b2ce..9bac23ef3 100644 --- a/src/mcpl/prose-grammar.ts +++ b/src/mcpl/prose-grammar.ts @@ -1,9 +1,8 @@ /** - * Explicit-prose routing grammar — the SINGLE definition shared by the - * delivery path (framework.deliverProse*) and the outgoing-stream router - * (prose-stream-router.ts). Extracted from framework.ts verbatim so the two - * parsers cannot drift: a streamed chunk and its eventual delivery must agree - * on what is a prefix, what is a modifier, and what is body. + * Explicit-prose routing grammar: the prefix parsers the delivery path uses + * (parseProsePrefix, parseHybridProsePrefix). The envelope splitters in + * framework.ts test a line's `>>` or `>>>` start on their own. The outgoing + * stream needs no parser of its own: it carries what delivery publishes. * * Grammar (docs/explicit-prose-routing.md): * `>>target [!] [body…]` — target = channel spec or `skip_reply` diff --git a/src/mcpl/prose-stream-router.ts b/src/mcpl/prose-stream-router.ts deleted file mode 100644 index e8337e7d7..000000000 --- a/src/mcpl/prose-stream-router.ts +++ /dev/null @@ -1,292 +0,0 @@ -/** - * Incremental prose router: turns a stream of text deltas into a stream of - * (channelId, delta) pairs AS THE MODEL GENERATES, so downstream surfaces - * (voice synthesis, streamed message rendering, future platforms) can act on - * speech before the turn completes. This is the emission half of MCPL Spec - * 14.3 (`channels/outgoing/chunk`) — the notification has existed in the - * wire types since the spec landed; this class is what finally produces it. - * - * Correctness contract: routing decisions here MUST agree with the delivery - * path (deliverProse/parseProsePrefix). Both use prose-grammar.ts; this - * router additionally holds back exactly as much text as needed to make each - * line's decision — the start of a line is buffered only until it either - * cannot be a `>>` prefix (flush as body) or its prefix token (+ optional - * ` !` modifier) completes (switch target, swallow the prefix). Body text - * after the decision point streams through with zero added latency. - * - * Suppression is fail-closed: text whose destination is unknown (explicit - * mode before the first `>>`, unresolvable/ambiguous targets, `>>skip_reply`) - * emits NOTHING — the delivery path will bounce/retain it. Text for a channel - * the `speakable` option refuses emits nothing either, and which servers see - * what is emitted is the registry's (sendOutgoingChunk). The stream can't - * know what delivery decides later: `speakable` is asked as the text is - * routed, so a channel declared inbound-only after its text was emitted, or - * one whose open then fails, is refused at delivery, though a consumer has - * seen the text. - * - * Three modes, mirroring proseRouting: - * - explicit: parse `>>` prefixes. A fresh turn's stream starts suppressed; - * a later stream of the turn starts at the turn's destination, - * and each round's end returns the stream to it (retarget). - * - hybrid: parse `>>>` envelopes, with the turn's locus as the - * fallback. An envelope with no target, or `>>>skip_reply`, - * holds back what follows until a later `>>>` names a target. - * - locus: NO prefix parsing (locus mode's `>>` lines are plain text); - * everything streams to the turn's locus. - */ - -export interface RoutedDelta { - channelId: string; - delta: string; -} - -type State = 'line-start' | 'indent' | 'maybe-prefix' | 'prefix-gap' | 'prefix-token' | 'after-token' | 'body'; - -/** Whitespace as the delivery path reads it (`trimStart()`, `\s`, `\S`). */ -const isSpace = (ch: string): boolean => /\s/.test(ch); - -export class ProseStreamRouter { - private target: string | null; - private state: State = 'line-start'; - private held = ''; // text held while a line's decision is pending - private token = ''; // the >>spec token being accumulated - private sawBang = false; - private accumulated = new Map(); - - constructor( - private opts: { - /** explicit parses `>>`; hybrid parses `>>>` while retaining locus fallback. */ - mode: 'explicit' | 'hybrid' | 'locus'; - /** Initial destination: the turn's locus (locus and hybrid mode); in - * explicit mode, null on a fresh turn, or the turn's destination on a - * later stream of the turn. */ - initialTarget: string | null; - /** Resolve a `>>spec` to a channelId, or null (unresolved/ambiguous). */ - resolve: (spec: string) => string | null; - /** Whether speech may be published to this channel now. Text for a - * channel it refuses is neither emitted nor kept for outgoing/complete, - * so the stream's end can't carry it there should that change before - * the stream ends. Asked as the text is routed, while delivery decides - * when its round is admitted: text refused here stays out of the - * stream even if the channel accepts speech again by then and the - * round posts it. Every channel, when left out. */ - speakable?: (channelId: string) => boolean; - }, - ) { - this.target = opts.initialTarget; - if (opts.mode === 'locus') this.state = 'body'; - } - - /** Per-channel full text emitted so far (for outgoing/complete). */ - byChannel(): ReadonlyMap { - return this.accumulated; - } - - /** - * Point the stream at a destination decided outside it. Explicit mode does - * this at each round's end, with the turn's destination: a private round's - * prefixes, a `>>skip_reply` envelope and a target that didn't resolve all - * leave that destination as it was, though the stream followed them while - * the round was written. A target refused for its server's inbound-only - * declaration clears it. - */ - retarget(target: string | null): void { - this.target = target; - } - - /** - * Drop everything routed so far and return to the constructed state. - * - * For a membrane refusal retry (RetryingEvent): the abandoned attempt's - * prose must not survive into the new one, and `accumulated` feeds - * outgoing/complete — leaving it would finalize text the model never - * actually said in the attempt that stands. - */ - reset(): void { - this.target = this.opts.initialTarget; - this.state = this.opts.mode === 'locus' ? 'body' : 'line-start'; - this.held = ''; - this.token = ''; - this.sawBang = false; - this.accumulated = new Map(); - } - - feed(delta: string): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (const ch of delta) this.step(ch, out); - return this.keep(out); - } - - /** - * The end of a segment: one round's text, which delivery parses on its - * own. The pending line's decision is made now. Held line-start text can - * no longer become a prefix, so it is body. A prefix whose target is - * written but has no body takes effect, as delivery reads `>>#chan` with - * nothing after it. The next segment starts a new line. - */ - endSegment(): RoutedDelta[] { - const out: RoutedDelta[] = []; - if (this.state === 'indent') this.emit(this.held, out); - if (this.state === 'maybe-prefix') { - // Hybrid's `>>>` with nothing after it is an envelope with no target: - // delivery bounces it and holds back what follows. Anything else held - // is text to delivery, explicit's bare `>>` included. - const arrows = this.held.trimStart(); - if (this.opts.mode === 'hybrid' && arrows === '>>>') this.target = null; - else this.emit(this.held, out); - } - // Hybrid's `>>>` and a gap with nothing after it: no target either. - if (this.state === 'prefix-gap') this.target = null; - if (this.state === 'prefix-token') this.applyTarget(); - this.held = ''; - this.token = ''; - if (this.opts.mode !== 'locus') this.state = 'line-start'; - return this.keep(out); - } - - /** End of stream: the last segment ends. */ - finish(): RoutedDelta[] { - return this.endSegment(); - } - - private step(ch: string, out: RoutedDelta[]): void { - switch (this.state) { - case 'body': - if (ch === '\n' && this.opts.mode !== 'locus') { - this.emit('\n', out); - this.state = 'line-start'; - } else { - this.emit(ch, out); - } - return; - - case 'line-start': - // Indentation before a prefix is held until the line's decision: the - // delivery path reads an indented `>>` (explicit) or `>>>` (hybrid) - // line as a prefix too, after any leading whitespace. - if (ch !== '\n' && isSpace(ch)) { - this.state = 'indent'; - this.held = ch; - return; - } - if (ch === '>') { this.state = 'maybe-prefix'; this.held = '>'; return; } - this.state = 'body'; - this.step(ch, out); - return; - - case 'indent': - if (ch !== '\n' && isSpace(ch)) { this.held += ch; return; } - if (ch === '>') { this.held += ch; this.state = 'maybe-prefix'; return; } - this.emit(this.held, out); - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - - case 'maybe-prefix': { - const prefix = this.opts.mode === 'hybrid' ? '>>>' : '>>'; - const arrows = this.held.trimStart(); - if (ch === '>' && arrows.length < prefix.length) { - this.held += '>'; - return; - } - if (arrows === prefix) { - if (this.opts.mode === 'hybrid' && (ch === ' ' || ch === '\t')) { - this.held = ''; - this.state = 'prefix-gap'; - return; - } - if (isSpace(ch)) { - if (this.opts.mode === 'explicit') { - // `>>` and then whitespace is a quoted arrow: body text, since - // delivery's prefix needs a target right after the arrows. - this.emit(this.held, out); - } else { - // `>>>` and then whitespace other than a space or tab (a - // newline included) is an envelope with no target. Delivery - // bounces it and holds back what follows until a later `>>>` - // names a target. - this.target = null; - } - // The whitespace itself is body, a newline included. - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - } - this.state = 'prefix-token'; - this.token = ch; - this.held = ''; - return; - } - this.emit(this.held, out); - this.held = ''; - this.state = 'body'; - this.step(ch, out); - return; - } - - case 'prefix-gap': - if (ch === ' ' || ch === '\t') return; - if (ch === '\n') { this.target = null; this.state = 'line-start'; return; } - // Other whitespace can't start hybrid's target (`[ \t]*\S+`): an - // envelope with no target, as above. - if (isSpace(ch)) { this.target = null; this.state = 'body'; return; } - this.state = 'prefix-token'; - this.token = ch; - return; - - case 'prefix-token': - if (ch === ' ' || ch === '\t') { this.applyTarget(); this.state = 'after-token'; this.sawBang = false; return; } - if (ch === '\n') { this.applyTarget(); this.state = 'line-start'; return; } - // Any other whitespace ends the token too (`\S+`), and starts the - // body: the grammar's gap after a token is spaces and tabs only. - if (isSpace(ch)) { this.applyTarget(); this.state = 'body'; this.step(ch, out); return; } - this.token += ch; - return; - - case 'after-token': - // Grammar: `([ \t]+!)?[ \t]*` — one optional `!` immediately after the - // first whitespace run is the continue-turn modifier, not body. - if (ch === ' ' || ch === '\t') return; - if (ch === '\n') { this.state = 'line-start'; return; } - if (ch === '!' && !this.sawBang) { this.sawBang = true; return; } - this.state = 'body'; - this.step(ch, out); - return; - } - } - - private applyTarget(): void { - const spec = this.token; - this.token = ''; - if (spec === 'skip_reply') { this.target = null; return; } - this.target = this.opts.resolve(spec); - } - - private emit(text: string, out: RoutedDelta[]): void { - if (this.target === null || !text) return; - out.push({ channelId: this.target, delta: text }); - } - - /** - * What one feed or segment end routed, less any channel `speakable` - * refuses, kept for outgoing/complete. A feed runs without yielding, so - * one answer per channel holds for all of it. - */ - private keep(out: RoutedDelta[]): RoutedDelta[] { - const kept = coalesce(out).filter((d) => this.opts.speakable?.(d.channelId) ?? true); - for (const d of kept) this.accumulated.set(d.channelId, (this.accumulated.get(d.channelId) ?? '') + d.delta); - return kept; - } -} - -function coalesce(deltas: RoutedDelta[]): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (const d of deltas) { - const last = out[out.length - 1]; - if (last && last.channelId === d.channelId) last.delta += d.delta; - else out.push({ ...d }); - } - return out; -} diff --git a/src/types/trace.ts b/src/types/trace.ts index a91f2322b..776f15338 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -173,6 +173,59 @@ export type TraceEvent = type: 'inference:turn_ended'; agentName: string; }) + | (TraceEventBase & { + /** + * Prose that delivery published, sent once the publish succeeds: the + * outgoing stream's chunk (MCPL Spec 14.3 `channels/outgoing/chunk`), + * emitted whether or not the channel's server takes the stream. It + * carries only what was published, so a consumer that speaks or shows + * it, such as a voice relay, never gets text that delivery withholds (a + * `skip_reply`, a silencing send, a private `think`, a hold, a bounce, + * a draft) or a publish the server didn't confirm (`delivered: true`): + * one refused before sending, one that threw, timed out or got an + * error answer, one the server reported undelivered + * (`delivered: false`), and one whose reply leaves `delivered` out. + * `inference:tokens` stays the model's text as it is written (held, as + * before, while a tool-result guard's batch is pending). A later + * publish to the same channel by the same stream opens with a paragraph + * break (`\n\n`), so a channel's chunks joined in `index` order are its + * text, its messages kept apart. + * `mcpl:speech-routed` also reports speech publishes, with no stream + * id, index or end, each once delivered: deliverSpeech's, the `sleep` + * tool's announcement among them, and a subconscious's + * `speak_in_channel`'s. A relay that voices the agent's prose takes + * this trace, with `inference:published_complete`, and not + * `mcpl:speech-routed` as well, and takes each message's id from its + * `messageId` to edit what it voiced. + */ + type: 'inference:published'; + agentName: string; + /** Where the prose is published. */ + channelId: string; + /** The stream's id, shared with its `channels/outgoing/*` notifications. */ + inferenceId: string; + /** Increases by one per chunk of the stream, across its channels. */ + index: number; + text: string; + /** The id the channel's server returned for this chunk's publish + * (`ChannelsPublishResult.messageId`); absent when it returned none. + * Each chunk is one publish, so each carries its own. */ + messageId?: string; + }) + | (TraceEventBase & { + /** + * A stream's end for one channel it published to (MCPL Spec 14.3 + * `channels/outgoing/complete`), sent on every exit path once the + * stream has ended and its deliveries have run, or, when the host + * stops, at once: `text` is everything its `inference:published` chunks + * carried there, and none follows it. + */ + type: 'inference:published_complete'; + agentName: string; + channelId: string; + inferenceId: string; + text: string; + }) // Tool lifecycle | (TraceEventBase & { diff --git a/test/channel-registry-routing.test.ts b/test/channel-registry-routing.test.ts index abbcf4fe8..874046a5a 100644 --- a/test/channel-registry-routing.test.ts +++ b/test/channel-registry-routing.test.ts @@ -166,6 +166,40 @@ test('routeSpeech surfaces a failure when the server reports delivered:false', a assert.ok(traces.some(t => t.type === 'mcpl:speech-route-failed' && t.outcome === 'failed')); }); +test('routeSpeech counts a publish reply that leaves delivered out as unconfirmed, not delivered', async () => { + const { registry, failures } = makeRegistry({}); + seedRegistered(registry, 'discord', 'ch-x'); + registry.handleIncoming('discord', incoming('ch-x', 'hi')); + + const res = await registry.routeSpeech('cairn', 'a reply', registry.resolveLocus('cairn')); + + assert.equal(res, null, 'no delivery is confirmed, so the stream gets none of it'); + assert.equal(failures.length, 1, 'and the route failure is reported'); + assert.equal(failures[0].outcome, 'unknown', 'as a post that may or may not exist'); +}); + +test('routeSpeech reports each delivered publish as mcpl:speech-routed, with nothing that ties it to a stream or ends one', async () => { + const { registry, traces } = makeRegistry({ delivered: true }); + seedRegistered(registry, 'discord', 'ch-x'); + registry.handleIncoming('discord', incoming('ch-x', 'hi')); + + const before = traces.length; + await registry.routeSpeech('cairn', 'first reply', registry.resolveLocus('cairn')); + await registry.routeSpeech('cairn', 'second reply', registry.resolveLocus('cairn')); + + // The first publish also opens the channel, and says so in a trace of its + // own; every other trace the two sends give is checked. + const routed = traces.slice(before).filter((t) => t.type !== 'mcpl:channel-opened-by-send'); + assert.deepEqual(routed.map((t) => ({ type: t.type, channelId: t.channelId, text: t.text, delivered: t.delivered })), [ + { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'first reply', delivered: true }, + { type: 'mcpl:speech-routed', channelId: 'ch-x', text: 'second reply', delivered: true }, + ], 'one trace per publish, with its whole text, and none that ends an utterance'); + for (const t of routed) { + assert.deepEqual(['inferenceId', 'streamId', 'index'].filter((key) => key in t), [], + 'no stream id or index, unlike inference:published'); + } +}); + test('routeSpeech routes a conversation fork to its HOME channel, not the global last-inbound (item 3)', async () => { // Two channels are live. chanA registered first; then a message arrives on // chanB, flipping the process-global defaultPublishChannel to chanB. A fork diff --git a/test/code-execution.test.ts b/test/code-execution.test.ts index 43645ed44..683700502 100644 --- a/test/code-execution.test.ts +++ b/test/code-execution.test.ts @@ -27,6 +27,7 @@ import type { } from '../src/index.js'; import { AgentFramework, PYTHON_RUNTIME_SOURCE, PyRunner, buildInjectedTools } from '../src/index.js'; import { createMockResponse, MockMembrane } from './helpers/mock-membrane.js'; +import type { ContentBlock } from '@animalabs/membrane'; import { PYTC_STOP_ROWS_DRIVER } from './helpers/pytc-stop-rows.js'; const ECHO_TOOLS: { pyName: string; toolName: string }[] = [ @@ -100,6 +101,39 @@ describe('buildInjectedTools', () => { }); describe('PyRunner (real python3)', () => { + it("no two runners' interpreters share a number, so a mark keyed by one can't match another's", async () => { + const first = new PyRunner({ onToolCall: async () => '' }); + const second = new PyRunner({ onToolCall: async () => '' }); + try { + assert.strictEqual(first.interpreter, null, 'no interpreter before the first script'); + await first.exec('x = 1', []); + await second.exec('x = 1', []); + assert.notStrictEqual(first.interpreter, null); + assert.notStrictEqual(second.interpreter, null); + assert.notStrictEqual(first.interpreter, second.interpreter, 'each interpreter has its own number'); + } finally { + first.dispose(); + second.dispose(); + } + assert.strictEqual(first.interpreter, null, 'and none once the runner is disposed'); + }); + + it('an interpreter killed by a signal the runner didn\'t send has no number: it is gone', async () => { + const runner = new PyRunner({ onToolCall: async () => '' }); + try { + await runner.exec('x = 1', []); + const child = (runner as unknown as { child: import('node:child_process').ChildProcess }).child; + assert.notStrictEqual(runner.interpreter, null, 'live while it runs'); + const exited = new Promise((resolve) => child.once('exit', resolve)); + process.kill(child.pid!, 'SIGKILL'); + await exited; + assert.equal(child.killed, false, 'the runner sent no signal'); + assert.strictEqual(runner.interpreter, null, 'and once the signal ended it, none is live'); + } finally { + runner.dispose(); + } + }); + it('runs a script and returns stdout', async () => { const runner = new PyRunner({ onToolCall: async () => '' }); try { @@ -1246,6 +1280,1171 @@ describe('framework code_execution integration (real python3)', () => { }); }); +// --------------------------------------------------------------------------- +// A round that runs code_execution: its script's calls count as the round's +// --------------------------------------------------------------------------- + +describe("a round that runs code_execution is routed once its scripts return (real python3)", () => { + /** echo, send_message and a `slow` tool held until the test lets it go; each call is logged. */ + class RoundToolModule implements Module { + readonly name = 'test'; + releaseSlow: () => void = () => {}; + private slowReleased = new Promise((resolve) => { this.releaseSlow = resolve; }); + constructor(private readonly log: string[]) {} + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + getTools(): ToolDefinition[] { + const tool = (name: string): ToolDefinition => ({ name, description: name, inputSchema: { type: 'object', properties: {} } }); + return [tool('echo'), tool('send_message'), tool('slow')]; + } + async handleToolCall(call: ToolCall): Promise { + const name = call.name.includes('--') ? call.name.slice(call.name.lastIndexOf('--') + 2) : call.name; + this.log.push(`tool:${name}`); + if (name === 'slow') { + await this.slowReleased; + this.log.push('slow:returned'); + } + return { success: true, data: { ok: true } }; + } + async onProcess(event: ProcessEvent, _state: ProcessState): Promise { + return event.type === 'external-message' + ? { addMessages: [{ participant: 'User', content: [{ type: 'text', text: 'go' }] }], requestInference: true } + : {}; + } + } + + const CHANNEL_TOOLS = [ + { name: 'skip_reply', description: 'End your turn without sending anything.', inputSchema: { type: 'object' as const, properties: {} } }, + { name: 'think', description: 'Reason privately.', inputSchema: { type: 'object' as const, properties: { content: { type: 'string' } } } }, + { name: 'channel_open', description: 'Open a channel.', inputSchema: { type: 'object' as const, properties: { channelId: { type: 'string' } } } }, + ]; + + async function scriptRound(mode: 'locus' | 'hybrid' | 'explicit' | 'disabled', responses: ReturnType[], + config: { sameRoundThinkTextPolicy?: 'private'; allowedTools?: string[] } = {}, options: { codeExecution?: boolean } = {}) { + const { tempDir, storePath } = tempStorePath('pytc-round-'); + const membrane = new MockMembrane(); + for (const response of responses) membrane.pushResponse(response); + const log: string[] = []; + const module = new RoundToolModule(log); + const framework = await AgentFramework.create({ + storePath, + membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test-model', systemPrompt: 'Act.', proseRouting: mode, ...config }], + modules: [module], + syncIntervalMs: 0, + ...(options.codeExecution === false ? {} : { codeExecution: { enabled: true } }), + }); + // The registry stub: skip_reply and think are on the agent's surface, so + // its scripts get them too, and skip_reply ends the turn as the real one + // does. + // What each `>>` label names; a test can rename one while a script runs. + const directory: Record = { '#alpha': 'chan-alpha' }; + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ + resolveLocus: () => 'world:test', + resolveProseTarget: (spec: string) => (directory[spec] ? { channelId: directory[spec] } : { error: `no channel ${spec}` }), + routeSpeech: async (_agent: string, text: string, channelId: string | null) => { + if (!channelId) return null; + log.push(`publish:${text}@${channelId}`); + return { delivered: true, channelId }; + }, + // A named or sticky destination posts through deliverSpeech, as with + // the real registry, which reports the post's outcome. + deliverSpeech: async (_agent: string, text: string, channelId: string | null) => { + if (!channelId) return { status: 'failed', reason: 'no channel', at: Date.now() }; + log.push(`publish:${text}@${channelId}`); + return { status: 'delivered', destination: { serverId: 'stub', channelId }, at: Date.now() }; + }, + resolveDestination: ({ channelId }: { channelId: string }) => ({ destination: { serverId: 'stub', channelId } }), + sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { + log.push(`chunk:${delta}@${channelId}`); + }, + handleChannelToolCall: async (name: string, input: Record | undefined) => { + log.push(`tool:${name}`); + if (name === 'skip_reply') return { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }; + // A successful channel_open moves the turn's locus, as the real one does. + if (name === 'channel_open') return { success: true, data: { channelId: input?.channelId } }; + return { success: true, data: {} }; + }, + getChannelTools: () => CHANNEL_TOOLS, + getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, + }, { get: (target, key: string) => (key in target ? (target as Record)[key] : () => undefined) }); + const run = async () => { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + await framework.runUntilIdle(); + }; + const done = async () => { + module.releaseSlow(); + await framework.stop(); + rmSync(tempDir, { recursive: true, force: true }); + }; + return { framework, module, log, run, done, directory }; + } + + const scriptCall = (input: Record, id = 'ce1') => + ({ type: 'tool_use', id, name: 'code_execution', input }) as ContentBlock; + /** The assistant's open drafts, oldest first, by their words and why each was held. */ + const heldDrafts = (framework: AgentFramework) => + (framework as unknown as { proseDrafts: { open(agent: string): Array<{ text: string; reason: string }> } }) + .proseDrafts.open('assistant').map((d) => ({ text: d.text, reason: d.reason })).reverse(); + // A script holds its round only when it can change the round's routing: + // its code names such a tool (scriptMayRoute). A script given this suffix + // names skip_reply on a line that never runs, so its round waits for it. + const HOLDS = '\nif False: await skip_reply({})'; + // An agent whose allowedTools names only code_execution and test--echo: a + // script reaches any other tool only by forging a call through echo's + // globals, and the host refuses that call at dispatch. + const LISTED = { allowedTools: ['code_execution', 'test--echo'] }; + const FORGE = 'make = test__echo.__globals__["_make_tool_fn"]\n'; + + for (const mode of ['locus', 'hybrid', 'explicit'] as const) { + it(`${mode}: a skip_reply inside the script keeps its round's prose from delivery and the stream`, async () => { + const prose = mode === 'explicit' ? '>>#alpha checking quietly' : 'checking quietly'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: prose }, + scriptCall({ code: 'await skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:skip_reply'), 'the script called skip_reply'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round's prose is kept, as a direct skip_reply keeps it"); + } finally { await h.done(); } + }); + + // With no EventGate, a skip_reply asking for a timed wake is refused, and + // the turn goes on. Refused inside the script, it counts as a direct one: + // its own round stays private, and the next round's prose is published. + it(`${mode}: a skip_reply refused inside the script keeps only its own round private`, async () => { + const kept = mode === 'explicit' ? '>>#alpha checking quietly' : 'checking quietly'; + const said = mode === 'explicit' ? '>>#alpha no timer here, so now' : 'no timer here, so now'; + const to = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: kept }, + scriptCall({ code: 'await skip_reply({"wake_in_seconds": 60})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: said }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(!h.log.includes('tool:skip_reply'), 'the host refused the call before the registry saw it'); + const published = h.log.filter((e) => e.startsWith('publish:')); + assert.deepStrictEqual(published, [`publish:no timer here, so now@${to}`], + "the refused call's round is kept, and the next round's prose is published"); + assert.ok(!h.log.some((e) => e.startsWith('chunk:checking quietly')), "nor does the kept round's prose stream"); + } finally { await h.done(); } + }); + + // A skip_reply the agent's allowedTools leaves out isn't among the + // script's functions, so the script forges the call, and the host refuses + // it at dispatch. It counts as a refused direct one does: its own round + // stays private, and the next round's prose is published. + it(`${mode}: a skip_reply the allowedTools refuses inside the script keeps only its own round private`, async () => { + const kept = mode === 'explicit' ? '>>#alpha checking quietly' : 'checking quietly'; + const said = mode === 'explicit' ? '>>#alpha not mine to skip, so now' : 'not mine to skip, so now'; + const to = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: kept }, + scriptCall({ code: `${FORGE}print(await make("skip_reply", "forged")({}))` }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: said }] as ContentBlock[]), + ], LISTED); + try { + await h.run(); + assert.ok(!h.log.includes('tool:skip_reply'), 'the host refused the call before the registry saw it'); + const published = h.log.filter((e) => e.startsWith('publish:')); + assert.deepStrictEqual(published, [`publish:not mine to skip, so now@${to}`], + "the refused call's round is kept, and the next round's prose is published"); + assert.ok(!h.log.some((e) => e.startsWith('chunk:checking quietly')), "nor does the kept round's prose stream"); + assert.deepStrictEqual(heldDrafts(h.framework), [], 'and nothing is held as a draft'); + } finally { await h.done(); } + }); + } + + // A send the agent's allowedTools leaves out posted nothing, so, refused + // inside the script as a direct one would be, it silences nothing: its round + // and the next are published, and nothing is held as a draft. + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a send the allowedTools refuses inside the script silences nothing`, async () => { + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: `${FORGE}print(await make("test--send_message", "forged")({}))` }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ], LISTED); + try { + await h.run(); + assert.ok(!h.log.includes('tool:send_message'), 'the host refused the send before the module saw it'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), + ['publish:posting the summary@world:test', 'publish:done@world:test'], + 'the round and the next are published, as with no send at all'); + assert.deepStrictEqual(heldDrafts(h.framework), [], 'and nothing is held as a draft'); + } finally { await h.done(); } + }); + + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a send inside the script silences its round and the turn's later prose`, async () => { + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await test__send_message({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the script sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + 'the send silences the round and what follows, as a direct send does'); + assert.deepStrictEqual(heldDrafts(h.framework), [ + { text: 'posting the summary', reason: 'explicit-send' }, + { text: 'done', reason: 'explicit-send' }, + ], 'and holds their words as drafts, as a direct send does'); + } finally { await h.done(); } + }); + + // A drafts resend is an explicit send, so it silences its round as + // send_message does: called from the round's script too. This one names a + // draft there isn't, so it's refused and sends nothing, as a direct one + // would be, and still counts, as a direct one does. + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a drafts resend inside the script holds its round and the turn's later prose as drafts`, async () => { + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: 'sending the earlier reply again' }, + scriptCall({ code: 'print(await drafts({"action": "resend", "draftIds": ["d-0000000"], "destination": "#alpha"}))' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + 'the resend silences the round and what follows, as a direct resend does'); + assert.deepStrictEqual(heldDrafts(h.framework), [ + { text: 'sending the earlier reply again', reason: 'explicit-send' }, + { text: 'done', reason: 'explicit-send' }, + ], 'and their words are held as drafts'); + } finally { await h.done(); } + }); + + // skip_reply's privacy outranks a send's hold, as for direct calls: a + // round whose script calls both keeps its prose private, and holds no draft. + for (const mode of ['locus', 'hybrid'] as const) it(`${mode}: a script that sends and calls skip_reply keeps its round's prose private, with no draft`, async () => { + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: 'posting the summary, then going quiet' }, + scriptCall({ code: 'await test__send_message({})\nawait skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message') && h.log.includes('tool:skip_reply'), 'the script sent and skipped'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], 'nothing is published'); + assert.deepStrictEqual(heldDrafts(h.framework), [], 'and the round is kept private, not held as a draft'); + } finally { await h.done(); } + }); + + it("explicit: a send inside the script silences neither its round nor the turn's later prose, as a direct send doesn't", async () => { + const h = await scriptRound('explicit', [ + createMockResponse([ + { type: 'text', text: '>>#alpha posting the summary' }, + scriptCall({ code: 'await test__send_message({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: '>>#alpha done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the script sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [ + 'publish:posting the summary@chan-alpha', 'chunk:posting the summary@chan-alpha', + 'publish:done@chan-alpha', 'chunk:\n\ndone@chan-alpha', + ], "explicit mode's prefixed prose is deliberate, so a send doesn't silence it"); + } finally { await h.done(); } + }); + + for (const mode of ['locus', 'hybrid', 'explicit'] as const) it(`${mode}: a think inside the script holds its round back under the private same-round policy`, async () => { + const prefix = mode === 'explicit' ? '>>#alpha ' : ''; + const target = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: `${prefix}working it through` }, + scriptCall({ code: 'await think({"content": "notes"})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: `${prefix}the answer` }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the script called think'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [`publish:the answer@${target}`], + 'the round beside the think stays private, as with a direct think, and the next round is delivered'); + } finally { await h.done(); } + }); + + it("the round's prose goes out once its script returns, not after a slow tool beside it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'on it' }, + scriptCall({ code: 'print(await test__echo({}))' + HOLDS }), + { type: 'tool_use', id: 'slow1', name: 'test--slow', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Let the slow tool go once the round's prose has gone out, or after a + // bound, so a round that waited for it still ends and fails the order. + const watch = setInterval(() => { if (h.log.includes('publish:on it@world:test')) h.module.releaseSlow(); }, 10); + // The bound only matters when the code is wrong: a round that waited + // for the slow tool still ends, and fails the order. + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:on it@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('tool:echo') < at('publish:on it@world:test'), "it waited for the script's own call"); + assert.ok(at('publish:on it@world:test') < at('slow:returned'), 'and not for the slow tool beside it'); + assert.ok(at('publish:on it@world:test') < at('chunk:on it@world:test'), 'and its chunk followed its publish'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a script whose code names no tool that changes routing doesn't hold its round: its prose goes out while the script runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The script waits in its slow call until the round's prose has gone out, + // or until a bound, so a round that waited for the script still ends. + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + /** Runs a one-script round whose prose should go out while its script waits in a slow call. */ + const publishedWhileScriptRuns = async (mode: 'locus' | 'explicit', code: string) => { + const prefix = mode === 'explicit' ? '>>#alpha ' : ''; + const target = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + const h = await scriptRound(mode, [ + createMockResponse([ + { type: 'text', text: `${prefix}running the analysis` }, + scriptCall({ code }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: `${prefix}done` }] as ContentBlock[]), + ]); + const published = `publish:running the analysis@${target}`; + const watch = setInterval(() => { if (h.log.includes(published)) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at(published) >= 0, 'the prose was delivered'); + assert.ok(at(published) < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }; + + it("a name inside a longer identifier, as say_hello or my_say, doesn't hold its round", async () => { + await publishedWhileScriptRuns('locus', 'say_hello = 1\nmy_say = 2\nsend_message_log = []\nawait test__slow({})'); + }); + + it("a comment that names a routing tool doesn't hold its round, since Python never runs it", async () => { + await publishedWhileScriptRuns('locus', 'await test__slow({}) # then skip_reply, or say so'); + }); + + it("explicit: a script that names only a send doesn't hold its round, since a send can't change an explicit round's routing", async () => { + await publishedWhileScriptRuns('explicit', 'await test__slow({})\nif False: await test__send_message({})'); + }); + + it("disabled: a script that names skip_reply doesn't hold its round, since nothing is routed", async () => { + const h = await scriptRound('disabled', [ + createMockResponse([ + { type: 'text', text: 'never routed' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + let waitedOn: boolean | undefined; + const watch = setInterval(() => { + if (!h.log.includes('tool:slow')) return; + clearInterval(watch); + waitedOn = rounds.has('assistant'); + h.module.releaseSlow(); + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.equal(waitedOn, false, 'no round waited on the script'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [], 'and nothing was published'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a script whose code names a world utterance, as say, holds its round until it returns", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'about to speak' }, + scriptCall({ code: 'await test__slow({})\nif False: await mcpl__world__say({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Once the script is in its slow call, a round routed at its tool-calls + // event has had its delivery queued long before; with the agent's queued + // deliveries run, the round must still be unpublished. Then the call goes. + let publishedWhileRunning: boolean | undefined; + let checking = false; + const watch = setInterval(() => { + if (checking || !h.log.includes('tool:slow')) return; + checking = true; + const queued = (h.framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + void Promise.resolve(queued).then(() => {}, () => {}).then(() => { + publishedWhileRunning = h.log.includes('publish:about to speak@world:test'); + h.module.releaseSlow(); + }); + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.equal(publishedWhileRunning, false, "the round wasn't published while its script ran"); + assert.ok(at('slow:returned') < at('publish:about to speak@world:test'), 'it went out once the script had returned'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a tool name a script builds at runtime goes unseen: its round is routed at its tool-calls event, as before scripts counted", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'not held' }, + scriptCall({ code: 'name = "skip" + "_reply"\nawait tools[name]({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('tool:skip_reply') >= 0, 'the script called skip_reply by a name it built'); + assert.ok(at('publish:not held@world:test') >= 0, + "the round was published anyway: its code names no tool that changes routing, so it didn't wait for the skip_reply that would have kept it back"); + } finally { await h.done(); } + }); + + it("under the public same-round policy, a script that calls think doesn't hold its round", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'thinking it over' }, + scriptCall({ code: 'await think({"content": "notes"})\nawait test__slow({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:thinking it over@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:thinking it over@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:thinking it over@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("without code execution, a code_execution call holds nothing: its round is routed at its tool-calls event", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'not a script after all' }, + scriptCall({ code: 'await skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ], {}, { codeExecution: false }); + try { + await h.run(); + assert.ok(h.log.includes('publish:not a script after all@world:test'), + "the call is answered as an unknown tool, whose result settles no round, so the round doesn't wait for it"); + } finally { await h.done(); } + }); + + it("a helper an earlier script defined holds a later script's round: the interpreter ran a script that names a tool that changes routing", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await post()' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the helper sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:defining a helper@world:test'], + "the send through the helper silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + it("a script that only calls a routing tool directly, by name or by key, leaves no mark: a later script's round isn't held for it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'thinking first' }, + scriptCall({ code: 'await think({"content": "notes"})\nawait tools["think"]({"content": "more"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(!h.log.includes('publish:thinking first@world:test'), 'the round whose script thought stays private'); + assert.ok(at('publish:running the analysis@world:test') >= 0, "the later round's prose was delivered"); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while its script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a key an earlier script kept holds a later script's round, as an alias does", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'keeping a key' }, + scriptCall({ code: 'name = "test--send_message"' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await tools[name]({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the kept key sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:keeping a key@world:test'], + "the send through the kept key silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + it("an alias an earlier script bound to a routing tool holds a later script's round, as a helper does", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'keeping a handle' }, + scriptCall({ code: 'post = test__send_message' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'posting the summary' }, + scriptCall({ code: 'await post({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:send_message'), 'the alias sent'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:keeping a handle@world:test'], + "the send through the alias silences its round and the turn's later prose, as a direct send does"); + } finally { await h.done(); } + }); + + /** The agent's interpreter mark: the names it holds, or undefined when there is none. */ + const markOf = (framework: AgentFramework) => { + const mark = (framework as unknown as { scriptRoutingInterpreters: Map }> }) + .scriptRoutingInterpreters.get('assistant'); + return mark && [...mark.names].sort(); + }; + + it("a script that calls a routing tool directly, beside a definition that doesn't use it, leaves no mark", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'formatting notes' }, + scriptCall({ code: 'def fmt(x):\n return x.upper()\nawait think({"content": fmt("notes")})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the script thought'); + assert.equal(markOf(h.framework), undefined, 'fmt can still be called, but it reaches no routing tool'); + } finally { await h.done(); } + }); + + it('a later mark in the same interpreter adds its names to the earlier ones', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'keeping a handle' }, + scriptCall({ code: 't = think' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.deepStrictEqual(markOf(h.framework), ['send_message', 'think'], 'the helper the first script left is still there'); + } finally { await h.done(); } + }); + + it('a script the busy runner refuses leaves no mark, though its code would have', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'two scripts at once' }, + scriptCall({ code: 'await test__slow({})' }, 'ce1'), + scriptCall({ code: 'post = test__send_message' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + // The control: the same script, run alone, marks. + createMockResponse([ + { type: 'text', text: 'now alone' }, + scriptCall({ code: 'post = test__send_message' }, 'ce3'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // Let the first script go once the second's refusal has come back, and + // read the mark once the first has finished, before the control runs. + const results: string[] = []; + let markAfterRefusal: string[] | undefined = ['unread']; + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') { + results.push(event.callId); + if (event.callId === 'ce2') h.module.releaseSlow(); + if (event.callId === 'ce1') markAfterRefusal = markOf(h.framework); + } + }); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(results, ['ce2', 'ce1', 'ce3'], "the second script was refused while the first ran"); + assert.equal(markAfterRefusal, undefined, 'the refused alias never ran, so it marked nothing'); + assert.deepStrictEqual(markOf(h.framework), ['send_message'], 'the same script, run alone, marks'); + } finally { + clearTimeout(bound); + await h.done(); + } + }); + + it("a mark left in an earlier interpreter doesn't hold a round once a newer interpreter runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'starting fresh' }, + scriptCall({ code: 'x = 1' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce3'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The helper's interpreter is reclaimed once its script returns; the next + // script starts a newer one, which is live when the third round is decided. + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'ce1') { + const runner = (h.framework as unknown as { codeExecutionRunners: Map }) + .codeExecutionRunners.get('assistant')!; + runner.reclaim('test'); + } + }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the third round was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while its script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("under the public same-round policy, a helper that calls think doesn't hold a later script's round", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def note():\n await think({"content": "notes"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await note()\nawait test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(h.log.includes('tool:think'), 'the helper thought'); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("under the private same-round policy, a helper that calls think holds a later script's round back", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def note():\n await think({"content": "notes"})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'working it through' }, + scriptCall({ code: 'await note()' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'the answer' }] as ContentBlock[]), + ], { sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + assert.ok(h.log.includes('tool:think'), 'the helper thought'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), + ['publish:defining a helper@world:test', 'publish:the answer@world:test'], + 'the round whose helper thought stays private, as with a direct think, and the rounds around it are delivered'); + } finally { await h.done(); } + }); + + it("once the interpreter is reclaimed, a script that names no such tool doesn't hold its round, since the helpers went with it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'defining a helper' }, + scriptCall({ code: 'async def post():\n await test__send_message({})' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([ + { type: 'text', text: 'running the analysis' }, + scriptCall({ code: 'await test__slow({})' }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The interpreter that ran the helper's script is reclaimed once that + // script returns, as an idle reclaim or a restart would. + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'ce1') { + const runner = (h.framework as unknown as { codeExecutionRunners: Map }) + .codeExecutionRunners.get('assistant')!; + runner.reclaim('test'); + } + }); + const watch = setInterval(() => { if (h.log.includes('publish:running the analysis@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:running the analysis@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:running the analysis@world:test') < at('slow:returned'), 'while the script was still running'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a background script's round is routed as soon as the script starts", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'starting a watcher' }, + scriptCall({ code: 'import asyncio\nawait asyncio.sleep(30)', background: true }), + { type: 'tool_use', id: 'slow1', name: 'test--slow', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const watch = setInterval(() => { if (h.log.includes('publish:starting a watcher@world:test')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('publish:starting a watcher@world:test') >= 0, 'the prose was delivered'); + assert.ok(at('publish:starting a watcher@world:test') < at('slow:returned'), 'without waiting for the slow tool'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a round with two code_execution calls waits for both: the second is refused at once, and the first's later skip_reply still counts", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'two scripts at once' }, + scriptCall({ code: 'await test__slow({})\nawait skip_reply({})' }, 'ce1'), + scriptCall({ code: 'print("second")' + HOLDS }, 'ce2'), + ] as ContentBlock[], 'tool_use'), + ]); + // Let the first script go once the second's refusal has come back. + const results: string[] = []; + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') { + results.push(event.callId); + if (event.callId === 'ce2') h.module.releaseSlow(); + } + }); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(results, ['ce2', 'ce1'], "the second script's refusal came back while the first ran"); + assert.ok(h.log.includes('tool:skip_reply'), "the first script's skip_reply ran"); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round waited for the first script, so its skip_reply kept the round's prose"); + } finally { + clearTimeout(bound); + await h.done(); + } + }); + + it("a background script's calls aren't its round's, even while a foreground script in the round runs", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'still mine to say' }, + scriptCall({ code: 'await test__send_message({})', background: true }, 'bg'), + scriptCall({ code: 'await test__slow({})' + HOLDS }, 'fg'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // The foreground script holds the round open until the background send has run. + const watch = setInterval(() => { if (h.log.includes('tool:send_message')) h.module.releaseSlow(); }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + const at = (entry: string) => h.log.indexOf(entry); + assert.ok(at('tool:send_message') >= 0 && at('tool:send_message') < at('slow:returned'), + 'the background send ran while the round waited on its foreground script'); + assert.ok(h.log.includes('publish:still mine to say@world:test'), "the background send didn't silence the round"); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("locus: a script round's prose goes to the locus it was written for, though the script opens another channel", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'for the room I was in' }, + scriptCall({ code: 'await channel_open({"channelId": "chan-other"})' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'now over here' }] as ContentBlock[]), + ]); + try { + await h.run(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), [ + 'publish:for the room I was in@world:test', + 'publish:now over here@chan-other', + ], 'the open moves later prose, not the round written before it'); + } finally { await h.done(); } + }); + + it("an ephemeral disposed while its round's script runs doesn't publish the round when the script returns", async () => { + const h = await scriptRound('locus', []); + const membrane = (h.framework as unknown as { membrane: MockMembrane }).membrane; + membrane.pushResponse(createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }), + ] as ContentBlock[], 'tool_use')); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + try { + const created = await h.framework.createEphemeralAgent({ name: 'eph-script', model: 'test-model', systemPrompt: 'Act.', allowedTools: 'all' }); + created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); + // The idle watchdog disposes the ephemeral while its script waits in the + // slow call. Its threshold is too long for python3's start to trip it, so + // the run is made to look idle only once the script is in the slow call, + // and the next poll disposes it. + const run = h.framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 15000, idleTimeoutMs: 60_000, idlePollMs: 20 }); + const stalled = assert.rejects(run, /stalled/); + h.framework.start(); + const deadline = Date.now() + 15000; + while (Date.now() < deadline && !h.log.includes('tool:slow')) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached the slow call'); + const runs = (h.framework as unknown as { ephemeralRuns: Map }).ephemeralRuns; + runs.get(created.agent.name)!.lastActivity = 0; + await stalled; + // The script now returns, and its result arrives with the framework still running. + let resulted!: () => void; + const result = new Promise((resolve) => { resulted = resolve; }); + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') resulted(); + }); + h.module.releaseSlow(); + await result; + await h.framework.runUntilIdle(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "the round's routing was never decided, so the late result publishes nothing"); + assert.ok(lines.some((line) => line.includes('eph-script: round [code_execution] -> 1 prose segment(s) withheld: its stream ended before its scripts returned')), + "the disposed ephemeral's stream withheld the round as it ended"); + } finally { + console.error = error; + await h.done(); + } + }); + + // Each way a stream can stop owning its round while the round's script runs: + // the agent stops waiting on it, is deregistered, or moves to a newer stream. + const leaveRound: Record void> = { + 'its agent is reset': (_framework, agent) => agent.reset(), + 'its agent is deregistered': (framework) => { + (framework as unknown as { agents: Map }).agents.delete('assistant'); + }, + 'its agent moves to a newer stream': (_framework, agent) => { + (agent as unknown as { _streamId: number })._streamId++; + }, + }; + for (const [how, leave] of Object.entries(leaveRound)) it(`a round is withheld when its script returns if, meanwhile, ${how}`, async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + ]); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + let stream: { cancel(): void } | undefined; + let agent: { state: { stream?: { cancel(): void } }; reset(): void } | undefined; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + void h.framework.runUntilIdle().catch(() => {}); + const started = Date.now(); + while (!h.log.includes('tool:slow') && Date.now() - started < 15000) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached its slow call'); + // The turn's routing state, whose receipt counts what the turn suppressed. + const routing = (h.framework as unknown as { turnRouting: Map }).turnRouting.get('assistant')!; + const suppressedBefore = routing.suppressed; + // The stream no longer owns the round, while it lives on. + agent = h.framework.getAgent('assistant')! as unknown as { state: { stream?: { cancel(): void } }; reset(): void }; + stream = agent.state.stream; + leave(h.framework, agent); + let resulted!: () => void; + const result = new Promise((resolve) => { resulted = resolve; }); + h.framework.onTrace((event) => { + if ((event.type === 'tool:completed' || event.type === 'tool:failed') && event.tool === 'code_execution') resulted(); + }); + h.module.releaseSlow(); + await result; + const until = Date.now() + 15000; + while (Date.now() < until && !lines.some((line) => line.includes('prose segment(s) withheld'))) await new Promise((r) => setTimeout(r, 10)); + assert.ok(lines.some((line) => line.includes('assistant: round [code_execution] -> 1 prose segment(s) withheld: its turn no longer waits on it')), + 'the script result found the round undecidable and withheld it'); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], 'and published nothing'); + assert.equal(routing.suppressed, suppressedBefore + 1, "and the turn's receipt counts the withheld segment as suppressed"); + assert.deepStrictEqual(routing.drafts, [], 'and holds none of it as a draft, since the script might have kept it private'); + } finally { + console.error = error; + stream?.cancel(); + // The departures are simulated, without the bookkeeping a real one does: + // a real deregistration also clears the name's active stream, and a real + // newer stream takes that entry over and clears it when it ends. Clear + // it and settle the agent, so the loop this test started finds the + // framework idle and the file can exit. + (h.framework as unknown as { activeStreams: Map }).activeStreams.delete('assistant'); + agent?.reset(); + await h.done(); + } + }); + + it("a round whose stream is stopped before its script returns is withheld", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'never decided' }, + scriptCall({ code: 'await test__slow({})\nawait skip_reply({})' }), + ] as ContentBlock[], 'tool_use'), + ]); + const lines: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); error(...args); }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + void h.framework.runUntilIdle().catch(() => {}); + // The script is running, held in its slow call. + const started = Date.now(); + while (!h.log.includes('tool:slow') && Date.now() - started < 15000) await new Promise((r) => setTimeout(r, 10)); + assert.ok(h.log.includes('tool:slow'), 'the script reached its slow call'); + await h.framework.stop(); + h.module.releaseSlow(); + await new Promise((r) => setTimeout(r, 50)); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:') || e.startsWith('chunk:')), [], + "whether the script would have silenced the round is unknown, so its prose isn't sent"); + assert.ok(lines.some((line) => line.includes('1 prose segment(s) withheld: its stream ended before its scripts returned')), + 'and the withheld round is logged, not dropped silently'); + } finally { + console.error = error; + await h.done(); + } + }); + + /** A pending round of another stream, as scriptRounds holds one: it records what it was asked to do. */ + const pendingRound = (agent: unknown, pending: string[] = ['elsewhere']) => { + const asked: string[] = []; + return { + asked, + round: { + agent, streamId: -1, scripts: new Map(), pending: new Set(pending), + route: () => { asked.push('route'); }, + withhold: (why: string) => { asked.push(`withhold: ${why}`); }, + }, + }; + }; + + it("a script's result settles only the round that dispatched it, though a newer round waits on the same call id", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the first round' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + // While the script runs, a newer round comes to wait under the name on a + // call with the same id, as a provider that reuses ids could make one. + const newer = pendingRound(h.framework.getAgent('assistant'), ['ce1']); + let swapped = false; + const watch = setInterval(() => { + if (!h.log.includes('tool:slow')) return; + clearInterval(watch); + rounds.set('assistant', newer.round); + swapped = true; + h.module.releaseSlow(); + }, 10); + // A script that never reaches its slow call still ends, and fails below. + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.ok(swapped, 'the newer round came to wait while the script ran'); + assert.deepStrictEqual(newer.asked, [], "the first round's result doesn't settle the newer round"); + assert.equal(rounds.get('assistant'), newer.round, 'which still waits'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); + + it("a newer round that doesn't wait displaces an older waiting round, so a call of its own with the same id can't settle it", async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the newer round' }, + scriptCall({ code: 'print("quick")' }, 'ce1'), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // An older round of another stream still waits on a call with the same + // id, as a provider that reuses ids could leave one. + const older = pendingRound({}, ['ce1']); + (h.framework as unknown as { scriptRounds: Map }).scriptRounds.set('assistant', older.round); + try { + await h.run(); + assert.deepStrictEqual(older.asked, ['withhold: a newer round replaced it before its scripts returned'], + "the older round is withheld, and the newer round's result settles nothing of it"); + assert.ok(h.log.includes('publish:the newer round@world:test'), 'and the newer round is routed at once'); + } finally { await h.done(); } + }); + + it('a round registered while another waits under the same agent name withholds the older one', async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'the newer round' }, + scriptCall({ code: 'print("quick")' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + const older = pendingRound({}); + (h.framework as unknown as { scriptRounds: Map }).scriptRounds.set('assistant', older.round); + try { + await h.run(); + assert.deepStrictEqual(older.asked, ['withhold: a newer round replaced it before its scripts returned'], + 'the older round is withheld, not overwritten silently'); + assert.ok(h.log.includes('publish:the newer round@world:test'), 'and the newer round is routed as usual'); + } finally { await h.done(); } + }); + + // A successor's round comes to wait under the agent's name while this stream + // runs: one of another agent object of the name, or one of the same agent + // object's later stream (a restart overlapping this one). + for (const whose of ['another agent object', "the same agent object's later stream"] as const) { + it(`a stream's teardown leaves alone a round that belongs to ${whose}`, async () => { + const h = await scriptRound('locus', [ + createMockResponse([ + { type: 'text', text: 'working' }, + { type: 'tool_use', id: 'r1', name: 'test--echo', input: {} }, + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[]), + ]); + // pendingRound's round has stream id -1, which no stream has. + const successor = pendingRound(whose === 'another agent object' ? {} : h.framework.getAgent('assistant')); + const rounds = (h.framework as unknown as { scriptRounds: Map }).scriptRounds; + h.framework.onTrace((event) => { + if (event.type === 'tool:completed' && event.callId === 'r1') rounds.set('assistant', successor.round); + }); + try { + await h.run(); + assert.ok(h.log.includes('publish:done@world:test'), 'the stream ran to its end'); + assert.deepStrictEqual(successor.asked, [], "this stream's teardown doesn't withhold the successor's round"); + assert.equal(rounds.get('assistant'), successor.round, 'nor remove it'); + } finally { await h.done(); } + }); + } + + it("explicit: a script round goes to the target its `>>` named when the round ended, though the label is renamed while the script runs", async () => { + const h = await scriptRound('explicit', [ + createMockResponse([ + { type: 'text', text: '>>#alpha written for alpha' }, + scriptCall({ code: 'await test__slow({})' + HOLDS }), + ] as ContentBlock[], 'tool_use'), + createMockResponse([{ type: 'text', text: '>>skip_reply done' }] as ContentBlock[]), + ]); + // While the script waits in its slow call, #alpha comes to name another channel. + const watch = setInterval(() => { + if (h.log.includes('tool:slow')) { h.directory['#alpha'] = 'chan-renamed'; h.module.releaseSlow(); } + }, 10); + const bound = setTimeout(() => h.module.releaseSlow(), 15000); + try { + await h.run(); + assert.deepStrictEqual(h.log.filter((e) => e.startsWith('publish:')), ['publish:written for alpha@chan-alpha'], + 'the round was admitted when it ended'); + } finally { + clearInterval(watch); + clearTimeout(bound); + await h.done(); + } + }); +}); + // --------------------------------------------------------------------------- // Background scripts (wake_agent) + spill-to-file // --------------------------------------------------------------------------- diff --git a/test/explicit-prose-routing.test.ts b/test/explicit-prose-routing.test.ts index 262fb958f..89179747c 100644 --- a/test/explicit-prose-routing.test.ts +++ b/test/explicit-prose-routing.test.ts @@ -26,7 +26,6 @@ import type { } from '../src/index.js'; import { AgentFramework } from '../src/index.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; -import { ProseStreamRouter } from '../src/mcpl/prose-stream-router.js'; import { draftState, type ProseDraftStore } from '../src/prose-drafts.js'; import { MembraneError } from '@animalabs/membrane'; import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; @@ -710,13 +709,13 @@ describe('explicit prose routing', () => { type RoutingState = { - pin?: string; destination: string | null; hybridHold?: 'private' | 'failed'; + pin?: string; hybridHold?: 'private' | 'failed'; continuation: boolean; deliveries: string[]; suppressed: number; drafts: Array<{ id: string; text: string }>; keptPrivate: number; triggerChannel?: string; locus?: string; engaged: Set; arrivals: Array<{ messageId: string; channelId: string }>; }; /** A turn's routing state, as the framework makes a fresh one. */ const routingState = (fields: Partial = {}): RoutingState => ({ - destination: null, continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, + continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, engaged: new Set(), arrivals: [], ...fields, }); @@ -1669,7 +1668,7 @@ describe('explicit prose routing', () => { await framework.stop(); }); - it('the stream reads leading whitespace and a token\'s end as delivery does', async () => { + it("delivery reads leading whitespace and a token's end before a `>>` prefix", async () => { const framework = await createFramework(); const routed = stubRegistry(framework); const gateway = framework as unknown as { deliverProse(agent: unknown, text: string): Promise }; @@ -1679,66 +1678,23 @@ describe('explicit prose routing', () => { for (const text of cases) { routed.length = 0; await gateway.deliverProse(agent, text); - const router = new ProseStreamRouter({ - mode: 'explicit', initialTarget: null, resolve: (spec) => (spec === '#alpha' ? 'chan-alpha' : null), - }); - const streamed = [...router.feed(text), ...router.finish()]; - assert.deepEqual(routed.map((r) => r.locus), ['chan-alpha'], `delivery sends ${JSON.stringify(text)} to #alpha`); - assert.ok(streamed.length > 0 && streamed.every((d) => d.channelId === 'chan-alpha'), - `the stream sends ${JSON.stringify(text)} there too`); + assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), [{ text: 'hello', locus: 'chan-alpha' }], + `delivery sends the body of ${JSON.stringify(text)} to #alpha`); } } finally { await framework.stop(); } }); - it("a round's destination, decided at its boundary, is the sticky destination its delivery leaves", async () => { - const framework = await createFramework(); - stubRegistry(framework); - const internals = framework as unknown as { - deliverProse(agent: unknown, text: string): Promise; - admitProse(text: string): unknown[]; - roundDestination(current: string | null, envelopes: unknown[]): string | null; - turnRouting: Map; - }; - const agent = framework.getAgent('assistant')!; - const rounds: string[][] = [ - ['>>#alpha first\n>>#alpha'], - [' >>#alpha indented'], - ['>>#nowhere bounced'], - ['>>skip_reply kept\nand unprefixed'], - ['unprefixed', '>>#alpha second segment'], - // A target refused for speech clears the destination (R1-1), and a - // later one that resolves sets it again. - ['>>#ears for the room'], - ['>>#alpha first\n>>#ears for the room'], - ['>>#ears for the room', '>>#alpha after'], - ]; - try { - for (const segments of rounds) { - for (const start of [null, 'chan-alpha']) { - internals.turnRouting.set('assistant', routingState({ pin: start ?? undefined })); - for (const segment of segments) await internals.deliverProse(agent, segment); - const admitted = segments.flatMap((segment) => internals.admitProse(segment)); - assert.equal(internals.roundDestination(start, admitted), internals.turnRouting.get('assistant')?.pin ?? null, - `${JSON.stringify(segments)} from ${start}`); - } - } - } finally { - await framework.stop(); - } - }); - - it('hybrid: a `>>>` envelope with no target streams nothing, as delivery bounces it', async () => { + it('hybrid: a `>>>` envelope with no target publishes nothing, so nothing streams, and holds the turn that queued it', async () => { const framework = await createFramework('hybrid'); const routed = stubRegistry(framework); const gateway = framework as unknown as { - deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, round: { routing: RoutingState; notice: 'later' }): Promise; + deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, + round: { routing: RoutingState; notice: 'later'; onPublished?: (channelId: string, text: string) => void }): Promise; turnRouting: Map; }; const agent = framework.getAgent('assistant')!; - // The registry's resolver trims its argument, as the router's must here. - const resolve = (spec: string) => (spec.trim() === '#alpha' ? 'chan-alpha' : null); const cases = ['>>>\r\nsecret', '>>>\nsecret', '>>>\u00a0#alpha secret', '>>> \u00a0#alpha secret']; try { for (const text of cases) { @@ -1746,78 +1702,56 @@ describe('explicit prose routing', () => { const queued: RoutingState = routingState(); const live: RoutingState = routingState(); gateway.turnRouting.set('assistant', live); - await gateway.deliverHybridProse(agent, text, 'chan-locus', true, { routing: queued, notice: 'later' }); + const streamed: string[] = []; + await gateway.deliverHybridProse(agent, text, 'chan-locus', true, + { routing: queued, notice: 'later', onPublished: (_channel, body) => streamed.push(body) }); assert.equal(queued.hybridHold, 'failed', 'the supplied turn receives the hold'); assert.equal(live.hybridHold, undefined, 'the live turn stays unheld'); - const router = new ProseStreamRouter({ mode: 'hybrid', initialTarget: 'chan-locus', resolve }); - const streamed = [...router.feed(text), ...router.finish()]; assert.equal(routed.length, 0, `delivery bounces ${JSON.stringify(text)}`); - assert.deepEqual(streamed, [], `and the stream sends none of ${JSON.stringify(text)}`); + assert.deepEqual(streamed, [], `and the stream hears none of ${JSON.stringify(text)}`); } } finally { await framework.stop(); } }); - it('hybrid: a segment that is only `>>>`, or `>>>` and a gap, holds back the next segment, as delivery does', async () => { + it('hybrid: a segment that is only `>>>`, or `>>>` and a gap, holds back the next segment, from delivery and the stream alike', async () => { const framework = await createFramework('hybrid'); const routed = stubRegistry(framework); const gateway = framework as unknown as { - deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, round: { routing: RoutingState; notice: 'later' }): Promise; + deliverHybridProse(agent: unknown, text: string, locus: string, allowLocus: boolean, + round: { routing: RoutingState; notice: 'later'; onPublished?: (channelId: string, text: string) => void }): Promise; turnRouting: Map; }; const agent = framework.getAgent('assistant')!; - const resolve = (spec: string) => (spec.trim() === '#alpha' ? 'chan-alpha' : null); try { for (const prefix of ['>>>', '>>> ', '>>>\t']) { routed.length = 0; const queued: RoutingState = routingState(); const live: RoutingState = routingState(); gateway.turnRouting.set('assistant', live); - await gateway.deliverHybridProse(agent, '>>>#alpha first', 'chan-locus', true, { routing: queued, notice: 'later' }); - await gateway.deliverHybridProse(agent, prefix, 'chan-locus', true, { routing: queued, notice: 'later' }); - await gateway.deliverHybridProse(agent, 'secret', 'chan-locus', true, { routing: queued, notice: 'later' }); + const streamed: Array<{ channelId: string; text: string }> = []; + const round = { + routing: queued, + notice: 'later' as const, + onPublished: (channelId: string, text: string) => { streamed.push({ channelId, text }); }, + }; + await gateway.deliverHybridProse(agent, '>>>#alpha first', 'chan-locus', true, round); + await gateway.deliverHybridProse(agent, prefix, 'chan-locus', true, round); + await gateway.deliverHybridProse(agent, 'secret', 'chan-locus', true, round); assert.deepEqual(queued.deliveries, ['chan-alpha'], 'the supplied turn records its delivery'); assert.ok(queued.drafts.some((d) => d.text === 'secret'), 'the supplied turn holds the held-back segment as its draft'); assert.deepEqual(live.deliveries, [], 'the live turn receives no delivery'); assert.deepEqual(live.drafts, [], 'the live turn holds no draft'); - const router = new ProseStreamRouter({ mode: 'hybrid', initialTarget: 'chan-locus', resolve }); - const streamed = [ - ...router.feed('>>>#alpha first'), ...router.endSegment(), - ...router.feed(prefix), ...router.endSegment(), - ...router.feed('secret'), ...router.finish(), - ]; assert.deepEqual(routed.map((r) => r.text), ['first'], `delivery sends only first, after ${JSON.stringify(prefix)}`); - assert.equal(streamed.filter((d) => d.delta.includes('secret')).length, 0, - `and the stream holds secret back after ${JSON.stringify(prefix)}`); + assert.deepEqual(streamed, [{ channelId: 'chan-alpha', text: 'first' }], + `and the stream hears first, where it was published, and holds secret back after ${JSON.stringify(prefix)}`); } } finally { await framework.stop(); } }); - it('the stream neither sends nor keeps text for a channel `speakable` refuses, and keeps what it routes there once `speakable` accepts it', () => { - let earsSpeakable = false; - const router = new ProseStreamRouter({ - mode: 'explicit', - initialTarget: null, - resolve: (spec) => (spec === '#ears' ? 'body:ears' : spec === '#alpha' ? 'chan-alpha' : null), - speakable: (channelId) => channelId !== 'body:ears' || earsSpeakable, - }); - const streamed = [ - ...router.feed('>>#ears refused words'), ...router.endSegment(), - ...router.feed('>>#alpha for alpha'), ...router.endSegment(), - ]; - earsSpeakable = true; - streamed.push(...router.feed('>>#ears spoken words'), ...router.finish()); - assert.deepEqual(streamed, [ - { channelId: 'chan-alpha', delta: 'for alpha' }, - { channelId: 'body:ears', delta: 'spoken words' }, - ]); - assert.deepEqual([...router.byChannel()], [['chan-alpha', 'for alpha'], ['body:ears', 'spoken words']], - 'outgoing/complete keeps only what was streamed'); - }); - it('hybrid: a hold beside a send writes the routing and drafts of its own round, not the live turn\'s', async () => { const framework = await createFramework('hybrid'); stubRegistry(framework); @@ -2242,9 +2176,9 @@ describe('explicit prose routing', () => { const declared = new Set(); registry.inboundOnlyServer = (channelId: string) => (declared.has(channelId) ? 'body' : undefined); // The turn's routing, as its first delivery sees it. - let turn: { pin?: string; destination: string | null } | undefined; + let turn: { pin?: string } | undefined; registry.deliverSpeech = async (_agent: string, text: string, locus?: string | null) => { - turn ??= (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); + turn ??= (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); routed.push({ text, locus: locus ?? null }); if (inboundOnly || refusal === 'delivered, then declared') declared.add(locus ?? ''); const destination = { serverId: 'stub', channelId: locus ?? '' }; @@ -2260,27 +2194,22 @@ describe('explicit prose routing', () => { if (inboundOnly) { assert.deepEqual(routed, [{ text: 'first', locus: 'chan-alpha' }], 'the later text never went to the refused channel'); assert.ok(bounces.some((t) => t.includes('this turn has no destination now')), `the later text bounced: ${bounces.join(' | ')}`); - // Both halves of the destination end: where delivery sends later - // text, and where a later stream of the turn would start. - assert.equal(turn?.pin, undefined); - assert.equal(turn?.destination, null); + assert.equal(turn?.pin, undefined, 'the destination later text would follow ends'); } else { assert.deepEqual(routed, [{ text: 'first', locus: 'chan-alpha' }, { text: 'and more', locus: 'chan-alpha' }]); // Each failed delivery bounces its own words; none bounces for want of a destination. assert.equal(bounces.filter((t) => t.includes('delivery to chan-alpha failed')).length, refusal === 'another failure' ? 2 : 0); assert.equal(bounces.some((t) => t.includes('this turn has no destination now')), false); assert.equal(turn?.pin, 'chan-alpha'); - assert.equal(turn?.destination, 'chan-alpha'); } await framework.stop(); }); } - it('explicit: a refusal at delivery ends a destination that names the refused channel even when the pin has moved off it', async () => { - // Each half ends while it still names the refused channel, on its own. - // Here another delivery (as a replaced stream's queued one can) moves the - // pin to #cafe during #alpha's delivery, while the destination still - // names #alpha: the pin stays, and the destination ends. + it('explicit: a refusal at delivery leaves the pin when another delivery has moved it off the refused channel', async () => { + // The pin ends only while it still names the refused channel. Here + // another delivery (as a replaced stream's queued one can) moves it to + // #cafe during #alpha's delivery, so it stays. membrane.pushResponse(createMockResponse([ { type: 'text', text: '>>#alpha first' }, { type: 'tool_use', id: 'c1', name: 'robot--move', input: { dir: 'north' } }, @@ -2293,15 +2222,15 @@ describe('explicit prose routing', () => { const registry = (framework as unknown as { channelRegistry: Record }).channelRegistry; const declared = new Set(); registry.inboundOnlyServer = (channelId: string) => (declared.has(channelId) ? 'body' : undefined); - let turn: { pin?: string; destination: string | null } | undefined; - let atFailure: { pin?: string; destination: string | null } | undefined; + let turn: { pin?: string } | undefined; + let atFailure: { pin?: string } | undefined; registry.deliverSpeech = async (_agent: string, text: string, locus?: string | null) => { - turn ??= (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); + turn ??= (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); routed.push({ text, locus: locus ?? null }); const destination = { serverId: 'stub', channelId: locus ?? '' }; if (locus === 'chan-alpha') { turn!.pin = 'discord:g:cafe'; - atFailure = { pin: turn!.pin, destination: turn!.destination }; + atFailure = { pin: turn!.pin }; declared.add('chan-alpha'); return { status: 'failed', destination, reason: 'inbound-only: server "body" declared it', at: Date.now() }; } @@ -2310,18 +2239,17 @@ describe('explicit prose routing', () => { feedTurn(framework, 2, [{ type: 'text', text: '>>skip_reply' }] as ContentBlock[]); trigger(framework); await framework.runUntilIdle(); - assert.deepEqual(atFailure, { pin: 'discord:g:cafe', destination: 'chan-alpha' }, 'the state the clear met'); + assert.deepEqual(atFailure, { pin: 'discord:g:cafe' }, 'the state the clear met'); assert.equal(turn?.pin, 'discord:g:cafe', 'the moved pin stays'); - assert.equal(turn?.destination, null, 'the destination that named the refused channel ends'); assert.deepEqual(routed, [{ text: 'first', locus: 'chan-alpha' }, { text: 'and more', locus: 'discord:g:cafe' }]); await framework.stop(); }); - it("explicit: a delivery refused after a later round named another channel leaves that round's destination", async () => { + it("explicit: a delivery refused while a later round waits behind it leaves that round its own destination", async () => { // Greta-2425's probe. Deliveries queue behind each other, and the turn - // goes on without waiting, so round 2's >>#cafe is admitted, setting the - // destination, before round 1's delivery to #alpha fails. The clear ends - // only a destination that still names #alpha. + // goes on without waiting, so round 2's >>#cafe is admitted before round + // 1's delivery to #alpha fails. The clear ends only the pin #alpha set, + // and round 2's delivery then pins #cafe for the text after it. membrane.pushResponse(createMockResponse([ { type: 'text', text: '>>#alpha first' }, { type: 'tool_use', id: 'c1', name: 'robot--move', input: { dir: 'north' } }, @@ -2338,19 +2266,24 @@ describe('explicit prose routing', () => { const registry = (framework as unknown as { channelRegistry: Record }).channelRegistry; const declared = new Set(); registry.inboundOnlyServer = (channelId: string) => (declared.has(channelId) ? 'body' : undefined); - const turnOf = () => (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); - let turn: { pin?: string; destination: string | null } | undefined; - let destinationAtFailure: unknown = 'never reached'; + const turnOf = () => (framework as unknown as { turnRouting: Map }).turnRouting.get('assistant'); + let turn: { pin?: string } | undefined; + // MockMembrane gives a turn's rounds to its one stream, so a round's + // tool calls, not the membrane's calls, show how far the turn has got. + let toolRounds = 0; + framework.onTrace((e) => { if ((e as { type: string }).type === 'inference:tool_calls_yielded') toolRounds++; }); + let roundsAtFailure = 0; registry.deliverSpeech = async (_agent: string, text: string, locus?: string | null) => { turn ??= turnOf(); routed.push({ text, locus: locus ?? null }); const destination = { serverId: 'stub', channelId: locus ?? '' }; if (locus === 'chan-alpha') { - // Hold #alpha's delivery until round 2 is admitted, then refuse it: - // its server has declared it inbound-only by now. + // Hold #alpha's delivery until round 2 is admitted, as its tool calls + // show, then refuse it: its server has declared it inbound-only by + // now. const deadline = Date.now() + 3000; - while (turn?.destination !== 'discord:g:cafe' && Date.now() < deadline) await new Promise((r) => setTimeout(r, 5)); - destinationAtFailure = turn?.destination ?? null; + while (toolRounds < 2 && Date.now() < deadline) await new Promise((r) => setTimeout(r, 5)); + roundsAtFailure = toolRounds; declared.add('chan-alpha'); return { status: 'failed', destination, reason: 'inbound-only: server "body" declared it', at: Date.now() }; } @@ -2359,23 +2292,21 @@ describe('explicit prose routing', () => { feedTurn(framework, 2, [{ type: 'text', text: '>>skip_reply' }] as ContentBlock[]); trigger(framework); await framework.runUntilIdle(); - assert.equal(destinationAtFailure, 'discord:g:cafe', 'round 2 was admitted before #alpha refused'); + assert.ok(roundsAtFailure >= 2, `round 2 was admitted before #alpha refused (${roundsAtFailure} tool rounds)`); assert.deepEqual(routed, [ { text: 'first', locus: 'chan-alpha' }, { text: 'second', locus: 'discord:g:cafe' }, { text: 'and more', locus: 'discord:g:cafe' }, ]); - // Delivery and a later stream of the turn agree on #cafe. assert.equal(turn?.pin, 'discord:g:cafe'); - assert.equal(turn?.destination, 'discord:g:cafe'); await framework.stop(); }); for (const [final, kept] of [['>>#ears ! for the room', false], ['>>#cafe ! for the room', true]] as const) { it(`explicit: a turn's final text (${kept ? 'its target kept' : 'its target refused'}) reaches nothing of its ' !' continuation, which starts a turn of its own`, async () => { - // The turn's final text is delivered after the turn settles, so neither - // its pin nor the destination a later stream would start at carries - // into the continuation: that's a new turn, with no destination. + // The turn's final text is delivered after the turn settles, so its + // pin doesn't carry into the continuation: that's a new turn, with no + // destination. membrane.pushResponse(createMockResponse([ { type: 'text', text: '>>#alpha first' }, { type: 'tool_use', id: 'c1', name: 'robot--move', input: { dir: 'north' } }, @@ -2593,8 +2524,8 @@ describe('explicit prose routing', () => { // ---- A start that continues a logical turn keeps the turn's routing ---- // A framework retry after a stream error, the compile a provider-capacity // hold defers a turn to, and the re-inference that consumes unstreamed tool - // results each continue the turn: the sticky destination, explicit mode's - // turn destination and hybrid's hold carry into the new stream. + // results each continue the turn: the sticky destination and hybrid's hold + // carry into the new stream, which streams what its deliveries publish. /** An organization acceleration 429, which holds the turn for a later compile. */ const accelerationLimit = (request: NormalizedRequest): MembraneError => new MembraneError({ @@ -2675,7 +2606,7 @@ describe('explicit prose routing', () => { for (const [label, mode, first, expected] of [ ['hybrid >>>skip_reply keeps later unprefixed prose private', 'hybrid', '>>>skip_reply an aside', []], ['hybrid >>>#alpha is sticky', 'hybrid', '>>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], - ['explicit >>#alpha is sticky, and the stream starts at the turn\'s destination', 'explicit', '>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], + ['explicit >>#alpha is sticky, and the later stream streams where delivery publishes', 'explicit', '>>#alpha first', [{ text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }]], ] as const) { it(`across ${start}, ${label}`, async () => { const { framework, routed, bounces, laterChunks } = await continueTurn(how, mode, first); @@ -2683,11 +2614,11 @@ describe('explicit prose routing', () => { // Publication as the registry stub saw it; bounce notices by their kind. assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), expected); assert.equal(bounces, 0, 'nothing bounced'); - if (mode === 'explicit') { - // The later stream streams its unprefixed text where delivery sends it. - assert.deepEqual(laterChunks.map((c) => c.channelId), laterChunks.map(() => 'chan-alpha')); - assert.equal(laterChunks.map((c) => c.delta).join(''), 'carrying on'); - } + // In every mode, the later stream streams what it published, where + // it published it: nothing under a hold, and its unprefixed text at + // the sticky destination. + assert.deepEqual(laterChunks.map((c) => ({ text: c.delta, locus: c.channelId })), + expected.filter((post) => post.text === 'carrying on')); } finally { await framework.stop(); } @@ -2698,7 +2629,7 @@ describe('explicit prose routing', () => { for (const [label, mode, left, expected] of [ ['hybrid: the hold keeps later unprefixed prose private', 'hybrid', { hybridHold: 'private', locus: 'world:commons' }, []], ['hybrid: the sticky destination', 'hybrid', { pin: 'chan-alpha', locus: 'world:commons' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], - ['explicit: the sticky destination, and the stream starts at the turn\'s destination', 'explicit', { pin: 'chan-alpha', destination: 'chan-alpha' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], + ['explicit: the sticky destination, and the stream streams where delivery publishes', 'explicit', { pin: 'chan-alpha' }, [{ text: 'carrying on', locus: 'chan-alpha' }]], ] as const) { it(`a re-inference on unstreamed tool results keeps the turn's routing: ${label}`, async () => { membrane.pushResponse(createMockResponse([{ type: 'text', text: 'carrying on' }] as ContentBlock[])); @@ -2721,10 +2652,9 @@ describe('explicit prose routing', () => { await within(framework.runUntilIdle(), 'the re-inference', 10000); assert.deepEqual(routed.map((r) => ({ text: r.text.trim(), locus: r.locus })), expected); assert.equal(context(framework).bounces, 0, 'nothing bounced'); - if (mode === 'explicit') { - assert.deepEqual(chunks.map((c) => c.channelId), chunks.map(() => 'chan-alpha')); - assert.equal(chunks.map((c) => c.delta).join(''), 'carrying on'); - } + // In every mode, the re-inference streams what it published, where it + // published it. + assert.deepEqual(chunks.map((c) => ({ text: c.delta, locus: c.channelId })), expected); } finally { await framework.stop(); } @@ -2748,7 +2678,7 @@ describe('explicit prose routing', () => { const cm = framework.getAgent('assistant')!.getContextManager(); const getMessage = cm.getMessage.bind(cm); try { - const earlier = routingState({ pin: 'chan-alpha', destination: 'chan-alpha' }); + const earlier = routingState({ pin: 'chan-alpha' }); internals.turnRouting.set('assistant', earlier); internals.openLogical.set('assistant', earlier); // Someone addresses the agent from another channel while the turn waits. @@ -2803,12 +2733,10 @@ describe('explicit prose routing', () => { if (text.trim() === 'zero') await held; }); let starts = 0; - let retryChunks = 0; framework.onTrace((event) => { if (event.type !== 'inference:started') return; starts++; if (starts === 2) { - retryChunks = chunks.length; membrane.pushResponse(createMockResponse([{ type: 'text', text: 'carrying on' }])); } if (starts >= 3) membrane.pushResponse(createMockResponse([{ type: 'text', text: `${p}skip_reply` }])); @@ -2845,7 +2773,6 @@ describe('explicit prose routing', () => { const routing = internals.turnRouting.get('assistant')!; if (mode === 'explicit') { assert.equal(routing.pin, 'discord:g:cafe', "round two's delivery hasn't run yet"); - assert.equal(routing.destination, 'chan-alpha', 'though round two set the destination as it ended'); } else { assert.equal(routing.hybridHold, undefined, "round two's delivery hasn't run yet"); } @@ -2858,11 +2785,16 @@ describe('explicit prose routing', () => { { text: 'first', locus: 'chan-alpha' }, { text: 'carrying on', locus: 'chan-alpha' }, ], "the replaced stream's posts go first, and the retry's text follows the destination they set"); - assert.deepEqual(chunks.slice(retryChunks).map((c) => c.channelId), chunks.slice(retryChunks).map(() => 'chan-alpha'), - "the retry's stream starts at the destination round two set"); + assert.deepEqual(chunks.map((c) => ({ text: c.delta.trim(), locus: c.channelId })), [ + { text: 'zero', locus: 'discord:g:cafe' }, + { text: 'first', locus: 'chan-alpha' }, + { text: 'carrying on', locus: 'chan-alpha' }, + ], "every chunk follows its publish: the replaced stream's late posts, then the retry's text at the destination they set"); } else { assert.deepEqual(posts, [{ text: 'zero', locus: 'discord:g:cafe' }], "the retry's unprefixed text stays private under the hold the late delivery set"); + assert.deepEqual(chunks.map((c) => ({ text: c.delta.trim(), locus: c.channelId })), [{ text: 'zero', locus: 'discord:g:cafe' }], + "only the late post streams: the retry's held text doesn't"); const receipt = context(framework).texts.filter((t) => t.startsWith('[delivered]')); assert.equal(receipt.length, 1, `one receipt for the logical turn: ${receipt.join(' | ')}`); assert.match(receipt[0]!, /discord:g:cafe/, "it reports the replaced stream's late post"); diff --git a/test/outgoing-stream.test.ts b/test/outgoing-stream.test.ts new file mode 100644 index 000000000..3d94d2e63 --- /dev/null +++ b/test/outgoing-stream.test.ts @@ -0,0 +1,747 @@ +/** + * The outgoing stream (MCPL Spec 14.3 `channels/outgoing/chunk` and + * `complete`, mirrored as the `inference:published` traces) carries what + * delivery published, sent once each publish succeeds. Text that + * delivery withholds never reaches it: a round that a silencing tool, + * `skip_reply` or a private `think` keeps back, text under a hybrid hold, a + * bounce, or a round the stream never finished. Nor does a publish that + * throws, that the server reports undelivered, or that it doesn't confirm. + * A voice server speaks what it gets, and words already spoken can't be + * taken back, so the stream can't run ahead of delivery. + */ + +import { afterEach, test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, Membrane, NormalizedRequest, NormalizedResponse, StreamEvent, YieldingStreamOptions } from '@animalabs/membrane'; +import { AgentFramework, type AgentConfig, type Module, type ModuleContext, type ToolCall, type ToolResult, + type ProcessEvent, type ProcessState } from '../src/index.js'; +import { MockYieldingStream, createMockResponse } from './helpers/mock-membrane.js'; +import { ChannelRegistry } from '../src/mcpl/channel-registry.js'; +import { ALL_CAPABILITY_PATHS, CapabilityGrant } from '../src/mcpl/capability-grant.js'; + +const dirs: string[] = []; +afterEach(() => { while (dirs.length) rmSync(dirs.pop()!, { recursive: true, force: true }); }); + +const text = (value: string): ContentBlock => ({ type: 'text', text: value }); +const call = (id: string, name: string, input: Record = {}): ContentBlock => + ({ type: 'tool_use', id, name, input }) as ContentBlock; +/** A round that ends in tool calls. */ +const round = (...blocks: ContentBlock[]) => createMockResponse(blocks, 'tool_use'); +/** A round that ends the stream. */ +const final = (...blocks: ContentBlock[]) => createMockResponse(blocks); + +/** Each stream the framework opens takes the next list of responses. */ +class ScriptMembrane { + streams = 0; + /** Wraps a stream's events, by its 1-based number. */ + wrap?: (index: number, events: AsyncIterable) => AsyncIterable; + constructor(readonly scripts: NormalizedResponse[][]) {} + streamYielding(_request: NormalizedRequest, _options: YieldingStreamOptions = {}) { + const script = this.scripts.shift(); + assert.ok(script, 'unexpected extra inference'); + const stream = new MockYieldingStream(script); + const index = ++this.streams; + const wrap = this.wrap; + if (wrap) { + const iterate = stream[Symbol.asyncIterator].bind(stream); + (stream as unknown as { [Symbol.asyncIterator]: () => AsyncIterator })[Symbol.asyncIterator] = + () => wrap(index, { [Symbol.asyncIterator]: iterate })[Symbol.asyncIterator](); + } + return stream; + } + asMembrane() { return this as unknown as Membrane; } +} + +class ToolModule implements Module { + readonly name = 'test'; + /** Runs inside a tool call, before its result: lets a test act mid-round. */ + during?: (call: ToolCall) => Promise; + async start(_ctx: ModuleContext) {} + async stop() {} + getTools() { + return [ + { name: 'read', description: 'Read', inputSchema: { type: 'object' as const, properties: {} } }, + { name: 'send_message', description: 'Explicit send', inputSchema: { type: 'object' as const, properties: {} } }, + ]; + } + async handleToolCall(call: ToolCall): Promise { + await this.during?.(call); + return { success: true, data: `payload-${call.id}` }; + } + /** Called with each external message's text as it is processed. */ + processed?: (content: string) => void; + async onProcess(event: ProcessEvent, _state: ProcessState) { + if (event.type === 'external-message') this.processed?.(String(event.content)); + return event.type === 'external-message' + ? { + addMessages: [{ + participant: 'user', + content: [{ type: 'text' as const, text: String(event.content) }], + ...((event as { metadata?: Record }).metadata ? { metadata: (event as { metadata?: Record }).metadata } : {}), + }], + requestInference: String(event.content) === 'go', + } + : {}; + } +} + +type Logged = + | { kind: 'chunk'; channelId: string; text: string; inferenceId: string; index: number } + | { kind: 'publish'; channelId: string; text: string } + | { kind: 'complete'; channelId: string; text: string; inferenceId: string } + | { kind: 'mark'; channelId: string; text: string }; + +/** + * The registry stub: #alpha, #cafe and @laria resolve, the turn's locus is + * world:test, and every chunk, publish and complete lands in one ordered + * log. A publish whose text is in `fails` fails. + */ +function registry(framework: AgentFramework, options: { streaming?: boolean } = {}) { + const log: Logged[] = []; + /** A publish whose text is held here doesn't return until the test lets it go. */ + const holds = new Map>(); + /** + * A publish of one of these texts fails: the server reports it undelivered + * (`delivered: false`); its reply confirms nothing, so it may or may not + * have been posted (`unknown`, as the real registry reads a reply that + * leaves `delivered` out, an error answer or a timeout); or it throws. + */ + const fails = new Map(); + /** A publish of one of these texts is answered with this message id, as a server that returns one. */ + const messageIds = new Map(); + const known: Record = { '#alpha': 'chan-alpha', '#cafe': 'chan-cafe', '@laria': 'discord:dm:99' }; + let stopped = false; + /** One post, with the outcome the real registry's deliverSpeech gives it. */ + const publish = async (body: string, channelId: string | null) => { + if (!channelId) return { status: 'failed' as const, reason: 'no channel', at: Date.now() }; + log.push({ kind: 'publish', channelId, text: body }); + await holds.get(body); + const failure = fails.get(body); + if (failure === 'throw') throw new Error('synthetic publish failure'); + const destination = { serverId: 'stub', channelId }; + const messageId = messageIds.get(body); + if (failure === 'unknown') { + return { status: 'unknown' as const, destination, reason: 'the stub returned no delivery receipt (delivery uncertain)', at: Date.now() }; + } + return failure === undefined + ? { status: 'delivered' as const, destination, ...(messageId !== undefined ? { messageId } : {}), at: Date.now() } + : { status: 'failed' as const, destination, reason: 'the stub reported delivered: false', at: Date.now() }; + }; + const target: Record = { + resolveLocus: () => 'world:test', + resolveProseTarget: (spec: string) => (known[spec] ? { channelId: known[spec] } : { error: `no channel ${spec}` }), + resolveDestination: ({ channelId }: { channelId: string }) => ({ destination: { serverId: 'stub', channelId } }), + // Locus prose posts through routeSpeech, a named or sticky destination + // through deliverSpeech, as with the real registry. + deliverSpeech: async (_agent: string, body: string, channelId: string | null) => publish(body, channelId), + routeSpeech: async (_agent: string, body: string, channelId: string | null) => { + const outcome = await publish(body, channelId); + return outcome.status === 'delivered' + ? { delivered: true, channelId: outcome.destination.channelId, ...(outcome.messageId !== undefined ? { messageId: outcome.messageId } : {}) } + : null; + }, + sendOutgoingChunk: (channelId: string, _agent: string, inferenceId: string, index: number, delta: string) => { + if (stopped) return; + log.push({ kind: 'chunk', channelId, text: delta, inferenceId, index }); + }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, body: string) => { + if (stopped) return; + log.push({ kind: 'complete', channelId, text: body, inferenceId }); + }, + // As the real registry's stopAll forgets its channels, a chunk or + // complete sent after it finds no server and goes nowhere. + stopAll: () => { + stopped = true; + log.push({ kind: 'mark', channelId: '', text: 'registry stopped' }); + }, + // skip_reply ends the turn, as the real registry's does; think is noted. + handleChannelToolCall: async (name: string) => (name === 'skip_reply' + ? { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } } + : { success: true, data: { noted: true } }), + getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, + getChannelTools: () => [], + }; + // A registry whose server doesn't take the stream sends no chunk or complete. + if (options.streaming === false) { + delete target.sendOutgoingChunk; + delete target.sendOutgoingComplete; + } + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(target, { + get: (t, key: string) => (key in t ? t[key] : () => undefined), + }); + return { + log, + holds, + fails, + messageIds, + chunks: () => log.filter((e): e is Extract => e.kind === 'chunk'), + publishes: () => log.filter((e) => e.kind === 'publish').map((e) => ({ channelId: e.channelId, text: e.text })), + completes: () => log.filter((e): e is Extract => e.kind === 'complete'), + }; +} + +async function harness(scripts: NormalizedResponse[][], config: Partial = {}, options: { streaming?: boolean; gate?: boolean } = {}) { + const dir = mkdtempSync(join(tmpdir(), 'af-outgoing-stream-')); dirs.push(dir); + const membrane = new ScriptMembrane(scripts); + const module = new ToolModule(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test', systemPrompt: 'system', ...config }], modules: [module], syncIntervalMs: 0, + // An EventGate gives the agent the sleep tool. + ...(options.gate ? { gate: { configPath: join(dir, 'gate.json') } } : {}), + }); + const stub = registry(framework, options); + const traces: Array<{ type: string; channelId: string; text: string; inferenceId: string; index?: number; messageId?: string }> = []; + framework.onTrace((event) => { + if (event.type === 'inference:published' || event.type === 'inference:published_complete') { + const { type, channelId, text: body, inferenceId } = event; + traces.push({ + type, channelId, text: body, inferenceId, + ...(event.type === 'inference:published' ? { index: event.index } : {}), + // Kept whenever the key is there, so a trace that carries it as undefined shows. + ...('messageId' in event ? { messageId: event.messageId } : {}), + }); + } + }); + /** + * Once the agent's queued deliveries have run. A stream's `complete` is + * chained on its deliveries in its teardown, ahead of this wait. + */ + const settled = async () => { + await (framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + }; + const run = async () => { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + await framework.runUntilIdle(); + await settled(); + }; + return { framework, membrane, module, run, settled, traces, ...stub }; +} + +test('a delivered round streams its text once its publish succeeds, and the published traces mirror the stream', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + try { + await h.run(); + const [id] = new Set(h.chunks().map((c) => c.inferenceId)); + assert.deepEqual(h.log, [ + { kind: 'publish', channelId: 'world:test', text: 'first' }, + { kind: 'chunk', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, + { kind: 'publish', channelId: 'world:test', text: 'second' }, + { kind: 'chunk', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, + { kind: 'complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, + ], 'each publish is streamed once it succeeds, and a later one to the same channel opens with a paragraph break'); + assert.deepEqual(h.traces, [ + { type: 'inference:published', channelId: 'world:test', text: 'first', inferenceId: id, index: 0 }, + { type: 'inference:published', channelId: 'world:test', text: '\n\nsecond', inferenceId: id, index: 1 }, + { type: 'inference:published_complete', channelId: 'world:test', text: 'first\n\nsecond', inferenceId: id }, + ]); + } finally { await h.framework.stop(); } +}); + +for (const mode of ['locus', 'hybrid', 'explicit'] as const) { + const aside = mode === 'explicit' ? '>>#alpha an aside I decided against' : 'an aside I decided against'; + + test(`${mode}: a round that calls skip_reply streams nothing`, async () => { + const h = await harness([[round(text(aside), call('skip', 'skip_reply'))]], { proseRouting: mode }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [], 'delivery keeps the round'); + assert.deepEqual(h.log, [], 'and the stream gets none of it'); + assert.deepEqual(h.traces, []); + } finally { await h.framework.stop(); } + }); + + test(`${mode}: a round beside a private think streams nothing, and the next round streams as it is published`, async () => { + const answer = mode === 'explicit' ? '>>#alpha the answer' : 'the answer'; + const h = await harness([[round(text(aside), call('plan', 'think', { content: 'planning' })), final(text(answer))]], + { proseRouting: mode, sameRoundThinkTextPolicy: 'private' }); + try { + await h.run(); + const where = mode === 'explicit' ? 'chan-alpha' : 'world:test'; + assert.deepEqual(h.publishes(), [{ channelId: where, text: 'the answer' }]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: where, text: 'the answer' }]); + } finally { await h.framework.stop(); } + }); +} + +for (const mode of ['locus', 'hybrid'] as const) { + test(`${mode}: a round with a send streams nothing, and neither does the turn's later prose`, async () => { + const h = await harness([[round(text('sending it directly'), call('send', 'test--send_message')), final(text('and done'))]], + { proseRouting: mode }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [], 'the send silences the turn from its round on'); + assert.deepEqual(h.log, [], 'and the stream carries none of it'); + } finally { await h.framework.stop(); } + }); +} + +test('explicit: a round starts where the sticky destination is, and its >> lines move it, as delivery does', async () => { + const h = await harness([[round(text('>>#alpha first'), call('one', 'test--read')), final(text('middle\n>>#cafe later'))]], + { proseRouting: 'explicit' }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'middle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: '\n\nmiddle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + assert.deepEqual(h.completes().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first\n\nmiddle' }, + { channelId: 'chan-cafe', text: 'later' }, + ]); + } finally { await h.framework.stop(); } +}); + +test('explicit: a {{unsent}} resend streams the retained text it publishes', async () => { + const original = 'the reply I meant for laria'; + // The unprefixed reply bounces and wakes a second turn, which resends it. + const h = await harness([[final(text(original))], [final(text('>>@laria {{unsent}}'))]], { proseRouting: 'explicit' }); + try { + await h.run(); + assert.deepEqual(h.publishes(), [{ channelId: 'discord:dm:99', text: original }]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'discord:dm:99', text: original }], + 'the stream carries the substituted text, not the token'); + } finally { await h.framework.stop(); } +}); + +test('hybrid: a publish that fails streams nothing, and the {{unsent}} resend of its retained text streams once', async () => { + const reply = 'the reply I meant for alpha'; + // The >>> post fails, so delivery retains its text and wakes a second turn, which resends it. + const h = await harness([[final(text(`>>>#alpha ${reply}`))], [final(text('>>>#alpha {{unsent}}'))]], { proseRouting: 'hybrid' }); + h.fails.set(reply, 'undelivered'); + // The failure applies to the first post alone, so the resend succeeds. + const failureOf = h.fails.get.bind(h.fails); + h.fails.get = (body: string) => { const failure = failureOf(body); h.fails.delete(body); return failure; }; + try { + await h.run(); + assert.deepEqual(h.publishes(), [{ channelId: 'chan-alpha', text: reply }, { channelId: 'chan-alpha', text: reply }], + 'the post failed, and the resend posted the retained text'); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'chan-alpha', text: reply }], + 'only the resend streams, once, so a voice server speaks the reply once'); + } finally { await h.framework.stop(); } +}); + +/** Two streams of one turn: the first round crosses the stream budget, so the turn restarts. */ +function budgetRestart(first: string, continued: string) { + const overBudget = round(text(first), call('one', 'test--read')); + overBudget.usage = { inputTokens: 100, outputTokens: 5 }; + return [[overBudget], [final(text(continued))]]; +} + +test("hybrid: a resumed stream's first round stays held under the hold an earlier stream left", async () => { + const h = await harness(budgetRestart('>>>skip_reply an aside', 'carrying on'), { proseRouting: 'hybrid', maxStreamTokens: 20 }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the turn restarted for its budget'); + assert.deepEqual(h.publishes(), [], 'delivery keeps the hold across the restart'); + assert.deepEqual(h.log, [], 'and the resumed stream sends nothing to the locus'); + } finally { await h.framework.stop(); } +}); + +test("hybrid: a resumed stream's first round streams to the pin an earlier stream left, and each stream completes", async () => { + const h = await harness(budgetRestart('>>>#alpha first', 'carrying on'), { proseRouting: 'hybrid', maxStreamTokens: 20 }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the turn restarted for its budget'); + assert.deepEqual(h.publishes(), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'carrying on' }, + ]); + const chunks = h.chunks(); + assert.deepEqual(chunks.map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'carrying on' }, + ], 'the resumed stream follows the pin, not the locus'); + assert.notEqual(chunks[0]!.inferenceId, chunks[1]!.inferenceId, 'each stream has its own id'); + assert.deepEqual(h.completes().map((c) => ({ text: c.text, inferenceId: c.inferenceId })), [ + { text: 'first', inferenceId: chunks[0]!.inferenceId }, + { text: 'carrying on', inferenceId: chunks[1]!.inferenceId }, + ], 'the stream that handed its turn to the restart completes too'); + } finally { await h.framework.stop(); } +}); + +test('a stream that fails partway streams none of its unfinished round, and its retry streams what it delivers', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('half a thought'))], [final(text('carrying on'))]]); + // The first stream errors where its last round would complete. + h.membrane.wrap = (index, events) => ({ + async *[Symbol.asyncIterator]() { + for await (const event of events) { + if (index === 1 && event.type === 'complete') { + yield { type: 'error', error: new Error('synthetic transient stream error') } as StreamEvent; + return; + } + yield event; + } + }, + }); + try { + await h.run(); + assert.equal(h.membrane.streams, 2, 'the error was retried'); + assert.deepEqual(h.publishes().map((p) => p.text), ['first', 'carrying on']); + assert.equal(h.chunks().some((c) => c.text.includes('half a thought')), false, 'the unfinished round never streams'); + assert.deepEqual(h.completes().map((c) => c.text), ['first', 'carrying on'], 'each stream completes with what it delivered'); + } finally { await h.framework.stop(); } +}); + +test('locus: after a mid-turn addressed re-pin, the next round streams to the new locus, where it is published', async () => { + const h = await harness([[round(text('station four, proceeding'), call('one', 'test--read')), final(text('answering you'))]]); + // While the tool runs, someone addresses the agent from a DM. The tool + // returns once the message has been processed, so the round's boundary + // weighs it. + h.module.during = async () => { + const seen = new Promise((resolve) => { + h.module.processed = (content) => { if (content === 'quick question over here?') resolve(); }; + }); + h.framework.pushEvent({ + type: 'external-message', source: 'test', content: 'quick question over here?', + metadata: { channelId: 'discord:dm:antra', tags: ['chat:addressed'] }, + } as unknown as ProcessEvent); + await seen; + }; + try { + await h.run(); + assert.deepEqual(h.publishes(), [ + { channelId: 'world:test', text: 'station four, proceeding' }, + { channelId: 'discord:dm:antra', text: 'answering you' }, + ]); + assert.deepEqual(h.chunks().map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'world:test', text: 'station four, proceeding' }, + { channelId: 'discord:dm:antra', text: 'answering you' }, + ], 'the stream follows the re-pin'); + } finally { await h.framework.stop(); } +}); + +test("explicit: a stream's complete waits for the deliveries it queued, so no chunk follows it", async () => { + // skip_reply ends the turn while round one's post is still out: explicit + // mode doesn't hold the turn for its queued deliveries. + const h = await harness([[round(text('>>#alpha first'), call('one', 'test--read')), round(text('an aside'), call('skip', 'skip_reply'))]], + { proseRouting: 'explicit' }); + let release!: () => void; + h.holds.set('first', new Promise((resolve) => { release = resolve; })); + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + // The turn has ended and its stream torn down, with 'first' still out. + await h.framework.runUntilIdle(); + h.log.push({ kind: 'mark', channelId: '', text: 'post released' }); + release(); + await h.settled(); + assert.deepEqual(h.log.map((e) => `${e.kind}:${e.text}`), [ + 'publish:first', + 'mark:post released', + 'chunk:first', + 'complete:first', + ], 'the chunk follows the post once it succeeds, the complete follows the chunk, and the private round streams nothing'); + } finally { + release(); + await h.framework.stop(); + } +}); + +/** How a test title says a publish failed. */ +const failedHow = (failure: 'undelivered' | 'unknown' | 'throw'): string => + failure === 'throw' ? 'throws' : failure === 'unknown' ? "the server doesn't confirm" : 'the server reports undelivered'; + +for (const failure of ['undelivered', 'unknown', 'throw'] as const) { + test(`a publish that ${failedHow(failure)} never streams, and the stream goes on with what is`, async () => { + const h = await harness([[round(text('lost'), call('one', 'test--read')), final(text('kept'))]]); + h.fails.set('lost', failure); + try { + await h.run(); + assert.deepEqual(h.publishes().map((p) => p.text), ['lost', 'kept'], 'both publishes were tried'); + assert.deepEqual(h.chunks().map((c) => c.text), ['kept'], 'only the one that succeeded streams'); + assert.deepEqual(h.completes().map((c) => c.text), ['kept'], 'and the complete carries only it'); + assert.equal(h.traces.some((t) => t.text.includes('lost')), false, 'nor do the published traces'); + } finally { await h.framework.stop(); } + }); +} + +test('a stream stopped mid-turn still completes what it published', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('never written'))]]); + // The tool doesn't return until the test is done with it. + let release!: () => void; + const held = new Promise((resolve) => { release = resolve; }); + let published!: () => void; + const firstOut = new Promise((resolve) => { published = resolve; }); + h.module.during = async () => { await held; }; + const unlog = h.framework.onTrace((event) => { if (event.type === 'inference:published') published(); }); + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + let timer: ReturnType | undefined; + await Promise.race([firstOut, new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error('the first round never streamed')), 10000); + })]).finally(() => clearTimeout(timer)); + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.completes().map((c) => ({ channelId: c.channelId, text: c.text })), [{ channelId: 'world:test', text: 'first' }], + 'the stopped stream ends its utterance with what it carried'); + } finally { + unlog?.(); + release(); + await h.framework.stop(); + } +}); + +// A stop() that finds a publish still out ends the stream before its teardown +// closes the connections the publish waits on, whatever becomes of the +// publish then: the close rejects it, or it succeeds too late to stream. +for (const late of ['rejects', 'succeeds'] as const) { + test(`a stop() with a publish still out completes what the stream sent before teardown, and a publish that ${late === 'rejects' ? 'teardown rejects' : 'succeeds after teardown'} never streams`, async () => { + const h = await harness([[ + round(text('first'), call('one', 'test--read')), + round(text('second'), call('two', 'test--read')), + final(text('never written')), + ]]); + // 'second' is still out when stop() begins; teardown settles it, as + // closing an MCPL connection settles a request it still holds. + let settleSecond!: () => void; + const secondHeld = new Promise((resolve, reject) => { + settleSecond = () => (late === 'rejects' ? reject(new Error('connection closed')) : resolve()); + }); + secondHeld.catch(() => {}); + h.holds.set('second', secondHeld); + // The second round's tool doesn't return until the test is done with it. + let releaseTwo!: () => void; + const twoHeld = new Promise((resolve) => { releaseTwo = resolve; }); + h.module.during = async (c) => { if (c.id === 'two') await twoHeld; }; + h.module.stop = async () => { + h.log.push({ kind: 'mark', channelId: '', text: 'teardown' }); + settleSecond(); + }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !h.log.some((e) => e.kind === 'publish' && e.text === 'second');) { + assert.ok(Date.now() < end, 'the second publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.log.filter((e) => e.kind !== 'publish').map((e) => `${e.kind}:${e.text}`), + ['chunk:first', 'complete:first', 'mark:registry stopped', 'mark:teardown'], + 'the stream completes before the registry forgets its channels and the teardown, with what it sent, and nothing streams after its complete'); + assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), ['inference:published:first', 'inference:published_complete:first'], + 'and the published traces say the same'); + } finally { + releaseTwo(); + await h.framework.stop(); + } + }); +} + +test('a stop() completes a stream the framework no longer tracks, as a disposed ephemeral\'s, before the registry forgets its channels', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + // The turn's last publish is still out when stop() begins; teardown rejects + // it, as closing an MCPL connection rejects a request it still holds. + let rejectSecond!: () => void; + const secondHeld = new Promise((_, reject) => { rejectSecond = () => reject(new Error('connection closed')); }); + secondHeld.catch(() => {}); + h.holds.set('second', secondHeld); + h.module.stop = async () => { + h.log.push({ kind: 'mark', channelId: '', text: 'teardown' }); + rejectSecond(); + }; + try { + h.framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void h.framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !h.log.some((e) => e.kind === 'publish' && e.text === 'second');) { + assert.ok(Date.now() < end, 'the last publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + // The stream, still awaiting that publish, is no longer tracked, as when + // an idle ephemeral is disposed, so stop() doesn't wait for it. + (h.framework as unknown as { activeStreams: Map }).activeStreams.delete('assistant'); + await h.framework.stop(); + await h.settled(); + assert.deepEqual(h.log.filter((e) => e.kind !== 'publish').map((e) => `${e.kind}:${e.text}`), + ['chunk:first', 'complete:first', 'mark:registry stopped', 'mark:teardown'], + 'the stream completes with what it sent before the registry forgets its channels, and nothing streams after'); + } finally { + await h.framework.stop(); + } +}); + +test("with the real channel registry, a stop() with a publish still out sends the stream's complete to the server before the registry forgets its channels", async () => { + // The stop tests above use the harness's stub registry. Here the publishes, + // the chunks, the complete and stopAll go through a real ChannelRegistry + // and a server it owns, so the registry's own teardown keeps the order honest. + const dir = mkdtempSync(join(tmpdir(), 'af-outgoing-real-')); dirs.push(dir); + const membrane = new ScriptMembrane([[ + round(text('first'), call('one', 'test--read')), + round(text('second'), call('two', 'test--read')), + final(text('never written')), + ]]); + const module = new ToolModule(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test', systemPrompt: 'system' }], modules: [module], syncIntervalMs: 0, + }); + // What reaches the server, with the registry's stopAll and the teardown marked. + const wire: string[] = []; + // 'second' is still out when stop() begins; teardown rejects it, as closing + // an MCPL connection rejects a request it still holds. + let rejectSecond!: () => void; + const secondHeld = new Promise((_, reject) => { rejectSecond = () => reject(new Error('connection closed')); }); + secondHeld.catch(() => {}); + const server = { + grant: new CapabilityGrant(new Set(ALL_CAPABILITY_PATHS), []), + sendChannelsPublish: async (params: { content: Array<{ text: string }> }) => { + const body = params.content[0]!.text; + wire.push(`publish:${body}`); + if (body === 'second') await secondHeld; + return { delivered: true }; + }, + sendChannelsOutgoingChunk: (params: { delta: string }) => { wire.push(`chunk:${params.delta}`); }, + sendChannelsOutgoingComplete: (params: { content: Array<{ text: string }> }) => { wire.push(`complete:${params.content[0]!.text}`); }, + sendChannelsTyping: () => {}, + }; + const registry = new ChannelRegistry({ getServer: () => server } as never, {} as never, () => {}, () => {}, {}); + (registry as unknown as { channels: Map }).channels.set('srv:world:test', + { serverId: 'srv', descriptor: { id: 'world:test', type: 'srv', label: 'world:test' }, open: true }); + const stopAll = registry.stopAll.bind(registry); + (registry as unknown as { stopAll: () => void }).stopAll = () => { wire.push('registry stopped'); stopAll(); }; + // The turn's locus and channel tools, as the harness gives them. + (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(registry, { + get: (target, key: string) => { + if (key === 'resolveLocus') return () => 'world:test'; + if (key === 'handleChannelToolCall') return async () => ({ success: true, data: {} }); + if (key === 'getChannelTools') return () => []; + const value = (target as unknown as Record)[key]; + return typeof value === 'function' ? (value as (...args: unknown[]) => unknown).bind(target) : value; + }, + }); + // The second round's tool doesn't return until the test is done with it. + let releaseTwo!: () => void; + const twoHeld = new Promise((resolve) => { releaseTwo = resolve; }); + module.during = async (c) => { if (c.id === 'two') await twoHeld; }; + module.stop = async () => { wire.push('teardown'); rejectSecond(); }; + try { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} }); + void framework.runUntilIdle().catch(() => {}); + for (const end = Date.now() + 10000; !wire.includes('publish:second');) { + assert.ok(Date.now() < end, 'the second publish went out'); + await new Promise((resolve) => setTimeout(resolve, 10)); + } + await framework.stop(); + await (framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); + assert.deepEqual(wire, ['publish:first', 'chunk:first', 'publish:second', 'complete:first', 'registry stopped', 'teardown'], + "the server gets the stream's complete before the registry forgets its channels, and nothing streams after it"); + } finally { + releaseTwo(); + await framework.stop(); + } +}); + +test('disabled: nothing streams, from a mid-turn round, trailing text or a text-only turn', async () => { + for (const [script, branches] of [ + [[round(text('mid-turn'), call('one', 'test--read')), final(text('trailing'))], ['mid-turn prose', 'tool-call trailing prose']], + [[final(text('text only'))], ['text-only prose']], + ] as const) { + const h = await harness([[...script]], { proseRouting: 'disabled' }); + const called: string[] = []; + h.module.during = async (call) => { called.push(call.name); }; + // Each branch the turn reaches says it routed nothing; the test checks it reached them. + const logged: string[] = []; + const error = console.error; + try { + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + try { await h.run(); } finally { console.error = error; } + for (const branch of branches) { + assert.ok(logged.includes(`[routing] assistant: ${branch} NOT routed (proseRouting=disabled)`), `the ${branch} branch ran`); + } + assert.deepEqual(called, script.length > 1 ? ['read'] : [], "the tool round's call was made"); + assert.deepEqual(h.log, [], 'nothing is published, streamed or completed'); + assert.deepEqual(h.traces, [], 'and no published trace is emitted'); + } finally { await h.framework.stop(); } + } +}); + +test("the sleep tool's announcement publishes through routeSpeech, as prose does, but isn't part of the stream", async () => { + const h = await harness([[round(text('going quiet now'), call('s1', 'sleep', { seconds: 600 }))]], {}, { gate: true }); + try { + await h.run(); + assert.ok(h.publishes().some((p) => p.channelId === 'world:test' && p.text.startsWith('\u{1F4A4}')), 'the sleep tool announced at the locus'); + assert.deepEqual(h.chunks().map((c) => c.text), ['going quiet now'], "the round's prose streams, and the announcement doesn't"); + } finally { + // EventGate.dispose leaves a pending sleep's timer running, which would + // keep this file alive for the whole sleep, so the test ends the sleep. + (h.framework as unknown as { eventGate: { clearSleep(): boolean } }).eventGate.clearSleep(); + await h.framework.stop(); + } +}); + +test("the published traces are emitted when the channel's server takes no stream", async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]], {}, { streaming: false }); + try { + await h.run(); + assert.deepEqual(h.chunks(), [], 'the server is sent no chunk'); + assert.deepEqual(h.traces.map((t) => `${t.type}:${t.text}`), [ + 'inference:published:first', + 'inference:published:\n\nsecond', + 'inference:published_complete:first\n\nsecond', + ], 'in-process consumers still hear what was published'); + } finally { await h.framework.stop(); } +}); + +// A publish that fails streams nothing, on every delivery path. +const failedPublishCases: Array<{ name: string; mode: 'locus' | 'hybrid' | 'explicit'; script: () => NormalizedResponse[]; lost: string; kept?: string }> = [ + { name: 'locus text-only', mode: 'locus', script: () => [final(text('lost'))], lost: 'lost' }, + { name: 'locus trailing', mode: 'locus', script: () => [round(text('kept'), call('one', 'test--read')), final(text('lost'))], lost: 'lost', kept: 'kept' }, + { name: 'hybrid unprefixed', mode: 'hybrid', script: () => [round(text('lost'), call('one', 'test--read')), final(text('kept'))], lost: 'lost', kept: 'kept' }, + { name: 'hybrid >>> target', mode: 'hybrid', script: () => [final(text('>>>#alpha lost'))], lost: 'lost' }, + { name: 'explicit mid-turn', mode: 'explicit', script: () => [round(text('>>#alpha lost'), call('one', 'test--read')), final(text('>>#cafe kept'))], lost: 'lost', kept: 'kept' }, + { name: 'explicit text-only', mode: 'explicit', script: () => [final(text('>>#alpha lost'))], lost: 'lost' }, +]; +// The id a server returns for a publish rides on that publish's trace, on every delivery path. +for (const c of failedPublishCases) { + test(`${c.name}: each published trace carries the message id its server returned for that publish`, async () => { + const h = await harness([c.script()], { proseRouting: c.mode }); + const ids = new Map([[c.lost, 'msg-1'], ...(c.kept ? [[c.kept, 'msg-2'] as [string, string]] : [])]); + for (const [body, id] of ids) h.messageIds.set(body, id); + try { + await h.run(); + const published = h.traces.filter((t) => t.type === 'inference:published'); + const bodyOf = (t: { text: string }) => t.text.replace(/^\n\n/, ''); + assert.deepEqual(published.map(bodyOf).sort(), [...ids.keys()].sort(), 'each publish streamed once'); + for (const t of published) assert.equal(t.messageId, ids.get(bodyOf(t)), `"${bodyOf(t)}" carries its own publish's id`); + } finally { await h.framework.stop(); } + }); +} + +test('a published trace carries no message id when the server returned none', async () => { + const h = await harness([[round(text('first'), call('one', 'test--read')), final(text('second'))]]); + try { + await h.run(); + const published = h.traces.filter((t) => t.type === 'inference:published'); + assert.equal(published.length, 2, 'both publishes streamed'); + assert.ok(published.every((t) => !('messageId' in t)), 'no id came back, so the traces carry none'); + } finally { await h.framework.stop(); } +}); + +for (const c of failedPublishCases) { + for (const failure of ['undelivered', 'unknown', 'throw'] as const) { + test(`${c.name}: a publish that ${failedHow(failure)} streams nothing`, async () => { + const h = await harness([c.script()], { proseRouting: c.mode }); + h.fails.set(c.lost, failure); + try { + await h.run(); + assert.ok(h.publishes().some((p) => p.text === c.lost), 'the publish was tried'); + assert.equal(h.chunks().some((ch) => ch.text.includes(c.lost)), false, 'no chunk carries it'); + assert.equal(h.completes().some((co) => co.text.includes(c.lost)), false, 'nor does a complete'); + assert.equal(h.traces.some((t) => t.text.includes(c.lost)), false, 'nor a published trace'); + if (c.kept) { + assert.deepEqual(h.chunks().map((ch) => ch.text), [c.kept], 'the publish beside it that succeeds still streams'); + } + } finally { await h.framework.stop(); } + }); + } +} diff --git a/test/present-while-acting.test.ts b/test/present-while-acting.test.ts index 7e4c6becb..10683cfa6 100644 --- a/test/present-while-acting.test.ts +++ b/test/present-while-acting.test.ts @@ -1091,7 +1091,7 @@ describe('present while acting', () => { // fallback's re-inference. Messages arrive before that starts. const open: Routing = { locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(['discord:guild:portables']), - destination: null, continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], + continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; internals.turnRouting.set('assistant', open); internals.openLogical.set('assistant', open); @@ -1120,7 +1120,7 @@ describe('present while acting', () => { }; const open: Routing = { locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), - destination: null, continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], + continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; internals.turnRouting.set('assistant', open); internals.openLogical.set('assistant', open); @@ -1271,7 +1271,7 @@ describe('present while acting', () => { recordTurnCheckpoint(agentName: string): void; }; const open: Routing = { - locus: 'chan-turn', engaged: new Set(), destination: null, + locus: 'chan-turn', engaged: new Set(), continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; internals.openLogical.set('assistant', open); @@ -1300,7 +1300,7 @@ describe('present while acting', () => { // live, and stop() ended every open turn. Its budget restart now runs, // and its stream fails and is retried. const turn: Routing = { - locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), destination: null, + locus: 'chan-turn', triggerChannel: 'chan-turn', engaged: new Set(), continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; internals.turnRouting.set('assistant', turn); @@ -1347,7 +1347,7 @@ describe('present while acting', () => { // Built by hand, as #312's silent-start test does: production runs no // silent turn while a prose turn is open. const open: Routing = { - locus: 'chan-turn', engaged: new Set(), destination: null, + locus: 'chan-turn', engaged: new Set(), continuation: false, deliveries: [], suppressed: 0, drafts: [], keptPrivate: 0, arrivals: [], }; internals.openLogical.set('assistant', open); diff --git a/test/prose-stream-router.test.ts b/test/prose-stream-router.test.ts deleted file mode 100644 index 8875ee237..000000000 --- a/test/prose-stream-router.test.ts +++ /dev/null @@ -1,197 +0,0 @@ -// ProseStreamRouter: streamed routing must agree with the delivery path's -// grammar (prose-grammar.parseProsePrefix + the envelope splitter) regardless -// of how the token stream is chopped. The last test is the property that -// matters most: char-by-char feeding ≡ one-shot feeding. -import { test } from 'node:test'; -import assert from 'node:assert/strict'; -import { ProseStreamRouter, type RoutedDelta } from '../src/mcpl/prose-stream-router.js'; - -const RESOLVE: Record = { - '#general': 'discord:g:100', - '#ops': 'discord:g:200', - '@sol': 'discord:dm:300', -}; - -function explicit(): ProseStreamRouter { - return new ProseStreamRouter({ - mode: 'explicit', - initialTarget: null, - resolve: (spec) => RESOLVE[spec] ?? null, - }); -} - -function run(r: ProseStreamRouter, text: string, chunk = text.length): RoutedDelta[] { - const out: RoutedDelta[] = []; - for (let i = 0; i < text.length; i += chunk) out.push(...r.feed(text.slice(i, i + chunk))); - out.push(...r.finish()); - return out; -} - -function textFor(deltas: RoutedDelta[], ch: string): string { - return deltas.filter((d) => d.channelId === ch).map((d) => d.delta).join(''); -} - -test('locus mode: pure passthrough, >> lines are plain text', () => { - const r = new ProseStreamRouter({ mode: 'locus', initialTarget: 'discord:g:100', resolve: () => null }); - const out = run(r, 'hello\n>>#ops not a prefix here\nbye'); - assert.equal(textFor(out, 'discord:g:100'), 'hello\n>>#ops not a prefix here\nbye'); -}); - -test('explicit: unprefixed leading text is suppressed (delivery would bounce it)', () => { - const out = run(explicit(), 'no destination yet\n>>#general now routed'); - assert.equal(textFor(out, 'discord:g:100'), 'now routed'); - assert.equal(out.every((d) => !d.delta.includes('no destination')), true); -}); - -test('explicit: multi-envelope switching, prefixes swallowed', () => { - const out = run(explicit(), '>>#general hello there\nstill general\n>>#ops aside for ops\n>>@sol dm line'); - assert.equal(textFor(out, 'discord:g:100'), 'hello there\nstill general\n'); - assert.equal(textFor(out, 'discord:g:200'), 'aside for ops\n'); - assert.equal(textFor(out, 'discord:dm:300'), 'dm line'); -}); - -test('quoted `>> arrow` and lone `>` stay body text', () => { - const out = run(explicit(), '>>#general look:\n>> quoted arrow\n> markdown quote\ndone'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n>> quoted arrow\n> markdown quote\ndone'); -}); - -test('skip_reply suppresses; later envelope resumes', () => { - const out = run(explicit(), '>>skip_reply private thought\n>>#ops public line'); - assert.equal(out.every((d) => !d.delta.includes('private')), true); - assert.equal(textFor(out, 'discord:g:200'), 'public line'); -}); - -test('unresolvable target suppresses that envelope only', () => { - const out = run(explicit(), '>>#nonexistent to nowhere\n>>#general to somewhere'); - assert.equal(out.every((d) => !d.delta.includes('nowhere')), true); - assert.equal(textFor(out, 'discord:g:100'), 'to somewhere'); -}); - -test('explicit: an indented prefix routes, as delivery reads it', () => { - const out = run(explicit(), ' >>#general indented reply\n\t>>#ops tabbed aside\n plain indented line'); - assert.equal(textFor(out, 'discord:g:100'), 'indented reply\n'); - assert.equal(textFor(out, 'discord:g:200'), 'tabbed aside\n plain indented line'); -}); - -test('explicit: indentation before a quoted arrow or a lone `>` stays body', () => { - const out = run(explicit(), '>>#general look:\n >> quoted arrow\n > quote'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n >> quoted arrow\n > quote'); -}); - -test('explicit: any leading whitespace before a prefix, as delivery trims it', () => { - const out = run(explicit(), '\u00a0>>#general nbsp\n\u2003>>#ops em space\n\r>>@sol after a CR'); - assert.equal(textFor(out, 'discord:g:100'), 'nbsp\n'); - assert.equal(textFor(out, 'discord:g:200'), 'em space\n'); - assert.equal(textFor(out, 'discord:dm:300'), 'after a CR'); -}); - -test('explicit: whitespace other than a space or tab ends the token and starts the body', () => { - const out = run(explicit(), '\t>>#ops\r\nhello'); - assert.equal(textFor(out, 'discord:g:200'), '\r\nhello'); -}); - -test('explicit: `>>` and then any whitespace is a quoted arrow', () => { - const out = run(explicit(), '>>#general look:\n>>\u00a0quoted'); - assert.equal(textFor(out, 'discord:g:100'), 'look:\n>>\u00a0quoted'); -}); - -test('explicit: a bare `>>` line is text, its newline included, as delivery sends it', () => { - const r = new ProseStreamRouter({ mode: 'explicit', initialTarget: 'discord:g:100', resolve: (spec) => RESOLVE[spec] ?? null }); - const out = [...r.feed('>>\nhello'), ...r.finish()]; - assert.equal(textFor(out, 'discord:g:100'), '>>\nhello'); -}); - -test('explicit: each segment starts a new line, and a prefix with no body takes effect at its end', () => { - // Delivery parses each round's text on its own: a round may begin with a - // prefix, and a round that is only a prefix sets the sticky destination. - const r = explicit(); - const out: RoutedDelta[] = []; - out.push(...r.feed('>>#general first'), ...r.endSegment()); - out.push(...r.feed('>>#ops second'), ...r.endSegment()); - out.push(...r.feed('>>@sol'), ...r.endSegment()); - out.push(...r.feed('to sol'), ...r.finish()); - assert.equal(textFor(out, 'discord:g:100'), 'first'); - assert.equal(textFor(out, 'discord:g:200'), 'second'); - assert.equal(textFor(out, 'discord:dm:300'), 'to sol'); -}); - -test('` !` continuation modifier is swallowed; `!` as body survives', () => { - const out = run(explicit(), '>>#general ! next turn continues\n>>#ops !bang body'); - assert.equal(textFor(out, 'discord:g:100'), 'next turn continues\n'); - // grammar: `([ \t]+!)?` — the first ! after whitespace is the modifier, - // remainder is body (matches parseProsePrefix on ">>#ops !bang body"). - assert.equal(textFor(out, 'discord:g:200'), 'bang body'); -}); - -test('prefix on its own line routes the following lines', () => { - const out = run(explicit(), '>>#general\nbody on next line\nand more'); - assert.equal(textFor(out, 'discord:g:100'), 'body on next line\nand more'); -}); - -test('byChannel accumulates exactly the emitted text', () => { - const r = explicit(); - run(r, '>>#general one\n>>#ops two'); - assert.equal(r.byChannel().get('discord:g:100'), 'one\n'); - assert.equal(r.byChannel().get('discord:g:200'), 'two'); -}); - -test('PROPERTY: char-by-char ≡ one-shot, across every chunk size', () => { - const text = 'lead-in ignored\n>>#general alpha\nbeta > gamma\n>> not a prefix\n>>#ops ! delta\n>>skip_reply hidden\n>>#general {{unsent}} tail'; - const oneShot = run(explicit(), text); - for (const size of [1, 2, 3, 5, 7, 11]) { - const chunked = run(explicit(), text, size); - for (const ch of Object.values(RESOLVE)) { - assert.equal(textFor(chunked, ch), textFor(oneShot, ch)); - } - } -}); - -function hybrid(): ProseStreamRouter { - return new ProseStreamRouter({ - mode: 'hybrid', - initialTarget: 'world:commons', - resolve: (spec) => RESOLVE[spec] ?? (spec === 'world:commons' ? 'world:commons' : null), - }); -} - -test('hybrid: unprefixed prose stays on the frozen locus', () => { - const out = run(hybrid(), 'ordinary world speech'); - assert.equal(textFor(out, 'world:commons'), 'ordinary world speech'); -}); - -test('hybrid: triple-arrow target switches publication and consumes the envelope', () => { - const out = run(hybrid(), '>>>#general cafe body'); - assert.equal(textFor(out, 'discord:g:100'), 'cafe body'); - assert.equal(out.every(d => !d.delta.includes('>>>')), true); - assert.equal(textFor(out, 'world:commons'), ''); -}); - -test('hybrid: permissive whitespace and cross-surface canonical ids', () => { - const out = run(hybrid(), ' >>> #ops spaced\n\t>>> world:commons home'); - assert.equal(textFor(out, 'discord:g:200'), 'spaced\n'); - assert.equal(textFor(out, 'world:commons'), 'home'); -}); - -test('hybrid: double arrows and mid-line triple arrows remain ordinary locus prose', () => { - const text = '>>#ops old syntax is prose\nlook >>>#ops quoted'; - const out = run(hybrid(), text); - assert.equal(textFor(out, 'world:commons'), text); - assert.equal(textFor(out, 'discord:g:200'), ''); -}); - -test('hybrid: missing target suppresses that envelope; a later valid envelope resumes', () => { - const out = run(hybrid(), '>>>#missing nowhere\nplain remains on failed target\n>>>world:commons restored'); - assert.equal(out.every(d => !d.delta.includes('nowhere') && !d.delta.includes('plain remains')), true); - assert.equal(textFor(out, 'world:commons'), 'restored'); -}); - -test('hybrid PROPERTY: char-by-char equals one-shot', () => { - const text = 'locus first\n>>>#general alpha\nbeta\n>>> #ops delta\n>>>world:commons home'; - const one = run(hybrid(), text); - for (const size of [1, 2, 3, 4, 7, 11]) { - const chunked = run(hybrid(), text, size); - for (const ch of ['world:commons', ...Object.values(RESOLVE)]) { - assert.equal(textFor(chunked, ch), textFor(one, ch)); - } - } -}); diff --git a/test/script-tool-names.test.ts b/test/script-tool-names.test.ts new file mode 100644 index 000000000..379b71809 --- /dev/null +++ b/test/script-tool-names.test.ts @@ -0,0 +1,157 @@ +/** + * scriptToolNames reads a code_execution script as Python reads it: which + * routing tools it names, which hold its own round, and which it leaves + * reachable by a later script, which mark the interpreter. code-execution's + * tests run the gate and the mark through real scripts; these pin the + * reading itself. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { scriptToolNames } from '../src/code-execution/script-tool-names.js'; + +const TOOLS = ['skip_reply', 'send_message', 'say', 'think']; +const read = (script: string) => { + const { names, reachable } = scriptToolNames(script, TOOLS); + return { names: [...names].sort(), reachable: [...reachable].sort() }; +}; + +test('a name counts where Python could call it, not inside a longer identifier', () => { + assert.deepEqual(read('say_hello = 1\nmy_say = 2\nsend_message_log = []'), { names: [], reachable: [] }); + assert.deepEqual(read('await mcpl__world__say({})'), { names: ['say'], reachable: [] }); +}); + +test('a direct call leaves nothing reachable, by name or by key', () => { + assert.deepEqual(read('await think({"content": "notes"})\nawait tools["think"]({"content": "more"})'), + { names: ['think'], reachable: [] }); + assert.deepEqual(read('await tools[ "world--say" ] ({})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('await say ({})'), { names: ['say'], reachable: [] }, 'a space before the call'); +}); + +test('no tools names nothing', () => { + assert.deepEqual([...scriptToolNames('await say({})', []).names], []); + assert.deepEqual([...scriptToolNames('await say({})', []).reachable], []); +}); + +test('an alias, a definition or a kept key leaves the tool reachable', () => { + assert.deepEqual(read('post = test__send_message'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('async def post():\n await test__send_message({})'), + { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('quiet = lambda: skip_reply({})'), { names: ['skip_reply'], reachable: ['skip_reply'] }); + assert.deepEqual(read('name = "test--send_message"'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read('name = "say"'), { names: ['say'], reachable: ['say'] }, 'a bare whole name, as a key kept in a variable'); +}); + +test('a key is a whole tool name, optionally namespaced, not a sentence that ends in one', () => { + assert.deepEqual(read('print("to speak, call world--say")'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('print("use mcpl__world__say")'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('k = "mcpl__world__say"'), { names: ['say'], reachable: ['say'] }); + // A prefix can hold punctuation besides `-` and `_`. + assert.deepEqual(read('post = tools["mcpl--zulip.work--say"]'), { names: ['say'], reachable: ['say'] }, 'a `.` in the prefix'); + assert.deepEqual(read('post = tools["discord:v1--say"]'), { names: ['say'], reachable: ['say'] }, 'a `:` in the prefix'); +}); + +test("a direct call outside a definition leaves nothing reachable, beside a definition that doesn't use the tool", () => { + assert.deepEqual(read('def fmt(x):\n return x\nawait say({"text": fmt("hi")})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('await say(sorted(xs, key=lambda x: x))'), { names: ['say'], reachable: [] }, 'a lambda after the call'); + assert.deepEqual(read('k = lambda x: x\nawait say({})'), { names: ['say'], reachable: [] }, "a lambda's span ends with its logical line"); + // A line that opens with a string stands where it's written: at column 0 it + // ends the def before it, as Python reads it. + assert.deepEqual(read('def f():\n return 1\n"%s" % say({})'), { names: ['say'], reachable: [] }, 'a call on a line that opens with a string'); + assert.deepEqual(read('def f():\n return 1\n"""note"""\nawait say({})'), { names: ['say'], reachable: [] }, 'a string-only line after a def'); + assert.deepEqual(read('def f():\n# a comment at column 0\n await say({})'), { names: ['say'], reachable: ['say'] }, "a comment doesn't end a def"); + assert.deepEqual(read('def f():\n return "("\nawait say({})'), { names: ['say'], reachable: [] }, "a bracket in a string continues no line"); + assert.deepEqual(read('print("a lambda"); await say({})'), { names: ['say'], reachable: [] }, "a lambda in a string isn't one"); +}); + +test('a name inside a definition is reachable, a called key included, whatever lines the definition spans', () => { + assert.deepEqual(read('def go():\n return tools["say"]({})'), { names: ['say'], reachable: ['say'] }); + // A header over several lines, and a body line continued inside brackets at + // no indentation, as Python allows; the call after the definition is direct. + assert.deepEqual(read('async def post(\n text,\n):\n x = [\n1,\n]\n await say({"text": text})\nawait skip_reply({})'), + { names: ['say', 'skip_reply'], reachable: ['say'] }); + assert.deepEqual(read('class Poster:\n async def go(self):\n await send_message({})\n\nawait think({})'), + { names: ['send_message', 'think'], reachable: ['send_message'] }); + // A line a string literal carries over continues its statement, whatever its + // indentation (Ramona-2416); so does a blank line written with \r\n. + assert.deepEqual(read('if True:\n def f():\n q = """\n"""; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a triple-quoted string'); + assert.deepEqual(read('if True:\n def f():\n q = "a\\\n"; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line opened inside a backslash-continued string'); + assert.deepEqual(read('def f():\r\n x = 1\r\n\r\n # a note\r\n await say({})\r\n'), { names: ['say'], reachable: ['say'] }, 'a blank \\r\\n line and a comment'); + // A backslash at a line's end continues it in code too. + assert.deepEqual(read('def f():\n x = 1 + \\\n2; await say({})'), { names: ['say'], reachable: ['say'] }, 'a line continued by a backslash in code'); + // ...a header's too: `async \` with `def` on the next line is one `async def`. + assert.deepEqual(read('async \\\ndef quiet():\n await skip_reply({})\nawait say({})'), { names: ['say', 'skip_reply'], reachable: ['skip_reply'] }, 'async joined to def by a backslash'); + assert.deepEqual(read('async \\\n \\\n def quiet():\n await skip_reply({})'), { names: ['skip_reply'], reachable: ['skip_reply'] }, 'a header over two backslash joins'); + assert.deepEqual(read('f = (lambda x:\n say(x))'), { names: ['say'], reachable: ['say'] }, "a lambda's body on its continued line"); +}); + +test("a class's own body runs as the class statement does, so a direct call there leaves nothing reachable", () => { + assert.deepEqual(read('class A:\n say({})'), { names: ['say'], reachable: [] }); + assert.deepEqual(read('class A:\n post = say'), { names: ['say'], reachable: ['say'] }, 'an alias kept on the class'); +}); + +test("a name at a line's end, before a line that opens with a bracket, is a reference, not a call", () => { + assert.deepEqual(read('alias = say\n(print("x"))'), { names: ['say'], reachable: ['say'] }); + assert.deepEqual(read('post = tools["say"]\n(print("x"))'), { names: ['say'], reachable: ['say'] }, 'a key, likewise'); +}); + +test('a comment names nothing, and its words are not code', () => { + assert.deepEqual(read('await test__slow({}) # then skip_reply, or say so'), { names: [], reachable: [] }); + assert.deepEqual(read('say("hi") # first class result'), { names: ['say'], reachable: [] }); +}); + +test('a sentence in a string counts for its own round but leaves nothing reachable', () => { + assert.deepEqual(read('# I think this works\ndef fmt(x):\n return x\nprint("They say the build is green")'), + { names: ['say'], reachable: [] }); + assert.deepEqual(read('def fmt(result):\n print(f"I would say {result}")'), { names: ['say'], reachable: [] }); +}); + +test('a triple-quoted string runs across lines, with quotes and # inside it', () => { + assert.deepEqual(read('note = """They said "skip_reply" # not a comment\nI think"""\npost = test__send_message'), + { names: ['send_message', 'skip_reply', 'think'], reachable: ['send_message'] }); +}); + +test('in a raw string, a backslash still keeps the quote after it inside the string', () => { + assert.deepEqual(read('pattern = r"\\" # say"\npost = test__send_message'), + { names: ['say', 'send_message'], reachable: ['send_message'] }); +}); + +test('an escaped quote stays inside its string', () => { + assert.deepEqual(read('msg = "she said \\"skip_reply\\" # done"'), { names: ['skip_reply'], reachable: [] }); +}); + +test('a name right after an escape in a string counts, as the string run as code would call it', () => { + assert.deepEqual(read('exec("x = 1\\nsay({})")'), { names: ['say'], reachable: [] }); + for (const escape of ['\\x0a', '\\u000a', '\\U0000000a', '\\N{LINE FEED}', '\\012']) { + assert.deepEqual(read(`exec("x = 1${escape}say({})")`), { names: ['say'], reachable: [] }, escape); + } +}); + +test("an f-string's call inside its braces counts for its own round but leaves nothing reachable", () => { + assert.deepEqual(read('print(f"result: {say({\'text\': \'hi\'})}")'), { names: ['say'], reachable: [] }); +}); + +test('a # inside a string starts no comment', () => { + assert.deepEqual(read('label = "say #3"; post = test__send_message'), + { names: ['say', 'send_message'], reachable: ['send_message'] }); +}); + +test("a string in one quote mark, not three, ends at its line's end, so a stray quote can't take the rest of the script", () => { + assert.deepEqual(read('x = "unclosed\npost = test__send_message'), { names: ['send_message'], reachable: ['send_message'] }); + assert.deepEqual(read("x = 'unclosed\npost = test__send_message"), { names: ['send_message'], reachable: ['send_message'] }); +}); + +test("what goes unseen: a name the code doesn't spell out, an enumeration, code from a file, a call in a string's text inside a definition, a call that outlives its script, an alias made inside a string or an f-string's field, code kept in a string", () => { + assert.deepEqual(read('name = "skip" + "_reply"\nawait tools[name]({})'), { names: [], reachable: [] }, 'built at runtime'); + assert.deepEqual(read('await tools["s\\x61y"]({})'), { names: [], reachable: [] }, 'written with escapes'); + assert.deepEqual(read('fns = list(tools.values())'), { names: [], reachable: [] }, 'an enumeration'); + assert.deepEqual(read('exec(open("helper.py").read())'), { names: [], reachable: [] }, 'code from a file'); + assert.deepEqual(read('def f():\n exec("say({})")'), { names: ['say'], reachable: [] }, "a call in a string's text inside a definition"); + assert.deepEqual(read('pending = say({"text": "hi"})'), { names: ['say'], reachable: [] }, 'a call made here...'); + assert.deepEqual(read('await pending'), { names: [], reachable: [] }, '...and awaited later'); + assert.deepEqual(read('asyncio.create_task(say({"text": "hi"}))'), { names: ['say'], reachable: [] }, 'a call scheduled as a task'); + assert.deepEqual(read('gen = (say({"text": m}) for m in msgs)'), { names: ['say'], reachable: [] }, 'a call left in a generator expression'); + assert.deepEqual(read('exec("post = say")'), { names: ['say'], reachable: [] }, 'an alias made inside a string the script runs'); + assert.deepEqual(read('print(f"{(post := say)}")'), { names: ['say'], reachable: [] }, "an alias made inside an f-string's field"); + assert.deepEqual(read('later = "say({\'text\': \'hi\'})"'), { names: ['say'], reachable: [] }, 'code kept in a string...'); + assert.deepEqual(read('await eval(later)'), { names: [], reachable: [] }, '...and run later'); +}); diff --git a/test/skip-reply-wake-xml-mode.test.ts b/test/skip-reply-wake-xml-mode.test.ts index 23edcf9a6..76bb53dea 100644 --- a/test/skip-reply-wake-xml-mode.test.ts +++ b/test/skip-reply-wake-xml-mode.test.ts @@ -2,7 +2,9 @@ * skip_reply's refused wake in XML tool mode: without an EventGate, a round * that calls skip_reply with wake_in_seconds is refused and the turn goes on. * That round's prose stays private, and every other round's prose is - * delivered as it would be without the refusal, in order. + * delivered as it would be without the refusal, in order. A skip_reply + * refused inside a round's code_execution script keeps its round private the + * same way. * * In XML tool mode, tool-calls events carry no roundContent, so no round is * routed live: the turn's prose is delivered at its end, from the turn-end @@ -41,6 +43,9 @@ const echo = (id: string, text: string, ...more: string[]): Round => ({ results: `${[id, ...more].map(() => '{"echoed":"x"}').join('')}`, }); const refusedWake = (id: string, text: string, wake: unknown = 60): Round => ({ text, calls: [{ id, name: 'skip_reply', input: { reason: 'later', wake_in_seconds: wake } }], results: 'refused' }); +/** A round whose one call is a foreground code_execution script. */ +const script = (id: string, text: string, code: string): Round => ({ text, calls: [{ id, name: 'code_execution', input: { code } }], results: '' }); +const REFUSED_IN_SCRIPT = 'await skip_reply({"wake_in_seconds": 60})'; class XmlStream implements YieldingStream { private events: StreamEvent[] = []; @@ -118,8 +123,13 @@ class EchoModule implements Module { async onProcess(_event: ProcessEvent, _state: ProcessState): Promise { return {}; } } -/** One XML-mode turn, with no EventGate unless asked: what reached the channel, and whether each round's result was an error. */ -async function xmlTurn(rounds: Round[], finalText: string, thinking = false, gate = false) { +/** + * One XML-mode turn, with no EventGate unless asked: what reached the channel, + * whether each round's result was an error, and which channel tools the + * registry handled. With scripts, code execution is on and skip_reply is a + * channel tool a script can call. + */ +async function xmlTurn(rounds: Round[], finalText: string, thinking = false, opts: { scripts?: boolean; gate?: boolean } = {}) { let stream: XmlStream | undefined; const membrane = { async complete(_r: NormalizedRequest): Promise { throw new Error('not used'); }, @@ -132,13 +142,21 @@ async function xmlTurn(rounds: Round[], finalText: string, thinking = false, gat const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: membrane as unknown as import('@animalabs/membrane').Membrane, agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], modules: [new EchoModule()], - ...(gate ? { gate: { configPath: join(dir, 'gate.json') } } : {}), + ...(opts.scripts ? { codeExecution: { enabled: true } } : {}), + ...(opts.gate ? { gate: { configPath: join(dir, 'gate.json') } } : {}), }); const routed: string[] = []; + const registrySaw: string[] = []; + const channelTools = opts.scripts + ? [{ name: 'skip_reply', description: 'End your turn without sending anything.', inputSchema: { type: 'object' as const, properties: {} } }] + : []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ - getChannelTools: () => [], resolveLocus: () => 'channel:test', + getChannelTools: () => channelTools, resolveLocus: () => 'channel:test', routeSpeech: async (_agent: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'channel:test' }; }, - handleChannelToolCall: async () => ({ success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }), + handleChannelToolCall: async (name: string) => { + registrySaw.push(name); + return { success: true, endTurn: true, data: { skipped: true, note: 'Turn ended; nothing sent.' } }; + }, }, { get: (target, prop: string) => prop in target ? target[prop as keyof typeof target] : () => undefined }); try { const agent = framework.getAgent('agent')!; @@ -146,7 +164,7 @@ async function xmlTurn(rounds: Round[], finalText: string, thinking = false, gat .startAgentStream(agent, { agentName: 'agent', reason: 'test', source: 'test', timestamp: Date.now() }); await framework.runUntilIdle(); const errors = (stream?.received ?? []).map((r) => (r[0] as { isError?: boolean } | undefined)?.isError === true); - return { routed, errors }; + return { routed, errors, registrySaw }; } finally { await framework.stop(); rmSync(dir, { recursive: true, force: true }); @@ -210,7 +228,7 @@ describe('skip_reply refused in XML tool mode: its round stays private, and the }); it('with a gate, a wake_in_seconds that isn\'t a number of seconds is refused the same way', async () => { - const r = await xmlTurn([echo('e1', 'ONE'), refusedWake('w1', 'REFUSED_PRIVATE', 'soon')], '\nFINAL_REPLY', false, true); + const r = await xmlTurn([echo('e1', 'ONE'), refusedWake('w1', 'REFUSED_PRIVATE', 'soon')], '\nFINAL_REPLY', false, { gate: true }); assert.deepEqual(r.errors, [false, true], 'the wake was refused and the turn went on'); assert.deepEqual(r.routed, ['ONE', 'FINAL_REPLY']); }); @@ -220,4 +238,16 @@ describe('skip_reply refused in XML tool mode: its round stays private, and the assert.deepEqual(r.errors, [false, true, false]); assert.deepEqual(r.routed, ['ONE', 'TWO', 'THIRD']); }); + + it('a skip_reply refused inside a code_execution script keeps its round private too', async () => { + const r = await xmlTurn([script('ce1', 'SCRIPT_PRIVATE', REFUSED_IN_SCRIPT)], '\nFINAL_REPLY', false, { scripts: true }); + assert.ok(!r.registrySaw.includes('skip_reply'), 'the host refused the call before the registry saw it'); + assert.deepEqual(r.routed, ['FINAL_REPLY']); + }); + + it('after a script round whose skip_reply was refused, the next round is delivered', async () => { + const r = await xmlTurn([script('ce1', 'SCRIPT_PRIVATE', REFUSED_IN_SCRIPT), echo('e1', 'TWO')], '\nTHIRD', false, { scripts: true }); + assert.ok(!r.registrySaw.includes('skip_reply'), 'the host refused the call before the registry saw it'); + assert.deepEqual(r.routed, ['TWO', 'THIRD']); + }); }); diff --git a/test/stop-step-throws.test.ts b/test/stop-step-throws.test.ts index 97a00e13b..2860d67b5 100644 --- a/test/stop-step-throws.test.ts +++ b/test/stop-step-throws.test.ts @@ -108,6 +108,8 @@ interface Row { /** The releases whose unread pass ran, in the unread pass's row. */ const unreadPassed: string[] = []; +/** The waiting outgoing completes that went out, in their row. */ +const sentCompletes: string[] = []; const ROWS: Row[] = [ { step: "the push coalescer's suspend()", inject: (fw, e) => throwIn(internals(fw), 'pushCoalescer', 'suspend', e) }, @@ -138,6 +140,20 @@ const ROWS: Row[] = [ }, }, { step: "cancelling provider admission", inject: (fw, e) => patchMethod(fw, 'cancelProviderAdmission', e) }, + { + step: "a stream's waiting outgoing complete", + inject: (fw, e) => { + // Two streams' completes wait: the first one's throws. + sentCompletes.length = 0; + const waiting = internals(fw).pendingOutgoingCompletes as Set<() => void>; + const first = () => { throw e; }; + const second = () => { sentCompletes.push('second'); }; + waiting.add(first); + waiting.add(second); + return () => { waiting.delete(first); waiting.delete(second); }; + }, + check: () => assert.deepEqual(sentCompletes, ['second'], "the other stream's complete still went out"), + }, { step: "the channel registry's stopAll()", inject: (fw, e) => throwIn(internals(fw), 'channelRegistry', 'stopAll', e) }, { step: "the event gate's dispose()", inject: (fw, e) => throwIn(internals(fw), 'eventGate', 'dispose', e) }, { diff --git a/test/tool-result-guard.test.ts b/test/tool-result-guard.test.ts index b17a62274..496afd5b6 100644 --- a/test/tool-result-guard.test.ts +++ b/test/tool-result-guard.test.ts @@ -349,7 +349,7 @@ test('native Membrane observes the first refusal even when guard is enabled by a const outgoing: string[] = []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(withDeliverSpeech({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, speech: string) => { + routeSpeech: async (_agent: string, speech: string, channelId: string) => { routed.push(speech); return { delivered: true, channelId: 'world:test' }; }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, @@ -378,7 +378,7 @@ test('native Membrane observes the first refusal even when guard is enabled by a } finally { await framework.stop(); } }); -test('explicit mode: a guard retry streams on to the sticky destination delivery uses', async () => { +test('explicit mode: a guard retry streams on to the sticky destination delivery uses, and each stream completes under its own id', async () => { const h = await harness([ [createMockResponse([ { type: 'text', text: '>>#alpha first' }, @@ -388,6 +388,7 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery ], { toolResultGuard: true, proseRouting: 'explicit', refusalHandling: { retries: 3 } }); const routed: Array<{ text: string; channelId: string }> = []; const streamed: Array<{ channelId: string; delta: string }> = []; + const completed: Array<{ channelId: string; text: string; inferenceId: string }> = []; (h.framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(withDeliverSpeech({ resolveProseTarget: (spec: string) => spec === '#alpha' ? { channelId: 'chan-alpha' } : { error: `no channel ${spec}` }, routeSpeech: async (_agent: string, speech: string, channelId: string) => { @@ -396,6 +397,9 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { streamed.push({ channelId, delta }); }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, text: string) => { + completed.push({ channelId, text, inferenceId }); + }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }), { get: (target, key: string) => key in target ? (target as Record)[key] : () => undefined }); @@ -409,10 +413,15 @@ test('explicit mode: a guard retry streams on to the sticky destination delivery const toAlpha = streamed.filter((d) => d.channelId === 'chan-alpha').map((d) => d.delta).join(''); assert.match(toAlpha, /continued/, 'the resumed stream follows delivery there'); assert.doesNotMatch(streamed.map((d) => d.delta).join(''), /discard-this-partial-output/); + assert.deepEqual(completed.map((c) => ({ channelId: c.channelId, text: c.text })), [ + { channelId: 'chan-alpha', text: 'first' }, + { channelId: 'chan-alpha', text: 'continued' }, + ], 'the stream that handed its turn to the retry completes what it streamed, and so does the retry'); + assert.notEqual(completed[0].inferenceId, completed[1].inferenceId, 'the retry streams under its own inference id'); } finally { await h.framework.stop(); } }); -test('explicit mode: a budget restart streams on to the destination the turn last addressed, before delivery catches up', { timeout: 15000 }, async () => { +test('explicit mode: after a budget restart, the resumed reply streams where delivery sends it, once the earlier posts have gone', { timeout: 15000 }, async () => { const h = await harness([ [ createMockResponse([ @@ -428,6 +437,10 @@ test('explicit mode: a budget restart streams on to the destination the turn las ], { proseRouting: 'explicit', maxStreamTokens: 1000 }); const routed: Array<{ text: string; channelId: string }> = []; const streamed: Array<{ channelId: string; delta: string }> = []; + // Each publish that succeeds and each chunk, in the order they happened. + const order: string[] = []; + // What delivery had started to publish when the restarted stream wrote. + let atRestart: string[] | undefined; // Delivery of 'first' is held until the resumed stream has written // 'continued', so '>>#cafe second' is still queued, and delivery's sticky // destination still #alpha, when the restart begins. @@ -451,6 +464,9 @@ test('explicit mode: a budget restart streams on to the destination the turn las for await (const event of events) { yield event; if (event.type === 'tokens' && event.content === 'continued') { + // The premise: the restarted stream is writing while 'first' is + // still held and '>>#cafe second' is still queued behind it. + atRestart = routed.map((r) => r.text); releaseFirst(); await secondDone; } @@ -470,10 +486,12 @@ test('explicit mode: a budget restart streams on to the destination the turn las routed.push({ text, channelId }); if (text === 'first') await firstHeld; if (text === 'second') secondDelivered(); + order.push(`published:${text}`); return { delivered: true, channelId }; }, sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { streamed.push({ channelId, delta }); + order.push(`chunk:${delta.trim()}`); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -482,6 +500,7 @@ test('explicit mode: a budget restart streams on to the destination the turn las await h.run(); assert.equal(h.membrane.requests.length, 2); assert.ok(restarts.includes('context_budget'), 'the turn restarted for its budget'); + assert.deepEqual(atRestart, ['first'], "the restarted stream wrote while 'first' was held and 'second' still queued"); assert.deepEqual(routed, [ { text: 'first', channelId: 'chan-alpha' }, { text: 'second', channelId: 'chan-cafe' }, @@ -492,6 +511,9 @@ test('explicit mode: a budget restart streams on to the destination the turn las assert.equal(streamed.some((d) => d.delta.includes('>>')), false, 'no prefix is streamed as text'); assert.deepEqual(streamed.filter((d) => d.delta.includes('continued')).map((d) => d.channelId), ['chan-cafe'], 'the resumed stream follows the destination the turn last addressed'); + assert.ok(order.indexOf('chunk:continued') > order.indexOf('published:continued') + && order.indexOf('published:continued') > order.indexOf('published:first'), + `the resumed reply streams once its own publish succeeds, after the earlier posts: ${order.join(', ')}`); } finally { releaseFirst(); secondDelivered(); @@ -669,11 +691,13 @@ test("locus mode: a resumed stream's deliveries queue behind the turn's earlier /** * The registry stub for one-stream explicit turns: #alpha and #cafe resolve, - * deliveries and stream chunks are recorded, and a `think` tool is offered. + * deliveries, stream chunks and stream completes are recorded, and a `think` + * tool is offered. */ function explicitRegistry(h: Awaited>) { const routed: Array<{ text: string; channelId: string }> = []; - const streamed: Array<{ channelId: string; delta: string }> = []; + const streamed: Array<{ channelId: string; delta: string; inferenceId: string }> = []; + const completed: Array<{ channelId: string; text: string; inferenceId: string }> = []; (h.framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy(withDeliverSpeech({ resolveProseTarget: (spec: string) => spec === '#alpha' ? { channelId: 'chan-alpha' } : spec === '#cafe' ? { channelId: 'chan-cafe' } : { error: `no channel ${spec}` }, @@ -681,20 +705,23 @@ function explicitRegistry(h: Awaited>) { routed.push({ text: text.trim(), channelId }); return { delivered: true, channelId }; }, - sendOutgoingChunk: (channelId: string, _agent: string, _id: string, _index: number, delta: string) => { - streamed.push({ channelId, delta }); + sendOutgoingChunk: (channelId: string, _agent: string, inferenceId: string, _index: number, delta: string) => { + streamed.push({ channelId, delta, inferenceId }); + }, + sendOutgoingComplete: (channelId: string, _agent: string, inferenceId: string, text: string) => { + completed.push({ channelId, text, inferenceId }); }, handleChannelToolCall: async () => ({ success: true, data: { noted: true } }), getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [{ name: 'think', description: 'Reason privately', inputSchema: { type: 'object', properties: { content: { type: 'string' } } } }], }), { get: (target, key: string) => key in target ? (target as Record)[key] : () => undefined }); - return { routed, streamed }; + return { routed, streamed, completed }; } test("explicit mode: within one stream, each round after a private round or >>skip_reply streams where delivery sends it", { timeout: 15000 }, async () => { - // The stream follows each prefix as it's written, a private round's too. - // At each round's end it returns to the turn's destination, which a private - // think's >>#cafe and a >>skip_reply envelope leave at #alpha. + // The stream carries what delivery publishes, where it publishes it. A + // private think's >>#cafe and a >>skip_reply envelope publish nothing and + // leave the sticky destination at #alpha. const h = await harness([[ createMockResponse([ { type: 'text', text: '>>#alpha first\n' }, @@ -731,10 +758,13 @@ test("explicit mode: within one stream, each round after a private round or >>sk } }); -test("explicit mode: a membrane retry within a stream starts again at the turn's destination", { timeout: 15000 }, async () => { +test("explicit mode: a membrane retry within a stream streams none of the refused attempt, and the stream goes on under its id", { timeout: 15000 }, async () => { // Round one set #alpha. The membrane then refuses the next attempt and // retries it; the retried attempt's unprefixed reply is delivered to - // #alpha, and streams there too. + // #alpha, and streams there too. The refused attempt was never delivered, + // so it never streamed, and the stream's one `complete` carries both + // delivered rounds. The stream's inference/lifecycle `started` and its + // terminal share that one id too. const h = await harness([[ createMockResponse([ { type: 'text', text: '>>#alpha first\n' }, @@ -758,22 +788,37 @@ test("explicit mode: a membrane retry within a stream starts again at the turn's }; return stream; }; - const { routed, streamed } = explicitRegistry(h); + const { routed, streamed, completed } = explicitRegistry(h); + const lifecycle: Array<{ inferenceId: string; phase: string }> = []; + (h.framework as unknown as { hookOrchestrator: unknown }).hookOrchestrator = { + beforeInference: async () => [], + emitLifecycle: (event: { inferenceId: string; phase: string }) => { lifecycle.push({ inferenceId: event.inferenceId, phase: event.phase }); }, + }; try { await h.run(); + // The stream's `complete` is chained on its deliveries, ahead of this wait. + await (h.framework as unknown as { proseSpeechChains: Map> }).proseSpeechChains.get('assistant'); assert.deepEqual(routed, [ { text: 'first', channelId: 'chan-alpha' }, { text: 'continued', channelId: 'chan-alpha' }, ]); - assert.deepEqual(streamed.filter((d) => d.delta.includes('continued')).map((d) => d.channelId), ['chan-alpha']); - assert.equal(streamed.some((d) => d.delta.includes('refused partial')), true, - 'the refused attempt had streamed, as text streams before a refusal is known'); + assert.deepEqual(streamed.map((d) => ({ channelId: d.channelId, delta: d.delta })), [ + { channelId: 'chan-alpha', delta: 'first' }, + { channelId: 'chan-alpha', delta: '\n\ncontinued' }, + ], 'the stream carries what was published, and none of the refused attempt'); + assert.equal(new Set(streamed.map((d) => d.inferenceId)).size, 1, 'the retry keeps the stream id'); + assert.deepEqual(completed, [{ channelId: 'chan-alpha', text: 'first\n\ncontinued', inferenceId: streamed[0]!.inferenceId }], + 'one complete closes the stream with everything it carried'); + assert.deepEqual(lifecycle, [ + { inferenceId: streamed[0]!.inferenceId, phase: 'started' }, + { inferenceId: streamed[0]!.inferenceId, phase: 'completed' }, + ], 'the lifecycle notifications start and end the same inference'); } finally { await h.framework.stop(); } }); -test("explicit mode: a round's later text block starts a new line for the stream, as delivery joins blocks", { timeout: 15000 }, async () => { +test("explicit mode: a round's later text block starts a new envelope, for delivery and the stream alike", { timeout: 15000 }, async () => { // Two text blocks in one round, the second opening with a prefix, the first // not ending with a newline. Delivery reads them joined by a newline. const h = await harness([[ @@ -784,27 +829,6 @@ test("explicit mode: a round's later text block starts a new line for the stream ] as ContentBlock[], 'tool_use'), createMockResponse([{ type: 'text', text: 'continued' }] as ContentBlock[]), ]], { proseRouting: 'explicit' }); - const open = h.membrane.streamYielding.bind(h.membrane); - h.membrane.streamYielding = (request: NormalizedRequest, options?: YieldingStreamOptions) => { - const stream = open(request, options); - const iterate = stream[Symbol.asyncIterator].bind(stream); - (stream as unknown as { [Symbol.asyncIterator]: () => AsyncIterator })[Symbol.asyncIterator] = async function* () { - const events = { [Symbol.asyncIterator]: iterate } as AsyncIterable<{ type?: string; content?: unknown }>; - let split = false; - for await (const event of events) { - if (!split && event.type === 'tokens' && typeof event.content === 'string' && event.content.includes('>>#cafe')) { - // The mock streams a round's text as one token: stream it as the - // two blocks the round holds. - split = true; - yield { type: 'tokens', content: '>>#alpha first part', meta: { type: 'text', visible: true, blockIndex: 0 } }; - yield { type: 'tokens', content: '>>#cafe second', meta: { type: 'text', visible: true, blockIndex: 1 } }; - continue; - } - yield event; - } - }; - return stream; - }; const { routed, streamed } = explicitRegistry(h); try { await h.run(); @@ -823,8 +847,8 @@ test("explicit mode: a round's targets are resolved once, when it ends, for its // From a review's probe. Both rounds that name #cafe have ended, and their // deliveries wait behind 'first', when the registry's #cafe starts naming // another channel. Each goes to the channel #cafe named when its round - // ended, where the stream sent it: the queued '>>#cafe second', and the - // resumed '>>#cafe continued', whose text-only round ended before the wait. + // ended, and streams there: the queued '>>#cafe second', and the resumed + // '>>#cafe continued', whose text-only round ended before the wait. const t = await heldRestart('explicit', { first: '>>#alpha first\n', second: '>>#cafe second', continued: '>>#cafe continued' }); try { await t.held; @@ -1013,11 +1037,11 @@ test("hybrid: a restarted turn whose last response is empty writes its receipt a test("a fresh turn starts new routing state in every mode, and leaves the earlier turn's as it was", async () => { for (const mode of ['explicit', 'hybrid', 'locus'] as const) { const h = await harness([[answer()]], { proseRouting: mode }); - type Routing = { pin?: string; destination: string | null; hybridHold?: 'private' | 'failed'; + type Routing = { pin?: string; hybridHold?: 'private' | 'failed'; continuation: boolean; deliveries: string[]; suppressed: number; drafts: unknown[]; keptPrivate: number; triggerChannel?: string; locus?: string; engaged: Set; arrivals: Array<{ messageId: string; channelId: string }> }; const internals = h.framework as unknown as { turnRouting: Map }; - const earlierFields = (): Routing => ({ pin: 'chan-earlier', destination: 'chan-earlier', hybridHold: 'failed', + const earlierFields = (): Routing => ({ pin: 'chan-earlier', hybridHold: 'failed', continuation: true, deliveries: ['chan-earlier'], suppressed: 2, drafts: [{ id: 'd-earlier' }], keptPrivate: 1, triggerChannel: 'chan-earlier', locus: 'chan-earlier', engaged: new Set(['chan-earlier']), arrivals: [{ messageId: 'earlier-message', channelId: 'chan-earlier' }] }); const earlier = earlierFields(); @@ -1027,7 +1051,6 @@ test("a fresh turn starts new routing state in every mode, and leaves the earlie const now = internals.turnRouting.get('assistant'); assert.notEqual(now, earlier, `${mode}: the new turn has its own routing state`); assert.equal(now?.pin, undefined, `${mode}: no sticky destination`); - assert.equal(now?.destination, null, `${mode}: no turn destination`); assert.equal(now?.hybridHold, undefined, `${mode}: no hybrid hold`); assert.equal(now?.continuation, false, `${mode}: no continuation`); assert.deepEqual(now?.deliveries, [], `${mode}: no deliveries carried into the new turn`); diff --git a/test/tool-wrapper-prose-guard-integration.test.ts b/test/tool-wrapper-prose-guard-integration.test.ts index 5d8010f0b..6cf9b8844 100644 --- a/test/tool-wrapper-prose-guard-integration.test.ts +++ b/test/tool-wrapper-prose-guard-integration.test.ts @@ -121,7 +121,9 @@ async function run(enabled?: boolean, allowedTools?: string[]) { const outgoing: string[] = []; (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, speech: string) => { routed.push(speech); return { delivered: true, channelId: 'world:test' }; }, + routeSpeech: async (_agent: string, speech: string, channelId: string) => { + routed.push(speech); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -172,7 +174,9 @@ describe('tool wrapper prose guard integration', () => { agents: [{ name: 'assistant', model: 'test', systemPrompt: 'sys', toolWrapperProseGuard: true }], modules: [module], }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ - resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string, channelId: string) => { + routed.push(text); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_c: string, _a: string, _id: string, _i: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -219,7 +223,7 @@ describe('tool wrapper prose guard integration', () => { assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); - it('buffers then delivers ordinary prose when enabled', async () => { + it("a guarded turn's ordinary prose streams what delivery publishes", async () => { const dir = mkdtempSync(join(tmpdir(), 'af-wrapper-guard-ordinary-')); dirs.push(dir); const membrane = new MockMembrane(); membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Ordinary answer.' }] as ContentBlock[])); @@ -230,7 +234,9 @@ describe('tool wrapper prose guard integration', () => { }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', - routeSpeech: async (_agent: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + routeSpeech: async (_agent: string, text: string, channelId: string) => { + routed.push(text); return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); @@ -240,7 +246,8 @@ describe('tool wrapper prose guard integration', () => { const all = framework.getAgent('assistant')!.getContextManager().getAllMessages() as Array<{ content: ContentBlock[]; metadata?: Record }>; await framework.stop(); assert.deepEqual(routed, ['Ordinary answer.']); - assert.deepEqual(outgoing, [], 'guarded turns buffer ordinary prose until completion'); + assert.deepEqual(outgoing, ['Ordinary answer.'], + 'the guard classifies before delivery, and the stream carries what delivery publishes'); assert.ok(all.some((m) => m.content.some((b) => b.type === 'text' && b.text === 'Ordinary answer.'))); assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); @@ -254,7 +261,7 @@ describe('tool wrapper prose guard integration', () => { membrane.pushResponse(createMockResponse([{ type: 'text', text: '\n', }] as ContentBlock[])); - const module = new HeartbeatModule(); const routed: string[] = []; + const module = new HeartbeatModule(); const routed: string[] = []; const outgoing: string[] = []; const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: membrane.asMembrane(), agents: [{ name: 'assistant', model: 'test', systemPrompt: 'sys', toolWrapperProseGuard: true }], @@ -262,6 +269,7 @@ describe('tool wrapper prose guard integration', () => { }); (framework as unknown as { channelRegistry: unknown }).channelRegistry = new Proxy({ resolveLocus: () => 'world:test', routeSpeech: async (_a: string, text: string) => { routed.push(text); return { delivered: true, channelId: 'world:test' }; }, + sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => { outgoing.push(delta); }, getDefaultPublishChannel: () => null, isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], }, { get: (target, prop: string) => (prop in target ? (target as Record)[prop] : () => undefined) }); framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); @@ -270,6 +278,8 @@ describe('tool wrapper prose guard integration', () => { await framework.stop(); assert.equal(module.calls, 1, 'genuine structured tool executes exactly once'); assert.deepEqual(routed, ['\n']); + assert.deepEqual(outgoing, ['\n'], + 'posted as ordinary prose, it streams as delivery publishes it'); assert.ok(all.some((m) => m.content.some((b) => b.type === 'text' && b.text.includes('')))); assert.ok(!all.some((m) => m.metadata?.kind === 'tool-wrapper-prose-contained')); }); @@ -413,7 +423,9 @@ describe('tool wrapper prose guard integration', () => { resolveLocus: () => 'world:test', startTyping: (channel: string) => typing.push(`start:${channel}`), stopTyping: (channel?: string) => typing.push(`stop:${channel ?? '(all)'}`), - routeSpeech: async () => ({ delivered: true, channelId: 'world:test' }), + routeSpeech: async (_agent: string, text: string, channelId: string) => { + return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => outgoing.push(['chunk', delta]), sendOutgoingComplete: (_channel: string, _agent: string, _id: string, text: string) => outgoing.push(['complete', text]), getDefaultPublishChannel: () => 'world:test', isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -423,11 +435,18 @@ describe('tool wrapper prose guard integration', () => { created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); const run = framework.runEphemeralToCompletion(created.agent, created.contextManager); framework.start(); - assert.deepEqual(await run, { speech: 'Ordinary ephemeral answer.', toolCallsCount: 0 }); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.ok(typing.includes('stop:(all)'), 'terminal physical frame stops typing after ephemeral caller disposal'); - assert.ok(outgoing.some(([kind, text]) => kind === 'complete' && text === 'Ordinary ephemeral answer.'), 'terminal physical frame finalizes outgoing stream'); - await framework.stop(); + try { + assert.deepEqual(await run, { speech: 'Ordinary ephemeral answer.', toolCallsCount: 0 }); + // The terminal frame's teardown stops typing and sends the complete. + const until = Date.now() + 5000; + while (Date.now() < until && !(typing.includes('stop:(all)') && outgoing.some(([kind]) => kind === 'complete'))) { + await new Promise((resolve) => setTimeout(resolve, 5)); + } + assert.ok(typing.includes('stop:(all)'), 'terminal physical frame stops typing after ephemeral caller disposal'); + assert.ok(outgoing.some(([kind, text]) => kind === 'complete' && text === 'Ordinary ephemeral answer.'), 'terminal physical frame finalizes outgoing stream'); + } finally { + await framework.stop(); + } }); it('releases a conversation name reservation after transient creation failure', async () => { @@ -452,7 +471,7 @@ describe('tool wrapper prose guard integration', () => { }); - it('idle-disposed ephemeral stops typing and finalizes every started outgoing stream', async () => { + it("idle-disposed ephemeral stops typing, and text it never delivered doesn't stream", async () => { const dir = mkdtempSync(join(tmpdir(), 'af-wrapper-guard-idle-terminal-')); dirs.push(dir); const framework = await AgentFramework.create({ storePath: join(dir, 'store'), membrane: new LateCompletionAfterIdleMembrane().asMembrane(), agents: [], modules: [] }); const events: string[] = []; @@ -460,7 +479,9 @@ describe('tool wrapper prose guard integration', () => { resolveLocus: () => 'world:test', startTyping: (channel: string) => events.push(`start:${channel}`), stopTyping: (channel?: string) => events.push(`stop:${channel ?? '(all)'}`), - routeSpeech: async () => ({ delivered: true, channelId: 'world:test' }), + routeSpeech: async (_agent: string, text: string, channelId: string) => { + return { delivered: true, channelId: 'world:test' }; + }, sendOutgoingChunk: (_channel: string, _agent: string, _id: string, _index: number, delta: string) => events.push(`chunk:${delta}`), sendOutgoingComplete: (_channel: string, _agent: string, _id: string, text: string) => events.push(`complete:${text}`), getDefaultPublishChannel: () => 'world:test', isChannelOpen: () => true, getDescriptor: () => undefined, getChannelTools: () => [], @@ -470,12 +491,18 @@ describe('tool wrapper prose guard integration', () => { created.contextManager.addMessage('user', [{ type: 'text', text: 'go' }]); const run = framework.runEphemeralToCompletion(created.agent, created.contextManager, { startupTimeoutMs: 500, idleTimeoutMs: 20, idlePollMs: 5 }); framework.start(); - await assert.rejects(run, /stalled/); - await new Promise((resolve) => setTimeout(resolve, 180)); - assert.ok(events.includes('stop:(all)'), 'disposer-owned frame stops typing'); - assert.ok(events.some((event) => event.startsWith('chunk:')), 'probe actually opened an outgoing stream'); - assert.ok(events.some((event) => event.startsWith('complete:')), 'every started outgoing stream receives a terminal complete'); - await framework.stop(); + try { + await assert.rejects(run, /stalled/); + // The disposed frame's typing stops in its teardown, which runs once + // its late answer has come in. + const until = Date.now() + 5000; + while (Date.now() < until && !events.includes('stop:(all)')) await new Promise((resolve) => setTimeout(resolve, 5)); + assert.ok(events.includes('stop:(all)'), 'disposer-owned frame stops typing'); + assert.deepEqual(events.filter((event) => event.startsWith('chunk:') || event.startsWith('complete:')), [], + "the partial text and the disposed frame's late answer were never delivered, so neither streams"); + } finally { + await framework.stop(); + } }); });