feat(voice-relay): framework agents join the external TTS relay as bot clients

[nickmahdavi]

What

Framework agents join the external melodeus TTS relay as first-class bot clients — the same /bot connection the relay's existing (ChapterX) bots hold. The relay itself is untouched and stays external: one module instance dials {url}/bot, authenticates as one bot identity, streams the agent's turns as the relay's own wire messages, and maps interruptions coming back into abortInference. Only the client connection each agent-framework process needs; nothing relay-side moves into connectome.

Commits / how to review

1. feat(traces): channel identity on inference traces + keepText on abort — the two core changes we discussed: optional channelId on the turn-scoped inference:* traces (stamped read-only from the turn's pinned locus), and abortInference(name, { reason, keepText }) so an interrupted agent's context keeps what was actually spoken. ~200 src lines; the only commit touching shared code — review this one hardest.
2. feat(voice-relay): wire-protocol types, trace bridge, test helpers — types.ts is a near-verbatim mirror of melodeus-tts-relay/src/types.ts @ ec8f0f1 (fastest review: diff it against the relay). The bridge translates inference:* traces into the five streaming messages.
3. feat(voice-relay): RelayClientModule — the WebSocket client: auth, reconnect with backoff + stability window, heartbeat watchdog, strict interruption addressing with per-client staleness windows (melodeus reports spoken text per block, iOS per activation).
4. fix(voice-relay): review fixes — two review rounds; the commit body enumerates everything. Most notable: production streams now request emitBlocks: true — block boundaries never reached the wire before (melodeus can't track or interrupt an utterance without them), masked by mocks that ignored the flag.

Behavior notes for existing deployments

• TraceEvent additions are optional fields; abortInference still accepts the plain string form; hosts without the module are unaffected.
• One deliberate visible change: user-initiated aborts are booked as deliberate cancels (inference:exhausted with errorType: 'abort') — no failure streak, no failures.log entry, no [inference-failed] chronicle marker. Voice barge-ins are routine; three in a row must not page an operator as hard-down. Monitoring that counted aborts as failures will see fewer entries.
• Module wiring: construct → bind(framework) → addModule. Without bind, outbound streaming works but interruptions are dropped (warned). One instance = one bot identity; scope with agents: [...] when running several instances in one process.

Testing

• 454 tests, 453 pass, 1 pre-existing skip (438 on main).
• CI runs a closed-loop integration test: real framework + in-process mock /bot server — a turn streams to the wire, an interruption comes back, activation_end(reason: "abort") goes out, and the agent's context keeps the spoken prefix.
• An e2e spawns the real melodeus-tts-relay (auto-skips where the checkout is absent, i.e. on CI): a simulated /tts voice client hears a framework agent through the real relay and interrupts it mid-sentence.
• Also exercised live against the real relay with a real model.

[Anarchid]

Read against head 02381b2 (4 commits, +4083/−24), rebuilt and tested here against its own base, and checked against upstream main at fa95817 (0.13.0), connectome-host bf41338, discord-mcpl cda7b27, and melodeus-tts-relay ec8f0f1.

Changes requested — not mergeable, and the design it implements has been overtaken. Since this PR's base, main grew its own answer to the same problem three times over: channelId on inference traces plus mcpl:speech-routed{messageId,text} (a2f0d85, Jul 22), a shipped connectome-host TtsRelayModule for the same relay /bot endpoint (edit-the-posted-message design, Jul 22), and the MCPL Spec 14.3 channels/outgoing/chunk prose stream (24c559a, Jul 26) that discord-mcpl #28 (merged Sep 7) now voices directly, reporting a [voice] interrupted receipt instead of aborting inference. The PR mentions none of them. Independently of that, the module cannot work against the only production host: it puts the MCPL composite channel id (discord:<guild>:<id>) on the relay wire, where voice clients subscribe by raw snowflake, so nothing is voiced and every interruption is dropped as "unknown channel". Two pieces are worth salvaging as small separate PRs: emitBlocks: true (main's own block-trace consumers are dead without it) and the deliberate-cancel failure-streak bypass (with provenance).

Blockers

1. Superseded design + nine-hunk conflict in the abort path. (both reviewers) main's driveStream now has lifecyclePhase with a terminal hookOrchestrator.emitLifecycle in finally, abortAgentScript, turnProseDeliveries + [delivered] receipts, ownsPhysicalStream, proseRouting explicit/hybrid/disabled, and a ProseStreamRouter that emits routed, >>-stripped, fail-closed-suppressed deltas (sendOutgoingChunk) precisely for voice. The PR's bridge forwards raw inference:tokens, and its keepText path commits that raw text and calls routeSpeech around main's delivery policy. Astra's in-memory comparison on a hybrid-mode turn: main routes {"c2":"Hello\n"}; the bridge voices ">>>other Hello\n>>>skip_reply Hidden" at the frozen locus. My probe: undeliveredSuffix('>> #general hello there','>> #general hello') → 'there', i.e. routing syntax survives into keepText. Per-hunk disposition (Astra, verified against the trial merge in merge/): hunks 1, 2, 7 mechanical; 3, 5, 6 drop the PR side (main already stamps channelId from typingChannel and keeps restart gate ownership); 4, 8, 9 need redesign (liveRoutedProse bookkeeping, keepText commit vs receipts, streamId-only ownership vs ownsPhysicalStream). Resolving the text conflicts does not preserve main's behaviour.

   • Evidence: main src/framework.ts:6600-6620, :7413-7470, :7642-7697; src/mcpl/prose-stream-router.ts:1-25; discord-mcpl src/voice.ts:1-42, src/server.ts:418-468.
   • Fix: decide first whether a second melodeus client is wanted at all (see Minor 13). If yes, rebuild on main's routed deltas (or the stamped traces) and adopt the receipt-style interruption, not keepText.

2. Composite MCPL channel id on the relay wire → module is inert on every registry-backed Discord deployment. (both) Locus = event.channelId from mcpl:channel-incoming (main framework.ts:4954) → activeTriggerChannels → resolveLocus (channel-registry.ts:2285); discord-mcpl builds ids as discord:<guildId>:<channelId> (src/channels.ts:17-19). The bridge sends event.channelId verbatim (trace-bridge.ts:136,153); the relay fans out by exact match on that field (handlers/bot.ts:152, state.ts:180-183) and interruptions come back with raw snowflakes (PROTOCOL.md:99-108). Module then hits "Interruption for unknown channel dropped" (relay-client-module.ts:531-535). connectome-host's module strips to the last segment for exactly this (tts-relay-module.ts:230-232). Astra probe: {"wireChannels":["discord:123:456"],"aborts":[],"warnings":["Interruption for unknown channel dropped"]}. Every PR test uses raw ids ('chan-loop', 'chan-e2e', stub registry resolving 'chan-live'), so the suite cannot see it.

   • Fix: explicit surface↔relay id translation at the module boundary, both directions; one test with a registered discord:g:c channel.

Major

3. Provider-timeout aborts are booked as deliberate cancels and never close the activation. (Astra) membrane emits aborted with reason: 'user' | 'timeout' | 'error' (origin/main src/membrane.ts:2932-2937, 3943-3950); the PR's aborted case stamps errorType: 'abort' unconditionally (framework.ts:5168-5174), bypassing the streak/failures.log/marker for adapter deadlines too. The bridge does not handle inference:exhausted, and inference:aborted only comes from abortInference, so a timed-out stream leaves activation_start unpaired on the wire. Probe: healthFailures:0, wire:["activation_start"]. framework.stop() also calls cancelStream directly (:955) with no aborted trace.

   • Fix: gate the bypass on event.reason === 'user' (or on an abortInference-originated flag); close activations idempotently on every terminal, using saved activation identity.

4. Interruptions after completion or after a tool-round post do nothing. (Astra; conhost contrast Claude) The module only calls abortInference, which returns false for an idle agent (agent.ts:783-797). Voice lags text, and connectome posts prose at round boundaries, so the common case is "the message is already on Discord and the stream has moved on": the full unspoken reply stands, context is untouched, and the header's claim that "context and posted message keep the words … spoken" (relay-client-module.ts:11-15) is false. test/inference-trace-channel.test.ts:287-315 enshrines keeping the full live post. The e2e sends its interruption after runUntilIdle and asserts only the spy call, not an effect. conhost's module edits the posted message and drops a context note (tts-relay-module.ts:441-474); discord-mcpl #28 sends a voiced/unvoiced receipt.

   • Fix: reconcile against mcpl:speech-routed message ids regardless of stream state, or adopt the receipt design.

5. Abort-drain window corrupts the successor stream's bookkeeping. (Astra) The new await turnSpeechChain + routeSpeech in the aborted case (framework.ts:5123-5129) runs after cancelStream set the agent idle, so a successor can start. The streamId !== myStreamId branch calls onInferenceEnded and breaks without settling; finally then calls stopTyping() unconditionally (:5285) and leaves the predecessor's pendingAssistantBlocks (old tool_use) under the agent name for the successor to flush. Probe: pending:[["a",[{"type":"tool_use","id":"old-tool"}]]], typingStops:1, gateEnds:2, settles:[]. Orphaned tool_use without tool_result is a 400 on the next request.

   • Fix: generation-owned state (main's ownsPhysicalStream/turn token pattern); settle before releasing ownership.

6. Divergence ≠ "all new": the dedupe heuristic re-commits already-committed rounds. (Astra; Claude probe agrees) undeliveredSuffix returns the whole report on the first mismatch (prose-segments.ts:75-79) and both the context commit and the channel post use it (framework.ts:5103-5113). A module that (re)connected mid-activation starts its window at the first chunk it saw; a whole-activation client then legitimately reports "Beta Gamma" against committed "Alpha\nBeta" → diverges → Beta committed and posted twice. Probe: committed:"Alpha\nBeta", toCommit:"Beta Gamma". test/prose-segments.test.ts:83-89 and inference-trace-channel.test.ts:347-357 encode the assumption.

   • Fix: correlate reports to block/activation offsets, not first-differing-character.

7. Default agents: all + one channel → the wrong agent gets aborted. (Astra) AF wakes every agent on a channel message by default; activeAgentByChannel keeps only the last inference:started (relay-client-module.ts:222-224) while streamedText accumulates per channel across agents under one botId (:653-666). Probe: Alice speaks, her interruption prefix-matches, abortInference('bob', {keepText:'Alice'}). Alice's words land in Bob's context.

   • Fix: one active agent per relay identity, or activation-aware addressing.

8. Two-window staleness guard drops legitimate lagging barge-ins and accepts repeated-prefix stale ones. (both; opposite halves) Each block_start erases the block window (:656-659); the activation window has no separators ('Round oneRound two'). A client still voicing block B when generation is in block C reports "Beta…" → no prefix match → "Stale interruption dropped" (probe: aborts:[]); the barge-in is silently ignored exactly when speak-while-acting makes it likely. Conversely an old "Hello" report aborts a new "Hello, again" turn on the same channel (probe: keepText:"Hello"). The incoming timestamp is discarded (:503-509).

   • Fix: bounded per-block records + timestamp correlation; treat ambiguous legacy reports conservatively.

9. Identity defaults defeat the relay's duplicate-speech suppression. (both) userId/username default to botId (:241-245). The relay turns a Discord message into bot_message_posted (not voiced) only when author id/username matches a connected bot (relay src/index.ts:104-119,143-146); a connectome agent posts through discord-mcpl's bot user, so with the defaults voice clients receive the streamed chunks and then the ordinary channel_message for the same reply. Probe: suppressedAsConnected:false. The config comment calls userId a mention-resolution concern. Inherited: conhost's module has the same weakness (tts-relay-module.ts:305-309).

   • Fix: require/resolve the posting bot's Discord identity or a discordName mapping; document the suppression role.

Minor

10. emitBlocks: true is a global stream-contract change, undisclosed as one. (Claude; Astra concurs) agent.ts:676 flips it for every agent, module or not; the PR body says hosts without the module are unaffected. It is nevertheless the most valuable line here: main agent.ts:771 is still false, so inference:content_block never fires on 0.13.0 and conhost's tts-relay-module.ts:398-418, tui.ts:1694, and web-ui-observers.ts:269 block lanes are dead today. Ship it alone, with a changelog fragment.
11. Backpressure drops terminal frames on a healthy socket. (Astra) :637-643 drops any message over MAX_BUFFERED_BYTES, including activation_end; heartbeats keep the watchdog happy, so the client holds a half-open activation indefinitely. Probe: wire:["activation_start","chunk"].
12. Asterisk-only suffix is committed/posted. (Astra) Comparison skips * but the tail uses trim(): undeliveredSuffix('*grins*','*grins*') → '*', '**Hello**' → '**' (my re-probe). Normalise the remainder with the same predicate.
13. Two clients for one relay identity, and no decision recorded. (Claude) conhost main still constructs its own TtsRelayModule (src/index.ts:400-401); running both against one botId is fatal by design (relay state.ts:54 + module :399-410). The PR needs to say whether it replaces the host module. Relay-side inherited bug found by Astra while probing: handlers/bot.ts:91-95 calls unregisterBot(botId) when the old socket closes, deleting the replacement that registerBot just stored (probe: newRegisteredAfterOldClose:false) — worth an issue on melodeus-tts-relay.
14. Channel-less interruption fallback is unreachable (both): the relay requires a string channelId (handlers/tts.ts:356-359); test/voice-relay-client.test.ts:387 enshrines dead code.
15. Doc slips (Astra): trace.ts JSDoc says ten channel-bearing traces; there are nine. "Pinned locus, else triggering channel" is wrong when a registry exists without a pin (inferenceTraceChannel deliberately returns undefined, framework.ts:5784-5786).

Verified fine

• PR head builds clean and passes 454/454 on its own base with the real relay pinned (PR body: 453 + 1 skip); e2e runs against melodeus-tts-relay@ec8f0f1.
• types.ts is a faithful mirror of relay types.ts@ec8f0f1 (renames, HeartbeatMessage, GuildMemberInfo only). ws ^8.18.0 already an AF dependency at base and main.
• Reconnect/backoff/watchdog: timers cleaned on stop and on replaced-connection (Astra timer probe: remaining:0); handshakeTimeout:10000; ±20 % jitter; stale-socket guards; no token in logs.
• Duplicate-abort keying: cancelStream does not bump streamId; store-after-success keeps the first keepText (probe kept:[["a:7","Hello"]]).
• visible = blockType === 'text' matches membrane's native and XML emitters. Ordinary error retries do pair failed with a fresh started.
• Relay facts: heartbeat 2 s; replacement close 1000 "Replaced by new connection"; botId mismatch dropped; interruption requires connected bot + string channelId.
• Fact-block audit: Astra confirmed 7/7 and corrected one prompt claim — the trace test does install a registry stub (inference-trace-channel.test.ts:107-124), it just resolves to a raw id.

What to salvage

1. emitBlocks: true + changelog fragment, own PR (fixes conhost/TUI/webui block lanes on main).
2. Deliberate-cancel bypass keyed on reason === 'user' (+ [inference-failed] marker suppression), own PR.
3. Decide the voice architecture explicitly (host TtsRelayModule vs discord-mcpl #28 vs a framework relay client) before any more relay-client code; if a framework client is wanted, it consumes routed deltas, translates ids, and reports receipts.

🔴 CHANGES REQUESTED

Reviewers: Claude (Fable 5.1, Claude Code) + Codex (GPT-6 Astra), independent passes merged; every Astra citation re-verified against source or by probe.

Method

Codex GPT-6 Astra (gpt-6-astra, xhigh, codex-cli 0.153.4, read-only sandbox, no network): 14 min wall, 84 commands, 3.38 M input (3.19 M cached), 23.1 k output (8.0 k reasoning); 11 findings, 11/11 citations re-verified, severities kept except #2 raised Major→Blocker. Probes ran in-memory against the built dist/ (no Chronicle stores, no sockets). Found by both: 1, 2, 8, 9, 14. Astra only: 3, 4, 5, 6, 7, 11, 12, 15, relay unregister race. Claude only: 10, 13, the discord-mcpl #28 / Spec 14.3 supersession context, and the build+e2e run. Astra could not reach GitHub (PR body supplied) or run the test suite (read-only sandbox).

— Reviewed by Claude Fable 5.1 via Claude Code and GPT-6 Astra via OpenAI Codex, merged and re-verified.

[slimepriestess]

Second reviewer's note, at head 02381b2, after the 9/8 dual review. I did not re-run the whole pass — it was already two independent readers with every citation re-verified — but I checked its load-bearing claims against current main (fa95817+) first-hand before writing this:

• Superseded, three ways — confirmed. main stamps channelId on the inference traces itself (src/types/trace.ts:50/97/112/240, from typingChannel in framework.ts:6695/6729/6742); connectome-host carries its own TtsRelayModule for the same /bot endpoint; and discord-mcpl's voice path (merged 9/7) voices routed prose directly and reports a status: 'interrupted' receipt (src/voice.ts:121/237) rather than aborting inference. This PR predates all three and references none.
• Inert on production — confirmed at the source: the module forwards event.channelId verbatim as the relay's channelId (relay-client-module.ts:223-224, 503-533), and what main puts there is the registry's composite id, not the raw snowflake the relay's voice clients subscribe by. Every interruption lands in the "unknown channel dropped" branch.
• Not mergeable — GitHub reports CONFLICTING; the 9/8 per-hunk disposition (three hunks needing redesign, not resolution) stands.
• Salvage item 2 is already in flight: "user abort is a deliberate cancel, not a failure — no streak, no [inference-failed] marker" is exactly agent-framework fix: user interrupt is a cancellation, not a failure (+ honest no-locus marker, loud gate-buffer drop) #134 (Lari), reviewed and with its one blocker's fix ready at c23ce76. Nothing to extract from here for that.
• Salvage item 1 still stands: main sets emitBlocks: false (src/agent.ts:771) while framework.ts:6734 carries a case 'block' consumer, so the block lane is dead on every host today. That is a real, small, separate PR — with the disclosure the 9/8 review asked for (it is a stream-contract change for every agent, not a module flag).

Recommendation to the maintainers: close this PR as superseded, with thanks — the diagnosis in the body (zero-retry default, the relay's need for block boundaries, barge-ins must not page as hard-down) was right and each of those has since landed or is landing by another route. If nickmahdavi wants to open the emitBlocks PR, that's the natural credit; otherwise I'm glad to open it citing this PR and its author as the origin. The architecture question the review closes on (host relay module vs. discord-mcpl voice vs. a framework relay client) is a decision to record before any more relay-client code, not something a rebase settles.

— Weft (Claude, via Ra's account, disclosed per convention)

[Anarchid]

(closing as superceded per review above)