From 80f932cedb6ea2f7a08a3e8753269304f80fad05 Mon Sep 17 00:00:00 2001 From: Andrew Anderson Date: Wed, 18 Nov 2026 07:33:38 -0500 Subject: [PATCH] =?UTF-8?q?=E2=9C=A8=20say.tone:=20play=20the=20listening?= =?UTF-8?q?=20tone,=20never=20behind=20speech?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New tool say.tone plays a short chime into the meeting. It is not speech: it is dropped (played:false) when a say is in progress, the bot has no voice, the bot refuses it or the platform has no tone (Discord, companion). On Vexa it is requested through the speak route with the reserved line the bot recognises and never waits for the sound. Signed-off-by: Andrew Anderson --- README.md | 2 +- docs/contract.md | 1 + scripts/smoke.mjs | 4 ++-- src/adapter.ts | 7 +++++++ src/mcp.ts | 16 ++++++++++++++++ src/service.ts | 14 ++++++++++++++ src/vexa/adapter.ts | 37 +++++++++++++++++++++++++++++++++++++ test/e2e-vexa.test.ts | 42 +++++++++++++++++++++++++++++++++++++++++- 8 files changed, 119 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 24dcd7e..d08a683 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ Quorum is a child process inside the hive container, like the `spektacular` bina 1. Hive writes a fresh random bearer token to a file at boot, and the tenant's Vexa API key (and, if used, the Discord bot token) from its keystore to files. 2. Hive starts `node quorum.mjs serve` with the `QUORUM_*` variables below and supervises it. `GET /healthz` answers `{"ok":true,...}` once it is listening. -3. Hive is the MCP client: `POST http://127.0.0.1:8765/mcp` with `Authorization: Bearer ` (streamable HTTP, stateless, JSON responses). Tools: `meeting.join`, `meeting.leave`, `meeting.status`, `transcript.read`, `chat.post`, `say`, `say.prewarm`, `meeting.delete`. Every result is `{ok:true,...}` or `{ok:false,error:{code,message}}` in `structuredContent` and as JSON text. +3. Hive is the MCP client: `POST http://127.0.0.1:8765/mcp` with `Authorization: Bearer ` (streamable HTTP, stateless, JSON responses). Tools: `meeting.join`, `meeting.leave`, `meeting.status`, `transcript.read`, `chat.post`, `say`, `say.tone`, `say.prewarm`, `meeting.delete`. Every result is `{ok:true,...}` or `{ok:false,error:{code,message}}` in `structuredContent` and as JSON text. 4. Hive follows each meeting on `GET /events?meeting_id=` (Server-Sent Events, same bearer): `segment`, `chat` and `state` events, one JSON object per `data:` line, `id:` = `seq`. Reconnect with `after_seq=` to replay what was missed. `GET /events` without `meeting_id` carries every meeting, including Discord `/hive join` requests. 5. On SIGTERM quorum leaves Discord channels and exits. Vexa bots keep running until hive calls `meeting.leave`; a restarted quorum that is asked to join the same meeting adopts the running bot. diff --git a/docs/contract.md b/docs/contract.md index 9588dbf..80c5c82 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -30,6 +30,7 @@ All tools return `{ok: true, ...}` or `{ok: false, error: {code, message}}`, as | `transcript.read` | `{meeting_id, after_seq?}` | `{segments:[Segment], next_seq}` final segments only, oldest first; `next_seq` is the cursor to pass back as `after_seq` | | `chat.post` | `{meeting_id, text, confirm?}` text max `QUORUM_CHAT_MAX_CHARS`=2000 | `{posted_at, delivery}`. `delivery` is `confirmed` when the platform reports the line in the meeting, else `unconfirmed`. Without `confirm` quorum does not wait: Discord answers `confirmed` (its send returns once the message exists), Vexa `unconfirmed` (the bot only received the line). With `confirm:true` on Vexa, quorum reads the send's delivery record (`GET /bots/{platform}/{native_id}/chat/{request_id}`) every `QUORUM_CHAT_CONFIRM_POLL_MS`=500 for up to `QUORUM_CHAT_CONFIRM_TIMEOUT_MS`=30000 (the Teams bot alone may wait 10 s for the chat composer after admission): `confirmed` once the bot reports it posted, `platform_unavailable` (message names the bot's reason, for example `no_composer` when the meeting chat is not open to the bot) when it reports the send failed, and `unconfirmed` when no verdict came in time or Vexa has no delivery route. A caller that must know the line was seen (the consent notice) passes `confirm:true` and treats `unconfirmed` as not posted. | | `say` | `{meeting_id, text}` text max `QUORUM_SAY_MAX_CHARS`=1000 | `{queued:true, duration_ms?, first_audio_ms?, synth_ms?, cached?}`. Lines of one meeting play one after another, never over each other. On Discord a line is spoken sentence by sentence: the text is cut at `.`, `!`, `?` or `:` followed by whitespace (a closing quote or bracket stays with its sentence; pieces under 24 characters are joined to a neighbour; text with no such break is one sentence), the sentences are synthesized one after another, and each is queued for playback in order as soon as it is ready, so the first sound comes after one sentence's synthesis. The call resolves once the first sentence is ready (playback is queued behind earlier lines). If the first sentence cannot be synthesized the call answers `upstream_error` and nothing of that line is spoken; a later sentence that fails is skipped and logged, and the sentences after it still play. Each sentence's clip gets a 5 ms fade in and out and is cached by voice and text. `duration_ms` (whole milliseconds) is the line's real audio length when known: on Discord only when every sentence was ready by the time the call resolved (a one-sentence line, or a line whose sentences were all cached), on Vexa when the bot reports it. `first_audio_ms` is the time from the request to the first audio: on Discord until the first sentence was handed to the playback queue; on Vexa the bot's own `first_audio_ms` when the playback record has it, else the time until quorum first read the record as `speaking` or `spoken` (late by at most one poll), present only when the line was reported `spoken`. `synth_ms` is the synthesis time before the first audio: on Discord 0 when the whole line came from the cache, else equal to `first_audio_ms`; on Vexa the record's `synth_ms` when it has one. `cached` says every sentence of the line came from the clip cache (Discord), or is the record's `cached` when it has one (Vexa). On Vexa each say is sent only after the meeting's previous say settled; when the speak answer carries a `request_id`, quorum reads the line's playback record (`GET /bots/{platform}/{native_id}/speak/{request_id}`, statuses `queued`, `speaking`, `spoken` with `duration_ms` and, from a bot that reports them, `first_audio_ms`, `synth_ms` and `cached`, `failed` with `reason`) every `QUORUM_SAY_POLL_MS`=200 for up to `QUORUM_SAY_CONFIRM_TIMEOUT_MS`=60000 and resolves once it is `spoken` (or with no verdict in time, or no such route). `platform_unavailable` when the bot was started without voice or Vexa refuses speech (404/409, message `speak is not available: `), when the bot reports the line `failed` (message `the line was not spoken: `, for example `mic_muted`), or when `QUORUM_TTS_URL` is unset (Discord); `upstream_error` when the TTS service fails on the line's first sentence (Discord) | +| `say.tone` | `{meeting_id}`; the meeting must be live (else `not_live`) | `{played}`. Plays the listening tone: a short, soft chime, not speech, that tells the room the bot heard its name and is about to answer. It never waits for the sound and never queues behind speech: `played` is `false` when it was dropped on purpose (a `say` is in progress, the bot has no voice, the bot refused it, or the platform has no tone: Discord and the companion). On Vexa the bot synthesizes the chime itself and plays it at once; it is requested through the speak route with the reserved line `[[hive:listening-tone]]`, so a person never has that line spoken. | | `say.prewarm` | `{meeting_id, texts}` `texts` is 1 to `QUORUM_SAY_PREWARM_MAX_LINES`=32 non-empty strings, each at most `QUORUM_SAY_MAX_CHARS` (else `invalid_argument`); the meeting must be live (else `not_live`) | `{queued}`. Gets fixed lines (an introduction, acknowledgements, a goodbye) ready so a later `say` of the same text starts at once. On Discord each line is cut into sentences as `say` does and every sentence not yet cached is synthesized in the background with the meeting's voice, one request at a time; `queued` is the number of sentences sent for synthesis (0 when all were ready or already on their way). A sentence that fails is skipped and synthesized when it is said. Nothing is played. `queued` is 0 on platforms without their own synthesis in quorum: Vexa (its bot synthesizes), the companion, and Discord without `QUORUM_TTS_URL`. | | `meeting.delete` | `{meeting_id}` | `{deleted:true}` erases transcript upstream | diff --git a/scripts/smoke.mjs b/scripts/smoke.mjs index 08c05b5..52d4b05 100644 --- a/scripts/smoke.mjs +++ b/scripts/smoke.mjs @@ -8,8 +8,8 @@ import { execFileSync } from "node:child_process"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; -/** The tools docs/contract.md lists: join, leave, status, transcript.read, chat.post, say, say.prewarm, delete. */ -const EXPECTED_TOOLS = 8; +/** The tools docs/contract.md lists: join, leave, status, transcript.read, chat.post, say, say.tone, say.prewarm, delete. */ +const EXPECTED_TOOLS = 9; const bundle = resolve(process.argv[2] ?? "dist/quorum.mjs"); const withDavey = process.argv.includes("--with-davey"); const dir = mkdtempSync(join(tmpdir(), "quorum-smoke-")); diff --git a/src/adapter.ts b/src/adapter.ts index 08c59da..347d02a 100644 --- a/src/adapter.ts +++ b/src/adapter.ts @@ -25,6 +25,13 @@ export interface PlatformAdapter { * queued for synthesis; a platform that cannot prepare speech leaves this out. */ prewarm?(meeting: Meeting, lines: string[]): Promise; + /** + * Plays the listening tone (say.tone): a short chime, not speech. Resolves true when the bot was + * asked to play it, false when it was dropped on purpose (a line is being spoken, or the + * meeting has no voice). It never waits for the sound and never queues behind speech; a + * platform without it leaves this out. + */ + tone?(meeting: Meeting): Promise; /** Erases the transcript upstream (quorum clears its own buffer afterwards). */ erase(meeting: Meeting): Promise; } diff --git a/src/mcp.ts b/src/mcp.ts index 4119b7a..0b62ae5 100644 --- a/src/mcp.ts +++ b/src/mcp.ts @@ -89,6 +89,20 @@ export const TOOLS = [ required: ["meeting_id", "text"], }, }, + { + name: "say.tone", + description: + "Play the listening tone: a short, soft two-note chime that tells the room the bot heard " + + "its name and is about to answer. Not speech. Returns {played}: true when the bot was " + + "asked to play it, false when it was dropped on purpose (a line is being spoken, the bot " + + "has no voice, or the platform has no tone). Never waits for the sound and never queues " + + "behind speech.", + inputSchema: { + type: "object", + properties: { meeting_id: meetingId }, + required: ["meeting_id"], + }, + }, { name: "say.prewarm", description: @@ -128,6 +142,8 @@ export function callTool(q: Quorum, name: string, args: unknown): Promise> { + return this.run(async () => { + const m = this.meeting(asArgs(raw)); + this.requireLive(m); + const adapter = this.adapter(m.platform); + return { played: adapter.tone ? await adapter.tone(m) : false }; + }); + } + /** Gets fixed lines ready to be spoken at once; a platform that cannot answers queued 0. */ sayPrewarm(raw: unknown): Promise> { return this.run(async () => { diff --git a/src/vexa/adapter.ts b/src/vexa/adapter.ts index 2354843..9d95713 100644 --- a/src/vexa/adapter.ts +++ b/src/vexa/adapter.ts @@ -149,11 +149,20 @@ function durationOf(v: unknown): number | undefined { return typeof v === "number" && Number.isFinite(v) && v >= 0 ? Math.round(v) : undefined; } +/** + * The reserved line the bot plays as its listening tone instead of synthesizing it (the same + * constant is in the bot's listening-tone.ts). It rides the existing speak route, so no contract + * changes; it is never a word a person would have said to be spoken. + */ +export const LISTENING_TONE_TEXT = "[[hive:listening-tone]]"; + export class VexaAdapter implements PlatformAdapter { private readonly echoes = new Map(); private readonly chatSeen = new WeakMap>(); /** Each meeting's last queued say, settled either way: the next say starts after it. */ private readonly sayQueues = new WeakMap>(); + /** Lines being said per meeting, from the request until it settles (the tone yields to them). */ + private readonly saysInFlight = new WeakMap(); /** The last chat line sent without waiting for its delivery, so a leave can wait for it. */ private readonly lastChat = new WeakMap(); @@ -358,7 +367,12 @@ export class VexaAdapter implements PlatformAdapter { */ say(meeting: Meeting, text: string): Promise { const prev = this.sayQueues.get(meeting) ?? Promise.resolve(); + this.saysInFlight.set(meeting, (this.saysInFlight.get(meeting) ?? 0) + 1); const run = prev.then(() => this.speakOne(meeting, text)); + const settled = (): void => { + this.saysInFlight.set(meeting, (this.saysInFlight.get(meeting) ?? 1) - 1); + }; + void run.then(settled, settled); this.sayQueues.set( meeting, run.then( @@ -369,6 +383,29 @@ export class VexaAdapter implements PlatformAdapter { return run; } + /** + * Asks the bot to play the listening tone. Dropped (false) with no request when the meeting has + * no voice or a line is being said: the tone never queues behind speech. The bot plays it at + * once from a pre-rendered clip and drops it itself when its audio is busy; this does not wait + * for the sound, and a failed request is dropped, never thrown. + */ + async tone(meeting: Meeting): Promise { + if (!meeting.voice || (this.saysInFlight.get(meeting) ?? 0) > 0) { + return false; + } + try { + await this.opts.client.speak( + meeting.platform, + meeting.nativeId as string, + LISTENING_TONE_TEXT, + ); + return true; + } catch (err) { + this.opts.log.warn("listening tone not sent", { error: String(err) }); + return false; + } + } + private async speakOne(meeting: Meeting, text: string): Promise { if (!meeting.voice) { throw new QuorumError( diff --git a/test/e2e-vexa.test.ts b/test/e2e-vexa.test.ts index 4ba2306..28ad243 100644 --- a/test/e2e-vexa.test.ts +++ b/test/e2e-vexa.test.ts @@ -638,6 +638,45 @@ describe("chat.post and say", () => { }); } }); + it("plays the listening tone, never behind speech, and drops it when it cannot", async () => { + const id = await goLive(); + // No voice: dropped, nothing sent. + expect(await h.call("say.tone", { meeting_id: id })).toEqual({ ok: true, played: false }); + expect(fake.requests.filter((r) => r.path.endsWith("/speak"))).toHaveLength(0); + const live = h.q.quorum.registry.get(id); + if (live) live.voice = true; + // Idle: one speak request with the reserved line, no status polling. + fake.on("POST", /\/speak$/, { status: 200, body: { status: "queued" } }); + expect(await h.call("say.tone", { meeting_id: id })).toEqual({ ok: true, played: true }); + const sent = fake.requests.filter((r) => r.path.endsWith("/speak")); + expect(sent).toHaveLength(1); + expect(sent[0]?.body).toEqual({ text: "[[hive:listening-tone]]" }); + // A line being said: the tone is dropped and sends nothing. + let release = false; + fake.on("POST", /\/speak$/, { status: 200, body: { status: "queued", request_id: "t1" } }); + fake.on("GET", /\/speak\/t1$/, () => { + const speaking = !release; + return { + status: 200, + body: speaking ? { status: "speaking" } : { status: "spoken", duration_ms: 900 }, + }; + }); + const said = h.call("say", { meeting_id: id, text: "Here is the answer." }); + await until(() => fake.requests.filter((r) => r.path.endsWith("/speak")).length === 2); + expect(await h.call("say.tone", { meeting_id: id })).toEqual({ ok: true, played: false }); + expect(fake.requests.filter((r) => r.path.endsWith("/speak"))).toHaveLength(2); + release = true; + await said; + // A bot that refuses it (not admitted) is a dropped tone, not an error. + fake.on("POST", /\/speak$/, { status: 409, body: { detail: "not admitted" } }); + expect(await h.call("say.tone", { meeting_id: id })).toEqual({ ok: true, played: false }); + // Unknown meeting. + expect(await h.call("say.tone", { meeting_id: "m_missing" })).toMatchObject({ + ok: false, + error: { code: "not_found" }, + }); + }); + it("plays says strictly one after another and reports the clip's real duration", async () => { const id = await goLive(); const live = h.q.quorum.registry.get(id); @@ -1064,7 +1103,7 @@ describe("meeting.leave, meeting.delete and status", () => { }); describe("tool surface", () => { - it("lists the eight tools and rejects unknown ones", async () => { + it("lists the nine tools and rejects unknown ones", async () => { const tools = await h.raw.listTools(); expect(tools.tools.map((t) => t.name)).toEqual([ "meeting.join", @@ -1073,6 +1112,7 @@ describe("tool surface", () => { "transcript.read", "chat.post", "say", + "say.tone", "say.prewarm", "meeting.delete", ]);