Research: Cloudflare Agents voice API as native voice layer for aegis-oss agents

[stackbilt-admin]

Overview

Cloudflare's @cloudflare/voice package (part of the Agents platform, currently Beta) provides a complete voice pipeline — STT, LLM turn handling, TTS, and conversation persistence — built on Durable Objects. Evaluate how this fits as an optional voice capability for aegis-oss agents.

Docs: https://developers.cloudflare.com/agents/api-reference/voice/

---

What the API provides

// Server-side: extend a DO with voice
export class MyAgent extends withVoice(AgentBase) {
  transcriber = new WorkersAIFluxSTT(env.AI);
  tts = new WorkersAITTS(env.AI);

  async onTurn(transcript: string, ctx: TurnContext) {
    // plug into aegis-oss memory + dispatch here
    return this.kernel.dispatch(transcript);
  }
}

// Client-side: React hook
const { status, transcript, startCall, endCall } = useVoiceAgent(agentUrl);

Key capabilities:

• Binary PCM audio over WebSocket, sentence-chunked streaming TTS
• Interruption handling (user speech aborts in-flight LLM/TTS)
• SQLite-backed conversation history via DO storage
• Hooks: afterTranscribe, beforeSynthesize, onInterrupt, lifecycle
• Workers AI STT/TTS (free tier) or Deepgram/ElevenLabs/Twilio third-party

---

Design questions for aegis-oss

• Composition model: Should withVoice wrap the agent's DO class, or should aegis-oss expose a VoiceCapability mixin that delegates to CF's mixin internally?
• Memory integration: The afterTranscribe hook is the natural injection point for aegis-oss memory context. How do we surface this cleanly?
• onTurn → kernel dispatch: onTurn returns a string/stream. Does this map 1:1 to the aegis-oss executor interface, or do we need an adapter?
• Packaging: Core bundle vs. optional @aegis-oss/voice addon package?
• Provider abstraction: Should aegis-oss abstract over CF's STT/TTS providers, or pass through CF's interface directly?
• Framework-agnostic client: VoiceClient (non-React) may matter for agents used outside browser contexts

---

Opportunity

The CF voice API aligns closely with aegis-oss's DO-native architecture. Shipping voice as a first-class optional capability would differentiate aegis-oss from other agent frameworks that require external voice infra.

---

Related

• Internal AEGIS daemon integration: Stackbilt-dev/aegis#590
• CF Agents platform: https://developers.cloudflare.com/agents/

[stackbilt-admin]

Design update: layer model + non-goal

After further review, adding two clarifications that should constrain the OSS design:

Non-goal: Cloudflare lock-in

The Cloudflare Voice integration is a first native runtime target, not the core execution model. aegis-oss must define provider-neutral contracts for sessions, memory injection, tool dispatch, streaming output, cancellation, and observability.

Cloudflare Agent / withVoice(Agent) belongs in an adapter package or runtime boundary. The core AEGIS kernel must remain executable from CLI, daemon, HTTP, WebSocket, MCP, and future runtimes without requiring Claude Code, Cloudflare Agents, or any single harness.

Required core abstraction: AegisExecutorPort

All interface adapters (voice, chat, CLI, MCP, scheduled runner) must consume a single port rather than calling executor internals directly:

export interface AegisTurnInput {
  sessionId: string;
  userId: string;
  text: string;
  signal?: AbortSignal;
  context?: Record<string, unknown>;
}

export type AegisTurnEvent =
  | { type: 'text.delta';   text: string }
  | { type: 'tool.call';    name: string; args: unknown }
  | { type: 'tool.result';  name: string; result: unknown }
  | { type: 'memory.write'; patch: unknown }
  | { type: 'warning';      message: string }
  | { type: 'done' };

export interface AegisExecutorPort {
  dispatch(input: AegisTurnInput): AsyncIterable<AegisTurnEvent>;
  cancel?(sessionId: string): Promise<void>;
}

VoiceSession.onTurn() iterates AegisTurnEvent and pipes text.delta events into the CF TTS pipeline. It does not call Claude streaming internals. Neither does the CLI daemon, the WebSocket handler, or the MCP bridge. The port is the only seam.

Layer model

aegis-oss core
  intent model, memory contracts, tool dispatch contract
  AegisExecutorPort + AegisTurnEvent
  policy / governance hooks, observability

runtime adapters (implement/wrap the port)
  Cloudflare Agent / DO adapter
  Claude Code harness adapter
  CLI / local daemon adapter
  HTTP / WebSocket adapter

interface adapters (consume the port)
  voice (VoiceSession)
  text chat (ChatSession)
  webhook (ARGUS)
  MCP bridge
  scheduled autonomous runner

Full spec: voice-integration-spec wiki + harness-independence wiki