Skip to content

feat(audio): Add managed TTS engine service with OpenAI /v1/audio/speech compatibility #11

Closed

Description

@spelech

🎯 Objective & Overview

Integrate a dedicated, lightweight Text-to-Speech (TTS) engine service—primarily Kokoro-FastAPI (CPU/GPU, Apache 2.0, 82M params) or AllTalk TTS—into LocalLLMServerManager. Provide full process lifecycle management and an OpenAI-compatible /v1/audio/speech proxy.


📂 Files & Components to Touch

  1. Configuration (LocalLLMServerManager.Shared/Models/AppSettings.cs):
    • Add:
      • string AudioEngineExecutablePath = "" (Path to python script, binary, or docker command)
      • string AudioEngineUrl = "http://127.0.0.1:8880"
      • string PreferredAudioVoice = "af_heart"
  2. Process Management (Services/AiEngineManager.cs & Interfaces/IAiEngineManager.cs):
    • Add Task<bool> StartAudioEngineAsync(string executablePath, ILogger logger)
    • Add Task<bool> StopAudioEngineAsync(ILogger logger)
    • Add Process? AudioProcess { get; }
  3. Endpoints (Endpoints/EngineEndpoints.cs & Endpoints/ModelProxyEndpoints.cs):
    • POST /api/audio/start — Spawns the audio engine process or starts the docker container.
    • POST /api/audio/stop — Gracefully stops the process.
    • POST /v1/audio/speech — Proxies OpenAI-format speech requests to AudioEngineUrl/v1/audio/speech.
    • GET /api/audio/voices — Returns list of available voice profiles.
  4. Tool Discovery (Services/ToolDiscoveryService.cs):
    • Auto-detect local Kokoro/AllTalk installations and Docker container availability.

🔌 API Contract

POST /v1/audio/speech (OpenAI Compatibility Standard)

{
  "model": "kokoro",
  "input": "The quick brown fox jumps over the lazy dog.",
  "voice": "af_heart",
  "response_format": "mp3",
  "speed": 1.0
}

Response: 200 OK with binary audio/mpeg or audio/wav stream.


🛠️ Step-by-Step Implementation Guide

  1. Update AppSettings.cs with the new audio configuration properties.
  2. Implement start/stop process handlers in AiEngineManager.cs.
  3. Add the proxy route in ModelProxyEndpoints.cs streaming chunked audio responses directly back to the HTTP client.
  4. Update UI settings in SettingsTabControl.axaml to display the Audio Engine status, path, and test voice synthesizer.

🧪 Verification

  • dotnet build
  • dotnet test --filter Category!=LiveExternal
  • npm run lint & npx tsc --noEmit

Activity

  1. google-labs-jules commented on Aug 23, 2026

    @google-labs-jules
    Contributor

    Jules is on it. When finished, you will see another comment and be able to review a PR.

  2. google-labs-jules commented on Aug 23, 2026

    @google-labs-jules
    Contributor

    Ready for a review! A PR has been created.

  3. spelech commented on Aug 24, 2026

    @spelech
    OwnerAuthor

    Resolved in v3.7.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions