Add OpenRouter as a native speech-to-text provider

[MyButtermilk]

Summary

Add a dedicated Scriber STT route for OpenRouter, separate from the existing OpenRouter use in summarization. The first curated/default model should be:

microsoft/mai-transcribe-1.5

The route should support Live Mic finalization and direct file/YouTube transcription, use streaming multipart as the preferred wire format, retain OpenRouter's documented base64 JSON schema as a narrowly controlled compatibility fallback, and never silently replay an ambiguous billable transcription request.

This issue should build on the reusable provider transport from #16 so warm OpenRouter calls can benefit from DNS/TCP/TLS connection reuse.

Motivation

Scriber already stores an OPENROUTER_API_KEY for text-generation workflows, but it cannot currently use OpenRouter's dedicated speech-to-text endpoint. A native STT route would provide:

• a second network path to microsoft/mai-transcribe-1.5 in addition to direct Azure Speech;
• one OpenRouter credential for multiple discoverable STT models;
• a practical way to benchmark direct Microsoft access against OpenRouter using identical audio;
• future access to Whisper and other transcription-only models without adding a bespoke adapter for every host.

OpenRouter exposes a dedicated /api/v1/audio/transcriptions endpoint and an STT model catalog filtered with output_modalities=transcription [1]. microsoft/mai-transcribe-1.5 is currently available through that route [2]. OpenRouter's current API reference documents base64 JSON as the canonical request schema, while its launch documentation also confirms support for OpenAI-style multipart/form-data uploads [3]. This makes multipart-first with a narrow JSON fallback a useful compatibility strategy.

OpenRouter currently forwards the MAI model to Azure, so this route must be treated as an alternative ingress path, not assumed to be faster than Microsoft direct. The issue should include an A/B benchmark rather than a hard-coded preference.

Proposed provider identity

Use a distinct provider key so STT behavior is not coupled to summarization routing:

openrouter_stt

Suggested configuration:

SCRIBER_OPENROUTER_STT_MODEL=microsoft/mai-transcribe-1.5
OPENROUTER_API_KEY=...

Reuse the existing OpenRouter API-key setting, but expose the transcription provider and model independently in the Settings UI.

Proposed implementation

1. Add a focused adapter

Create a module such as:

src/openrouter_stt.py

Suggested public boundaries:

async def transcribe_with_openrouter_stt(...): ...

class OpenRouterTranscribeSTTService(STTService): ...

The service can follow the existing buffered Azure MAI pattern for Live Mic:

• append PCM frames to the existing bounded spool abstraction;
• finalize once on the terminal frame;
• emit one final TranscriptionFrame;
• support the silent-recording skip contract;
• close/reset all temporary streams on success, failure, cancellation, and destruction.

The direct file path should pass a seekable file object to the same transport helper rather than loading the whole file into memory.

2. Use multipart as the fast path

Preferred request:

POST https://openrouter.ai/api/v1/audio/transcriptions
Authorization: Bearer <key>
HTTP-Referer: https://scriber.local
X-OpenRouter-Title: Scriber

Suggested multipart fields:

file=<seekable audio stream>
model=microsoft/mai-transcribe-1.5
language=de            # omit for auto detection
temperature=0

Only send optional OpenAI-style fields such as response_format after they have been verified against the OpenRouter STT endpoint and selected model. Do not send timestamp or diarization fields universally because support is model/provider-specific.

Important transport requirements:

• pass the file object directly to aiohttp.FormData;
• let aiohttp generate the multipart boundary and Content-Type;
• begin uploading without creating a complete base64 or JSON copy;
• preserve stream position and close ownership explicitly;
• integrate with the shared session from Reuse provider HTTP sessions and add connection-level STT diagnostics #16.

3. Retain a narrow base64 JSON fallback

Compatibility request:

{
  "model": "microsoft/mai-transcribe-1.5",
  "input_audio": {
    "data": "<base64 raw audio bytes>",
    "format": "wav"
  },
  "language": "de",
  "temperature": 0
}

Fallback is allowed only when the server explicitly rejects the multipart wire format, for example:

• HTTP 415;
• HTTP 400 or 422 with a bounded error body clearly referring to multipart, content type, input_audio, or request-body format.

Do not change wire format after:

• authentication or permission errors;
• invalid/unknown model errors;
• quota/payment/rate-limit errors;
• generic provider 5xx responses;
• connect/read/total timeouts;
• cancellation or an ambiguous connection loss after upload begins.

The JSON fallback should have an explicit maximum input size and bounded memory behavior. If it cannot be streamed safely in the first implementation, reject oversized fallback inputs rather than allocating an unbounded base64 string.

4. Enforce no ambiguous replay

This is a non-idempotent, potentially billable POST. Set application-level retries to zero.

A request that may have been accepted must surface an error and require an explicit user retry. Do not automatically fail over from OpenRouter to direct Azure, or from Azure to OpenRouter, after an ambiguous timeout or connection reset.

Safe transport recovery before request bytes are committed may remain the responsibility of the HTTP stack, but Scriber must never issue a second application POST silently.

5. Parse the OpenRouter response and bounded diagnostics

Expected success shape:

{
  "text": "...",
  "usage": {
    "seconds": 9.2,
    "cost": 0.000508
  }
}

Requirements:

• return text through the existing transcription boundary;
• preserve the structured response for job metadata where appropriate;
• optionally record the X-Generation-Id response header for support diagnostics;
• never log audio, transcript text, the API key, authorization headers, or the base64 payload;
• limit and redact provider error bodies using Scriber's existing error helpers.

6. Add model discovery

Query:

GET https://openrouter.ai/api/v1/models?output_modalities=transcription

Implementation ideas:

• cache the filtered catalog for a bounded TTL;
• always retain the configured model even if catalog refresh temporarily fails;
• expose only models whose output modality includes transcription;
• provide microsoft/mai-transcribe-1.5 as the curated default;
• keep a free-text override for newly released models;
• do not couple STT discovery to summarization model lists.

A minimal first release may ship only the curated MAI model, but the adapter and settings contract should not prevent dynamic discovery later.

7. Integrate through Scriber

Likely touch points:

• src/config.py
  • API-key map entry for openrouter_stt;
  • provider label;
  • default/configured STT model.
• src/runtime/provider_dependencies.py
  • register the adapter if required by packaged import checks.
• src/core/provider_capabilities.py
  • supports_live_streaming=False;
  • supports_direct_file_upload=True;
  • injects_immediately_in_live_mode=False;
  • no diarization or word-timestamp claims unless verified;
  • supports_five_hour_meeting=False until a long-file route is proven.
• src/pipeline.py
  • Live Mic buffered service;
  • direct file/YouTube helper;
  • frozen execution route/model handling.
• frontend settings, shared API types, German/English translations, provider icon, and validation gates.
• job persistence and replay-safe error categorization.

8. Keep custom vocabulary honest

Do not map Scriber's CUSTOM_VOCAB to an unsupported universal prompt field. OpenRouter supports provider-specific options, but they must be enabled only for a model/provider combination whose behavior is verified and documented.

Direct Microsoft MAI may therefore remain the richer route for Azure-specific phrase-list behavior, while OpenRouter provides portability and an alternative ingress path.

Non-goals

• Automatically choosing OpenRouter because it is assumed to be faster.
• Replacing the direct Azure MAI provider.
• Pretending OpenRouter is a realtime streaming STT service.
• Claiming diarization, word timestamps, or five-hour meeting support without evidence.
• Cross-provider automatic retries.
• Reusing summarization model routing or fallback rules for audio transcription.

Acceptance criteria

• OpenRouter STT appears as an independent provider in Settings.
• The existing OpenRouter key can authenticate the STT route without changing summarization behavior.
• microsoft/mai-transcribe-1.5 is the default model and can be overridden.
• Live Mic produces one committed final transcript through OpenRouter.
• File and YouTube workflows can use direct OpenRouter upload where the input is within the supported boundary.
• Multipart succeeds without constructing a complete base64/JSON copy.
• JSON fallback occurs only after an explicit multipart-format rejection.
• Authentication, model, quota, rate-limit, generic provider, timeout, and cancellation failures produce exactly one application attempt.
• Cancellation aborts the active upload and closes temporary resources.
• Model discovery returns only transcription-capable models and degrades safely when unavailable.
• Logs and support bundles contain no key, audio, transcript, filename, or base64 payload.
• Existing Azure MAI behavior and tests remain unchanged.

Suggested tests

Request-shape tests

• Multipart contains the correct file, model, optional language, and deterministic temperature.
• Auto language omits the language field.
• The file body is consumed incrementally rather than read fully before request start.
• JSON fallback uses raw base64 data without a data-URI prefix and reports the correct format.

Fallback/retry matrix

• 415 -> exactly one JSON fallback.
• Format-specific 400/422 -> exactly one JSON fallback.
• 401, 402, 404, 429, generic 500/502/503 -> no fallback and no retry.
• connect timeout, read timeout, connection reset after upload start, cancellation -> no second POST.

Integration tests

• Exact known transcript from a fixed WAV/MP3 fixture.
• Shared connection reuse with Reuse provider HTTP sessions and add connection-level STT diagnostics #16: warm second call reports connection reuse.
• Live terminal frame emits one final result and then closes cleanly.
• Direct file route retains the frozen model even if settings change mid-job.
• Packaged runtime import and frontend provider/type gates pass.

Benchmark plan

Compare direct Azure MAI and OpenRouter MAI using the exact same encoded audio bytes:

• one cold call and at least ten warm calls per route;
• alternating A/B order;
• identical model, language, and local preprocessing;
• no post-processing or automatic retries;
• median and p95 for upload start, provider response, and committed text;
• transcript equality/quality check;
• connection-created versus connection-reused evidence.

The outcome should inform the UI description or documentation, not silently change the user's selected route.

References

1. https://openrouter.ai/docs/guides/overview/multimodal/stt
2. https://openrouter.ai/microsoft/mai-transcribe-1.5
3. https://openrouter.ai/blog/announcements/announcing-audio-apis
4. Speed up OpenRouter dictation by 22% without forcing IPv4 DevEmperor/DictateKeyboard#210