Skip to content

Latest commit

 

History

History
490 lines (370 loc) · 56 KB

ARCHITECTURE.md

File metadata and controls

490 lines (370 loc) · 56 KB

Architecture

Utterform uses Tauri 2 as its desktop shell, Rust for all privileged or compute-heavy work, and Svelte with TypeScript for the interface.

Boundaries

  • audio.rs — device discovery, CPAL capture, bounded handoff, RMS envelope, native cutoff, temporary WAV lifecycle, Whisper normalization
  • feedback.rs — synthesized start/stop clicks and distinct post-delivery Done chime through CPAL output, played to completion on outputs that suspend when idle; the productive Done carries a generation the next recording makes stale
  • platform.rs — Omarchy-only native window-decoration policy
  • activation.rs — the single way to reveal the one main window, and which tray gestures ask for it; on macOS the app is an accessory (menu-bar) application for good, so revealing also activates it
  • tray.rs — Tauri's native tray on Windows/macOS, an own StatusNotifierItem on Linux so a left click arrives; both show a red dot while recording and a five-second green dot after successful delivery
  • actions.rs — the prompts Utterform ships with, and how a user's replacement resolves against them
  • transcription/openai.rs — GPT Transcribe and Responses API calls
  • context.rs — what the Context action reads: the selection (Linux), else the clipboard; see Context
  • transcription/codex.rs — text actions on the ChatGPT subscription through the Codex CLI login (experimental); see Text actions on ChatGPT
  • transcription/local.rs — blocking local Whisper inference
  • packaging/cmake/portable-cpu.cmake — the instruction set whisper.cpp is compiled for, and why it is not the build machine's
  • models.rs — curated model catalog, downloads, progress events, and SHA-256 verification
  • cli.rs — what a command line or a second launch asks the running app to do
  • hotkey.rs — a reserved key combination where the OS grants one, routed into the same intent as the command line
  • src/lib/shortcut.ts — reading a key combination off the keyboard in the one spelling hotkey.rs parses
  • typing/ — putting the finished text into the focused window: paste or keystrokes, through wtype/xdotool on Linux and SendInput on Windows
  • output.rs — independent clipboard, file and cursor delivery
  • settings.rs — non-secret JSON settings
  • history.rs — bounded, atomic, device-local text history; local titles, no AI calls
  • secrets.rs — OS credential-store access
  • commands.rs — thin Tauri command boundary; owns the Record → Transcribe → Transform → Deliver events of a batch recording
  • diagnostics.rs — the single diagnostic boundary: event format, levels, ids and references, Failure, redaction (diagnostics/redact.rs), the bounded support report (diagnostics/report.rs), run marker, panic hook and the support actions
  • src/lib/diagnostics.ts — the interface's bridge for failures no command saw
  • src/ — presentation and focused-window keyboard interaction

The frontend never receives an API key or temporary audio path. Network calls originate in Rust.

Recording lifecycle

  1. Resolve the selected device by CPAL's stable device ID, falling back to the system default.
  2. Capture native f32, i16, or u16 samples.
  3. Send small sample chunks over a bounded channel to a WAV writer thread.
  4. Stop capture and finalize the temporary WAV.
  5. Send the WAV to GPT Transcribe, or normalize it to 16 kHz mono f32 for local Whisper.
  6. Optionally transform the transcript with a separate text-model request.
  7. Persist the completed text to local history (if enabled), before external delivery.
  8. Deliver to each selected output independently.
  9. Delete the temporary recording when the artifact leaves scope, unless transcription failed and it was kept.

When a recording fails (0.7.13)

The audio is the one thing that cannot be recreated, so failures keep it.

  • process_recording in commands.rs is the single path from artifact to delivery; finish_recording and retry_recording both enter it. A transient failure (connect, transport, body, HTTP 429/5xx) is retried once after two seconds.
  • Any remaining transcription failure moves the artifact and its request into retained::RetainedRecordings, a bounded queue of five. empty_result and empty_recording are not kept: there is nothing to retry with.
  • retry_recording uses current settings, forces clipboard output and never types at the cursor, since the request came from Utterform's own window. Failure re-queues.
  • Dropping an artifact deletes its file, so whoever takes one must deliver or restore it. Files left at exit are removed by cleanup_stale_recordings on the next start.
  • A stream that is gone sets stream_error; the native watchdog in check_limit ends the batch recording at once. finalize then returns the partial audio with interrupted set instead of an error, unless nothing was captured. Partial audio never earns the Done cue. Live capture keeps its own failure path.
  • start_recording rejects a key that is missing but not a keyring that cannot answer. The frontend no longer gates on its cached hasApiKey: the backend can actually read.
  • notify_if_unattended shows a desktop notification when the main window is not both visible and focused, falling back to revealing it. The frontend calls it from setError and for delivery warnings. The text is shown, never logged.

secrets::reconnect replaces the Linux keyring store after a platform failure, because keyring holds one Secret Service session for the process lifetime and that session ends with the daemon. macOS and Windows have no such session.

Audio files (0.13.0)

A file is turned into what a recording leaves behind and then takes the recording's path. audio_file::decode probes the content with Symphonia (Ogg, Matroska/WebM, MP4/M4A, MP3, WAV, FLAC, AIFF, CAF; AAC, MP3, FLAC, Vorbis, PCM, ADPCM, ALAC) and decodes Opus through symphonia-adapter-libopus, which compiles libopus into the binary with CMake — Symphonia has no Opus decoder of its own. Each packet is mixed to mono and streamed through a small resampler (box average when going down, linear interpolation when going up) into a temporary 16 kHz mono 16-bit WAV from audio::new_temporary_wav, named like a recording so the stale cleanup knows it. The result is an ordinary RecordingArtifact; nothing downstream knows where the audio came from, and dropping it deletes the copy. Undecodable packets are skipped and counted; a cut-off file keeps what came before the cut; more than two hours is refused while decoding.

commands::transcribe_file takes a path, a source (drop, open, command_line, open_with) and the usual ProcessRequest, forces clipboard on and cursor off, refuses while a recording or Live dictation runs or another file is in progress (FileBusy), and calls process_recording with Origin::File. That origin logs mode=file and keeps no copy when transcription fails: the original is still there, so opening it again is the retry.

Files from outside queue in PendingFiles (at most 20): the launch's own arguments, a second instance's arguments resolved against its working directory, and macOS' RunEvent::Opened. cli::files_from reads --file <path>, --file=<path>, file:// URLs and a bare argument that names an existing file — the form "Open with" and the Windows registry use — so utterform toggle keeps meaning what it did. The interface hears file-waiting, takes paths with take_pending_file one at a time while it is free, and also drains the queue once at start. Drag and drop uses the webview's tauri://drag-enter/-leave/-drop events.

Uploads: transcription::openai cuts audio longer than twelve minutes into parts of about ten (split_at_pauses), moving each cut to the quietest 100 ms window in the 30 s before the mark, uploads them in order and joins the texts with a space. A silent part adds nothing; any other failure fails the whole file. Local Whisper takes the whole buffer. RecordingArtifact::whisper_pcm now reads frame by frame and skips resampling at 16 kHz, so a two-hour file costs one buffer of about 460 MB rather than three.

"Open with" registration: Linux's desktop entry lists audio MIME types with Exec=… %F; Windows' NSIS hook writes Utterform.AudioFile into each type's OpenWithProgids and an Applications\utterform.exe entry, under HKCU, and removes them on a real uninstall — Tauri's own fileAssociations is not used because its NSIS macro replaces the type's default program; macOS' Info.plist declares CFBundleDocumentTypes for public.audio with rank Alternate.

Events: file.requested (source, count), file.started (recording, source), file.decoded (codec, source rate and channels, skipped packets, audio and decode ms), file.failed (class, stage start or decode), file.packets_skipped, file.requests_dropped, transcription.split (parts), transcription.part_failed, then the recording's own transcription.* → recording.completed mode=file. Names, paths and extensions of files are never logged.

Background capture & feedback

Capture is owned by Rust/CPAL, not the WebView. Focus changes, minimization, and closing the window to tray do not stop a recording. Explicit Stop processes it, Escape discards it, and tray Quit discards active audio before exiting. No microphone capture starts merely by launching the app. The in-window keys stay focused-window shortcuts; global dictation is a separate path described below.

A backend's buffer under- or overrun report (ErrorKind::Xrun) is a gap in a stream that keeps running — Windows raises it through AUDCLNT_BUFFERFLAGS_DATA_DISCONTINUITY, macOS through a processor-overload listener — and is counted, not fatal. Only an error meaning the stream is gone ends a recording. note_stream_error is the one place that decides.

Pause/resume is an explicit, idempotent set_recording_paused command returning native recording status. CPAL stays open for cross-platform reliability; an atomic gate discards paused callbacks before conversion/allocation/writing. No silent gap is inserted. The OS may continue to show its microphone-use indicator. A native clock excludes paused intervals from elapsed time and the cutoff while keeping session identity unchanged. Stop can consume paused audio; Escape and tray Quit still discard it. The UI serializes pause commands, ignores stale polling responses, and handles a watchdog-completion race without processing twice.

A native watchdog checks the active session every 250 ms and finalizes capture at ten minutes of unpaused recording, even if the WebView is suspended. The completed artifact stays in native state until processing consumes it once. An event starts processing immediately when the WebView is running; status polling catches up after a hidden/suspended window resumes. Session identity prevents an old watchdog from stopping a later recording. The UI reads native elapsed time rather than incrementing a JS timer.

Only a bounded RMS-derived envelope crosses IPC, at most 10 times per second while visible and recording; no raw audio reaches the frontend. The full-window violet/blue ambient field responds to this envelope behind stationary controls. Reduced-motion mode disables field movement. Cues are generated locally, quiet and short, and can be disabled in Settings.

A cue is played to the end rather than for a fixed time. An output device that suspends when idle — Bluetooth, HDMI, most docks — needs a few hundred milliseconds to carry any sound and queues 150-250 ms more, and closing a CPAL stream discards what the device still holds. So every cue carries a 60 ms silent lead-in and a 300 ms silent tail, and the stream stays open until the device has taken all of it; the three-second deadline is a safety net, not the expected wait. Every cue is opened, played, finished and closed on a thread of its own, never on the thread that answered the interface. The start cue's output stream is opened once the microphone is running, because opening a capture stream can reconfigure the device that plays the cue; capture starts immediately and discards until that cue has been played, which is what keeps it out of its own recording and what the recording clock counts from. Every cue outcome is written to the log. A speaker failure still never fails a recording, but it is reported rather than dropped: a start cue that could not be played raises a desktop notification in its place, because a hidden window offers nothing else.

The Done cue is the one cue nothing waits for. finish_recording — batch and live alike — returns as soon as history has been attempted and every requested output has run; the Done cue is started then, on its own thread, through feedback::DoneCues, and the interface is ready for the next dictation the moment the command answers, while the cue and the ribbon's release run on. Only one productive Done exists at a time: each carries the generation it was issued in, issuing the next makes the previous stale, and every real recording start — commands::begin_recording for batch and local, live::start for GPT Live — makes them all stale before its own start cue is scheduled. A stale cue hands the device silence from its next buffer, its thread closes the stream, and the log says it was silenced and when. Making a cue stale is one atomic increment that never waits for the audio thread, the device or the deadline; dropping a Playback handle does not do it. Start, Stop and the cues Settings plays carry no generation and are never cut short. Together with the start cue's own arming rule — capture discards until the start cue has been played — this keeps the tail of the previous Done out of the next recording. processing in the interface still covers transcription, transformation, the history attempt and every requested output; an intent that arrives during it is dropped, never queued, and the first one after the answer starts exactly one recording.

Omarchy is detected only in a Hyprland desktop session with an Omarchy installation/path. Native decorations are disabled before first showing the window. Other desktops, macOS, and Windows retain their standard decorations. No compositor config or automatic Omarchy theme integration is added.

Shared visual identity

src-tauri/icons/app-icon.svg is the single source for the in-app brand (imported by Vite), Settings header, and generated desktop icons. Run npm run icons after changing it. The generator uses Tauri's renderer, copies only desktop assets, and canonicalizes ICNS chunk order for byte-stable regeneration. The app identifier, executable name and storage paths remain unchanged. The tray continues to use Tauri's default window icon.

Settings (since 0.9.0) is one page over the main window, using the same theme tokens and custom SelectMenu as the main controls. Opening motion is disabled under reduced motion; keyboard focus is contained and restored, the search field takes focus on open, and Escape closes an open selector, then a detail page, then the search, then Settings.

Rows and search. src/lib/settings-catalog.ts lists every row with its category (Recording, Transcription, AI, Output, App) and the extra words that find it. The categories are a tablist with arrow-key navigation and roving focus; a non-empty search replaces the category with every matching row across all of them, each labelled with its category. A setting that is not in the catalog can be neither shown nor found, so a new setting starts there. Rows that hold technical detail (local model list, text model id and effort, an action's prompt) open a detail page one level down instead of showing it in front. Explanations live in title tooltips.

No Save. Every control writes at once through change() in App.svelte: the interface updates, then save_settings runs. Text fields pass a 500 ms delay so a write waits for a pause in typing; closing Settings flushes a pending write. Writes are chained on one promise, so an older one can never land after a newer one. A short "✓ Saved" confirms success; a failed write stays as "Not saved" with the backend's message. Values that cannot be stored (a vocabulary term with </>, a language that is not a code) are refused at their field and never reach change(). The dictation key is stored, then registered behind any capture that is still giving the old key back; a key another application holds is reported at the field and stays stored, so a later start or toggle can try it again. Autostart is not a setting: the switch writes the operating system's entry at once and is reset to what autostart_enabled reads back.

Prompts

actions.rs holds the shipped prompts and is the only place their text exists. The interface fetches them through list_built_in_actions rather than keeping a copy, so what Settings shows, what Reset restores, and what a recording runs cannot drift apart.

AppSettings::action_overrides stores only what the user replaced, per action id, with name and prompt independently optional: an action that was renamed still follows a later, better default prompt. A blank replacement resolves back to the default in actions::instructions, so an emptied box cannot fail the recording that used it. Resolution happens in Rust with the settings a recording already loads, which is why the tray, the dictation key and utterform --toggle run the edited prompts without the frontend sending any prompt text.

plain is in the same list with no prompt at all: it is shown in the editor as the action that never reaches a text model, and has nothing to edit.

Text actions on ChatGPT (0.10.0)

AppSettings::text_provider chooses who runs a text action: openai_key (the Responses API with the keyring key) or codex (the user's ChatGPT subscription). Transcription is never affected. transcription::transform dispatches; the instructions resolve the same way for both (openai::action_instructions).

codex.rs has no login of its own. It reads auth.json from $CODEX_HOME or ~/.codex (%USERPROFILE%\.codex on Windows), sends the access token and account id to https://chatgpt.com/backend-api/codex/responses with originator: utterform, and reads the answer from the server-sent-event stream the endpoint insists on (stream: true, input as a message list). An access token within five minutes of its exp, or one the server answers with 401, is renewed once at auth.openai.com/oauth/token under the Codex CLI's public client id. Refresh tokens are single-use, so the renewal is written back to the same file — atomically, mode 0600 — or the CLI would be signed out; before renewing, the file is read again and a token the CLI renewed in the meantime is taken instead. Approach taken from kit (jli-software/kit).

Models come from the CLI's models_cache.json: only visibility: list, in its priority order. codex_model: None means Auto: the first …-luna/…-mini, else the first listed. Effort minimal is sent as low, the Codex models' lowest level.

Decided with Jonas on 2026-11-07: marked Experimental in Settings, and no fallback to the API key — a ChatGPT failure is the usual transform.fallback to the plain transcript with ChatGPT's reason. begin_recording checks the login like a missing key, before anything is said. transform.* events carry provider=openai_key|codex; nothing from auth.json is ever logged or sent to the interface, which gets only codex_status (signed in, model ids and names). open_codex_setup opens one fixed URL.

Context: selected text and clipboard (0.11.0)

The built-in action context (actions::CONTEXT) turns the transcript into an instruction and the selected or copied text into its material. Decided with Jonas on 2026-11-07: a separate action, no per-action switch; plain dictation and every other action never read context. Linux reads the selection; Windows and macOS use the clipboard only, because reaching the selection there means a simulated Ctrl+C / Cmd+C, which interrupts a terminal's running program.

When. begin_recording captures once the microphone is open and before the command returns, so a stop cannot overtake it. context::capture tries the selection (context_selection) and then the clipboard (context_clipboard); blank text is none, and a clipboard whose text matches the fingerprint of Utterform's last delivered text (remember_own_output, a hash held in memory) is skipped. The result waits in CapturedContext keyed by recording id; finish_recording moves it into the request (ProcessRequest::context, serde(skip)), so a kept recording retries with the same material. cancel_recording and the next start clear it.

Linux selection. wl-paste --primary --no-newline --type text under Wayland, xclip -o -selection primary or xsel --output --primary under X11, with stdout read on a thread and the child killed after 800 ms, so an unresponsive selection owner cannot hold the recording. A non-zero exit ("No selection") is none. context_support tells Settings whether a tool is there and which package to install otherwise.

Model input. transcription::transform hands the Context action <instruction>…</instruction> and <material source="selection|clipboard">…</material> (or <material/>) to whichever provider is set; the shipped prompt says the material is data and never instructions. The material is capped at context::MAX_CHARS (20,000 characters, cut on a character boundary). A missing or cut material adds a delivery warning; neither stops the run.

Audio files. An audio file opened while Context is the chosen action (0.13.0) runs as Plain with a note: it has no spoken instruction and nothing was read at its start.

Diagnostics. context.captured (source, characters, truncated) or context.none (which sources were on). The text itself is never logged, like a transcript.

Action per application (0.12.0)

Rule. settings.app_actions holds { app: { id, name }, action } pairs, empty by default. action is a built-in id or custom:<id>. When the dictation key starts a recording (applyIntent → startRecording(true)), the interface asks foreground_app for the application in front, adds it to settings.recent_apps (newest first, eight at most, written by the settings save the start already does) and looks for a rule (lib/app-actions.ts: matched on the id or the name, case-insensitive, so a name typed by hand matches as well; a rule whose action was removed is ignored). A match whose action differs from the window's sets appAction for this one recording: effectiveAction follows it for start_recording, finish_recording and a custom action's prompt, the action menu shows it disabled, its label names the app, and announce_app_action shows a notification. showResult, setError and cancelRecording clear it. A start from the window's button or Space never asks, so the window's choice holds there; live dictation stays Plain and only remembers the app. Removing a custom action removes the rules that pointed at it.

Timing. The query runs after phase is starting, so the existing guards against double starts still hold, and before the microphone opens. On Hyprland it is one request on the command socket (a few milliseconds); spawn_blocking keeps it off the main thread.

Which application (foreground.rs). Utterform itself is never the answer: a window whose process id is Utterform's own yields none.

Platform Source id
Hyprland command socket j/activewindow class, else initialClass
Sway swaymsg -t get_tree -r, the focused leaf app_id, else X11 class
X11 xprop -root _NET_ACTIVE_WINDOW, then WM_CLASS and _NET_WM_PID class (second WM_CLASS string)
Windows GetForegroundWindow → process → QueryFullProcessImageNameW file name without .exe
macOS NSWorkspace.frontmostApplication bundle identifier

The compositor is asked before X11: on Wayland, xprop only sees XWayland windows. GNOME and KDE on Wayland offer no such query; foreground_support says so and Settings disables + App. Child processes are killed after 400 ms. display_name turns an id into what Settings shows (org.mozilla.Thunderbird → Thunderbird, com.mitchellh.ghostty → Ghostty); macOS uses the localized name.

Diagnostics. foreground.read (source, found, elapsed_ms) for every query and recording.app_action (action) when a rule takes over. The application is never logged: which program someone dictates into says as much as a window title.

Microphone in the tray (1.1.0)

One setting, two places. The tray's Microphone submenu writes settings.input_device, the field Settings → Recording → Microphone writes; None is the system default. tray_microphones::choose loads settings.json, changes only that field, saves it, and then emits microphone-chosen with the id. The window copies it into its settings without saving; a pending or later save already carries it. The backend saves first so the choice holds even if no window ever loaded. A recording that is running keeps its microphone; the choice applies to the next start_recording.

The menu. tray_microphones::entries turns the known devices and the choice into lines: System default (name of the default device), then every device, exactly one checked. A chosen id that is not connected gets a greyed, checked line "Chosen microphone (not connected, using system default)", because audio::select_device falls back to the default then. Menu ids are microphone:default and microphone:device:<cpal id>. Windows and macOS rebuild Tauri's native menu with tray.set_menu (labels escape &); Linux updates the SubMenu of its own StatusNotifierItem through the ksni handle (labels escape _).

Staying current. The devices are read at startup (off the event loop), whenever the window calls list_input_devices (its copy is reused), and on Windows and macOS on TrayIconEvent::Enter, at most every two seconds, so a microphone plugged in since shows up before the menu opens. save_settings and complete_onboarding move the checkmark. Linux has no such hover event; there the list follows the window.

Diagnostics. tray.microphone_chosen (default or device) and tray.microphones_failed (step). Device names are not logged.

Microphone level (1.2.0)

What it holds. settings.microphone_level: a percentage, or None to leave the level to the operating system (the default, and what an older settings file reads as). mic_level.rs runs one thread, started in setup, that receives the target over a channel — from init (the settings file), save_settings, complete_onboarding and the tray's microphone choice — and, while a level is set, reads it every two seconds and writes it back when it is more than one point away (rounding in the system moves it by one). Nothing is read or written while no level is set.

Per system. Windows: IMMDeviceEnumerator::GetDevice with the endpoint id from the cpal id wasapi:<id>, else the default capture endpoint, then IAudioEndpointVolume::{Get,Set}MasterVolumeLevelScalar — the Sound settings slider; COM is initialised once on the thread. Linux: wpctl get-volume/set-volume @DEFAULT_AUDIO_SOURCE@, else pactl get-source-volume/set-source-volume @DEFAULT_SOURCE@; the ALSA ids Utterform records from do not map to PipeWire nodes, so it is always the default source. macOS: osascript input volume of (get volume settings) / set volume input volume, the default input.

Settings. Recording → Level: the switch reads the current level (read_microphone_level) and starts there; if that fails the switch stays off and the reason is shown. The slider saves after 300 ms.

Diagnostics. mic_level.target (level or off), mic_level.restored (from, to; at most once a minute), mic_level.failed once until it works again, mic_level.read_failed. No device names.

First-run assistant (0.14.0)

Who sees it. settings.onboarding_completed. AppSettings::default() — what load returns when there is no settings file — has false; a stored file without the field deserializes it as true (#[serde(default = "already_set_up")]), so an upgrade never shows the assistant. An unreadable settings file stays an error, not a new user. The interface's DEFAULT_SETTINGS say true: the backend alone decides who is new.

One decision for intents. setup loads the settings once and sets the managed Onboarding flag; save_settings and complete_onboarding keep it current. cli::Intent::route(setting_up) decides for the launch intent, the second instance and the global hotkey (deliver_intent): while setting up, every intent — including --toggle, the hotkey and a stale autostart entry — reveals the window and none is forwarded or kept as the startup intent, so nothing records afterwards out of nowhere. Otherwise the earlier matrix holds (only Show reveals; Autostart reaches nothing). The interface also ignores remote-intent, its keyboard shortcuts and audio files while the assistant is open, and main is inert beneath it.

Draft, then Finish. lib/OnboardingAssistant.svelte keeps an OnboardingDraft (lib/onboarding.ts): mode, key text, local model, one language code, vocabulary, ChatGPT, dictation key, autostart. Moving between the four steps calls nothing that changes state. applyDraft maps it onto the loaded settings, touching only those fields; a cloud model chosen earlier survives a local choice. Finish then runs, in order: save_settings with onboarding_completed: false (a crash resumes setup), set_openai_api_key if one was typed for the cloud, download_local_model if the chosen model is missing (progress from the window's existing listener), apply_global_hotkey if the key changed, enable/disable autostart if the switch changed (read back, believed over the request), and complete_onboarding. A failing stage returns to its step with the message; what already succeeded is remembered and not repeated, and setup stays incomplete. Skip calls only complete_onboarding(loaded settings, skipped: true).

Links. open_openai_api_keys opens the compiled-in https://platform.openai.com/api-keys and takes no argument, like open_codex_setup; no opener permission reaches the WebView.

Diagnostics. onboarding.shown (source first_launch or startup_intent), onboarding.intent_consumed (source, intent), onboarding.skipped, onboarding.completed (engine, cloud model, local model id, text provider, language and vocabulary counts, hotkey on/off), onboarding.finish_failed (stage settings) and onboarding.key_page_open_failed. The other stages fail inside their existing commands, which log their own events. Never the key, the language, the words or a typed draft.

What the local engine is compiled for

whisper.cpp is built from source by whisper-rs-sys, and ggml's CMake targets the machine doing the compiling unless told otherwise. For a release built on someone else's server that is a trap rather than an optimization: the 0.7.2 Linux binary carried the runner's AVX-512 and AMX instructions and was stopped with SIGILL on an Intel N300 while loading a model — inside ggml_backend_cpu_device_get_extra_buffers_type, plain C++ frame code, which is before ggml's own runtime dispatch can choose a kernel and therefore beyond its help. The bug had been there since the first release; only the pairing of build machine and user machine decided whether it fired.

packaging/cmake/portable-cpu.cmake sets GGML_NATIVE=OFF, which makes ggml use its explicit defaults instead: SSE4.2, AVX, AVX2, FMA, F16C and BMI2 on x86-64, with AVX-512, VNNI and AMX left out; on Apple Silicon clang already targets apple-m1, the oldest Mac supported. The file is deliberately free of CMAKE_SYSTEM_NAME, which would mark the build as cross-compiling and drop ggml to plain x86-64 with no AVX2 at all. It arrives through CMAKE_TOOLCHAIN_FILE in .cargo/config.toml, because whisper-rs-sys accepts no defines from its dependents.

cmake-rs never tells Cargo which environment variables it read, so a cached whisper-rs-sys survives a change to that file — which is why the CI cache key carries a suffix to retire one, and why two checks look at the result rather than at the intent: scripts/test-linux-system-package.sh disassembles the packaged binary and refuses AVX-512 or AMX inside a ggml or whisper symbol (other crates carry such code too, but reach it through a CPUID check first), and a test in local.rs reads the compiled feature set through whisper_print_system_info on every platform. The same line is written to the log the first time a local transcription runs.

Transcription context

vocabulary is a list of literal terms. GPT Transcribe takes them as keywords[], the parameter the API offers for expected terms; local Whisper takes them joined as initial_prompt, its own way of biasing a spelling. The API refuses a keyword containing <, >, CR or LF — and refuses the whole request with it — so terms are entered one per line, the interface names one it would have to drop, and usable_keywords filters them again before sending. transcription_context is the API's free-form prompt and reaches GPT Transcribe only.

text_effort becomes reasoning.effort on the Responses call, and is sent only when the user chose a level. Left on Auto nothing is sent, so a text model that does not offer the level we would otherwise have guessed cannot reject the request.

Text history

history.json lives in Tauri's app-local-data directory, separately from settings and exported files. The newest 100 texts are retained, newest first, including clipboard-only results. Titles are the first words of the text (Unicode-safe, at most 64 characters plus ellipsis), without timestamps or additional model calls. The UI restores the latest entry and keeps the displayed text while another recording runs.

New entries include createdAtMs (Unix milliseconds). Missing timestamps from older histories are recovered from their Unix-nanosecond IDs at read time, without rewriting files on load; the next normal append persists the compatible additional field. The UI uses local calendar days and de-CH formatting: Today · HH:mm · N min ago today, DD.MM.YYYY on earlier days, with full date/time in the result's tooltip. A lightweight timer refreshes relative ages every 15 seconds. Unknown/invalid dates are labelled unavailable instead of guessed.

Writes are serialized with a mutex, use an owner-only temporary file in the same directory, flush data, and atomically replace the old file. Unreadable/corrupt history is never silently overwritten; the UI still receives the new text with a persistence warning. History is unencrypted and local to each device, not stored in Git or synchronized by the app. Users can disable future history and separately confirm clearing existing history in Settings. Clearing does not touch the clipboard or exported files.

Global dictation

One intent, two ways in. A compositor binding runs utterform --toggle and the single-instance plugin hands the command line to the running app; a reserved key combination fires in the backend. Both emit the same remote-intent event, so the interface owns exactly one recording state machine and every guard against double starts applies to both.

Omarchy's dictation layer (4.5 and later) is a third caller of the same command line. Omarchy runs omarchy-dictation <start|stop|toggle> from its own bindings — Alt + Space, and Right Alt held for push-to-talk — and that calls omarchy-dictation-<backend> under a lock, one request after the other. packaging/omarchy/ holds Utterform's backend: the adapter hands each request to a running Utterform and waits for the second process to exit (about 50 ms), which keeps press and release in order; a request that has to start Utterform launches it detached through uwsm-app, because the started process is the app and never exits; stop with no Utterform running does nothing, since Omarchy also sends it when another backend is chosen. Whether Utterform runs is asked with pgrep -x utterform for the user, not through the single-instance plugin's D-Bus name, which is the plugin's own business. The installer copies the three commands into the launcher's directory and adds one row to the user's menu extension; Omarchy binds the keys only while a command named after the backend — utterform — exists.

Push-to-talk releases the key while the start cue may still be playing, and start_recording answers only after it has (see below). A stop that arrives while the interface is starting is therefore kept and finishes the recording as soon as it runs; a toggle in that moment is still dropped, so a bouncing key cannot become a start and a stop.

Key combinations are reserved only where the platform grants them: Windows, macOS, and Linux under X11. A Wayland session registers nothing — the compositor owns the keyboard — and Settings shows what to bind there instead. The plugin is installed from setup rather than at build time, so a host that refuses to create a hotkey manager costs the dictation key and not the application; a key another application already holds is reported in Settings, where it was entered, and the previously working key is released only after the new one parses.

Neither path raises the window. The audio cues are therefore the confirmation, and they come from Rust rather than the WebView, so they sound whether or not the window is visible. Because the start cue is the only sign the microphone is live, it is played to completion before capture keeps anything, and a notification stands in when no output device will play it. The tray icon carries a red dot for as long as a recording runs, set from the same commands that start and end capture, so there is a confirmation that does not depend on a speaker at all.

The combination itself is recorded rather than typed. shortcut.ts turns a keydown into the string hotkey.rs parses and nothing else: the physical key comes from KeyboardEvent.code, so a layout cannot change which shortcut a key produces, and every token it emits — A–Z, 0–9, Space, F9, Up, Comma, Super — is one global-hotkey already accepts, which is why what is shown, what settings.json holds and what is registered are one string. Rust still validates on save; there is no second syntax. A bare key is refused in the field for the same reason it is refused in parse. ShortcutRecorder.svelte reads keys in the capture phase on the window, so the dialog's Escape, the focus trap and the main window's shortcuts cannot take a key meant for the recorder, and Escape ends the reading alone.

A shortcut the operating system is holding for Utterform would never reach the field it is being replaced in, so the reading releases it — apply_global_hotkey(null) — for exactly that moment and registers it again when the reading ends, in a serialized queue Save waits on. A new key is still only made permanent by Save. On Wayland nothing is registered and nothing is suspended; a compositor binding that fires mid-reading is turned away in the interface instead.

Starting with the session

tauri-plugin-autostart owns the entry: HKCU Run on Windows, a per-user LaunchAgent on macOS, ~/.config/autostart/Utterform.desktop on Linux. The entry is the only state — no field in AppSettings mirrors it — so Settings asks isEnabled when it opens, writes only a change and only on Save, and asks again afterwards rather than trusting the call. A system that will not answer costs the switch and nothing else.

Every entry carries --autostart, and that argument is the reason it can exist: cli.rs reads an unknown or bare command line as Show, so without an intent of its own a login would open the window in front of whatever the user signed in to do. Intent::Autostart raises no window, is never stored as a startup intent, and is never emitted as remote-intent — so it does nothing on this instance and nothing at all when it meets a running one. On Linux it also settles the graphics backend before GTK exists: every other Linux start goes through the launcher the installer writes, which pins GDK_BACKEND=x11, and an autostart entry runs the executable directly, so the same policy is applied in run() for that one kind of start and a backend the session chose is left alone.

Delivery to the focused window

Clipboard, file and cursor are independent, and typing runs last so a delivery failure there costs a warning rather than the text.

Paste is the default because synthesized keystrokes cross the compositor, the input method and the receiving application one character at a time, and each can drop or reorder one when input arrives faster than a person could type — the terminal that turns "Session" into "ession". A paste moves the whole text at once, so there is nothing to reorder. The chord depends on the window: terminals reserve plain Ctrl+V for the shell and take the text on Ctrl+Shift+V on Linux and Shift+Insert on Windows, while those chords mean something else entirely in a browser or editor, so the focused window decides. Hyprland answers over its own socket, X11 through xdotool, Windows through the foreground window's class and the program behind it; an unrecognized window gets the chord every graphical toolkit agrees on. On Windows every key of a chord carries its scan code and names the left-hand modifier, because Windows Terminal reads both and ignores a bare virtual key. Guessing wrong costs a paste that does not arrive, never the text.

Paste delivery is told whether the clipboard output already ran, so it neither writes the transcript twice nor takes the clipboard when it did not have to. When it does write, the text stays there — restoring the previous contents would race the receiving window's read and could paste the wrong text.

Keystrokes stay selectable for the windows that refuse a paste, with a leading Shift press and release for the Wayland clients that swallow the first character a fresh virtual keyboard sends, and a delay between keys, clamped so a hand-edited settings file cannot stall delivery for minutes. Text always reaches wtype and xdotool on stdin, so a transcript starting with a dash is never read as options.

Windows needs no helper program: one SendInput call appends its whole batch to the input queue atomically, so both methods deliver the text intact. Characters go in as Unicode rather than scan codes, making the transcript independent of the active keyboard layout, and every line ending becomes exactly one Return. Windows blocks input from a normal process to an elevated window; that is reported rather than counted as success.

macOS posts CoreGraphics keyboard events (typing/macos.rs): ⌘V for a paste, since a Mac terminal takes the same chord as every other window, and one Unicode keyboard event per character for keystrokes, paced by the delay. The window server drops events from a process without the Accessibility grant and says nothing, so macos.rs checks the grant first, opens the system request on the first delivery, and reports the refusal as a warning. The same module refuses to record through a microphone macOS has denied, and Entitlements.plist carries the audio-input entitlement the hardened runtime requires before macOS asks for the microphone at all. See MACOS.md.

Diagnostics

Diagnostics are local, bounded and free of user content, and diagnostics.rs is the only code that knows how they are written.

Storage. tauri-plugin-log 2.9 supplies the filtered logger and writes to stderr; Utterform's bounded writer stores utterform.log in Tauri's log directory (app_log_dir: $XDG_DATA_HOME/software.jli.utterform/logs on Linux, ~/Library/Logs/software.jli.utterform on macOS, %LOCALAPPDATA%\software.jli.utterform\logs on Windows). It checks the size after every record and rotates at 2 MiB to utterform.1.log and utterform.2.log, so the disk budget is about 6 MiB. It buffers at most one already-bounded record and drops that record after a write or rotation failure, preventing a full disk or a removed/locked log directory from growing process memory without limit. The level is INFO in release builds and DEBUG in debug builds, and a filter admits only the utterform/utterform_lib targets and the frontend bridge's utterform::frontend — dependency output is never written. The logger is built with the plugin's split and attached in setup before anything else runs, rather than registered as a plugin: its only other content is a JavaScript log command the WebView must not reach, and a log directory that cannot be created then costs the file instead of the launch (events still reach stderr and diagnostics.file_unavailable says why). The 0.7.8 utterform.log in app-local-data is not moved, deleted or written.

Events. One line per event: 2026-09-16T08:00:00.123Z INFO utterform run=1a2b3c4d recording.opened recording=3 device="Studio microphone" open_ms=42. Names are stable and dotted (app.*, recording.*, file.*, transcription.*, transform.*, history.*, delivery.*, live.*, cue.*, settings.*, hotkey.*, models.*, secrets.*, support.*, window.*, tray.*, typing.*, frontend.error); fields are key=value, quoted and escaped when they contain whitespace, quotes, = or backslashes. Every line carries run, eight hex digits for this process, so a file shared by several runs stays attributable. Every batch and Live recording gets a process-local recording id from one counter, carried through StartedRecording, ActiveRecording, RecordingArtifact and the Live Session — separate from the Instant that identifies a capture session. A run of a recording therefore reads as recording.requested → recording.opened → recording.armed → recording.stopped → transcription.started/completed → transform.started/completed or transform.fallback → history.saved → delivery.completed → recording.completed, and a Live one adds live.input_ready, live.connected, live.streaming, live.stream_completed and the input worker's events.

Failures and references. Low-level code returns a diagnostics::Failure: the message the user reads, kept apart from a closed-vocabulary class, an optional HTTP status and an optional detail (an io::ErrorKind, a keyring variant, an audio backend's error). Failure has no Display, so its message — which can quote an OpenAI error body — cannot become a log field by accident. The code that knows the context owns the one event: failure! logs ERROR, fallback! logs WARN for work that continued without that part, and both append (ref <run>-<n>) to the message; the same reference is the event's ref field. Guidance the user acts on (add a key, choose an output, a model not yet downloaded, install wtype, grant Accessibility, start Live from another application's text field) is logged at INFO with outcome=guidance and no reference; typing::is_guidance and typing::is_capture_guidance recognise the platform modules' own refusal sentences for that. Commands that only pass a failure on do not log it again.

Privacy. Allowed: version, OS/architecture, desktop and session kind, run/recording/reference ids, durations, counts, enabled modes, model and action ids (a custom action is only custom), HTTP statuses, microphone and output device names, program/application names, window classes and bundle ids, error classes, and the platform typing modules' own refusal sentences, which name tools, programs and counts. The text model is named only when it looks like a model id; anything else typed into that field is custom. Never: API keys or authorization material, transcripts, history, clipboard, selection or output-file contents, audio, prompts and prompt overrides, vocabulary, language hints, recording context, window titles, names and paths of opened audio files, request/response bodies, WebSocket deltas, raw third-party errors, full settings or history JSON, output folders (a saved file is named by its basename). Linux Live focus events keep their existing rendering: event name, window address, workspace id. Every record is still normalized by redact::line — one physical line, no control or direction-override characters, home directories reduced to $HOME, sk-… keys, bearer tokens and Authorization: values replaced — and bounded to 16 KiB; a copied report applies the same again, legacy lines included. The sanitizer cannot recognise dictated prose; the protection is that it never reaches diagnostics.

Bounds. Nothing is logged from the audio callback, the writer loop, per delta, per keystroke, per animation frame or per status poll: glitches, delivered chunks, WebSocket events and audio bytes are counted and reported once. Limiter keeps Linux Live focus events at 200 per session and reports live.focus_events_suppressed; the frontend bridge admits 50 reports per run (20 per mounted interface on its own side); a failing get_recording_status is reported once per run; undelivered model-download progress events are counted and reported once.

Unclean exits and panics. setup reads utterform.running in app-local-data: present means the previous run never reached RunEvent::Exit (app.previous_run_unclean, and the report header says so), then it is atomically replaced with this run's id. RunEvent::Exit writes app.exiting and removes the marker if it is this run's. Tray Quit still discards capture first. A force-kill or power loss is therefore reported as unclean, by design; a marker that cannot be read or written is logged and never blocks startup. The panic hook, installed once the logger is attached and chained to the previous hook, records thread, source location (build-machine paths reduced), the payload without the text std quotes when a string is sliced at the wrong place (is not a char boundary, out of bounds of, when slicing), and a captured backtrace without the panic machinery's own frames, bounded to 8 KiB; it guards against recursion. Aborts, segmentation faults and native driver crashes never reach a hook: they appear only as an unclean previous run, and no crash dumps are collected.

The interface. App.svelte registers error and unhandledrejection listeners on mount and removes the same handlers on destroy. createFailureReporter sends a source category, the error's name and message (500 characters), a stack without the serving origin (2,000 characters) and the current phase — never the rejection value itself, which becomes Non-error value of type … — and swallows every failure of its own, so logging cannot cause the next rejection. A command rejects with a string that already carries its reference, so only an Error thrown in the interface is reported from setError; autostart plugin failures, which no Utterform command sees, are reported explicitly. The backend accepts only known source and phase values, reduces origins again and returns the reference shown beside the message.

Support actions. diagnostics_info returns the backend's own resolved path; copy_diagnostics, open_log_file and open_log_folder take no arguments at all. The report is built on a blocking thread from the tails of the current, rotated and legacy logs — never settings, history, recordings or exports — newest first, reading only as many bytes as fit, dropping a partial first line so no UTF-8 sequence is split, and stopping at the first file that no longer fits whole so the kept lines are one unbroken stretch. The header names format version, generation time, version, OS, architecture, desktop, run id, previous-run state, log path and suppressed frontend reports; the whole text never exceeds 256 KiB and says when earlier lines were omitted. Opening uses tauri-plugin-opener's Rust API — registered without its link-click script and without any JavaScript permission — open_path for the file and reveal_item_in_dir for the folder, falling back to opening the folder where the desktop offers no way to select a file. Settings disables only the action in flight and shows its result or failure in the Diagnostics row; the log path is the row's tooltip.

Known limits. Diagnostics are best-effort. If the current log cannot be written or rotated — for example because the disk is full, the directory was removed or Windows has locked a file — that record is still written to stderr but may be absent from the files and copied report. The next record retries; no failed record is retained in memory.

Deliberate MVP constraints

  • Batch transcription only
  • One dictation key, not a set of separately bindable global shortcuts
  • Autostart on or off, with no separate "start minimized" choice: a login start is always silent
  • No global dictation key under Wayland; the compositor binding covers it
  • No Live Dictation on macOS
  • CPU local inference by default
  • No updater
  • No files-as-clipboard-objects
  • No local LLM for transformations
  • No telemetry, upload or crash dumps: diagnostics stay on the device, and a native crash shows only as an unclean previous run

GPU backends and release signing require separate platform work and testing.

Live dictation

live.rs owns one numbered session at a time: an OpenAI Realtime transcription socket, the microphone's live PCM tap and a dedicated native input thread. Deltas are appended in order, split into short chunks and handed to the input worker over a bounded queue; nothing is retried or replayed, and a final transcript that differs from the live text stays in Utterform. The worker owns the platform typer (typing/live.rs) for the whole session and drops it before the next session may start; between deltas it only reads queued focus events, and each chunk confirms the target once before typing. A stop request holds typing for 700 ms so a still-held shortcut modifier cannot combine with typed keys.

On Omarchy/Hyprland the typer is a persistent Wayland virtual keyboard plus Hyprland's event socket. Only a confirmed change of the active window address or the target closing is a loss; workspace, monitor, layer, submap and config-reload events are logged but ignored, and IPC failures are reported as technical faults. Because Hyprland matches bindings against a virtual keyboard by keycode through its own layout, characters are placed on keycodes without default bindings, editing, browser or media meaning, and with DOM codes so Chromium on Wayland accepts them; the map is seeded with the most frequent characters and extended on demand. Windows uses Unicode SendInput with WinEvent foreground/focus/desktop observation on the worker thread and waits for physically held modifiers directly.

Tray status (0.7.13, replacing the rule below)

Red is recording. Green now means the text reached Utterform, set just before output delivery rather than after it, because the text can be fetched from the window or history whatever the clipboard and cursor do. Amber is Status::Failed, reported from the interface's setError through notify_if_unattended(failed: true), the one path every failure of every kind passes through; a warning on delivered text passes failed: false and stays green. Amber expires after five seconds unless Indicator::kept is set, in which case it is what the icon rests on: green and red still take over and it returns afterwards. adopt lets a retry of an older recording turn green. All platforms share status_badge; Linux pre-renders one pixmap per status. The Done cue keeps the stricter meaning of every requested output having worked.

Tray delivery confirmation (0.7.12)

tray::TrayState holds Idle/Recording/Success, a generation token and at most one abortable five-second async expiry. Status transitions, timeout token checks and platform icon writes all run on the event loop, so an already queued stale timeout cannot clear a new recording/success. Stop/cancel invalidate the previous timer; processing is unbadged. Batch success reuses the Done cue's transformation/output predicate but does not depend on sound. Live success requires nonempty text, an actual output and no processing/output failures (history is best-effort). Manual history-copy and cue previews do not claim a new recording completed. Tray failures stay decorative and emit bounded tray.update_failed events (state, queue_status, expiry_state, queue_expiry, find_tray, icon, set_icon, set_tooltip, status_notifier_item) without text.