diff --git a/CHANGELOG.md b/CHANGELOG.md index 88ee2eef..cf22a71b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,18 +10,31 @@ This project uses semantic versioning for public releases. Use `MAJOR.MINOR.PATC ## [1.2.0] - 2026-06-26 -The project is now **Fluent**, a faster, more token-efficient fork of Free Flow. This release rebrands the app and trims the dictation pipeline so each dictation uses fewer tokens and completes more quickly, without changing the core experience. +The project is now **Fluent**, a fork of Free Flow reworked for speed, transcription accuracy, and token efficiency. This release rebrands the app and folds in the improvements made since the fork. ### Changed - Renamed the app and project from FreeFlow to Fluent, including the bundle identifier (`com.inhaq.fluent`), app menus, permission prompts, build tooling, and the project website. +### Added + +- Post-Processing Reasoning setting (Automatic / Off / Low / Medium / High) that controls the cleanup model's `reasoning_effort`, letting you trade accuracy for fewer tokens and faster cleanup. This was not configurable in Free Flow. +- Optional warm-microphone session that keeps an idle capture session alive between dictations so recording starts without `AVCaptureSession` cold-start latency, with an automatic idle cool-down. +- Fast dictation mode that skips the screenshot and vision/context LLM call for plain dictation, with a "Capture screenshot for dictation" opt-in for when you want richer context. +- Custom vocabulary is now sent to the transcriber as a biasing prompt, so names, jargon, and acronyms are spelled correctly in the raw transcript. +- `os_signpost` instrumentation across the dictation pipeline (trigger, recorder ready, stop, audio finalized, transcript received, paste) for measuring end-to-end latency. +- Optional streamed multipart upload path for transcription requests. + ### Improved -- Plain dictation now skips screenshot capture entirely and sends only lightweight app and window metadata to the context model, removing the most expensive step from the common dictation path and saving the image tokens it used to spend. -- Screenshots, when still needed (such as in Edit Mode), are captured at a smaller default dimension (768px), JPEG-compressed, and cropped of surrounding whitespace before upload, lowering image token usage. -- The default cleanup model runs with low reasoning effort and reasoning output disabled, reducing token overhead and latency. -- Reasoning blocks are always stripped from model output so reasoning-oriented models stay fast and never paste their internal scratch work. +- Reworked speech-to-text using techniques from the Whisper paper ("Robust Speech Recognition via Large-Scale Weak Supervision"): robust per-segment decoding based on `no_speech_prob`, `avg_logprob`, and `compression_ratio` that rejects hallucinations while preserving quiet, low-confidence real speech. +- Added an audio front-end (80 Hz high-pass, loudness normalization with a safe peak ceiling and gain cap, and maximum-quality 16 kHz resampling) to improve recognition of soft or distant speech, with a full fallback to the original recording on any error. +- Transcription now decodes at `temperature=0` for stable, repeatable output. +- Rewrote the default cleanup system prompt from roughly 720 to about 370 words while preserving every behavioral rule, roughly halving the fixed prompt tokens per cleanup call. +- Lowered the default context screenshot dimension from 1024px to 768px (about 44% fewer pixels) and crop surrounding whitespace before upload. +- ``/`` reasoning blocks are now always stripped from model output across the whole model family, so reasoning models never paste their scratch work. +- Optimized the audio capture hot path (decode each buffer once, Accelerate/vDSP metering, throttled level updates), launch (async microphone discovery), run-history loading (lazy screenshots), and Settings rendering for a smoother, more responsive app. +- Added `qwen/qwen3.6-27b` to the model list and removed the GitHub star promo card and its launch-time network request. diff --git a/README.md b/README.md index 81d0fb6c..cbe7d09d 100644 --- a/README.md +++ b/README.md @@ -25,22 +25,38 @@ ## Overview -Fluent is a free Mac dictation app for fast AI transcription, context-aware cleanup, and voice-driven text editing, with no monthly subscription. It started as a fork of [Free Flow](https://github.com/zachlatta/freeflow) and has been reworked to be **more efficient, to use fewer tokens per dictation, and to feel noticeably faster** end to end. +Fluent is a free Mac dictation app for fast AI transcription, context-aware cleanup, and voice-driven text editing, with no monthly subscription. It started as a fork of [Free Flow](https://github.com/zachlatta/freeflow), and since the fork it has been reworked along three lines: it **feels faster from the moment you press the key**, it **converts speech to text more accurately**, and it **uses fewer tokens per dictation** while putting that cost under your control. -## Why Fluent Is Faster and Cheaper +## Faster From Key-Press to Paste -Fluent keeps Free Flow's full feature set but trims the expensive parts of the dictation pipeline so you spend fewer tokens and wait less: +Fluent attacks latency at both ends of a dictation, so recording begins almost the instant you press the shortcut and text lands shortly after you release it: -- **No screenshot on plain dictation.** Capturing and uploading a screenshot to the vision context model is the single most expensive step of context collection. For ordinary dictation, the app and window metadata are enough, so Fluent skips the screenshot entirely. Image capture only happens for Edit Mode (and an explicit opt-in). This removes thousands of image tokens from the typical dictation and cuts a slow step out of the hot path. -- **Smaller, compressed screenshots when they are used.** When a screenshot is needed, Fluent sends it at a 768px default dimension instead of 1024px (about 44% fewer pixels, and image tokens scale with pixel area), JPEG-compresses it, and crops surrounding whitespace before upload. -- **A lean default cleanup model.** Cleanup runs on `openai/gpt-oss-20b` with reasoning effort set to low and reasoning output disabled, so you pay for the cleaned text instead of a long hidden chain of thought. -- **Reasoning tokens never leak through.** Any ``-style reasoning blocks are always stripped from model output, so reasoning-heavy models stay fast and never paste their scratch work. +- **Instant start with a warm microphone (opt-in).** Starting an `AVCaptureSession` is one of the slowest steps when you press the key. Fluent can keep an idle capture session warm between dictations so the next recording starts without that cold-start delay, then automatically cools the session down after a short idle period to limit battery and mic-indicator impact. +- **Fast mode removes work from the release path.** For plain dictation, Fluent skips both the active-window screenshot and the vision/context LLM call entirely, so finishing a dictation no longer waits on a context request. Screenshot context is reserved for Edit Mode and an explicit opt-in. +- **A leaner audio hot path.** Each captured audio buffer is now decoded once and shared across the file writer, the realtime stream, and the level meter (instead of three separate decodes), RMS metering uses Accelerate/vDSP, and the live waveform is throttled to ~30 Hz, cutting CPU and main-thread wakeups during recording. +- **A faster launch and UI.** Microphone discovery moved off the launch path, run-history screenshots load lazily, and the Settings cards were made cheap to render, so the app opens and navigates without stalls. +- **Measured, not guessed.** A built-in `os_signpost` pipeline records each milestone (trigger to recorder-ready to stop to audio-finalized to transcript-received to paste), so end-to-end latency can be profiled in Instruments. -The result is the same dictation quality with a smaller token bill and lower latency on every dictation. +## More Accurate Speech-to-Text + +Speech-to-text was reworked using techniques from speech-recognition research, in particular the Whisper paper ["Robust Speech Recognition via Large-Scale Weak Supervision"](https://arxiv.org/abs/2212.04356): + +- **Robust per-segment decoding.** Instead of a brittle hardcoded list of phrases to discard, Fluent applies the paper's robust-decoding signals — `no_speech_prob`, `avg_logprob`, and `compression_ratio` — per segment to drop genuine non-speech and looping hallucinations while keeping quiet, low-confidence real speech that the old approach used to throw away. +- **An audio front-end tuned for quiet voices.** Before transcription, recordings pass through an 80 Hz high-pass that strips DC offset and sub-speech rumble (which lowers effective SNR), loudness normalization toward a consistent level with a safe peak ceiling and gain cap, and maximum-quality resampling to 16 kHz to avoid aliasing. Any failure falls back to the original recording untouched. +- **Vocabulary-biased, deterministic transcription.** Your custom vocabulary is now sent to the transcriber as a biasing prompt so names, jargon, and acronyms are spelled correctly in the raw transcript, and decoding runs at `temperature=0` for stable, repeatable output. Previously custom vocabulary only influenced the cleanup step, never the transcription itself. + +## Fewer Tokens, On Your Terms + +Fluent trims the fixed cost of every dictation and, unlike Free Flow, lets you decide how much the model is allowed to think: + +- **User-configurable reasoning effort (new in Fluent).** A Post-Processing Reasoning setting (Automatic / Off / Low / Medium / High) controls the model's `reasoning_effort`. Turning it down — or Off — cuts the hidden reasoning tokens you pay for and speeds up cleanup. Free Flow gave you no control here; in Fluent it is a setting. +- **A leaner cleanup prompt.** The default cleanup system prompt was rewritten from roughly 720 to about 370 words while preserving every behavioral rule, roughly halving the fixed prompt tokens sent on every post-processing call. +- **Smaller, compressed screenshots when used.** When a screenshot is actually needed, it is captured at a 768px default dimension instead of 1024px (about 44% fewer pixels, and image tokens scale with pixel area), then whitespace-cropped before upload. +- **Reasoning tokens never leak through.** ``/`` reasoning blocks are always stripped from model output across the whole model family, so reasoning-oriented models stay fast and never paste their scratch work. ## Quick Start -1. Download the app from above or [click here](https://github.com/inhaq/fluent/releases/latest/download/Fluent.dmg) +1. Download the app from above, or [download Fluent.dmg directly](https://github.com/inhaq/fluent/releases/latest/download/Fluent.dmg) 2. Get a free Groq API key from [groq.com](https://groq.com/) 3. Hold `Fn` to talk, or tap `Command-Fn` to start and stop dictation, and have whatever you say pasted into the current text field @@ -48,7 +64,9 @@ The result is the same dictation quality with a smaller token bill and lower lat - **Custom shortcuts:** Customize both hold-to-talk and toggle dictation shortcuts. If your toggle shortcut extends your hold shortcut, you can start in hold mode and press the extra modifier keys to latch into tap mode without stopping the recording. - **Context-aware cleanup:** Fluent can read nearby app context so names, terms, and phrases are spelled correctly when you dictate into email, terminals, docs, and other apps. -- **Custom vocabulary:** Add names, jargon, and project-specific words that Fluent should preserve during cleanup. +- **Custom vocabulary:** Add names, jargon, and project-specific terms. Fluent now feeds them to the transcriber as a biasing prompt so they are spelled correctly in the raw transcript, not just during cleanup. +- **Reasoning control:** Choose how hard the cleanup model thinks (Automatic / Off / Low / Medium / High) to trade accuracy for speed and token cost. +- **Speed toggles:** Optionally keep the microphone warm between dictations for instant starts, and opt into capturing a screenshot for plain dictation when you want richer context. - **OpenAI-compatible providers:** Use Groq by default, or configure a custom model and API URL in settings. ## Edit Mode @@ -57,7 +75,7 @@ Edit Mode lets you highlight existing text and transform it with a spoken instru ## Privacy -There is no Fluent server, so Fluent does not store or retain your data. The only information that leaves your computer are API calls to your configured transcription and LLM provider. +There is no Fluent server, so Fluent does not store or retain your data on infrastructure we run. The audio and text of your dictations only ever leave your computer as API calls to the transcription and LLM provider you configure. The app also contacts GitHub to check for updates; that is the only other outbound request. ## Custom Cleanup diff --git a/website/llms.txt b/website/llms.txt index fbd93532..e3629692 100644 --- a/website/llms.txt +++ b/website/llms.txt @@ -17,7 +17,13 @@ Fluent uses Groq by default for transcription and LLM-powered cleanup. It suppor ## How Fluent Differs From Free Flow -Fluent keeps Free Flow's features but is optimized for efficiency and speed. Plain dictation skips screenshot capture by default, so most dictations send only lightweight app and window metadata to the context model instead of a full image. When a screenshot is used (for example in Edit Mode), it is downscaled and JPEG-compressed before being sent, which lowers image token usage. The default cleanup model runs with low reasoning effort and reasoning output disabled. The combined effect is fewer tokens billed per dictation and lower end-to-end latency. +Fluent keeps Free Flow's features but was reworked along three lines after the fork: speed, transcription accuracy, and token efficiency. + +Speed: an optional warm-microphone session keeps an idle capture session alive so recording starts without AVCaptureSession cold-start latency, and a fast dictation mode skips the screenshot and vision/context model call for plain dictation so finishing a dictation does not wait on a context request. The audio capture path, launch, and UI were also optimized, and os_signpost instrumentation measures end-to-end latency. + +Accuracy: speech-to-text was reworked using techniques from the Whisper paper ("Robust Speech Recognition via Large-Scale Weak Supervision"), including robust per-segment decoding (no_speech_prob, avg_logprob, compression_ratio), an audio front-end (high-pass filter, loudness normalization, high-quality resampling), deterministic temperature=0 decoding, and custom-vocabulary biasing of the raw transcript. + +Token efficiency: a user-configurable Post-Processing Reasoning setting (Automatic/Off/Low/Medium/High) controls reasoning_effort (not configurable in Free Flow), the default cleanup prompt was roughly halved in length, screenshots default to 768px when used, and reasoning blocks are always stripped from output. ## Privacy