Skip to content

feat(voice): add optional local dictation with microphone feedback - #96

Draft
angel1411337-del wants to merge 1 commit into
AlienTechSoftware:mainfrom
angel1411337-del:feat/voice-dictation
Draft

angel1411337-del wants to merge 1 commit into
AlienTechSoftware:mainfrom
angel1411337-del:feat/voice-dictation

Conversation

@angel1411337-del

Copy link
Copy Markdown
Contributor

What and why

This adds optional native dictation that uses a local whisper.cpp executable and model the user selects. Nothing is downloaded and no audio leaves the computer. Recording becomes active only after real microphone samples arrive. Setup includes device selection and separate microphone and recognizer checks. The panel shows startup, level and time, processing, review and actionable failures, including errors when there is no data for five seconds or the stream stalls.

Provider-independent core and runtime crates (crates/voice-core, crates/voice-runtime) own bounded audio, coalesced previews, cancellation and subprocess cleanup. Transcripts are reviewed by default and inserted as one Undo/Redo transaction. Page, revision and cursor tickets prevent stale automatic insertion. Before changing the page, insertion checks the unchanged upstream serialization, and it keeps rejected words for review.

The feature is off by default (cargo build -p notecraft --features voice). The default app dependency tree excludes CPAL and the capture runtime. This pull request does not change the formats or Markdown implementation and does not depend on another pull request; it also builds and passes its UI tests on top of #92 (literal text). On today's main, the serializer cannot keep some punctuation literal, so those words are refused and stay editable and copyable in review. With #92, words it can keep literal are inserted exactly. Multiline insertion requires a paragraph end or appending, and refuses cases that would shift indexed layout. English and pseudo catalogs are supplied; other locales fall back to English. Only trusted executables should be selected. Temporary-file cleanup is best effort, not secure erasure.

Quality level L3 (see SKILLS/nc-devx/SKILL.md)

  • The branch is rebased on main (4286f7b) and has no merge commits: one commit
  • cargo xtask ci passes locally on Rust 1.97 (see below); the pinned 1.99.1 run is this pull request's CI
  • Coverage: the coverage job's warnings are read; cargo xtask coverage --update is in
    this pull request if coverage rose (to read once CI has run)
  • Docs match the change: docs/commands.md (voice.*), docs/how-to/dictate.md, docs/how-to/README.md, docs/gaps.md, docs/feature-parity.md, docs/architecture.md, README.md
  • A new user action is a CommandSpec with a test: the voice.* commands, with engine tests for origin and redaction
  • Page or demo changes: none to the demo; cargo xtask demo-check passes
  • UI changes: setup, listening and review were inspected in light and dark themes (rendered by the tests)
  • New files under assets/ or docs/images/: none

How it was checked

On the current main (4286f7b), with Rust 1.97.0 locally (this environment could not download the pinned 1.99.1; GitHub CI runs it):

  • With voice disabled, cargo xtask ci passed: fmt, clippy, tests (983 passed, 0 failed, 2 existing manual large-fixture tests ignored), i18n-audit, layers, assets, docs, text, demo-check. cargo build --workspace --locked also passed.
  • cargo build -p notecraft --features voice --locked passed.
  • Optional UI library (cargo test -p notecraft-ui-egui --features voice --lib): 307 tests passed, including 36 voice tests.
  • Core and runtime with all features: 23 passed; one opt-in device test is ignored by default.
  • Optional all-target Clippy with -D warnings (craft-voice-runtime/microphone, notecraft/voice) passed.
  • Stacked on fix(editor): preserve literal Markdown text across saves and edits #92, the optional UI library tests and Clippy also pass, so the two pull requests can merge in either order. Two test changes made that possible:

Audio evidence, from the earlier run on a71a3c6 (the voice-core and voice-runtime sources are byte-identical in this branch):

  • Capture path, synthetic input. The opt-in real CPAL/ALSA test read a synthetic FIFO-backed ALSA input, not a microphone: 60 callbacks, 32,000 converted samples, non-silent data, and the device reopened.
  • Recognizer, recorded speech. A real whisper.cpp CPU whisper-cli with ggml-tiny.en-q5_1 transcribed whisper.cpp's samples/jfk.wav recording through this runtime's transcribe example, giving the expected sentence.

Not checked, and needing separate verification:

  • capture from a physical microphone, and acoustic accuracy on live speech;
  • device behavior and permission prompts on Windows and macOS. The macOS plist, entitlement and packager syntax were validated, but nothing ran on a Mac.
  • the new three-OS voice CI job, which runs for the first time on this pull request.

Exact manual acceptance steps are in docs/how-to/dictate.md.

Written with AI assistance (Codex and Claude), reviewed and tested as above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KowAZudvkUiQS623fiB331

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant