Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/VOICE-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Superseded by [the voice bar](#the-voice-bar-0261). Kept for the facts and for w

## Facts (Pixel 7 Pro, 2026-10-31)

- **Engine:** Voxtype 1.1.1 (`voxtype-bin`; `wtype` installed) runs as a systemd user daemon (`voxtype.service`) with Omarchy's config (`/usr/share/omarchy/default/voxtype/config.toml`): Whisper `base.en` is the default model, output `type`, `max_duration_secs = 60`, `state_file = "auto"`, `[hotkey] enabled = false`. Voice settings > Longest recording (omamobile-shell 0.2.73) sets `audio.max_duration_secs` in the user's config to 1, 2, 5 or 10 minutes, and the bar counts down to it; the wait after Done grows with the recording.
- **Engine:** Voxtype 1.1.1 (`voxtype-bin`; `wtype` installed) runs as a systemd user daemon (`voxtype.service`) with Omarchy's config (`/usr/share/omarchy/default/voxtype/config.toml`): Whisper `base.en` is the default model, output `type`, `max_duration_secs = 60`, `state_file = "auto"`, `[hotkey] enabled = false`. From omamobile-shell 0.2.82 the root image ships it ready: `voxtype-bin` in `rootfs/packages.base`, base.en in `omamobile-voice` 0.1.0 (`/usr/share/omamobile/voice/models`, fetched at build time from a pinned whisper.cpp commit and sha256), and `omamobile-setup-user` doing what `omarchy-voxtype-install` does once per user without the download (Omarchy's config, a link to the model in `~/.local/share/voxtype/models`, `systemctl --user enable voxtype.service`, Omarchy's install invitation marked done). Voice settings > Longest recording (omamobile-shell 0.2.73) sets `audio.max_duration_secs` in the user's config to 1, 2, 5 or 10 minutes, and the bar counts down to it; the wait after Done grows with the recording.
- **Control:** `voxtype record start|stop|toggle|cancel`. `record start` takes `--type | --clipboard | --paste | --file[=PATH]`, `--model <name>` (for this recording only: any downloaded model; the daemon keeps it loaded), `--no-osd`, `--auto-submit` and `--smart-auto-submit` (`voxtype record start --help` on the phone).
- **State:** `$XDG_RUNTIME_DIR/voxtype/state` holds `idle`, `recording`, `streaming` or `transcribing`. `voxtype-audio-bridge` prints one NDJSON frame per audio block (`{"peak","rms","vad","ts_ms"}`) from the daemon's `audio.sock`. Voxtype's own `voxtype-shared/{StateReader,AudioBridge}.qml` show how to read both; the plugin follows them and doesn't import them.
- **OSD:** Voxtype's GTK overlay anchors bottom centre, over the dock and the home pill, so every recording the shell starts passes `--no-osd`.
Expand Down
2 changes: 1 addition & 1 deletion docs/release-notes/2026-11-DRAFT.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ From [docs/DEVICES.md](../DEVICES.md) on the release day:

### New in the last weeks

- **Voice input:** a voice bar on the keyboard with live words as you speak, on-device speech recognition, and a setting for the longest recording.
- **Voice input, ready out of the box:** a voice bar on the keyboard with live words as you speak, on-device speech recognition, and a setting for the longest recording. Voxtype and its English speech model (Whisper base.en) come in the image, so dictation works from the first login with no setup step and no download, offline too.
- **Media controls** with album art in the notification shade and on the lock screen.
- **Lock screen:** a glance view (time, date, weather, notifications), the battery level, and a clock that follows the bar's 12/24-hour setting.
- **Fluid home and drawer gestures**, pill gestures, and hiding or showing the keyboard from the home pill.
Expand Down
4 changes: 3 additions & 1 deletion packages/omamobile-shell/PKGBUILD
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ depends=(omarchy omamobile-apps 'omamobile-omarchy>=0.1.11' omamobile-owl-widget
# Both are in Omarchy's base package list, which root images install.
foot xdg-terminal-exec # guard for hand-built/partial installations
xdg-desktop-portal-gnome) # the file chooser (portal/hyprland-portals.conf)
optdepends=('fprintd: fingerprint setup and unlock (with omamobile-fingerprint)')
optdepends=('fprintd: fingerprint setup and unlock (with omamobile-fingerprint)'
'omamobile-voice: the speech model for voice input, so Voxtype works with no download (the root image has it)'
'voxtype-bin: voice input (the root image has it; or Install > AI > Dictation)')
backup=(etc/xdg/xdg-desktop-portal/hyprland-portals.conf)
install=omamobile-shell.install

Expand Down
40 changes: 39 additions & 1 deletion packages/omamobile-shell/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Omarchy Mobile's phone profile for Omarchy's own shell. It isn't a fork: these a
| `plugins/omamobile.back` | The back gesture: swipe in from a side edge for Escape |
| `plugins/omamobile.update` | The update sheet: Omarchy's update for a phone, through `omamobile-update` (package `omamobile-omarchy`) |
| `plugins/omamobile.voice` | The voice bar, through Voxtype: hold the pill in an app or the search bar on the home screen (or the keyboard's Mic key, Power + Volume Up, the Dictation tile, whose chevron opens the settings); it types into the app, sends to the agent in its terminal, or asks it from the home screen (docs/VOICE-PLAN.md) |
| `bin/omamobile-voice-preview` | The voice bar's live preview: a second capture of the microphone (in memory only), transcribed in chunks with tiny.en and Whisper's context cut to the chunk while Voxtype records, one JSON line per pass; the bar shows the words in the theme's muted colour until the daemon's final text replaces them. Voxtype 1.1.1 can't hand its own partial text over (docs/VOICE-PLAN.md, phase 3) |
| `bin/omamobile-voice-preview` | The voice bar's live preview: a second capture of the microphone (in memory only), transcribed in chunks with tiny.en (base.en when tiny.en isn't downloaded) and Whisper's context cut to the chunk while Voxtype records, one JSON line per pass; the bar shows the words in the theme's muted colour until the daemon's final text replaces them. Voxtype 1.1.1 can't hand its own partial text over (docs/VOICE-PLAN.md, phase 3) |
| `plugins/omamobile.access` | The permission sheet: "Allow Camera to use the camera?" with the app's icon and two large buttons, in place of xdg-desktop-portal-gtk's desktop dialog |
| `plugins/omamobile.fingerprint` | The under-display fingerprint sensor: target during verification, enrollment screen |
| `profile/shell.json` | The phone layout the setup seeds once |
Expand Down Expand Up @@ -50,6 +50,44 @@ which normal root images install. The explicit dependencies are a guard
for hand-built or partial installations: both agent launch and installing
a selection need Omarchy's terminal launcher.

## Voice input

Voice works out of the box on a fresh install: no Install > AI > Dictation
step and no download. The root image installs Voxtype (`voxtype-bin` from
[omarchy], in `rootfs/packages.base`) and Whisper's base.en speech model
(`packages/omamobile-voice`, 141 MiB in `/usr/share/omamobile/voice/models`).
The model counts twice in the image: the image also carries its own copy of
the [omamobile] repository (`/var/lib/omamobile/repo`, for offline
reinstalls), where the package archive is another 141 MiB that won't
compress. With voxtype-bin (71 MB installed, its archive is in Omarchy's
repository, not ours) the root image grows by about 370 MB unpacked and
about 325 MB compressed (2.8 GB before).

At the user's first setup `omamobile-setup-user` does what Omarchy's
`omarchy-voxtype-install` does, minus the download: it copies Omarchy's
Voxtype config, links the shipped model into
`~/.local/share/voxtype/models` (the only place Voxtype looks), enables
the packaged `voxtype.service` user daemon (it starts with the graphical
session) and marks Omarchy's "Install Dictation" invitation done. The
config and the link happen once per user, and a config or model the user
already has stays; the enable is retried at each login until it works
(firstboot may run without a user manager). The settings' Fast (tiny.en,
75 MB) and Accurate (small.en) models still download on request; without
tiny.en the live preview uses base.en, with a pass every 2.5 s instead of
1.2 s. Phones installed before this keep their setup: `omamobile-voice` is
an optional dependency, and Omarchy's own install works as before.

To remove dictation completely: Remove > Dictation (Omarchy's
`omarchy-voxtype-remove`) takes out voxtype-bin, the user's Voxtype config
and the model link, and setup leaves it removed; the shipped model stays
until `sudo pacman -R omamobile-voice` frees its 141 MiB. Installing
Dictation again through Omarchy's menu then downloads base.en itself.

Building `omamobile-voice` needs huggingface.co the first time on a build
machine: the model comes from a pinned commit of whisper.cpp's model
repository with a pinned sha256, and makepkg keeps it in its source cache
afterwards.

## File dialogs

Open, save and choose-a-folder requests through the portal (Qt, GTK, Electron and Flatpak apps alike) get Nautilus's libadwaita file chooser from xdg-desktop-portal-gnome, not xdg-desktop-portal-gtk's GTK 3 desktop chooser, whose Places sidebar took half of the 480 px width. Nautilus's chooser folds its sidebar away at phone width and has touch-sized rows; `hypr/phone.lua` tiles it, even when it has a parent window, so it fills the screen between the bar and the on-screen keyboard. Only FileChooser goes to the gnome backend: its other interfaces need gnome-shell. It runs without gnome-shell (its Mutter service channel is missing, so it falls back to a plain Wayland connection). xdg-desktop-portal-gtk stays as the fallback and for its other interfaces. GTK apps that would draw their own chooser in-process are sent to the portal too (`GTK_USE_PORTAL=1` for GTK 3, `GDK_DEBUG=portals` for GTK 4, in `environment.d/20-omamobile-osk.conf`); Qt Widgets apps keep Omarchy's `QT_QPA_PLATFORMTHEME=gtk3` for now.
Expand Down
44 changes: 41 additions & 3 deletions packages/omamobile-shell/bin/omamobile-setup-user
Original file line number Diff line number Diff line change
Expand Up @@ -402,8 +402,8 @@ if omarchy-done ensure omamobile-back-plugin; then
add_plugin omamobile.back
fi
# Voice input (omamobile.voice): hold the home pill to ask the agent, the
# keyboard strip's mic key to dictate, through Voxtype when it is
# installed (Install > AI > Dictation); without it the sheet says so.
# keyboard strip's mic key to dictate, through Voxtype (in the root image,
# set up below; or Install > AI > Dictation); without it the sheet says so.
if omarchy-done ensure omamobile-voice-plugin; then
add_plugin omamobile.voice
fi
Expand All @@ -415,6 +415,45 @@ if omarchy-done ensure omamobile-menu-voice; then
sed -i '0,/^{/s//{\n "setup.voice": {"icon":"\xf3\xb0\x8d\xac","label":"Voice","description":"Speech model and how voice input starts","action":"omarchy-shell shell summon omamobile.voice '"'"'{\\"page\\":\\"settings\\"}'"'"'"},/' "$extension"
fi
fi
# Voice out of the box: the root image ships voxtype-bin (rootfs/packages.base)
# and Whisper's base.en (omamobile-voice, in /usr/share/omamobile/voice/models),
# so a new user gets what Omarchy's Install > AI > Dictation
# (omarchy-voxtype-install) sets up, without its 142 MB download: Omarchy's
# Voxtype config, the model in Voxtype's models directory (a link to the
# shipped file; Voxtype looks only in ~/.local/share/voxtype/models) and the
# packaged user daemon enabled, so it starts with the graphical session.
# Omarchy's "Install Dictation" invitation is marked done, as its install
# would leave it. Only with both installed. The config, the link and the
# invitation happen once per user: a config or a model the user already has
# stays, and one they delete later isn't brought back. The enable has its
# own marker, set only when it succeeds, so a failed one (firstboot carries
# on without a user manager) is tried again at the next login. A user who
# removes Dictation (Remove > Dictation takes voxtype-bin out) gets nothing.
voxconf=~/.config/voxtype/config.toml
voxmodels=${OMAMOBILE_VOICE_MODELS:-/usr/share/omamobile/voice/models}
if command -v "${OMAMOBILE_VOXTYPE:-voxtype}" >/dev/null && [[ -f $voxmodels/ggml-base.en.bin ]]; then
if omarchy-done ensure omamobile-voxtype-setup; then
omarchy-done mark voxtype-install-invitation
voxdefault=${OMARCHY_PATH:-/usr/share/omarchy}/default/voxtype/config.toml
if [[ ! -f $voxconf && -f $voxdefault ]]; then
{ mkdir -p "${voxconf%/*}" && cp "$voxdefault" "$voxconf"; } ||
echo "omamobile-setup-user: could not copy Omarchy's Voxtype config" >&2
fi
mkdir -p ~/.local/share/voxtype/models || true
for model in "$voxmodels"/ggml-*.bin; do
[[ -e ~/.local/share/voxtype/models/${model##*/} ]] ||
ln -sfn "$model" ~/.local/share/voxtype/models/"${model##*/}" ||
echo "omamobile-setup-user: could not link ${model##*/}" >&2
done
fi
if omarchy-done check omamobile-voxtype-setup && ! omarchy-done check omamobile-voxtype-enabled; then
if systemctl --user enable voxtype.service; then
omarchy-done mark omamobile-voxtype-enabled
else
echo "omamobile-setup-user: could not enable voxtype.service; trying again at the next login" >&2
fi
fi
fi
# Voxtype on the phone (docs/VOICE-PLAN.md, phase 3): Omarchy's base.en
# stays the model, with Whisper's audio context cut to the clip's length
# (context_window_optimization, for clips under 22.5 s). On the Pixel 7 Pro
Expand All @@ -429,7 +468,6 @@ fi
# marker by more than a minute): a model the user chose since stays.
# Voxtype ignores a per-recording `--model` it hasn't prepared, so this is
# the daemon's own config; it restarts once, and only if it was running.
voxconf=~/.config/voxtype/config.toml
voxdone=~/.local/state/omarchy/done
if [[ -f $voxconf ]]; then
voxrestart=0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
# data_status the operator and technology, when registered
# dictation voxtype: on while recording, busy while transcribing, off
# when idle; down when its user service isn't running, none
# without voxtype (it is an optional Omarchy install)
# without voxtype (in the root image; Remove > Dictation
# takes it out)

t() { timeout 3 "$@" 2>/dev/null; }
have() { command -v "$1" >/dev/null 2>&1; }
Expand Down
5 changes: 3 additions & 2 deletions packages/omamobile-shell/plugins/omamobile.voice/Voice.qml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ import "Readable.js" as Readable
import "../omamobile.launcher/LauncherModel.js" as LauncherModel

// Voice input for the phone (docs/VOICE-PLAN.md): one voice bar, through
// Voxtype's user daemon (voxtype.service, Omarchy's Install > AI >
// Voxtype's user daemon (voxtype.service; in the root image, set up by
// omamobile-setup-user with the shipped base.en; or Omarchy's Install > AI >
// Dictation). Speech is turned into text on the phone.
//
// The ways in, as on Android: a hold on the pill in an app, or on the
Expand Down Expand Up @@ -525,7 +526,7 @@ Item {
id: startProc
onExited: function(exitCode) {
if (exitCode !== 0 && root.phase === "starting")
root.fail("Voxtype isn't running. Install it with Install > AI > Dictation, or start voxtype.service.")
root.fail("Voxtype isn't running. Start voxtype.service, or install it with Install > AI > Dictation.")
}
}

Expand Down
Loading