Local engine model management: desktop UI by rexlunae · Pull Request #487 · rexlunae/RustyClaw

rexlunae · 2026-08-18T05:45:34Z

What & why

The desktop "Local Engines & Models" dialog becomes a real management surface, and the composer's model picker reflects what is actually available and running locally. Depends on #486 (core/gateway/protocol) — merge in order.

Changes

Parameters editor (components/engines.rs)

Each engine tab gains a Parameters editor: context window, device, huge pages, mmap, lazy weights, max output tokens, max concurrency, default model (a dropdown of the local model list), and auto-start. Saved via the new EngineConfigSet command and persisted to config.toml; applied on the next Start/Load, with a Restart button to apply immediately.
The form state is keyed per engine and re-seeds when the gateway reports a changed config; edits never apply to a stale base.

Running-model indication

The model table marks loaded models as running (from the engine registry's host process detection) vs "on disk", and the composer's model dropdown appends a ● running marker to models the local engine reports as loaded (via the new ProviderModelListResult.loaded field).

Load/Unload feedback

The clicked row's button shows Loading… and is disabled until the gateway answers; the outcome (success or an honest error, e.g. "served by a joshua started outside RustyClaw (port 8080)") renders as a dismissible inline alert in the dialog, plus the existing toast.

Usability

The dialog auto-loads the active engine's model list when it opens instead of waiting for a tab click.
Model actions can carry per-model context overrides (Joshua --n-ctx).

Testing

cargo check --workspace, clippy clean; desktop (117) and view (87) tests pass.
Verified live against a preview gateway + desktop client: parameter save/round-trip, running markers for real on-host servers, Loading…/inline-error feedback, and auto-loading contents on open.

Local inference engines now have real model management backing the UI: - EngineConfig gains typed, optional parameters (context_length, device, huge_pages, mmap, lazy_weights, max_output_tokens, max_concurrency) that Joshua maps to its serve flags (--n-ctx/--device/--huge-pages/--mmap/ --lazy-weights/--max-output-tokens/--max-concurrency); llama.cpp maps context_length to --ctx-size. extra_args remains the escape hatch and still wins on the command line. - The engine registry detects servers already running on the host (joshua serve / llama-server, Linux, best-effort) and reports them as running with the models they serve, so the UI shows what is actually loaded even when a server was started outside RustyClaw. - stop() now receives the engine config so engines scope their kill to their own port instead of pkill'ing every matching server on the host; Load/Unload/Start are honest about externally-run servers (refuse to kill them, report which port they are on, don't spawn a duplicate). - provider_models_with_local_fallback() merges the live provider API list with the engine's on-disk model list (GGUF scans for Joshua/llama.cpp), clears the fetch error when local models exist, and reports which models are loaded — the data behind on-disk model pickers. - Joshua's spawn reports a real error when the server never answers (instead of a forever "may still be loading"). - The gateway's per-turn model-call deadline is configurable via RUSTYCLAW_MODEL_TIMEOUT_SECS (default 180) — local engines on loaded machines can exceed the old hard-coded cap while prefilling. - Wire/protocol: EngineConfigSet client command, EngineInfoDto carries the full engine config, ProviderModelListResult carries loaded-model ids. Config example + CHANGELOG updated.

- joshua_serve_flags only emits --device/--huge-pages for the values Joshua actually accepts (auto/cpu/metal/cuda and transparent/2mb/1gb/ huge), so a hostile free-form config value can no longer inject shell syntax into the spawn command. Numeric fields were already safe. - Joshua's spawn no longer declares failure when the server has simply not answered yet: after the health-probe window it checks whether the spawned process is still alive and reports "still loading" if so, reserving the error for a process that actually exited. Switching to a large model no longer shows a scary error while the model loads fine moments later. - llama.cpp remove resolves the model name back to the scanned on-disk path (file stem, possibly inside a per-repo subdirectory) before deleting, and fails with the available list when nothing matches — previously it rm -f'd a non-existent path and reported phantom success while the file stayed on disk. - ProviderModels.models/.error gain the doc comments the style guide requires for public fields.

Protocol: the enriched payloads now ride in new frames with new pinned discriminants instead of widening existing messages — the wire format is positional bincode and deserialize_frame rejects trailing bytes, so adding fields to ProviderModelListResult/EngineInfoDto broke older peers (the project's own protocol rule: new capabilities need new frames). EngineConfigList (after EngineListResult) and ProviderModelLoadedList (after ProviderModelListResult) carry the engine configs and the loaded- model markers; the old frames are byte-identical to before. Joshua start: only a server on the engine's own configured port counts as 'already running'. A joshua started outside RustyClaw on another port no longer prevents the configured server from starting (or Restart from bringing it back). llama.cpp stop: reports honestly — 'no llama-server is running on port N' when nothing matched, instead of always claiming success while a detected server on another port keeps serving. llama.cpp start: applies the typed context window (--ctx-size) on manual starts too, matching the auto-start path. Clients gain minimal arms for the two new frames here; the desktop/TUI consume them in their UI PRs.

- Joshua spawn: the 'process exited' hard failure is only reported on Linux, where process inspection exists; on other platforms a slow load falls back to the informational 'may still be loading' instead of a false failure. Same for the llama.cpp stop check. - llama.cpp auto-start: the service definition always passes the resolved --port (not only when one is configured), so the port-scoped stop can actually identify auto-started servers instead of silently matching nothing. - llama.cpp stop: on non-Linux (no process inspection) it falls back to stopping every llama-server rather than claiming success while one keeps running. - llama.cpp start: the configured models directory is interpolated via sh_quote instead of naive single quotes. - running_server_cmdlines: no unused-variable warning on non-Linux builds (cfg_attr allow with the Linux-only body). - Joshua load: 'already loaded' is only claimed when the configured endpoint actually serves the model; a joshua started outside RustyClaw on another port no longer makes Load report success while the configured server has nothing loaded.

- Bump WIRE_PROTOCOL_VERSION to 4: EngineConfig gained its typed parameter fields (the shipped EngineConfigSet payload widened positionally) and the EngineConfigList / ProviderModelLoadedList frames are new, so a mismatched peer must fail at the first affected frame instead of mis-parsing. - Joshua: sh_quote the model path and every extra_arg interpolated into the serve command line; server_process_alive is Linux-only (dead code and an unused parameter on other platforms). - llama.cpp: stop() keeps its port/pattern bindings inside the Linux block (no unused-variable warnings on macOS/non-Linux); start() sh_quotes extra_args; load() falls back to the typed context_length. - engine_start_command: llama.cpp built-in flags now come first so a hand-written --port/--ctx-size in extra_args overrides them (auto-start). - Ollama: load() applies the typed context_length (--num-ctx) when no per-load override is given; list_models marks models resident per /api/ps instead of never marking anything loaded. - LM Studio: list_models no longer claims every listed model is running.

- llama.cpp list_models: a model served by a running llama-server but living outside the scanned models dir (e.g. started manually with an explicit --model path) was dropped from the list — loaded ids are now surfaced like Joshua's list_models does. - Port-scoped stop/liveness patterns terminate the digit run (joshua pkill '...127.0.0.1:{port}( |$)' and llamacpp '...--port {port}( |$)', and joshua's server_process_alive now requires a non-digit after the port), so a short port (1234/808) can no longer match a server on 12345/8080. - Engine status only reports Running when a detected server sits on the engine's own configured port (or the configured endpoint answers); a foreign server on another port no longer hides the Start button or turns Stop into a no-op that claims success. - validate_engine_config() rejects structurally unusable configs (port 0) before they are persisted or reach the pkill/pgrep patterns; the gateway's EngineConfigSet handler refuses and reports the error instead of relying on the port's u16 type alone.

…odel pickers The Local Engines & Models dialog becomes a real management surface and the composer picker reflects what is actually available locally: - Each engine tab gains a Parameters editor persisted via EngineConfigSet (context window, device, huge pages, mmap, lazy weights, max output tokens, max concurrency, default model picked from the local model list, auto-start). Applied on the next Start/Load, with a Restart button to apply immediately. - The model table marks loaded models as "running" (from the engine registry's host process detection) alongside on-disk ones, and the composer's model dropdown appends a "● running" marker to models the local engine reports as loaded. - Load/Unload give real feedback: the clicked row's button shows "Loading…" until the gateway answers, and the outcome (success or an honest error) is rendered inline in the dialog, dismissible. - The dialog auto-loads the active engine's model list when it opens, instead of waiting for a tab click. - The engines dialog's EngineModelAction now carries per-model context overrides for Joshua (--n-ctx).

- Engine configs and loaded-model markers now arrive in their own frames (EngineConfigList / ProviderModelLoadedList) matching the protocol change on the backend: the panel entries get their config patched from EngineConfigList, and the picker's running markers come from ProviderModelLoadedList. - Load actions honour the context window saved in the engine parameters (the gateway maps it per engine: --n-ctx / --ctx-size / --num-ctx), so the saved value actually applies for llama.cpp and Ollama instead of being silently ignored. - The Restart button now works for engines with an unrelated server running on another port (backend start guard is port-scoped); it stops and restarts the engine's own server.

- A load/unload action whose answer never arrives (dropped connection) left the model buttons stuck on Loading… forever: the Disconnected handler now clears the in-flight action marker and its result. - Saving engine parameters no longer blanks out enabled/endpoint/port/ models_dir/extra_args when the EngineConfigList snapshot has not arrived: the Save button is disabled (with a hint) until the real configs are in, since the panel entries are placeholders before that. - Drop the stray, unreachable global_settings.rs dialog from the branch (it was never declared as a module and references types that do not exist); the on-disk copy is left untouched.

The squashed backend kept the placeholder arms for the new ProviderModelLoadedList/EngineConfigList frames; keep the desktop's real handling (patched configs + loaded markers + the configs_received guard).

devin-ai-integration

Devin Review found 1 new potential issue.

View 9 additional findings in Devin Review.

devin-ai-integration · 2026-08-18T08:33:06Z

+                    state
+                        .write()
+                        .engine_model_action_pending = Some((engine.clone(), model.clone()));


🟡 Model buttons can stay stuck on "Loading…" when the request never leaves the app

The clicked model is flagged as busy (engine_model_action_pending = Some(...) at crates/rustyclaw-desktop/src/app/dialogs.rs:809-811) before it is known whether the request can actually be sent, so when it cannot the row's buttons stay greyed out and stuck on "Loading…".
Impact: Load/Unload/Remove for that model become unusable until the connection drops or the app is restarted, with no error shown.

Pending flag set unconditionally; only a gateway answer or a Disconnected event clears it

crates/rustyclaw-desktop/src/app/dialogs.rs:809-811 sets engine_model_action_pending before gateway.read() is checked. If gw is None, nothing is ever sent. Likewise, if client.send(...) returns Err (queue closed/full path), the spawned task only logs (tracing::error!("Failed to send model action: {}", e)). The flag is cleared only in GatewayEvent::EngineActionResult (crates/rustyclaw-desktop/src/app_support.rs:1007-1013) or on GatewayEvent::Disconnected (crates/rustyclaw-desktop/src/app_support.rs:191). Meanwhile the dialog disables all three action buttons for that row via pending_here (crates/rustyclaw-desktop/src/components/engines.rs:704-708, 729-731, 742-745, 758).

Setting the flag only after a successful enqueue (or clearing it in the error branch, and surfacing the failure through engine_action_result) keeps the buttons honest.

Prompt for agents

In crates/rustyclaw-desktop/src/app/dialogs.rs, `on_model_action` sets `state.write().engine_model_action_pending = Some((engine, model))` before checking that a gateway client exists and before knowing whether `client.send(GatewayCommand::EngineModelAction { .. })` succeeded. If there is no client, or the send errors (the spawned task only logs), the flag is never cleared and the engines dialog keeps the row's Load/Unload/Remove buttons disabled with a "Loading…" label (see the `pending_here` usage in crates/rustyclaw-desktop/src/components/engines.rs). Only `EngineActionResult` or a `Disconnected` event clears it (crates/rustyclaw-desktop/src/app_support.rs). Fix by only marking the action pending once the command has actually been enqueued, and clearing the pending marker (optionally recording an inline failure via `engine_action_result`) in the send-error branch.

Was this helpful? React with 👍 or 👎 to provide feedback.

rexlunae mentioned this pull request Aug 18, 2026

Local engine model management: TUI #488

Merged