feat(ollama): route ACP agents to local Ollama models via env injection - #626
Draft
ChristianLuciani wants to merge 14 commits into
Draft
feat(ollama): route ACP agents to local Ollama models via env injection#626ChristianLuciani wants to merge 14 commits into
ChristianLuciani wants to merge 14 commits into
Conversation
Adds support for Ollama Launch (v0.15+), allowing AionCore users to run ACP agents with Ollama-hosted models via `ollama launch <agent>`, eliminating the need for API keys or provider configuration. Supported agents: claude, opencode, codex, copilot, pi, hermes, droid, qwen. Implementation: - Add OLLAMA_LAUNCH_MAP constant and helper functions to aionui-common - Create ollama module in aionui-ai-agent for runtime PATH detection - Add ollama_compatible computed field to AgentMetadata for UI filtering - Add use_ollama boolean flag to AcpBuildExtra for per-session control - Modify ACP factory to delegate to `ollama launch <agent>` when enabled - Graceful fallback to native launch preserves existing behaviour Testing: 619 existing + 6 new Ollama tests pass (625 total), cargo clippy -- -D warnings clean, cargo fmt clean, backwards compatible.
Ollama Launch requires an explicit --model when running without a TTY
('model selection requires an interactive terminal'). The previous
implementation ran 'ollama launch <agent>' with no --model, which
would fail in AionCore's headless child-process environment.
Changes:
- Add ollama_model: Option<String> to AcpBuildExtra
- Pass --model <ollama_model> -y when ollama_model is set
- Fall back to native launch with a WARN when use_ollama=true
but ollama_model is missing
- Add 4 deserialization tests for the new field
- Ensure struct-literal tests include the new field
Replace #[serde(default)] on use_ollama with an explicit custom Deserialize implementation. The Raw helper struct uses Option<bool> for use_ollama and maps None → false, ensuring that the presence of ollama_model alone never toggles use_ollama. This avoids potential serde version differences across CI environments where the derived default behavior may vary.
Contributor
Author
|
The frontend companion for this is now up as a draft: iOfficeAI/AionUi#3602 — it adds the Guid-page Ollama Launch model selector gated on |
…rapping ollama launch QA on the frontend companion (AionUi draft PR) surfaced that spawning 'ollama launch <agent> --model <model> -y' as the ACP child process always fails with 'Initialize handshake timed out after 30s'. Root cause, verified empirically with a PATH shim on Ollama 0.32.0: 'ollama launch claude' resolves the claude binary on PATH and execs the agent's *interactive TUI* with provider env vars applied (and forwards --model to it). It is not an ACP server wrapper, so a headless spawn never answers the ACP initialize request on stdio. Fix: keep the agent's native ACP command (bridge) and inject the same environment that ollama launch injects, captured from 0.32.0: ANTHROPIC_BASE_URL=http://127.0.0.1:11434 ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_API_KEY= (cleared) ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU}_MODEL=<model> CLAUDE_CODE_SUBAGENT_MODEL=<model> Verified headless: the claude ACP bridge answers initialize immediately with this env applied. Scope: OLLAMA_LAUNCH_MAP is replaced by OLLAMA_COMPATIBLE_BACKENDS, restricted to 'claude' — the only backend whose env mapping is verified end-to-end. Other agents (codex, qwen, ...) need their own verified mappings (some use config files, not env) and can be added in follow-ups. A unit test enforces that every backend advertised as ollama_compatible has an env mapping, so the silent-fallback branch is unreachable.
The prompt turn failed with JSON-RPC -32603 (errorKind: model_not_found) whenever the user's ~/.claude settings carried a persisted settings.model: the claude ACP bridge resolves the session model with priority ANTHROPIC_MODEL > settings.model > default, and build_ollama_env set no ANTHROPIC_MODEL, so a provider model that does not exist on Ollama leaked into the session (verified by driving @agentclientprotocol/claude-agent-acp 0.58.1 headless over stdio). ollama launch itself does not set ANTHROPIC_MODEL because it runs the interactive TUI where users can pick a model; headless ACP has no picker, so the selected Ollama model must be pinned explicitly. Also add the six telemetry/nonessential-traffic suppression variables the real ollama launch claude injects (captured from Ollama 0.32.1 via a PATH shim) so the agent does not call Anthropic endpoints with the placeholder credentials. Verified end-to-end: aioncore --local conversation with use_ollama=true and ollama_model=qwen2.5-coder:7b returns a model response through the full ACP stack.
ollama launch qwen configures qwen-code with pure environment variables (captured from Ollama 0.32.1 via a PATH shim): OPENAI_API_KEY=ollama, OPENAI_BASE_URL=http://127.0.0.1:11434/v1 and OPENAI_MODEL=<model>, plus --model/--auth-type openai CLI flags that are redundant headless — with OPENAI_API_KEY set, qwen --acp resolves the openai auth path and OPENAI_MODEL pins the session model (verified end-to-end against qwen-code 0.19.10 with a clean HOME, and through aioncore --local where the prompt turn completed against a local Ollama model). The remaining ollama launch integrations were probed the same way and cannot be driven by environment alone, so they stay excluded and are documented in the OLLAMA_COMPATIBLE_BACKENDS doc comment: codex receives -c model_providers.* CLI overrides, kimi a --config <json> flag its acp subcommand does not accept, and pi/droid get no injection at all (their launchers rewrite user config files interactively).
ChristianLuciani
changed the title
Bring feat/ollama-launch-acp up to date with upstream main. Conflict resolutions: - Keep both mcp_resolve (session-port) and ollama modules - Session-port still routes claude/codex by default; bypass to ACP when use_ollama && ollama_compatible so env injection can apply - Drop cli_detect.rs (removed upstream in iOfficeAI#678); ollama field only needed on remaining AgentMetadata test fixtures - Restore OLLAMA_COMPATIBLE_BACKENDS constants + tests on main layout
…n qwen version docs - Move OLLAMA_OPENAI_BASE_URL from ollama/mod.rs to aionui-common::constants so it lives alongside OLLAMA_DEFAULT_BASE_URL as the single source of truth. - Add VERSION-SENSITIVE comment block for qwen-code 0.19.10 auth resolution, matching the thoroughness of the claude mapping docs. - Fix AgentManagementRow.ollama_compatible doc comment: the mechanism is env injection, not 'Ollama Launch'. - Add cross-crate constant-consistency test. - Update integration test to use the canonical constant.
ChristianLuciani
marked this pull request as ready for review
July 27, 2026 20:51
resolve_agent_command_spec_with_ollama was injecting Ollama provider env vars (e.g. OPENAI_API_KEY=ollama) BEFORE apply_acp_launch_policy appended the host runtime_env. When the host had a real OPENAI_API_KEY, it overwrote the Ollama placeholder at the end of the vec, causing qwen-code to call the real OpenAI API with a model that doesn't exist there → the prompt turn failed. Fix: split the Ollama injection out of the command-resolution wrapper into a standalone inject_ollama_env() that runs AFTER apply_acp_launch_policy, so Ollama vars are truly the last entries in spec.env and override catalog, cc-switch, and host runtime_env variables alike. Added test: ollama_env_overrides_runtime_env verifies the qwen flow where a host OPENAI_API_KEY=sk-real must not leak past the Ollama route.
qwen-code requires --auth-type=openai to resolve the OpenAI auth path even when OPENAI_API_KEY is set in the environment. Without it the ACP bridge starts but the prompt turn fails silently (verified against qwen-code 0.19.10). The previous code assumed these flags were 'redundant for headless ACP' — that was incorrect. Fix: inject_ollama_env now appends --auth-type=openai and --model <model> to spec.args for the qwen backend, matching what 'ollama launch qwen' passes. Claude is unaffected (it reads provider config from env vars alone with no CLI auth flag needed). Updated the VERSION-SENSITIVE doc comment in ollama/mod.rs to reflect the corrected understanding, and extended ollama_env_overrides_runtime_env to verify the CLI flags are present.
ChristianLuciani
added a commit
to ChristianLuciani/AionUi
that referenced
this pull request
Jul 27, 2026
Adds an Ollama Launch selector to the Guid page "+" menu (and the mobile action sheet) for agents the backend flags as ollama_compatible. Picking a local model sends use_ollama + ollama_model in the conversation extra payload so AionCore spawns the agent via 'ollama launch <agent> --model <model>' instead of its native command; picking Off keeps the native launch. The fragment is only emitted together with a model, since the backend requires an explicit model for headless launches. Model discovery reads the local Ollama HTTP API (/api/tags) with a short timeout and degrades to an empty-state hint when Ollama is not running. The selector is desktop-only for now because remote WebUI sessions run in a browser on a different host than the one spawning agents. Companion to iOfficeAI/AionCore#626.
ChristianLuciani
added a commit
to ChristianLuciani/AionUi
that referenced
this pull request
Jul 27, 2026
…tion design The backend (iOfficeAI/AionCore#626) routes model calls to Ollama via provider env injection into the agent's native ACP command, not via 'ollama launch <agent>' (which starts an interactive TUI that never answers the ACP initialize handshake). Update the file-level doc, buildOllamaLaunchExtra comment, agentTypes.ts, and useGuidAssistantSelection.ts to match the final design.
ChristianLuciani
marked this pull request as draft
July 30, 2026 20:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⏸️ Paused — see discussion in #730