fix: make the fully-local (no API key) path work — PROXY_PORT, --provider ollama, proxy base URL, ONNX model paths - #228
Open
vidaunited wants to merge 5 commits into
Conversation
added 5 commits
August 18, 2026 06:08
`PROXY_PORT` is documented in `--help` but was read nowhere, so the proxy port was the hardcoded field `cli-proxy.ts:56`. Any process already on :3000 — Docker Desktop, a dev server — broke every proxied provider with no way out. A port fix has two halves, and fixing one is invisible: cli-proxy.ts decides where the proxy LISTENS claudeAgentDirect.ts decides where the SDK DIALS (hardcoded localhost:3000) With only the first, `PROXY_PORT=3991` prints a confident `🔧 Proxy URL: http://localhost:3991` and then fails with `Connection error` — the banner reports the bind, never the dial, so the log argues the fix worked. Measured on one identical command, everything else held constant: cli-proxy.ts only rc=1, no model output, "Connection error" both halves rc=0, model responded, no error The onnx client keeps `ONNX_PROXY_PORT` (default 3001) to match `cli-proxy.ts`'s own `parseInt(process.env.ONNX_PROXY_PORT || '3001')` — collapsing all four onto PROXY_PORT would break the ONNX path. Refs ruvnet#81, which reported this fix shipping in 2.0.1-alpha.5; both halves are absent from 2.1.2.
`--provider ollama` had no code path in the agent runner. It fell through every
provider guard into the `ANTHROPIC_API_KEY is required` branch — an error whose
three suggested alternatives never mention ollama, so there was no way to tell
the provider was unimplemented rather than misconfigured.
The capability was half-present: router.ts constructs an OllamaProvider and
router/providers/ollama.ts ships, but src/proxy/ has anthropic-to-{gemini,onnx,
openrouter,requesty} and no anthropic-to-ollama, so the CLI had nothing to
route to.
This needs no new proxy. Ollama exposes an OpenAI-compatible
/v1/chat/completions, which is what AnthropicToOpenRouterProxy already speaks,
and that class already accepts an openrouterBaseUrl. So startOllamaProxy()
reuses it against OLLAMA_HOST/v1 with a placeholder key — Ollama ignores
Authorization, and requiring a real key would defeat the point of a local
provider.
- shouldUseOllama(), matching the existing shouldUseX shape
- an early return in shouldUseOpenRouter() so a stray OPENROUTER_API_KEY
cannot capture an explicit ollama request via key-based auto-selection
- isOllama in the API-key guard, so a keyless local provider is not rejected
- ollama listed in --help and in the missing-key error
- OLLAMA_HOST / OLLAMA_MODEL honoured, defaulting to
http://localhost:11434 and qwen2.5-coder:7b
Verified end to end against 2.1.2 with no API key set anywhere: 5/5 exact
responses on qwen2.5-coder:7b, and the run is refused correctly when Ollama is
down rather than silently falling back to a paid API.
Refs ruvnet#146, which reports the same provider gap in the config wizard.
`agentic-flow proxy --provider openrouter` built AnthropicToOpenRouterProxy without openrouterBaseUrl, so the standalone proxy could only ever talk to https://openrouter.ai/api/v1. Three construction sites, one inconsistent: cli-proxy.ts startOpenRouterProxy passes it proxy/anthropic-to-openrouter.ts entry passes it cli-proxy.ts runStandaloneProxy did not The constructor already defaults to the public OpenRouter URL when the field is absent, so passing it through is non-breaking. This matters because `proxy` is the documented way to put an Anthropic-shaped endpoint in front of Claude Code or Cursor, and as written it could not target a self-hosted OpenAI-compatible backend (Ollama, vLLM, LM Studio, LiteLLM, an internal gateway) — the main reason to run a local proxy. The workaround was to bypass the CLI and run the proxy module directly, since only its own entrypoint read the variable. Verified against 2.1.2: the startup banner now reports the configured base URL and requests reach it.
The Phi-4-mini download (~4.6GB) used a localPath hardcoded RELATIVE to
process.cwd(), with no environment override, so it landed in whatever directory
the command ran from. Measured: 915MB in about 60 seconds into a git repository
root, untracked and unignored.
Consequences:
- the same model is re-downloaded once per working directory
- a `git add -A` in that repo will stage several GB
- no way to point it at a shared cache, and in particular not at
~/.agentic-flow/models, where `embeddings init` already puts its model —
the two subsystems disagreed about where models live
Paths now resolve to AGENTIC_FLOW_MODEL_DIR, defaulting to
~/.agentic-flow/models, and are exported so the reader can resolve the same
path the writer uses (see the following commit).
) The downloader writes models/phi-4-mini/... while every consumer but one defaulted to models/phi-4/... — no `-mini`. The two can never coincide, so a fully completed ~4.6GB download was not found by the code that needed it. The mismatched default is live on the ordinary CLI path, because no modelPath is supplied unless ONNX_MODEL_PATH is set: cli-proxy.ts new AnthropicToONNXProxy({ modelPath: process.env.ONNX_MODEL_PATH }) anthropic-to-onnx.ts forwards config.modelPath to ONNXLocalProvider onnx-local.ts falls back to ./models/phi-4/... <- never written All six sites now resolve from the single exported PHI4_MODEL_PATH, which also takes them out of process.cwd() (previous commit): router/providers/onnx-local.ts was phi-4 router/providers/onnx-phi4.ts was phi-4 router/router.ts was phi-4 router/test-onnx-local.ts was phi-4 router/test-onnx-benchmark.ts was phi-4 router/providers/onnx-local-optimized.ts was phi-4-mini, but still cwd-relative onnx-local.ts and onnx-local-optimized.ts already imported from model-downloader.js, so this adds no new dependency edge for them. Together with ruvnet#226 this is what makes `--provider onnx` — the option the missing-API-key error steers users toward — actually usable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes five defects that together make
agentic-flowunusable for a fully local,no-API-key setup. Each commit is atomic and maps to one issue.
5273a11PROXY_PORThonoured on the bind but not on the diala07574d--provider ollamaaccepted and then silently ignoredf0cc1earunStandaloneProxydropsANTHROPIC_PROXY_BASE_URLbb3f2f5process.cwd()2d2d24aThey are in one PR because three of the five touch
src/cli-proxy.tsandseparate PRs would conflict on merge. Happy to split per issue if you prefer —
say the word and I'll open five.
#223 —
PROXY_PORTon the dialcli-proxy.tsreadPROXY_PORTwhen starting the proxy butclaudeAgentDirect.tshardcoded
http://localhost:3000in three places, so settingPROXY_PORTmoved thelistener and left the client dialling the old port. This is a regression of #81.
Also fixes the ONNX sibling at line 58 (
ONNX_PROXY_PORT, default 3001), which hadthe same shape.
Verified: with
:3000deliberately held by another process,PROXY_PORT=3999now completes a request end to end. Before the change the same invocation failed
ECONNREFUSED 127.0.0.1:3000.#224 —
--provider ollama--provider ollamaparsed without error, then fell through everyshouldUseX()guard to the Anthropic path and demanded
ANTHROPIC_API_KEY. There was no codepath to a local Ollama at all.
Ollama already serves an OpenAI-compatible
/v1/chat/completions, so this needs nonew proxy class —
AnthropicToOpenRouterProxyis reused withopenrouterBaseUrl: ${host}/v1and a placeholder key (Ollama ignores auth).9 edits: the guard, the branch, the launcher, the key check, the error text, the
display line and the help text.
Verified:
--provider ollama --model qwen2.5-coder:7bwith no API key set atall, asked to echo a nonsense sentinel token, returned it exactly 5 times out
of 5. The sentinel was chosen to have no plausible alternate spelling after a
first attempt used a real word and the model "corrected" it.
#225 —
openrouterBaseUrlin the standalone proxyrunStandaloneProxy()constructed the proxy without passingopenrouterBaseUrl,so
ANTHROPIC_PROXY_BASE_URLwas silently ignored in exactly the mode where auser is most likely to set it.
#226 / #227 — ONNX model paths
model-downloader.tswrote the model underjoin(process.cwd(), '.agentic-flow', ...)while six reader sites each rebuilt their own path — some relative to
cwd, somenot. Running the CLI from a different directory silently re-downloaded, and the
reader could look somewhere the writer had never written.
#226 exports
MODEL_ROOT/PHI4_MODEL_PATH/PHI4_MODEL_DATA_PATHrooted at~/.agentic-flow/models(overridable viaAGENTIC_FLOW_MODEL_DIR).#227 points all six readers at the exported constant, so writer and readers cannot
diverge again.
Verification notes, including the limits
dist/with the equivalentedits applied, not against a fresh
npm run buildof this branch — the clone hasno
node_modulesand this repo's install is large. The TypeScript here is thesame change, typecheck-clean, but the built artefact was not re-verified from
this branch and I'd rather say so than imply otherwise.
tsc --noEmit --skipLibCheck. ZeroTS1xxxsyntaxerrors introduced. Two pre-existing diagnostics (
TS2591,TS18048) areunchanged — confirmed by stashing the branch and re-running: 2 before, 2 after.
Anthropic" is the error shape, not the exit code: before the change, 7 runs
produced
proxy_errorand 0 produced a completion; after, 0proxy_errorandthe completions above.
ECONNREFUSEDwent 2 → 0 for PROXY_PORT is documented in --help but read nowhere in 2.1.2 (regression of #81) #223 on the same basis.byte-identical to the original.
Tested on macOS 26.6.1 (arm64), Node v22.23.0, Ollama 0.32.5 with
qwen2.5-coder:7b.