Skip to content

fix: make the fully-local (no API key) path work — PROXY_PORT, --provider ollama, proxy base URL, ONNX model paths - #228

Open
vidaunited wants to merge 5 commits into
ruvnet:mainfrom
vidaunited:fix/local-provider-and-proxy-port
Open

fix: make the fully-local (no API key) path work — PROXY_PORT, --provider ollama, proxy base URL, ONNX model paths#228
vidaunited wants to merge 5 commits into
ruvnet:mainfrom
vidaunited:fix/local-provider-and-proxy-port

Conversation

@vidaunited

Copy link
Copy Markdown

Fixes five defects that together make agentic-flow unusable for a fully local,
no-API-key setup. Each commit is atomic and maps to one issue.

Commit Issue Defect
5273a11 #223 PROXY_PORT honoured on the bind but not on the dial
a07574d #224 --provider ollama accepted and then silently ignored
f0cc1ea #225 runStandaloneProxy drops ANTHROPIC_PROXY_BASE_URL
bb3f2f5 #226 ONNX model paths resolved under process.cwd()
2d2d24a #227 ONNX downloader and readers resolve different paths

They are in one PR because three of the five touch src/cli-proxy.ts and
separate PRs would conflict on merge. Happy to split per issue if you prefer
say the word and I'll open five.


#223PROXY_PORT on the dial

cli-proxy.ts read PROXY_PORT when starting the proxy but claudeAgentDirect.ts
hardcoded http://localhost:3000 in three places, so setting PROXY_PORT moved the
listener and left the client dialling the old port. This is a regression of #81.

Also fixes the ONNX sibling at line 58 (ONNX_PROXY_PORT, default 3001), which had
the same shape.

Verified: with :3000 deliberately held by another process, PROXY_PORT=3999
now completes a request end to end. Before the change the same invocation failed
ECONNREFUSED 127.0.0.1:3000.

#224--provider ollama

--provider ollama parsed without error, then fell through every shouldUseX()
guard to the Anthropic path and demanded ANTHROPIC_API_KEY. There was no code
path to a local Ollama at all.

Ollama already serves an OpenAI-compatible /v1/chat/completions, so this needs no
new proxy class — AnthropicToOpenRouterProxy is reused with
openrouterBaseUrl: ${host}/v1 and a placeholder key (Ollama ignores auth).
9 edits: the guard, the branch, the launcher, the key check, the error text, the
display line and the help text.

Verified: --provider ollama --model qwen2.5-coder:7b with no API key set at
all
, asked to echo a nonsense sentinel token, returned it exactly 5 times out
of 5
. The sentinel was chosen to have no plausible alternate spelling after a
first attempt used a real word and the model "corrected" it.

#225openrouterBaseUrl in the standalone proxy

runStandaloneProxy() constructed the proxy without passing openrouterBaseUrl,
so ANTHROPIC_PROXY_BASE_URL was silently ignored in exactly the mode where a
user is most likely to set it.

#226 / #227 — ONNX model paths

model-downloader.ts wrote the model under join(process.cwd(), '.agentic-flow', ...)
while six reader sites each rebuilt their own path — some relative to cwd, some
not. Running the CLI from a different directory silently re-downloaded, and the
reader could look somewhere the writer had never written.

#226 exports MODEL_ROOT / PHI4_MODEL_PATH / PHI4_MODEL_DATA_PATH rooted at
~/.agentic-flow/models (overridable via AGENTIC_FLOW_MODEL_DIR).
#227 points all six readers at the exported constant, so writer and readers cannot
diverge again.


Verification notes, including the limits

  • Behaviour was verified against the built 2.1.2 dist/ with the equivalent
    edits applied, not against a fresh npm run build of this branch — the clone has
    no node_modules and this repo's install is large. The TypeScript here is the
    same change, typecheck-clean, but the built artefact was not re-verified from
    this branch
    and I'd rather say so than imply otherwise.
  • Each commit typechecks with tsc --noEmit --skipLibCheck. Zero TS1xxx syntax
    errors introduced. Two pre-existing diagnostics (TS2591, TS18048) are
    unchanged — confirmed by stashing the branch and re-running: 2 before, 2 after.
  • For --provider ollama is unimplemented in the agent path and fails with "ANTHROPIC_API_KEY is required" #224 the discriminator between "the local path ran" and "it fell through to
    Anthropic" is the error shape, not the exit code: before the change, 7 runs
    produced proxy_error and 0 produced a completion; after, 0 proxy_error and
    the completions above. ECONNREFUSED went 2 → 0 for PROXY_PORT is documented in --help but read nowhere in 2.1.2 (regression of #81) #223 on the same basis.
  • The edits were applied and reverted round-trip; the reverted tree was
    byte-identical to the original.

Tested on macOS 26.6.1 (arm64), Node v22.23.0, Ollama 0.32.5 with qwen2.5-coder:7b.

Mohammed Al-Ajmi added 5 commits August 18, 2026 06:08
`PROXY_PORT` is documented in `--help` but was read nowhere, so the proxy port
was the hardcoded field `cli-proxy.ts:56`. Any process already on :3000 — Docker
Desktop, a dev server — broke every proxied provider with no way out.

A port fix has two halves, and fixing one is invisible:

  cli-proxy.ts          decides where the proxy LISTENS
  claudeAgentDirect.ts  decides where the SDK DIALS  (hardcoded localhost:3000)

With only the first, `PROXY_PORT=3991` prints a confident
`🔧 Proxy URL: http://localhost:3991` and then fails with `Connection error` —
the banner reports the bind, never the dial, so the log argues the fix worked.

Measured on one identical command, everything else held constant:

  cli-proxy.ts only   rc=1, no model output, "Connection error"
  both halves         rc=0, model responded, no error

The onnx client keeps `ONNX_PROXY_PORT` (default 3001) to match
`cli-proxy.ts`'s own `parseInt(process.env.ONNX_PROXY_PORT || '3001')` —
collapsing all four onto PROXY_PORT would break the ONNX path.

Refs ruvnet#81, which reported this fix shipping in 2.0.1-alpha.5; both halves are
absent from 2.1.2.
`--provider ollama` had no code path in the agent runner. It fell through every
provider guard into the `ANTHROPIC_API_KEY is required` branch — an error whose
three suggested alternatives never mention ollama, so there was no way to tell
the provider was unimplemented rather than misconfigured.

The capability was half-present: router.ts constructs an OllamaProvider and
router/providers/ollama.ts ships, but src/proxy/ has anthropic-to-{gemini,onnx,
openrouter,requesty} and no anthropic-to-ollama, so the CLI had nothing to
route to.

This needs no new proxy. Ollama exposes an OpenAI-compatible
/v1/chat/completions, which is what AnthropicToOpenRouterProxy already speaks,
and that class already accepts an openrouterBaseUrl. So startOllamaProxy()
reuses it against OLLAMA_HOST/v1 with a placeholder key — Ollama ignores
Authorization, and requiring a real key would defeat the point of a local
provider.

  - shouldUseOllama(), matching the existing shouldUseX shape
  - an early return in shouldUseOpenRouter() so a stray OPENROUTER_API_KEY
    cannot capture an explicit ollama request via key-based auto-selection
  - isOllama in the API-key guard, so a keyless local provider is not rejected
  - ollama listed in --help and in the missing-key error
  - OLLAMA_HOST / OLLAMA_MODEL honoured, defaulting to
    http://localhost:11434 and qwen2.5-coder:7b

Verified end to end against 2.1.2 with no API key set anywhere: 5/5 exact
responses on qwen2.5-coder:7b, and the run is refused correctly when Ollama is
down rather than silently falling back to a paid API.

Refs ruvnet#146, which reports the same provider gap in the config wizard.
`agentic-flow proxy --provider openrouter` built AnthropicToOpenRouterProxy
without openrouterBaseUrl, so the standalone proxy could only ever talk to
https://openrouter.ai/api/v1.

Three construction sites, one inconsistent:

  cli-proxy.ts  startOpenRouterProxy       passes it
  proxy/anthropic-to-openrouter.ts  entry  passes it
  cli-proxy.ts  runStandaloneProxy         did not

The constructor already defaults to the public OpenRouter URL when the field is
absent, so passing it through is non-breaking.

This matters because `proxy` is the documented way to put an Anthropic-shaped
endpoint in front of Claude Code or Cursor, and as written it could not target a
self-hosted OpenAI-compatible backend (Ollama, vLLM, LM Studio, LiteLLM, an
internal gateway) — the main reason to run a local proxy. The workaround was to
bypass the CLI and run the proxy module directly, since only its own entrypoint
read the variable.

Verified against 2.1.2: the startup banner now reports the configured base URL
and requests reach it.
The Phi-4-mini download (~4.6GB) used a localPath hardcoded RELATIVE to
process.cwd(), with no environment override, so it landed in whatever directory
the command ran from. Measured: 915MB in about 60 seconds into a git repository
root, untracked and unignored.

Consequences:
  - the same model is re-downloaded once per working directory
  - a `git add -A` in that repo will stage several GB
  - no way to point it at a shared cache, and in particular not at
    ~/.agentic-flow/models, where `embeddings init` already puts its model —
    the two subsystems disagreed about where models live

Paths now resolve to AGENTIC_FLOW_MODEL_DIR, defaulting to
~/.agentic-flow/models, and are exported so the reader can resolve the same
path the writer uses (see the following commit).
)

The downloader writes models/phi-4-mini/... while every consumer but one
defaulted to models/phi-4/... — no `-mini`. The two can never coincide, so a
fully completed ~4.6GB download was not found by the code that needed it.

The mismatched default is live on the ordinary CLI path, because no modelPath is
supplied unless ONNX_MODEL_PATH is set:

  cli-proxy.ts          new AnthropicToONNXProxy({ modelPath: process.env.ONNX_MODEL_PATH })
  anthropic-to-onnx.ts  forwards config.modelPath to ONNXLocalProvider
  onnx-local.ts         falls back to ./models/phi-4/...   <- never written

All six sites now resolve from the single exported PHI4_MODEL_PATH, which also
takes them out of process.cwd() (previous commit):

  router/providers/onnx-local.ts            was phi-4
  router/providers/onnx-phi4.ts             was phi-4
  router/router.ts                          was phi-4
  router/test-onnx-local.ts                 was phi-4
  router/test-onnx-benchmark.ts             was phi-4
  router/providers/onnx-local-optimized.ts  was phi-4-mini, but still cwd-relative

onnx-local.ts and onnx-local-optimized.ts already imported from
model-downloader.js, so this adds no new dependency edge for them.

Together with ruvnet#226 this is what makes `--provider onnx` — the option the
missing-API-key error steers users toward — actually usable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant