feat: add generic local provider (--local) with onboarding#4
Open
styles01 wants to merge 7 commits into
Open
Conversation
styles01
force-pushed
the
feat/local-provider
branch
from
a86c7fa to
225aa73
Compare
… name capture bug Add --local as a first-class Claude Code provider backed by a user-configured local gateway (e.g. Flow LLM). Includes ai local-onboard for interactive setup, provider registration, status output, docs, and smoke test coverage. Fix: _local_provider_select_model sent display output (model list banner) to stdout, which command substitution captured into the model name variable. The resulting blob was written to config and sent as the model ID to the API. Redirect display echoes to stderr so only the selected model name is captured. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds --dgx flag for accessing a remote Ollama instance over VPN (default: http://DGXSPARK-A:11434). Configurable via DGX_HOST and DGX_MODEL_HIGH/MID/LOW env vars. Works with Claude Code runtime. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…add timeouts - Added dgx to fallback conditionals in scripts/ai (was only in setup.sh) - Added dgx to tool_supported_providers in tools/claude-code.sh - Added dgx to --chrome incompatibility warning in both scripts/ai and setup.sh - Added --connect-timeout 5 to curl calls in dgx.sh (was missing, risked hangs over VPN) - Fixed provider_model_available grep to avoid prefix false positives Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…mpatibility Claude Code 2.1+ ignores the ANTHROPIC_MODEL env var, so pipe it through as a --model CLI flag instead. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The local provider probe only treated 400/422 responses as evidence
that the Messages API is implemented. Backends like flow return 503
with a valid Anthropic error envelope (`{"type":"error","error":{...}}`)
when the requested model is registered but not loaded — this still
proves the API exists and works, so it should be "ready" too. The
same applies to any other 4xx with a JSON body (e.g. flow on an
unknown model).
Without this, `ai --local` against flow fails the probe and reports
"flow returned an unexpected response from the Messages API probe"
even though flow is healthy. provider_setup_env will still fail
later with a specific message if the configured default model is
also unloaded.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude Code 2.1+ emits `{"type":"thinking","thinking":"..."}` blocks
in addition to (or sometimes instead of) `{"type":"text","text":"..."}`
blocks. The jq filters in tool_execute_prompt() only matched
type=="text", so thinking blocks were silently dropped from the
display stream.
Update all three filters to match both types and use (.text // .thinking)
for the value. Mirrors the fix in cc21-thinking-blocks memory.
Non-thinking models are unaffected (they emit text blocks).
For thinking-enabled models (gemma-4 it, deepseek, kimi, etc.) the
user now sees both the model's reasoning and its response.
Co-Authored-By: Claude <noreply@anthropic.com>
Add HANDOFF-flow-thinking-blocks.md documenting the upstream flow
bug where thinking-only content blocks in conversation history caused
a 400 ("Content block type 'thinking' is not supported in this
Anthropic MVP") on the next user turn. The bug has since been fixed
in flow; this doc remains as a regression reference and a template
for similar future bugs in the proxy layer.
Reproduction, root cause analysis, and three suggested fixes are
included.
Also update CHANGELOG with an Unreleased entry covering:
- the local provider probe fix
- the claude-code thinking-block display fix
- this handoff doc
Co-Authored-By: Claude <noreply@anthropic.com>
styles01
force-pushed
the
feat/local-provider
branch
from
907f689 to
8316d7e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--localas a first-class Claude Code provider, mirroring the existing--ollamaand--lmstudiopatternsai local-onboardfor interactive setup of a user-configured local Anthropic-compatible gateway (e.g. Flow LLM)_local_provider_select_modelsent display output to stdout, causing the model list banner to be captured into the model name variable by command substitution — resulting in an invalid model ID being sent to the APIWhat this enables
Changes
providers/local.sh— new provider: validates config, sets Anthropic-compatible env vars, resolves modelsscripts/lib/local-provider-manager.sh— onboarding flow, config loading, model discovery, endpoint probingscripts/ai—--localflag parsing,local-onboardsubcommandscripts/lib/provider-loader.sh— registerlocalproviderscripts/ai-status— show local provider availability/configscripts/lib/core-utils.sh— shared helpers for local providersetup.sh— deploy local provider files and libsecrets.example.sh— local provider config referencedocs/PROVIDERS.md— usage documentation for--localREADME.md— quick-start examplestest/automation/run_tests.sh— smoke coverage for local provider pathtools/claude-code.sh— minor model flag handlingexamples/README.md— index updateBug fix detail
_local_provider_select_modelechoed the "Discovered models" banner and numbered list to stdout. Sincedefault_model=$(_local_provider_select_model "1")captures all stdout, the entire display text was concatenated into the model name, written to the config file, and sent as the model ID to the API — producing a 400 error. Fix: redirect display output to stderr (>&2).Test plan
ai local-onboardcompletes and saves clean config to~/.ai-runner/local-provider.shai --locallaunches Claude Code pointed at the configured backendai --local --model <name>overrides the default modelai-statusshows local provider config and availability🤖 Generated with Claude Code