Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 33 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,18 @@ It currently supports the following local backends:
- llama-swap
- oMLX
- MLX-VLM
- Any other server with an OpenAI-compatible `/v1/models` endpoint (`openai`, used when nothing more specific matches)

Instead of creating one provider per server, this plugin keeps one `local` provider and lets you register multiple named targets. Each target is probed at runtime, and its currently loaded models are exposed automatically.
Instead of creating one provider per server, this plugin keeps one `local` provider and lets you register multiple named targets. Each target is probed at runtime, and its models are exposed automatically.

## Features

- Adds a `local` provider to OpenCode
- Supports multiple local URLs under one provider
- Supports multiple servers under one provider
- Includes supported default `127.0.0.1` targets automatically
- Detects loaded models at runtime
- Detects models at runtime and fills in context size, tool calling, and vision support where the server reports them
- Routes each model to the correct target URL
- API key auth currently unsupported
- Optional API key per target, sent as `Authorization: Bearer <key>`
- Uses OpenCode global config, not project-local config

## Example
Expand All @@ -47,43 +48,49 @@ Default targets are enabled automatically for these backends and ports:
- Exo: `http://127.0.0.1:52415`
- llama-swap: `http://127.0.0.1:8080`
- oMLX: `http://127.0.0.1:8000`
- MLX-VLM: `http://127.0.0.1:8000`
- MLX-VLM: `http://127.0.0.1:8080`

If your local providers do not need auth, you can start using the `local` provider immediately.
If your local servers run on these ports and do not need an API key, you can start using the `local` provider immediately.

**Note: oMLX requires API authentication to be disabled in the web app settings for the plugin to detect and probe models.**

**Note: API key authentication is currently unsupported.**
**Note: oMLX with API authentication enabled needs a custom target with its API key set (see below).**

## Custom Targets

If you need non-default hosts or ports, use the CLI auth flow to add an explicit target:
Add a server on another host or port, or one that needs an API key, with **Add Custom Target**. Run it from the TUI with `/connect` → **Local LLM Provider**, or from the CLI:

```bash
opencode auth login --provider local --method "Add Custom Target"
```

This will prompt for:
It asks for:

- a target ID, like `studio` or `remote-ollama`
- the local provider URL
- the API key (enter `none` since API keys are currently unsupported)
- the server URL, with or without `/v1`
- the server's API key. Leave it empty if the server has none. Enter `{env:VAR_NAME}` to store a reference to an environment variable instead of the key itself
- which models to list: only loaded models, or every model the server offers (useful for llama-swap, which loads models on demand)

The plugin checks that the server answers before it saves anything, then adds the target to your OpenCode global config. Run it again to add more servers.

The target is then stored in OpenCode global config.
`/connect` only lists **Local LLM Provider** once the provider has at least one model. If none of your servers are reachable yet, add the first target from the CLI.

You can also add explicit targets manually in config if needed:
You can also add targets manually in config:

```json
{
"provider": {
"local": {
"name": "Local Provider",
"options": {
"includeDefaults": true,
"targets": {
"studio": {
"url": "http://192.168.1.10:1234/v1",
"kind": "lmstudio"
},
"swap": {
"url": "https://llama-swap.example.com/v1",
"kind": "llamaswap",
"apiKey": "{env:LLAMA_SWAP_API_KEY}",
"includeUnloaded": true
}
}
}
Expand All @@ -92,53 +99,34 @@ You can also add explicit targets manually in config if needed:
}
```

Explicit targets override the built-in defaults when they use the same ID.
The CLI custom-target method is the supported way to add explicit targets without editing config directly.

## Resulting Config
Target fields:

The plugin stores explicit targets in OpenCode global config under the `local` provider:

```json
{
"provider": {
"local": {
"name": "Local Provider",
"options": {
"includeDefaults": true,
"targets": {
"studio": {
"url": "http://127.0.0.1:1234/v1",
"kind": "lmstudio"
}
}
}
}
}
}
```
- `url` (required): the server URL
- `kind` (optional): one of `ollama`, `lmstudio`, `llamacpp`, `vllm`, `exo`, `llamaswap`, `omlx`, `mlxvlm`, `openai`. If you leave it out, the backend is detected. `openai` treats any server as a plain OpenAI-compatible endpoint
- `apiKey` (optional): sent as `Authorization: Bearer <key>` when probing the server and with every chat request to its models. Use `{env:VAR_NAME}` to keep the key out of the config file
- `includeUnloaded` (optional, llama-swap only): list every model llama-swap offers, not only the loaded ones. The plugin reads their details from `/v1/models` and does not load them

With `includeDefaults: true`, the built-in default `127.0.0.1` targets are also checked at runtime even though they are not written into config.
Explicit targets override the built-in defaults when they use the same ID. A key set in `provider.local.options.apiKey` applies to every target that has no `apiKey` of its own.

**Note: API key authentication is currently unsupported.**
With `includeDefaults: true`, the built-in default `127.0.0.1` targets are also checked at runtime even though they are not written into config. Set it to `false` if you only use custom targets.

## How Models Appear

Models are discovered dynamically from each configured target. (Only **loaded** models)
Models are discovered dynamically from each configured target. By default, only **loaded** models are listed.

To avoid collisions, model IDs are prefixed with the target ID:

- `ollama/llama3.2`
- `studio/qwen2.5-coder`

Each generated model keeps its own target URL internally, so requests go to the correct backend.
Each generated model keeps its own target URL and API key internally, so requests go to the correct backend.

## Notes

- Model detection is runtime-based, not static
- If loaded models change in your local server, OpenCode will see the updated list on the next provider refresh
- Built-in default `127.0.0.1` targets are enabled unless you set `includeDefaults` to `false`
- **API key authentication is currently unsupported**
- If a custom target cannot be probed, the reason (for example `HTTP 401` for a wrong key) is written to the OpenCode log

## Development

Expand Down
38 changes: 21 additions & 17 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading