Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,8 @@ jobs:
- uses: actions/checkout@v4
- name: ProviderName generator drift (checked-in files must match generator output)
run: python3 scripts/gen_provider_names.py --check
- name: Providers doc generator drift (docs/api/providers.md must match generator output)
run: python3 scripts/gen_providers_doc.py --check
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- uses: actions/setup-node@v4
Expand Down
5 changes: 4 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ to set up a development environment, run the tests, and submit changes.
```
aimux/
├── aimux-core/ # Core abstractions: LanguageModel / Provider / Message / StreamPart
├── aimux-providers/ # 325 provider implementations + cassettes
├── aimux-providers/ # provider implementations + cassettes (counts: docs/api/providers.md)
├── aimux-stream/ # SSE / NDJSON stream parsing
├── aimux-provider-utils/ # HTTP utilities: retry, backoff, error parsing, API-key loading
├── aimux-ffi/ # C ABI (opaque handle + JSON + push callback) for non-native bindings
Expand Down Expand Up @@ -72,6 +72,9 @@ aimux distinguishes three kinds of providers:
provider-specific behavior.
3. **Modality-specific** — speech, image, video, transcription, etc.

Step-by-step checklists (with the generator and CI rules) live in
[docs/contributing/adding-a-provider.md](docs/contributing/adding-a-provider.md).

Before submitting a provider, read `rfc/0006-provider-development.md` for the
minimum acceptance criteria, core contracts, and required tests.

Expand Down
53 changes: 29 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
<img src="assets/aimux-banner.png" alt="aimux banner" width="100%">
</p>

> **A unified LLM access layer written in Rust. One API for 329 AI providers.**
> **A unified LLM access layer written in Rust. One API for [327 AI providers](docs/api/providers.md).**

[![CI](https://github.com/arcships/aimux/actions/workflows/ci.yml/badge.svg)](https://github.com/arcships/aimux/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-stable-orange.svg)](https://www.rust-lang.org/)
[![Providers](https://img.shields.io/badge/providers-329-green.svg)](docs/api/providers.md)
[![Providers](https://img.shields.io/badge/providers-327-green.svg)](docs/api/providers.md)
[![Bindings](https://img.shields.io/badge/bindings-8-9cf.svg)](bindings/)
[![crates.io](https://img.shields.io/crates/v/aimux-core)](https://crates.io/crates/aimux-core)
[![npm](https://img.shields.io/npm/v/@arcships/aimux)](https://www.npmjs.com/package/@arcships/aimux)
Expand All @@ -30,12 +30,13 @@ difference: aimux is an access layer, those are orchestration layers.

## Why aimux

- **329 provider modules** — 251 registry-backed OpenAI-compatible
(unified `provider(name, ...)` entry) + 10 native protocol
implementations (OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere,
Mistral, xAI, Anthropic-AWS) + 68 standalone/modality/local/search providers
(OpenRouter, DeepSeek, Ollama, vLLM, ElevenLabs, KlingAI, Tavily, …).
Full list: [docs/api/providers.md](docs/api/providers.md).
- **327 providers** (as of 2026-09-24) — 251 registry-backed OpenAI-compatible
(unified `provider(name, ...)` entry) + 76 typed providers (native protocol
implementations such as OpenAI/Anthropic/Google/Bedrock/Vertex, local
engines like Ollama/vLLM, and speech/image/video/search modality providers).
Counts by category and the full list live in
[docs/api/providers.md](docs/api/providers.md) — generated, and the single
source of truth for provider counts (only the badge above repeats the total).
- **Unified, object-safe interface** — the `LanguageModel` trait supports
`Box<dyn>` so providers are interchangeable without changing call sites.
- **Full multimodal** — text, streaming, tool calling, embeddings, image,
Expand All @@ -49,7 +50,7 @@ difference: aimux is an access layer, those are orchestration layers.
fallback (RFC-0021); `MoaModel` aggregates parallel reference models
mixture-of-agents style (RFC-0022). Both are plain `LanguageModel`s.
- **Config-driven provider registry** — `provider-registry.json` describes
each of the 251 OpenAI-compatible providers (base URL, env var, profile
each registry-backed OpenAI-compatible provider (base URL, env var, profile
quirks: top_k, tools, response_format, streaming usage, max_tokens key);
one unified `provider(name, ...)` entry in every binding.
- **Fast and small** — Rust core, release profile tuned for binary size
Expand Down Expand Up @@ -96,7 +97,7 @@ middleware, and telemetry per request).
```
aimux/
├── aimux-core # Core abstractions: LanguageModel / Provider / Message / StreamPart
├── aimux-providers # 329 provider implementations (251 registry-backed + native)
├── aimux-providers # Provider implementations — registry-backed + typed (docs/api/providers.md)
├── aimux-stream # SSE / NDJSON stream parsing
├── aimux-provider-utils # One-exchange HTTP helpers, response handlers, API-key loading
├── aimux-ffi # C ABI (opaque handles + JSON results + owned aimux_error_t *) for non-native bindings
Expand All @@ -120,7 +121,7 @@ cargo add aimux-core aimux-providers
| Crate | Description | crates.io |
|-------|-------------|-----------|
| `aimux-core` | Core abstractions: `LanguageModel` / `Provider` / `Message` / `StreamPart` | [crates.io](https://crates.io/crates/aimux-core) |
| `aimux-providers` | 325 provider implementations | [crates.io](https://crates.io/crates/aimux-providers) |
| `aimux-providers` | Provider implementations — [registry-backed + typed](docs/api/providers.md) | [crates.io](https://crates.io/crates/aimux-providers) |
| `aimux-stream` | SSE / NDJSON stream parsing | [crates.io](https://crates.io/crates/aimux-stream) |
| `aimux-provider-utils` | One-exchange HTTP helpers and typed response handlers | [crates.io](https://crates.io/crates/aimux-provider-utils) |
| `aimux-ffi` | C ABI for non-native bindings | [crates.io](https://crates.io/crates/aimux-ffi) |
Expand Down Expand Up @@ -283,8 +284,9 @@ let model = provider_from_env("deepseek", "deepseek-chat", None)?;
// model usage is identical — it's all dyn LanguageModel
```

All 251 OpenAI-compatible providers are registry-backed: `provider(name, ...)`
in every binding, with typed `ProviderName` (enum/union/consts per language).
All registry-backed OpenAI-compatible providers share one entry:
`provider(name, ...)` in every binding, with typed `ProviderName`
(enum/union/consts per language).
The retired per-provider shell types (`XxxConfig`/`XxxProvider`) are gone —
see [docs/API.md](docs/API.md#providers).

Expand All @@ -294,17 +296,19 @@ see [docs/API.md](docs/API.md#providers).

## Provider coverage

| Type | Count | Examples |
|------|:-----:|----------|
| Native protocol | 10 | OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, Anthropic-AWS |
| OpenAI-compatible (registry) | 251 | Groq, Fireworks, Together, Perplexity, Ollama Cloud, DeepSeek, Alibaba Tongyi, Zhipu, Baidu, Tencent, Moonshot, SiliconFlow… |
| OpenAI-compatible (standalone + Vertex-hosted) | 35 | OpenRouter, Hugging Face, Ollama, vLLM, SGLang, Llama.cpp, LiteLLM Proxy, Vertex-hosted DeepSeek/Qwen/Llama… |
| Speech / transcription | 10 | ElevenLabs, Deepgram, AssemblyAI, AWS Polly, Cartesia, Hume, Gladia, RevAI, LMNT, Fal |
| Image / video | 8 | Black Forest Labs, Replicate, Luma, Prodia, KlingAI, Recraft, Stability, RunwayML |
| Embeddings / rerank / search | 13 | Voyage, Jina, Tavily, Exa, Firecrawl, Serper, SearXNG, You.com… |
| Other (Responses API, Bedrock/Mantle) | 2 | generic Responses API wrapper, Bedrock Mantle |
Counts by category live in
[docs/api/providers.md](docs/api/providers.md#totals) — generated from the
registry and `lib.rs`, and the single source of truth. The rough shape of
the roster:

Full list: [rfc/0004-provider-inventory.md](rfc/0004-provider-inventory.md).
- **Native protocol** — OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, Anthropic-AWS, Voyage, Codex, OpenRouter
- **OpenAI-compatible (registry-backed)** — Groq, Fireworks, Together, Perplexity, Ollama Cloud, DeepSeek, Alibaba Tongyi, Zhipu, Baidu, Tencent, Moonshot, SiliconFlow…
- **OpenAI-compatible (standalone + Vertex-hosted)** — Hugging Face, Ollama, vLLM, SGLang, Llama.cpp, LiteLLM Proxy, Vertex-hosted DeepSeek/Qwen/Llama…
- **Speech / transcription** — ElevenLabs, Deepgram, AssemblyAI, AWS Polly, Cartesia, Hume, Gladia, RevAI, LMNT, Fal
- **Image / video** — Black Forest Labs, Replicate, Luma, Prodia, KlingAI, Recraft, Stability, RunwayML
- **Embeddings / rerank / search** — Voyage, Jina, Tavily, Exa, Firecrawl, Serper, SearXNG, You.com…

Full list with per-category counts: [docs/api/providers.md](docs/api/providers.md).

## Language bindings

Expand Down Expand Up @@ -338,7 +342,8 @@ Tests run on cassette playback — no network and no keys. See
|-----|----------|
| [docs/API.md](docs/API.md) | **API overview** — shared reference + links to per-language guides |
| [docs/api/reference.md](docs/api/reference.md) | **API reference** — public types & functions lookup |
| [docs/api/providers.md](docs/api/providers.md) | **Provider list** — all 325 providers with entry points (generated) |
| [docs/contributing/adding-a-provider.md](docs/contributing/adding-a-provider.md) | **Adding a provider** — the three-case checklist (registry row / new protocol / single modality) |
| [docs/api/providers.md](docs/api/providers.md) | **Provider list** — every provider with entry points and the canonical counts (generated) |
| [docs/api/](docs/api/) | **Per-language API guides** — Node.js, Python, Rust, Go, C/C++, Swift, Kotlin, Flutter |
| [docs/error-model.md](docs/error-model.md) | **错误模型** — 跨语言错误形态与兼容性约定 |
| [docs/PROJECT-OVERVIEW.md](docs/PROJECT-OVERVIEW.md) | Project overview, design decisions, benchmarks |
Expand Down
7 changes: 4 additions & 3 deletions docs/API.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# aimux API Documentation

> Unified LLM service access layer — one API to access 325 AI providers
> Unified LLM service access layer — one API to access every provider
> ([providers.md](api/providers.md) carries the count)

## Table of Contents

Expand Down Expand Up @@ -63,7 +64,7 @@ model)` (Go), `Model.openai(apiKey, modelId)` (Java, Kotlin, Flutter),
Multimodal, local-inference and search providers have their own constructors
too — full list: [reference.md](api/reference.md).

### OpenAI-compatible (251)
### OpenAI-compatible (registry-backed)

One function in every binding:

Expand All @@ -85,7 +86,7 @@ provider(name, api_key?, model_id, config?) // all languages
```

字符串形式(`provider("groq", ...)`)在全部语言中同样可用——两种写法等价。
- Full list (251, name / env var / base URL): [providers.md](api/providers.md)
- Full list (name / env var / base URL): [providers.md](api/providers.md)
- Custom endpoint: registry name + `base_url` override, or the OpenAI
constructor with a base URL

Expand Down
41 changes: 22 additions & 19 deletions docs/PROJECT-OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ aimux does not do agent loops, RAG, or orchestration — it focuses solely on un
| Metric | Value |
|------|------|
| Rust code | 144,500+ lines |
| AI providers | 329 (251 registry-backed OpenAI-compatible + 10 native protocols + 38 standalone/local/speech/image/video) |
| AI providers | 327 (251 registry-backed OpenAI-compatible + 76 typed; counts by category in [api/providers.md](api/providers.md), generated) |
| Modality traits | 8 (text/embedding/image/video/speech/transcription/reranking/search) |
| Test cassettes | 2,650 recorded replays |
| Test files | 118 |
Expand Down Expand Up @@ -102,7 +102,7 @@ The comparison between aimux and the OpenAI official SDK is truly equivalent —

| Trait | Capability | Example Providers |
|-------|------|-------------|
| `LanguageModel` | Text generation + streaming + tool calling | OpenAI / Anthropic / Google / DeepSeek / 325 providers |
| `LanguageModel` | Text generation + streaming + tool calling | OpenAI / Anthropic / Google / DeepSeek / all providers ([providers.md](api/providers.md)) |
| `EmbeddingModel` | Vector embeddings | OpenAI / Cohere / Voyage / generic-compatible |
| `ImageModel` | Image generation | Black Forest Labs / Replicate / Fal / KlingAI |
| `VideoModel` | Video generation | Google Veo / Replicate |
Expand All @@ -113,15 +113,18 @@ The comparison between aimux and the OpenAI official SDK is truly equivalent —

### Categorization of providers

| Type | Count | Representatives |
|------|:---:|------|
| Native protocol | 11 | OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, DeepSeek |
| OpenAI-compatible (registry) | 251 | Groq, Fireworks, Together, Perplexity, Ollama, OpenRouter, Alibaba Tongyi, Zhipu, Baidu, Tencent, iFlytek, Moonshot AI, SiliconFlow… |
| Voice/transcription | 7 | ElevenLabs, Deepgram, AssemblyAI, Cartesia… |
| Image/video | 8 | Black Forest Labs, Replicate, Fal, KlingAI… |
| Search | 11 | Tavily, Exa, Serper, Firecrawl… |
Counts by category live in [api/providers.md](api/providers.md#totals)
(generated — the single source of truth). Representatives:

| Type | Representatives |
|------|------|
| Native protocol | OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, Codex, OpenRouter |
| OpenAI-compatible (registry) | Groq, Fireworks, Together, Perplexity, Ollama, Alibaba Tongyi, Zhipu, Baidu, Tencent, Moonshot AI, SiliconFlow… |
| Voice/transcription | ElevenLabs, Deepgram, AssemblyAI, Cartesia… |
| Image/video | Black Forest Labs, Replicate, Fal, KlingAI… |
| Search | Tavily, Exa, Serper, Firecrawl… |

See [rfc/0004-provider-inventory.md](../rfc/0004-provider-inventory.md) for the full list.
See [api/providers.md](api/providers.md) for the full list with counts.
All registry providers are accessed via the unified `provider(name, ...)` entry point
— see [API.md](API.md#providers).

Expand Down Expand Up @@ -158,10 +161,10 @@ aimux/
│ ├── ImageModel / VideoModel
│ ├── SpeechModel / TranscriptionModel
│ └── RerankingModel / SearchModel
├── aimux-providers # 325 provider implementations
│ ├── 11 native protocols # standalone model + convert, handles provider-specific differences
│ ├── 251 OpenAI compatible # registry-backed: provider-registry.json + provider(name, ...) entry (RFC-0017 phase 4)
│ └── modalities/search # voice / image / video / search implementations
├── aimux-providers # provider implementations (canonical counts: docs/api/providers.md)
│ ├── native protocols # standalone model + convert, handles provider-specific differences
│ ├── OpenAI compatible # registry-backed: provider-registry.json + provider(name, ...) entry (RFC-0017 phase 4)
│ └── modalities/search # voice / image / video / search implementations
├── aimux-stream # SSE / NDJSON streaming parsing
├── aimux-provider-utils # One-exchange HTTP helpers, response handlers, API-key loading
├── aimux-ffi # C ABI (FFI infrastructure, shared by all bindings)
Expand Down Expand Up @@ -270,10 +273,10 @@ Fundamental difference: **aimux is an access layer, LangChain/Mastra is an orche
Your app
└── Orchestration layer (LangChain / Mastra / custom loop)
└── Access layer (aimux) ← here
└── 325 AI providers
└── AI providers (count: docs/api/providers.md)
```

aimux does not compete with LangChain; instead it serves as the layer beneath LangChain — LangChain handles the agent loop / RAG / chain, while aimux handles unified access to 325 providers. Running the access layer in Rust delivers performance and memory behavior far exceeding a JS-implemented access layer.
aimux does not compete with LangChain; instead it serves as the layer beneath LangChain — LangChain handles the agent loop / RAG / chain, while aimux handles unified access to every provider ([providers.md](api/providers.md)). Running the access layer in Rust delivers performance and memory behavior far exceeding a JS-implemented access layer.

### aimux vs rig / rust-genai

Expand Down Expand Up @@ -405,7 +408,7 @@ Not a single trick, but systematic design choices:
| Does not do | Zod validation / middleware / telemetry | Does | Extra CPU per request |
| Compilation | AOT compiled to native code | JIT | Cold start + steady state |

### 3. The way to unify 325 providers
### 3. The way to unify all providers

Don't write an independent model for each provider — that would explode. Use `OpenAICompatProfile` to describe the differences:

Expand All @@ -420,7 +423,7 @@ let profile = OpenAICompatProfile {
};
```

The 11 native protocols have independent models + convert (handling differences such as Anthropic message format / Google generateContent / Bedrock SigV4), while the 251 OpenAI-compatible providers are registry-backed (provider-registry.json + unified provider(name, ...) entry, RFC-0017 phase 4).
The native protocol providers have independent models + convert (handling differences such as Anthropic message format / Google generateContent / Bedrock SigV4), while the OpenAI-compatible providers are registry-backed (provider-registry.json + unified provider(name, ...) entry, RFC-0017 phase 4).

### 4. Recorded testing with 2650 cassettes

Expand All @@ -436,7 +439,7 @@ aimux's type design directly targets Vercel AI SDK V4 provider types — the `Ge

### Completed

- [x] 329 providers integrated (10 native + 251 registry-backed OpenAI-compatible + voice/image/video/search)
- [x] 327 providers integrated (registry-backed OpenAI-compatible + native/modality typed; counts by category in [providers.md](api/providers.md), generated)
- [x] 8 modality traits (text/embedding/image/video/speech/transcription/reranking/search)
- [x] 7 language bindings (Node/Python/Swift/Kotlin/Flutter/C/Rust)
- [x] 2650 cassette recorded tests
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Public documentation for aimux — a unified LLM access layer written in Rust.
|-----|----------|
| [API.md](API.md) | **API overview** — features, shared reference tables, factory functions, coverage matrix |
| [api/reference.md](api/reference.md) | **API reference** — all public types & functions, with sources |
| [api/providers.md](api/providers.md) | **Provider list** — all 325 providers with entry points (generated) |
| [api/providers.md](api/providers.md) | **Provider list** — every provider with entry points and the canonical counts (generated) |
| [api/gaps.md](api/gaps.md) | **Binding API gaps** — per-binding missing API tracking (Swift/Kotlin/Flutter multimodal, Go base_url, search factories, C ABI `_with_base`), with C ABI function mapping and reference implementations |
| [api/](api/) | **Per-language guides** — Node.js, Python, Rust, Go, C/C++, Swift, Kotlin, Flutter, Java |
| [error-model.md](error-model.md) | **错误模型** — 错误来源、跨语言映射、所有权与兼容性约定 |
Expand Down
Loading
Loading