Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,8 @@ jobs:
- uses: actions/checkout@v4
- name: Providers doc generator drift (docs/api/providers.md must match generator output)
run: python3 scripts/gen_providers_doc.py --check
- name: Provider boundaries (no call-level reads or provider-name branches in providers, removed names stay removed)
run: bash scripts/check_provider_boundaries.sh
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: ts-rs export drift (checked-in types must match generator output)
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,6 @@ quirks-corpus/
# Generated audit artifacts — keep conclusions in Markdown, not raw output.
/docs/quality-audit/**/lcov.info
/docs/quality-audit/**/*.log

# AI SDK protocol fixture recorder (scripts/aisdk-fixtures) — npm install output
scripts/aisdk-fixtures/node_modules/
283 changes: 274 additions & 9 deletions CHANGELOG.md

Large diffs are not rendered by default.

29 changes: 23 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ to set up a development environment, run the tests, and submit changes.
aimux/
├── aimux-core/ # Core abstractions: LanguageModel / Provider / Message / StreamPart
├── aimux-providers/ # provider implementations + cassettes (counts: docs/api/providers.md)
├── aimux-stream/ # SSE parsing
├── aimux-provider-utils/ # HTTP utilities: retry, backoff, error parsing, API-key loading, streamed tool-call tracking
├── aimux-stream/ # SSE decoding
├── aimux-provider-utils/ # HTTP utilities: Fetch transport, Resolvable settings, error parsing, API-key loading, streamed tool-call tracking
├── aimux-ffi/ # C ABI (opaque handle + JSON + push callback) for non-native bindings
├── bindings/ # Node, Python, Swift, Kotlin, Flutter, Go, C — share one Rust core
├── rfc/ # Design docs (RFCs)
Expand Down Expand Up @@ -38,6 +38,20 @@ cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
```

Provider changes also run the generator and boundary checks CI runs in the
`contract-tests` job:

```bash
python3 scripts/gen_providers_doc.py --check # docs/api/providers.md
python3 scripts/gen_ts_types.py --check # ts-rs types for the Node binding
bash scripts/check_provider_boundaries.sh # factory-shape rules for provider packages
```

The Node and Python bindings are separate workspaces: run `cargo fmt` and
`cargo clippy -- -D warnings` inside each, then `npm ci && npm run build &&
npm test` (Node) or `maturin build --release`, `pip install` the wheel and
`python -m pytest tests/` (Python).

## Testing

Tests run entirely on **cassette playback** — no network access and no API keys
Expand All @@ -62,10 +76,13 @@ aimux distinguishes three kinds of providers:
1. **Native protocol** — providers with their own request/response model
(OpenAI, Anthropic, Google, Bedrock, …). Implement the full `convert`/model
path and handle provider-specific differences.
2. **OpenAI-compatible thin wrapper** — the majority. Describe differences via
`OpenAICompatProfile` (top_k, tools, response_format, streaming usage,
request-body post-processing) so the thin wrapper does not erase
provider-specific behavior.
2. **OpenAI-compatible preset** — the majority. One row of
`aimux-providers/src/provider_registry.json` (base URL, key variable or
`auth: none`, optional URL variable and template parameters) that
is embedded and parsed once into a runtime table. Create presets by name
through `provider(name, ...)`. Vendors with behavior of their own get a package on
the shared compatible chat model (`groq/`, `deepseek/`) instead of a
branch on their name.
3. **Modality-specific** — speech, image, video, transcription, etc.

Step-by-step checklists (with the generator and CI rules) live in
Expand Down
83 changes: 41 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<img src="assets/aimux-banner.png" alt="aimux banner" width="100%">
</p>

> **A unified LLM access layer written in Rust. One API for [327 AI providers](docs/api/providers.md).**
> **A unified LLM access layer written in Rust. One API for [327 provider names](docs/api/providers.md).**

[![CI](https://github.com/arcships/aimux/actions/workflows/ci.yml/badge.svg)](https://github.com/arcships/aimux/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
Expand All @@ -30,13 +30,10 @@ difference: aimux is an access layer, those are orchestration layers.

## Why aimux

- **327 providers** (as of 2026-09-24) — 251 registry-backed OpenAI-compatible
(unified `provider(name, ...)` entry) + 76 typed providers (native protocol
implementations such as OpenAI/Anthropic/Google/Bedrock/Vertex, local
engines like Ollama/vLLM, and speech/image/video/search modality providers).
Counts by category and the full list live in
[docs/api/providers.md](docs/api/providers.md) — generated, and the single
source of truth for provider counts (only the badge above repeats the total).
- **327 provider names** — 282 registry-backed OpenAI-compatible presets
+ 45 vendor packages, covering text and other modalities. Local engines
such as Ollama/vLLM are presets. `provider_names()` lists the Rust names;
the generated catalogue lives in [docs/api/providers.md](docs/api/providers.md).
- **Unified, object-safe interface** — the `LanguageModel` trait supports
`Box<dyn>` so providers are interchangeable without changing call sites.
- **Full multimodal** — text, streaming, tool calling, embeddings, image,
Expand All @@ -49,10 +46,12 @@ difference: aimux is an access layer, those are orchestration layers.
- **Composite models** (new in 0.3.0) — `RouterModel` routes each call with
fallback (RFC-0021); `MoaModel` aggregates parallel reference models
mixture-of-agents style (RFC-0022). Both are plain `LanguageModel`s.
- **Config-driven provider registry** — `provider-registry.json` describes
each registry-backed OpenAI-compatible provider (base URL, env var, profile
quirks: top_k, tools, response_format, streaming usage, max_tokens key);
one unified `provider(name, ...)` entry in every binding.
- **Config-driven provider registry** — `provider_registry.json` describes
each registry-backed OpenAI-compatible provider (base URL, env var, auth
mode, template parameters); the embedded JSON is parsed once into a runtime
descriptor table. Rust uses `create_provider(name, PresetSettings)` for
presets and vendor packages; bindings retain `provider(name, ...)`.
Packages follow the AI SDK shape: `XxxProviderSettings` + `create_xxx()` + a default `xxx()`.
- **Fast and small** — Rust core, release profile tuned for binary size
(`lto`, `codegen-units=1`, `panic="abort"`, `strip`, `opt-level="z"`).
- **8 language bindings** from one core: Node, Python, Swift, Kotlin, Flutter,
Expand Down Expand Up @@ -148,14 +147,12 @@ console.log(result.text)

```rust
use aimux_core::prelude::*;
use aimux_providers::{OpenAIConfig, OpenAIProvider};
use aimux_providers::openai::openai;

#[tokio::main]
async fn main() -> Result<(), AiMuxError> {
let provider = OpenAIProvider::new(
OpenAIConfig::new(std::env::var("OPENAI_API_KEY")?)
);
let model = provider.model("gpt-4o");
// The default provider loads OPENAI_API_KEY when a request is made.
let model = openai().chat("gpt-4o");

let result = generate_text(
&model,
Expand All @@ -182,8 +179,8 @@ let result = stream_text(
let mut stream = result.stream;
while let Some(part) = stream.next().await {
match part? {
StreamPart::TextDelta { delta, .. } => print!("{}", delta),
StreamPart::Finish { .. } => println!("\n[done]"),
TextStreamPart::TextDelta { delta, .. } => print!("{}", delta),
TextStreamPart::Finish { .. } => println!("\n[done]"),
_ => {}
}
}
Expand Down Expand Up @@ -220,9 +217,10 @@ use aimux_core::moa::{MoaConfig, MoaModel};
use aimux_core::router::{FallbackPolicy, RouterConfig, RouterModel, RuleRouter};

// Router: pick one model per call, fall back on failure (RFC-0021).
let provider = aimux_providers::openai::openai();
let children: Vec<ChildModel> = vec![
Arc::new(provider.model("gpt-4o")) as ChildModel,
Arc::new(provider.model("gpt-4o-mini")) as ChildModel,
Arc::new(provider.chat("primary-model-id")) as ChildModel,
Arc::new(provider.chat("secondary-model-id")) as ChildModel,
];
let routed = RouterModel::new(
children.clone(),
Expand All @@ -234,7 +232,7 @@ let routed = RouterModel::new(
// MoA: fan out to references, aggregate their answers (RFC-0022).
let moa = MoaModel::new(
children,
Arc::new(provider.model("gpt-4o")) as ChildModel,
Arc::new(provider.chat("primary-model-id")) as ChildModel,
MoaConfig::default(),
);

Expand Down Expand Up @@ -273,35 +271,36 @@ session.close()
## Switch providers

```rust
// OpenAI → DeepSeek: only the provider name changes (registry-backed;
// key read from the provider's env var)
use aimux_providers::{provider, provider_from_env};
use aimux_core::provider_registry::create_provider_registry;
use aimux_providers::{create_provider, default_providers, provider_names, PresetSettings};

let model = provider("deepseek", None, "deepseek-chat", None)?;
let model = provider_from_env("deepseek", "deepseek-chat", None)?;
// Keys are loaded when a request is made.
let provider = create_provider("deepseek", PresetSettings::default())?;
let model = provider.language_model("deepseek-chat")?;

// Or assemble the built-in providers and resolve a provider:model id.
let registry = create_provider_registry(default_providers(), Default::default());
let model = registry.language_model("deepseek:deepseek-chat")?;
let names = provider_names();
// model usage is identical — it's all dyn LanguageModel
```

All registry-backed OpenAI-compatible providers share one entry:
`provider(name, ...)` in every binding. Provider names are runtime strings;
the generated provider list is the discovery surface.
The retired per-provider shell types (`XxxConfig`/`XxxProvider`) are gone —
see [docs/API.md](docs/API.md#providers).

> **Scope:** OpenAI-compatible → `provider(name, ...)`; others (Anthropic,
> multimodal, local…) → their constructors. List:
> [docs/api/providers.md](docs/api/providers.md).
Rust by-name creation covers vendor packages and all 282 presets. Each vendor
also has `create_xxx(XxxProviderSettings {..})` and a default instance `xxx()`.
Bindings keep their own `provider(name, ...)` API. Runtime model listing uses
`Provider::discovery()` and then `ProviderDiscovery::list_models()` — see
[docs/api/rust.md](docs/api/rust.md#providers).

## Provider coverage

Counts by category live in
[docs/api/providers.md](docs/api/providers.md#totals) — generated from the
registry and `lib.rs`, and the single source of truth. The rough shape of
The generated catalogue is [docs/api/providers.md](docs/api/providers.md).
The accepted Rust names come from `provider_names()`. The rough shape of
the roster:

- **Native protocol** — OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, Anthropic-AWS, Voyage, Codex, OpenRouter
- **OpenAI-compatible (registry-backed)** — Groq, Fireworks, Together, Perplexity, Ollama Cloud, DeepSeek, Alibaba Tongyi, Zhipu, Baidu, Tencent, Moonshot, SiliconFlow…
- **OpenAI-compatible (standalone + Vertex-hosted)** — Hugging Face, Ollama, vLLM, SGLang, Llama.cpp, LiteLLM Proxy, Vertex-hosted DeepSeek/Qwen/Llama…
- **Native protocol** — OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, Groq, DeepSeek, xAI, Anthropic-AWS, Voyage
- **OpenAI-compatible (registry-backed)** — Fireworks, Together, Perplexity, OpenRouter, Ollama Cloud, Alibaba Tongyi, Zhipu, Baidu, Tencent, Moonshot, SiliconFlow…
- **Local and Vertex-hosted presets** — Ollama, vLLM, SGLang, Llama.cpp, LiteLLM Proxy, Vertex-hosted DeepSeek/Qwen/Llama…
- **Responses API for text** — xAI and Hugging Face (no chat-completions entry)
- **Speech / transcription** — ElevenLabs, Deepgram, AssemblyAI, AWS Polly, Cartesia, Hume, Gladia, RevAI, LMNT, Fal
- **Image / video** — Black Forest Labs, Replicate, Luma, Prodia, KlingAI, Recraft, Stability, RunwayML
- **Embeddings / rerank / search** — Voyage, Jina, Tavily, Exa, Firecrawl, Serper, SearXNG, You.com…
Expand Down
14 changes: 7 additions & 7 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ aimux 是 **provider 接入与治理的运行时**:
│ ├─→ #174 Auth L1 ──→ #175 Auth L2 ← 依赖 B1
│ └─→ #167 transport-level replay ──→ #179 replay 子命令 ← 依赖 B1
├─ C 轨 FFI(现 123 个 extern "C",#166 基线 109)→ ops 协议(C1 升级为传输无关协议:op 表 / 二进制帧 / 版本协商)
│ └─ C2 构造器转发 shim 是 B4/B5 前置
│ └─ C2 构造器转发 shim 取消:B4/B5 的直接构造调用点随 provider 工厂切换一次迁移(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系))
├─ D 轨 8 种绑定重写为 ops 薄封装(依赖 C1)+ D8 类型镜像生成
├─ E 轨 内部清理(#164 已合入,改为 master 上独立 PR)+ 文档三层重组
│
Expand Down Expand Up @@ -66,30 +66,30 @@ aimux 是 **provider 接入与治理的运行时**:
| 项 | 内容 |
|---|---|
| B2–B8 | 17 个 LanguageModel 实现 → ~7 个协议;33 个 wrapper 退役;responses 家族合并(遵循 #166 修正 3/6) |
| #174 / #175 | registry `auth` schema + `apply_auth()`;Credential 解析序 + CredentialStore + TokenRefresher |
| #174 / #175 | registry `auth` 列作为 preset descriptor 数据,鉴权在各厂商包内以 headers 闭包 / fetch 装饰器实现;CredentialStore + TokenRefresher 随宿主回调后置(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系)) |
| L2 补齐 | `StreamPart::Raw` / `provider_metadata` 在各协议接线——投影不下的不得静默丢弃 |
| **S3** | `cargo-bloat` 审计 0.5.0 增重(Android `.so` 13.4 → 21 MB);feature gating 仅作可选小体积路径,默认全量 |
| ~~调研 RFC~~ | ~~L2 数据模型选型(AI SDK 形态 / Open Responses items / 自有),指标见 RFC-0036 §3~~ 撤销:选型已定为 AI SDK V4 形态(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系)) |

版本语义:Rust API 大 breaking(33 wrapper 退役);C ABI 经 C2 转发不受影响。
版本语义:Rust API、binding wire 与 C ABI 在全链路对齐切换时一次性 breaking(33 wrapper 退役、构造器替换),不经 C2 转发(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系))。

### 0.8.0 ——「ops 协议落地」(~4-5 周)

| 项 | 内容 |
|---|---|
| C1 | 9 个导出 + `dispatch`,**同一 dispatch 同时服务 FFI 与 stdio CLI**;旧导出保留共存 |
| C1 | 9 个导出 + `dispatch`,**同一 dispatch 同时服务 FFI 与 stdio CLI**;旧符号按切换门一次替换,不共存(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系)) |
| C3 | `aimux_error_*` 访问器 → 错误 JSON 信封(含派生 `retry_after_ms`) |
| D1–D7 | 8 种绑定迁到 ops 薄封装。D1 Kotlin(只依赖 Java artifact)与 D5 Flutter ffigen 过渡版不依赖 C1,可在 0.6/0.7 提前做 |
| D8 | 类型镜像生成:serde → JSON Schema → 各语言,`--check` 门禁扩到全部输出 |
| P | stdio 入口往返开销纳入性能门禁 |

版本语义:新 ABI 加入,旧导出并存(#166 要求共存至少一个 minor 版本);绑定按各自节奏迁移。“共存至少一个 minor”不再作为承诺:全链路对齐切换本身没有共存期,之后 ops ABI 引入时的旧符号也按切换门一次替换;C2 转发 shim 与 C4 旧导出清理随之取消(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系))。
版本语义:新 ABI 按切换门一次替换旧符号,不提供共存期(#166 原要求的“共存至少一个 minor”撤销);全链路对齐切换本身同样没有共存期,C2 转发 shim 与 C4 旧导出清理随之取消(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系))。

### 0.9.0 ——「治理」(~3-4 周)

| 项 | 内容 |
|---|---|
| #167 | transport-level replay:mock 挂 HTTP 层跑真协议代码,覆盖全协议 / 全模态;`ProviderRecord` = registry 行 + protocol |
| #167 | transport-level replay:mock 挂 HTTP 层跑真协议代码,覆盖全协议 / 全模态;录制只存身份,不保存可重建的 `ProviderRecord`(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系)) |
| 漂移检测 | #170 扩展:定期重录,与 cassette 字节级 diff |
| 能力矩阵 | provider × 能力,每格由探测或录制支撑 |
| CLI | `aimux probe / replay / diff`(#179 replay 子命令并入);#181 debug CLI 并入 |
Expand All @@ -98,7 +98,7 @@ aimux 是 **provider 接入与治理的运行时**:

| 项 | 内容 |
|---|---|
| C4 | 8 种绑定全部迁移后删除旧导出、旧头文件、旧 FFI 测试 |
| ~~C4~~ | ~~8 种绑定全部迁移后删除旧导出、旧头文件、旧 FFI 测试~~ 取消:旧导出在切换时随绑定一起删除,1.0 不再补删(调整:见 [docs/aisdk-architecture-alignment.md §0.7](docs/aisdk-architecture-alignment.md#07-与-roadmap--rfc-0036-既有承诺的关系)) |
| 冻结范围 | ops 协议与 L0 / L1 契约。**L2 独立版本化,不随 1.0 冻结** |
| 发布判据 | #166 ledger 全勾(B9 / #171 后半除外);错误模型 #95 与请求管线 #164 稳定一个周期;各门禁(S2 / P)连续绿 |

Expand Down
1 change: 1 addition & 0 deletions aimux-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ httpdate = { workspace = true }
rand = { workspace = true }
base64 = "0.22"
jsonschema = { workspace = true }
regex = "1"

[dev-dependencies]
tokio = { workspace = true, features = ["macros", "rt", "test-util"] }
Expand Down
Loading
Loading