Self-hosted OpenAI-compatible gateway. One Node process, one SQLite database, one admin UI.
SmolRouter sits between OpenAI-compatible clients and one or more upstream providers. Clients authenticate with a gateway key scoped to specific model aliases. The gateway handles routing, key pools, failover, TPS-aware routing, caching, budgets, and telemetry.
There are no runtime npm dependencies, no database server, and no build step. Node 22.13 or newer is required for node:sqlite.
- OpenAI-compatible
POST /v1/chat/completions,GET /v1/models, and generic/v1/*JSON paths. - Model aliases and provider routes with priority tiers and round-robin, least-used, or cache-aware balancing.
- Provider key pools with round-robin rotation and automatic auth-failure benching.
- Failover with capped backoff, cooldowns, recovery probes, and proactive health checks.
- TPS-aware routing with a preferred route, effective-TPS floor, fallback, and recovery probes.
- Exact response cache, singleflight coalescing, and prompt-cache savings tracking.
- Upstream and downstream pricing, per-key budget rules, and currency conversion.
- Capability routing for
tools,vision,audio,json,reasoning, and max context. - Attempt telemetry, usage reports, latency and TPS reports, Prometheus metrics, and audit rows.
- SQLite backups, restore tooling, retention maintenance, and config export/import.
- Admin UI in plain HTML, CSS, and ES modules.
Requirements: Node.js 22.13 or newer.
npm run setup
npm start- Gateway:
http://127.0.0.1:8787/v1 - Admin UI:
http://127.0.0.1:8787/admin
First request:
curl http://127.0.0.1:8787/v1/chat/completions \
-H "Authorization: Bearer YOUR_GATEWAY_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "your-model-alias",
"messages": [{ "role": "user", "content": "Hello via SmolRouter." }],
"stream": true
}'See Installation for setup details, reverse proxies, upgrades, and restore.
| Guide | Contents |
|---|---|
| Documentation index | All guides. |
| About | Design, architecture, tech stack, compatibility. |
| Installation | Setup, first start, reverse proxy, upgrades, restore. |
| Usage | Client configuration, requests, headers, dashboard, errors. |
| TPS-aware routing | Preferred routes, demotion, fallback, probes, configuration. |
| Billing and budgets | Pricing layers, budget rules, currency, LiteLLM surfaces. |
| Operations | Health, metrics, jobs, backups, retention, configuration. |
| API reference | Response envelopes, endpoint map, Postman collection. |
| Security | Authentication, key handling, CSRF, privacy, audit. |
| Development | Tests, layout, migrations, contribution rules. |
| Changelog | Release history. |
Every environment variable is documented in .env.example. The Postman collection is at postman/smolrouter.postman_collection.json.
- Admin passwords use scrypt. Sessions are signed, HttpOnly cookies.
- Admin mutations require a session cookie and
X-CSRF-Token. - Gateway keys are SHA-256 hashed and scoped to models.
- Provider keys are AES-256-GCM encrypted at rest and never returned.
- Telemetry, usage, and TPS records store metadata only. Prompt and completion content is not stored.
- The optional response cache keeps response bodies in memory only. It is bounded and can be disabled per model.
See Security.
Any provider that implements the OpenAI API can be used. Tested or documented providers include OpenAI, OpenRouter, Google Gemini's OpenAI-compatible endpoints, Vertex AI adapters, Nvidia NIM, DeepSeek, Groq, and Cerebras.
Other POST /v1/* paths are routed when the JSON body contains a model field.
npm testTests use temporary SQLite databases and stub upstreams; no live provider is required. See Development.
SmolRouter is licensed under the GNU Affero General Public License v3 (AGPL-3.0). See LICENSE.
