Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Added

- Provider-scoped client endpoints under `/provider/<name>` with filtered OpenAI, Anthropic and
Gemini model catalogs, provider-pinned text execution and isolated session affinity. Named
OpenAI-compatible upstreams such as `mistral` are supported. Antigravity CLI setup now uses
its enterprise model gateway environment variables.

- Mistral monthly API and Vibe Code budget history for API-key upstreams on `api.mistral.ai`,
with cached subscription plans and optional `budget` and `currency` window fields. These
usage-only windows use account weight 1.
Expand Down Expand Up @@ -62,6 +67,9 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Fixed

- Two Mistral keys of one customer (a console key and a Vibe key) are recorded as one account,
so the monthly API and Vibe Code budgets are not counted twice.

- `soonest-reset-first` no longer sends new work to an account it knows is spent while the
account with allowance left is only on a short hold. Seen on a live hub: one "overloaded"
answer put the usable Codex account on a five-minute hold and the next conversations were
Expand Down
30 changes: 30 additions & 0 deletions docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,36 @@ Two hard rules apply throughout this page.
Follow [setup](setup.md) for installation and keys. [Environments](environments.md) covers machines
that run agents, and [box setup](box-setup.md) gives an ordered checklist.

## Pin a client to one provider

Use `https://<hub>/provider/<name>` as the client base URL. Provider names match registry keys
and are case-insensitive, including configured `api-keys.openai-compatibility` names such as
`mistral`. For example, set an OpenAI-compatible client's base URL to
`https://<hub>/provider/mistral/v1`, or a Gemini client's base URL to
`https://<hub>/provider/antigravity` when it appends `/v1beta` itself. Authenticate with a hub
access key.

The prefix supports these routes.

- `GET /provider/<name>/v1/models` and `/v1/models/<model>` use the existing OpenAI or Anthropic
catalog shape selected by the client headers.
- `GET /provider/<name>/v1beta/models` and `/v1beta/models/<model>` return Gemini model metadata.
- `POST /provider/<name>/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/v1/messages`
and `/v1/messages/count_tokens` use the existing HTTP handlers, including streaming.
- `POST /provider/<name>/v1beta/models/<model>:generateContent`, `:streamGenerateContent` and
`:countTokens` use the Gemini handlers.

Catalogs contain that provider's registered models. Requests select credentials from that
provider's accounts, including when another provider serves the same model ID. A model outside
the scope gets the usual `model_not_found` error. An unknown provider returns HTTP 404.

Configured public model aliases, credential prefixes and thinking suffixes keep their usual
meaning within the selected provider. Keep any credential prefix in the model ID, such as
`team/model-name`. The `auto` model selects from the scoped catalog. Session affinity bindings
are isolated by provider scope. Scoped routes resolve through the registry and bypass plugin
model routers. Payload rules and request interceptors still run through the existing execution
pipeline.

## Capability table

The table describes this checkout. **Text APIs** means the hub's translated Chat Completions,
Expand Down
57 changes: 36 additions & 21 deletions docs/t3code.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,34 +41,49 @@ signs in directly and does not go through the hub.

## Antigravity (Gemini)

T3 Code's Antigravity provider runs the Antigravity CLI (`agy`). With its **Gemini API key** sign-in
the CLI sends Gemini API requests, and it reads the endpoint from `GOOGLE_GEMINI_BASE_URL`. That
lets the stock provider use the hub's Antigravity account. It needs an `agy` release that knows
this variable; 1.5.9 does.
T3 Code's Antigravity provider runs the Antigravity CLI (`agy`). Its enterprise model gateway
mode discovers models from the hub and sends requests through the hub's Antigravity accounts.
This needs a recent `agy` release with gateway support.

1. Open **Settings → Providers → Antigravity**.
2. Set the sign-in method to **Gemini API key** and enter a hub access key, not a Google key.
3. Add the environment variable `GOOGLE_GEMINI_BASE_URL` with the hub's root URL, such as
`https://<hub>.<tailnet>.ts.net:8317`.
2. Set the sign-in method to **Gemini API key** and enter a hub access key.
3. Add these provider environment variables. Mark the access key as sensitive.

The model list comes from the CLI, not from the hub, so it also shows Gemini models your
Antigravity account does not serve. Those fail with `unknown provider for model`. Hide them in
the model picker and keep the ones `GET /v1beta/models` on the hub returns. The Google account
sign-in method does not go through the hub.
```text
AGY_LLM_GATEWAY_URL=https://<hub>/provider/antigravity
AGY_LLM_GATEWAY_API_KEY=<hub access key>
```

The model list comes from the hub and contains the models served by its Antigravity accounts.
The Google account sign-in method connects directly to Google.

## Mistral (OpenCode and Vibe)

Mistral has two monthly budgets on one account: API usage and Vibe Code. Which one a request
draws on is decided by Mistral from the client that sent it.

- **OpenCode** pointed at the hub uses the hub's Mistral upstream (an `api-keys.openai-compatibility`
entry for `https://api.mistral.ai/v1`) and draws on the API budget.
- **Mistral Vibe**, added in T3 Code as an ACP agent and signed in with Mistral, talks to Mistral
directly and draws on the Vibe Code budget. It does not pass through the hub.

The hub reads both budgets with its Mistral key and shows them on the Overview page as money
spent of the monthly amount, with the reset date, whichever client did the spending. See
[quota history](quota-history.md).
draws on depends on the key that sends it: a key made in the Mistral console draws on the API
budget, and the key Mistral Vibe gets when you sign in with Mistral draws on the Vibe Code budget.
To run both through the hub, give the hub both keys as two upstreams.

1. Keep the console key as the `mistral` upstream (`api-keys.openai-compatibility`, base URL
`https://api.mistral.ai/v1`). OpenCode and other clients use it as `<model>`.
2. Add the Vibe key as a second upstream with the same base URL and models and `prefix: vibe`, and
set `force-model-prefix: true`. Its models are then reachable only as `vibe/<model>`, so nothing
else spends the Vibe budget by accident. On macOS, Vibe keeps its key in the keychain under the
service `ai.mistral.vibe`.
3. Declare the reasoning levels Mistral accepts on models that have them, for example
`thinking: {levels: [none, high], zero-allowed: true}` on `mistral-vibe-cli-latest`. Without it
the hub turns "thinking off" into `reasoning_effort: low`, which Mistral rejects.
4. Point Vibe at the hub in `~/.vibe/config.toml`: a `mistral` provider with
`api_base = "https://<hub>.<tailnet>.ts.net:8317/v1"`, `backend = "mistral"`,
`preserve_api_base = true`, and `api_key_env_var = "MISTRAL_API_KEY"`, with the hub access key
in `~/.vibe/.env`. Add each model as `[[models]]` with `name = "vibe/<model>"` and the alias Vibe
shows, such as `mistral-medium-3.5` for `vibe/mistral-vibe-cli-latest`.

T3 Code's Mistral Vibe agent (ACP) reads the same files, so it follows.

The hub reads both budgets and shows them on the Overview page as money spent of the monthly
amount, with the reset date. Two keys of one Mistral customer are recorded as one account, so
neither allowance is counted twice. See [quota history](quota-history.md).

## Other upstream providers

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
func TestQuotaHistoryMistralAPIKeyEndpoint(t *testing.T) {
budget := `{"vibe_budget":{"usage_percentage":0.0,"initial_budget":150.0,"currency":"USD","reset_at":"2026-12-01T00:00:00Z","payg_enabled":false},"api_budget":{"usage_percentage":2.59892596,"initial_budget":15.0,"currency":"USD","reset_at":"2026-12-01T00:00:00Z","payg_enabled":false},"usage_percentage":2.59892596,"initial_budget":15.0,"currency":"USD","reset_at":"2026-12-01T00:00:00Z"}`
upstream := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
if req.Method != "GET" || req.Host != "api.mistral.ai" || req.Header.Get("Authorization") != "Bearer fixture-mistral-key" {
if req.Method != "GET" || (req.Host != "api.mistral.ai" && req.Host != "console.mistral.ai") || req.Header.Get("Authorization") != "Bearer fixture-mistral-key" {
t.Error("billing request must use the upstream's own Bearer key and host")
w.WriteHeader(http.StatusBadRequest)
return
Expand All @@ -31,6 +31,8 @@ func TestQuotaHistoryMistralAPIKeyEndpoint(t *testing.T) {
_, _ = w.Write([]byte(budget))
case "/v1/billing/subscription":
_, _ = w.Write([]byte(`{"plan_name":"EDU","payg_enabled":false,"vibe_payg_enabled":false,"enterprise":false}`))
case "/api/vibe/whoami":
_, _ = w.Write([]byte(`{"customer_id":"fixture-customer"}`))
default:
t.Errorf("unexpected billing path %q", req.URL.Path)
w.WriteHeader(http.StatusNotFound)
Expand Down
58 changes: 58 additions & 0 deletions internal/api/provider_routes.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package api

import (
"errors"
"fmt"
"net/http"
"strings"

"github.com/gin-gonic/gin"
"github.com/rune-oai/t3code-proxy/internal/interfaces"
"github.com/rune-oai/t3code-proxy/internal/registry"
"github.com/rune-oai/t3code-proxy/internal/util"
"github.com/rune-oai/t3code-proxy/sdk/api/handlers"
"github.com/rune-oai/t3code-proxy/sdk/api/handlers/claude"
"github.com/rune-oai/t3code-proxy/sdk/api/handlers/gemini"
"github.com/rune-oai/t3code-proxy/sdk/api/handlers/openai"
"github.com/rune-oai/t3code-proxy/sdk/cliproxy/auth"
)

func (s *Server) setupProviderRoutes(openaiHandler *openai.OpenAIAPIHandler, claudeHandler *claude.ClaudeCodeAPIHandler, responsesHandler *openai.OpenAIResponsesAPIHandler, geminiHandler *gemini.GeminiAPIHandler) {
group := s.engine.Group("/provider/:provider", AuthMiddleware(s.accessManager))
group.Use(func(c *gin.Context) {
provider := strings.ToLower(strings.TrimSpace(c.Param("provider")))
modelRegistry := registry.GetGlobalRegistry()
if !modelRegistry.HasProvider(provider) {
provider = util.OpenAICompatibleProviderKey(provider)
}
if !modelRegistry.HasProvider(provider) {
message := fmt.Sprintf("unknown provider %q", c.Param("provider"))
errMsg := &interfaces.ErrorMessage{StatusCode: http.StatusNotFound, Error: errors.New(message)}
switch {
case strings.Contains(c.FullPath(), "/v1beta/"):
c.JSON(http.StatusNotFound, gin.H{"error": gin.H{"code": http.StatusNotFound, "message": message, "status": "NOT_FOUND"}})
case strings.Contains(c.FullPath(), "/messages") || (strings.Contains(c.FullPath(), "/v1/models") && isAnthropicModelsRequest(c)):
claudeHandler.WriteErrorResponse(c, errMsg)
default:
s.handlers.WriteErrorResponse(c, errMsg)
}
c.Abort()
return
}
c.Request = c.Request.WithContext(auth.WithProviderScope(c.Request.Context(), provider))
c.Next()
})
group.GET("/v1/models", s.unifiedModelsHandler(openaiHandler, claudeHandler))
group.GET("/v1/models/*model", func(c *gin.Context) {
c.Set(handlers.ModelDetailIDContextKey, strings.TrimPrefix(c.Param("model"), "/"))
s.unifiedModelsHandler(openaiHandler, claudeHandler)(c)
})
group.POST("/v1/chat/completions", openaiHandler.ChatCompletions)
group.POST("/v1/completions", openaiHandler.Completions)
group.POST("/v1/responses", responsesHandler.Responses)
group.POST("/v1/messages", claudeHandler.ClaudeMessages)
group.POST("/v1/messages/count_tokens", claudeHandler.ClaudeCountTokens)
group.GET("/v1beta/models", s.geminiModelsHandler(geminiHandler))
group.GET("/v1beta/models/*action", s.geminiGetHandler(geminiHandler))
group.POST("/v1beta/models/*action", geminiHandler.GeminiHandler)
}
Loading
Loading