Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,17 +105,17 @@ Gateway uploads renew a 30-second idle read deadline when bytes arrive, with a f

Gateway admission allows model changes with reasoning, compaction and reconstructed WebSocket history when the current bound account remains usable and supports the requested model. WebSocket continuation identity survives normalization so rebuilt transcripts cannot authorize account migration. Complete portable history may still replace an unsupported binding; a durable rebind marker protects older model-scoped state after recorded account moves, legacy-scope adoption and restart. Additive migration 019 preserves existing bindings and treats their current account as the continuity baseline; earlier account moves cannot be reconstructed. Incompatible model continuations return HTTP 409 with `conversation_model_conflict`, distinct from an unavailable account binding, and request history and every supported language provide model-specific recovery guidance. Live provider and desktop acceptance remain separate.

Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences rather than strict bindings: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, unavailable resources fall back to normal scheduling, and a full affinity table skips the preference instead of rejecting traffic. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource without changing its binding. Claude Code subagents share their main session key. Explicit sessions keep the existing strict binding rules.
Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key. Explicit sessions keep the existing strict binding rules.

API channels can be edited in place at `/channels`: name, platform, protocol, base URL and, optionally, the API key. Leaving the key blank keeps the stored secret. Edits retain the channel ID, group assignments, limits and request history; endpoint, protocol or key changes reset verification, and endpoint or protocol changes invalidate the model catalog. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope and the name becomes available again.
API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history, and a key change resets verification. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.

Network proxies are shown as compact cards in a responsive grid whose column count follows the page width. Each card shows its status and latency, an IP purity panel, and the exit IP, location, network, verdict and bound accounts exactly once, merging the proxy check with the IP report. The panel reads the exit IP observed by the last successful check, never the proxy endpoint, from the public ip.net.coffee lookup used by its IP scoring page and shows its trust score with the same five purity tiers, network type, risk flags, ASN, ISP, location and verdict. Only public exit IPs are sent; results are cached per IP for six hours, forced refreshes are paced to one per 30 seconds per IP, and unchecked proxies make no lookup. The report is informational and does not affect routing or proxy checks.
Network proxies are shown as compact cards in a responsive grid whose column count follows the page width. Each card shows its status and latency, an IP purity panel, and the exit IP, location, network, verdict and bound accounts exactly once, merging the proxy check with the IP report. Opening the page only reads reports the server already holds; an administrator must explicitly request a lookup, which sends the exit IP observed by the last successful check, never the proxy endpoint, to the public ip.net.coffee lookup used by its IP scoring page. The panel shows its trust score with the same five purity tiers, network type, risk flags, ASN, ISP, location and verdict. Only public exit IPs are sent; results are cached per IP for six hours, concurrent lookups for one IP share a single request, attempts are paced to one per 30 seconds per IP from when they start, and rate-limited responses back off for five minutes. The report is informational and does not affect routing or proxy checks.

Workspace administrators can delete a resource allowance once no admitted request still holds one of its reservations. Deletion removes the allowance, its revisions, price snapshots, key bindings and allowance ledger in one audited transaction; request history and usage statistics are retained. Keys that were bound to it keep their resource group without a budget limit, and the group's resources are no longer locked by the allowance.

OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Pricing reads image-generation output prices per image token. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.
OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.

OpenCode Go API channels show upstream spending windows from the official key-authenticated `GET https://opencode.ai/zen/go/v1/usage` endpoint: the rolling five-hour, weekly and monthly percentages with their reset times. Reads share the quota cache, refresh bounds and failure states of subscription usage. These observations are informational and do not gate channel scheduling.
OpenCode Go API channels whose base URL is the official `https://opencode.ai` origin show upstream spending windows from the key-authenticated `GET https://opencode.ai/zen/go/v1/usage` endpoint; relay URLs using the preset never send their key there. The windows are the rolling five-hour, weekly and monthly percentages with their reset times. Reads share the quota cache, refresh bounds and failure states of subscription usage. These observations are informational and do not gate channel scheduling.

## Brand Commitments

Expand Down
20 changes: 10 additions & 10 deletions internal/accounts/apikey.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,8 @@ func normalizeBaseURL(value string) (string, error) {
return strings.TrimRight(u.String(), "/"), nil
}

// APIChannelUpdate edits an API channel in place. An empty APIKey keeps the stored key.
// APIChannelUpdate renames a channel or replaces its key. An empty APIKey keeps the stored key;
// BaseURL, Platform and Protocol must match the stored channel.
type APIChannelUpdate struct {
Name string
APIKey string
Expand All @@ -114,8 +115,8 @@ type APIChannelUpdate struct {
Protocol string
}

// UpdateAPIChannel changes channel settings without replacing its resource ID, group membership,
// limits or conversation bindings. Endpoint and protocol changes invalidate the model catalog.
// UpdateAPIChannel changes the name or key without replacing the resource ID, group membership,
// limits or conversation bindings.
func (s *Service) UpdateAPIChannel(ctx context.Context, id string, input APIChannelUpdate) (Account, error) {
name, err := NormalizeName(input.Name)
if err != nil {
Expand Down Expand Up @@ -151,8 +152,12 @@ func (s *Service) UpdateAPIChannel(ctx context.Context, id string, input APIChan
if credential.APIPlatform == "" && credential.APIProtocol == "" {
credential.APIPlatform, credential.APIProtocol = previous.APIPlatform, previous.APIProtocol
}
endpointChanged := previous.BaseURL != credential.BaseURL || previous.APIConfig() != credential.APIConfig()
connectionChanged := endpointChanged || credential.AccessToken != previous.AccessToken
// Credentials and conversation bindings stay scoped to the original upstream: a stored key must
// never be sent to another endpoint or protocol, so those changes still require a new channel.
if previous.BaseURL != credential.BaseURL || previous.APIConfig() != credential.APIConfig() {
return Account{}, ErrIdentity
}
connectionChanged := credential.AccessToken != previous.AccessToken
status := row.Status
if connectionChanged {
status = "unverified"
Expand All @@ -168,11 +173,6 @@ func (s *Service) UpdateAPIChannel(ctx context.Context, id string, input APIChan
return Account{}, err
}
}
if endpointChanged {
if err := q.InvalidateAccountCatalog(ctx, id); err != nil {
return Account{}, err
}
}
now := s.now().Unix()
if name != row.Name {
if err := q.RenameAccount(ctx, db.RenameAccountParams{ID: id, TenantID: s.tenantID, Name: name, UpdatedAt: now}); err != nil {
Expand Down
19 changes: 15 additions & 4 deletions internal/allocations/allocations.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ type Rate struct {
Input int64 `json:"input"`
Cached int64 `json:"cached"`
Output int64 `json:"output"`
// ImageInput charges image input tokens of image models; zero falls back to Input.
ImageInput int64 `json:"image_input,omitempty"`
}
type Config struct {
Mode string `json:"mode"`
Expand Down Expand Up @@ -240,7 +242,7 @@ func normalize(c *Config) error {
}
models[model] = true
c.Rates[i].Model = model
if r.Input <= 0 || r.Cached < 0 || r.Output <= 0 || r.Input > 1_000_000_000 || r.Cached > 1_000_000_000 || r.Output > 1_000_000_000 {
if r.Input <= 0 || r.Cached < 0 || r.Output <= 0 || r.ImageInput < 0 || r.Input > 1_000_000_000 || r.Cached > 1_000_000_000 || r.Output > 1_000_000_000 || r.ImageInput > 1_000_000_000 {
return ErrInput
}
}
Expand Down Expand Up @@ -674,7 +676,7 @@ func (s *Service) applyPrices(ctx context.Context, q *db.Queries, groupID int64,
config.Rates = make([]Rate, 0, len(ids))
for _, model := range ids {
if price, ok := s.pricing.Lookup(model); ok {
config.Rates = append(config.Rates, Rate{Model: model, Input: price.Input, Cached: price.Cached, Output: price.Output})
config.Rates = append(config.Rates, Rate{Model: model, Input: price.Input, Cached: price.Cached, Output: price.Output, ImageInput: price.ImageInput})
continue
}
if rate, ok := previousByModel[model]; ok {
Expand All @@ -687,12 +689,21 @@ func (s *Service) applyPrices(ctx context.Context, q *db.Queries, groupID int64,
return &catalog, nil
}
func Cost(r Rate, input, output, cached int64) (int64, error) {
if input < 0 || output < 0 || cached < 0 || cached > input || input > 1_000_000_000 || output > 1_000_000_000 {
return CostWithImages(r, input, 0, output, cached)
}

// CostWithImages settles input by modality: imageInput tokens are part of input and use the image rate.
func CostWithImages(r Rate, input, imageInput, output, cached int64) (int64, error) {
if input < 0 || output < 0 || cached < 0 || imageInput < 0 || cached+imageInput > input || input > 1_000_000_000 || output > 1_000_000_000 {
return 0, ErrInput
}
imageRate := r.ImageInput
if imageRate == 0 {
imageRate = r.Input
}
// Round only after summing all token categories; even sub-micro charges remain nonzero.
sum := new(big.Int)
for _, v := range [][2]int64{{input - cached, r.Input}, {cached, r.Cached}, {output, r.Output}} {
for _, v := range [][2]int64{{input - cached - imageInput, r.Input}, {imageInput, imageRate}, {cached, r.Cached}, {output, r.Output}} {
if v[1] < 0 {
return 0, ErrInput
}
Expand Down
21 changes: 21 additions & 0 deletions internal/allocations/allocations_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1772,3 +1772,24 @@ func TestManagedAccountDeletionExplainsPoolLock(t *testing.T) {
t.Fatal("missing actionable lock error", err)
}
}

func TestCostSettlesImageInputByModality(t *testing.T) {
// Rates are micro-USD per million tokens: text input $5/M, image input $8/M, image output $30/M.
rate := Rate{Input: 5_000_000, Cached: 1_250_000, Output: 30_000_000, ImageInput: 8_000_000}
cost, err := CostWithImages(rate, 1_000_000, 400_000, 1_000_000, 100_000)
// 500k text * 5 + 400k image * 8 + 100k cached * 1.25 + 1M output * 30 = 2.5 + 3.2 + 0.125 + 30 USD.
if err != nil || cost != 35_825_000 {
t.Fatal(cost, err)
}
if plain, err := Cost(rate, 1_000_000, 1_000_000, 100_000); err != nil || plain != 34_625_000 {
t.Fatal("text-only cost changed", plain, err)
}
fallback := rate
fallback.ImageInput = 0
if cost, err := CostWithImages(fallback, 1_000_000, 400_000, 0, 0); err != nil || cost != 5_000_000 {
t.Fatal("missing image rate must use the input rate", cost, err)
}
if _, err := CostWithImages(rate, 100, 60, 0, 50); !errors.Is(err, ErrInput) {
t.Fatal("overlapping categories accepted", err)
}
}
10 changes: 6 additions & 4 deletions internal/allocations/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,11 @@ func Begin(ctx context.Context, q *db.Queries, r Request, now int64, location *t

type Completion struct {
Input, Output, Cached int64
Known bool
Dispatched bool
Rejected bool
// ImageInput is the image share of Input, reported by image generation responses.
ImageInput int64
Known bool
Dispatched bool
Rejected bool
}

func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int64, manual bool) error {
Expand Down Expand Up @@ -237,7 +239,7 @@ func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int
if err != nil {
return err
}
cost, err = Cost(rate, c.Input, c.Output, c.Cached)
cost, err = CostWithImages(rate, c.Input, c.ImageInput, c.Output, c.Cached)
if err != nil {
return err
}
Expand Down
2 changes: 2 additions & 0 deletions internal/backup/backup_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,8 @@ ALTER TABLE accounts DROP COLUMN reset_check_at;
ALTER TABLE accounts DROP COLUMN usage_revision;
ALTER TABLE account_affinity DROP COLUMN rebound;
ALTER TABLE account_affinity DROP COLUMN model;
DROP INDEX account_groups_active_name;
ALTER TABLE account_affinity DROP COLUMN derived;
ALTER TABLE account_groups DROP COLUMN session_sticky;
ALTER TABLE account_groups DROP COLUMN deleted_at;
ALTER TABLE account_groups DROP COLUMN routing_preference;
Expand Down
2 changes: 1 addition & 1 deletion internal/channels/channels.go
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ func (s *Service) Delete(ctx context.Context, id string) error {
return s.resources.Delete(ctx, id)
}

// Update edits name, endpoint, platform, protocol and optionally the key while keeping the channel ID.
// Update renames a channel and optionally replaces its key while keeping the channel ID.
func (s *Service) Update(ctx context.Context, id string, input Input) (Channel, error) {
if _, err := s.Get(ctx, id); err != nil {
return Channel{}, err
Expand Down
6 changes: 6 additions & 0 deletions internal/gateway/affinity.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ func sessionDigest(userID, groupID int64, session string) [32]byte {
return sha256.Sum256([]byte(fmt.Sprintf("%d:%d:%s", userID, groupID, session)))
}

// derivedDigest hashes stickiness preferences in a namespace explicit session digests cannot produce:
// explicit inputs always begin with a numeric user ID.
func derivedDigest(userID, groupID int64, session string) [32]byte {
return sha256.Sum256([]byte(fmt.Sprintf("derived:%d:%d:%s", userID, groupID, session)))
}

func sessionBinding(ctx context.Context, q *db.Queries, params db.ListSessionAffinitiesParams, scope string) (*db.ListSessionAffinitiesRow, error) {
bindings, err := q.ListSessionAffinities(ctx, params)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion internal/gateway/allocations.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ func (e *observation) settleAllocation(ctx context.Context, q *db.Queries) error
}
known := e.record.InputTokens != nil && e.record.OutputTokens != nil && (e.usageFinal || e.record.Outcome == "success" || e.record.Outcome == "incomplete")
rejected := e.record.UpstreamStatus != nil && *e.record.UpstreamStatus >= 400
err := allocations.Finish(ctx, q, e.record.RequestID, allocations.Completion{Input: value(e.record.InputTokens), Output: value(e.record.OutputTokens), Cached: value(e.record.CachedTokens), Known: known, Dispatched: e.upstreamDispatched, Rejected: rejected}, e.service.now().UnixMilli(), false)
err := allocations.Finish(ctx, q, e.record.RequestID, allocations.Completion{Input: value(e.record.InputTokens), ImageInput: e.imageInputTokens, Output: value(e.record.OutputTokens), Cached: value(e.record.CachedTokens), Known: known, Dispatched: e.upstreamDispatched, Rejected: rejected}, e.service.now().UnixMilli(), false)
if errors.Is(err, sql.ErrNoRows) {
return ErrAllocationAccounting
}
Expand Down
15 changes: 15 additions & 0 deletions internal/gateway/history.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@ func requestReasoningEffort(input map[string]json.RawMessage, kind Kind) string
}

type observation struct {
// imageInputTokens is the image share of input tokens reported by Images responses; it only
// affects allowance settlement and is not stored in request history.
imageInputTokens int64
route *RouteExplanation
kind Kind
service *Service
Expand Down Expand Up @@ -381,6 +384,7 @@ func (e *observation) observe(raw []byte) {
Output *int64 `json:"output_tokens"`
Details struct {
Cached *int64 `json:"cached_tokens"`
Image *int64 `json:"image_tokens"`
} `json:"input_tokens_details"`
}
var value struct {
Expand Down Expand Up @@ -408,4 +412,15 @@ func (e *observation) observe(raw []byte) {
e.record.InputTokens = valid(tokens.Input)
e.record.OutputTokens = valid(tokens.Output)
e.record.CachedTokens = valid(tokens.Details.Cached)
e.imageInputTokens = 0
// Settlement rejects overlapping categories, so an inconsistent split keeps the text input rate.
if image := valid(tokens.Details.Image); image != nil && e.record.InputTokens != nil {
cached := int64(0)
if e.record.CachedTokens != nil {
cached = *e.record.CachedTokens
}
if cached+*image <= *e.record.InputTokens {
e.imageInputTokens = *image
}
}
}
Loading