Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,15 +105,15 @@ Gateway uploads renew a 30-second idle read deadline when bytes arrive, with a f

Gateway admission allows model changes with reasoning, compaction and reconstructed WebSocket history when the current bound account remains usable and supports the requested model. WebSocket continuation identity survives normalization so rebuilt transcripts cannot authorize account migration. Complete portable history may still replace an unsupported binding; a durable rebind marker protects older model-scoped state after recorded account moves, legacy-scope adoption and restart. Additive migration 019 preserves existing bindings and treats their current account as the continuity baseline; earlier account moves cannot be reconstructed. Incompatible model continuations return HTTP 409 with `conversation_model_conflict`, distinct from an unavailable account binding, and request history and every supported language provide model-specific recovery guidance. Live provider and desktop acceptance remain separate.

Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation already bound under its raw `metadata.user_id` key before opt-in keeps that binding, and only unbound conversations adopt the normalized key. Explicit sessions keep the existing strict binding rules.
Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation keeps whichever raw or normalized key already holds its binding whether the setting is on or off; the setting only decides whether an unbound conversation creates a normalized binding. Explicit sessions keep the existing strict binding rules.

API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history; a key change resets verification and invalidates the model catalog in the same transaction. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.

Network proxies are shown as compact cards in a responsive grid whose column count follows the page width. Each card shows its status and latency, an IP purity panel, and the exit IP, location, network, verdict and bound accounts exactly once, merging the proxy check with the IP report. Opening the page only reads reports the server already holds; an administrator must explicitly request a lookup, which sends the exit IP observed by the last successful check, never the proxy endpoint, to the public ip.net.coffee lookup used by its IP scoring page. The panel shows its trust score with the same five purity tiers, network type, risk flags, ASN, ISP, location and verdict. Only public exit IPs are sent; results are cached per IP for six hours, concurrent lookups for one IP share a single request, attempts are paced to one per 30 seconds per IP from when they start, rate-limited responses back off for five minutes, and a new IP is refused while all 256 cache slots hold in-flight lookups. The report is informational and does not affect routing or proxy checks.

Workspace administrators can delete a resource allowance once no admitted request still holds one of its reservations. Deletion removes the allowance, its revisions, price snapshots, key bindings and allowance ledger in one audited transaction; request history and usage statistics are retained. Keys that were bound to it keep their resource group without a budget limit, and the group's resources are no longer locked by the allowance.

OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens; manual settlement of a pending request accepts the same optional image input count. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.
OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens; manual settlement of a pending request accepts the same optional image input count, stores it with the entry, and treats only an identical breakdown as a retry; a different split is a conflict. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.

OpenCode Go API channels whose base URL is the official `https://opencode.ai` origin show upstream spending windows from the key-authenticated `GET https://opencode.ai/zen/go/v1/usage` endpoint; relay URLs using the preset never send their key there. The windows are the rolling five-hour, weekly and monthly percentages with their reset times. Reads share the quota cache, refresh bounds and failure states of subscription usage. These observations are informational and do not gate channel scheduling.

Expand Down
11 changes: 11 additions & 0 deletions internal/allocations/allocations_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1826,6 +1826,17 @@ func TestManualSettlementChargesImageInputAtImageRate(t *testing.T) {
if cost != 35_825_000 {
t.Fatalf("manual settlement ignored the image input split: %d", cost)
}
// An identical retry succeeds; the same totals with another image split are a conflict.
if err := s.Settle(ctx, 9, r.ID, Completion{Input: 1_000_000, ImageInput: 400_000, Cached: 100_000, Output: 1_000_000}); err != nil {
t.Fatal("identical settlement retry failed", err)
}
if err := s.Settle(ctx, 9, r.ID, Completion{Input: 1_000_000, ImageInput: 600_000, Cached: 100_000, Output: 1_000_000}); !errors.Is(err, ErrSettlement) {
t.Fatal("a different image split was treated as a retry", err)
}
var stored, image int64
if err := conn.QueryRow(`SELECT cost, image_input_tokens FROM allocation_entries WHERE request_id=?`, r.ID).Scan(&stored, &image); err != nil || stored != 35_825_000 || image != 400_000 {
t.Fatal("settlement details changed", stored, image, err)
}
overlap := r
overlap.ID = "synthetic-image-overlap"
if err := Begin(ctx, q, overlap, now.Unix(), time.UTC); err != nil {
Expand Down
5 changes: 3 additions & 2 deletions internal/allocations/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,8 @@ func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int
return nil
}
if manual {
if e.State == "settled" && e.Manual == 1 && e.InputTokens == c.Input && e.OutputTokens == c.Output && e.CachedTokens == c.Cached {
// Only an identical breakdown is a retry; a different image split would have charged differently.
if e.State == "settled" && e.Manual == 1 && e.InputTokens == c.Input && e.OutputTokens == c.Output && e.CachedTokens == c.Cached && e.ImageInputTokens == c.ImageInput {
return nil
}
if e.State != "pending" || c.Input < e.InputTokens || c.Output < e.OutputTokens {
Expand Down Expand Up @@ -248,7 +249,7 @@ func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int
// Partial observations remain available for correction but are not a charge.
cost = 0
}
return q.FinishAllocationEntry(ctx, db.FinishAllocationEntryParams{RequestID: id, FinishedAt: now, State: state, InputTokens: c.Input, OutputTokens: c.Output, CachedTokens: c.Cached, Cost: cost, Manual: bit(manual)})
return q.FinishAllocationEntry(ctx, db.FinishAllocationEntryParams{RequestID: id, FinishedAt: now, State: state, InputTokens: c.Input, OutputTokens: c.Output, CachedTokens: c.Cached, ImageInputTokens: c.ImageInput, Cost: cost, Manual: bit(manual)})
}
func (s *Service) Settle(ctx context.Context, scheme int64, id string, c Completion) error {
tx, err := s.conn.BeginTx(ctx, nil)
Expand Down
1 change: 1 addition & 0 deletions internal/backup/backup_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ ALTER TABLE accounts DROP COLUMN usage_revision;
ALTER TABLE account_affinity DROP COLUMN rebound;
ALTER TABLE account_affinity DROP COLUMN model;
DROP INDEX account_groups_active_name;
ALTER TABLE allocation_entries DROP COLUMN image_input_tokens;
ALTER TABLE account_affinity DROP COLUMN derived;
ALTER TABLE account_groups DROP COLUMN session_sticky;
ALTER TABLE account_groups DROP COLUMN deleted_at;
Expand Down
49 changes: 36 additions & 13 deletions internal/gateway/selection.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,20 +89,43 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i
return "", digest, err
}
derived := false
if hints, ok := stickyFrom(ctx); ok && policy.Routing.SessionSticky {
normalize := hints.Session != "" && (session == "" || hints.Replace)
if normalize && session != "" {
// A conversation bound under its raw metadata key before opt-in keeps that binding:
// provider-held state must continue on its original account.
rawScope := provider
if rawScope == "" {
rawScope = "auto"
if hints, ok := stickyFrom(ctx); ok {
sticky := policy.Routing.SessionSticky
bindingScope := provider
if bindingScope == "" {
bindingScope = "auto"
}
// bound reports whether a key already owns a binding; ambiguous legacy bindings count as bound.
bound := func(hash [32]byte) (bool, error) {
binding, err := sessionBinding(ctx, q, db.ListSessionAffinitiesParams{GroupID: groupID, UserID: userID, SessionHash: hash[:], Now: s.now().Unix()}, bindingScope)
if errors.Is(err, ErrAffinityUnavailable) {
return true, nil
}
bound, err := sessionBinding(ctx, q, db.ListSessionAffinitiesParams{GroupID: groupID, UserID: userID, SessionHash: digest[:], Now: s.now().Unix()}, rawScope)
if err != nil && !errors.Is(err, ErrAffinityUnavailable) {
return "", digest, err
return binding != nil, err
}
normalize := false
switch {
case hints.Session == "" || session != "" && !hints.Replace:
case hints.Derived:
// Preferences are the opt-in feature itself and disappear with the setting.
normalize = sticky
default:
// Explicit Claude conversations keep whichever key already holds their binding, whether the
// setting is on or off, so provider-held state never moves when the toggle changes.
rawBound := false
if session != "" {
if rawBound, err = bound(digest); err != nil {
return "", digest, err
}
}
if !rawBound {
normalizedBound, err := bound(sessionDigest(userID, groupID, hints.Session))
if err != nil {
return "", digest, err
}
// The setting only decides whether an unbound conversation creates a normalized binding.
normalize = normalizedBound || sticky
}
normalize = bound == nil && err == nil
}
if normalize {
session, derived = hints.Session, hints.Derived
Expand All @@ -113,7 +136,7 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i
}
}
// Subagents prefer their parent's resource among admitted candidates, as Codex reviews already do.
if parentThread == "" && hints.Parent != "" && hints.Parent != session {
if sticky && parentThread == "" && hints.Parent != "" && hints.Parent != session {
parentThread = hints.Parent
}
}
Expand Down
36 changes: 36 additions & 0 deletions internal/gateway/sticky_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -236,3 +236,39 @@ func TestStickinessHonorsBindingsMadeBeforeOptIn(t *testing.T) {
}
}
}

func TestNormalizedBindingsSurviveOptOut(t *testing.T) {
s, _, _ := stickyGateway(t, true)
ctx := context.Background()
raw := `{"device_id":"synthetic","session_id":"synthetic-opt-out"}`
hints := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-opt-out", Replace: true})
original, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, "")
if err != nil {
t.Fatal(err)
}
s, _, _ = stickyGatewayFrom(t, s, false)
if _, err := s.accounts.SetEnabled(ctx, original, false); err != nil {
t.Fatal(err)
}
// Turning the setting off must not turn a bound continuation into a new session.
got, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, "")
if err == nil || got != original {
t.Fatalf("normalized binding was dropped after opt-out: %q %v", got, err)
}
if _, err := s.accounts.SetEnabled(ctx, original, true); err != nil {
t.Fatal(err)
}
if got, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, ""); err != nil || got != original {
t.Fatal("continuation left its normalized binding", got, err)
}
// With the setting off, unbound conversations keep using their raw key and create no normalized binding.
fresh := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-unbound", Replace: true})
if _, _, err := s.selectAllocationAccount(fresh, 1, 1, `{"session_id":"synthetic-unbound"}`, "", "synthetic-model", Responses, 0, pinnedAffinity, ""); err != nil {
t.Fatal(err)
}
normalized := sessionDigest(1, 1, "claude:synthetic-unbound")
var count int
if err := s.db.QueryRow("SELECT count(*) FROM account_affinity WHERE session_hash=?", normalized[:]).Scan(&count); err != nil || count != 0 {
t.Fatal("opt-out created a normalized binding", count, err)
}
}
26 changes: 15 additions & 11 deletions internal/storage/db/allocations.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 18 additions & 17 deletions internal/storage/db/models.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading