Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Gateway admission allows model changes with reasoning, compaction and reconstruc

Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation keeps whichever raw or normalized key already holds its binding whether the setting is on or off; the setting only decides whether an unbound conversation creates a normalized binding. Explicit sessions keep the existing strict binding rules.

Resource groups also offer optional quota failover, disabled by default. When enabled, an existing Responses conversation whose bound account has exhausted its quota moves to another admitted account of the same provider instead of failing. The binding records 16-byte fingerprints (at most 4,096) of every encrypted reasoning or compaction item present at the move, and those items are withheld from the new account on that and every later turn, so only readable transcript and the new account's own state reach it. Requests that reference provider-held state (`previous_response_id`, `conversation`, `container` or `item_reference`) never move, and disablement, cooldown, saturation or model mismatch still never move a conversation.
Resource groups also offer quota failover, enabled by default for new and existing groups and switchable per group. When enabled, an existing Responses conversation whose bound account has exhausted its quota moves to another admitted account of the same provider instead of failing. The binding records 16-byte fingerprints (at most 4,096) of every encrypted reasoning or compaction item present at the move, and those items are withheld from the new account on that and every later turn, so only readable transcript and the new account's own state reach it. Requests that reference provider-held state (`previous_response_id`, `conversation`, `container` or `item_reference`) never move, and disablement, cooldown, saturation or model mismatch still never move a conversation.

API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history; a key change resets verification and invalidates the model catalog in the same transaction. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.

Expand Down
2 changes: 1 addition & 1 deletion internal/gateway/selection.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i
}
if err := quotaAdmission(*bound, model); err != nil {
var quota *QuotaError
// Opt-in only: the default keeps the strict rule that conversations never move.
// Groups that turn failover off keep the strict rule that conversations never move.
if !errors.As(err, &quota) || !policy.Routing.QuotaFailover || carrier == nil || !carrier.Movable || existing.Provider != scope {
return id, digest, err
}
Expand Down
14 changes: 14 additions & 0 deletions internal/groups/groups_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -410,3 +410,17 @@ func TestDeletedGroupNamesNeverCollide(t *testing.T) {
t.Fatal("active names are no longer unique", err)
}
}

func TestQuotaFailoverDefaultsOnAndCanBeTurnedOff(t *testing.T) {
_, service, _, account := fixture(t)
ctx := context.Background()
pool, err := service.Save(ctx, 0, groups.Input{Name: "Synthetic failover pool", Enabled: true, AccountIDs: []string{account.ID}})
if err != nil || !pool.Routing.QuotaFailover {
t.Fatalf("new pool lacks default quota failover: %+v, %v", pool.Routing, err)
}
off := groups.Routing{Preference: "protocol", AllowAPIFallback: true}
updated, err := service.Save(ctx, pool.ID, groups.Input{Name: "Synthetic failover pool", Enabled: true, AccountIDs: []string{account.ID}, Routing: &off})
if err != nil || updated.Routing.QuotaFailover {
t.Fatalf("quota failover could not be turned off: %+v, %v", updated.Routing, err)
}
}
4 changes: 2 additions & 2 deletions internal/groups/resources.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ type Routing struct {
AllowAPIFallback bool `json:"allow_api_fallback"`
// SessionSticky derives conversation keys for sessionless clients and lets subagents inherit their parent's resource.
SessionSticky bool `json:"session_sticky"`
// QuotaFailover lets an existing conversation leave a quota-exhausted account; encrypted state
// from the previous account is withheld from the new one.
// QuotaFailover (default on) lets an existing conversation leave a quota-exhausted account;
// encrypted state from the previous account is withheld from the new one.
QuotaFailover bool `json:"quota_failover"`
}

Expand Down
5 changes: 3 additions & 2 deletions internal/storage/db/groups.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions internal/storage/migrations/036_quota_failover_default.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
-- Quota failover becomes the default: existing groups opt in; new groups are created with it (CreateGroup).
UPDATE account_groups SET quota_failover = 1;
5 changes: 3 additions & 2 deletions internal/storage/queries/groups.sql
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ SELECT * FROM account_groups WHERE id=sqlc.arg(id) AND tenant_id=sqlc.arg(tenant
SELECT count(*) FROM account_groups WHERE tenant_id=sqlc.arg(tenant_id) AND deleted_at IS NULL;

-- name: CreateGroup :execlastid
INSERT INTO account_groups(tenant_id,name,enabled,created_at,updated_at)
VALUES(sqlc.arg(tenant_id),sqlc.arg(name),sqlc.arg(enabled),sqlc.arg(now),sqlc.arg(now));
-- Quota failover is on by default; SetGroupRouting applies an explicit choice.
INSERT INTO account_groups(tenant_id,name,enabled,created_at,updated_at,quota_failover)
VALUES(sqlc.arg(tenant_id),sqlc.arg(name),sqlc.arg(enabled),sqlc.arg(now),sqlc.arg(now),1);

-- name: FindGroupName :one
SELECT id FROM account_groups WHERE tenant_id=sqlc.arg(tenant_id) AND name=sqlc.arg(name) COLLATE NOCASE AND deleted_at IS NULL;
Expand Down
2 changes: 1 addition & 1 deletion web/src/components/GroupEditor.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ function GroupForm({
)
const [sticky, setSticky] = useState(value?.routing.session_sticky ?? false)
const [failover, setFailover] = useState(
value?.routing.quota_failover ?? false,
value?.routing.quota_failover ?? true,
)
const [restricted, setRestricted] = useState(
value?.restricted_models ?? false,
Expand Down
4 changes: 2 additions & 2 deletions web/src/lib/groups.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ const groupSchema = z.object({
preference: z.enum(['protocol', 'subscription_first', 'api_first']),
allow_api_fallback: z.boolean(),
session_sticky: z.boolean().optional().default(false),
quota_failover: z.boolean().optional().default(false),
quota_failover: z.boolean().optional().default(true),
})
.default({
preference: 'protocol',
allow_api_fallback: true,
session_sticky: false,
quota_failover: false,
quota_failover: true,
}),
id: z.number().int().positive(),
name: z.string().min(1),
Expand Down
4 changes: 2 additions & 2 deletions web/src/pages/Groups.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ it('selects subscriptions and channels separately with API fallback off by defau
preference: 'subscription_first',
allow_api_fallback: false,
session_sticky: false,
quota_failover: false,
quota_failover: true,
})
return Promise.resolve(
response({
Expand Down Expand Up @@ -130,7 +130,7 @@ it('creates a pool with only selected accounts', async () => {
preference: 'subscription_first',
allow_api_fallback: false,
session_sticky: false,
quota_failover: false,
quota_failover: true,
},
model_policy: { restricted: false, models: [] },
})
Expand Down