diff --git a/PRODUCT.md b/PRODUCT.md index 7780ac67..1ca294d4 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -105,7 +105,7 @@ Gateway uploads renew a 30-second idle read deadline when bytes arrive, with a f Gateway admission allows model changes with reasoning, compaction and reconstructed WebSocket history when the current bound account remains usable and supports the requested model. WebSocket continuation identity survives normalization so rebuilt transcripts cannot authorize account migration. Complete portable history may still replace an unsupported binding; a durable rebind marker protects older model-scoped state after recorded account moves, legacy-scope adoption and restart. Additive migration 019 preserves existing bindings and treats their current account as the continuity baseline; earlier account moves cannot be reconstructed. Incompatible model continuations return HTTP 409 with `conversation_model_conflict`, distinct from an unavailable account binding, and request history and every supported language provide model-specific recovery guidance. Live provider and desktop acceptance remain separate. -Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation already bound under its raw `metadata.user_id` key before opt-in keeps that binding, and only unbound conversations adopt the normalized key. Explicit sessions keep the existing strict binding rules. +Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation keeps whichever raw or normalized key already holds its binding whether the setting is on or off; the setting only decides whether an unbound conversation creates a normalized binding. Explicit sessions keep the existing strict binding rules. API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history; a key change resets verification and invalidates the model catalog in the same transaction. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused. @@ -113,7 +113,7 @@ Network proxies are shown as compact cards in a responsive grid whose column cou Workspace administrators can delete a resource allowance once no admitted request still holds one of its reservations. Deletion removes the allowance, its revisions, price snapshots, key bindings and allowance ledger in one audited transaction; request history and usage statistics are retained. Keys that were bound to it keep their resource group without a budget limit, and the group's resources are no longer locked by the allowance. -OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens; manual settlement of a pending request accepts the same optional image input count. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified. +OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens; manual settlement of a pending request accepts the same optional image input count, stores it with the entry, and treats only an identical breakdown as a retry; a different split is a conflict. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified. OpenCode Go API channels whose base URL is the official `https://opencode.ai` origin show upstream spending windows from the key-authenticated `GET https://opencode.ai/zen/go/v1/usage` endpoint; relay URLs using the preset never send their key there. The windows are the rolling five-hour, weekly and monthly percentages with their reset times. Reads share the quota cache, refresh bounds and failure states of subscription usage. These observations are informational and do not gate channel scheduling. diff --git a/internal/allocations/allocations_test.go b/internal/allocations/allocations_test.go index a417bd52..178d4011 100644 --- a/internal/allocations/allocations_test.go +++ b/internal/allocations/allocations_test.go @@ -1826,6 +1826,17 @@ func TestManualSettlementChargesImageInputAtImageRate(t *testing.T) { if cost != 35_825_000 { t.Fatalf("manual settlement ignored the image input split: %d", cost) } + // An identical retry succeeds; the same totals with another image split are a conflict. + if err := s.Settle(ctx, 9, r.ID, Completion{Input: 1_000_000, ImageInput: 400_000, Cached: 100_000, Output: 1_000_000}); err != nil { + t.Fatal("identical settlement retry failed", err) + } + if err := s.Settle(ctx, 9, r.ID, Completion{Input: 1_000_000, ImageInput: 600_000, Cached: 100_000, Output: 1_000_000}); !errors.Is(err, ErrSettlement) { + t.Fatal("a different image split was treated as a retry", err) + } + var stored, image int64 + if err := conn.QueryRow(`SELECT cost, image_input_tokens FROM allocation_entries WHERE request_id=?`, r.ID).Scan(&stored, &image); err != nil || stored != 35_825_000 || image != 400_000 { + t.Fatal("settlement details changed", stored, image, err) + } overlap := r overlap.ID = "synthetic-image-overlap" if err := Begin(ctx, q, overlap, now.Unix(), time.UTC); err != nil { diff --git a/internal/allocations/runtime.go b/internal/allocations/runtime.go index eefb1bdd..285825c4 100644 --- a/internal/allocations/runtime.go +++ b/internal/allocations/runtime.go @@ -211,7 +211,8 @@ func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int return nil } if manual { - if e.State == "settled" && e.Manual == 1 && e.InputTokens == c.Input && e.OutputTokens == c.Output && e.CachedTokens == c.Cached { + // Only an identical breakdown is a retry; a different image split would have charged differently. + if e.State == "settled" && e.Manual == 1 && e.InputTokens == c.Input && e.OutputTokens == c.Output && e.CachedTokens == c.Cached && e.ImageInputTokens == c.ImageInput { return nil } if e.State != "pending" || c.Input < e.InputTokens || c.Output < e.OutputTokens { @@ -248,7 +249,7 @@ func Finish(ctx context.Context, q *db.Queries, id string, c Completion, now int // Partial observations remain available for correction but are not a charge. cost = 0 } - return q.FinishAllocationEntry(ctx, db.FinishAllocationEntryParams{RequestID: id, FinishedAt: now, State: state, InputTokens: c.Input, OutputTokens: c.Output, CachedTokens: c.Cached, Cost: cost, Manual: bit(manual)}) + return q.FinishAllocationEntry(ctx, db.FinishAllocationEntryParams{RequestID: id, FinishedAt: now, State: state, InputTokens: c.Input, OutputTokens: c.Output, CachedTokens: c.Cached, ImageInputTokens: c.ImageInput, Cost: cost, Manual: bit(manual)}) } func (s *Service) Settle(ctx context.Context, scheme int64, id string, c Completion) error { tx, err := s.conn.BeginTx(ctx, nil) diff --git a/internal/backup/backup_test.go b/internal/backup/backup_test.go index 22cef3dd..7432a3ad 100644 --- a/internal/backup/backup_test.go +++ b/internal/backup/backup_test.go @@ -213,6 +213,7 @@ ALTER TABLE accounts DROP COLUMN usage_revision; ALTER TABLE account_affinity DROP COLUMN rebound; ALTER TABLE account_affinity DROP COLUMN model; DROP INDEX account_groups_active_name; +ALTER TABLE allocation_entries DROP COLUMN image_input_tokens; ALTER TABLE account_affinity DROP COLUMN derived; ALTER TABLE account_groups DROP COLUMN session_sticky; ALTER TABLE account_groups DROP COLUMN deleted_at; diff --git a/internal/gateway/selection.go b/internal/gateway/selection.go index 8a8b40c2..86643522 100644 --- a/internal/gateway/selection.go +++ b/internal/gateway/selection.go @@ -89,20 +89,43 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i return "", digest, err } derived := false - if hints, ok := stickyFrom(ctx); ok && policy.Routing.SessionSticky { - normalize := hints.Session != "" && (session == "" || hints.Replace) - if normalize && session != "" { - // A conversation bound under its raw metadata key before opt-in keeps that binding: - // provider-held state must continue on its original account. - rawScope := provider - if rawScope == "" { - rawScope = "auto" + if hints, ok := stickyFrom(ctx); ok { + sticky := policy.Routing.SessionSticky + bindingScope := provider + if bindingScope == "" { + bindingScope = "auto" + } + // bound reports whether a key already owns a binding; ambiguous legacy bindings count as bound. + bound := func(hash [32]byte) (bool, error) { + binding, err := sessionBinding(ctx, q, db.ListSessionAffinitiesParams{GroupID: groupID, UserID: userID, SessionHash: hash[:], Now: s.now().Unix()}, bindingScope) + if errors.Is(err, ErrAffinityUnavailable) { + return true, nil } - bound, err := sessionBinding(ctx, q, db.ListSessionAffinitiesParams{GroupID: groupID, UserID: userID, SessionHash: digest[:], Now: s.now().Unix()}, rawScope) - if err != nil && !errors.Is(err, ErrAffinityUnavailable) { - return "", digest, err + return binding != nil, err + } + normalize := false + switch { + case hints.Session == "" || session != "" && !hints.Replace: + case hints.Derived: + // Preferences are the opt-in feature itself and disappear with the setting. + normalize = sticky + default: + // Explicit Claude conversations keep whichever key already holds their binding, whether the + // setting is on or off, so provider-held state never moves when the toggle changes. + rawBound := false + if session != "" { + if rawBound, err = bound(digest); err != nil { + return "", digest, err + } + } + if !rawBound { + normalizedBound, err := bound(sessionDigest(userID, groupID, hints.Session)) + if err != nil { + return "", digest, err + } + // The setting only decides whether an unbound conversation creates a normalized binding. + normalize = normalizedBound || sticky } - normalize = bound == nil && err == nil } if normalize { session, derived = hints.Session, hints.Derived @@ -113,7 +136,7 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i } } // Subagents prefer their parent's resource among admitted candidates, as Codex reviews already do. - if parentThread == "" && hints.Parent != "" && hints.Parent != session { + if sticky && parentThread == "" && hints.Parent != "" && hints.Parent != session { parentThread = hints.Parent } } diff --git a/internal/gateway/sticky_test.go b/internal/gateway/sticky_test.go index 234237d0..5c0c7dfc 100644 --- a/internal/gateway/sticky_test.go +++ b/internal/gateway/sticky_test.go @@ -236,3 +236,39 @@ func TestStickinessHonorsBindingsMadeBeforeOptIn(t *testing.T) { } } } + +func TestNormalizedBindingsSurviveOptOut(t *testing.T) { + s, _, _ := stickyGateway(t, true) + ctx := context.Background() + raw := `{"device_id":"synthetic","session_id":"synthetic-opt-out"}` + hints := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-opt-out", Replace: true}) + original, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, "") + if err != nil { + t.Fatal(err) + } + s, _, _ = stickyGatewayFrom(t, s, false) + if _, err := s.accounts.SetEnabled(ctx, original, false); err != nil { + t.Fatal(err) + } + // Turning the setting off must not turn a bound continuation into a new session. + got, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, "") + if err == nil || got != original { + t.Fatalf("normalized binding was dropped after opt-out: %q %v", got, err) + } + if _, err := s.accounts.SetEnabled(ctx, original, true); err != nil { + t.Fatal(err) + } + if got, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, ""); err != nil || got != original { + t.Fatal("continuation left its normalized binding", got, err) + } + // With the setting off, unbound conversations keep using their raw key and create no normalized binding. + fresh := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-unbound", Replace: true}) + if _, _, err := s.selectAllocationAccount(fresh, 1, 1, `{"session_id":"synthetic-unbound"}`, "", "synthetic-model", Responses, 0, pinnedAffinity, ""); err != nil { + t.Fatal(err) + } + normalized := sessionDigest(1, 1, "claude:synthetic-unbound") + var count int + if err := s.db.QueryRow("SELECT count(*) FROM account_affinity WHERE session_hash=?", normalized[:]).Scan(&count); err != nil || count != 0 { + t.Fatal("opt-out created a normalized binding", count, err) + } +} diff --git a/internal/storage/db/allocations.sql.go b/internal/storage/db/allocations.sql.go index e45ebe45..0d810d17 100644 --- a/internal/storage/db/allocations.sql.go +++ b/internal/storage/db/allocations.sql.go @@ -463,18 +463,19 @@ func (q *Queries) DeleteAllocationScheme(ctx context.Context, id int64) (int64, } const finishAllocationEntry = `-- name: FinishAllocationEntry :exec -UPDATE allocation_entries SET finished_at=?,state=?,input_tokens=?,output_tokens=?,cached_tokens=?,cost=?,manual=? WHERE request_id=? +UPDATE allocation_entries SET finished_at=?1,state=?2,input_tokens=?3,output_tokens=?4,cached_tokens=?5,image_input_tokens=?6,cost=?7,manual=?8 WHERE request_id=?9 ` type FinishAllocationEntryParams struct { - FinishedAt int64 - State string - InputTokens int64 - OutputTokens int64 - CachedTokens int64 - Cost int64 - Manual int64 - RequestID string + FinishedAt int64 + State string + InputTokens int64 + OutputTokens int64 + CachedTokens int64 + ImageInputTokens int64 + Cost int64 + Manual int64 + RequestID string } func (q *Queries) FinishAllocationEntry(ctx context.Context, arg FinishAllocationEntryParams) error { @@ -484,6 +485,7 @@ func (q *Queries) FinishAllocationEntry(ctx context.Context, arg FinishAllocatio arg.InputTokens, arg.OutputTokens, arg.CachedTokens, + arg.ImageInputTokens, arg.Cost, arg.Manual, arg.RequestID, @@ -492,7 +494,7 @@ func (q *Queries) FinishAllocationEntry(ctx context.Context, arg FinishAllocatio } const getAllocationEntry = `-- name: GetAllocationEntry :one -SELECT request_id, scheme_id, revision_id, user_id, account_id, model, mode, window_start, reset_at, started_at, finished_at, state, input_tokens, output_tokens, cached_tokens, cost, manual FROM allocation_entries WHERE request_id=? +SELECT request_id, scheme_id, revision_id, user_id, account_id, model, mode, window_start, reset_at, started_at, finished_at, state, input_tokens, output_tokens, cached_tokens, cost, manual, image_input_tokens FROM allocation_entries WHERE request_id=? ` func (q *Queries) GetAllocationEntry(ctx context.Context, requestID string) (AllocationEntry, error) { @@ -516,6 +518,7 @@ func (q *Queries) GetAllocationEntry(ctx context.Context, requestID string) (All &i.CachedTokens, &i.Cost, &i.Manual, + &i.ImageInputTokens, ) return i, err } @@ -621,7 +624,7 @@ func (q *Queries) GetTenantAllocationScheme(ctx context.Context, arg GetTenantAl } const listAllocationPending = `-- name: ListAllocationPending :many -SELECT request_id, scheme_id, revision_id, user_id, account_id, model, mode, window_start, reset_at, started_at, finished_at, state, input_tokens, output_tokens, cached_tokens, cost, manual FROM allocation_entries WHERE scheme_id=?1 AND (?2=0 OR user_id=?2) AND state='pending' ORDER BY started_at DESC,request_id DESC LIMIT 256 +SELECT request_id, scheme_id, revision_id, user_id, account_id, model, mode, window_start, reset_at, started_at, finished_at, state, input_tokens, output_tokens, cached_tokens, cost, manual, image_input_tokens FROM allocation_entries WHERE scheme_id=?1 AND (?2=0 OR user_id=?2) AND state='pending' ORDER BY started_at DESC,request_id DESC LIMIT 256 ` type ListAllocationPendingParams struct { @@ -656,6 +659,7 @@ func (q *Queries) ListAllocationPending(ctx context.Context, arg ListAllocationP &i.CachedTokens, &i.Cost, &i.Manual, + &i.ImageInputTokens, ); err != nil { return nil, err } diff --git a/internal/storage/db/models.go b/internal/storage/db/models.go index e834c59f..08723509 100644 --- a/internal/storage/db/models.go +++ b/internal/storage/db/models.go @@ -60,23 +60,24 @@ type AlertState struct { } type AllocationEntry struct { - RequestID string - SchemeID int64 - RevisionID int64 - UserID int64 - AccountID string - Model string - Mode string - WindowStart int64 - ResetAt int64 - StartedAt int64 - FinishedAt int64 - State string - InputTokens int64 - OutputTokens int64 - CachedTokens int64 - Cost int64 - Manual int64 + RequestID string + SchemeID int64 + RevisionID int64 + UserID int64 + AccountID string + Model string + Mode string + WindowStart int64 + ResetAt int64 + StartedAt int64 + FinishedAt int64 + State string + InputTokens int64 + OutputTokens int64 + CachedTokens int64 + Cost int64 + Manual int64 + ImageInputTokens int64 } type AllocationRevision struct { diff --git a/internal/storage/migrations/034_allocation_image_input.sql b/internal/storage/migrations/034_allocation_image_input.sql new file mode 100644 index 00000000..88428437 --- /dev/null +++ b/internal/storage/migrations/034_allocation_image_input.sql @@ -0,0 +1,3 @@ +-- Settled allowance entries keep the image share of input so manual retries compare the exact +-- breakdown that determined the charge. +ALTER TABLE allocation_entries ADD COLUMN image_input_tokens INTEGER NOT NULL DEFAULT 0 CHECK(image_input_tokens>=0); diff --git a/internal/storage/queries/allocations.sql b/internal/storage/queries/allocations.sql index 3cb14bfc..2f380b6d 100644 --- a/internal/storage/queries/allocations.sql +++ b/internal/storage/queries/allocations.sql @@ -73,7 +73,7 @@ SELECT * FROM allocation_entries WHERE request_id=?; -- name: AllocationHasEntries :one SELECT EXISTS(SELECT 1 FROM allocation_entries WHERE scheme_id=?); -- name: FinishAllocationEntry :exec -UPDATE allocation_entries SET finished_at=?,state=?,input_tokens=?,output_tokens=?,cached_tokens=?,cost=?,manual=? WHERE request_id=?; +UPDATE allocation_entries SET finished_at=sqlc.arg(finished_at),state=sqlc.arg(state),input_tokens=sqlc.arg(input_tokens),output_tokens=sqlc.arg(output_tokens),cached_tokens=sqlc.arg(cached_tokens),image_input_tokens=sqlc.arg(image_input_tokens),cost=sqlc.arg(cost),manual=sqlc.arg(manual) WHERE request_id=sqlc.arg(request_id); -- name: RecoverAllocationEntries :exec UPDATE allocation_entries SET state='pending' WHERE state='active' AND EXISTS( SELECT 1 FROM allocation_schemes s JOIN account_groups g ON g.id=s.group_id