Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,15 +105,15 @@ Gateway uploads renew a 30-second idle read deadline when bytes arrive, with a f

Gateway admission allows model changes with reasoning, compaction and reconstructed WebSocket history when the current bound account remains usable and supports the requested model. WebSocket continuation identity survives normalization so rebuilt transcripts cannot authorize account migration. Complete portable history may still replace an unsupported binding; a durable rebind marker protects older model-scoped state after recorded account moves, legacy-scope adoption and restart. Additive migration 019 preserves existing bindings and treats their current account as the continuity baseline; earlier account moves cannot be reconstructed. Incompatible model continuations return HTTP 409 with `conversation_model_conflict`, distinct from an unavailable account binding, and request history and every supported language provide model-specific recovery guidance. Live provider and desktop acceptance remain separate.

Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key. Explicit sessions keep the existing strict binding rules.
Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation already bound under its raw `metadata.user_id` key before opt-in keeps that binding, and only unbound conversations adopt the normalized key. Explicit sessions keep the existing strict binding rules.

API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history, and a key change resets verification. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.
API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history; a key change resets verification and invalidates the model catalog in the same transaction. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.

Network proxies are shown as compact cards in a responsive grid whose column count follows the page width. Each card shows its status and latency, an IP purity panel, and the exit IP, location, network, verdict and bound accounts exactly once, merging the proxy check with the IP report. Opening the page only reads reports the server already holds; an administrator must explicitly request a lookup, which sends the exit IP observed by the last successful check, never the proxy endpoint, to the public ip.net.coffee lookup used by its IP scoring page. The panel shows its trust score with the same five purity tiers, network type, risk flags, ASN, ISP, location and verdict. Only public exit IPs are sent; results are cached per IP for six hours, concurrent lookups for one IP share a single request, attempts are paced to one per 30 seconds per IP from when they start, and rate-limited responses back off for five minutes. The report is informational and does not affect routing or proxy checks.
Network proxies are shown as compact cards in a responsive grid whose column count follows the page width. Each card shows its status and latency, an IP purity panel, and the exit IP, location, network, verdict and bound accounts exactly once, merging the proxy check with the IP report. Opening the page only reads reports the server already holds; an administrator must explicitly request a lookup, which sends the exit IP observed by the last successful check, never the proxy endpoint, to the public ip.net.coffee lookup used by its IP scoring page. The panel shows its trust score with the same five purity tiers, network type, risk flags, ASN, ISP, location and verdict. Only public exit IPs are sent; results are cached per IP for six hours, concurrent lookups for one IP share a single request, attempts are paced to one per 30 seconds per IP from when they start, rate-limited responses back off for five minutes, and a new IP is refused while all 256 cache slots hold in-flight lookups. The report is informational and does not affect routing or proxy checks.

Workspace administrators can delete a resource allowance once no admitted request still holds one of its reservations. Deletion removes the allowance, its revisions, price snapshots, key bindings and allowance ledger in one audited transaction; request history and usage statistics are retained. Keys that were bound to it keep their resource group without a budget limit, and the group's resources are no longer locked by the allowance.

OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.
OpenAI Images requests (`POST /v1/images/generations` and `/v1/images/edits`, JSON or multipart) route only to Codex subscriptions and forward to the ChatGPT backend's direct Images endpoints. Codex catalogs include the built-in `gpt-image-*` models so they appear in model lists, group allowlists and pricing; image models are rejected on text endpoints, and text models on Images endpoints. Streaming partial images is not supported. Request history records the `images` operation with reported token usage. Allowance pricing for image models charges output at the image output rate and settles input by modality, using the image input rate for reported image input tokens; manual settlement of a pending request accepts the same optional image input count. CC Switch picker suggestions omit image models. Live ChatGPT image acceptance remains unverified.

OpenCode Go API channels whose base URL is the official `https://opencode.ai` origin show upstream spending windows from the key-authenticated `GET https://opencode.ai/zen/go/v1/usage` endpoint; relay URLs using the preset never send their key there. The windows are the rolling five-hour, weekly and monthly percentages with their reset times. Reads share the quota cache, refresh bounds and failure states of subscription usage. These observations are informational and do not gate channel scheduling.

Expand Down
4 changes: 4 additions & 0 deletions internal/accounts/apikey.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,10 @@ func (s *Service) UpdateAPIChannel(ctx context.Context, id string, input APIChan
if err := s.persist(ctx, q, id, credential, status); err != nil {
return Account{}, err
}
// A new key may grant a different model set; discovery must not reuse the old snapshot.
if err := q.InvalidateAccountCatalog(ctx, id); err != nil {
return Account{}, err
}
}
now := s.now().Unix()
if name != row.Name {
Expand Down
45 changes: 45 additions & 0 deletions internal/allocations/allocations_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"fmt"
"path/filepath"
"slices"
"strconv"
"strings"
"testing"
"time"
Expand Down Expand Up @@ -1793,3 +1794,47 @@ func TestCostSettlesImageInputByModality(t *testing.T) {
t.Fatal("overlapping categories accepted", err)
}
}

func TestManualSettlementChargesImageInputAtImageRate(t *testing.T) {
s, conn, user, account := fixture(t)
ctx := context.Background()
now := time.Date(2030, 1, 2, 12, 0, 0, 0, time.UTC)
s.now = func() time.Time { return now }
config := `{"mode":"amount","period":"day","members":[{"user_id":` + strconv.FormatInt(user, 10) + `,"limit":1000000000}],"rates":[{"model":"gpt-image-2","input":5000000,"cached":1250000,"output":30000000,"image_input":8000000}]}`
if _, err := conn.Exec(`INSERT INTO allocation_schemes(id,name,group_id,enabled,created_at) VALUES(9,'Synthetic image allowance',2,1,?)`, now.Unix()); err != nil {
t.Fatal(err)
}
if _, err := conn.Exec(`INSERT INTO allocation_revisions(id,scheme_id,effective_at,accounting_anchor_at,config) VALUES(90,9,?,?,?)`, now.Unix()-60, now.Unix()-60, config); err != nil {
t.Fatal(err)
}
q := db.New(conn)
r := Request{ID: "synthetic-image-edit", SchemeID: 9, UserID: user, GroupID: 2, AccountID: account, Model: "gpt-image-2", StartedAt: now.Unix()}
if err := Begin(ctx, q, r, now.Unix(), time.UTC); err != nil {
t.Fatal(err)
}
if err := q.RecoverAllocationEntries(ctx, 1); err != nil {
t.Fatal(err)
}
if err := s.Settle(ctx, 9, r.ID, Completion{Input: 1_000_000, ImageInput: 400_000, Cached: 100_000, Output: 1_000_000}); err != nil {
t.Fatal(err)
}
var cost int64
if err := conn.QueryRow(`SELECT cost FROM allocation_entries WHERE request_id=?`, r.ID).Scan(&cost); err != nil {
t.Fatal(err)
}
// 500k text * $5 + 400k image * $8 + 100k cached * $1.25 + 1M image output * $30, in micro-USD.
if cost != 35_825_000 {
t.Fatalf("manual settlement ignored the image input split: %d", cost)
}
overlap := r
overlap.ID = "synthetic-image-overlap"
if err := Begin(ctx, q, overlap, now.Unix(), time.UTC); err != nil {
t.Fatal(err)
}
if err := q.RecoverAllocationEntries(ctx, 1); err != nil {
t.Fatal(err)
}
if err := s.Settle(ctx, 9, overlap.ID, Completion{Input: 100, ImageInput: 60, Cached: 50}); !errors.Is(err, ErrInput) {
t.Fatal("overlapping image and cached input accepted", err)
}
}
16 changes: 15 additions & 1 deletion internal/gateway/selection.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,21 @@ func (s *Service) selectAllocationAccount(ctx context.Context, userID, groupID i
}
derived := false
if hints, ok := stickyFrom(ctx); ok && policy.Routing.SessionSticky {
if hints.Session != "" && (session == "" || hints.Replace) {
normalize := hints.Session != "" && (session == "" || hints.Replace)
if normalize && session != "" {
// A conversation bound under its raw metadata key before opt-in keeps that binding:
// provider-held state must continue on its original account.
rawScope := provider
if rawScope == "" {
rawScope = "auto"
}
bound, err := sessionBinding(ctx, q, db.ListSessionAffinitiesParams{GroupID: groupID, UserID: userID, SessionHash: digest[:], Now: s.now().Unix()}, rawScope)
if err != nil && !errors.Is(err, ErrAffinityUnavailable) {
return "", digest, err
}
normalize = bound == nil && err == nil
}
if normalize {
session, derived = hints.Session, hints.Derived
digest = sessionDigest(userID, groupID, session)
if derived {
Expand Down
44 changes: 44 additions & 0 deletions internal/gateway/sticky_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@ import (
func stickyGateway(t *testing.T, sticky bool) (*Service, string, string) {
t.Helper()
s, parent, other := reviewGateway(t)
return stickyGatewayFrom(t, s, sticky, parent, other)
}

func stickyGatewayFrom(t *testing.T, s *Service, sticky bool, ids ...string) (*Service, string, string) {
t.Helper()
parent, other := "", ""
if len(ids) == 2 {
parent, other = ids[0], ids[1]
}
detail, err := groups.New(s.db).Get(context.Background(), 1)
if err != nil {
t.Fatal(err)
Expand Down Expand Up @@ -192,3 +201,38 @@ func TestDerivedPreferencesNeverBlockExplicitSessions(t *testing.T) {
t.Fatal("derived request was rejected")
}
}

func TestStickinessHonorsBindingsMadeBeforeOptIn(t *testing.T) {
s, _, _ := stickyGateway(t, false)
ctx := context.Background()
raw := `{"device_id":"synthetic","session_id":"synthetic-claude-session"}`
original, _, err := s.selectAccount(ctx, 1, 1, raw, "", "synthetic-model", Responses)
if err != nil {
t.Fatal(err)
}
s, _, _ = stickyGatewayFrom(t, s, true)
if _, err := s.accounts.SetEnabled(ctx, original, false); err != nil {
t.Fatal(err)
}
hints := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-claude-session", Replace: true})
// The raw metadata binding stays authoritative; provider state must not move to a fallback.
got, _, err := s.selectAllocationAccount(hints, 1, 1, raw, "", "synthetic-model", Responses, 0, pinnedAffinity, "")
if err == nil || got != original {
t.Fatalf("bound conversation moved after opt-in: %q %v", got, err)
}
if _, err := s.accounts.SetEnabled(ctx, original, true); err != nil {
t.Fatal(err)
}
// Unbound conversations adopt the normalized key, shared by the whole agent tree.
fresh := withStickyHints(ctx, stickyHints{Session: "claude:synthetic-new-session", Replace: true})
first, _, err := s.selectAllocationAccount(fresh, 1, 1, `{"session_id":"synthetic-new-session","agent_id":"a"}`, "", "synthetic-model", Responses, 0, pinnedAffinity, "")
if err != nil {
t.Fatal(err)
}
for _, agent := range []string{"b", "c", "d"} {
got, _, err := s.selectAllocationAccount(fresh, 1, 1, `{"session_id":"synthetic-new-session","agent_id":"`+agent+`"}`, "", "synthetic-model", Responses, 0, pinnedAffinity, "")
if err != nil || got != first {
t.Fatal("agent tree did not share the normalized binding", agent, got, err)
}
}
}
10 changes: 8 additions & 2 deletions internal/server/allocations.go
Original file line number Diff line number Diff line change
Expand Up @@ -216,15 +216,21 @@ func (h *allocationHTTP) settle(w http.ResponseWriter, r *http.Request) {
Input *int64 `json:"input"`
Output *int64 `json:"output"`
Cached *int64 `json:"cached"`
// ImageInput is the optional image share of Input for image model requests.
ImageInput *int64 `json:"image_input"`
}
if !decodeJSON(w, r, &in) {
return
}
if in.Input == nil || in.Output == nil || in.Cached == nil {
if in.Input == nil || in.Output == nil || in.Cached == nil || in.ImageInput != nil && *in.ImageInput < 0 {
allocationError(w, allocations.ErrInput)
return
}
err = h.gateway.Allocations().Settle(r.Context(), id, in.RequestID, allocations.Completion{Input: *in.Input, Output: *in.Output, Cached: *in.Cached})
var imageInput int64
if in.ImageInput != nil {
imageInput = *in.ImageInput
}
err = h.gateway.Allocations().Settle(r.Context(), id, in.RequestID, allocations.Completion{Input: *in.Input, ImageInput: imageInput, Output: *in.Output, Cached: *in.Cached})
if err != nil {
allocationError(w, err)
return
Expand Down
15 changes: 15 additions & 0 deletions internal/server/channels_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@ package server

import (
"encoding/json"
"github.com/murongg/SubLane/internal/upstream"
"io"
"net/http"
"strings"
"testing"
"time"

"github.com/murongg/SubLane/internal/groups"
)
Expand Down Expand Up @@ -205,10 +207,23 @@ func TestChannelSettingsCanBeEditedInPlace(t *testing.T) {
t.Fatal("edit removed group membership", value)
}
}
if err := f.accounts.SaveCatalog(t.Context(), channel.ID, 0, []string{"synthetic-old-model"}, time.Now().Unix(), upstream.CatalogSource("openai")); err != nil {
t.Fatal(err)
}
var revision int64
if err := f.connection.QueryRow(`SELECT models_revision FROM accounts WHERE id=?`, channel.ID).Scan(&revision); err != nil {
t.Fatal(err)
}
rotated := request(h, "PUT", "/api/channels/"+channel.ID, origin, map[string]string{"name": "Synthetic renamed", "api_key": "synthetic-next-key", "base_url": "https://one.example.test/v1", "platform": "custom", "protocol": "openai"}, owner)
if rotated.Code != 200 {
t.Fatal(rotated.Code, rotated.Body.String())
}
// A replacement key must not reuse the previous key's model snapshot.
var cleared bool
var next int64
if err := f.connection.QueryRow(`SELECT models_snapshot IS NULL, models_revision FROM accounts WHERE id=?`, channel.ID).Scan(&cleared, &next); err != nil || !cleared || next <= revision {
t.Fatal("key replacement kept the old catalog", cleared, next, revision, err)
}
if stored, err := f.accounts.Prepare(t.Context(), channel.ID, nil); err != nil || stored.AccessToken != "synthetic-next-key" {
t.Fatal("key was not replaced", err)
}
Expand Down
17 changes: 12 additions & 5 deletions internal/server/ipreport.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,11 @@ func (c *ipReportCache) fetch(ctx context.Context, ip string) (upstream.IPReport
now := c.now()
entry := c.entries[ip]
if entry == nil {
c.evictLocked()
if !c.makeRoomLocked() {
// Every slot holds an in-flight lookup; admitting another IP would exceed the bound.
c.mu.Unlock()
return upstream.IPReport{}, ipReportCooldown, errIPReportPaced
}
entry = &ipReportEntry{}
c.entries[ip] = entry
}
Expand Down Expand Up @@ -111,10 +115,11 @@ func (c *ipReportCache) fetch(ctx context.Context, ip string) (upstream.IPReport
return report, 0, nil
}

// makeRoomLocked reports whether a new entry fits, evicting the oldest idle entry when full.
// Called with the lock held; in-flight entries are never evicted.
func (c *ipReportCache) evictLocked() {
func (c *ipReportCache) makeRoomLocked() bool {
if len(c.entries) < ipReportEntries {
return
return true
}
oldest := ""
for key, value := range c.entries {
Expand All @@ -125,9 +130,11 @@ func (c *ipReportCache) evictLocked() {
oldest = key
}
}
if oldest != "" {
delete(c.entries, oldest)
if oldest == "" {
return false
}
delete(c.entries, oldest)
return true
}

func (h *proxyHTTP) exitIP(w http.ResponseWriter, r *http.Request) (string, bool) {
Expand Down
Loading