Skip to content

feat: per-account client fingerprints, wider failover and faster CI #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Alumos merged 1 commit into from
Nov 12, 2026
Merged

feat: per-account client fingerprints, wider failover and faster CI #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions .github/workflows/cache.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
name: Build cache

# Actions caches are scoped to the ref that saved them: pull request and tag runs cannot read each
# other's caches, but every ref can read the default branch's. After each merge this workflow
# compiles (without testing) on main, so the next pull request and release tag restore an almost
# current build and recompile only what changed.
on:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: build-cache
cancel-in-progress: true

jobs:
go:
name: Go caches
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
cache: false
# The keys match the CI jobs that restore them.
- name: Restore Go build cache
id: build-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-build-${{ runner.os }}-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
go-build-${{ runner.os }}-${{ hashFiles('go.sum') }}-
go-build-${{ runner.os }}-
- name: Compile packages and tests
run: |
mkdir -p web/dist && touch web/dist/.keep
go build ./...
go vet ./...
- name: Save Go build cache
if: steps.build-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ steps.build-cache.outputs.cache-primary-key }}
- name: Clear the normal build cache
run: go clean -cache
- name: Restore Go race build cache
id: race-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Paths are part of the cache version and must match the CI restore exactly.
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-race-${{ runner.os }}-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
go-race-${{ runner.os }}-${{ hashFiles('go.sum') }}-
go-race-${{ runner.os }}-
- name: Compile race-instrumented tests
# -run '^$' builds and links every test binary without running a test.
run: go test -race -run '^$' ./... > /dev/null
- name: Save Go race build cache
if: steps.race-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ steps.race-cache.outputs.cache-primary-key }}

containers:
name: Container caches (${{ matrix.arch }})
runs-on: ${{ matrix.runner }}
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
runner: ubuntu-24.04
- arch: arm64
runner: ubuntu-24.04-arm
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
id: buildx
- name: Cache BuildKit mounts
id: mount-cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: buildkit-mounts
key: buildkit-mounts-${{ matrix.arch }}-${{ hashFiles('go.sum', 'web/pnpm-lock.yaml') }}-${{ hashFiles('**/*.go', 'web/src/**') }}
restore-keys: |
buildkit-mounts-${{ matrix.arch }}-${{ hashFiles('go.sum', 'web/pnpm-lock.yaml') }}-
buildkit-mounts-${{ matrix.arch }}-
- name: Inject BuildKit mounts
uses: reproducible-containers/buildkit-cache-dance@5422eac04292c961a382e0f584ea0f03ad9da723 # v3.4.0
with:
builder: ${{ steps.buildx.outputs.name }}
dockerfile: Dockerfile
cache-dir: buildkit-mounts
skip-extraction: ${{ steps.mount-cache.outputs.cache-hit }}
- name: Build image layers
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
platforms: linux/${{ matrix.arch }}
push: false
build-args: |
VERSION=0.1.0-dev
REVISION=${{ github.sha }}
SOURCE_URL=${{ github.server_url }}/${{ github.repository }}
cache-from: type=gha,scope=container-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=container-${{ matrix.arch }}
provenance: false
74 changes: 70 additions & 4 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,20 @@ jobs:
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
cache: false
# Build outputs are keyed by source, so each commit restores the closest earlier build and
# recompiles only what changed. Only the build part saves, so the cache holds a full build.
- name: Restore Go build cache
id: go-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-build-${{ runner.os }}-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
go-build-${{ runner.os }}-${{ hashFiles('go.sum') }}-
go-build-${{ runner.os }}-
- uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
with:
version: 9.12.2
Expand Down Expand Up @@ -68,6 +82,14 @@ jobs:
- name: Verify key migration with encrypted credentials
if: matrix.part == 'build'
run: SUBLANE_TEST_MIGRATION_BINARY="$PWD/bin/sublane" node --test --test-name-pattern='real CLI' scripts/separate-key.test.mjs
- name: Save Go build cache
if: matrix.part == 'build' && steps.go-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ steps.go-cache.outputs.cache-primary-key }}
- name: Validate workflow and shell scripts
if: matrix.part == 'lint'
run: |
Expand All @@ -86,41 +108,65 @@ jobs:
strategy:
fail-fast: false
matrix:
group: [gateway-server-1, gateway-server-2, gateway-server-3, gateway-server-4, gateway-server-5, other-1, other-2, other-3]
group: [gateway-server-1, gateway-server-2, gateway-server-3, gateway-server-4, gateway-server-5, other-1, other-2, other-3, other-4]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
cache: false
# Race-instrumented objects differ from normal builds, so they keep their own cache. The first
# shard compiles the gateway and server with every dependency and saves for the others.
- name: Restore Go race build cache
id: go-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: go-race-${{ runner.os }}-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
go-race-${{ runner.os }}-${{ hashFiles('go.sum') }}-
go-race-${{ runner.os }}-
- name: Test with race detector
shell: bash
env:
TEST_GROUP: ${{ matrix.group }}
run: |
# Shards partition by test name and package; each final shard is a complement so
# examples, fuzz seeds, future test names and new packages are always tested.
heavy_a='/internal/(allocations|accounts)$'
heavy_a='/internal/allocations$'
heavy_b='/internal/(backup|auth|alerts)$'
heavy_c='/internal/accounts$'
case "$TEST_GROUP" in
gateway-server-1) go test -race -timeout=20m -run '^Test[A-B]' ./internal/gateway ./internal/server ;;
gateway-server-2) go test -race -timeout=20m -run '^Test[C-G]' ./internal/gateway ./internal/server ;;
gateway-server-3) go test -race -timeout=20m -run '^Test[H-N]' ./internal/gateway ./internal/server ;;
gateway-server-4) go test -race -timeout=20m -run '^Test[O-R]' ./internal/gateway ./internal/server ;;
gateway-server-5) go test -race -timeout=20m -skip '^Test[A-R]' ./internal/gateway ./internal/server ;;
other-1|other-2|other-3)
other-1|other-2|other-3|other-4)
all=$(go list ./... | grep -vE '/internal/(gateway|server)$')
case "$TEST_GROUP" in
other-1) package_list=$(grep -E "$heavy_a" <<< "$all") ;;
other-2) package_list=$(grep -E "$heavy_b" <<< "$all") ;;
other-3) package_list=$(grep -vE "$heavy_a|$heavy_b" <<< "$all") ;;
other-3) package_list=$(grep -vE "$heavy_a|$heavy_b|$heavy_c" <<< "$all") ;;
other-4) package_list=$(grep -E "$heavy_c" <<< "$all") ;;
esac
mapfile -t packages <<< "$package_list"
go test -race -timeout=20m "${packages[@]}"
;;
*) echo "unknown test group: $TEST_GROUP" >&2; exit 1 ;;
esac
- name: Save Go race build cache
if: matrix.group == 'gateway-server-1' && steps.go-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ steps.go-cache.outputs.cache-primary-key }}

containers:
name: Container (${{ matrix.arch }})
Expand All @@ -145,6 +191,26 @@ jobs:
with:
node-version-file: .node-version
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
id: buildx
# Layer caching cannot keep RUN --mount=type=cache directories, so every image rebuilt Go and
# frontend dependencies from scratch. Persist those mounts per architecture and source state.
- name: Cache BuildKit mounts
id: mount-cache
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: buildkit-mounts
key: buildkit-mounts-${{ matrix.arch }}-${{ hashFiles('go.sum', 'web/pnpm-lock.yaml') }}-${{ hashFiles('**/*.go', 'web/src/**') }}
restore-keys: |
buildkit-mounts-${{ matrix.arch }}-${{ hashFiles('go.sum', 'web/pnpm-lock.yaml') }}-
buildkit-mounts-${{ matrix.arch }}-
- name: Inject BuildKit mounts
uses: reproducible-containers/buildkit-cache-dance@5422eac04292c961a382e0f584ea0f03ad9da723 # v3.4.0
with:
builder: ${{ steps.buildx.outputs.name }}
# The mounts are discovered from the Dockerfile, so new cache mounts are persisted too.
dockerfile: Dockerfile
cache-dir: buildkit-mounts
skip-extraction: ${{ steps.mount-cache.outputs.cache-hit }}
- name: Build image for the native runner
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
Expand Down
6 changes: 5 additions & 1 deletion PRODUCT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,11 @@ Gateway admission allows model changes with reasoning, compaction and reconstruc

Resource groups offer optional session stickiness, disabled by default. When enabled, requests that carry no gateway session derive a conversation key from client conversation headers, Codex turn metadata, Claude Code session metadata or, as a fallback, a hash of the system prompt and first user message, following CLIProxyAPI's session-affinity order. Derived keys create one-hour preferences in their own namespace, so they never read or rewrite an explicit binding: the preferred resource ranks first only after every policy, quota, allowance, cooldown and capacity check, and unavailable resources fall back to normal scheduling. Preferences may hold at most a quarter of the shared affinity table, are skipped rather than rejected when it is full, and are evicted first when an explicit conversation needs a slot. Subagents identified by Codex parent threads, Claude parent sessions or `X-Parent-Session-Id` prefer their parent's resource through read-only lookups of the parent's explicit binding or preference. Claude Code subagents share their main session key; a conversation keeps whichever raw or normalized key already holds its binding whether the setting is on or off; the setting only decides whether an unbound conversation creates a normalized binding. Explicit sessions keep the existing strict binding rules.

Resource groups also offer quota failover, enabled by default for new and existing groups and switchable per group. When enabled, an existing Responses conversation whose bound account has exhausted its quota moves to another admitted account of the same provider instead of failing. The binding records 16-byte fingerprints (at most 4,096) of every encrypted reasoning or compaction item present at the move, and those items are withheld from the new account on that and every later turn, so only readable transcript and the new account's own state reach it. Requests that reference provider-held state (`previous_response_id`, `conversation`, `container` or `item_reference`) never move, and disablement, cooldown, saturation or model mismatch still never move a conversation.
Resource groups also offer account failover, enabled by default for new and existing groups and switchable per group. When enabled, an existing Responses conversation moves to another admitted account of the same provider instead of failing when its bound account has exhausted its quota or allowance, is cooling down for five minutes or more, is disabled, needs reauthorization, was deleted or left the pool; if no account can take it, the original account's error is returned. The binding records 16-byte fingerprints (at most 4,096) of every encrypted reasoning or compaction item present at the move, and those items are withheld from the new account on that and every later turn, so only readable transcript and the new account's own state reach it. Requests that reference provider-held state (`previous_response_id`, `conversation`, `container` or `item_reference`) never move, and saturation, short cooldowns or model mismatch never move a conversation. Groups with failover off keep the strict rule that conversations never switch accounts.

Each upstream account presents one client fingerprint, following CLIProxyAPI. Claude accounts keep their durable device ID and pin one device profile (User-Agent and X-Stainless OS, architecture, runtime and package versions) per account. Codex accounts pin the platform and terminal of the first native Codex client that uses them (stored on the account) and present it with SubLane's managed Codex version to every later client, whatever machine or tool it runs on; accounts never seen by a native client keep the bare version agent. Different accounts may present different fingerprints.

Provider requests whose connection fails before any byte is sent (DNS, TCP dial or proxy CONNECT) are retried up to twice after 250 ms and 1 s on the same account; failures after a request may have reached the provider are never replayed.

API channels can be renamed and given a replacement key in place at `/channels`; leaving the key blank keeps the stored secret. Base URL, platform and protocol remain fixed so a stored key never reaches another upstream; those changes still require a separate channel. Edits retain the channel ID, group assignments, limits and request history; a key change resets verification and invalidates the model catalog in the same transaction. Resource groups can be deleted when no active personal key or resource allowance references them. Deletion retires the group: routing, grants, resources, model policy and bindings are removed in one audited transaction, while request history and usage aggregates keep their workspace scope. Retired groups keep their names; uniqueness applies only to active groups, so the name can be reused.

Expand Down
28 changes: 15 additions & 13 deletions internal/accounts/credential.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,21 @@ import (

// Credential is internal secret material. HTTP responses must use Account instead.
type Credential struct {
BaseURL string `json:"base_url,omitempty"`
APIPlatform string `json:"api_platform,omitempty"`
APIProtocol string `json:"api_protocol,omitempty"`
ProxyURL string `json:"-"`
Provider string `json:"provider,omitempty"`
Metadata map[string]json.RawMessage `json:"metadata,omitempty"`
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
IDToken string `json:"id_token,omitempty"`
AccountID string `json:"account_id"`
Email string `json:"email,omitempty"`
Plan string `json:"plan,omitempty"`
ExpiresAt int64 `json:"expires_at"`
BaseURL string `json:"base_url,omitempty"`
APIPlatform string `json:"api_platform,omitempty"`
APIProtocol string `json:"api_protocol,omitempty"`
ProxyURL string `json:"-"`
// ClientPlatform is the account's pinned client platform (for example "(Mac OS 15.5.0; arm64) iTerm.app/3.6.1").
ClientPlatform string `json:"-"`
Provider string `json:"provider,omitempty"`
Metadata map[string]json.RawMessage `json:"metadata,omitempty"`
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
IDToken string `json:"id_token,omitempty"`
AccountID string `json:"account_id"`
Email string `json:"email,omitempty"`
Plan string `json:"plan,omitempty"`
ExpiresAt int64 `json:"expires_at"`
// Keep rejection with the encrypted token so a restart cannot make it eligible for fallback.
Rejected bool `json:"access_token_rejected,omitempty"`
}
Expand Down
Loading