Skip to content

feat(ai-gateway): add project usage governance - #543

Open
dviejokfs wants to merge 2 commits into
mainfrom
feat/ai-gateway-project-governance
Open

feat(ai-gateway): add project usage governance#543
dviejokfs wants to merge 2 commits into
mainfrom
feat/ai-gateway-project-governance

Conversation

@dviejokfs

@dviejokfs dviejokfs commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • attribute deployment-token AI usage to project, environment, deployment, and token IDs
  • expose project/environment/token filters across AI usage APIs
  • add operator-only instance, project, environment, and token governance policies for model allowlists, RPM, and monthly cost
  • enforce limits across console nodes with PostgreSQL advisory locks, shared rate events, and atomic cost reservations
  • convert unresolved streamed requests into durable conservative debits while releasing explicit upstream failures
  • keep multimodal, streaming, and month-boundary accounting fail-closed, including conflicting OpenAI output-cap aliases

Policies apply only from trusted deployment-token attribution. BYOK requests still obey allowlists and RPM limits but do not consume operator-funded monthly budgets. Internal control-plane AI calls remain outside this deployment-token HTTP boundary.

Closes #376

Evidence

AI governance lifecycle and concurrency (real PostgreSQL/TimescaleDB)

cargo test -p temps-ai-gateway --test governance_integration -- --nocapture
cargo test: 4 passed (1 suite, 9.78s)

Coverage includes cross-instance token RPM enforcement, concurrent project budget reservation, atomic replacement of a reservation with actual usage, and durable conservative settlement after an unresolved stream exceeds the provider timeout window.

Gateway behavior and regression coverage

cargo test --lib -p temps-ai-gateway
cargo test: 186 passed (1 suite, 0.10s)

Includes trusted scope attribution, scoped usage filtering, token policies, cost conversion, fail-closed remote-image projection for budgeted calls, streaming usage forcing, and conflicting output-limit alias protection.

Migration applied on a fresh local database

GET /api/ai/governance -> HTTP 200

The fresh database reports project_id, environment_id, deployment_id, deployment_token_id, and billing_period on ai_usage_logs, plus the governance rate-event and cost-reservation tables.

Migration rollback (real TimescaleDB)

cargo test -p temps-migrations --test migration_tests test_migration_down -- --exact --nocapture
cargo test: 1 passed, 12 filtered out (1 suite, 10.83s)

Repository gates

cargo check --lib
cargo clippy -p temps-ai-gateway --lib -- -D warnings
cargo test --lib -p temps-entities -p temps-migrations
git diff origin/main...HEAD --check
cargo check: 0 errors
cargo clippy: 0 errors
cargo test: 53 passed (2 suites)
diff check: clean

Rust review: APPROVE. Migration/test review: APPROVE. Security re-review: APPROVE.

Attribute deployment-token usage to projects and environments, and enforce shared model, rate, and monthly cost policies with atomic database-backed reservations.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

📓 Changelog preview

This is what your commits will add to the generated CHANGELOG.md at release time (via git-cliff). Do not edit CHANGELOG.md by hand — it is generated from your Conventional Commit messages.

## [Unreleased]

### Added

- **ai-gateway:** Add project usage governance

### Fixed

- **ai-gateway:** Harden governance lifecycle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI gateway: per-project/per-token rate limiting and cost attribution for deployment-token callers

1 participant