Support single instance image by using local key set & sqlite

[Copilot]

Summary

Adds support for running the LLM Gateway as a single-instance deployment without PostgreSQL or Redis dependencies. When DATABASE_URL starts with sqlite:, the gateway automatically uses SQLite for persistence and an in-memory cache for key/route lookups.

Changes

New files

• src/db.rs — DbPool enum wrapping either PgPool or SqlitePool, with db_query_as!, db_execute!, and db_query_scalar! macros for database-agnostic query dispatch
• src/cache.rs — Cache enum supporting both Redis (multi-instance) and in-memory (single-instance) key/route caching
• migrations_sqlite/ — SQLite-compatible migration files (4 files mirroring the PostgreSQL migrations)

Modified files

• Cargo.toml — Added sqlite feature to sqlx
• src/config.rs — Made redis_url optional; auto-detect SQLite mode from DATABASE_URL
• src/state.rs — Uses DbPool + Cache instead of PgPool + ConnectionManager
• src/main.rs — Dual-mode initialization: PG+Redis or SQLite+InMemory based on DATABASE_URL
• src/services/*.rs — All services updated to use portable SQL and DbPool/Cache abstractions
• src/middleware/auth.rs — Uses Cache instead of Redis directly
• src/routes/admin.rs — Uses Cache instead of Redis directly
• src/routes/proxy.rs — Uses Cache instead of Redis directly
• Dockerfile — Copies migrations_sqlite/ into the build
• .env.example — Documents single-instance SQLite configuration
• README.md — Documents single-instance mode, updated architecture diagram

Usage

Single-instance mode (no external dependencies)

export DATABASE_URL="sqlite:llm_gateway.db?mode=rwc"
export ADMIN_KEY="my-secret"
cargo run

Full deployment (PostgreSQL + Redis)

export DATABASE_URL="postgres://postgres:postgres@localhost:5432/llm_gateway"
export REDIS_URL="redis://127.0.0.1:6379"
export ADMIN_KEY="my-secret"
cargo run

Design

• Database abstraction: DbPool enum + macros dispatch queries to either PgPool or SqlitePool. SQL uses $1/$2 parameter syntax which sqlx supports for both databases.
• Cache abstraction: Cache enum wraps either Redis ConnectionManager or an in-memory HashMap-based store with RwLock for thread safety.
• Portable SQL: Most queries work identically on both databases. Dashboard hourly bucketing uses date_trunc (PG) vs strftime (SQLite). NOW() replaced with bind parameters for portability.
• SQLite optimizations: WAL journal mode and foreign keys enabled on startup.

[Copilot]

Pull request overview

This PR adds a “single-instance” deployment mode that runs the gateway with SQLite for persistence and an in-memory cache for key/model route lookups, while preserving the existing PostgreSQL + Redis mode.

Changes:

• Introduces DbPool + query-dispatch macros to support PostgreSQL and SQLite with shared service logic.
• Introduces Cache abstraction to support Redis or in-memory caching, and updates services/middleware/routes to use it.
• Adds SQLite migrations and updates startup/config/docs to auto-select the correct DB/cache mode.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/state.rs	Switches shared state from PgPool/Redis manager to DbPool/Cache.
src/main.rs	Initializes Pg vs SQLite pool, runs correct migrations, selects Redis vs in-memory cache, warms caches.
src/config.rs	Makes redis_url optional; adds SQLite auto-detection helper.
src/db.rs	Adds DbPool enum and query/execute/scalar macros for DB-agnostic dispatch.
src/cache.rs	Adds Cache enum with Redis + in-memory implementations for SET/HASH ops used by services.
src/services/provider_service.rs	Updates provider CRUD to use DbPool and portable timestamp binds.
src/services/model_service.rs	Updates model CRUD + route resolution to use DbPool + Cache instead of Redis directly.
src/services/key_service.rs	Updates key CRUD/validation to use DbPool + Cache instead of Redis directly.
src/services/log_service.rs	Ports logging + dashboard queries to DbPool, adds SQLite-friendly SQL variants.
src/middleware/auth.rs	Switches user-key auth middleware to validate via Cache + DbPool.
src/routes/admin.rs	Updates admin handlers to use Cache and refresh caches after provider/model changes.
src/routes/proxy.rs	Updates proxy routing to resolve model routes via Cache.
migrations_sqlite/001_init.sql	Adds initial SQLite schema for keys/providers/models.
migrations_sqlite/002_request_logs.sql	Adds SQLite request log table + indexes.
migrations_sqlite/003_token_budget.sql	Adds SQLite token budget columns.
migrations_sqlite/004_token_coefficients.sql	Adds SQLite token coefficient columns.
Cargo.toml	Enables sqlx SQLite feature.
Cargo.lock	Adds transitive deps needed for SQLite builds (e.g., cc).
Dockerfile	Copies SQLite migrations into the image build context.
.env.example	Documents SQLite single-instance configuration.
README.md	Documents single-instance mode and updates architecture/usage docs.

Comments suppressed due to low confidence (1)

src/services/log_service.rs:145

• where_clause is reused for both count_query and data_query, but it hard-codes filter placeholders as $3/$4 to account for LIMIT $1 OFFSET $2. That makes count_query invalid (it has no $1/$2 binds), so filtered log listing will error at runtime. Consider building separate WHERE clauses (one indexed from $1 for the count query and one from $3 for the data query), or switch to sqlx::QueryBuilder to avoid manual placeholder indexing entirely.

    // Build dynamic WHERE clauses
    let mut conditions: Vec<String> = vec![];
    if params.key_id.is_some() {
        conditions.push("r.user_key_id = $3".to_string());
    }
    if params.model.is_some() {
        let idx = if params.key_id.is_some() { 4 } else { 3 };
        conditions.push(format!("r.model_requested = ${idx}"));
    }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

SQLite PRAGMAs are executed once against the pool, but settings like PRAGMA foreign_keys = ON are per-connection in SQLite. With max_connections(5), some pooled connections may still have foreign keys disabled, leading to inconsistent referential integrity. Prefer configuring these via SqliteConnectOptions / SqlitePoolOptions::connect_with (or after_connect) so every new connection enables foreign_keys (and WAL/busy_timeout if needed).