feat(keys): add granular API key limits with persistence

[mahdiwafy]

Focused revision (split from PR #77 per maintainer feedback).

Scope:

• schema/migration (005-add-api-key-limits)
• repository/API validation (apiKeyUsageRepo, apiKeysRepo)
• enforcement across modalities (chat handlers, /v1/models)
• concurrency semantics + persistence
• tests

This is the persistence-backed API-key limits feature only — no proxy rotation, no token-saver removal, no unrelated UI.

[mahdiwafy]

Verified: npm run build ✅, focused vitest 42 passed ✅. Includes migration 005, repository/API validation, enforcement across chat handlers and /v1/models.

[Vanszs]

Request changes

Cross-check against the latest main found the central limits implementation is not production-safe:

1. Usage is not persistent. apiKeyUsageRepo.js stores counters in global._apiKeyCounters; restart, PM2 workers, containers, and replicas reset or diverge enforcement. The migration persists configuration only.
2. Admission and recording are non-atomic. checkApiKeyLimits() runs before processing, while recordApiKeyUsage() runs later. Concurrent requests can all pass the same limit.
3. Token enforcement uses body.max_tokens, which is an output ceiling, not total prompt+completion usage. When omitted it becomes zero, so token limits are bypassed for large prompts.
4. /v1/models records request usage before the operation succeeds, while chat records after response usage. Define one counting contract and apply it consistently.
5. Invalid input can become unlimited. parseLimitInt() maps malformed values to null; routes do not strictly validate limits or expiresAt. Invalid dates produce NaN and skip expiry enforcement.
6. No changed tests prove persistence across restart, concurrency, migration upgrades, malformed payloads, streaming/non-streaming accounting, or ACL/custom-node preservation.

Please rebase onto current main. Replace the process-local check/record design with atomic persistent admission/reservation (or explicitly narrow the feature and remove persistence claims), define token accounting semantics, reject invalid API input with 400, and add reproducible CI tests. ACL checks appear preserved, but they need regression coverage before merge.

[Copilot]

Pull request overview

Adds persistence-backed configuration for granular per-API-key limits (expiry + request/token caps) and wires enforcement/usage tracking through the SSE chat pipeline, the /v1/models endpoint, and the dashboard key editor.

Changes:

• Extend apiKeys schema + add migration 005 to persist per-key limit fields (expiry, rpm/rph/rpd, token caps).
• Introduce an in-memory per-process counter repo (apiKeyUsageRepo) and thread apiKeyInfo through open-sse handlers to record usage after responses.
• Expose/edit limits via dashboard + keys API routes, and add richer ACL-deny logging.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/sse/services/auth.js	Adds warn-level logging for ACL blocks (providers/combos/kinds).
src/sse/handlers/chat.js	Adds per-key limit check and passes apiKeyInfo into handleChatCore.
src/lib/localDb.js	Re-exports limit/usage helpers from apiKeyUsageRepo.
src/lib/db/schema.js	Bumps schema version and adds new nullable limit columns to apiKeys.
src/lib/db/repos/apiKeyUsageRepo.js	New in-memory counters + limit check / record / snapshot helpers.
src/lib/db/repos/apiKeysRepo.js	Persists/loads new limit fields; adds expiresAt enforcement during validation.
src/lib/db/migrations/index.js	Registers migration 005.
src/lib/db/migrations/005-add-api-key-limits.js	Adds new apiKeys columns via ALTER TABLE when missing.
src/app/api/v1/models/route.js	Applies limit check + request counting to /v1/models when API keys are required.
src/app/api/keys/route.js	Returns per-key usage snapshot; accepts limit fields on key creation.
src/app/api/keys/[id]/route.js	Returns per-key usage snapshot; accepts limit updates on key update.
src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.js	Adds UI state + form fields for editing per-key limits and displaying badges.
open-sse/handlers/chatCore/streamingHandler.js	Threads apiKeyInfo into streaming completion usage tracking.
open-sse/handlers/chatCore/sseToJsonHandler.js	Threads apiKeyInfo into SSE→JSON usage tracking.
open-sse/handlers/chatCore/requestDetail.js	Records per-key usage (via recordApiKeyUsage) when usage tokens are available.
open-sse/handlers/chatCore/nonStreamingHandler.js	Threads apiKeyInfo into non-streaming usage tracking.
open-sse/handlers/chatCore.js	Adds apiKeyInfo to handleChatCore signature and shared ctx.

Suppressed comments (4)

src/lib/db/repos/apiKeyUsageRepo.js:108

• getDateKey() buckets are based on local time, but the RPD retryAfterMs calculation is based on epoch-day modulo (UTC boundary). This can yield a wrong Retry-After (and even negative values around DST/timezone boundaries). Compute retry-after to the next local midnight to match the bucket key.

    const date = getDateKey();
    const current = bumpCounter(counters.rpd, keyId, date, 0);
    if (current + 1 > rpdLimit) {
      return { allowed: false, reason: `Rate limit exceeded: ${rpdLimit} requests per day`, retryAfterMs: 86400000 - (Date.now() % 86400000) };
    }

src/lib/db/repos/apiKeyUsageRepo.js:208

• getApiKeyUsageSnapshot() can return stale counts from a previous minute/hour/day because it reads the raw map entry without checking the current bucket key (unlike checkApiKeyLimits, which resets buckets via bumpCounter(..., 0)). This makes the dashboard usage display misleading and can show non-zero usage even after the window has rolled over.

export function getApiKeyUsageSnapshot(apiKeyInfo) {
  if (!apiKeyInfo) return null;
  const keyId = apiKeyInfo.id;
  return {
    rpm: { limit: apiKeyInfo.rpm, used: (counters.rpm.get(keyId)?.count || 0) },

src/lib/db/repos/apiKeyUsageRepo.js:86

• checkApiKeyLimits() checks current + 1 > limit, but it never reserves the request in the counter. Since recordApiKeyUsage() is only called later (on successful completion via saveUsageStats, or explicitly in /v1/models), concurrent/in-flight requests can all pass this check and then increment later, effectively bypassing RPM/RPH/RPD limits under load.

  const rpmLimit = apiKeyInfo.rpm;
  if (rpmLimit != null) {
    const minute = getMinuteTs();
    const current = bumpCounter(counters.rpm, keyId, minute, 0);
    if (current + 1 > rpmLimit) {

open-sse/handlers/chatCore/requestDetail.js:106

• saveUsageStats() returns early when both in/out tokens are 0, which prevents the new recordApiKeyUsage(apiKeyInfo, ...) call from running. That means RPM/RPH/RPD limits won't count responses that omit usage tokens (or report zeros). Consider still recording the request count even when token usage is unavailable.

  const inTokens = tokens.input_tokens ?? tokens.prompt_tokens ?? 0;
  const outTokens = tokens.output_tokens ?? tokens.completion_tokens ?? 0;

  if (inTokens === 0 && outTokens === 0) return;

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.