Skip to content

Add Status tab with auto-revival, latency-based smart routing, cost tracking, and bug fixes - #11

Open
MIsmail3018 wants to merge 2 commits into
p32929:masterfrom
MIsmail3018:feature/status-dashboard-and-revival
Open

Add Status tab with auto-revival, latency-based smart routing, cost tracking, and bug fixes#11
MIsmail3018 wants to merge 2 commits into
p32929:masterfrom
MIsmail3018:feature/status-dashboard-and-revival

Conversation

@MIsmail3018

Copy link
Copy Markdown

What

Three new features for the rotato admin dashboard and key rotation pipeline:

  1. Auto-Revival - re-tests quarantined keys every 1 hour against their provider base URL and auto-unquarantines them if they come back online. Prevents permanent key loss when a provider briefly rejects requests.
  2. Smart Routing (latency-based) - rolling 20-sample latency window per key with avg/p50/p95 stats. getNextKey() now sorts candidates by average latency, fastest first. State persists across restarts.
  3. Per-Model Cost Tracking - records tokens_in / tokens_out / cost per request from upstream usage blocks. 24h summary by provider AND by model. 10k-record ring buffer.

New REST endpoints (admin auth required)

  • GET /admin/api/status - named keys + dead keys + 24h usage
  • GET /admin/api/latency - latency stats per key
  • GET /admin/api/cost - cost/token summary 24h + by provider/model
  • GET /admin/api/quarantine - full quarantine state
  • POST /admin/api/quarantine/clear - body {key?} clears one or all
  • POST /admin/api/revival/run - triggers AutoRevival immediately

Dashboard

New "Status" tab in public/admin.html with:

  • 4 stat cards (providers, named keys, dead keys, requests/24h)
  • Dead keys panel with human names + quarantine expiry
  • Per-provider quota bars (red >=90%, yellow >=70%, green <70%)
  • Per-key usage rows with 429 count
  • Smart Routing card (blue left-border, latency rank with medals)
  • Cost Summary card (green left-border, 24h totals)
  • Auto-refresh every 10s when tab active

Files changed

  • public/admin.html - +279 lines (Status tab + 3 new render functions)
  • src/keyRotator.js - +366 lines (LatencyTracker, CostTracker, AutoRevival classes)
  • src/openaiClient.js - +42 lines (latency recording on success/error, cost recording)
  • src/server.js - +205 lines (6 new endpoints, AutoRevival wiring)

Total: 4 files changed, 858 insertions(+), 34 deletions(-)

Tested

2026-06-07 with 14 real requests: 6 keys tracked, latency range 1053ms-4601ms, 13497 tokens, 0 errors. AutoRevival scheduler fires at 30s after startup and every 1h thereafter.

Notes for reviewer

  • AutoRevival uses the same default base URLs as the openai client; if a provider has a custom BASE_URL in .env, it will be picked up from config.providers.
  • Quota limits in the dashboard (50/day OpenRouter, 300/day OmniRoute, etc.) are hardcoded in JS - they should be made provider-configurable in a follow-up.
  • The cost field is recorded as 0 because rotato does not currently know per-model pricing; a follow-up could parse model_pricing.json if you maintain one upstream.

Update: Bug fixes from user review

After the first review surfaced 4 data bugs in the Status tab, this branch now also includes:

  1. Providers stat — was showing 0 (Object.keys on Map). Fixed with Array.from(getProviders().keys()).
  2. omni-route-key dead — was marked dead from HTTP 403, but 403 means endpoint unavailable, not key invalid. Removed 403 from deadKeyStatusCodes; AutoRevival.testKey now distinguishes true=alive, false=key-dead, null=endpoint-unavailable.
  3. Quota over-count — was including rotato's own auth-gate rejects. Filtered at server, exposed as separate auth_rejected_24h stat.
  4. (no key) rows — was polluting per-provider display. Hidden in dashboard, replaced with new Auth Rejected stat card.

Also filtered out zero-count ghost providers (e.g. 'openai'/'unknown' with 0 requests) from the response.

Total: 4 files, +40/-11 on top of the original +858/-34.

… admin dashboard

Three new features integrated into the Status tab and exposed via REST endpoints:

1. AutoRevival (~/rotato/src/keyRotator.js)
   - Re-tests quarantined keys every 1 hour against their provider base URL
   - Auto-unquarantines keys that are alive again (HTTP <400)
   - Manual trigger via POST /admin/api/revival/run
   - Prevents 'permanent' dead-key loss when a provider comes back online

2. LatencyTracker + smart routing (~/rotato/src/openaiClient.js)
   - Rolling 20-sample latency window per key (avg/p50/p95)
   - State persisted to ~/rotato/latency_state.json across restarts
   - getNextKey() now sorts candidates by average latency, fastest first
   - Dashboard shows live latency rank with gold/silver/bronze medals

3. CostTracker (~/rotato/src/keyRotator.js)
   - Records tokens_in/tokens_out/cost per request from upstream usage blocks
   - 24h summary by provider AND by model
   - Input vs output token split on dashboard
   - Last 10,000 records kept (ring buffer)

New REST endpoints (all require admin auth):
- GET  /admin/api/status      - named keys + dead keys + 24h usage
- GET  /admin/api/latency     - latency stats per key
- GET  /admin/api/cost        - cost/token summary 24h + by provider/model
- GET  /admin/api/quarantine  - full quarantine state
- POST /admin/api/quarantine/clear - body {key?} clears one or all
- POST /admin/api/revival/run - triggers AutoRevival immediately

Dashboard (~/rotato/public/admin.html) adds a new 'Status' tab with:
- 4 stat cards (providers, named keys, dead keys, requests/24h)
- Dead keys panel with human names + quarantine expiry
- Per-provider quota bars (red >=90%, yellow >=70%, green <70%)
- Per-key usage rows with 429 count
- Smart Routing card (blue left-border, latency rank)
- Cost Summary card (green left-border, 24h totals)
- Auto-refresh every 10s when tab active

Tested 2026-06-07 with 14 real requests: 6 keys tracked, latency range
1053ms-4601ms, 13497 tokens, 0 errors. AutoRevival scheduler fires at
30s after startup and every 1h thereafter.
1. Providers stat showed 0 — `Object.keys()` on a Map returns [].
   Fix: use `Array.from(this.config.getProviders().keys())`.

2. omni-route-key marked Dead (HTTP 403) — 403 is 'endpoint
   forbidden' (OmniRoute doesn't expose GET /v1/models), not
   'key invalid'. Fix: remove 403 from deadKeyStatusCodes in
   openaiClient.js (was: Set([401, 403]); now: Set([401])).
   Also: AutoRevival.testKey now returns null on 403/404 to
   mean 'endpoint not available, don't mark dead' (distinct
   from false=key invalid, true=key alive).

3. Provider quota bars included rotato's own auth-gate rejects.
   Fix: filter out entries where status=401 and keyUsed is null
   (these are rotato rejecting missing/invalid ACCESS_KEY) before
   building per-provider usage. Count them as auth_rejected_24h
   instead, exposed as separate stat in the response.

4. '(no key)' rows polluting per-provider display — same root
   cause as p32929#3, plus filter in dashboard: hide rows where
   name === '(no key)' from the per-provider cards.

Bonus: filter out providers from the response that aren't in the
configured list (cleaned up 'openai' and 'unknown' with 0
requests showing as fake provider rows).

Manually cleared the stale omni-route-key entry from
quarantine_state.json since the trigger condition no longer
exists.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant