Expose tenant-scoped usage events through authenticated CLI

[lluisinthedesert]

Goal

Let public CLI users inspect their own Understudy gateway usage/traces after they authenticate, without exposing ClickHouse credentials, arbitrary SQL, cross-tenant data, or raw payloads by default.

This is the public-safe version of our internal manual verification flow where we confirm live GEPA/DSPy adapter calls in understudy_metrics.events.

Non-goals

• Do not hand out ClickHouse users/passwords to public customers.
• Do not expose arbitrary SQL or table/column discovery through the CLI.
• Do not expose cross-org, cross-project, or founder/admin analytics.
• Do not return raw prompts, completions, captures, or R2 objects in this issue.
• Do not make this a broad dashboard project.

Proposed surface

Add a small tenant-scoped usage API in the platform, then a thin CLI command in this repo.

Possible CLI shape:

understudy usage recent --project rehearsal --limit 20 --json
understudy usage summary --project rehearsal --since 24h --json

The API should authenticate with the same user/API-key context as the rest of the CLI and server-side enforce:

• org scope from auth, not client input alone;
• optional project/key filters only if the caller owns them;
• bounded time windows and row limits;
• allowlisted fields only;
• redacted payloads by default;
• rate limits suitable for CLI polling/debugging.

Minimum returned fields

For recent rows:

• ts
• request_id
• project_id or project slug if cheaply joinable
• endpoint
• provider
• requested_model
• served_model
• input_tokens
• output_tokens
• total_ms
• status_code
• outcome

For summary:

• request count
• success/error counts
• token totals
• p50/p95 total_ms
• grouped by model and project

Security requirements

• Platform owns the ClickHouse credential; CLI never sees it.
• All queries are parameterized/constructed from allowlisted filters.
• Auth-derived org scope is mandatory in every query.
• Default response never includes request/response payload bodies.
• Error messages must not leak table names beyond the public API contract or internal query text.
• Audit/rate-limit this endpoint separately from inference if needed.

Acceptance criteria

• A signed-in CLI user can list their own recent usage events after making a gateway call.
• A signed-in CLI user cannot request another org/project/key's rows.
• CLI command works with --json and prints no secrets.
• Tests cover tenant scope, limit/window bounds, and rejected cross-tenant filters.
• Docs explain that this is a scoped usage view, not direct ClickHouse access.

[lluisinthedesert]

Leaving this open after PR #30. The merged CLI now has auth/project/model-route surfaces, but tenant-scoped usage inspection still needs a platform API plus a thin CLI command. Cleaned the proposed binary name to understudy.

[bharath-avl]

read through this and PR #30, makes sense the tenant-scoped auth stuff has to sit on the platform side first.

would it help if I built out the CLI part in the meantime? like understudy usage recent / usage summary with the flags and json shape you already spec'd, just pointed at a mocked response for now. that way the command's already built and tested once the platform api is ready, and it's not blocked on backend work I can't really touch anyway.

no worries if the shape's still likely to change and this'd just be wasted work, just figured i'd ask before assuming