Analytics — observe command invocations, latency, error rates, callers

[SyniRon]

Right now feature utilization is guessed, not measured. Goal: capture per-command invocation counts, latency, error rates, and who's calling what.

Open:

• Stack choice — Prometheus + Grafana via existing 7Cav infra? Lighter (SQLite counter behind a /stats slash command)? OpenTelemetry?
• PII posture for caller IDs (Discord ID vs. hashed vs. omitted)
• Whether this should be aligned with parallel analytics work on related projects so a single observability surface emerges

Sizing: unknown until stack picked.

[SyniRon]

This was generated by AI during triage.

Triage Notes

What we've established so far:

• This is bot-service operational observability (which commands ran, how long, who) — distinct from the broader Discord guild analytics roadmap that Add recurring report for recent joiners with Star Citizen role #98 is the first foothold for. Both could eventually share a storage layer; they answer different questions and shouldn't be conflated at scope time.
• Sentry is already wired (PR 0.7.7, manual capture per ADR 0001) and is the only observability dependency cavbot2 has today.
• No metrics endpoint, scrape target, or /stats command exists today.
• 7Cav-org infra ≠ syniron-personal infra — options that would require new infrastructure should account for 7Cav-org infra cost, not cross-pollination from personal stacks.

Stack option comparison — please pick a direction:

1. Extend Sentry (Recommended starting point).
   Sentry's Performance Monitoring captures transactions/spans with sentry-go's transaction API. Each command becomes a transaction; latency / call counts / error correlation come for free; the data lives next to existing error captures. Cost: zero new infra, one Go dependency upgrade, ~per-command instrumentation diff. Limit: Sentry's free-tier transaction quota — would need to check 7Cav's plan.

2. SQLite + /stats slash command.
   In-process SQLite (mattn/go-sqlite3 or modernc.org/sqlite for pure-Go) writes one row per invocation ({ts, command, caller_id, latency_ms, error}). A /stats command queries it with parameterized aggregates (top callers, p50/p95 latency, error rates by command, last 24h activity). Cost: one Go dependency, one file on disk, one new command. Benefit: self-contained, no external service, Discord-native UX. Limit: no live dashboard — you query via Discord.

3. Prometheus + Grafana.
   /metrics HTTP endpoint exposed by the bot, Prometheus scrapes, Grafana visualizes. Cost: meaningful — needs a Prom instance + Grafana on 7Cav infra (or wiring into something existing on the 7Cav org side), a scrape target reachable from Prom, dashboard JSON. Benefit: proper time-series dashboards, alerting. Best long-term observability story; worst short-term effort.

4. OpenTelemetry.
   Vendor-neutral instrumentation; exports to Sentry, Prometheus, Tempo/Jaeger, etc. Forward-looking but rarely worth the abstraction overhead unless multi-backend export is a real near-term need.

5. Umami — flagged but explicitly out for this issue (different ecosystem; revisit in a future research session if 7Cav stands one up).

Questions blocking scope:

• Stack choice. Which of the above? My pick is starting with (1) Sentry-extension as the cheapest meaningful win, with (2) SQLite as a deferred follow-up if the Discord-native /stats UX turns out to be valuable. Acceptable, or different direction?
• PII posture for caller IDs. Options: (a) Discord ID raw, (b) hashed Discord ID (HMAC with a secret salt — pseudonymous, still groupable per-user), (c) Discord username only, (d) omit entirely (aggregate-only). My pick: (b) hashed with an env-var salt — lets you ask "who's the heaviest user of X" without storing raw identifiers in the telemetry backend. Push back?
• Cross-project alignment. The body mentions "parallel analytics work on related projects so a single observability surface emerges" — which projects, and is there a 7Cav-org-wide observability stack being designed that this should slot into? If yes, that probably moots the option comparison above and the answer is "wait for the org-wide decision."
• Scope distinction confirmation. Confirming this is bot-service telemetry only, NOT the broader Discord guild analytics roadmap (Add recurring report for recent joiners with Star Citizen role #98). They share a future SQLite-shaped opportunity but should be designed independently.