feat(queryapi): serve metric queries with server-enforced project isolation

[0xmc]

Closes #207.

queryapi now serves its five Prometheus-shaped metric routes instead of returning 501. Every metric query is confined to the caller's project by the server, with the project taken only from the verified identity, the same rule the logs path follows.

How tenancy is enforced

• storage.MetricStore sits alongside LogStore. The prometheus backend is a narrow net/http client for any Prometheus-query-API server, not a reverse proxy.
• Each backend request is built from typed fields and carries extra_label=<project label>=<project>. Client parameters are never copied, so a caller can't add, replace, or remove the filter. A request with no project sends nothing.
• extra_label is a VictoriaMetrics extension, and servers that don't implement it ignore it. So tenancy is an explicit, required flag (--prometheus-tenancy=extra-label) rather than something implied by the backend URL. Startup fails without it.

Proof it refuses

task test-integration-prometheus seeds two projects into VictoriaMetrics v1.114.0 and, as one project, checks query, query_range, series, labels, and label/{name}/values, with the result cache on. Every negative check is paired with a positive one.

With the extra_label line removed, every endpoint's isolation check fails (the other project's series, labels, and values come back, and sum returns both projects' total). With it restored, all pass. The task fails unless the test reports a pass, so a skipped or renamed test can't go green. CI runs it as Test (queryapi integration-prometheus).

Other changes

• Prometheus HTTP API parameters: fractional Unix seconds, float-second steps, --max-range, the 11,000-point limit, a timeout capped at --query-timeout, and the Prometheus error envelope. Backend failures are logged, not returned.
• VictoriaMetrics returns query timeouts as HTTP 422 with a numeric errorType; they now surface as 503 timeout.
• /readyz gains metrics-storage. The logs check is renamed storage → logs-storage to match, which changes /readyz?verbose output.
• --logs-storage replaces --storage, which stays as a deprecated alias for one release. The deployment keeps the STORAGE env var, so overlays need no edit.
• New flags: --metrics-storage (default fake), --prometheus-url, --prometheus-tenancy, --prometheus-project-label, --max-range, --max-response-bytes.
• OpenAPI updated to 0.4.0 for real PromQL responses.

Rollout

Merging changes nothing in-cluster: --metrics-storage defaults to fake. Pointing queryapi at VictoriaMetrics, moving cloud-portal over, and closing the direct path to VictoriaMetrics happen in infra and cloud-portal.

Follow-ups

• Tighten the timeout match to VictoriaMetrics's message prefix, so a malformed query that echoes "timeout exceeded" reports bad_data.
• PromQL parse-and-rewrite as a second tenancy mode.

🤖 Generated with Claude Code