Looking Glass Backend #1041

Goal

Extend Galactic's fabric API to support the customer-facing Looking Glass service with scoped service access, incremental router results, complete coverage, and backend rate-limiting guardrails.

Reuse the existing route lookup, BGP summary, ping, and traceroute implementations, router discovery, and execution scheduler. An authorized service submits a bounded diagnostic with trusted project and request identity; Galactic executes it through the shared cell scheduler and returns each router's observation as it finishes.

Protect fabric routers and probe destinations from excessive diagnostic traffic, including requests originating from a public site. Enforce backend limits independently of upstream admission and public ingress protections.

This epic covers Galactic's backend contract, implementation, operational controls, and qualification. The public website, session API and lifecycle, project IAM, federation, Datum Connect, customer clients, and public ingress rate limiting are owned outside Galactic.

Enhancement: Looking Glass.

Done when

• A dedicated, cell-scoped service identity can invoke the four supported diagnostics without operator privileges or direct access to node-only methods. Unauthorized callers and cross-cell identities are rejected. Project and request attribution is accepted only through the authorized service contract.
• The service receives complete router outcomes as individual routers finish, followed by final coverage from the same discovery snapshot. Expected, successful, failed, and omitted routers are accounted for consistently; observations retain router identity and sample time.
• Typed errors distinguish backend failures, unavailable capabilities, and throttling from network observations such as packet loss or an unanswered hop. Throttled callers receive retry guidance through the backend protocol.
• Customer requests share the existing execution scheduler with operator and FabricQuery traffic. Trusted project identity drives fair scheduling, so all customer traffic is not treated as one service account's workload.
• Configurable burst and sustained admission limits, concurrent execution caps, and bounded queues protect each cell. Per-project and per-destination probe budgets account for diagnostic cost and router fan-out. Limits remain effective across gateway replicas and leadership changes; limiter failures cannot permit unbounded execution.
• Backend destination restrictions, execution deadlines, packet budgets, and output limits apply to every customer diagnostic regardless of upstream validation. Slow consumers cannot hold execution slots indefinitely or cause unbounded buffering.
• Caller cancellation, stream failure, deadline expiry, and loss of execution ownership stop queued and in-flight work within documented and measured bounds. Customer diagnostic streams are not automatically replayed after interruption or ownership replacement.
• Existing operator and FabricQuery interfaces remain compatible. The new contract is published in a consumable API version, and session results are not persisted in Kubernetes status or routine logs.
• Operators can configure limits, observe admission and execution outcomes, diagnose throttling, and disable customer diagnostic execution independently of routing. Metrics avoid unbounded customer and destination labels.
• All milestones have delivered their artifacts and passed their acceptance checks. Qualification uses real fabric routers and an independent service client, covering authorization, incremental results, incomplete coverage, fairness, overload, slow consumers, cancellation, limiter failure, and leadership changes, with reproducible results and no material routing impact against agreed thresholds.

Milestones

• Define the service execution contract, attribution, coverage, and limit semantics #1042 — Define the service execution contract, attribution, coverage, and limit semantics
• Implement scoped service identity and backend authorization #1043 — Implement scoped service identity and backend authorization
• Deliver incremental router outcomes and complete coverage #1044 — Deliver incremental router outcomes and complete coverage
• Integrate project fairness, rate limits, and aggregate probe budgets #1045 — Integrate project fairness, rate limits, and aggregate probe budgets
• Enforce bounded streaming, cancellation, and execution ownership #1046 — Enforce bounded streaming, cancellation, and execution ownership
• Deliver backend configuration, diagnostics, and operational controls #1047 — Deliver backend configuration, diagnostics, and operational controls
• Qualify service integration and failure behavior on the target platform #1048 — Qualify service integration and failure behavior on the target platform