feat(web): discover configured queues through the gateway

doc/rfc/web-library.md

@@ -10,19 +10,19 @@ Phase one is read-only: request summary, history, and queue receipt list ([statu
 
 ## URL reference
 
-The entry point lists the queues configured by the host, with each queue linking to its landing page. The queue is the top-level navigation context. Singular resource segments identify what the user is opening; the ID remains a value supplied by the gateway, not a string the UI interprets. The reference host currently serves only `demo-queue`, so its queue list initially has one entry.
+The entry point lists gateway-configured queues through `ListQueues`, including empty queues, with each queue linking to its landing page. The queue is the top-level navigation context; resource IDs remain gateway-supplied values that the UI does not interpret.
 
 | Resource or view | Route | Example | Availability |
 |---|---|---|---|
-| Queue directory | `/` | `/` lists configured queues, including a link to `/demo-queue` | Reference host |
+| Queue directory | `/` | `/` lists queues returned by `ListQueues` | Reference host |
 | Queue landing and request list | `/<queue>` | `/demo-queue` | Reference host |
 | Request detail | `/<queue>/request/<request-id>` | `/demo-queue/request/10` | Reference host |
 | Request history tab | `/<queue>/request/<request-id>?view=history` | `/demo-queue/request/10?view=history` | Reference host |
 | Batch detail | `/<queue>/batch/<batch-id>` | `/demo-queue/batch/4` | Deferred; requires a batch read contract |
 | GitHub logical change | `/<queue>/change/github/<host>/<org>/<repo>/pull/<pr>` | `/demo-queue/change/github/github.com/uber/submitqueue/pull/123` | Demo receipt-window scan; unbounded lookup deferred |
 | Phabricator logical change | `/<queue>/change/phab/<host>/D<revision>` | `/demo-queue/change/phab/phabricator.example.com/D12345` | Demo receipt-window scan; unbounded lookup deferred |
 
-The queue directory replaces the automatic redirect to `demo-queue`. Queue discovery belongs to the host's configured queue registry; the directory does not infer queue names from requests or require a new gateway discovery API.
+The queue directory replaces the automatic redirect to `demo-queue`. The library maps queue models; the host fetches and validates them through the published gateway API, without inferring queues from received requests.
 
 The request and batch examples follow [scoped sequential resource IDs](scoped-resource-ids.md): `"10"` and `"4"` are stored string IDs, and the route supplies their queue and resource context. The UI passes each ID unchanged to the gateway. The mockups below retain earlier illustrative slash-containing IDs; those labels do not override the scoped-ID contract.
 

web/package_test/consumer.ts

@@ -1,5 +1,5 @@
 import { RequestList, QueueDirectory, ChangeSubmissions, AutoRefresh } from "@submitqueue/web-submitqueue";
-import { loadRequestList, loadChangeSubmissions } from "@submitqueue/web-submitqueue/server";
+import { loadRequestList, loadChangeSubmissions, loadQueueDirectory } from "@submitqueue/web-submitqueue/server";
 import { createFakeGatewayReader } from "@submitqueue/web-submitqueue/testing";
 
 void RequestList;
@@ -9,3 +9,4 @@ void QueueDirectory;
 void ChangeSubmissions;
 void AutoRefresh;
 void loadChangeSubmissions;
+void loadQueueDirectory;

web/service/submitqueue/README.md

@@ -2,9 +2,9 @@
 
 This Next.js application demonstrates how to host `@submitqueue/web-submitqueue`. It is intentionally local/demo wiring rather than a production authentication or deployment contract.
 
-The host owns the `/<queue>` landing page and `/<queue>/request/<full-request-id>` detail route, the fixed `demo-queue` selection, HTTP Basic authentication, server-side Connect transport, readiness checks, and environment validation. Request IDs remain readable and are passed to the gateway without interpreting their internal format. `/healthz` validates configuration and performs a bounded gateway ping so Compose readiness reflects the host's only upstream. The browser never connects to the gRPC gateway directly.
+The host owns the `/<queue>` landing page and `/<queue>/request/<full-request-id>` detail route, HTTP Basic authentication, server-side Connect transport, readiness checks, and environment validation. Request IDs remain readable and are passed to the gateway without interpreting their internal format. `/healthz` validates configuration and performs a bounded gateway ping so Compose readiness reflects the host's only upstream. The browser never connects to the gRPC gateway directly.
 
-`/` lists the configured queues instead of redirecting to the demo queue. Request detail defaults to Summary and uses `?view=history` for shareable History navigation. The host supplies Next-specific refresh completion, dynamic rendering, authentication on every protected route, and CSS; none of those framework decisions live in the reusable library.
+`/` fetches configured queues through the gateway's `ListQueues` RPC, including queues with no requests. Queue, request, and change routes validate queue names against the same discovered catalog, with no demo-only allowlist. Discovery errors remain errors rather than being reported as missing queues. Request detail defaults to Summary and uses `?view=history` for shareable History navigation. The host supplies Next-specific refresh completion, dynamic rendering, authentication on every protected route, and CSS; none of those framework decisions live in the reusable library.
 
 GitHub, Phabricator, and git change pages mirror the canonical change URI's authority/path. The host adapter validates those provider-specific paths; React components do not parse URIs. Fake-provider `sq-files`/`sq-fake` query hints remain in the submitted backend values but are excluded from UI labels and navigation URLs.
 

web/service/submitqueue/src/app/(protected)/[queue]/change/[...reference]/page.tsx

@@ -4,7 +4,7 @@ import { connection } from "next/server";
 import { notFound, redirect } from "next/navigation";
 import { NextRefresh } from "../../../../../components/next-refresh";
 import { ChangeView } from "../../../../../components/change-view";
-import { DEMO_QUEUE } from "../../../../../server/config";
+import { loadConfiguredQueue } from "../../../../../server/queues";
 import { parseChangePath, changeHref } from "../../../../../server/change";
 import { readChangeSubmissions } from "../../../../../server/change-reader";
 import { resolveGatewayClient } from "../../../../../server/gateway";
@@ -25,7 +25,11 @@ export default async function ChangePage({ params, searchParams }: {
   const path = await params;
   const queue = decodePathSegment(decodeURIComponent(path.queue));
   const reference = parseChangePath(path.reference.map(decodeURIComponent));
-  if (queue !== DEMO_QUEUE || reference === null) {
+  if (reference === null) {
+    notFound();
+  }
+  const configured = await loadConfiguredQueue(queue);
+  if (configured.ok && !configured.data) {
     notFound();
   }
   const search = await searchParams;
@@ -42,7 +46,7 @@ export default async function ChangePage({ params, searchParams }: {
     redirect(latestHref);
   }
   const requestWindow = pageWindow ?? defaultRequestWindow();
-  const result = await loadChangeSubmissions(
+  const result = configured.ok ? await loadChangeSubmissions(
     async () => {
       const page = await readChangeSubmissions(resolveGatewayClient(), queue, reference, requestWindow);
       return {
@@ -62,7 +66,7 @@ export default async function ChangePage({ params, searchParams }: {
       window: scansQueue ? { fromMs: requestWindow.fromMs, toMs: requestWindow.toMs } : null,
     },
     { diagnostics: gatewayDiagnostics },
-  );
+  ) : configured;
   return <main className="shell">
     {result.ok ? <ChangeView model={result.data} /> : <ErrorState error={result.error} />}
     <div className="page-actions"><NextRefresh refreshHref={pageWindow ? latestHref : undefined}

web/service/submitqueue/src/app/(protected)/[queue]/page.tsx

@@ -12,7 +12,7 @@ import Link from "next/link";
 import { notFound, redirect } from "next/navigation";
 import { connection } from "next/server";
 
-import { DEMO_QUEUE } from "../../../server/config";
+import { loadConfiguredQueue } from "../../../server/queues";
 import { gatewayDiagnostics } from "../../../server/diagnostics";
 import { resolveDemoGateway } from "../../../server/gateway";
 import { requireAuthorization } from "../../../server/request-auth";
@@ -47,7 +47,8 @@ export default async function QueueRequestsPage({
   await requireAuthorization();
 
   const queue = decodePathSegment(decodeURIComponent((await params).queue));
-  if (queue !== DEMO_QUEUE) {
+  const configured = await loadConfiguredQueue(queue);
+  if (configured.ok && !configured.data) {
     notFound();
   }
 
@@ -63,7 +64,7 @@ export default async function QueueRequestsPage({
   }
   const requestWindow = pageWindow ?? defaultRequestWindow();
 
-  const loaded = await loadRequestList(
+  const loaded = configured.ok ? await loadRequestList(
     resolveDemoGateway,
     {
       queue,
@@ -73,7 +74,7 @@ export default async function QueueRequestsPage({
       pageToken: requestWindow.pageToken,
     },
     { diagnostics: gatewayDiagnostics },
-  );
+  ) : configured;
   const result = loaded.ok && loaded.data.nextPageToken ? {
     ...loaded,
     data: { ...loaded.data, nextPageToken: encodeRequestPage(queue, requestWindow, loaded.data.nextPageToken, secret) },

web/service/submitqueue/src/app/(protected)/[queue]/request/[...sqid]/page.tsx

@@ -11,7 +11,7 @@ import type { Metadata } from "next";
 import { notFound, redirect } from "next/navigation";
 import { connection } from "next/server";
 
-import { DEMO_QUEUE } from "../../../../../server/config";
+import { loadConfiguredQueue } from "../../../../../server/queues";
 import { gatewayDiagnostics } from "../../../../../server/diagnostics";
 import { resolveDemoGateway } from "../../../../../server/gateway";
 import { requireAuthorization } from "../../../../../server/request-auth";
@@ -39,7 +39,11 @@ export default async function RequestPage({
 
   const { queue: queuePath, sqid: sqidPath } = await params;
   const queue = decodePathSegment(decodeURIComponent(queuePath));
-  if (queue !== DEMO_QUEUE || sqidPath.length === 0) {
+  if (sqidPath.length === 0) {
+    notFound();
+  }
+  const configured = await loadConfiguredQueue(queue);
+  if (configured.ok && !configured.data) {
     notFound();
   }
   const sqid = sqidPath.map(value => decodePathSegment(decodeURIComponent(value))).join("/");
@@ -49,14 +53,14 @@ export default async function RequestPage({
     redirect(paths.request(queue, sqid, { view }));
   }
 
-  const result = await loadRequestDetail(
+  const result = configured.ok ? await loadRequestDetail(
     resolveDemoGateway,
     {
       queue,
       sqid,
     },
     { diagnostics: gatewayDiagnostics },
-  );
+  ) : configured;
 
   if (!result.ok && result.error.kind === "not-found") {
     notFound();

web/service/submitqueue/src/app/(protected)/page.tsx

@@ -1,7 +1,10 @@
 import { connection } from "next/server";
-import { QueueDirectory } from "@submitqueue/web-submitqueue";
+import { ErrorState, QueueDirectory } from "@submitqueue/web-submitqueue";
+import { loadQueueDirectory } from "@submitqueue/web-submitqueue/server";
 
-import { HOST_QUEUES } from "../../server/config";
+import { resolveGatewayClient } from "../../server/gateway";
+import { gatewayDiagnostics } from "../../server/diagnostics";
+import { NextRefresh } from "../../components/next-refresh";
 import { requireAuthorization } from "../../server/request-auth";
 
 export const dynamic = "force-dynamic";
@@ -10,5 +13,12 @@ export const revalidate = 0;
 export default async function HomePage() {
   await connection();
   await requireAuthorization();
-  return <main className="shell"><QueueDirectory queues={HOST_QUEUES} /></main>;
+  const result = await loadQueueDirectory(resolveGatewayClient(), { diagnostics: gatewayDiagnostics });
+  return <main className="shell">
+    {result.ok ? <QueueDirectory queues={result.data} /> : <ErrorState error={result.error} />}
+    <div className="page-actions"><NextRefresh
+      terminal={!result.ok && !result.error.retryable}
+      transientFailureCount={!result.ok && result.error.retryable ? 1 : 0}
+    /></div>
+  </main>;
 }

web/service/submitqueue/src/app/not-found.tsx

@@ -6,7 +6,7 @@ export default function NotFound() {
       <section className="state-card" aria-labelledby="not-found-title">
         <p className="eyebrow">Request lookup</p>
         <h1 id="not-found-title">Request not found</h1>
-        <p>The request may not exist in the demo queue or may have expired.</p>
+        <p>The queue or request may not exist, or its retained data may have expired.</p>
         <Link className="button-link" href="/">
           Return to queues
         </Link>

web/service/submitqueue/src/server/config.ts

@@ -1,9 +1,5 @@
 import "server-only";
 
-export const DEMO_QUEUE = "demo-queue";
-export const HOST_QUEUES = [
-  { name: DEMO_QUEUE, description: "Local read-only demo backed by the SubmitQueue gateway" },
-];
 export const GATEWAY_DEADLINE_MS = 5_000;
 export const GATEWAY_HEALTH_DEADLINE_MS = 1_000;
 

web/service/submitqueue/src/server/queues.test.ts

@@ -0,0 +1,27 @@
+import { describe, expect, it, vi } from "vitest";
+import { createFakeGatewayReader } from "@submitqueue/web-submitqueue/testing";
+import { Code, ConnectError } from "@connectrpc/connect";
+
+const resolver = vi.hoisted(() => ({ resolveGatewayClient: vi.fn() }));
+vi.mock("./gateway", () => resolver);
+
+import { loadConfiguredQueue } from "./queues";
+
+describe("gateway-backed host queue validation", () => {
+  it("accepts configured queues other than the former demo-only queue", async () => {
+    resolver.resolveGatewayClient.mockReturnValue(createFakeGatewayReader({
+      queues: [{ name: "empty-queue" }],
+    }));
+    expect(await loadConfiguredQueue("empty-queue")).toEqual({ ok: true, data: true });
+    expect(await loadConfiguredQueue("unknown")).toEqual({ ok: true, data: false });
+  });
+
+  it("does not treat a failed queue discovery as a missing queue", async () => {
+    resolver.resolveGatewayClient.mockReturnValue(createFakeGatewayReader({
+      queuesError: new ConnectError("unavailable", Code.Unavailable),
+    }));
+    expect(await loadConfiguredQueue("demo-queue")).toMatchObject({
+      ok: false, error: { kind: "transient", retryable: true },
+    });
+  });
+});

web/service/submitqueue/src/server/queues.ts

@@ -0,0 +1,13 @@
+import "server-only";
+
+import type { LoadResult } from "@submitqueue/web-submitqueue";
+import { loadQueueDirectory } from "@submitqueue/web-submitqueue/server";
+import { gatewayDiagnostics } from "./diagnostics";
+import { resolveGatewayClient } from "./gateway";
+
+export async function loadConfiguredQueue(queue: string): Promise<LoadResult<boolean>> {
+  const directory = await loadQueueDirectory(resolveGatewayClient(), { diagnostics: gatewayDiagnostics });
+  return directory.ok
+    ? { ok: true, data: directory.data.some(value => value.name === queue) }
+    : directory;
+}

web/submitqueue/README.md

@@ -21,4 +21,6 @@ Request list/detail result views retain the last successful snapshot on transien
 
 The package-consumer checks typecheck without Next.js and reject Next.js dependencies or imports in the emitted library.
 
+`loadQueueDirectory` maps the gateway's `ListQueues` response into serializable queue models and sanitized errors. It uses a separate structural queue-reader interface so request-only adapters remain valid; the host supplies the client and decides where to render or validate the discovered catalog.
+
 Normal queue/resource paths remain readable. Exceptional dot-only path segments use a visible `~` escape so browsers cannot normalize them away; `decodePathSegment` reverses that transport escape without interpreting an ID. Change-version navigation is an optional host callback over already supplied URLs, keeping router integration out of the library.

web/submitqueue/src/components.tsx

@@ -40,10 +40,10 @@ function RequestChanges({ request, links = {}, labels = {} }: {
 export function QueueDirectory({ queues, basePath }: { queues: readonly QueueModel[]; basePath?: string }) {
   const paths = new WebPaths(basePath === undefined ? {} : { basePath });
   return <section className="sq-directory">
-    <h1>Queues</h1><p className="sq-secondary">Queues configured by this host</p>
+    <h1>Queues</h1><p className="sq-secondary">Available submit queues</p>
     {queues.length === 0 ? <p>No queues are configured.</p> :
       <ul>{queues.map(queue => <li key={queue.name}>
-        <h2><a href={paths.requests(queue.name)}>{queue.name}</a></h2><p>{queue.description}</p>
+        <h2><a href={paths.requests(queue.name)}>{queue.name}</a></h2>{queue.description ? <p>{queue.description}</p> : null}
       </li>)}</ul>}
   </section>;
 }

web/submitqueue/src/server.test.ts

@@ -7,11 +7,43 @@ import {
   loadChangeSubmissions,
   loadRequestDetail,
   loadRequestList,
+  loadQueueDirectory,
   requestDetailIsComplete,
   requestDetailRefreshState,
   safeInt64ToNumber,
 } from "./server";
 
+describe("queue discovery mapping", () => {
+  it("lists gateway-configured queues, including ones without requests", async () => {
+    const reader = createFakeGatewayReader({ queues: [{ name: "demo-queue" }, { name: "empty-queue" }] });
+    const result = await loadQueueDirectory(reader);
+    expect(result).toEqual({
+      ok: true, data: [{ name: "demo-queue", description: "" }, { name: "empty-queue", description: "" }],
+    });
+  });
+
+  it("returns an empty directory when the gateway has no configured queues", async () => {
+    expect(await loadQueueDirectory(createFakeGatewayReader({ queues: [] }))).toEqual({ ok: true, data: [] });
+  });
+
+  it("classifies discovery failures without leaking the gateway message", async () => {
+    const onGatewayError = vi.fn();
+    const cause = new ConnectError("private upstream detail", Code.Unavailable);
+    const result = await loadQueueDirectory(
+      createFakeGatewayReader({ queuesError: cause }),
+      { diagnostics: { onGatewayError } },
+    );
+    expect(result).toMatchObject({ ok: false, error: { kind: "transient", retryable: true } });
+    expect(JSON.stringify(result)).not.toContain("private upstream detail");
+    expect(onGatewayError).toHaveBeenCalledWith(expect.objectContaining({ operation: "queues", cause }));
+  });
+
+  it.each([{ queues: [{ name: "" }] }, { queues: [{ name: "same" }, { name: "same" }] }])("rejects malformed discovery data", async ({ queues }) => {
+    const result = await loadQueueDirectory(createFakeGatewayReader({ queues }));
+    expect(result).toMatchObject({ ok: false, error: { kind: "internal" } });
+  });
+});
+
 describe("server presentation mapping", () => {
   beforeEach(() => vi.clearAllMocks());
 

web/submitqueue/src/server.ts

@@ -9,6 +9,7 @@ import type {
   RequestDetailModel,
   RequestListModel,
   RequestSummaryModel,
+  QueueModel,
   WebError,
 } from "./models.js";
 import { isTerminalStatus } from "./status.js";
@@ -71,7 +72,14 @@ export interface GatewayReader {
 
 export type GatewayResolver = (queue: string) => GatewayReader;
 
-export type GatewayOperation = "list" | "summary" | "history" | "change";
+export interface GatewayQueueReader {
+  listQueues(
+    input: Record<string, never>,
+    options?: GatewayCallOptions,
+  ): Promise<{ queues: readonly { name: string }[] }>;
+}
+
+export type GatewayOperation = "list" | "summary" | "history" | "change" | "queues";
 
 export interface GatewayDiagnostic {
   operation: GatewayOperation;
@@ -195,6 +203,28 @@ function reportGatewayError(
   }
 }
 
+export async function loadQueueDirectory(
+  gateway: GatewayQueueReader,
+  options: GatewayLoadOptions = {},
+): Promise<LoadResult<QueueModel[]>> {
+  try {
+    const response = await gateway.listQueues({});
+    const names = new Set<string>();
+    const queues: QueueModel[] = [];
+    for (const queue of response.queues) {
+      if (!queue.name || names.has(queue.name)) {
+        return { ok: false, error: INTERNAL_ERROR };
+      }
+      names.add(queue.name);
+      queues.push({ name: queue.name, description: "" });
+    }
+    return { ok: true, data: queues };
+  } catch (error) {
+    reportGatewayError(options, "queues", "", null, error);
+    return { ok: false, error: classifyGatewayError(error) };
+  }
+}
+
 export function requestDetailIsComplete(model: RequestDetailModel): boolean {
   if (model.historyError !== null || !isTerminalStatus(model.request.status)) {
     return false;