test: add end-to-end test spanning order, kitchen, and notification

.claude/CONTEXT.md

@@ -24,32 +24,23 @@ Full plan: `~/.claude/plans/lets-read-the-files-unified-galaxy.md`
 | 3 | Redis idempotency — replace in-memory Idempotency class | ✅ Done |
 | 4 | Retry-with-backoff in mq.ts | ✅ Done |
 | 5 | Testcontainers integration tests | ✅ Done |
-| 6 | End-to-end test | ⏳ Pending |
+| 6 | End-to-end test | ✅ Done |
 
 ## Current state
 
-**Phase 5 complete, on branch `phase-5-testcontainers` (not yet merged).** Every service now has a real Testcontainers-backed integration test (no mocks for Postgres/RabbitMQ/Redis): `services/order/tests/place-order.test.ts`, `services/kitchen/tests/order-events.test.ts`, `services/notification/tests/order-events.test.ts`, `services/menu/tests/menu.test.ts`. Root `bunfig.toml` sets `[test] timeout = 60000` (container startup exceeds bun's 5s default).
+**ALL PHASES COMPLETE (0 through 6) — the build plan is finished.** Phase 6 (`tests/e2e/order-flow.test.ts`, new `@quickbite/e2e` workspace package) wires order, kitchen, and notification together in one test: one Postgres container hosting all 3 databases (`order_db`/`kitchen_db`/`notification_db`, created via raw `CREATE DATABASE` statements — mirroring `docker/init.sql`'s shape), one shared RabbitMQ + Redis container, each service's real entrypoint dynamically imported in sequence (same env-var + dynamic-import technique as Phase 5). A real `POST /orders` via `server.inject()` drives the whole chain; the test polls both kitchen's and notification's own databases until the full happy path completes.
 
-**Key technique**: every service's `db.ts` reads `DATABASE_URL`/`RABBITMQ_URL`/`REDIS_URL` from `process.env` at module-load time (top-level `await ensureSchema()`). Static imports are hoisted before test code runs, so each test sets env vars in `beforeAll` *then* uses a dynamic `await import(...)` to load the module — no service source changes needed for this part. Order's route handler did need one small refactor: `buildServer()` was extracted out of `index.ts` into its own `services/order/src/server.ts`, so tests can `server.inject()` against it without triggering `index.ts`'s real `connect()`+`listen()` startup chain. Kitchen/notification's tests don't need this — they dynamically import the whole `index.ts` (with `PORT=0` for an ephemeral port) precisely because they DO want the real consumer wiring to run.
+**The one real architectural problem this phase surfaced, and the actual fix**: running `bun test` from the repo root executes every `*.test.ts` file in ALL packages within a SINGLE OS process by default. Since every service's `db.ts` reads `DATABASE_URL` from `process.env` once at module-load time, and Bun caches dynamically-imported modules by absolute path, a later test file calling `await import("../../services/order/src/server.ts")` could receive an already-cached module instance still pointing at an EARLIER test file's (already-stopped) Testcontainers Postgres. Tried Bun's `--isolate` flag (gives each file a fresh module/global context) — it didn't fully fix this, because `process.env` is OS-process-wide regardless of `--isolate`, and it also silently stopped honoring `bunfig.toml`'s test timeout, causing new failures. The actual fix: every package that has tests now has its own `"test": "bun test"` script, and the root script became `bun run --filter '*' --sequential test` — Bun's workspace filter spawns each package's tests as a genuinely separate OS process, which is real isolation, not just module-context isolation. Root `package.json`, every relevant `package.json`, and `.github/workflows/ci.yml`'s test step were updated to match.
 
-**Three real bugs found while getting these tests to actually pass, not implementation choices**:
-1. `@testcontainers/postgresql`'s default wait strategy ANDs `Wait.forHealthCheck()` with `Wait.forListeningPorts()` — but the plain `postgres:16` image has no Docker `HEALTHCHECK` (that's exactly why `docker-compose.yml` manually adds a `pg_isready` one), so it hung forever waiting for a health status that would never appear. Fixed in every Postgres-using test by overriding with `Wait.forLogMessage(/database system is ready to accept connections/, 2)`.
-2. `RecoveringChannelModel` (amqplib's recovery wrapper, Phase 2) emits its own `'error'` event separate from `'disconnect'` — an EventEmitter `'error'` with no listener crashes the whole process in Node/Bun. `connect()` in `packages/shared/src/mq.ts` only listened for `'connect'`/`'disconnect'`/`'reconnect-failed'`; added an `'error'` handler. This is a real production hardening fix, not just a test fix — a service could have crashed on certain connection failures before this.
-3. Notification's first integration test initially asserted exact event-arrival order (`["order.placed", "order.accepted", "order.ready"]`) — flaky, because the test publishes all 3 events back-to-back with no delay (unlike the real flow, where kitchen naturally waits ~3s between accepted and ready), so concurrent handler completions can race. Fixed to assert presence via a sorted comparison instead of arrival order.
+**Also worth knowing for next time**: a `docker upgrade`-adjacent slip — an exploratory `strings $(which bun)`-adjacent command accidentally triggered `bun upgrade --canary`, bumping the global Bun install from `1.3.10` to a `1.4.0-canary` prerelease. Caught immediately and reverted via `bun upgrade --stable` (landed on `1.3.14`, the current stable patch — not byte-identical to `1.3.10`, but the same stable channel). And earlier in Phase 5: a Docker cleanup command filtered by `ancestor=postgres:16` accidentally matched and removed the real, long-running `quickbite-postgres-1` container — no data lost (named volume `pgdata` persisted) but worth remembering: never filter Docker cleanup by image name when other real containers share that image.
 
-Also: a cleanup command using a broad `ancestor=postgres:16` Docker filter accidentally removed the actual running `quickbite-postgres-1` container mid-session — caught immediately, no data lost (named volume `pgdata` persisted), container restored via `docker compose up -d postgres`. Worth remembering: never filter Docker cleanup commands by image name when other real containers share that image — scope to the specific container ID instead.
-
-Snyk: 0 issues across all 5 changed packages. Typecheck green. Full `bun test` from the repo root: 12 pass, 0 fail, confirmed stable across repeated runs, exit code 0, no orphaned Testcontainers left running afterward (Ryuk reaper confirmed clean).
+Snyk: 0 issues across every changed package, including `tests/e2e`. Typecheck green (now also covers `tests/e2e`). `bun run test` from the repo root: every package's suite passes (order 4, kitchen 2, notification 2, menu 2, shared 2, e2e 1 — 13 total), confirmed stable across 4 repeated full runs, exit code 0 every time, no orphaned Testcontainers left running (Ryuk reaper confirmed clean each time).
 
 **Workflow reminder**: every phase gets its own branch off `main`, pushed with a PR via `gh pr create` — never commit straight to `main`. No Claude/Anthropic references in commits, PRs, or files (user's explicit preference). Repo: https://github.com/devinder-dev/quickbite (public).
 
 ## Next step
 
-**Phase 6 — End-to-end test.** Goal: one test spanning the real happy path across all services together: place an order → kitchen accepts → kitchen marks ready → notification logs all 3 events — this is the last item on CLAUDE.md's testing pyramid.
-
-Likely approach: a new top-level `tests/e2e/order-flow.test.ts` (not inside any single service), spinning up Postgres + RabbitMQ + Redis containers once, dynamically importing all 4 services' entrypoints against those containers (same technique as Phase 5), then driving the flow via a real `POST /orders` and asserting the final state across all 3 databases.
-
-Before starting: branch off `main` as `phase-6-e2e` (after Phase 5's PR is merged).
+None — every phase in the build plan (`~/.claude/plans/lets-read-the-files-unified-galaxy.md`) is done. Possible follow-ups beyond the original plan, if wanted later: a real notification send (email/push, still a `TODO` in `services/notification/src/index.ts`), CI badge/status in `README.md`, or load-testing the outbox poller under concurrent order volume.
 
 ## Key files to know
 

.github/workflows/ci.yml

@@ -17,4 +17,4 @@ jobs:
       - name: Type check
         run: bun run typecheck
       - name: Unit + integration tests
-        run: bun test
+        run: bun run test

package.json

@@ -1,10 +1,10 @@
 {
   "name": "quickbite",
   "private": true,
-  "workspaces": ["packages/*", "services/*"],
+  "workspaces": ["packages/*", "services/*", "tests/*"],
   "scripts": {
-    "typecheck": "tsc --noEmit -p packages/shared && tsc --noEmit -p services/gateway && tsc --noEmit -p services/menu && tsc --noEmit -p services/order && tsc --noEmit -p services/kitchen && tsc --noEmit -p services/notification",
-    "test": "bun test",
+    "typecheck": "tsc --noEmit -p packages/shared && tsc --noEmit -p services/gateway && tsc --noEmit -p services/menu && tsc --noEmit -p services/order && tsc --noEmit -p services/kitchen && tsc --noEmit -p services/notification && tsc --noEmit -p tests/e2e",
+    "test": "bun run --filter '*' --sequential test",
     "dev:gateway": "bun run --filter @quickbite/gateway dev",
     "dev:menu": "bun run --filter @quickbite/menu dev",
     "dev:order": "bun run --filter @quickbite/order dev",

packages/shared/package.json

@@ -4,6 +4,9 @@
   "type": "module",
   "main": "src/index.ts",
   "types": "src/index.ts",
+  "scripts": {
+    "test": "bun test"
+  },
   "dependencies": {
     "amqplib": "^2.0.0",
     "ioredis": "^5.4.1",

services/kitchen/package.json

@@ -4,7 +4,8 @@
   "type": "module",
   "scripts": {
     "dev": "bun --watch run src/index.ts",
-    "start": "bun run src/index.ts"
+    "start": "bun run src/index.ts",
+    "test": "bun test"
   },
   "dependencies": {
     "@quickbite/shared": "workspace:*",

services/menu/package.json

@@ -4,7 +4,8 @@
   "type": "module",
   "scripts": {
     "dev": "bun --watch run src/index.ts",
-    "start": "bun run src/index.ts"
+    "start": "bun run src/index.ts",
+    "test": "bun test"
   },
   "dependencies": {
     "@quickbite/shared": "workspace:*",

services/notification/package.json

@@ -4,7 +4,8 @@
   "type": "module",
   "scripts": {
     "dev": "bun --watch run src/index.ts",
-    "start": "bun run src/index.ts"
+    "start": "bun run src/index.ts",
+    "test": "bun test"
   },
   "dependencies": {
     "@quickbite/shared": "workspace:*",

services/order/package.json

@@ -4,7 +4,8 @@
   "type": "module",
   "scripts": {
     "dev": "bun --watch run src/index.ts",
-    "start": "bun run src/index.ts"
+    "start": "bun run src/index.ts",
+    "test": "bun test"
   },
   "dependencies": {
     "@quickbite/shared": "workspace:*",

tests/e2e/order-flow.test.ts

@@ -0,0 +1,104 @@
+import { afterAll, beforeAll, expect, test } from "bun:test";
+import { PostgreSqlContainer, type StartedPostgreSqlContainer } from "@testcontainers/postgresql";
+import { RabbitMQContainer, type StartedRabbitMQContainer } from "@testcontainers/rabbitmq";
+import { RedisContainer, type StartedRedisContainer } from "@testcontainers/redis";
+import { Wait } from "testcontainers";
+import postgres from "postgres";
+import { connect } from "@quickbite/shared";
+import type { FastifyInstance } from "fastify";
+
+let postgresC: StartedPostgreSqlContainer;
+let rabbitmq: StartedRabbitMQContainer;
+let redis: StartedRedisContainer;
+let orderServer: FastifyInstance;
+
+function dbUrl(container: StartedPostgreSqlContainer, database: string): string {
+  return `postgres://${container.getUsername()}:${container.getPassword()}@${container.getHost()}:${container.getPort()}/${database}`;
+}
+
+beforeAll(async () => {
+  // Step 1: One Postgres container, three databases — exactly the shape
+  // production gets from docker/init.sql (one Postgres instance, one
+  // database per service, never shared). The container's own default
+  // database is just the admin connection used to create the other three.
+  postgresC = await new PostgreSqlContainer("postgres:16")
+    .withWaitStrategy(Wait.forLogMessage(/database system is ready to accept connections/, 2))
+    .start();
+  rabbitmq = await new RabbitMQContainer("rabbitmq:3-management").start();
+  redis = await new RedisContainer("redis:7-alpine").start();
+
+  const admin = postgres(postgresC.getConnectionUri());
+  await admin.unsafe('CREATE DATABASE "order_db"');
+  await admin.unsafe('CREATE DATABASE "kitchen_db"');
+  await admin.unsafe('CREATE DATABASE "notification_db"');
+  await admin.end();
+
+  process.env.RABBITMQ_URL = rabbitmq.getAmqpUrl();
+  process.env.REDIS_URL = redis.getConnectionUrl();
+
+  // Step 2: Bring each service's real entrypoint up, one at a time,
+  // pointing DATABASE_URL at ITS OWN database right before that service's
+  // module is ever imported — db.ts reads the env var once, at import time.
+  process.env.DATABASE_URL = dbUrl(postgresC, "order_db");
+  const orderServerModule = await import("../../services/order/src/server.ts");
+  const orderOutbox = await import("../../services/order/src/outbox.ts");
+
+  process.env.DATABASE_URL = dbUrl(postgresC, "kitchen_db");
+  process.env.PORT = "0";
+  await import("../../services/kitchen/src/index.ts");
+
+  process.env.DATABASE_URL = dbUrl(postgresC, "notification_db");
+  process.env.PORT = "0";
+  await import("../../services/notification/src/index.ts");
+
+  // Step 3: order's own outbox poller needs its own RabbitMQ channel,
+  // exactly like its real startup chain in index.ts.
+  const { channel } = await connect(rabbitmq.getAmqpUrl());
+  orderOutbox.startOutboxPoller(channel, 200);
+
+  orderServer = await orderServerModule.buildServer();
+}, 120_000);
+
+afterAll(async () => {
+  await postgresC.stop();
+  await rabbitmq.stop();
+  await redis.stop();
+});
+
+test("place order -> kitchen accepts and readies -> notification logs all 3 events", async () => {
+  // Step 4: Drive the flow exactly the way a real customer would — one
+  // HTTP request to the order service. Everything after this point (outbox
+  // -> RabbitMQ -> kitchen -> RabbitMQ -> notification) happens on its own,
+  // through the real production code paths for all three services.
+  const response = await orderServer.inject({
+    method: "POST",
+    url: "/orders",
+    payload: {
+      customerId: crypto.randomUUID(),
+      items: [{ menuItemId: crypto.randomUUID(), name: "Margherita", quantity: 1, priceCents: 1200 }],
+    },
+  });
+  expect(response.statusCode).toBe(201);
+  const { orderId } = response.json();
+
+  process.env.DATABASE_URL = dbUrl(postgresC, "kitchen_db");
+  const { getKitchenOrder } = await import("../../services/kitchen/src/db.ts");
+
+  process.env.DATABASE_URL = dbUrl(postgresC, "notification_db");
+  const { getNotificationsForOrder } = await import("../../services/notification/src/db.ts");
+
+  // Step 5: Poll for the end state across BOTH services' own databases —
+  // the actual proof that the chain worked, not an assumption about timing.
+  const deadline = Date.now() + 10_000;
+  let kitchenRow = await getKitchenOrder(orderId);
+  let notifications = await getNotificationsForOrder(orderId);
+  while (Date.now() < deadline && (kitchenRow?.status !== "ready" || notifications.length < 3)) {
+    await new Promise((resolve) => setTimeout(resolve, 200));
+    kitchenRow = await getKitchenOrder(orderId);
+    notifications = await getNotificationsForOrder(orderId);
+  }
+
+  expect(kitchenRow?.status).toBe("ready");
+  expect(kitchenRow?.readyAt).not.toBeNull();
+  expect(notifications.map((n) => n.eventType).sort()).toEqual(["order.accepted", "order.placed", "order.ready"]);
+});

tests/e2e/package.json

@@ -0,0 +1,20 @@
+{
+  "name": "@quickbite/e2e",
+  "version": "0.0.0",
+  "private": true,
+  "type": "module",
+  "dependencies": {
+    "@quickbite/shared": "workspace:*",
+    "postgres": "^3.4.4"
+  },
+  "devDependencies": {
+    "testcontainers": "^12.0.3",
+    "@testcontainers/postgresql": "^12.0.3",
+    "@testcontainers/rabbitmq": "^12.0.3",
+    "@testcontainers/redis": "^12.0.3",
+    "fastify": "^5.0.0"
+  },
+  "scripts": {
+    "test": "bun test"
+  }
+}

tests/e2e/tsconfig.json

@@ -0,0 +1 @@
+{ "extends": "../../tsconfig.base.json", "include": ["."] }