diff --git a/.claude/CONTEXT.md b/.claude/CONTEXT.md index 2b7cf8b..61c3e4f 100644 --- a/.claude/CONTEXT.md +++ b/.claude/CONTEXT.md @@ -24,32 +24,23 @@ Full plan: `~/.claude/plans/lets-read-the-files-unified-galaxy.md` | 3 | Redis idempotency — replace in-memory Idempotency class | ✅ Done | | 4 | Retry-with-backoff in mq.ts | ✅ Done | | 5 | Testcontainers integration tests | ✅ Done | -| 6 | End-to-end test | ⏳ Pending | +| 6 | End-to-end test | ✅ Done | ## Current state -**Phase 5 complete, on branch `phase-5-testcontainers` (not yet merged).** Every service now has a real Testcontainers-backed integration test (no mocks for Postgres/RabbitMQ/Redis): `services/order/tests/place-order.test.ts`, `services/kitchen/tests/order-events.test.ts`, `services/notification/tests/order-events.test.ts`, `services/menu/tests/menu.test.ts`. Root `bunfig.toml` sets `[test] timeout = 60000` (container startup exceeds bun's 5s default). +**ALL PHASES COMPLETE (0 through 6) — the build plan is finished.** Phase 6 (`tests/e2e/order-flow.test.ts`, new `@quickbite/e2e` workspace package) wires order, kitchen, and notification together in one test: one Postgres container hosting all 3 databases (`order_db`/`kitchen_db`/`notification_db`, created via raw `CREATE DATABASE` statements — mirroring `docker/init.sql`'s shape), one shared RabbitMQ + Redis container, each service's real entrypoint dynamically imported in sequence (same env-var + dynamic-import technique as Phase 5). A real `POST /orders` via `server.inject()` drives the whole chain; the test polls both kitchen's and notification's own databases until the full happy path completes. -**Key technique**: every service's `db.ts` reads `DATABASE_URL`/`RABBITMQ_URL`/`REDIS_URL` from `process.env` at module-load time (top-level `await ensureSchema()`). Static imports are hoisted before test code runs, so each test sets env vars in `beforeAll` *then* uses a dynamic `await import(...)` to load the module — no service source changes needed for this part. Order's route handler did need one small refactor: `buildServer()` was extracted out of `index.ts` into its own `services/order/src/server.ts`, so tests can `server.inject()` against it without triggering `index.ts`'s real `connect()`+`listen()` startup chain. Kitchen/notification's tests don't need this — they dynamically import the whole `index.ts` (with `PORT=0` for an ephemeral port) precisely because they DO want the real consumer wiring to run. +**The one real architectural problem this phase surfaced, and the actual fix**: running `bun test` from the repo root executes every `*.test.ts` file in ALL packages within a SINGLE OS process by default. Since every service's `db.ts` reads `DATABASE_URL` from `process.env` once at module-load time, and Bun caches dynamically-imported modules by absolute path, a later test file calling `await import("../../services/order/src/server.ts")` could receive an already-cached module instance still pointing at an EARLIER test file's (already-stopped) Testcontainers Postgres. Tried Bun's `--isolate` flag (gives each file a fresh module/global context) — it didn't fully fix this, because `process.env` is OS-process-wide regardless of `--isolate`, and it also silently stopped honoring `bunfig.toml`'s test timeout, causing new failures. The actual fix: every package that has tests now has its own `"test": "bun test"` script, and the root script became `bun run --filter '*' --sequential test` — Bun's workspace filter spawns each package's tests as a genuinely separate OS process, which is real isolation, not just module-context isolation. Root `package.json`, every relevant `package.json`, and `.github/workflows/ci.yml`'s test step were updated to match. -**Three real bugs found while getting these tests to actually pass, not implementation choices**: -1. `@testcontainers/postgresql`'s default wait strategy ANDs `Wait.forHealthCheck()` with `Wait.forListeningPorts()` — but the plain `postgres:16` image has no Docker `HEALTHCHECK` (that's exactly why `docker-compose.yml` manually adds a `pg_isready` one), so it hung forever waiting for a health status that would never appear. Fixed in every Postgres-using test by overriding with `Wait.forLogMessage(/database system is ready to accept connections/, 2)`. -2. `RecoveringChannelModel` (amqplib's recovery wrapper, Phase 2) emits its own `'error'` event separate from `'disconnect'` — an EventEmitter `'error'` with no listener crashes the whole process in Node/Bun. `connect()` in `packages/shared/src/mq.ts` only listened for `'connect'`/`'disconnect'`/`'reconnect-failed'`; added an `'error'` handler. This is a real production hardening fix, not just a test fix — a service could have crashed on certain connection failures before this. -3. Notification's first integration test initially asserted exact event-arrival order (`["order.placed", "order.accepted", "order.ready"]`) — flaky, because the test publishes all 3 events back-to-back with no delay (unlike the real flow, where kitchen naturally waits ~3s between accepted and ready), so concurrent handler completions can race. Fixed to assert presence via a sorted comparison instead of arrival order. +**Also worth knowing for next time**: a `docker upgrade`-adjacent slip — an exploratory `strings $(which bun)`-adjacent command accidentally triggered `bun upgrade --canary`, bumping the global Bun install from `1.3.10` to a `1.4.0-canary` prerelease. Caught immediately and reverted via `bun upgrade --stable` (landed on `1.3.14`, the current stable patch — not byte-identical to `1.3.10`, but the same stable channel). And earlier in Phase 5: a Docker cleanup command filtered by `ancestor=postgres:16` accidentally matched and removed the real, long-running `quickbite-postgres-1` container — no data lost (named volume `pgdata` persisted) but worth remembering: never filter Docker cleanup by image name when other real containers share that image. -Also: a cleanup command using a broad `ancestor=postgres:16` Docker filter accidentally removed the actual running `quickbite-postgres-1` container mid-session — caught immediately, no data lost (named volume `pgdata` persisted), container restored via `docker compose up -d postgres`. Worth remembering: never filter Docker cleanup commands by image name when other real containers share that image — scope to the specific container ID instead. - -Snyk: 0 issues across all 5 changed packages. Typecheck green. Full `bun test` from the repo root: 12 pass, 0 fail, confirmed stable across repeated runs, exit code 0, no orphaned Testcontainers left running afterward (Ryuk reaper confirmed clean). +Snyk: 0 issues across every changed package, including `tests/e2e`. Typecheck green (now also covers `tests/e2e`). `bun run test` from the repo root: every package's suite passes (order 4, kitchen 2, notification 2, menu 2, shared 2, e2e 1 — 13 total), confirmed stable across 4 repeated full runs, exit code 0 every time, no orphaned Testcontainers left running (Ryuk reaper confirmed clean each time). **Workflow reminder**: every phase gets its own branch off `main`, pushed with a PR via `gh pr create` — never commit straight to `main`. No Claude/Anthropic references in commits, PRs, or files (user's explicit preference). Repo: https://github.com/devinder-dev/quickbite (public). ## Next step -**Phase 6 — End-to-end test.** Goal: one test spanning the real happy path across all services together: place an order → kitchen accepts → kitchen marks ready → notification logs all 3 events — this is the last item on CLAUDE.md's testing pyramid. - -Likely approach: a new top-level `tests/e2e/order-flow.test.ts` (not inside any single service), spinning up Postgres + RabbitMQ + Redis containers once, dynamically importing all 4 services' entrypoints against those containers (same technique as Phase 5), then driving the flow via a real `POST /orders` and asserting the final state across all 3 databases. - -Before starting: branch off `main` as `phase-6-e2e` (after Phase 5's PR is merged). +None — every phase in the build plan (`~/.claude/plans/lets-read-the-files-unified-galaxy.md`) is done. Possible follow-ups beyond the original plan, if wanted later: a real notification send (email/push, still a `TODO` in `services/notification/src/index.ts`), CI badge/status in `README.md`, or load-testing the outbox poller under concurrent order volume. ## Key files to know diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c423bd8..ee1de78 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,4 +17,4 @@ jobs: - name: Type check run: bun run typecheck - name: Unit + integration tests - run: bun test + run: bun run test diff --git a/bun.lock b/bun.lock index 626e999..a4ee0ed 100644 --- a/bun.lock +++ b/bun.lock @@ -86,6 +86,21 @@ "testcontainers": "^12.0.3", }, }, + "tests/e2e": { + "name": "@quickbite/e2e", + "version": "0.0.0", + "dependencies": { + "@quickbite/shared": "workspace:*", + "postgres": "^3.4.4", + }, + "devDependencies": { + "@testcontainers/postgresql": "^12.0.3", + "@testcontainers/rabbitmq": "^12.0.3", + "@testcontainers/redis": "^12.0.3", + "fastify": "^5.0.0", + "testcontainers": "^12.0.3", + }, + }, }, "packages": { "@balena/dockerignore": ["@balena/dockerignore@1.0.2", "", {}, "sha512-wMue2Sy4GAVTk6Ic4tJVcnfdau+gx2EnG7S+uAEe+TWJFqE4YoWN4/H8MSLj4eYJKxGg26lZwboEniNiNwZQ6Q=="], @@ -140,6 +155,8 @@ "@protobufjs/utf8": ["@protobufjs/utf8@1.1.1", "", {}, "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg=="], + "@quickbite/e2e": ["@quickbite/e2e@workspace:tests/e2e"], + "@quickbite/gateway": ["@quickbite/gateway@workspace:services/gateway"], "@quickbite/kitchen": ["@quickbite/kitchen@workspace:services/kitchen"], diff --git a/package.json b/package.json index f15ce3e..313242a 100644 --- a/package.json +++ b/package.json @@ -1,10 +1,10 @@ { "name": "quickbite", "private": true, - "workspaces": ["packages/*", "services/*"], + "workspaces": ["packages/*", "services/*", "tests/*"], "scripts": { - "typecheck": "tsc --noEmit -p packages/shared && tsc --noEmit -p services/gateway && tsc --noEmit -p services/menu && tsc --noEmit -p services/order && tsc --noEmit -p services/kitchen && tsc --noEmit -p services/notification", - "test": "bun test", + "typecheck": "tsc --noEmit -p packages/shared && tsc --noEmit -p services/gateway && tsc --noEmit -p services/menu && tsc --noEmit -p services/order && tsc --noEmit -p services/kitchen && tsc --noEmit -p services/notification && tsc --noEmit -p tests/e2e", + "test": "bun run --filter '*' --sequential test", "dev:gateway": "bun run --filter @quickbite/gateway dev", "dev:menu": "bun run --filter @quickbite/menu dev", "dev:order": "bun run --filter @quickbite/order dev", diff --git a/packages/shared/package.json b/packages/shared/package.json index 637ac2b..f9882fb 100644 --- a/packages/shared/package.json +++ b/packages/shared/package.json @@ -4,6 +4,9 @@ "type": "module", "main": "src/index.ts", "types": "src/index.ts", + "scripts": { + "test": "bun test" + }, "dependencies": { "amqplib": "^2.0.0", "ioredis": "^5.4.1", diff --git a/services/kitchen/package.json b/services/kitchen/package.json index 1056156..8d50f28 100644 --- a/services/kitchen/package.json +++ b/services/kitchen/package.json @@ -4,7 +4,8 @@ "type": "module", "scripts": { "dev": "bun --watch run src/index.ts", - "start": "bun run src/index.ts" + "start": "bun run src/index.ts", + "test": "bun test" }, "dependencies": { "@quickbite/shared": "workspace:*", diff --git a/services/menu/package.json b/services/menu/package.json index 8981780..79685c7 100644 --- a/services/menu/package.json +++ b/services/menu/package.json @@ -4,7 +4,8 @@ "type": "module", "scripts": { "dev": "bun --watch run src/index.ts", - "start": "bun run src/index.ts" + "start": "bun run src/index.ts", + "test": "bun test" }, "dependencies": { "@quickbite/shared": "workspace:*", diff --git a/services/notification/package.json b/services/notification/package.json index efa72ac..3dab322 100644 --- a/services/notification/package.json +++ b/services/notification/package.json @@ -4,7 +4,8 @@ "type": "module", "scripts": { "dev": "bun --watch run src/index.ts", - "start": "bun run src/index.ts" + "start": "bun run src/index.ts", + "test": "bun test" }, "dependencies": { "@quickbite/shared": "workspace:*", diff --git a/services/order/package.json b/services/order/package.json index 4b8e348..0837185 100644 --- a/services/order/package.json +++ b/services/order/package.json @@ -4,7 +4,8 @@ "type": "module", "scripts": { "dev": "bun --watch run src/index.ts", - "start": "bun run src/index.ts" + "start": "bun run src/index.ts", + "test": "bun test" }, "dependencies": { "@quickbite/shared": "workspace:*", diff --git a/tests/e2e/order-flow.test.ts b/tests/e2e/order-flow.test.ts new file mode 100644 index 0000000..414763b --- /dev/null +++ b/tests/e2e/order-flow.test.ts @@ -0,0 +1,104 @@ +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { PostgreSqlContainer, type StartedPostgreSqlContainer } from "@testcontainers/postgresql"; +import { RabbitMQContainer, type StartedRabbitMQContainer } from "@testcontainers/rabbitmq"; +import { RedisContainer, type StartedRedisContainer } from "@testcontainers/redis"; +import { Wait } from "testcontainers"; +import postgres from "postgres"; +import { connect } from "@quickbite/shared"; +import type { FastifyInstance } from "fastify"; + +let postgresC: StartedPostgreSqlContainer; +let rabbitmq: StartedRabbitMQContainer; +let redis: StartedRedisContainer; +let orderServer: FastifyInstance; + +function dbUrl(container: StartedPostgreSqlContainer, database: string): string { + return `postgres://${container.getUsername()}:${container.getPassword()}@${container.getHost()}:${container.getPort()}/${database}`; +} + +beforeAll(async () => { + // Step 1: One Postgres container, three databases — exactly the shape + // production gets from docker/init.sql (one Postgres instance, one + // database per service, never shared). The container's own default + // database is just the admin connection used to create the other three. + postgresC = await new PostgreSqlContainer("postgres:16") + .withWaitStrategy(Wait.forLogMessage(/database system is ready to accept connections/, 2)) + .start(); + rabbitmq = await new RabbitMQContainer("rabbitmq:3-management").start(); + redis = await new RedisContainer("redis:7-alpine").start(); + + const admin = postgres(postgresC.getConnectionUri()); + await admin.unsafe('CREATE DATABASE "order_db"'); + await admin.unsafe('CREATE DATABASE "kitchen_db"'); + await admin.unsafe('CREATE DATABASE "notification_db"'); + await admin.end(); + + process.env.RABBITMQ_URL = rabbitmq.getAmqpUrl(); + process.env.REDIS_URL = redis.getConnectionUrl(); + + // Step 2: Bring each service's real entrypoint up, one at a time, + // pointing DATABASE_URL at ITS OWN database right before that service's + // module is ever imported — db.ts reads the env var once, at import time. + process.env.DATABASE_URL = dbUrl(postgresC, "order_db"); + const orderServerModule = await import("../../services/order/src/server.ts"); + const orderOutbox = await import("../../services/order/src/outbox.ts"); + + process.env.DATABASE_URL = dbUrl(postgresC, "kitchen_db"); + process.env.PORT = "0"; + await import("../../services/kitchen/src/index.ts"); + + process.env.DATABASE_URL = dbUrl(postgresC, "notification_db"); + process.env.PORT = "0"; + await import("../../services/notification/src/index.ts"); + + // Step 3: order's own outbox poller needs its own RabbitMQ channel, + // exactly like its real startup chain in index.ts. + const { channel } = await connect(rabbitmq.getAmqpUrl()); + orderOutbox.startOutboxPoller(channel, 200); + + orderServer = await orderServerModule.buildServer(); +}, 120_000); + +afterAll(async () => { + await postgresC.stop(); + await rabbitmq.stop(); + await redis.stop(); +}); + +test("place order -> kitchen accepts and readies -> notification logs all 3 events", async () => { + // Step 4: Drive the flow exactly the way a real customer would — one + // HTTP request to the order service. Everything after this point (outbox + // -> RabbitMQ -> kitchen -> RabbitMQ -> notification) happens on its own, + // through the real production code paths for all three services. + const response = await orderServer.inject({ + method: "POST", + url: "/orders", + payload: { + customerId: crypto.randomUUID(), + items: [{ menuItemId: crypto.randomUUID(), name: "Margherita", quantity: 1, priceCents: 1200 }], + }, + }); + expect(response.statusCode).toBe(201); + const { orderId } = response.json(); + + process.env.DATABASE_URL = dbUrl(postgresC, "kitchen_db"); + const { getKitchenOrder } = await import("../../services/kitchen/src/db.ts"); + + process.env.DATABASE_URL = dbUrl(postgresC, "notification_db"); + const { getNotificationsForOrder } = await import("../../services/notification/src/db.ts"); + + // Step 5: Poll for the end state across BOTH services' own databases — + // the actual proof that the chain worked, not an assumption about timing. + const deadline = Date.now() + 10_000; + let kitchenRow = await getKitchenOrder(orderId); + let notifications = await getNotificationsForOrder(orderId); + while (Date.now() < deadline && (kitchenRow?.status !== "ready" || notifications.length < 3)) { + await new Promise((resolve) => setTimeout(resolve, 200)); + kitchenRow = await getKitchenOrder(orderId); + notifications = await getNotificationsForOrder(orderId); + } + + expect(kitchenRow?.status).toBe("ready"); + expect(kitchenRow?.readyAt).not.toBeNull(); + expect(notifications.map((n) => n.eventType).sort()).toEqual(["order.accepted", "order.placed", "order.ready"]); +}); diff --git a/tests/e2e/package.json b/tests/e2e/package.json new file mode 100644 index 0000000..57aef96 --- /dev/null +++ b/tests/e2e/package.json @@ -0,0 +1,20 @@ +{ + "name": "@quickbite/e2e", + "version": "0.0.0", + "private": true, + "type": "module", + "dependencies": { + "@quickbite/shared": "workspace:*", + "postgres": "^3.4.4" + }, + "devDependencies": { + "testcontainers": "^12.0.3", + "@testcontainers/postgresql": "^12.0.3", + "@testcontainers/rabbitmq": "^12.0.3", + "@testcontainers/redis": "^12.0.3", + "fastify": "^5.0.0" + }, + "scripts": { + "test": "bun test" + } +} diff --git a/tests/e2e/tsconfig.json b/tests/e2e/tsconfig.json new file mode 100644 index 0000000..d3a506c --- /dev/null +++ b/tests/e2e/tsconfig.json @@ -0,0 +1 @@ +{ "extends": "../../tsconfig.base.json", "include": ["."] }