From 638bcfd74ce9e445f720064cdf4de00bcf4c8bee Mon Sep 17 00:00:00 2001 From: Devinder Singh Date: Sun, 25 Oct 2026 14:11:13 +0100 Subject: [PATCH 1/2] docs: record stack decisions (Fastify, dashboard) and update team context Adds decision records 0001 (Fastify for the API) and 0002 (Vite, React, Tailwind, React Router, axios for the dashboard). Updates docs/CONTEXT.md with the accepted foundation: walking-skeleton plan, lease-based queue, Standard Webhooks signing, retry schedule, data model v1, API v1, deploy over SSH with a forced command, S3 backups with Terraform, the HTTP to HTTPS redirect and DNS for cdn and grafana. README now lists the real stack and address names. --- README.md | 15 ++- docs/CONTEXT.md | 124 ++++++++++++------- docs/decisions/0001-web-framework-fastify.md | 50 ++++++++ docs/decisions/0002-dashboard-stack.md | 39 ++++++ 4 files changed, 179 insertions(+), 49 deletions(-) create mode 100644 docs/decisions/0001-web-framework-fastify.md create mode 100644 docs/decisions/0002-dashboard-stack.md diff --git a/README.md b/README.md index a82ae32..30c47aa 100644 --- a/README.md +++ b/README.md @@ -21,15 +21,18 @@ Relay sits in the middle: | Service | Address | Status | |---|---|---| -| API | https://api.myrelay.tech | In progress | -| Dashboard (static files) | https://cdn.myrelay.tech | Planned | -| Metrics and logs | https://grafana.myrelay.tech | Planned | +| API | https://api.myrelay.tech | In progress (test app live with HTTPS) | +| Relay dashboard (static files) | https://cdn.myrelay.tech | Planned | +| Ops dashboard: metrics and logs (Grafana) | https://grafana.myrelay.tech | Planned | ## Stack -Bun + TypeScript · PostgreSQL (also the job queue, via `SKIP LOCKED`) · k3s on our own VPS · -Traefik + cert-manager (Let's Encrypt) · GitHub Actions → GHCR · Prometheus, Grafana, Loki · -Terraform for the backup bucket and cluster add-ons. +Bun 1.4 + TypeScript + Fastify · React + Vite + Tailwind dashboard · +PostgreSQL 18 (also the job queue, via `SKIP LOCKED`) · +k3s on our own VPS · Traefik + cert-manager (Let's Encrypt) · GitHub Actions → GHCR · +Prometheus, Grafana, Loki · Amazon S3 for backups, with the bucket made by Terraform. + +Why we chose what we chose: [`docs/decisions/`](docs/decisions/). ## Team diff --git a/docs/CONTEXT.md b/docs/CONTEXT.md index f798ceb..64e1f28 100644 --- a/docs/CONTEXT.md +++ b/docs/CONTEXT.md @@ -10,10 +10,12 @@ Chas Academy's "Avancerad fullstackutveckling, test och DevOps". - **Team:** Devinder Singh (backend + infrastructure) and Abdikhani Mohamed (frontend + quality). Every PR is reviewed by the other person. Both must be able to explain the whole system. -- **Dates:** Tenta 4 Wed 28 Oct · G complete target Fri 30 Oct · **oral defense Wed 4 Nov +- **Dates:** Tenta 4 Wed 28 Oct · G target around Sat 31 Oct · **oral defense Wed 4 Nov (live demo)** · code submission Sun 8 Nov. -- **Strategy:** build all G requirements first, then VG (2+ replicas, zero-downtime deploys, - autoscaling shown under load). +- **Strategy:** a **walking skeleton** first (a tiny API that the pipeline deploys to production, + already with 2 pods and zero-downtime rollouts), then the features on top. All G requirements + before VG. Quality and understanding come before speed: dates are targets, and on 4 Nov we show + what is done and explain it well. ## How we work @@ -28,8 +30,11 @@ Chas Academy's "Avancerad fullstackutveckling, test och DevOps". `ssh relay` / `kubectl get` / `gh api`), instead of relying on "done". - **Commands that ask questions** (`adduser`, `passwd`, `ufw enable`, a first `sudo`) run alone, never inside a pasted block. Pasted lines get swallowed as answers (real incident, 22 Oct). -- **Docs over memory.** Check current official docs for Bun, Hono, cert-manager, k3s and so on +- **Docs over memory.** Check current official docs for Bun, Fastify, cert-manager, k3s and so on instead of guessing APIs. +- **By code, not by clicking.** Manifests, scripts, Terraform and `gh` wherever an API exists. + Web panels only where none does (one.com VPS, get.tech DNS). +- **Decisions are written down** in `docs/decisions/` (one short record per choice). - **Security scans.** Scan new security-sensitive code (auth, SQL, signing, webhooks, user input) with Snyk where available, and fix findings before the PR. @@ -39,53 +44,75 @@ Chas Academy's "Avancerad fullstackutveckling, test och DevOps". - Secrets live only in **GitHub Actions secrets** or **Kubernetes Secrets**. Never in git. - The kubeconfig is an admin password. It stays on the server in `~/.kube/config`. - Ask the other person before any destructive command on the VPS, the database or DNS. -- Never expose to the internet: Postgres (5432), the Kubernetes API (6443), `/metrics`. -- Endpoint URLs from customers must not point to internal or private addresses (SSRF). +- Never expose to the internet: Postgres (5432), the Kubernetes API (6443), `/metrics` + (served on its own port 9090, which no Ingress points to). +- Endpoint URLs from customers must not point to internal, private or our own VPS addresses + (SSRF). Checked when an endpoint is created **and** before every delivery attempt; redirects + are never followed. ## How Relay works -1. A producer sends `POST /events` with its API key. +1. A producer sends `POST /v1/events` with its API key (`Authorization: Bearer relay_live_…`). 2. The API validates it and, in **one transaction**, saves the event plus one **delivery** row - per subscribed endpoint, then answers **202 Accepted**. -3. Worker pods claim due deliveries with `SELECT … FOR UPDATE SKIP LOCKED`. -4. The worker signs the body (HMAC-SHA256, header `Relay-Signature: t=,v1=`, - signed string `t + "." + body`) and POSTs it with a 10 s timeout. -5. 2xx = delivered. Error or timeout = retry with exponential backoff + jitter - (draft: 1 min, 5 min, 30 min, 2 h, 6 h, 12 h). After the last try: **dead**, replayable. -6. At-least-once delivery: each event has a unique id; producers may send `Idempotency-Key`. - -**Data model:** `owners` (id, name, api_key_hash) · `endpoints` (id, owner_id, url, secret, -event_types[], paused_until, created_at) · `events` (id, owner_id, type, payload jsonb, -idempotency_key, created_at; UNIQUE (owner_id, idempotency_key)) · `deliveries` (id, event_id, -endpoint_id, status pending|delivered|dead, attempts, next_attempt_at, last_status_code, -updated_at; INDEX (status, next_attempt_at)) · `attempts` (id, delivery_id, status_code, -duration_ms, error, created_at). - -**API:** `POST /endpoints` · `POST /events` · `GET /deliveries?status=` · `GET /deliveries/:id` -· `POST /deliveries/:id/replay` · `GET /health/live` (no DB check) · `GET /health/ready` -(checks DB) · `GET /metrics` (cluster-internal only). + per matching endpoint (exact event type, or an endpoint listening to `*`), then answers + **202 Accepted**. A repeated `Idempotency-Key` returns the original answer. +3. Worker pods **claim** due deliveries with `FOR UPDATE SKIP LOCKED` and take a 60 s **lease**: + `attempts + 1`, `next_attempt_at = now() + 60 s`, a fresh `lease_id`, commit. No transaction or + connection is held during the HTTP call. +4. The worker checks the URL again (SSRF), signs with **Standard Webhooks** (headers + `webhook-id` = event id, `webhook-timestamp`, `webhook-signature: v1,`), and POSTs with a 10 s timeout. It signs again on **every** attempt. +5. 2xx = delivered. Error, redirect or timeout = retry with exponential backoff and ±20% jitter: + 5 s, 1 min, 5 min, 30 min, 2 h, 6 h, 12 h (8 tries, about 21 h). After the last try: **dead**, + replayable. Tests and the demo use a shorter schedule from an environment variable. +6. The result is saved only `WHERE lease_id = ` (a **fencing token**). A crashed + worker's lease runs out and the row becomes due again by itself. +7. At-least-once delivery: receivers drop duplicates by `webhook-id`. + +**Data model v1** (Postgres 18.6, every id is a UUIDv7): +`owners` (id, name) · `api_keys` (owner_id, prefix, key_hash = SHA-256 UNIQUE, revoked_at) · +`endpoints` (owner_id, name, url, signing_secret `whsec_…`, event_types[] default `{*}`, +disabled_at) · `events` (owner_id, type, payload jsonb object, idempotency_key; UNIQUE (owner_id, +idempotency_key)) · `deliveries` (owner_id, event_id, endpoint_id, status pending|delivered|dead, +attempts, next_attempt_at, lease_id, last_status_code, last_error, delivered_at; UNIQUE (event_id, +endpoint_id); partial INDEX (next_attempt_at) WHERE status = 'pending') · `attempts` +(delivery_id, started_at, duration_ms, status_code, error, response_excerpt). + +**API v1:** `GET /v1/me` · `POST /v1/endpoints` (returns the signing secret once) · +`GET /v1/endpoints` · `POST /v1/events` · `GET /v1/deliveries?status=&endpoint_id=&cursor=` · +`GET /v1/deliveries/:id` · `POST /v1/deliveries/:id/replay` · `GET /health/live` (no DB check) · +`GET /health/ready` (checks DB, 503 while shutting down) · `GET /version` (git SHA) · +`/metrics` on port 9090 only. Errors: `{"error":{"code","message","request_id"}}`. Another +owner's id answers 404. Bodies: JSON only, max 256 KiB. CORS: `https://cdn.myrelay.tech` only. +Owners and API keys are created with a `create-owner` script (no signup page). ## Stack and decisions | Area | Choice | Not chosen | |---|---|---| -| Runtime | Bun 1.4 + TypeScript | Node (more common, but we know Bun) | +| Runtime | Bun **1.4.4**, pinned everywhere (laptop, CI, Docker). 1.4 is a new rewrite in Rust | Node (more common, but we know Bun; it stays our fallback) | +| Web framework | **Fastify 5** ([decision 0001](decisions/0001-web-framework-fastify.md)): `buildApp()` + `listen({ host: "0.0.0.0" })`, pino logs, JSON Schema validation, `@fastify/cors`, `inject()` tests | Hono (native on Bun, but new to us), Express, Elysia | +| Database library | postgres.js behind one file `src/db.ts` | `Bun.sql` (ties the data layer to the new runtime), Drizzle | +| Migrations | dbmate (plain SQL), run by a Kubernetes Job before each rollout. Expand/contract rule | drizzle-kit, an initContainer (would rerun on every scale-up) | | Unit + integration tests | `bun test`; integration against a real Postgres (Docker locally, service container in GitHub Actions) | Testcontainers (no official Bun support), mocked DB | +| Dashboard | **Vite + React + TypeScript, Tailwind CSS v4, React Router, axios** (one client in `src/api.ts`), plain React state with a small refresh loop while a delivery is pending, served by nginx ([decision 0002](decisions/0002-dashboard-stack.md)) | Next.js, plain CSS, `fetch`, TanStack Query | | E2E tests | Playwright, runs on Node in the dashboard folder | Cypress | -| Queue | Postgres `SKIP LOCKED` | RabbitMQ (stretch goal only, as a "doorbell"; dual-write problem) | -| Database | Postgres in k3s, StatefulSet + local-path volume with Retain | Neon / Supabase (brief: everything on our own VPS) | +| Queue | Postgres `SKIP LOCKED` + lease with fencing token | RabbitMQ (stretch goal only, as a "doorbell"; dual-write problem) | +| Signing | Standard Webhooks, re-signed on every attempt | Our own header format | +| IDs | UUIDv7 (built into Postgres 18). Prefixes only on secrets: `relay_live_`, `whsec_` | bigint counters, prefixed ids | +| Database | `postgres:18.6` StatefulSet, volume mounted at `/var/lib/postgresql`, our own StorageClass `local-path-retain` (the default deletes data) | Neon / Supabase (brief: everything on our own VPS) | | Orchestration | k3s v1.36.5 (Traefik, metrics-server, local-path, secrets encryption on) | Docker Compose | -| HTTPS | cert-manager v1.21.2 + Let's Encrypt, staging first | certbot by hand | -| Images | GHCR, tag = git SHA | Docker Hub | -| CI/CD | GitHub Actions: test → build → push → deploy | Argo CD | -| Observability | kube-prometheus-stack + Loki (Grafana Cloud free tier as fallback) | Datadog | -| Backups | Nightly CronJob: pg_dump + gzip → S3-compatible bucket, one tested restore | Volume snapshots | -| IaC | Kubernetes YAML in git; Terraform for the backup bucket, GitHub settings and Helm add-ons (Phase 4) | Terraform for everything (no provider for one.com or get.tech) | -| Scaling / zero downtime | HPA on CPU; RollingUpdate maxUnavailable 0, readiness probe, graceful SIGTERM | KEDA (stretch), blue-green | - -**Still open:** web framework (Hono, Elysia or Express), database library (`Bun.sql`, -postgres.js or Drizzle), how Actions reaches the cluster (SSH vs Kubernetes API), -backup storage (R2, B2 or AWS), final retry schedule, Grafana address. +| HTTPS | cert-manager v1.21.2 + Let's Encrypt, staging first; HTTP → HTTPS redirect in Traefik with `allowACMEByPass` | certbot by hand | +| Images | GHCR (public package), tag = git SHA | Docker Hub | +| CI/CD | GitHub Actions: test → build → push → deploy over SSH as a `deploy` user with a forced command (port 6443 stays closed), then check `/version` | Self-hosted runner, opening 6443, Argo CD | +| Observability | Slim kube-prometheus-stack + Loki + Alloy (Promtail is end-of-life); Grafana Cloud free tier as fallback | Datadog | +| Backups | Nightly CronJob: `pg_dump` → **Amazon S3** (eu-north-1), one timed restore | Volume snapshots, R2, B2 | +| IaC | Kubernetes YAML and scripts in git; **Terraform only for the S3 backup bucket** and its IAM user | Terraform for everything (no provider for one.com or get.tech; Helm via Terraform needs 6443) | +| Scaling / zero downtime | HPA on the API (2–4 pods, no `replicas:` field); workers fixed at 2; RollingUpdate maxUnavailable 0, readiness probe, preStop sleep, graceful SIGTERM | KEDA (stretch), blue-green | + +**Left out on purpose:** rate limiting per IP (Traefik sees every client as `10.42.0.1`), +encrypting the signing-secret column (we protect the backups instead), OpenAPI tooling +(`docs/api.md` + shared types instead). ## Infrastructure facts @@ -96,9 +123,12 @@ backup storage (R2, B2 or AWS), final retry schedule, Grafana address. - **Firewalls:** one.com's network firewall + ufw. ufw allows 22, 80, 443 and the k3s networks 10.42.0.0/16 and 10.43.0.0/16. one.com's firewall allows 22, 80 and 443. - **kubectl:** only on the server (`ssh relay`, then `kubectl …`). Port 6443 stays closed. -- **Domain:** `myrelay.tech`, DNS at get.tech. `api.myrelay.tech` has an A record to the VPS. - **No AAAA records**: the cluster is IPv4-only, and Let's Encrypt prefers IPv6 when it exists. - Planned: `cdn.myrelay.tech` (dashboard), `grafana.myrelay.tech` (name not final). +- **Domain:** `myrelay.tech`, DNS at get.tech. A records for `api`, `cdn` (the Relay dashboard) + and `grafana` (the ops dashboard: metrics and logs) all point to the VPS. **No AAAA records**: + the cluster is IPv4-only, and Let's Encrypt prefers IPv6 when it exists. +- **HTTP → HTTPS:** `k8s/platform/traefik-config.yaml` (a HelmChartConfig) redirects port 80 to + 443 for every host with a 308, except `/.well-known/acme-challenge/`, so certificate checks still + work over plain HTTP. ## Git workflow @@ -113,4 +143,12 @@ backup storage (R2, B2 or AWS), final retry schedule, Grafana address. Phase 0 (server foundation) is done: VPS hardened, k3s, cert-manager with Let's Encrypt issuers, and a temporary hello-world app on https://api.myrelay.tech with a trusted -certificate. Next: Phase 1, the API skeleton (Bun) with tests. +certificate. + +Phase 1 (walking skeleton) has started: HTTP → HTTPS redirect (1.1) and DNS for cdn and +grafana (1.2) are done. Next: 1.3, the API skeleton in `api/` (Bun 1.4.4 + Fastify) with +`/health/live`, `/health/ready`, `/version` and the first `bun test`. + +The plan in order: 1 walking skeleton · 2 database foundation · 3 core API · 4 worker · +5 dashboard on cdn (Abdikhani, in parallel) · 6 backups + Terraform (Abdikhani, in parallel) · +7 metrics and logs (Abdikhani, in parallel) · 8 VG under load · 9 docs and defense · 10 submit. diff --git a/docs/decisions/0001-web-framework-fastify.md b/docs/decisions/0001-web-framework-fastify.md new file mode 100644 index 0000000..4f8cee3 --- /dev/null +++ b/docs/decisions/0001-web-framework-fastify.md @@ -0,0 +1,50 @@ +# 0001. Web framework: Fastify + +- **Status:** Accepted +- **Date:** 2026-10-25 +- **Decided by:** Devinder Singh and Abdikhani Mohamed + +## Context + +The Relay API needs an HTTP framework. Our runtime is **Bun 1.4.4**, pinned because Bun 1.4 is the +first release of Bun rewritten in Rust (August 2026). The API must: + +- route requests and read path parameters (`/v1/deliveries/:id`); +- validate JSON bodies (`POST /v1/events`); +- write JSON logs with a request id on every line; +- limit body size (256 KiB) and allow CORS from `https://cdn.myrelay.tech`; +- shut down gracefully on SIGTERM, which is needed for zero-downtime rollouts; +- be testable without starting a server. + +Both of us are examined individually and must be able to explain every line. + +## Options considered + +| Option | For | Against | +|---|---|---| +| **Fastify 5** | Both of us have used it. JSON logging (pino) with request ids, body validation (JSON Schema), body limit and request ids are built in. `inject()` tests without a server. Runs on Node unchanged. | Built for Node's `http` module, so on Bun it runs through Bun's Node-compatibility layer, which Bun 1.4 rewrote. Not tested by us on 1.4.4 yet. Listens on `localhost` by default. More concepts (plugins, encapsulation, hooks). | +| Hono 4 | Runs on Bun's native server (`Bun.serve`). Web-standard Request/Response. CORS, body limit and request id built in. `app.request()` tests. | New for both of us. Logging and validation must be added (own logger, Zod). | +| Express | Most widely known. | Older design without web-standard Request/Response; needs extra packages for the features above. | +| Elysia | Fast, Bun-first. | Runs only on Bun; more "magic" that is harder to explain. | + +## Decision + +We use **Fastify 5** on Bun 1.4.4. + +Familiarity was the deciding factor: we can write, review and explain Fastify code with +confidence. Its built-in pino logging and JSON Schema validation also cover two needs we +would otherwise build ourselves. + +## Consequences + +- **Extra check in step 1.3:** before building on it, we run the app and its tests on Bun 1.4.4 and + put it under some load, to confirm Bun's Node-compatibility layer behaves. +- **Fallback:** if Bun causes problems, the same Fastify code runs on Node without changes. +- **Listen address:** the server must listen with `host: "0.0.0.0"`, because the default + (`localhost`) can't be reached from outside a container. +- **App structure:** `buildApp()` creates the app (one fresh instance per test, used with `inject()`); + `server.ts` only calls `listen()` and handles SIGTERM with `app.close()`. +- **Logging:** pino, built into Fastify, writing JSON to stdout. We keep our agreed field names + (`request_id`, `event_id`, `delivery_id`, `endpoint_id`, `status_code`, `duration_ms`). +- **Validation:** JSON Schema on each route. A bad body gets a 400 automatically. +- **CORS:** the `@fastify/cors` plugin, limited to `https://cdn.myrelay.tech` (and localhost in CI). diff --git a/docs/decisions/0002-dashboard-stack.md b/docs/decisions/0002-dashboard-stack.md new file mode 100644 index 0000000..9dfe78b --- /dev/null +++ b/docs/decisions/0002-dashboard-stack.md @@ -0,0 +1,39 @@ +# 0002. Dashboard stack: Vite, React, Tailwind, React Router, axios + +- **Status:** Accepted +- **Date:** 2026-10-25 +- **Decided by:** Devinder Singh and Abdikhani Mohamed + +## Context + +The Relay dashboard is a small single-page app on `https://cdn.myrelay.tech`, served as static +files. It has four screens (Connect, Delivery Log, Delivery Detail with Replay, Endpoints) and +talks to `https://api.myrelay.tech` with the owner's API key. The Delivery Detail screen should +update by itself while a delivery is pending, so the live demo can show retries happening. + +Both of us know HTML, CSS, Tailwind, React, TypeScript, fetch and axios. We are examined +individually, so the code must be easy for both of us to read and explain. + +## Decision + +| Part | Choice | Not chosen, and why | +|---|---|---| +| Build tool | **Vite** + React + TypeScript | Next.js: adds a server; the brief wants static files on `cdn.` | +| Styling | **Tailwind CSS v4** (`@tailwindcss/vite`) | Plain CSS: fine, but we both know Tailwind and it keeps spacing and colours consistent | +| Pages | **React Router** | Hand-made screen switching: no real URLs, so `/deliveries/:id` can't be shared or reloaded | +| API calls | **axios**, one shared client in `src/api.ts` | `fetch`: also fine, but axios reads more clearly to us (base URL, auth header and error handling set once) | +| Server data | **Plain React state** (`useState` + `useEffect`), with a small refresh loop on the detail screen while a delivery is pending | TanStack Query: powerful, but adds concepts we don't need for four screens | +| Serving | **nginx** (unprivileged image) with the built files | — | + +## Consequences + +- `src/api.ts` is the only file that talks to the API. It creates one axios client with + `baseURL` from `VITE_API_URL`, a timeout, and an interceptor that adds + `Authorization: Bearer ` (the key is kept in `sessionStorage`). +- The detail screen reloads every 2 seconds only while the delivery is `pending`, and stops when + it leaves the page. Replay triggers one reload. +- nginx must send unknown paths to `index.html`, so reloading `/deliveries/:id` works with + React Router. +- The API must allow CORS from `https://cdn.myrelay.tech` (and localhost in CI). +- The backend keeps using `fetch` in the worker to deliver webhooks: it needs + `redirect: "manual"` and `AbortSignal.timeout()`, and runs on the server, not in the browser. From b06a2755a71841fb9cb417fabedbf4065730c822 Mon Sep 17 00:00:00 2001 From: Devinder Singh Date: Sun, 25 Oct 2026 14:25:42 +0100 Subject: [PATCH 2/2] docs: add API code structure decision (Zod, layers, error handler) Adds decision record 0003: Zod validates bodies, queries and environment variables; code is split into routes, services and repositories; one Fastify error handler gives every error the same shape. Decision 0001 now points to 0003 for validation, and CONTEXT and README list Zod. --- README.md | 2 +- docs/CONTEXT.md | 3 +- docs/decisions/0001-web-framework-fastify.md | 6 +- docs/decisions/0003-api-code-structure.md | 92 ++++++++++++++++++++ 4 files changed, 99 insertions(+), 4 deletions(-) create mode 100644 docs/decisions/0003-api-code-structure.md diff --git a/README.md b/README.md index 30c47aa..888d02e 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ Relay sits in the middle: ## Stack -Bun 1.4 + TypeScript + Fastify · React + Vite + Tailwind dashboard · +Bun 1.4 + TypeScript + Fastify + Zod · React + Vite + Tailwind dashboard · PostgreSQL 18 (also the job queue, via `SKIP LOCKED`) · k3s on our own VPS · Traefik + cert-manager (Let's Encrypt) · GitHub Actions → GHCR · Prometheus, Grafana, Loki · Amazon S3 for backups, with the bucket made by Terraform. diff --git a/docs/CONTEXT.md b/docs/CONTEXT.md index 64e1f28..ec63b40 100644 --- a/docs/CONTEXT.md +++ b/docs/CONTEXT.md @@ -91,8 +91,9 @@ Owners and API keys are created with a `create-owner` script (no signup page). | Area | Choice | Not chosen | |---|---|---| | Runtime | Bun **1.4.4**, pinned everywhere (laptop, CI, Docker). 1.4 is a new rewrite in Rust | Node (more common, but we know Bun; it stays our fallback) | -| Web framework | **Fastify 5** ([decision 0001](decisions/0001-web-framework-fastify.md)): `buildApp()` + `listen({ host: "0.0.0.0" })`, pino logs, JSON Schema validation, `@fastify/cors`, `inject()` tests | Hono (native on Bun, but new to us), Express, Elysia | +| Web framework | **Fastify 5** ([decision 0001](decisions/0001-web-framework-fastify.md)): `buildApp()` + `listen({ host: "0.0.0.0" })`, pino logs, Zod validation, `@fastify/cors`, `inject()` tests | Hono (native on Bun, but new to us), Express, Elysia | | Database library | postgres.js behind one file `src/db.ts` | `Bun.sql` (ties the data layer to the new runtime), Drizzle | +| API code structure | **Zod 4** for bodies, queries and env vars ([decision 0003](decisions/0003-api-code-structure.md)); layers routes → services → repositories; one Fastify error handler with one error shape; small files, one job each | Fastify JSON Schema, `fastify-type-provider-zod`, logic in route handlers | | Migrations | dbmate (plain SQL), run by a Kubernetes Job before each rollout. Expand/contract rule | drizzle-kit, an initContainer (would rerun on every scale-up) | | Unit + integration tests | `bun test`; integration against a real Postgres (Docker locally, service container in GitHub Actions) | Testcontainers (no official Bun support), mocked DB | | Dashboard | **Vite + React + TypeScript, Tailwind CSS v4, React Router, axios** (one client in `src/api.ts`), plain React state with a small refresh loop while a delivery is pending, served by nginx ([decision 0002](decisions/0002-dashboard-stack.md)) | Next.js, plain CSS, `fetch`, TanStack Query | diff --git a/docs/decisions/0001-web-framework-fastify.md b/docs/decisions/0001-web-framework-fastify.md index 4f8cee3..8818d24 100644 --- a/docs/decisions/0001-web-framework-fastify.md +++ b/docs/decisions/0001-web-framework-fastify.md @@ -32,7 +32,7 @@ Both of us are examined individually and must be able to explain every line. We use **Fastify 5** on Bun 1.4.4. Familiarity was the deciding factor: we can write, review and explain Fastify code with -confidence. Its built-in pino logging and JSON Schema validation also cover two needs we +confidence. Its built-in pino logging (with a request id on every line) also covers a need we would otherwise build ourselves. ## Consequences @@ -46,5 +46,7 @@ would otherwise build ourselves. `server.ts` only calls `listen()` and handles SIGTERM with `app.close()`. - **Logging:** pino, built into Fastify, writing JSON to stdout. We keep our agreed field names (`request_id`, `event_id`, `delivery_id`, `endpoint_id`, `status_code`, `duration_ms`). -- **Validation:** JSON Schema on each route. A bad body gets a 400 automatically. +- **Validation:** Zod, called in each route ([decision 0003](0003-api-code-structure.md)). We don't + use Fastify's built-in JSON Schema validation: with Zod each body is described once and gives + both the runtime check and the TypeScript type. - **CORS:** the `@fastify/cors` plugin, limited to `https://cdn.myrelay.tech` (and localhost in CI). diff --git a/docs/decisions/0003-api-code-structure.md b/docs/decisions/0003-api-code-structure.md new file mode 100644 index 0000000..3fc97ad --- /dev/null +++ b/docs/decisions/0003-api-code-structure.md @@ -0,0 +1,92 @@ +# 0003. API code structure: Zod, layers and one error handler + +- **Status:** Accepted +- **Date:** 2026-10-25 +- **Decided by:** Devinder Singh and Abdikhani Mohamed + +## Context + +The API and the worker will grow to around ten routes, a queue and several database tables. +We are examined individually on every line, so the code must be easy to read, explain and test. +We want small files with one job each, no surprises, and a clear answer to "where does this +logic live?". + +## Decision + +### Validation with Zod + +Every request body, query string and the environment variables are checked with **Zod 4**. +One Zod schema gives both the runtime check and the TypeScript type (`z.infer`), so nothing is +described twice. + +```ts +const body = CreateEventBody.parse(request.body); // a wrong body throws, the error handler answers 422 +``` + +We call `parse()` ourselves in the route. No plugin. + +| Not chosen | Why | +|---|---| +| Fastify's JSON Schema validation | The type must be written a second time by hand, or generated with more tooling | +| `fastify-type-provider-zod` | Hides the parse call, and requires `@fastify/swagger` and `openapi-types` as extra packages | + +### Layers, one direction only + +``` +routes/ HTTP only: check input with Zod, call a service, send the answer + ▼ +services/ the rules: fan-out, idempotency, replay, transactions + ▼ +repositories/ SQL only: one file per table group, returns typed rows + ▼ +Postgres +``` + +- A layer only calls the layer below it. Nobody skips a layer. +- Repositories take the database handle as their first argument (`db`), so one service can + run several repositories inside **one transaction** (`sql.begin(tx => …)`). +- Pure logic (retry timing, signing, the SSRF address check) lives in small files with no + database or HTTP, so it can be unit-tested directly. + +### One error handler + +Code never builds error responses itself. It throws, and **one** Fastify error handler +(`error-handler.ts`) turns every error into the same shape: +`{"error": {"code", "message", "request_id"}}`. + +| Error | Answer | +|---|---| +| `ZodError` | 422 `validation_failed`, with the fields that were wrong | +| Our own `AppError` (for example `notFound`, `conflict`) | Its own status and code (404 `not_found`, 409 `already_delivered`, …) | +| Fastify's own client errors | Their status: 400 bad JSON, 413 body too large, 415 not JSON | +| Anything else | Logged in full, answered as 500 `internal_error` with no details | + +### Files + +``` +api/src/ +├─ server.ts start and graceful shutdown (the only file that listens on a port) +├─ app.ts buildApp(): plugins, error handler and routes wired together +├─ config.ts environment variables, checked with Zod at startup +├─ errors.ts AppError and helpers (notFound, conflict) +├─ error-handler.ts the one place errors become HTTP answers +├─ db.ts the Postgres connection +├─ schemas/ Zod schemas and their types +├─ routes/ one file per resource +├─ services/ one file per resource +└─ repositories/ one file per table group +``` + +Files stay small: roughly under 150 lines, one job each. Folders are created only when a step +needs them. + +## Consequences + +- **Testing:** pure functions get unit tests; repositories get integration tests against a real + Postgres; routes get `inject()` tests that go through the real error handler. +- **Bad configuration fails fast:** a missing or wrong environment variable stops the app at + startup with a clear message, before Kubernetes marks the pod as ready. +- **No leaks:** a database error can never reach a client with its SQL or stack trace, because + only the error handler writes error responses. +- [Decision 0001](0001-web-framework-fastify.md) refers here for validation: we use Zod instead of + Fastify's built-in JSON Schema.