diff --git a/CHANGELOG.md b/CHANGELOG.md index b6ade9f..aaf3b91 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,10 @@ project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - CI correctness smoke coverage for the asyncpg prototype and rollback semantics - Architecture decision retaining synchronous SQLAlchemy for v1.3.0 until representative measurements justify a separately scoped migration +- Guarded k6 load-testing profiles for health baselines, authenticated reads, + and isolated per-virtual-user refresh-token rotation +- Explicit workload error-rate and p95/p99 starting thresholds plus a staged, + repeatable measurement guide for controlled environments ## [1.2.0] - 2026-08-09 diff --git a/LOAD_TESTING.md b/LOAD_TESTING.md new file mode 100644 index 0000000..1aed2b4 --- /dev/null +++ b/LOAD_TESTING.md @@ -0,0 +1,88 @@ +# Load testing + +The repository includes a bounded [Grafana k6](https://grafana.com/docs/k6/latest/) +example for end-to-end HTTP behavior. It complements the database-only harness +in [DATABASE_BENCHMARKS.md](DATABASE_BENCHMARKS.md): k6 measures routing, JWT, +password hashing, database access, middleware, and response serialization as one +system. + +## Safety and scope + +Run the example only against an isolated local or explicitly authorized staging +environment. It creates disposable users and refresh-token sessions and does not +delete them. Use a disposable database or restore a snapshot after the run. + +Remote targets are rejected unless `ALLOW_REMOTE_TARGET=true`. That override is +an acknowledgement, not a safety guarantee. Never point this script at +production, and confirm the expected traffic with the environment owner before +raising concurrency. + +The default thresholds are reviewable starting points, not universal SLOs: + +- fewer than 1% failed workload requests; +- more than 99% successful checks; +- workload p95 below 500 ms and p99 below 1,000 ms. + +Change thresholds only from measured service objectives. Retain the k6 summary, +application metrics, PostgreSQL/Redis metrics, CPU, memory, commit SHA, and +environment configuration with every result. + +## Profiles + +| Profile | Behavior | Intended use | +| --- | --- | --- | +| `health` | Repeated `GET /health/live` after a readiness probe | Network and middleware baseline | +| `authenticated` | One user/session per VU, repeated `GET /auth/me`, periodic refresh rotation | Representative authentication lifecycle | + +Registration and login happen outside the tagged workload latency thresholds. +They are still checked and visible in the k6 result. Each virtual user owns its +refresh token so rotations do not create artificial replay failures. + +## Local smoke run + +Start PostgreSQL and Redis, apply migrations, and start the API as described in +[DEVELOPMENT.md](DEVELOPMENT.md). Install k6 1.x, then run: + +```bash +k6 run -e PROFILE=health -e VUS=2 -e DURATION=10s load_tests/api.js +k6 run -e PROFILE=authenticated -e VUS=5 -e DURATION=30s load_tests/api.js +``` + +PowerShell uses the same commands. The defaults target +`http://127.0.0.1:8000`, use five virtual users for 30 seconds, and rotate each +virtual user's refresh token every 20 iterations. + +Useful environment variables: + +| Variable | Default | Meaning | +| --- | --- | --- | +| `BASE_URL` | `http://127.0.0.1:8000` | API origin without a trailing slash | +| `PROFILE` | `authenticated` | `health` or `authenticated` | +| `VUS` | `5` | Concurrent virtual users | +| `DURATION` | `30s` | k6 duration expression | +| `REFRESH_EVERY` | `20` | Authenticated iterations between rotations | +| `RUN_ID` | current timestamp | Suffix making generated usernames unique | +| `LOAD_TEST_PASSWORD` | disposable local value | Password for generated users | +| `ALLOW_REMOTE_TARGET` | unset | Must be exactly `true` for a remote host | + +Validate the script without sending traffic: + +```bash +k6 inspect load_tests/api.js +``` + +## Staged workload + +Establish a baseline before looking for capacity limits: + +1. Run `health` with one VU to record network/middleware latency. +2. Run `authenticated` at 1, 5, 10, and 25 VUs for at least five minutes each. +3. Repeat every point three times on the same commit and unchanged environment. +4. Stop when error rate, pool wait, CPU saturation, or tail latency violates the + service objective; do not keep increasing traffic through an unhealthy API. +5. Diagnose bcrypt cost, database pool pressure, Redis latency, worker count, + and tracing export separately before changing the application architecture. + +CI shared runners should only validate correctness and script structure. Do not +use their timings as a release performance gate. A release gate belongs in a +controlled staging environment with stable infrastructure and retained results. diff --git a/README.md b/README.md index 0021df2..5376c3a 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,7 @@ contributor workflows, and local troubleshooting. | [API examples](API_EXAMPLES.md) | Register, authenticate, rotate tokens, call admin routes, and inspect operations endpoints | | [Architecture](ARCHITECTURE.md) | Understand module boundaries, request flow, authentication, transactions, and extension points | | [Database benchmarks](DATABASE_BENCHMARKS.md) | Reproduce sync/async PostgreSQL comparisons and understand the v1.3 architecture decision | +| [Load testing](LOAD_TESTING.md) | Exercise health and authenticated HTTP lifecycles with bounded k6 workloads and explicit thresholds | | [Local development](DEVELOPMENT.md) | Set up a checkout, run common commands, contribute, or troubleshoot locally | | [Deployment](DEPLOYMENT.md) | Configure a production host, release safely, terminate TLS, and operate the service | | [Monitoring](MONITORING.md) | Configure probes, Prometheus, multi-worker metrics, alerts, logs, and incident diagnosis | @@ -250,6 +251,10 @@ and async database benchmark and uploads its machine-readable JSON artifact. See [DATABASE_BENCHMARKS.md](DATABASE_BENCHMARKS.md); shared-runner timings are not used as performance gates. +End-to-end k6 examples cover a health baseline and an authenticated lifecycle +with isolated per-VU refresh rotation. See [LOAD_TESTING.md](LOAD_TESTING.md) +for safety guards, starting thresholds, and a staged workload methodology. + CI runs against PostgreSQL 17 rather than silently substituting SQLite. It also verifies that the built wheel contains and can import the application. diff --git a/ROADMAP.md b/ROADMAP.md index 612a568..cafb664 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -59,7 +59,7 @@ Status: in progress - Transactional outbox and background worker processing (completed) - Safe OIDC discovery/JWKS caching and invalidation guidance (completed) - Async database evaluation and performance benchmarks (completed; sync retained) -- Load-testing examples +- Load-testing examples (completed) - OpenTelemetry tracing example (completed) ## v2.0 — Deployment patterns at scale diff --git a/load_tests/api.js b/load_tests/api.js new file mode 100644 index 0000000..bba658f --- /dev/null +++ b/load_tests/api.js @@ -0,0 +1,136 @@ +import http from "k6/http"; +import { check, fail, sleep } from "k6"; + +const BASE_URL = (__ENV.BASE_URL || "http://127.0.0.1:8000").replace(/\/$/, ""); +const PROFILE = __ENV.PROFILE || "authenticated"; +const VUS = positiveInteger("VUS", 5); +const DURATION = __ENV.DURATION || "30s"; +const REFRESH_EVERY = positiveInteger("REFRESH_EVERY", 20); +const PASSWORD = __ENV.LOAD_TEST_PASSWORD || "local-load-test-password"; +const RUN_ID = (__ENV.RUN_ID || `${Date.now()}`) + .replace(/[^a-zA-Z0-9_-]/g, "") + .slice(0, 40); + +if (!["health", "authenticated"].includes(PROFILE)) { + throw new Error("PROFILE must be health or authenticated"); +} + +assertSafeTarget(BASE_URL); + +export const options = { + vus: VUS, + duration: DURATION, + discardResponseBodies: true, + thresholds: { + checks: ["rate>0.99"], + "http_req_failed{phase:workload}": ["rate<0.01"], + "http_req_duration{phase:workload}": ["p(95)<500", "p(99)<1000"], + }, +}; + +function positiveInteger(name, fallback) { + const value = Number.parseInt(__ENV[name] || `${fallback}`, 10); + if (!Number.isInteger(value) || value < 1) { + throw new Error(`${name} must be a positive integer`); + } + return value; +} + +function assertSafeTarget(value) { + const parsed = value.match(/^https?:\/\/(\[[^\]]+\]|[^/:]+)(?::\d+)?(?:\/.*)?$/i); + if (!parsed) { + throw new Error("BASE_URL must be an absolute HTTP(S) URL"); + } + const hostname = parsed[1].toLowerCase(); + const localHosts = new Set(["localhost", "127.0.0.1", "[::1]", "host.docker.internal"]); + if (!localHosts.has(hostname) && __ENV.ALLOW_REMOTE_TARGET !== "true") { + throw new Error( + "Refusing a remote target. Set ALLOW_REMOTE_TARGET=true only for an authorized non-production environment.", + ); + } +} + +function workloadTags(endpoint) { + return { tags: { phase: "workload", endpoint } }; +} + +export function setup() { + const ready = http.get(`${BASE_URL}/health/ready`, workloadTags("readiness")); + if (!check(ready, { "target is ready": (response) => response.status === 200 })) { + fail(`Target is not ready: HTTP ${ready.status}`); + } + + if (PROFILE === "health") { + return { users: [] }; + } + + const users = []; + for (let index = 1; index <= VUS; index += 1) { + const username = `load_${RUN_ID}_${index}`; + const registration = http.post( + `${BASE_URL}/register/`, + JSON.stringify({ username, password: PASSWORD }), + { headers: { "Content-Type": "application/json" }, tags: { phase: "setup", endpoint: "register" } }, + ); + if (!check(registration, { "load-test user registered": (response) => response.status === 200 })) { + fail(`Could not register ${username}: HTTP ${registration.status}`); + } + users.push(username); + } + return { users }; +} + +let tokens; + +function login(username) { + const response = http.post( + `${BASE_URL}/login/`, + { username, password: PASSWORD }, + { + headers: { "X-Device-Name": `k6-vu-${__VU}` }, + tags: { phase: "setup", endpoint: "login" }, + responseType: "text", + }, + ); + if (!check(response, { "load-test user logged in": (result) => result.status === 200 })) { + fail(`Could not log in ${username}: HTTP ${response.status}`); + } + return response.json(); +} + +function authenticatedIteration(data) { + if (!tokens) { + tokens = login(data.users[__VU - 1]); + } + + const me = http.get(`${BASE_URL}/auth/me`, { + headers: { Authorization: `Bearer ${tokens.access_token}` }, + ...workloadTags("auth_me"), + }); + check(me, { "authenticated read succeeds": (response) => response.status === 200 }); + + if (__ITER > 0 && __ITER % REFRESH_EVERY === 0) { + const rotated = http.post( + `${BASE_URL}/auth/refresh`, + JSON.stringify({ refresh_token: tokens.refresh_token }), + { + headers: { "Content-Type": "application/json" }, + ...workloadTags("refresh"), + responseType: "text", + }, + ); + if (check(rotated, { "refresh rotation succeeds": (response) => response.status === 200 })) { + tokens = rotated.json(); + } + } +} + +export default function (data) { + if (PROFILE === "health") { + const response = http.get(`${BASE_URL}/health/live`, workloadTags("liveness")); + check(response, { "liveness succeeds": (result) => result.status === 200 }); + } else { + authenticatedIteration(data); + } + sleep(0.1); +} diff --git a/tests/test_documentation.py b/tests/test_documentation.py index 3ba23ae..abf76b8 100644 --- a/tests/test_documentation.py +++ b/tests/test_documentation.py @@ -16,6 +16,7 @@ "CODE_OF_CONDUCT.md", "CONTRIBUTING.md", "DATABASE_BENCHMARKS.md", + "LOAD_TESTING.md", "DEPLOYMENT.md", "DEVELOPMENT.md", "MONITORING.md", @@ -89,6 +90,7 @@ def test_readme_indexes_the_task_guides(): "API_EXAMPLES.md", "ARCHITECTURE.md", "DATABASE_BENCHMARKS.md", + "LOAD_TESTING.md", "DEPLOYMENT.md", "DEVELOPMENT.md", "MONITORING.md",