Skip to content

Commit 186909e

Browse files
aksOpsclaude
andcommitted
chore(test): Playwright e2e suite + unified make regression target
Adds the third testing layer the project was missing: a real headless Chromium driving the built vite bundle with /api + /events mocked at the page level. Complements go unit tests (correctness) and vitest (component logic) by catching integration + render bugs — e.g. the "resets in 0 sec" bug and the sort-to-top bug we shipped today would both have been caught here. New: - ui/playwright.config.ts — chromium project, vite preview webserver on 127.0.0.1:4173 (IPv4 explicit so Playwright's probe matches), trace + screenshot retained on failure. - ui/e2e/fixtures/mocks.ts — installMocks(page, overrides) sets up /api/bootstrap, /api/sessions (raw Session[]), /api/quota, /api/feed, /events/** (no-op SSE). authenticate(page) seeds the ctm.token localStorage key so the paste screen is skipped. - ui/e2e/{auth,dashboard,quota-strip}.spec.ts — 6 initial cases covering: paste-screen visibility, token → dashboard, session card render + mode badge, last-tool-call ⏵ element, quota strip shows real "resets in N min|hr|day" (not 0 sec), percent values. - ui/e2e/README.md — how to run, how to grow (append a case per fix/feature), how to debug (trace.zip inspection). Makefile: - make e2e — pnpm build && playwright test - make regression — unified pre-merge pack that fails fast on the first non-zero step: go build → go test → go test -race on internal/serve/... → govulncheck → tsc --noEmit → vitest → pnpm audit (high+) → make e2e. ~19 s on a warm cache. vitest.config.ts: exclude e2e/** so vitest doesn't pick up Playwright specs and choke on test.describe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 32e35d4 commit 186909e

11 files changed

Lines changed: 379 additions & 5 deletions

Makefile

Lines changed: 43 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,16 @@ UI_DIR := ui
88
UI_DIST := $(UI_DIR)/dist
99
EMBED_DIST := internal/serve/dist
1010

11-
.PHONY: ui build dev clean help
11+
.PHONY: ui build dev clean help e2e regression
1212

1313
help:
1414
@echo "Targets:"
15-
@echo " ui — install + build React UI, sync to $(EMBED_DIST)"
16-
@echo " build — make ui && go build ./..."
17-
@echo " dev — pnpm --prefix ui dev + go run . serve in parallel"
18-
@echo " clean — remove $(UI_DIST) and $(EMBED_DIST)"
15+
@echo " ui — install + build React UI, sync to $(EMBED_DIST)"
16+
@echo " build — make ui && go build ./..."
17+
@echo " dev — pnpm --prefix ui dev + go run . serve in parallel"
18+
@echo " e2e — Playwright tests against vite preview, mocked API surface"
19+
@echo " regression — full pre-merge pack: go build/test/race/vuln + ui tsc/vitest/audit/e2e"
20+
@echo " clean — remove $(UI_DIST) and $(EMBED_DIST)"
1921

2022
ui:
2123
@echo "==> pnpm install"
@@ -40,3 +42,39 @@ dev:
4042

4143
clean:
4244
rm -rf $(UI_DIST) $(EMBED_DIST)
45+
46+
# Playwright E2E. Uses Chromium from ~/.cache/ms-playwright (installed via
47+
# `pnpm exec playwright install chromium`). Mocks /api + /events at page
48+
# level so tests don't need a running daemon or fixture DB. Run
49+
# `pnpm --prefix ui exec playwright install chromium` once before first run.
50+
e2e:
51+
@echo "==> pnpm build (needed for vite preview)"
52+
cd $(UI_DIR) && pnpm build
53+
@echo "==> playwright test"
54+
cd $(UI_DIR) && pnpm exec playwright test
55+
56+
# Unified regression pack. Runs everything a PR must clear before merge.
57+
# Fails fast — first non-zero exit stops the run. Total wall time on this
58+
# machine is ~60-90s depending on Go cache state.
59+
#
60+
# Contract: every shipped bug fix or new feature adds a test case that
61+
# executes under one of these steps (unit / vitest / e2e). The pack grows;
62+
# it does not get replaced. See ui/e2e/README.md.
63+
regression:
64+
@echo "==> go build ./..."
65+
go build ./...
66+
@echo "==> go test ./..."
67+
go test ./...
68+
@echo "==> go test -race ./internal/serve/..."
69+
go test -race ./internal/serve/...
70+
@echo "==> govulncheck ./..."
71+
govulncheck ./...
72+
@echo "==> pnpm -C ui tsc --noEmit"
73+
pnpm -C $(UI_DIR) exec tsc --noEmit
74+
@echo "==> pnpm -C ui test (vitest)"
75+
pnpm -C $(UI_DIR) test
76+
@echo "==> pnpm audit (High/Critical fails; Medium/Low reported)"
77+
cd $(UI_DIR) && pnpm audit --audit-level=high
78+
@echo "==> make e2e"
79+
$(MAKE) e2e
80+
@echo "==> regression pack OK"

ui/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,8 @@ coverage/
77

88
# tsc incremental build cache
99
*.tsbuildinfo
10+
11+
# Playwright E2E artifacts
12+
test-results/
13+
playwright-report/
14+
.playwright/

ui/e2e/README.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Playwright E2E — ctm serve UI
2+
3+
Fast, deterministic browser tests for the React SPA. Mocks `/api/**` and
4+
`/events/**` per-page so the suite never needs a running daemon or fixture
5+
DB, and runs against a production vite-preview bundle so CSS and asset
6+
resolution match what ships to users.
7+
8+
## Running
9+
10+
```sh
11+
make e2e # from repo root — rebuilds bundle + runs suite
12+
pnpm exec playwright test # from ui/ — assumes dist/ is already built
13+
make regression # runs this suite plus Go/vitest/audit
14+
```
15+
16+
One-time setup on a fresh clone:
17+
```sh
18+
pnpm --prefix ui install
19+
pnpm --prefix ui exec playwright install chromium
20+
```
21+
22+
## Growing the pack
23+
24+
**Every shipped bug fix or feature adds a test here that would have caught
25+
it.** The pack is append-only — do not replace existing specs when you
26+
extend them. Name new spec files after the feature or regression ID
27+
(`quota-strip.spec.ts`, `auth.spec.ts`, …) and group related cases inside
28+
a single `test.describe` block.
29+
30+
## Mocks
31+
32+
`e2e/fixtures/mocks.ts` exposes `installMocks(page, overrides?)` and
33+
`authenticate(page)`. Defaults return a single happy-path session, a
34+
quota snapshot with future reset times, an empty feed, and a held-open
35+
SSE stream. Override per-case:
36+
37+
```ts
38+
await installMocks(page, {
39+
sessions: [{ ...alpha, is_active: false }],
40+
authenticated: false, // forces /api/bootstrap → 401
41+
});
42+
```
43+
44+
Shapes mirror the real API (`/api/sessions``Session[]`, `/api/feed`
45+
`ToolCallRow[]`) — if you change either contract in `internal/serve/api`,
46+
update the fixture too.
47+
48+
## Debugging
49+
50+
```sh
51+
pnpm exec playwright test --debug # launch inspector
52+
pnpm exec playwright test auth.spec.ts --ui # interactive UI mode
53+
playwright show-trace test-results/<name>/trace.zip
54+
```
55+
56+
Traces are retained only on failure (see `playwright.config.ts`).

ui/e2e/auth.spec.ts

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import { test, expect } from "@playwright/test";
2+
import { installMocks } from "./fixtures/mocks";
3+
4+
test.describe("Auth / token paste flow", () => {
5+
test("shows paste screen when no token is stored", async ({ page }) => {
6+
await installMocks(page, { authenticated: false });
7+
await page.goto("/");
8+
// The paste screen headline is just "ctm"; assert on the paste
9+
// textarea + its bearer-token label to avoid coupling to copy.
10+
await expect(page.getByRole("textbox")).toBeVisible();
11+
await expect(page.getByText(/bearer token/i)).toBeVisible();
12+
});
13+
14+
test("accepting a token transitions to the dashboard", async ({ page }) => {
15+
await installMocks(page, { authenticated: false });
16+
await page.goto("/");
17+
18+
const input = page.getByRole("textbox");
19+
await input.fill("pasted-token-value");
20+
21+
// Swap bootstrap mock to 200 so form submission passes auth.
22+
await installMocks(page, { authenticated: true });
23+
24+
await page.getByRole("button", { name: /continue/i }).click();
25+
await expect(
26+
page.getByRole("link", { name: /alpha/i }).first(),
27+
).toBeVisible();
28+
});
29+
});

ui/e2e/dashboard.spec.ts

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import { test, expect } from "@playwright/test";
2+
import { authenticate, installMocks } from "./fixtures/mocks";
3+
4+
test.describe("Dashboard", () => {
5+
test.beforeEach(async ({ page }) => {
6+
await authenticate(page);
7+
await installMocks(page);
8+
});
9+
10+
test("renders the seeded session card", async ({ page }) => {
11+
await page.goto("/");
12+
// Desktop auto-navigates to the top session so "alpha" appears in
13+
// both the list card and detail heading — assert on the link card.
14+
await expect(
15+
page.getByRole("link", { name: /alpha/i }).first(),
16+
).toBeVisible();
17+
await expect(page.getByText(/safe/i).first()).toBeVisible();
18+
});
19+
20+
test("shows last tool call time on the card", async ({ page }) => {
21+
// Seeded tool call is at 16:28; attached at 16:00. Card should carry
22+
// both time readings (⏵ for tool call, plain for attached).
23+
await page.goto("/");
24+
const card = page.getByRole("link", { name: /alpha/i }).first();
25+
await expect(card).toBeVisible();
26+
// ⏵ glyph rendered for the tool-call badge.
27+
await expect(card.locator('time[aria-label="last tool call"]')).toBeVisible();
28+
});
29+
});

ui/e2e/fixtures/mocks.ts

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
import type { Page, Route } from "@playwright/test";
2+
3+
/**
4+
* Default mock surface: one happy-path session, one quota snapshot,
5+
* no attention alerts, empty feed. Tests override per-case by calling
6+
* `installMocks(page, { overrides })` or by adding their own route()
7+
* handlers after.
8+
*/
9+
export interface MockOverrides {
10+
bootstrap?: unknown;
11+
sessions?: unknown;
12+
quota?: unknown;
13+
feed?: unknown;
14+
/** When false, /api/bootstrap returns 401 so the paste screen renders. */
15+
authenticated?: boolean;
16+
}
17+
18+
const defaultSession = {
19+
name: "alpha",
20+
uuid: "00000000-0000-0000-0000-000000000001",
21+
mode: "safe",
22+
workdir: "/home/dev/projects/ctm",
23+
created_at: "2026-04-21T15:00:00Z",
24+
last_attached_at: "2026-04-21T16:00:00Z",
25+
last_tool_call_at: "2026-04-21T16:28:00Z",
26+
is_active: true,
27+
tmux_alive: true,
28+
context_pct: 42,
29+
tokens: { input_tokens: 12_340, output_tokens: 2_100, cache_tokens: 55_000 },
30+
};
31+
32+
const defaultQuota = {
33+
weekly_pct: 48,
34+
five_hr_pct: 24,
35+
// A reset 3 hours into the future — verifies the relativeFuture helper.
36+
weekly_resets_at: new Date(Date.now() + 3 * 3600_000).toISOString(),
37+
five_hr_resets_at: new Date(Date.now() + 45 * 60_000).toISOString(),
38+
};
39+
40+
export async function installMocks(
41+
page: Page,
42+
overrides: MockOverrides = {},
43+
): Promise<void> {
44+
const authed = overrides.authenticated !== false;
45+
46+
await page.route("**/api/bootstrap", (route: Route) => {
47+
if (!authed) return route.fulfill({ status: 401 });
48+
return route.fulfill({
49+
contentType: "application/json",
50+
body: JSON.stringify(
51+
overrides.bootstrap ?? {
52+
version: "e2e-test",
53+
port: 37778,
54+
has_webhook: false,
55+
},
56+
),
57+
});
58+
});
59+
60+
await page.route("**/api/sessions**", (route: Route) => {
61+
return route.fulfill({
62+
contentType: "application/json",
63+
body: JSON.stringify(overrides.sessions ?? [defaultSession]),
64+
});
65+
});
66+
67+
await page.route("**/api/quota", (route: Route) => {
68+
return route.fulfill({
69+
contentType: "application/json",
70+
body: JSON.stringify(overrides.quota ?? defaultQuota),
71+
});
72+
});
73+
74+
await page.route("**/api/feed**", (route: Route) => {
75+
return route.fulfill({
76+
contentType: "application/json",
77+
body: JSON.stringify(overrides.feed ?? []),
78+
});
79+
});
80+
81+
// Keep SSE calls quiet — route with a held-open no-op stream.
82+
await page.route("**/events/**", (route: Route) => {
83+
return route.fulfill({
84+
status: 200,
85+
contentType: "text/event-stream",
86+
body: ": ok\n\n",
87+
});
88+
});
89+
}
90+
91+
/** Store an auth token in localStorage so the SPA skips the paste screen. */
92+
export async function authenticate(page: Page): Promise<void> {
93+
await page.addInitScript(() => {
94+
window.localStorage.setItem("ctm.token", "e2e-test-token");
95+
});
96+
}
97+
98+
export { defaultSession, defaultQuota };

ui/e2e/quota-strip.spec.ts

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
import { test, expect } from "@playwright/test";
2+
import { authenticate, installMocks } from "./fixtures/mocks";
3+
4+
test.describe("QuotaStrip", () => {
5+
test.beforeEach(async ({ page }) => {
6+
await authenticate(page);
7+
await installMocks(page);
8+
});
9+
10+
test("shows future reset time, not 0 sec (regression 4e9694f)", async ({
11+
page,
12+
}) => {
13+
// Default mock has 5h reset 45 minutes out, weekly 3 hours out.
14+
// Before the relativeFuture fix, both rendered "0 sec".
15+
await page.goto("/");
16+
17+
const strip = page.getByRole("region", { name: /rate limit usage/i });
18+
await expect(strip).toBeVisible();
19+
20+
const text = await strip.innerText();
21+
expect(text).not.toContain("resets in 0 sec");
22+
expect(text).toMatch(/resets in \d+ (min|hr|day)/);
23+
});
24+
25+
test("renders percentage values", async ({ page }) => {
26+
await page.goto("/");
27+
const strip = page.getByRole("region", { name: /rate limit usage/i });
28+
await expect(strip).toContainText("24%"); // 5h
29+
await expect(strip).toContainText("48%"); // weekly
30+
});
31+
});

ui/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
},
3434
"devDependencies": {
3535
"@eslint/js": "10.0.1",
36+
"@playwright/test": "^1.59.1",
3637
"@tailwindcss/vite": "^4.2.2",
3738
"@testing-library/jest-dom": "^6.9.1",
3839
"@testing-library/react": "^16.3.2",

ui/playwright.config.ts

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
/// <reference types="node" />
2+
import { defineConfig, devices } from "@playwright/test";
3+
4+
/**
5+
* Playwright E2E config. Drives the React SPA against a mocked /api + /events
6+
* surface via `page.route` so tests stay fast and deterministic; backed by the
7+
* vite preview server (built bundle, not dev server) so CSS/asset resolution
8+
* matches production.
9+
*/
10+
export default defineConfig({
11+
testDir: "./e2e",
12+
outputDir: "./test-results",
13+
timeout: 15_000,
14+
expect: { timeout: 3_000 },
15+
fullyParallel: true,
16+
forbidOnly: !!process.env.CI,
17+
retries: process.env.CI ? 1 : 0,
18+
reporter: process.env.CI ? "github" : "list",
19+
20+
use: {
21+
baseURL: "http://127.0.0.1:4173",
22+
trace: "retain-on-failure",
23+
screenshot: "only-on-failure",
24+
},
25+
26+
projects: [
27+
{
28+
name: "chromium",
29+
use: { ...devices["Desktop Chrome"] },
30+
},
31+
],
32+
33+
// webServer assumes `dist/` is already built. `make e2e` runs
34+
// `pnpm build` first; running `playwright test` directly also works
35+
// if you've recently built. Vite preview serves the static bundle —
36+
// no dev-server HMR noise — matching what ships to users.
37+
webServer: {
38+
command:
39+
"pnpm exec vite preview --port 4173 --strictPort --host 127.0.0.1",
40+
url: "http://127.0.0.1:4173",
41+
reuseExistingServer: !process.env.CI,
42+
timeout: 30_000,
43+
stdout: "ignore",
44+
stderr: "pipe",
45+
},
46+
});

0 commit comments

Comments
 (0)