Skip to content

Clean up Playwright fixtures after setup failures #406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thomas9120 merged 1 commit into from
Oct 11, 2026
Merged

Clean up Playwright fixtures after setup failures #406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,18 @@ npm run test:frontend

Runs the Playwright smoke test for browser-level shared-state sync. This is also the only suite that can cover the Configure sampler preset panel, because `renderFlags()` destroys and rebuilds it — the `<select>` an assertion reads is a different element than the one that was clicked, which a `node:vm` harness cannot reproduce.

On Windows, Codex's sandbox can redirect `LOCALAPPDATA`, causing Playwright to
look in an empty browser cache even when Chromium is installed for the normal
user. `PLAYWRIGHT_BROWSERS_PATH` can select an existing cache (see
[Playwright's browser cache documentation](https://playwright.dev/docs/browsers#managing-browser-binaries)),
but this alone does not grant network access. If Chromium reports
`ERR_NETWORK_ACCESS_DENIED` when opening the local fixture server, run
`npm run test:frontend` through Codex's approved command escalation or a normal
local terminal. Do not repeatedly reinstall browsers into the redirected cache.
The pop-out fixtures register server and browser cleanup immediately after each
resource starts, so setup failures exit promptly; a child-process regression
check verifies this with an intentionally missing Chromium cache.

Both browser suites serve `ui/` through the shared Node HTTP/1.1 server in `tests/frontend/ui_server.cjs`. Persistent connections avoid opening a socket for every asset, which can exhaust Windows socket buffers during repeated page loads (`net::ERR_NO_BUFFER_SPACE`). A regression check verifies that HTML, JavaScript, and CSS requests reuse one connection. Python is not needed to serve the browser fixtures.

The browser suite has named `node:test` scenarios with fresh browser contexts and API fixtures. Shared controls, authentication, inference, Chat streaming/recovery/compaction, samplers, external servers, custom launch arguments, backend activation, model switching, preset focus, model folders, and Monitor behavior run independently. A scenario failure does not prevent the remaining scenarios from running. Smoke-test failures, including startup and reload timeouts, retain the original error as their cause and include captured page errors and failed requests, or explicit indications that none were captured. Normal request cancellations during navigation are excluded. Injected Configure rendering failures and connection resets verify these diagnostics on both initial load and reload. The external-server reload check waits for a restored inference target before asserting its exact identity, so a wrong identity is reported directly. To run one scenario:
Expand Down
87 changes: 37 additions & 50 deletions tests/frontend/chat_popout_integration.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ const assert = require("node:assert/strict");
const fs = require("node:fs");
const path = require("node:path");
const { randomUUID } = require("node:crypto");
const { spawnSync } = require("node:child_process");
const os = require("node:os");
const { test } = require("node:test");
const { startUiServer } = require("./ui_server.cjs");

Expand All @@ -25,6 +27,21 @@ const CHAT_HOST_STATUS = "#chat-window-host-status";
const CHAT_OPEN_FULL_GUI = "#chat-window-open-full-gui";
const API_SECRET = "fixture-api-secret-must-not-persist";

test("browser launch failure closes the fixture server", { timeout: 15_000 }, () => {
const env = { ...process.env, PLAYWRIGHT_BROWSERS_PATH: path.join(os.tmpdir(), randomUUID()) };
delete env.NODE_TEST_CONTEXT;
const result = spawnSync(process.execPath, [
"--test", "--test-name-pattern=^direct Chat window URL", __filename,
], {
env,
encoding: "utf8",
timeout: 10_000,
});
assert.ifError(result.error);
assert.equal(result.status, 1, "missing Chromium must fail promptly rather than hang");
assert.match(result.stdout + result.stderr, /Executable doesn't exist/);
});

function json(value) {
return JSON.stringify(value);
}
Expand Down Expand Up @@ -243,17 +260,14 @@ async function assertNoSecret(page, label) {

test("direct Chat window URL without an opener shows a safe unavailable shell", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
const callsResult = await installApiRoutes(context);
const pageErrors = [];
const page = await context.newPage();
page.on("pageerror", error => pageErrors.push(error.message));
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

await page.goto(`${server.baseUrl}?${POPUP_QUERY}`, { waitUntil: "domcontentloaded" });
await page.locator(CHAT_WINDOW_PLACEHOLDER).waitFor({ state: "visible" });
Expand All @@ -280,17 +294,14 @@ test("direct Chat window URL without an opener shows a safe unavailable shell",

test("Chat pop-out contains URL construction and live host getter failures", { timeout: 60_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
const pageErrors = [];
context.on("page", page => page.on("pageerror", error => pageErrors.push(error.message)));
await installInitScript(context, representativeConversation(), randomUUID());
const { calls } = await installApiRoutes(context);
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});
const main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
await selectChat(main);
Expand Down Expand Up @@ -334,7 +345,9 @@ test("Chat pop-out contains URL construction and live host getter failures", { t

test("Chat recovery focuses a safe control when no server disables the composer", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
await installInitScript(context, representativeConversation(), randomUUID());
await installApiRoutes(context, {
Expand All @@ -345,11 +358,6 @@ test("Chat recovery focuses a safe control when no server disables the composer"
runtime_generation: null,
},
});
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

const main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -382,15 +390,12 @@ test("Chat recovery focuses a safe control when no server disables the composer"

test("Recover chat here explains ownership contention and retries after the owner closes", { timeout: 60_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
await installInitScript(context, representativeConversation(), randomUUID());
const { calls } = await installApiRoutes(context);
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

const owner = await context.newPage();
await owner.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -437,7 +442,9 @@ test("Recover chat here explains ownership contention and retries after the owne

test("Chat initialization failure leaves an unavailable shell without breaking the GUI", { timeout: 60_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
let chatUiRouteHit = false;
await context.route(/\/js\/chat\/chat-main\.js(?:\?|$)/, async route => {
Expand All @@ -452,11 +459,6 @@ test("Chat initialization failure leaves an unavailable shell without breaking t
const pageErrors = [];
const page = await context.newPage();
page.on("pageerror", error => pageErrors.push(error.message));
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

await page.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
await page.waitForFunction(() => window.__phase4ChatInitFailureFixture === true);
Expand All @@ -480,7 +482,9 @@ test("Chat initialization failure leaves an unavailable shell without breaking t

test("real same-context Chat pop-out use and return cycle", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
const pageErrors = [];
context.on("page", page => page.on("pageerror", error => pageErrors.push(error.message)));
Expand All @@ -490,11 +494,6 @@ test("real same-context Chat pop-out use and return cycle", { timeout: 120_000 }
const { calls } = await installApiRoutes(context);
let main;
let popup;
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -695,7 +694,9 @@ test("real same-context Chat pop-out use and return cycle", { timeout: 120_000 }

test("blocked and blank pop-outs preserve ownership and a fresh transfer identity", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
const pageErrors = [];
context.on("page", page => page.on("pageerror", error => pageErrors.push(error.message)));
Expand All @@ -712,11 +713,6 @@ test("blocked and blank pop-outs preserve ownership and a fresh transfer identit
});
const { calls } = await installApiRoutes(context);
let main;
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -895,19 +891,16 @@ test("blocked and blank pop-outs preserve ownership and a fresh transfer identit

test("idle popup close and deletion recovery use an independently seeded transcript", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
await context.addInitScript(() => { window.__LLAMA_GUI_TEST_HOOKS__ = true; });
const pageErrors = [];
context.on("page", page => page.on("pageerror", error => pageErrors.push(error.message)));
const conversation = representativeConversation();
await installInitScript(context, conversation, `phase4-idle-${randomUUID()}`);
const { calls } = await installApiRoutes(context);
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

const main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -1034,7 +1027,9 @@ test("idle popup close and deletion recovery use an independently seeded transcr

test("popup reload and main reload recover the newest checkpoint without resend", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({ headless: true });
t.after(() => browser.close());
const context = await browser.newContext();
const conversation = representativeConversation();
const runNonce = `phase4-reload-${randomUUID()}`;
Expand All @@ -1043,11 +1038,6 @@ test("popup reload and main reload recover the newest checkpoint without resend"
const { calls } = await installApiRoutes(context);
let main;
let popup;
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

main = await context.newPage();
await main.goto(server.baseUrl, { waitUntil: "domcontentloaded" });
Expand Down Expand Up @@ -1179,21 +1169,18 @@ test("popup reload and main reload recover the newest checkpoint without resend"

test("insecure mapped origin keeps single-window Chat fallback usable", { timeout: 120_000 }, async t => {
const server = await startUiServer();
t.after(() => server.close());
const browser = await chromium.launch({
headless: true,
args: ["--host-resolver-rules=MAP popout-test.invalid 127.0.0.1"],
});
t.after(() => browser.close());
const context = await browser.newContext();
const conversation = representativeConversation();
const runNonce = `phase4-insecure-${randomUUID()}`;
await installInitScript(context, conversation, runNonce);
await installApiRoutes(context);
const main = await context.newPage();
t.after(async () => {
await context.close();
await browser.close();
await server.close();
});

const port = new URL(server.baseUrl).port;
await main.goto(`http://popout-test.invalid:${port}/`, { waitUntil: "domcontentloaded" });
Expand Down
Loading