Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/frames-arm-when-and-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'playwright-layout-guard': minor
---

Three additions that came out of the trial on public pages:

- **`ignoreFrames`** leaves single iframes alone, by a text that their address contains (`*` for any characters), for widgets you do not control. The box of the iframe in your page is still watched.
- **`armWhen`** is a CSS selector of something that is shown when an app that draws itself after the load event is ready: the guard reports nothing until it matches, takes the layout then, and reports from there on. `guard.armed()` says whether the guard is watching, and a test whose selector never matched fails instead of passing without having watched anything.
- **A command line**: `npx playwright-layout-guard scan <address...>` opens pages in a browser and reports every box that moved after the load, next to the layout shift score of the browser (Chromium). Addresses can come from a file (`--file`), the report can be text, JSON, Markdown or HTML (`--format`, `--out`), the options come from a JSON file (`--options`), `--immediate` and `--arm-when` set `start` and `armWhen`, `--screenshots` saves the picture with the moved boxes marked, and the exit code is 0, 1 (something moved) or 2 (wrong command, a page that could not be loaded, an `--arm-when` element that never showed). The script of the trial is the first version of it.
79 changes: 60 additions & 19 deletions README.md

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,14 @@ test.use({ layoutGuard: { ignore: ['.ad-slot', '.live-ticker', '[data-carousel]'

The selectors are CSS selectors; the element and everything inside it is left alone. Keep the list short and say why in a comment: every entry is a place where the guard is blind.

An embedded widget in an iframe has a page of its own that the guard watches too. Leave it out by its address with `ignoreFrames`:

```ts
test.use({ layoutGuard: { ignoreFrames: ['widget.example'] } });
```

The iframe box in your page is still watched (it can push your content), the page inside it is not.

## An app that renders after the load

A client-side rendered app is an empty page at the load event and fills in afterwards, which looks like a page that moves. Wait until the app has rendered, then tell the guard to start from there:
Expand All @@ -76,6 +84,14 @@ test('the dashboard', async ({ page, guard }) => {
});
```

Or let the guard wait for the app by itself: name something that is shown when it is ready.

```ts
test.use({ layoutGuard: { armWhen: '#app' } });
```

The guard reports nothing until the element is shown, takes the layout then, and reports from there on. If the element never shows (a wrong selector, an app that failed to start), the test fails and says so, because nothing was watched.

Server-rendered pages need none of this. A framework that renders on the server and then **hydrates** is what the guard is best at: the page must look the same before and after.

## Several pages in one test
Expand Down
2 changes: 1 addition & 1 deletion docs/survey.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ What this says:
- **`immediate` finds most of what CLS finds** (8 of 9), but 30 of 46 loads report something and 16 hit the limit: a page that is still being delivered in pieces does move, and the guard says so. Use it for your own pages when the load itself is the thing to check, not as a default.
- **The default mode is quiet on most real pages** (38 of 46 loads without a report). What it reports was checked by hand:
- **Real layout changes after the load:** web.dev (a footer pushed 78 px down), nodejs.org (the navigation settles 4 px at a time), bbc.com (a few pixels when a script finishes) and the Vue guide (the whole app moves 72 px up when a bar above it goes away). Some pages show up in one run and not in the next (MDN on a phone: a sidebar appears at 1 s and pushes the whole article 392 px down, with a CLS of 0 because the article was below the window), because what a page does after the load depends on timing.
- **Small frames:** react.dev embeds frames that are 15 px high and grow to 20 px. They are reported with the name of the frame (`[frame /path]`). There is no option yet to leave out a single frame (listed as an idea in the README).
- **Small frames:** react.dev embeds frames that are 15 px high and grow to 20 px. They are reported with the name of the frame (`[frame /path]`). Since 0.4 a frame can be left out with `ignoreFrames`.

## Pages with a report or a poor CLS

Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@
"sideEffects": false,
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"bin": {
"playwright-layout-guard": "./dist/bin.js"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
Expand Down
4 changes: 4 additions & 0 deletions src/bin.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env node
import { main } from './cli.js';

process.exitCode = await main(process.argv.slice(2));
214 changes: 214 additions & 0 deletions src/cli.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
import { describe, expect, it } from 'vitest';

import {
exitCodeOf,
fileNameOf,
mergeOptions,
normalizeUrl,
type PageResult,
parseCli,
parseUrlList,
renderReport,
renderText,
} from './cli.js';
import type { Shift } from './types.js';

const shift = (element: string, extra: Partial<Shift> = {}): Shift => ({
kind: 'moved',
element,
text: 'Hello',
before: { top: 100, left: 0, width: 600, height: 40 },
after: { top: 160, left: 0, width: 600, height: 40 },
affected: 2,
at: 1200,
url: '/',
...extra,
});

const calm: PageResult = { url: 'https://a.example/', http: 200, cls: 0, armed: true, shifts: [] };
const moving: PageResult = {
url: 'https://b.example/',
finalUrl: 'https://b.example/home',
http: 200,
cls: 0.1281,
armed: true,
shifts: [shift('main'), shift('footer', { affected: 0 })],
screenshot: 'pictures/02-b-example.png',
};
const broken: PageResult = {
url: 'https://c.example/',
error: 'page.goto: Timeout 45000ms exceeded.',
};
const waiting: PageResult = { url: 'https://d.example/', http: 200, armed: false, shifts: [] };

describe('parseCli', () => {
it('reads addresses and options', () => {
const parsed = parseCli([
'scan',
'example.com',
'http://localhost:3000/a',
'--browser',
'webkit',
'--format=json',
'--fail-on',
'never',
'--wait',
'1.5',
'--timeout',
'10',
'--no-scroll',
'--immediate',
'--arm-when',
'#app',
'--out',
'report/layout.json',
'--options',
'guard.json',
'--device',
'iPhone 13',
'--file',
'urls.txt',
'--screenshots',
'pictures',
]);
expect(parsed).toEqual({
kind: 'scan',
scan: {
urls: ['https://example.com/', 'http://localhost:3000/a'],
file: 'urls.txt',
browser: 'webkit',
format: 'json',
out: 'report/layout.json',
failOn: 'never',
wait: 1.5,
scroll: false,
timeout: 10_000,
immediate: true,
armWhen: '#app',
optionsFile: 'guard.json',
device: 'iPhone 13',
screenshots: 'pictures',
},
});
});

it('has defaults', () => {
expect(parseCli(['scan', 'https://example.com'])).toMatchObject({
kind: 'scan',
scan: { browser: 'chromium', format: 'text', failOn: 'shift', wait: 4, scroll: true },
});
});

it('answers help, version and no command', () => {
expect(parseCli(['--help'])).toEqual({ kind: 'help' });
expect(parseCli(['-h'])).toEqual({ kind: 'help' });
expect(parseCli([])).toEqual({ kind: 'help' });
expect(parseCli(['--version'])).toEqual({ kind: 'version' });
});

it('says what is wrong', () => {
const message = (args: string[]): string => {
const parsed = parseCli(args);
return parsed.kind === 'error' ? parsed.message : parsed.kind;
};
expect(message(['crawl', 'example.com'])).toContain('unknown command "crawl"');
expect(message(['scan', 'example.com', '--browser', 'edge'])).toContain(
'--browser is one of chromium, firefox, webkit',
);
expect(message(['scan', 'example.com', '--format', 'xml'])).toContain('--format');
expect(message(['scan', 'example.com', '--fail-on', 'error'])).toContain('--fail-on');
expect(message(['scan', 'example.com', '--wait', 'soon'])).toContain('--wait needs a number');
expect(message(['scan', 'ftp://example.com'])).toContain('not an http or https address');
expect(message(['scan', 'example.com', '--nope'])).toContain('--nope');
});
});

describe('addresses', () => {
it('adds https when there is no scheme', () => {
expect(normalizeUrl('example.com/a')).toBe('https://example.com/a');
expect(normalizeUrl('file:///etc/passwd')).toBeUndefined();
});

it('reads a list: one per line, comments and empty lines left out', () => {
expect(parseUrlList('# pages\nexample.com\n\nexample.org/a # second\r\n#last\n')).toEqual([
'example.com',
'example.org/a',
]);
});

it('makes a file name from an address', () => {
expect(fileNameOf('https://www.example.com/docs/intro?a=1', 1)).toBe(
'02-www-example-com-docs-intro.png',
);
});
});

describe('options', () => {
it('puts the flags of the command line over the file', () => {
expect(
mergeOptions({ ignore: ['.ad'], start: 'load' }, { immediate: true, armWhen: '#app' }),
).toEqual({ ignore: ['.ad'], start: 'immediate', armWhen: '#app' });
expect(mergeOptions({ start: 'immediate' }, { immediate: false, armWhen: undefined })).toEqual({
start: 'immediate',
});
});
});

describe('exit code', () => {
it('is 0 for calm pages and 1 for a shift', () => {
expect(exitCodeOf([calm], 'shift')).toBe(0);
expect(exitCodeOf([calm, moving], 'shift')).toBe(1);
expect(exitCodeOf([moving], 'never')).toBe(0);
});

it('is 2 when a page could not be loaded or the guard never started to watch', () => {
expect(exitCodeOf([broken], 'never')).toBe(2);
expect(exitCodeOf([moving, broken], 'shift')).toBe(2);
expect(exitCodeOf([waiting], 'shift')).toBe(2);
});
});

describe('the reports', () => {
it('writes text with a block per page and the score of the browser', () => {
const text = renderText([calm, moving, broken, waiting]);
expect(text).toContain('https://a.example/ (HTTP 200, CLS 0)\n nothing moved');
expect(text).toContain(
'https://b.example/ (now https://b.example/home) (HTTP 200, CLS 0.1281)',
);
expect(text).toContain('2 shifts:');
expect(text).toContain('1. <main> "Hello" moved 60 px down, with 2 elements inside it');
expect(text).toContain('picture: pictures/02-b-example.png');
expect(text).toContain('could not be loaded: page.goto: Timeout');
expect(text).toContain('never started to watch');
expect(text).toContain('4 pages: 1 page with shifts (2 shifts), 1 could not be loaded');
});

it('writes JSON that can be read back', () => {
const json = JSON.parse(renderReport([calm, moving, broken], 'json', 'webkit', 'then')) as {
version: number;
browser: string;
pages: PageResult[];
};
expect(json).toMatchObject({ version: 1, browser: 'webkit' });
expect(json.pages[1]?.shifts).toHaveLength(2);
expect(json.pages[2]?.error).toContain('Timeout');
});

it('writes a Markdown table with the biggest move of each page', () => {
const markdown = renderReport([calm, moving, broken], 'markdown', 'chromium');
expect(markdown).toContain('## Layout guard');
expect(markdown).toContain('| https://b.example/ | 2 | 0.1281 | <main>');
expect(markdown).toContain('| https://c.example/ | could not be loaded');
});

it('writes an HTML page that escapes what it shows', () => {
const html = renderReport(
[{ ...moving, shifts: [shift('div.<script>')] }, broken],
'html',
'chromium',
);
expect(html).toContain('<!doctype html>');
expect(html).toContain('&lt;script&gt;');
expect(html).not.toContain('div.<script>');
});
});
Loading
Loading