Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,13 @@ jobs:
- name: Test
run: composer test

# The demo app installs the package from this checkout, as a project does: PHPStan reports
# the string check, and Pest shows the line of the test, not a frame of the package.
# Both runs fail on purpose, so their output is read, not their exit code.
# The demo cart has an empty total: the string check stays green, PHPStan reports it, the
# check by selector fails at the test line. Those two fail on purpose: their output is read.
- name: Demo
working-directory: demo/app
run: |
composer install --no-interaction --no-progress
vendor/bin/pest tests/StringTest.php
phpstan=$(vendor/bin/phpstan analyse --no-progress --error-format=json || true)
grep -q html.markupAsString <<< "$phpstan"
pest=$(vendor/bin/pest --colors=never tests/CartTest.php || true)
Expand Down
72 changes: 45 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
[![PHP Version](https://img.shields.io/packagist/php-v/sorge-it/phpunit-pest-html-assertions)](composer.json)
[![PHPStan](https://img.shields.io/badge/PHPStan-level%20max-brightgreen)](phpstan.neon)

Check rendered HTML with CSS selectors, never as a string. Built for tests that AI coding agents
write and run: in Laravel, Livewire, Symfony and TYPO3.
**Your coding agent's tests are green, and the page is broken.** Agents check HTML the quickest
way, as a string, and such a check passes by accident and fails for nothing. This package makes
every check ask the DOM, and keeps your agent doing it.

```php
expect($this->get('/cart'))
Expand All @@ -19,11 +20,49 @@ expect($this->get('/cart'))

**Works with:** PHPUnit · Pest · Laravel · Livewire · Symfony · TYPO3 · PSR-7 · Laravel Boost · Claude Code · PHPStan · Rector

![PHPStan reports a check of markup as a string; the check by selector fails and shows the region](https://raw.githubusercontent.com/sorge-it/phpunit-pest-html-assertions/main/demo/output/demo.gif)
![A string check stays green on a broken page, PHPStan reports it, and the check by selector fails and shows the region](https://raw.githubusercontent.com/sorge-it/phpunit-pest-html-assertions/main/demo/output/demo.gif)

PHPStan reports a check of markup as a string. The check by selector fails, names the region,
gives the reason and shows the HTML of the region, at the line of the test. The demo is recorded
from the app in [`demo/`](demo).
The total on the cart page is empty. A string check stays green, PHPStan reports it, and the check
by selector goes red at the line of the test and shows the HTML. The demo runs the app in
[`demo/`](demo), and the CI runs it on every change.

## Why this exists

I hold my tests to one rule: green means the page works, red means it does not.

Tests that read HTML as a string break that rule in both directions. One of mine went red because
one more `<span>` wrapped a dot, and the page was fine. The other direction is worse:

```php
// The total is empty on the page. The test stays green:
// it finds the marker, not the amount.
expect($html)->toContain('data-total');

// This one goes red and shows the region.
expect($html)->toHaveSelectorText('[data-total]', '42.00 EUR');
```

A string check also passes when its text sits only in an attribute or a script. Some string checks
test nothing at all. Some are green by luck.

This matters more now than it used to. Coding agents produce code faster than anyone can click
through it. Testing the UI by hand on every change costs too much, so in practice it does not
happen. Browser tests are too slow and too heavy to cover every detail; they keep their job for
JavaScript and CSS. That leaves the HTML the server renders, checked in a feature test. The agent
writes these tests for almost nothing, and we need many of them. They only help if they are right.

In one of my Laravel apps, I counted 502 lines in 53 test files that checked the page as a string.
The agent had not done anything wrong. It had used the tools it had. So I gave it better ones:

- before it writes, a [skill](#for-coding-agents) gives it the rules;
- when it checks its work, [PHPStan](#phpstan) reports a check of markup as a string;
- when a test fails, the [message](#when-a-check-fails) shows the region as HTML, at the line of
the test.

The same app now runs 591 checks by CSS selector, and PHPStan stops a new string check of markup
before it lands.

I built this so I can trust a green test again. I hope it lets you trust yours.

## Installation

Expand Down Expand Up @@ -67,16 +106,6 @@ final class CartTest extends TestCase
Every method of the trait starts with `assertHtml`. So the trait also works in a Symfony
`WebTestCase`, which has its own `assertSelectorExists()` and similar methods.

## Why

Coding agents write many tests. Without help, they check HTML the way it is easiest to type:
`assertSee('<span class="dot">')`, a regular expression over the markup. Those tests fail when one
more `<span>` wraps a dot, though the page looks the same. And they pass when the text they look for
sits only in an attribute or a script.

This package reads the DOM instead. `symfony/dom-crawler` parses the page with `Dom\HTMLDocument`, as
a browser does, and every check selects with CSS.

## Compared with

| Tool | What it checks | The difference |
Expand All @@ -87,17 +116,6 @@ a browser does, and every check selects with CSS.
| Laravel Dusk, Pest's browser tests | a real browser | for JavaScript, CSS and clicks; slower. This package checks the HTML the server renders, in a unit or feature test |
| HTML snapshots | the whole page | fail on every change; an update accepts the whole new page |

## Built for coding agents

The package acts at each step of the loop in which an agent writes a test:

| Step | What acts |
|---|---|
| Before the agent writes | A [skill](#for-coding-agents) gives the agent the checks and the rules for selectors: through Laravel Boost, Claude Code or the `skills` CLI. Boost also adds a short guideline to every task. |
| When the agent checks its work | The [PHPStan rules](#phpstan) `html.markupAsString` and `html.classAsString` report a check of markup or of a class as a string and say what to use instead. |
| When a test fails | The [message](#when-a-check-fails) names the region, what was asked, what was found, and shows the HTML of the region. It points at the line of the test, not into this package. The agent can fix the test without a browser. |
| In an existing suite | A [Rector rule](#rector) rewrites the mechanical forms of crawler code. |

## Requirements

- PHP 8.3, 8.4 or 8.5
Expand Down
3 changes: 2 additions & 1 deletion demo/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ build: ## Build the image: VHS with PHP and Composer
install: build ## Install the demo app, with the package from this checkout
$(RUN) sh -c 'cd demo/app && rm -rf vendor composer.lock && composer install --no-interaction --no-progress -q'

preflight: install ## Check that both scenes show what the recording expects
preflight: install ## Check that every scene shows what the recording expects
$(RUN) sh -c 'cd demo/app && vendor/bin/pest tests/StringTest.php'
$(RUN) sh -c 'cd demo/app && vendor/bin/phpstan analyse --no-progress --error-format=json | grep -q "html.markupAsString"'
$(RUN) sh -c 'cd demo/app && vendor/bin/pest --colors=never tests/CartTest.php | grep -q "at tests/CartTest.php"'
$(RUN) sh -c 'cd demo/app && ! vendor/bin/pest --colors=never tests/CartTest.php | grep -q "src/PHPUnit/"'
Expand Down
4 changes: 2 additions & 2 deletions demo/app/cart.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

declare(strict_types=1);

/** The page of a cart, as a view would render it. */
/** The page of a cart, as a view renders it, with a bug: the total is empty. */
function cart(): string
{
return <<<'HTML'
Expand All @@ -12,7 +12,7 @@ function cart(): string
<li data-name="Apple">Apple</li>
<li data-name="Pear">Pear</li>
</ul>
<p data-total>42.00 EUR</p>
<p data-total></p>
</main>
HTML;
}
4 changes: 2 additions & 2 deletions demo/app/social.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

declare(strict_types=1);

// The screen of the social preview: the name, then the failure without Pest's header.
// The screen of the social preview: the name, then the failure up to the line of the test.

exec('vendor/bin/pest --colors=always tests/CartTest.php', $lines);

Expand All @@ -13,7 +13,7 @@
$failure[] = $line;
}

if (str_contains($line, 'Tests:')) {
if (str_contains($line, '➜')) {
break;
}
}
Expand Down
8 changes: 3 additions & 5 deletions demo/app/tests/CartTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@

declare(strict_types=1);

use function SorgeIt\PhpunitPestHtmlAssertions\Pest\html;
it('shows the total', function () {
$html = cart();

it('sells ripe pears', function () {
expect(html(cart()))
->within('[data-cart]')
->toHaveSelectorText('[data-name="Pear"]', 'Pear, ripe', 'The shop sells ripe pears only.');
expect($html)->toHaveSelectorText('[data-total]', '42.00 EUR');
});
6 changes: 4 additions & 2 deletions demo/app/tests/StringTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

declare(strict_types=1);

it('sells ripe pears', function () {
expect(cart())->toContain('<li data-name="Pear">Pear, ripe</li>');
it('shows the total', function () {
$html = cart();

expect($html)->toContain('data-total');
});
13 changes: 10 additions & 3 deletions demo/demo.tape
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# The animated demo in the README: PHPStan reports a check of markup as a
# string, the check by selector fails and names the region.
# The animated demo in the README: the total on the cart page is empty. A
# string check stays green, PHPStan reports it, and the check by selector
# fails and shows the region.
# Run through `make -C demo gif`; https://github.com/charmbracelet/vhs

Output demo/output/demo.gif
Expand All @@ -23,6 +24,12 @@ Sleep 800ms
Type "cat tests/StringTest.php"
Enter
Sleep 2s
Type "vendor/bin/pest tests/StringTest.php"
Enter
Sleep 4s
Type "grep data-total cart.php"
Enter
Sleep 2.5s
Type "vendor/bin/phpstan analyse --no-progress"
Enter
Sleep 6s
Expand All @@ -31,7 +38,7 @@ Type "clear"
Enter
Type "cat tests/CartTest.php"
Enter
Sleep 2.5s
Sleep 2s
Type "vendor/bin/pest tests/CartTest.php"
Enter
Sleep 9s
Binary file modified demo/output/demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified demo/output/social.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading