Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "sorge-it",
"description": "Skills for the PHP packages of Sorge IT.",
"owner": {
"name": "Stefan Sorge",
"url": "https://github.com/sorge-it"
},
"plugins": [
{
"name": "phpunit-pest-html-assertions",
"source": "./",
"description": "Check rendered HTML in PHPUnit and Pest tests with CSS selectors, never as a string."
}
]
}
13 changes: 13 additions & 0 deletions .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "phpunit-pest-html-assertions",
"description": "Check rendered HTML in PHPUnit and Pest tests with CSS selectors, never as a string: the skill of sorge-it/phpunit-pest-html-assertions.",
"author": {
"name": "Stefan Sorge",
"url": "https://github.com/sorge-it"
},
"homepage": "https://github.com/sorge-it/phpunit-pest-html-assertions",
"repository": "https://github.com/sorge-it/phpunit-pest-html-assertions",
"license": "MIT",
"keywords": ["php", "phpunit", "pest", "html", "testing", "css-selector"],
"skills": ["./resources/boost/skills/"]
}
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
* text=auto eol=lf

/.claude-plugin export-ignore
/.github export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
/CODE_OF_CONDUCT.md export-ignore
/CONTRIBUTING.md export-ignore
/SECURITY.md export-ignore
/composer.lock export-ignore
/context7.json export-ignore
/demo export-ignore
/fixtures export-ignore
/phpstan.neon export-ignore
/phpunit.xml export-ignore
Expand Down
5 changes: 5 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<!--
What went wrong or what is missing, and why this change is the answer.

A test fails without the change, and `composer test` passes. See CONTRIBUTING.md.
-->
13 changes: 13 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,16 @@ jobs:

- name: Test
run: composer test

# The demo app installs the package from this checkout, as a project does: PHPStan reports
# the string check, and Pest shows the line of the test, not a frame of the package.
# Both runs fail on purpose, so their output is read, not their exit code.
- name: Demo
working-directory: demo/app
run: |
composer install --no-interaction --no-progress
phpstan=$(vendor/bin/phpstan analyse --no-progress --error-format=json || true)
grep -q html.markupAsString <<< "$phpstan"
pest=$(vendor/bin/pest --colors=never tests/CartTest.php || true)
grep -q 'at tests/CartTest.php' <<< "$pest"
if grep -q 'src/PHPUnit/' <<< "$pest"; then echo 'A frame of the package is in the trace.'; exit 1; fi
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,41 @@
# Changelog

## 1.4.0

### Added

- A guideline for Laravel Boost in `resources/boost/guidelines/core.blade.php`. Boost gives it to the
agent with every task, the skill only when a task needs it. Boost installs both once the package is
picked on `php artisan boost:install` or listed in `packages` of `boost.json`; the README said
before that Boost found the skill by itself.
- The skill installs in Claude Code as a plugin of this repository, and in other agents with
`npx skills add sorge-it/phpunit-pest-html-assertions`. See the README, section For coding agents.
- A demo in the README, recorded with VHS from the app in `demo/`.

### Fixed

- A failure points at the line of the test. Before, Pest showed the line in
`src/PHPUnit/Assertion.php` of this package, and the line of the test came only further down. The
package now adds its `src` directory to PHPUnit's `ExcludeList`, as PHPUnit does with its own
code. The printed trace of PHPUnit leaves out these frames too. `HTML_ASSERTIONS_SHOW_FRAMES=1`,
set in the shell, keeps them, for a bug report. The checkout of the package keeps them always.

### Changed

- The package requires Composer 2.1 (`composer-runtime-api: ^2.1`).
- Composer loads the new `src/PHPUnit/Autoload.php` in every process that loads `vendor/autoload.php`:
it adds the `src` directory to PHPUnit's `ExcludeList` and has no other effect.
- A deprecation triggered inside the package names PHPUnit as its source, not a third party. Its
category (self, direct, indirect) stays the same.
- A problem comes as a pull request, not as an issue: issues are closed. See CONTRIBUTING.md.
- A vulnerability is reported privately through GitHub, not by email. See SECURITY.md.

### Upgrade

- Composer 1 and Composer 2.0 stay on 1.3. Update Composer to 2.1 or later.
- A test that expects frames of this package in a trace runs with `HTML_ASSERTIONS_SHOW_FRAMES=1`
set in the shell. In `phpunit.xml` it comes too late: Composer loads the package first.

## 1.3.0

### Added
Expand Down
6 changes: 6 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Code of conduct

This project follows the [Contributor Covenant, version 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).

Report unacceptable behavior to GitHub with "Report content" on the comment or pull request, or
through [GitHub's report form](https://github.com/contact/report-abuse).
39 changes: 39 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Contributing

Pull requests are welcome. Issues are closed: a problem comes as a pull request that fixes it.

## A problem

Open a pull request with:

- a test that fails without the change;
- the change;
- a description of what went wrong and why the change fixes it.

If you do not know how to fix it, describe the problem to a coding agent and let it draft the pull
request. Read the change and run the tests before you open it.

In a project, a failure hides the frames of this package; the checkout of the package keeps them. To
see where a problem in your project comes from inside the package, run the test with
`HTML_ASSERTIONS_SHOW_FRAMES=1` set in the shell, for example
`HTML_ASSERTIONS_SHOW_FRAMES=1 vendor/bin/pest`. In `phpunit.xml` it comes too late.

## A feature

Open a pull request with tests and a section in the README. The package checks the HTML a server
renders; see "Not in scope" in the README. A feature outside of that is declined.

## Before you open it

```sh
composer test # Pint and Rector as a dry run, PHPStan at max, type coverage 100 %, the test suites
composer lint # Rector, then Pint
```

The code must run on PHP 8.3. When a message or the output of a check changes, record the demo again
with `make -C demo all` (needs Docker).

## Review

A pull request can be reviewed when it is ready for review and `composer test` passes in the CI. A
draft without activity can be closed. Pull requests opened in bulk are closed without review.
158 changes: 100 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
[![Latest Version](https://img.shields.io/packagist/v/sorge-it/phpunit-pest-html-assertions)](https://packagist.org/packages/sorge-it/phpunit-pest-html-assertions)
[![Total Downloads](https://img.shields.io/packagist/dt/sorge-it/phpunit-pest-html-assertions)](https://packagist.org/packages/sorge-it/phpunit-pest-html-assertions)
[![License](https://img.shields.io/packagist/l/sorge-it/phpunit-pest-html-assertions)](LICENSE.md)
[![PHP Version](https://img.shields.io/packagist/php-v/sorge-it/phpunit-pest-html-assertions)](composer.json)
[![PHPStan](https://img.shields.io/badge/PHPStan-level%20max-brightgreen)](phpstan.neon)

Check rendered HTML with CSS selectors, never as a string. Built for tests that AI coding agents
write and run: in Laravel, Livewire, Symfony and TYPO3.
Expand All @@ -15,50 +17,13 @@ expect($this->get('/cart'))
->not->toHaveSelector('[data-errors]');
```

**Works with:** PHPUnit · Pest · Laravel · Livewire · Symfony · TYPO3 · PSR-7 · Laravel Boost · PHPStan · Rector
**Works with:** PHPUnit · Pest · Laravel · Livewire · Symfony · TYPO3 · PSR-7 · Laravel Boost · Claude Code · PHPStan · Rector

## Why

Coding agents write a large share of our tests. Left alone, they check HTML the way it is easiest to
type: `assertSee('<span class="dot">')`, a regular expression over the markup. Those tests fail when
one more `<span>` wraps a dot, though the page looks the same. And they pass when the text they look
for sits only in an attribute or a script.

This package asks the DOM instead. `symfony/dom-crawler` parses the page with `Dom\HTMLDocument`, as
a browser does, and every check selects with CSS.

## Built for coding agents

Each part of the package acts at one step of the loop in which an agent writes a test:

| Step | Part |
|---|---|
| Before the agent writes | A [Laravel Boost](#laravel-boost) skill gives the agent the checks and the rules for selectors. |
| When the agent checks its work | The [PHPStan rules](#phpstan) `html.markupAsString` and `html.classAsString` report a check of markup or of a class as a string and say what to use instead. |
| When a test fails | The [message](#when-a-check-fails) names the region, what was asked, what was found, and shows the HTML of the region. The agent can fix the test without a browser. |
| In an existing suite | A [Rector rule](#rector) rewrites the mechanical forms of crawler code. |

## Parts

The package has four parts:

| Part | Namespace | What it holds |
|---|---|---|
| PHPUnit | `SorgeIt\PhpunitPestHtmlAssertions\PHPUnit` | the constraints, `Html` (a page or a region of it) and the `AssertsHtml` trait |
| Pest | `SorgeIt\PhpunitPestHtmlAssertions\Pest` | the expectations and the function `html()` |
| PHPStan | `SorgeIt\PhpunitPestHtmlAssertions\PHPStan` | the rules `html.markupAsString` and `html.classAsString`, which report a check of markup or of a class as a string |
| Rector | `SorgeIt\PhpunitPestHtmlAssertions\Rector` | a rule that rewrites the mechanical forms of crawler code |

## Requirements

- PHP 8.3, 8.4 or 8.5
- PHPUnit 12.5 or 13
- Pest 4 or 5, optional, for the expectations
- the PHP extensions `dom` and `mbstring`
- Symfony DomCrawler and CssSelector 7.4 or 8.1
![PHPStan reports a check of markup as a string; the check by selector fails and shows the region](https://raw.githubusercontent.com/sorge-it/phpunit-pest-html-assertions/main/demo/output/demo.gif)

The CI tests three stacks: PHP 8.3 with PHPUnit 12, Pest 4, Laravel 12 and Symfony 7.4; PHP 8.4 with
the newest versions; PHP 8.5 with the versions of `composer.lock`.
PHPStan reports a check of markup as a string. The check by selector fails, names the region,
gives the reason and shows the HTML of the region, at the line of the test. The demo is recorded
from the app in [`demo/`](demo).

## Installation

Expand Down Expand Up @@ -102,6 +67,60 @@ final class CartTest extends TestCase
Every method of the trait starts with `assertHtml`. So the trait also works in a Symfony
`WebTestCase`, which has its own `assertSelectorExists()` and similar methods.

## Why

Coding agents write many tests. Without help, they check HTML the way it is easiest to type:
`assertSee('<span class="dot">')`, a regular expression over the markup. Those tests fail when one
more `<span>` wraps a dot, though the page looks the same. And they pass when the text they look for
sits only in an attribute or a script.

This package reads the DOM instead. `symfony/dom-crawler` parses the page with `Dom\HTMLDocument`, as
a browser does, and every check selects with CSS.

## Compared with

| Tool | What it checks | The difference |
|---|---|---|
| `assertSee()`, `assertSeeHtml()`, `toContain()` | a string | passes on text in an attribute or a script, fails when the markup around the text changes |
| Symfony's `assertSelectorTextContains()` and the other `assertSelector…()` methods | CSS selectors | only in a `WebTestCase`, on the client's last response; no regions; the message shows no HTML |
| `sinnbeck/laravel-dom-assertions` | CSS selectors | Laravel's test responses, views and Livewire components; this package also reads Symfony, TYPO3 and PSR-7 responses and adds PHPStan rules |
| Laravel Dusk, Pest's browser tests | a real browser | for JavaScript, CSS and clicks; slower. This package checks the HTML the server renders, in a unit or feature test |
| HTML snapshots | the whole page | fail on every change; an update accepts the whole new page |

## Built for coding agents

The package acts at each step of the loop in which an agent writes a test:

| Step | What acts |
|---|---|
| Before the agent writes | A [skill](#for-coding-agents) gives the agent the checks and the rules for selectors: through Laravel Boost, Claude Code or the `skills` CLI. Boost also adds a short guideline to every task. |
| When the agent checks its work | The [PHPStan rules](#phpstan) `html.markupAsString` and `html.classAsString` report a check of markup or of a class as a string and say what to use instead. |
| When a test fails | The [message](#when-a-check-fails) names the region, what was asked, what was found, and shows the HTML of the region. It points at the line of the test, not into this package. The agent can fix the test without a browser. |
| In an existing suite | A [Rector rule](#rector) rewrites the mechanical forms of crawler code. |

## Requirements

- PHP 8.3, 8.4 or 8.5
- PHPUnit 12.5 or 13
- Pest 4 or 5, optional, for the expectations
- the PHP extensions `dom` and `mbstring`
- Symfony DomCrawler and CssSelector 7.4 or 8.1
- Composer 2.1 or later

The CI tests three stacks: PHP 8.3 with PHPUnit 12, Pest 4, Laravel 12 and Symfony 7.4; PHP 8.4 with
the newest versions; PHP 8.5 with the versions of `composer.lock`.

## Namespaces

| Namespace | What it holds |
|---|---|
| `SorgeIt\PhpunitPestHtmlAssertions\PHPUnit` | the constraints, `Html` (a page or a region of it) and the `AssertsHtml` trait |
| `SorgeIt\PhpunitPestHtmlAssertions\Pest` | the expectations and the function `html()` |
| `SorgeIt\PhpunitPestHtmlAssertions\PHPStan` | the rules `html.markupAsString` and `html.classAsString`, which report a check of markup or of a class as a string |
| `SorgeIt\PhpunitPestHtmlAssertions\Rector` | a rule that rewrites the mechanical forms of crawler code |

The guideline and the skill for coding agents lie in `resources/boost/`.

## What a check reads

Each check takes one of these:
Expand Down Expand Up @@ -133,7 +152,7 @@ Each row gives the call and the sentence of its message when it fails, after the
| `toHaveSelector('[data-cart]')` | `assertHtmlSelectorExists` | has a node matching "[data-cart]" |
| — | `assertHtmlSelectorNotExists` | does not have a node matching "[data-cart]" |
| `toHaveSelectorCount('li', 3)` | `assertHtmlSelectorCount` | has 3 nodes matching "li" |
| `toHaveSelectorCountAtLeast('li', 1)` | `assertHtmlSelectorCountAtLeast` | has at least 1 nodes matching "li" |
| `toHaveSelectorCountAtLeast('li', 2)` | `assertHtmlSelectorCountAtLeast` | has at least 2 nodes matching "li" |
| `toHaveSelectorText('h1', 'Orders')` | `assertHtmlSelectorTextSame` | has one node matching "h1" with the text "Orders" |
| `toHaveSelectorTextContaining('h1', 'Ord')` | `assertHtmlSelectorTextContains` | has one node matching "h1" whose text contains "Ord" |
| `toHaveAnySelectorText('li', 'Apple')` | `assertHtmlAnySelectorTextSame` | has a node matching "li" with the text "Apple" |
Expand All @@ -145,7 +164,7 @@ Each row gives the call and the sentence of its message when it fails, after the
| `toHaveSelectorClass('[data-status]', 'bg-red-500')` | `assertHtmlSelectorClass` | has one node matching "[data-status]" with the class "bg-red-500" |
| `toHaveText('Apple Pear')` | `assertHtmlTextSame` | has the text "Apple Pear" |
| `toHaveTextContaining('Apple')` | `assertHtmlTextContains` | has a text that contains "Apple" |
| `toHaveTextCount('Apple', 1)` | `assertHtmlTextCount` | has the text "Apple" 1 times |
| `toHaveTextCount('Apple', 2)` | `assertHtmlTextCount` | has the text "Apple" 2 times |
| `toAppearBefore('[data-head]', '[data-body]')` | `assertHtmlSelectorBefore` | has the node matching "[data-head]" before the node matching "[data-body]" |
| `toHaveTitle('Orders')` | `assertHtmlPageTitleSame` | has the title "Orders" |
| `toHaveInputValue('email', 'anna@example.com')` | `assertHtmlInputValueSame` | has one field named "email" with the value "anna@example.com" |
Expand Down Expand Up @@ -278,9 +297,13 @@ NotOneNode: (page) > [data-cart]: a check of one node needs exactly one match. T

`dump()` and `dd()` of Pest show the same path and region.

A failure points at the line of the test. The trace leaves out the frames of this package, as it
leaves out those of PHPUnit. For a bug report, run the test with `HTML_ASSERTIONS_SHOW_FRAMES=1`
set in the shell, not in `phpunit.xml`: the trace then keeps them.

## Which selector

1. The element, its role, `aria-*`, its label or its visible text.
1. The tag, its role, `aria-*`, its label or its visible text.
2. A `data-*` marker: the functional marker that the app (Alpine, scripts) and the tests share. A CSS
class is for the look, not for a selector.
3. A class is a value to check (`toHaveSelectorClass('[data-status]', 'bg-red-500')`), not the
Expand Down Expand Up @@ -416,31 +439,50 @@ change slightly: `texts()` leaves out a script, a style or a `template` inside a

Add it to the `rector.php` of a project for a migration: `->withRules([CrawlerToHtmlRector::class])`.

## Laravel Boost
## For coding agents

The package ships a skill in the [Agent Skills](https://agentskills.io) format and a short guideline
for Laravel Boost. With them, an agent writes these checks instead of string checks.

**Laravel Boost** lists the package on `php artisan boost:install`, also as a `require-dev`
dependency. Pick it in the list of third-party guidelines and skills, or add it to `packages` in
`boost.json`. Boost then writes the guideline into the guideline file of the agent, for example
`CLAUDE.md`, and installs the skill:

```json
{
"packages": ["sorge-it/phpunit-pest-html-assertions"]
}
```

**Claude Code** installs the skill as a plugin of this repository:

The package ships a skill for [Laravel Boost](https://github.com/laravel/boost) in
`resources/boost/skills/`. Boost finds it in every direct dependency and gives it to your coding
agent, so the agent writes these checks instead of string checks.
```
/plugin marketplace add sorge-it/phpunit-pest-html-assertions
/plugin install phpunit-pest-html-assertions@sorge-it
```

**Other agents** (Cursor, Codex, GitHub Copilot and more) install it with the
[`skills` CLI](https://github.com/vercel-labs/skills):

```sh
npx skills add sorge-it/phpunit-pest-html-assertions
```

## Not in scope

- CSS: whether a rule takes effect is a question for a browser test (`getComputedStyle`).
- Accessibility: `axe-core` in a browser test.
- HTML snapshots: a snapshot takes every change as the new truth.
- HTML snapshots: see [Compared with](#compared-with).

## Development

```sh
composer test # Pint and Rector as a dry run, PHPStan at max, type coverage 100 %, the test suites
composer lint # Rector, then Pint
```

Bug reports and pull requests are welcome on
[GitHub](https://github.com/sorge-it/phpunit-pest-html-assertions/issues).
A problem comes as a pull request that fixes it, not as an issue. How to run the checks of the
package: [CONTRIBUTING.md](CONTRIBUTING.md).

## Security

Please report a security problem by email to github@sorge-it.de, not in a public issue.
Report a vulnerability privately through GitHub. See [SECURITY.md](SECURITY.md).

## License

Expand Down
14 changes: 14 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Security policy

## Supported versions

The latest minor release of 1.x. A fix is released there only.

## Reporting a vulnerability

Do not report a vulnerability in a public pull request. Report it privately through GitHub:
[Report a vulnerability](https://github.com/sorge-it/phpunit-pest-html-assertions/security/advisories/new).

If you have a fix, add it in the private fork of the advisory. There is no fixed response time.

The package runs in test suites (`require-dev`), not in production code.
Loading
Loading