Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 0 additions & 1 deletion .changeset/bump-deps-2026-11-14.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
---
"e2e": patch
"@e2e-dev/web": patch
"@e2e-dev/smol": patch
---

Expand Down
6 changes: 6 additions & 0 deletions .changeset/exact-operation-timeout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"e2e": patch
"@e2e-dev/core": patch
---

A step's own `timeout` reaches the engine exactly, so an error that names it, such as a download wait's, no longer reads a millisecond short.
5 changes: 5 additions & 0 deletions .changeset/init-web-hint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"e2e": patch
---

`e2e init` labels the web engine `Chromium, Firefox` instead of `Playwright`.
6 changes: 6 additions & 0 deletions .changeset/resolve-label-hook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"e2e": patch
"@e2e-dev/core": patch
---

`resolveExpression` takes `matches`, `hasText`, and `order` options, so an engine whose platform reads more than the tree carries (a browser's text, hidden, and state rules, and its match order) decides queries, text filters, and the order of matches by its own rules.
6 changes: 6 additions & 0 deletions .changeset/scroll-until-load-pause.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"e2e": patch
"@e2e-dev/core": patch
---

An agent scrolling to a text no longer gives up on a list that pauses while it loads its next page: the list counts as ended only once the screen stays unchanged for a second as well as three pages.
5 changes: 5 additions & 0 deletions .changeset/smol-prepare-browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@e2e-dev/smol": patch
---

`prepare` gets a `browser` and `screen` on the warm browser's open tab as its second argument, so a sign-in reads like a test: `prepare: async (_cdpEndpoint, { browser, screen }) => { ... }`. smol detaches them once `prepare` resolves. The DevTools endpoint stays the first argument. Those calls may take 120 seconds in all; `prepareTimeout` raises it. Requires `@e2e-dev/web` 0.17.0 or newer.
6 changes: 6 additions & 0 deletions .changeset/vision-only-wait-pixels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"e2e": patch
"@e2e-dev/core": patch
---

`agent.waitFor` with `vision: 'only'` compares only the screenshot between judgments, the one thing the judge is shown, so a screen read that fell back to pixels alone no longer costs another model call.
5 changes: 5 additions & 0 deletions .changeset/web-attach-browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@e2e-dev/web": patch
---

Add `attachBrowser(cdpEndpoint, options?)`: `browser` and `screen` on a running Chromium's open tab in its default context, outside a test. `close()` detaches and leaves the browser, the tab, and its cookies and storage as they were.
18 changes: 18 additions & 0 deletions .changeset/web-bidi-only.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
"@e2e-dev/web": minor
---

**Breaking:** the web engine drives Chromium and Firefox over WebDriver BiDi and no longer uses Playwright. Tests, fixtures, and `web()` options stay the same, except:

- WebKit is gone: `web({ browser: 'webkit' })` is `INVALID_CONFIG`.
- `surfaceOf` and `PlaywrightLiveSurface` are removed, so there is no raw Playwright `Page` or `BrowserContext` to reach for.
- `browser.locator` takes CSS and XPath, alone or in `>>` chains: no `text=`, `nth=`, or other Playwright engines. Chain `nth()` and screen queries instead.
- `connect.reconnectEndpoint` and `BrowserLease.reconnectEndpoint` are removed, and `reconnectEndpoint` in `connect` is `INVALID_CONFIG`: a dropped connection fails the attempt, and the next attempt attaches again through `cdpEndpoint`.
- With a `scope: 'attempt'` browser provider, `app.clearState()` and saved sessions throw `UNSUPPORTED_CAPABILITY`: the attempt runs in the fresh browser's own context.
- `setInputFiles` on a browser provider's browser sends the files' bytes to the page, up to 50 MB in all; a directory there throws `UNSUPPORTED_CAPABILITY`, and so does a download on a provider without `downloads`.
- On Firefox, `browser.emulateMedia({ reducedMotion })` (set `web({ reducedMotion })` instead), `route.continue({ url })`, and a drag that starts an HTML5 drag throw `UNSUPPORTED_CAPABILITY`.
- Browsers are new pinned builds: Chrome for Testing 157.0.8103.38 (the headless shell, or full Chrome when headed) and Firefox 159.0.1. They install into `E2E_BROWSERS_PATH`, else `e2e/browsers` in the user cache, with `npx @e2e-dev/web install [chromium|firefox] [--with-deps] [--headed]`, or on the first run. `PLAYWRIGHT_BROWSERS_PATH` is no longer read.
- Chromium video needs an ffmpeg with VP9: `E2E_FFMPEG_PATH`, else `ffmpeg` on the `PATH`, else a pinned build the first recording downloads into the browser cache (`npx @e2e-dev/web install ffmpeg` fetches it ahead). A run that records no video never downloads it. Windows on Arm has no pinned build: set `E2E_FFMPEG_PATH` there.
- The old `ms-playwright` browser cache is no longer used and can be deleted.
- Every browser and ffmpeg archive is checked against a pinned SHA-256 before it is unpacked. `E2E_BROWSERS_DOWNLOAD_HOST` names an `https` mirror (plain `http` only on `localhost`) serving the Chrome for Testing, Mozilla archive, and ffmpeg-static paths. Downloads honor `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` (on Node 22, with `NODE_USE_ENV_PROXY=1`).
- Concurrent installs into one cache wait for each other and say so; one whose owner died or stopped responding is taken over. Ctrl-C or SIGTERM during `e2e-web install` stops the download, removes the partial files, and exits 130 or 143. On Linux, `--with-deps` also installs `xz-utils`, which unpacking Firefox needs.
10 changes: 5 additions & 5 deletions .dev/skills/babysit/references/review-triage.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ Reply with the one fact that disproves it, then resolve. Common shapes:

- **Pre-existing.** `git blame` or `git log -L` shows the flagged lines
predate the branch. A real, cheap issue gets a follow-up, not scope creep.
- **Misread library semantics.** Cite the library source or docs (Playwright,
the AI SDK, agent-device, node).
- **Misread library semantics.** Cite the library source or docs (the
WebDriver BiDi spec, chromium-bidi, the AI SDK, agent-device, node).
- **Already guaranteed.** Name the shared helper, type, closed schema, or
`e2e/engine` contract that makes it impossible.
- **Intentional tradeoff.** Link the PR body line, the commit body, or the
Expand Down Expand Up @@ -83,11 +83,11 @@ before any reviewer does.
symlinks on Windows.
12. A new build input or benchmark dependency missing from
`.github/filters.yml`, so the benchmark suites skip a change they cover.
13. Core naming an engine noun (`Web`, `browser`, `page`, `route`,
`playwright`) outside the `init` presets.
13. Core naming an engine noun (`Web`, `browser`, `page`, `route`, `bidi`)
outside the `init` presets.
14. A scripted rename of our API spelling (a fixture name, a call shape)
that also rewrote another library's code: a Playwright `page.getByRole`
in a migration sample, a raw `Page` from `surfaceOf`. Grep the diff for
in a migration sample. Grep the diff for
the old spelling on receivers that are not ours.
15. Negative-only security tests: dropping a result or reason makes absence
and size checks pass. Assert the expected redacted value exists before
Expand Down
2 changes: 1 addition & 1 deletion .dev/skills/verify/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ repeat.

```bash
pnpm install --frozen-lockfile
pnpm --filter @e2e-dev/web exec playwright-core install chromium
pnpm build # every consumer below runs dist; rebuild after each change
node packages/web/dist/bin.js install chromium firefox
```

A stale `dist` is the most common false result. Rebuild before trusting any
Expand Down
2 changes: 1 addition & 1 deletion .dev/skills/writing-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ diagrams and code carry the detail.
## Title

- Conventional Commits, matching `git log`:
`fix(playwright): provision the browser in a runner-side prepare hook`.
`fix(web): provision the browser in a runner-side prepare hook`.
PRs squash-merge with the number appended, so the title is the commit.
- Mark spec or API breaks with `!` (`feat(config)!: ...`), same as the commit.
- Name the outcome, not the activity. "install builds with the appPath
Expand Down
31 changes: 23 additions & 8 deletions .github/actions/chromium/action.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
name: chromium
description: >-
Installs Playwright's Chromium and its system packages, with the .deb files
apt downloads for them restored from and saved to the Actions cache.
Installs the Chrome for Testing build @e2e-dev/web pins and its system
packages, with the .deb files apt downloads for them restored from and
saved to the Actions cache.

# `--with-deps` installs about 38 MB of packages from the Ubuntu mirror, and
# the mirror often serves them at about 200 kB/s: the step takes 2 to 5
Expand All @@ -10,14 +11,14 @@ description: >-
# the packages whose version changed since the cache was saved.
inputs:
only-shell:
description: Install only the headless shell, not the full browser.
description: Install only the headless shell, not also the full browser a headed run launches.
default: 'true'

runs:
using: composite
steps:
# The prefix holds what decides the package list: the Ubuntu release and
# the Playwright version. The suffix is a digest of the .deb file names,
# the Chrome build. The suffix is a digest of the .deb file names,
# so a key names one exact set of package versions.
# The directory sits under /var/cache, not the runner's temp directory:
# apt downloads as `_apt`, which cannot traverse /home/runner and would
Expand All @@ -32,13 +33,14 @@ runs:
run: |
set -euo pipefail
. /etc/os-release
PLAYWRIGHT="$(node -p "require('./packages/web/package.json').dependencies['playwright-core']")"
CHROME="$(sed -n "s/^export const CHROMIUM_BUILD = '\(.*\)';$/\1/p" packages/web/src/install/provision.ts)"
[ -n "$CHROME" ] || { echo "::error::CHROMIUM_BUILD not found in packages/web/src/install/provision.ts"; exit 1; }
sudo install -d -o "$(id -u)" -g "$(id -g)" -m 755 "$ARCHIVES"
sudo install -d -o _apt -g root -m 700 "$ARCHIVES/partial"
printf 'Dir::Cache::Archives "%s/";\nAPT::Keep-Downloaded-Packages "true";\n' "$ARCHIVES" \
| sudo tee /etc/apt/apt.conf.d/99chromium-archives > /dev/null
echo "archives=$ARCHIVES" >> "$GITHUB_OUTPUT"
echo "prefix=chromium-apt-$ID-$VERSION_ID-$(uname -m)-playwright-$PLAYWRIGHT" >> "$GITHUB_OUTPUT"
echo "prefix=chromium-apt-$ID-$VERSION_ID-$(uname -m)-chrome-$CHROME" >> "$GITHUB_OUTPUT"
# No entry has exactly this key: restore-keys picks the newest entry with
# the prefix.
- name: Restore the archives
Expand All @@ -48,11 +50,24 @@ runs:
path: ${{ steps.apt.outputs.archives }}/*.deb
key: ${{ steps.apt.outputs.prefix }}
restore-keys: ${{ steps.apt.outputs.prefix }}-
# The install command is the engine's own CLI, so it needs the built
# package. A job that builds first skips this step. `e2e` and the engine
# are each other's dev dependencies, so a filtered build would order them
# arbitrarily, and the engine needs `e2e`'s types.
- name: Build the web engine
shell: bash
run: |
set -euo pipefail
[ -f packages/web/dist/bin.js ] || { pnpm --filter @e2e-dev/core run build && pnpm --filter e2e run build && pnpm --filter @e2e-dev/web run build; }
# The install runs apt-get through sudo itself.
- name: Install Chromium
shell: bash
env:
ONLY_SHELL: ${{ inputs.only-shell == 'true' && '--only-shell' || '' }}
run: pnpm --filter @e2e-dev/web exec playwright-core install chromium --with-deps $ONLY_SHELL
ONLY_SHELL: ${{ inputs.only-shell }}
run: |
set -euo pipefail
node packages/web/dist/bin.js install chromium --with-deps
[ "$ONLY_SHELL" = 'true' ] || node packages/web/dist/bin.js install chromium --with-deps --headed
# autoclean deletes the archives the mirror no longer serves, so the set
# changes only when a package version does.
- name: Name the archive set
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,10 @@ jobs:
- uses: ./.github/actions/workspace-install
with:
agent-device-version: ${{ inputs.agent-device-version }}
- name: Install browsers
uses: ./.github/actions/chromium
- name: Build
run: pnpm run build
- name: Install browsers
uses: ./.github/actions/chromium
- name: Run ${{ matrix.script }}
continue-on-error: ${{ matrix.optional }}
env:
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ jobs:
- uses: ./.github/actions/workspace-install
with:
agent-device-version: ${{ inputs.agent-device-version }}
- name: Install browsers
uses: ./.github/actions/chromium
- name: Build
run: pnpm run build
- name: Install browsers
uses: ./.github/actions/chromium
- name: Run test
env:
# The reporter posts the comment with the job's token.
Expand Down Expand Up @@ -144,10 +144,10 @@ jobs:
- uses: ./.github/actions/workspace-install
with:
agent-device-version: ${{ inputs.agent-device-version }}
- name: Install browsers
uses: ./.github/actions/chromium
- name: Build
run: pnpm run build
- name: Install browsers
uses: ./.github/actions/chromium
- name: Run test:agent
env:
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,11 +124,9 @@ jobs:
with:
working-directory: ${{ matrix.tests }}
agent-device-version: ${{ inputs.agent-device-version }}
# The runner downloads its own Chromium on first use; the system
# libraries it links against need root.
- name: Chromium system dependencies
- name: Chromium and its system dependencies
working-directory: ${{ matrix.tests }}
run: npx playwright-core install-deps chromium
run: npx --no-install e2e-web install chromium --with-deps
- name: Run the tests
id: test
working-directory: ${{ matrix.tests }}
Expand Down
63 changes: 62 additions & 1 deletion .github/workflows/spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ jobs:
uses: ./.github/actions/chromium
with:
only-shell: false
# @e2e-dev/web's integration suite, locator parity included, runs on Firefox too.
- name: Install Firefox
run: node packages/web/dist/bin.js install firefox --with-deps
- name: Unit and integration tests
run: pnpm test

Expand Down Expand Up @@ -176,11 +179,69 @@ jobs:
AGENT_DEVICE_VERSION: ${{ inputs.agent-device-version }}
run: node scripts/install-smoke.ts ${{ matrix.pnpm }}

# The testbed's own tests on Firefox. The tests ported from Playwright
# (tag playwright) assert Chromium's errors and run on Chromium in `check`.
firefox:
name: testbed (firefox)
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: pnpm/action-setup@9fd676a19091d4595eefd76e4bd31c97133911f1 # v4.2.0
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 26
cache: pnpm
- uses: ./.github/actions/workspace-install
with:
agent-device-version: ${{ inputs.agent-device-version }}
- name: Build
run: pnpm run build
- name: Install Firefox
run: node packages/web/dist/bin.js install firefox --with-deps
- name: Testbed on Firefox
working-directory: apps/testbed
run: node node_modules/e2e/dist/cli/bin.js run --config e2e.firefox.config.ts --exclude-tag playwright
- name: Upload report
if: ${{ !cancelled() }}
continue-on-error: true
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: testbed-firefox
path: |
apps/testbed/.e2e/report.json
apps/testbed/.e2e/results
if-no-files-found: warn
retention-days: 7

# The web engine on Windows: its own installer, the watchdog that ends the
# browser's process tree, and a launch and attempt on each browser.
windows:
name: web (windows)
runs-on: windows-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: pnpm/action-setup@9fd676a19091d4595eefd76e4bd31c97133911f1 # v4.2.0
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 26
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm --filter @e2e-dev/core run build && pnpm --filter e2e run build && pnpm --filter @e2e-dev/web run build
- name: Install Chromium and Firefox
run: node packages/web/dist/bin.js install chromium firefox
# A Firefox launch takes 8 to 17 s on the Windows runner, so a test that
# launches twice needs more than the suite's 30 s.
- name: Launch, attempt, and close on each browser
working-directory: packages/web
run: npx vitest run --testTimeout=90000 --hookTimeout=90000 tests/integration/lifecycle.test.ts tests/integration/keyboard.test.ts tests/integration/tabs.test.ts

# The one check the ruleset requires from this workflow. A skipped job
# passes (the change cannot reach it); a failed or cancelled one fails it.
gate:
name: spec gate
needs: [check, test, bun, install]
needs: [check, test, bun, install, firefox, windows]
if: ${{ always() }}
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 5
Expand Down
13 changes: 8 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,14 +158,17 @@ suites that consume the built packages the way a user would.
(peer). Each worker slot boots one warm browser machine from the pinned
Playwright Ubuntu image; a Node TCP relay exposes DevTools, and SDK agent
readiness allows machine startup without waiting for published ports. In
`attempt` scope every attempt runs in a copy-on-write branch. With `app`,
`attempt` scope (the provider's default, which `@e2e-dev/web` rejects
with `INVALID_CONFIG` for now) every attempt runs in a copy-on-write
branch. With `app`,
the app under test runs in the same machine, so a branch also copies its
running app and data on the machine. When maintaining this integration,
update the provider's SDK dev dependency to a published version and raise
its peer minimum only if the provider needs new SDK behavior. Keep the
Playwright image and Chromium version in sync with `@e2e-dev/web`. Run
image's Chromium close to the build `@e2e-dev/web` pins (`CHROMIUM_BUILD`
in `packages/web/src/install/provision.ts`). Run
`pnpm --filter @e2e-dev/smol test:unit` and `pnpm check`, then verify live
machines in both attempt and worker scopes with an app running in the VM.
machines in worker scope with an app running in the VM.
- `packages/limrun` — the Limrun hosted mobile device provider. Each worker
leases one remote device, installs its app, and releases the device at exit.
- `packages/decision` — the published `@e2e-dev/decision` package: a
Expand Down Expand Up @@ -385,8 +388,8 @@ pnpm --filter @e2e-dev/testbed run test:headed
- `pnpm typecheck` runs `build` first, then per-package `typecheck`. The
package `typecheck` covers `tests/**`, which is what makes
`tests/types/sdk-types.ts` a test.
- Integration tests need Chromium: `pnpm --filter @e2e-dev/web exec
playwright-core install chromium`. The web engine's `prepare` hook
- Integration tests need Chromium: `node packages/web/dist/bin.js install
chromium` after `pnpm build`. The web engine's `prepare` hook
also installs a missing browser once per run, in the runner, before `plan`
is emitted and the run's clock starts.

Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,10 @@ pnpm check # lint -> check:dead-code -> typecheck -> docs:check
pnpm test # unit + integration
```

Integration tests need Chromium:
Integration tests need Chromium. After `pnpm build`:

```sh
pnpm --filter @e2e-dev/web exec playwright-core install chromium
node packages/web/dist/bin.js install chromium
```

Docs are part of the change, not a follow-up. A behavior change updates its
Expand Down
Loading
Loading