diff --git a/.changeset/e2e-core.md b/.changeset/e2e-core.md new file mode 100644 index 0000000000..6ffbd96fd0 --- /dev/null +++ b/.changeset/e2e-core.md @@ -0,0 +1,14 @@ +--- +'e2e': patch +'@e2e-dev/core': patch +'@e2e-dev/web': patch +'@e2e-dev/mobile': patch +--- + +New package `@e2e-dev/core`: drive a browser or a native app from plain code, with no test runner, config file, or report. `launch(web())` prepares an engine and returns a host; `host.newSession()` opens an isolated session holding `screen`, `app`, `platform`, and the engine's fixture. `session.saveState()` captures state in memory, and `SessionState.export({ key })` returns it only AES-256-GCM encrypted, as `session-state-1` bytes. Secrets go to `launch({ secrets })` and fill through `host.secret(name)`. `createAgent(session, { model, cache })` from `@e2e-dev/core/agent` adds the agent: with `cache`, every `act` names its recording with `cacheKey`. + +`@e2e-dev/web` and `@e2e-dev/mobile` now need `e2e` only for their main entry, which adds the engine's fixture to `test`, so their `e2e` peer is optional. `@e2e-dev/core/agent` also exports what `e2e/agent` does, `defineTool` among it. + +Device targets take `app.linkBase`: `app.open('/orders/42')` then opens the path as a link in the pinned app, through the new optional engine hook `session.openLink` (protocol method `session/openLink`). `@e2e-dev/mobile` implements it. + +`@e2e-dev/web/engine` and `@e2e-dev/mobile/engine` export the engine factories and their types without `test`, so a program that only drives an engine never loads the test runner. `defineEngine()` and `EngineHandle` now carry the fixtures an engine contributes as a type, and `defineEngine` requires a factory for each one `Fixtures` names: `web()` is an `EngineHandle`, `mobile()` an `EngineHandle`. `defineEngine` refuses a fixture named `close`, `saveState`, or `setDefaultTimeout`, the methods of a session. `@e2e-dev/core/engine` (and `e2e/engine`) exports `isRefusedLinkScheme`, the scheme list app links are refused by, which `@e2e-dev/mobile` now uses for `device.openLink`. diff --git a/AGENTS.md b/AGENTS.md index e1d50d8519..fea4af43a7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,7 +9,8 @@ There is no separate spec. The code is the contract, pinned in three places: - The emitted `packages/e2e/dist/index.d.ts` (and `dist/engine/index.d.ts`, `dist/runner.d.ts`, `dist/oauth/*.d.ts`), and `@e2e-dev/core`'s - `packages/core/dist/engine/index.d.ts`, are the public API. `packages/e2e/tests/types/sdk-types.ts` holds compile-time + `packages/core/dist/core/*.d.ts` and `dist/engine/index.d.ts`, + are the public API. `packages/e2e/tests/types/sdk-types.ts` holds compile-time assertions (`@ts-expect-error` lines) for the parts that are easy to loosen by accident; it runs under the package `typecheck`, never under vitest. - Wire formats live in `packages/e2e/schema/*.schema.json` (report-1, @@ -42,10 +43,15 @@ suites that consume the built packages the way a user would. the replay cache (`src/cache/`), secrets, and the session half of a worker: `run/engine-sessions.ts` (engine init and dispose, each attempt's session from launch to close, which the runner's `TargetExecutor` - extends), `run/standalone.ts` (the worker a runner host or MCP keeps - alive, with a `WorkerPolicy` per kind), and `run/standalone-attempt.ts` - (one attempt: reservation, launch, fixtures, deadline, close). Its one - public entry is `@e2e-dev/core/engine`. It imports nothing of `e2e`: + extends), `run/standalone.ts` (the worker an `@e2e-dev/core` host, a runner + host, or MCP keeps alive, with a `WorkerPolicy` per kind), and + `run/standalone-attempt.ts` (one attempt: reservation, launch, fixtures, + deadline, close). `src/core/` is its own surface: `launch` opens a + `core` attempt (nothing published to `globalThis`, no process secret + registry, no trace, no session deadline), `SessionState` the + `session-state-1` export, and `createAgent` (`@e2e-dev/core/agent`) the agent + with a per-`cacheKey` replay cache settled at session close. It imports + nothing of `e2e`: `tests/unit/package-imports.test.ts` in `packages/e2e` fails on a dependency `@e2e-dev/core`'s manifest does not declare. `e2e` reaches its modules as `@e2e-dev/core/lib/` (`./lib/*` in its exports), which is @@ -93,8 +99,11 @@ suites that consume the built packages the way a user would. checks that need a stored login and are never part of `pnpm test`. - `packages/web` — the published `@e2e-dev/web` package: the browser engine, built with the public `defineEngine`, contributing the - `browser` fixture and `expect(browser)`. It depends on `@e2e-dev/core` and - `e2e` (peers), never the reverse; a target names it explicitly as + `browser` fixture and `expect(browser)`. `@e2e-dev/web/engine` (and + `@e2e-dev/mobile/engine`) export the factory and types without `test`, so + `@e2e-dev/core` programs never import `e2e`'s main entry. It depends on + `@e2e-dev/core` (peer), never the reverse, and on `e2e` as an optional peer + that only the main entry loads; a target names it explicitly as `engine: web()`. There is no default engine and no well-known id registry in core. It imports from `@e2e-dev/core/engine` only: the semantics every engine must reproduce (error taxonomy, text and URL matching, assertion @@ -103,7 +112,7 @@ suites that consume the built packages the way a user would. - `packages/mobile` — the published `@e2e-dev/mobile` package: the iOS/Android engine on agent-device, built with the same public `defineEngine`, contributing the `device` fixture. Like the web engine it - depends on `@e2e-dev/core` and `e2e` (peers), never the reverse; the + depends on `@e2e-dev/core` (peer) and optionally `e2e`, never the reverse; the engine implementation imports from `@e2e-dev/core/engine` only, and a target names it as `engine: mobile()`. The `@e2e-dev/mobile/tools` subpath holds the agent-side `open_app`, `swipe`, @@ -367,7 +376,7 @@ pnpm --filter @e2e-dev/testbed run test:headed - Package `test` scripts do **not** build. Root `build` and `test` order the packages explicitly rather than relying on topological sort, because `e2e` devDepends on the web engine for its browser-backed integration tests - while the engine peer-depends on `e2e` — pnpm reports that cycle + while the engine optionally peer-depends on `e2e` — pnpm reports that cycle on every install. `@e2e-dev/core` builds first: `e2e` and every engine compile against its `dist`. - `pnpm typecheck` runs `build` first, then per-package `typecheck`. The @@ -713,7 +722,7 @@ trees, on both platforms, without a device. entry. On `0.x` the bump is `patch` for a feature or fix; never pick `minor` on your own. A breaking change is a `minor` with the summary opening `**Breaking:**`; a `minor` or `major` for a release the maintainer - calls opens `**Release:**` (`scripts/check-changeset-bumps.ts`, `pnpm check`). Peer ranges point one way only (engine -> `@e2e-dev/core` and `e2e`, integration -> + calls opens `**Release:**` (`scripts/check-changeset-bumps.ts`, `pnpm check`). Peer ranges point one way only (engine -> `@e2e-dev/core` and optionally `e2e`, integration -> engine) and read `>= `, the floor the oldest sibling the package works with. Raise it when a package starts relying on a newer sibling's export or contract field; diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 045e703bd8..1676e087fe 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -289,9 +289,10 @@ committed. The following are the public contract. A change that breaks any of them is a breaking change and follows the rules below: -- Public types exported from `e2e`, `e2e/engine`, and `@e2e-dev/core/engine`, - as emitted in `dist/index.d.ts` and `dist/engine/index.d.ts` of `e2e`, - and `dist/engine/index.d.ts` of `@e2e-dev/core`. Removing an +- Public types exported from `e2e`, `e2e/engine`, `@e2e-dev/core`, + `@e2e-dev/core/agent`, and `@e2e-dev/core/engine`, as emitted in `dist/index.d.ts` + and `dist/engine/index.d.ts` of `e2e`, and `dist/core/*.d.ts` and + `dist/engine/index.d.ts` of `@e2e-dev/core`. Removing an export, narrowing an accepted input, widening a returned type, or changing a fixture's runtime behavior all count. - Config keys in `e2e.config.ts`, their types, and their defaults. @@ -335,8 +336,8 @@ together in one release, with a changeset for each, so an engine and a runner from the same release always match. `e2e` and `@e2e-dev/core` are a changesets `fixed` group: they always share a version, and `e2e` depends on that exact `@e2e-dev/core`. Engines declare a peer range on `@e2e-dev/core` that points one way -only (engine to core, `>=x <1`), plus one on `e2e` for the main entry that -adds the engine's fixture to `test`, and each integration (`@e2e-dev/kernel`, `@e2e-dev/lightpanda`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`, `@e2e-dev/browserstack`, `@e2e-dev/e2b`) does +only (engine to core, `>=x <1`), plus an optional one on `e2e` for the main +entry that adds the engine's fixture to `test`, and each integration (`@e2e-dev/kernel`, `@e2e-dev/lightpanda`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`, `@e2e-dev/browserstack`, `@e2e-dev/e2b`) does the same on the engines it plugs into; do not make it mutual or narrow it. ### Driver nightly compatibility diff --git a/docs/core.mdx b/docs/core.mdx new file mode 100644 index 0000000000..42a98c2574 --- /dev/null +++ b/docs/core.mdx @@ -0,0 +1,280 @@ +--- +title: "@e2e-dev/core" +description: Drive a browser or a native app from plain code, with an optional agent and no test runner. +--- + +`@e2e-dev/core` drives a browser or an iOS or Android app from plain code. It +uses the e2e engines, `screen`, and the agent, without the test runner: there +is no config file, no report, and no `expect`. Use it for scripts, +automation, scrapers, and bots. + +## Install + + + +```bash npm +npm install @e2e-dev/core @e2e-dev/web +``` + +```bash pnpm +pnpm add @e2e-dev/core @e2e-dev/web +``` + +```bash bun +bun add @e2e-dev/core @e2e-dev/web +``` + + + +Install `@e2e-dev/mobile` instead of, or beside, `@e2e-dev/web` to drive a +device. `@e2e-dev/core` needs no `e2e`, the test runner, which is built on it. + + +Import an engine from its `/engine` entry, as the examples do. The package's +main entry is for tests: it loads `e2e`. + + +## Drive a browser + +`launch` prepares an engine and returns a host. A session is one isolated +browser context on that host, and it holds `screen`, `app`, `platform`, and +the engine's own fixture, here `browser`. + +```ts title="prices.ts" +import { launch } from '@e2e-dev/core'; +import { web } from '@e2e-dev/web/engine'; + +await using shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); + +for (const sku of ['A-1001', 'B-2040']) { + await using session = await shop.newSession(); + await session.app.open(`/products/${sku}`); + if (await session.screen.getByText('Sold out', { exact: false, visible: true }).isVisible()) continue; + const price = await session.screen.getByTestId('sale-price').textContent(); + console.log(sku, price); +} +``` + +`screen` and `app` are the same API as in a test: see +[screen](/reference/screen) and [app](/reference/app). `await using` closes +each session at the end of its block, and the host, with its browser, at the +end of the file. It needs Node.js 24 or Bun. On Node.js 22, call `close()` in +a `finally`: + +```ts +const shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); +try { + const session = await shop.newSession(); + await session.app.open('/products/A-1001'); +} finally { + await shop.close(); // closes the open session too +} +``` + +| Rule | Behavior | +| --- | --- | +| Ownership | Each `launch` takes its own engine instance: `launch(web())`. A second host on the same instance fails with `WORKER_PROTOCOL`. | +| Sessions | One session at a time on a host. Close it before `newSession()`. For parallel work, launch more hosts. | +| Timeouts | An action, a wait, or a read gets 30000 ms. `app.open()` and `app.restart()` get 120000 ms. `launch({ timeout })` or `session.setDefaultTimeout(ms)` sets one value for all of them. There is no session deadline. | +| Cancellation | `launch({ signal })` and `newSession({ signal })` stop what is running. | +| Ctrl-C | The host closes on SIGINT, SIGTERM, and SIGHUP, so its browser or device session does not stay behind. Then the program ends by the signal, unless it listens for the signal itself. Turn it off with `handleSignals: false`. | +| Files | Screenshots and downloads go to a temp directory that is deleted when the host closes. Set `artifactsDir` to keep them. | + +The [reference](/reference/core) lists every option. + +## Drive an app on a device + +Pass `mobile()` and the app. The same `screen` code runs on a browser and +on a device. + +```ts +import { launch } from '@e2e-dev/core'; +import { mobile } from '@e2e-dev/mobile/engine'; + +await using shopApp = await launch(mobile({ platform: 'ios' }), { + app: { bundleId: 'com.example.shop', linkBase: 'https://shop.example.com' }, +}); +await using session = await shopApp.newSession(); +await session.app.open('/products/A-1001'); +``` + +`app.linkBase` makes `app.open(path)` open a link in the app: +`https://shop.example.com/products/A-1001` as a universal link here, or +`shopapp://products/A-1001` with `linkBase: 'shopapp://'`. Without it, +`app.open()` launches the app and takes no path. `linkBase` also works in an +`e2e.config.ts` target; see [config](/reference/config#linkbase). + +The iOS simulator can stop a custom-scheme link at an "Open in ...?" prompt. +Tap its `Open` button when it shows: +`session.screen.getByRole('button', 'Open')`. + +## Keep a signed-in state + +`session.saveState()` captures a browser session's cookies and storage in +memory. Close that session, then start another from the state with +`newSession({ state })`. The mobile engine has no state to capture, so there +it fails with `UNSUPPORTED_CAPABILITY`. + +```ts +const state = await session.saveState(); +await session.close(); +await using next = await shop.newSession({ state }); +``` + +To keep a state between program runs, encrypt it with a 32-byte key that you +keep, for example in a secret store. There is no plaintext export. + +```ts +import { randomBytes } from 'node:crypto'; +import { SessionState } from '@e2e-dev/core'; + +const key = randomBytes(32); +const bytes = await state.export({ key }); // write these to disk +const restored = await SessionState.import(bytes, { key }); +``` + +A state restores only on a host of the same engine, platform, and app. Another +is `SESSION_MISMATCH`. Two addresses of one app, such as `localhost` and +`127.0.0.1`, are two apps unless both hosts set the same `app.identity`. A +wrong key or a changed byte is `SESSION_INVALID`. + +## Pass secrets + +Give secret values to `launch`, then fill them by name. + +```ts +await using shop = await launch(web(), { + app: { url: 'https://shop.example.com' }, + secrets: { SHOP_PASSWORD: process.env.SHOP_PASSWORD! }, +}); +await session.screen.getByLabel('Password').fill(shop.secret('SHOP_PASSWORD')); +``` + +The values never appear in the errors a session throws, in `screen.tree()`, +or in what the agent reads. A read your code asks for, such as +`textContent()`, returns the page's text as it is. An exported state holds +the values the session knew, encrypted, so a host launched without them still +redacts them. + +An agent gets a secret through its params, by name only: +`agent.act('Sign in with {password}', { params: { password: shop.secret('SHOP_PASSWORD') } })`. + +## Handle errors + +Every failure is an error with a `code`. Check it with `isE2EError`, not +`instanceof`. + +```ts +import { isE2EError } from '@e2e-dev/core'; + +try { + await session.screen.getByRole('button', 'Add to cart').tap(); +} catch (error) { + if (isE2EError(error) && error.code === 'LOCATOR_NOT_FOUND') console.log('not on sale'); + else throw error; +} +``` + +| Code | When | +| --- | --- | +| `LOCATOR_NOT_FOUND` | Nothing matched before the timeout. A wait that ran out has this code too. | +| `LOCATOR_AMBIGUOUS` | The locator matched more than one node. | +| `ACTION_FAILED` | The action did not finish. It may still have reached the app, so do not repeat a payment. | +| `APP_UNREACHABLE` | Nothing answered at the app's URL. | +| `CANCELLED` | A `signal` aborted the operation, or the session is closed. | +| `WORKER_PROTOCOL` | A second session on a host, a second host on one engine instance, or a closed host. | + +[Errors](/reference/errors) lists every code. + +## Let the agent act + +`createAgent` from `@e2e-dev/core/agent` puts an agent over one session. Its +`act`, `assert`, `waitFor`, and `extract` work as they do in a +[test](/reference/agent). The agent needs the AI SDK, and the example reads a +typed result with `zod`: install `ai` and `zod` beside the engine. + +```ts title="checkout.ts" +import { launch } from '@e2e-dev/core'; +import { createAgent } from '@e2e-dev/core/agent'; +import { web } from '@e2e-dev/web/engine'; +import { gateway } from 'ai'; +import { z } from 'zod'; + +await using shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); +await using session = await shop.newSession(); +await session.app.open('/products/A-1001'); + +const agent = createAgent(session, { + model: gateway('anthropic/claude-sonnet-5-5'), + cache: { dir: '.shop-bot/cache' }, +}); + +await agent.act('Add the item to the cart and apply the coupon with the largest discount', { + cacheKey: 'cart-best-coupon', +}); +// A check after the step lets the cache keep its recording. +await session.screen.getByRole('button', /pay now/i).waitFor(); + +const order = await agent.extract('the order total', { schema: z.object({ total: z.number() }) }); +if (order.total <= 39_000) { + try { + await session.screen.getByRole('button', /pay now/i).tap(); + } catch (error) { + // A failed tap may still have reached the server: check the order list, never retry the payment. + console.error('check /my/orders'); + throw error; + } +} else { + console.log(`the total, ${order.total}, is over the limit: the order stays unpaid`); +} +``` + +`createAgent` takes the options of an [`agents` entry](/reference/config#agents), +such as `model`, `judge`, `maxSteps`, and `context`, plus `cache`. +There is no default model: an agent without one fails its first call with +`MODEL_UNAVAILABLE`. An `act` step has its own timeout, 120000 ms: set it per +call with `timeout`. + +### Replay recorded steps + +With `cache`, `act` records each flow and replays it on the next run with no +model call. Every `act` then names its recording with `cacheKey`, or runs live +with `cache: false`. An `act` without one fails with `INVALID_ARGUMENT`, and +TypeScript refuses it when the `cache` option is written in the `createAgent` +call. + +A recording is keyed by the `cacheKey`, the instruction, its params, the app, +and how many times the session made the same call. Pass a value that changes +between runs as a param: a value the agent reads off the screen is decided by +the model again on each run. + +A recording is saved when the session closes, or its host does, and only when +a check passed after the step, such as `locator.waitFor()`, `agent.assert`, +or `agent.waitFor`. When an agent step fails, the recordings not yet confirmed +at that moment are dropped. A failure of your own code's locator call, such +as a `waitFor` that ran out, drops nothing. Read [Replay cache](/cache) for +how a replay checks the screen and hands off to the model. + +| Option | Default | Meaning | +| --- | --- | --- | +| `dir` | `.e2e/cache` | The directory, from the `launch` `cwd`. | +| `mode` | `read-write` | `read-only` replays and never writes. | +| `strict` | `false` | A recording that no longer replays fails with `REPLAY_STALE`. | +| `store` | - | A custom [`CacheStore`](/cache) in place of the files. | + +## Next + + + + Every option of `launch`, a session, a saved state, and `createAgent`. + + + Every query and action a session holds. + + + What `act`, `assert`, `waitFor`, and `extract` take and return. + + + Choose and configure the model the agent calls. + + diff --git a/docs/docs.json b/docs/docs.json index c9e8ea0a7d..6bb008a432 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -74,7 +74,8 @@ "pages": [ "web", "mobile", - "api-testing" + "api-testing", + "core" ] }, { @@ -179,7 +180,8 @@ "reference/expect", "reference/screen", "reference/app", - "reference/agent" + "reference/agent", + "reference/core" ] }, { diff --git a/docs/examples/core/checkout.ts b/docs/examples/core/checkout.ts new file mode 100644 index 0000000000..857e06894f --- /dev/null +++ b/docs/examples/core/checkout.ts @@ -0,0 +1,33 @@ +import { launch } from '@e2e-dev/core'; +import { createAgent } from '@e2e-dev/core/agent'; +import { web } from '@e2e-dev/web/engine'; +import { gateway } from 'ai'; +import { z } from 'zod'; + +await using shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); +await using session = await shop.newSession(); +await session.app.open('/products/A-1001'); + +const agent = createAgent(session, { + model: gateway('anthropic/claude-sonnet-5-5'), + cache: { dir: '.shop-bot/cache' }, +}); + +await agent.act('Add the item to the cart and apply the coupon with the largest discount', { + cacheKey: 'cart-best-coupon', +}); +// A check after the step lets the cache keep its recording. +await session.screen.getByRole('button', /pay now/i).waitFor(); + +const order = await agent.extract('the order total', { schema: z.object({ total: z.number() }) }); +if (order.total <= 39_000) { + try { + await session.screen.getByRole('button', /pay now/i).tap(); + } catch (error) { + // A failed tap may still have reached the server: check the order list, never retry the payment. + console.error('check /my/orders'); + throw error; + } +} else { + console.log(`the total, ${order.total}, is over the limit: the order stays unpaid`); +} diff --git a/docs/examples/core/prices.ts b/docs/examples/core/prices.ts new file mode 100644 index 0000000000..21bff06431 --- /dev/null +++ b/docs/examples/core/prices.ts @@ -0,0 +1,12 @@ +import { launch } from '@e2e-dev/core'; +import { web } from '@e2e-dev/web/engine'; + +await using shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); + +for (const sku of ['A-1001', 'B-2040']) { + await using session = await shop.newSession(); + await session.app.open(`/products/${sku}`); + if (await session.screen.getByText('Sold out', { exact: false, visible: true }).isVisible()) continue; + const price = await session.screen.getByTestId('sale-price').textContent(); + console.log(sku, price); +} diff --git a/docs/mobile.mdx b/docs/mobile.mdx index a087f59dd6..b29ff2aa00 100644 --- a/docs/mobile.mdx +++ b/docs/mobile.mdx @@ -779,7 +779,7 @@ export function cloudDevices(options: { provider: string; device: string; osVers | On the web | On a device | | --- | --- | -| `app.open('/path')` and the agent's `navigate` verb | No app URL and no `navigate`. `app.open()` launches `app.bundleId` fresh, and `device.openLink(url)` opens a link into it | +| `app.open('/path')` and the agent's `navigate` verb | No app URL and no `navigate`. `app.open()` launches `app.bundleId` fresh, and `device.openLink(url)` opens a link into it. With [`app.linkBase`](/reference/config#linkbase), `app.open('/path')` opens the path as a link | | `test.setup` saves a session, `session:` restores it | Not available. A simulator has no portable session snapshot; sign in per test or seed the app | | Secret input | `type_secret` and `screen.getByLabel(...).fill(secret)` fill declared secrets into secure fields | | `--video` writes a WebM screencast | `--video` writes an MP4 of the device screen with taps shown (`videoTouches: false` hides them), or keeps a device provider's own recording | diff --git a/docs/package.json b/docs/package.json index 8cfe2d7e9c..ceef7af1f7 100644 --- a/docs/package.json +++ b/docs/package.json @@ -11,6 +11,7 @@ "@ai-sdk/typesafe-ai": "3.0.40", "@e2e-dev/browser-run": "workspace:*", "@e2e-dev/browserstack": "workspace:*", + "@e2e-dev/core": "workspace:*", "@e2e-dev/decision": "workspace:*", "@e2e-dev/e2b": "workspace:*", "@e2e-dev/limrun": "workspace:*", diff --git a/docs/reference/app.mdx b/docs/reference/app.mdx index 182230d1b1..ef3fe31253 100644 --- a/docs/reference/app.mdx +++ b/docs/reference/app.mdx @@ -64,13 +64,16 @@ await app.open('/settings/billing'); // a path relative to app.url await app.open('https://example.com/login'); // any http(s) URL ``` -On a device target, which has no URL, `open()` launches the pinned app fresh -and takes no path. +On a device target, which has no URL, `open()` launches the pinned app fresh. +A path opens as a link in the app when the target declares +[`app.linkBase`](/reference/config#linkbase), and is `APP_URL_REQUIRED` +without it. With a link base, an absolute link opens as given, custom app +schemes included. | Throws | When | | --- | --- | -| `APP_URL_REQUIRED` | A path, on a target that declares no `app.url`. | -| `POLICY_DENIED` | A URL whose scheme is not `http:` or `https:`. `about:blank` is allowed. | +| `APP_URL_REQUIRED` | A path, on a target that declares no `app.url` (or, on a device, no `app.linkBase`). | +| `POLICY_DENIED` | A URL whose scheme is not `http:` or `https:`. `about:blank` is allowed. On a device with `app.linkBase`, a link whose scheme is `file:`, `data:`, `javascript:`, `view-source:`, `blob:`, or `filesystem:`. | | `APP_UNREACHABLE` | The app URL could not be reached. | | `ACTION_FAILED` | The operation failed or timed out. | | `UNSUPPORTED_CAPABILITY` | No `session.open` on a target with a URL, or no `session.restart` on one without. On a device with no `app.bundleId`, also the target's build not installed yet ([mobile](/reference/mobile#the-app-under-test)). | @@ -211,7 +214,8 @@ export interface App { /** * Opens the app: the declared URL, a path relative to it, or any absolute * http(s) URL. On a device target, which has no URL, it launches the pinned - * app fresh and takes no path. + * app fresh; a path opens as a link in it when the target declares + * `app.linkBase`. */ open(path?: string): Promise; /** diff --git a/docs/reference/config.mdx b/docs/reference/config.mdx index 141144797f..73bdfe65a5 100644 --- a/docs/reference/config.mdx +++ b/docs/reference/config.mdx @@ -353,6 +353,7 @@ targets: [ | [`url`](#url) | `string` | - | | [`bundleId`](#bundleid) | `string` | - | | [`appPath`](#apppath) | `string` | - | +| [`linkBase`](#linkbase) | `string` | - | | [`identity`](#identity) | `string` | derived from the app | | [`environment`](#environment) | `'test' \| 'staging' \| 'production'` | `'test'` or `'production'` | | [`launchArguments`](#launcharguments) | `string[]` | - | @@ -367,6 +368,7 @@ target. A target with `app` needs an [engine](#engine). `launchArguments`. - A `mobile()` target needs `app.bundleId` or `app.appPath`. `app.url` on a mobile target is not supported yet. +- `app.linkBase` needs an engine that opens links, such as `mobile()`. - An unknown key in `app` or `app.command` fails with a did-you-mean. @@ -420,6 +422,29 @@ appPath: './build/App.app', Without `bundleId`, the app it installs is the one launched. See [the device engine](/reference/mobile#engine) for when it is installed. +#### `linkBase` + +Where `app.open(path)` opens on a device target: the path joins the link base, +and the link opens in the pinned app. + +```ts +linkBase: 'https://shop.example.com', // app.open('/orders/42') opens https://shop.example.com/orders/42 +linkBase: 'shopapp://', // app.open('/orders/42') opens shopapp://orders/42 +``` + +- One slash joins the base and the path; none follows a bare scheme + (`shopapp://`). +- An absolute link passed to `app.open` opens as given, a custom app scheme + included; `file:`, `data:`, `javascript:`, `view-source:`, `blob:`, and + `filesystem:` links are `POLICY_DENIED`. +- Without `linkBase`, `app.open()` on a device launches the app and takes no + path (`APP_URL_REQUIRED`). + +| Throws | When | +| --- | --- | +| `INVALID_CONFIG` | The engine opens no links (a `web()` target), the value is not an absolute URL or a scheme, it has a query or fragment (an empty `?` or `#` included), it has leading or trailing whitespace, or it names `file:`, `data:`, `javascript:`, `view-source:`, `blob:`, or `filesystem:`. | +| `POLICY_DENIED` | `app.open` got an absolute link with one of those schemes. | + #### `identity` The stable identity keying replay cache and session entries. diff --git a/docs/reference/core.mdx b/docs/reference/core.mdx new file mode 100644 index 0000000000..a3bfb1f411 --- /dev/null +++ b/docs/reference/core.mdx @@ -0,0 +1,277 @@ +--- +title: "@e2e-dev/core" +description: launch, hosts, sessions, saved state, and createAgent, the API of a program that drives a browser or a device without the test runner. +--- + +`@e2e-dev/core` is the API of a program that drives a browser or a device +with no test runner around it. `launch` returns a host, a host opens +sessions, and `createAgent` puts an agent over a session. The +[guide](/core) shows each of them in use. + +```ts title="prices.ts" +import { launch } from '@e2e-dev/core'; +import { web } from '@e2e-dev/web/engine'; + +await using shop = await launch(web(), { app: { url: 'https://shop.example.com' } }); +await using session = await shop.newSession(); +await session.app.open('/products/A-1001'); +console.log(await session.screen.getByTestId('sale-price').textContent()); +``` + +## Reference + +### Entries + +| Import | From | +| --- | --- | +| [`launch`](#launch), [`SessionState`](#sessionstate), `isE2EError` | `@e2e-dev/core` | +| [`createAgent`](#createagent), `defineTool` | `@e2e-dev/core/agent` | +| `web` | `@e2e-dev/web/engine` | +| `mobile` | `@e2e-dev/mobile/engine` | +| `defineEngine` | `@e2e-dev/core/engine` | + +`@e2e-dev/core/lib/*` is private. Its modules can change in any release. + +### Launching + +| Function | Returns | +| --- | --- | +| [`launch`](#launch) | `Promise` | + +#### `launch` + +Prepares an engine and returns the host that opens sessions on it. The first +launch of `web()` downloads a browser. + +```ts +const shop = await launch(web(), { + app: { url: 'https://shop.example.com' }, + timeout: 10_000, + artifactsDir: 'artifacts', +}); +``` + +| Option | Type | Default | +| --- | --- | --- | +| [`app`](#app) | `LaunchApp` | - | +| `secrets` | `Record` | - | +| `headed` | `boolean` | `false` | +| [`timeout`](#timeout) | `number` | per operation | +| `signal` | `AbortSignal` | - | +| `handleSignals` | `boolean` | `true` | +| `artifactsDir` | `string` | a temp directory | +| `cwd` | `string` | `process.cwd()` | +| `env` | `NodeJS.ProcessEnv` | `process.env` | +| `onNotice` | `(message: string) => void` | - | + +- `secrets`: values by name, 6 characters or more each. See + [Pass secrets](/core#pass-secrets). +- `headed`: shows the browser window. A device ignores it. +- `signal`: cancels the launch, and every session and operation on the host. +- `handleSignals`: closes the host on SIGINT, SIGTERM, and SIGHUP, then lets + the signal end the program, unless the program listens for it itself. +- `artifactsDir`: where screenshots, downloads, and recordings go, from + `cwd`. The default directory is deleted when the host closes. +- `cwd`: what upload paths, `artifactsDir`, and an agent's cache directory + resolve from. +- `onNotice`: progress outside operations, such as a browser download or a + device lease. + +| Throws | When | +| --- | --- | +| `INVALID_ARGUMENT` | The first argument is not an engine, or an option or `app` key is unknown. | +| `INVALID_CONFIG` | The engine refuses the app: `web()` without `app.url`, `mobile()` without `app.bundleId` or `app.appPath`. | +| `WORKER_PROTOCOL` | Another host owns this engine instance. Call the factory again: `launch(web())`. | + + + +##### `app` + +What `app.open()` opens on a browser, or launches on a device. + +| Field | For | Meaning | +| --- | --- | --- | +| `url` | `web()` | The app's URL. `app.open(path)` resolves against it. | +| `bundleId` | `mobile()` | The installed app to launch. | +| `appPath` | `mobile()` | The app build the program runs against, a `.app` or an `.apk`. See [config](/reference/config#apppath). | +| `linkBase` | `mobile()` | What `app.open(path)` joins the path to and opens as a link. See [config](/reference/config#linkbase). | +| `identity` | both | The app's name for saved states and recordings. Default: the URL or the bundle id. | +| `launchArguments` | `mobile()` | Arguments the app launches with. | +| `permissions` | both | Permissions to grant or deny. | + + + +##### `timeout` + +The default timeout of one operation, in milliseconds. Without it: + +| Operation | Default | +| --- | --- | +| An action, a wait, or a read | 30000 | +| `app.open()`, `app.restart()`, `app.clearState()` | 120000 | +| An agent step | 120000, or the call's own `timeout` | + +`timeout` sets the first two rows. There is no session deadline. + +### Host + +| Member | Returns | +| --- | --- | +| `platform` | `string` | +| [`newSession`](#newsession) | `Promise` | +| [`secret`](#secret) | `Secret` | +| [`close`](#close) | `Promise` | + +`platform` is `web`, `ios`, `android`, or a custom engine's own label. + +#### `newSession` + +Opens a session: one isolated browser context, or the app on the device. A +host holds one session at a time. + +```ts +const session = await shop.newSession({ state, signal: AbortSignal.timeout(60_000) }); +``` + +| Option | Type | Default | +| --- | --- | --- | +| `state` | [`SessionState`](#sessionstate) | - | +| `signal` | `AbortSignal` | - | + +| Throws | When | +| --- | --- | +| `WORKER_PROTOCOL` | A session is open on the host, or the host is closed. | +| `SESSION_MISMATCH` | `state` was saved on another engine, platform, or app. | +| `SESSION_EXPIRED` | `state` is past its expiry. | + +#### `secret` + +Returns the handle of a secret passed to `launch`, for `fill` or an agent's +params. An unknown name throws `SECRET_UNAVAILABLE`. + +```ts +await session.screen.getByLabel('Password').fill(shop.secret('SHOP_PASSWORD')); +``` + +#### `close` + +Closes the open session, then the engine. A second call does nothing. A host +is also disposable: `await using` closes it. A cleanup that fails throws an +`AggregateError` of the failures. + +### Session + +| Member | Returns | +| --- | --- | +| `screen` | [`Screen`](/reference/screen) | +| `app` | [`App`](/reference/app) | +| `platform` | `string` | +| `browser` | [`Browser`](/reference/web), with `web()` | +| `device` | [`Device`](/reference/mobile), with `mobile()` | +| [`saveState`](#savestate) | `Promise` | +| [`setDefaultTimeout`](#setdefaulttimeout) | `void` | +| `close` | `Promise` | + +`close` ends the session, as `await using` does. After it, every operation +fails with `CANCELLED`. + +#### `saveState` + +Captures the session's cookies and storage in memory. An engine with no +state to capture, such as `mobile()`, throws `UNSUPPORTED_CAPABILITY`. + +```ts +const state = await session.saveState(); +``` + +#### `setDefaultTimeout` + +Sets the default timeout of the operations that start from now on, in +milliseconds: actions, waits, reads, and app opens and restarts. + +```ts +session.setDefaultTimeout(5_000); +``` + +### SessionState + +| Member | Returns | +| --- | --- | +| `engine` | `string` | +| `platform` | `string` | +| [`export`](#export) | `Promise` | +| [`SessionState.import`](#sessionstate-import) | `Promise` | + +#### `export` + +Encrypts the state with a 32-byte key (AES-256-GCM) into bytes you can +write to disk. There is no plaintext export. + +```ts +const bytes = await state.export({ key }); +``` + +The bytes name the engine, the platform, and the names of the secrets in the +clear. The cookies, the storage, and the secret values are encrypted. + +#### `SessionState.import` + +Decrypts bytes that `export` wrote. + +```ts +const state = await SessionState.import(bytes, { key }); +``` + +| Throws | When | +| --- | --- | +| `SESSION_INVALID` | Another key, a changed byte, or bytes that `export` did not write. | +| `SESSION_EXPIRED` | The state is past its expiry. | +| `INVALID_ARGUMENT` | The key is not 32 bytes. | + +### Agent + +| Function | Returns | +| --- | --- | +| [`createAgent`](#createagent) | an agent | + +#### `createAgent` + +Creates an agent over one session. `assert`, `waitFor`, and `extract` are +the [agent fixture's](/reference/agent). `act` differs in one way: with +`cache` it takes a `cacheKey`. + +```ts +const agent = createAgent(session, { + model: gateway('anthropic/claude-sonnet-5-5'), + cache: { dir: '.shop-bot/cache' }, +}); +const step = await agent.act('Add the item to the cart', { cacheKey: 'add-to-cart' }); +``` + +| Option | Type | Default | +| --- | --- | --- | +| `model`, `judge`, `context`, and the rest | an [`agents` entry](/reference/config#agents) | - | +| [`cache`](#cache) | `AgentCacheOptions` | no cache | + + + +##### `cache` + +| Field | Type | Default | +| --- | --- | --- | +| `dir` | `string` | `.e2e/cache` | +| `mode` | `'read-write' \| 'read-only'` | `'read-write'` | +| `strict` | `boolean` | `false` | +| `store` | [`CacheStore`](/cache) | the file store | + +With `cache`, each `act` passes `{ cacheKey: '' }`, or `{ cache: false }` +to run live. [Replay recorded steps](/core#replay-recorded-steps) says when a +recording is saved. + +## Related + +- [@e2e-dev/core guide](/core) +- [screen](/reference/screen), [app](/reference/app), and + [agent](/reference/agent): what a session and an agent hold. +- [Errors](/reference/errors): every error code. +- [Engines](/engines): write an engine of your own. diff --git a/docs/reference/engine-protocol.mdx b/docs/reference/engine-protocol.mdx index 7fc0a1c75b..5848148d93 100644 --- a/docs/reference/engine-protocol.mdx +++ b/docs/reference/engine-protocol.mdx @@ -231,6 +231,7 @@ params are the member's arguments by name, with every `OperationContext` as | `artifacts/startVideo` | `{ operation }` | `null` | | `artifacts/stopVideo` | `{ operation }` | `VideoSegment[]` | | `session/open` | `{ url, operation }` | `null` | +| `session/openLink` | `{ url, operation }` | `null` | | `session/back`, `session/restart`, `session/reset` | `{ operation }` | `null` | | `session/resize` | `{ size: { width, height }, operation }` | `null` | | `session/switchTab` | `{ index, operation }` | `null` | diff --git a/docs/reference/engine.mdx b/docs/reference/engine.mdx index 22c01dd470..51854e774a 100644 --- a/docs/reference/engine.mdx +++ b/docs/reference/engine.mdx @@ -12,8 +12,9 @@ implementation, derives its capabilities, and returns a handle for `targets` in the config. The runner drives it over the [engine protocol](/reference/engine-protocol), in its own process; see [Engines](/engines). Use `test.extend` fixtures for resources outside the UI, -such as APIs or databases. `@e2e-dev/core/engine` is the same module, and a -published engine imports it from there and peers on `@e2e-dev/core`. +such as APIs or databases. `@e2e-dev/core/engine` is the same module: a +published engine imports it from there and peers on `@e2e-dev/core`, so it runs +under the runner and under [@e2e-dev/core](/core) alike. This sketch assumes a `connect` adapter for the platform: @@ -67,11 +68,13 @@ Validates an engine spec, freezes it, computes its ```ts import { defineEngine, isEngineHandle, type EngineHandle } from 'e2e/engine'; -export function defineEngine(spec: Engine): EngineHandle; +export function defineEngine(spec: Engine): EngineHandle; +// With a fixture beside screen, app, and platform, `spec.fixtures` must hold +// a factory returning each one Fixtures names. export function isEngineHandle(value: unknown): value is EngineHandle; /** A validated engine: branded, frozen, capabilities computed. */ -export interface EngineHandle extends Engine { +export interface EngineHandle extends Engine { readonly capabilities: ReadonlySet; } ``` @@ -79,6 +82,13 @@ export interface EngineHandle extends Engine { Every function on the handle is bound to the spec it came from, so a class instance is as valid a body as an object literal. +`Fixtures` types what the engine's [`fixtures`](#contributed-fixtures) contribute, such as +`{ browser: Browser }`. The compiler requires a factory in `fixtures` for each +fixture it names, returning that fixture; `screen`, `app`, and `platform` need +none, since every session holds them. Nothing checks it at runtime. With it, `launch` from [`@e2e-dev/core`](/core) types a session's fixtures +without a hand-written argument: return `defineEngine<{ device: Device }>(...)` +from the factory. + `defineEngine` throws `INVALID_CONFIG` when the spec breaks one of these rules: @@ -463,7 +473,7 @@ engine. | `test.setup` sessions, the `session:` option | `state` | | `app.screenshot()`, assertion evidence, the failure screenshot | `artifacts` | | every [`video` mode](/reference/config#recording) but `off`, from the config, a target, a test, or `--video` | `artifacts` | -| `app.open()` | `session.open`; on a target with no app URL, `session.restart` | +| `app.open()` | `session.open`; on a target with no app URL, `session.restart`, or `session.openLink` for a path on a target with `app.linkBase` | | `app.back()` | `session.back` | | `app.restart()` | `session.restart` | | `app.clearState()` | `session.reset` | @@ -1433,6 +1443,7 @@ Steering hooks behind the universal `app` fixture and the agent's `navigate`, | Hook | Behind | | --- | --- | | `open(url)` | `app.open()`, the agent's `navigate` verb, the reopen after `restart` and `reset` | +| `openLink(url)` | `app.open(path)` on a target with `app.linkBase`: the joined link, its scheme checked, opened in the app under test. A target can declare `linkBase` only when its engine has this hook. | | `back()` | `app.back()`, the agent's `back` verb | | `resize(size)` | the agent's `resize_viewport` verb; `size` is `{ width, height }` in CSS pixels and holds for the rest of the attempt | | `switchTab(index)` | the agent's `switch_tab` verb; `index` is zero-based in the order the tabs opened, as [`tabs`](#observation) counts them. An index with no open tab is `ACTION_FAILED`. | @@ -1467,6 +1478,13 @@ export interface EngineSession { * origin policy. Absent on a surface without addressable locations. */ open?(url: string, context: OperationContext): Promise; + /** + * Opens a link in the app under test, as a deep link or a universal link + * would: `app.open(path)` on a target that declares `app.linkBase` lands + * here with the link joined and its scheme checked. A device engine's hook; + * a target whose engine lacks it cannot declare `linkBase`. + */ + openLink?(url: string, context: OperationContext): Promise; /** Navigates back once in the surface's history. */ back?(context: OperationContext): Promise; /** Sets the viewport to `size` in CSS pixels, for this page and every page the attempt opens after it. */ @@ -1613,6 +1631,7 @@ reproduces these semantics; import them instead of carrying a copy. | [`resolveExpression`](#resolveexpression) | `SemanticNode[]` | | [`urlMatches`](#urlmatches) | `boolean` | | [`siteOf`, `sameSite`](#site-helpers) | `string`, `boolean` | +| [`isRefusedLinkScheme`](#isrefusedlinkscheme) | `boolean` | | [`pollCondition`, `Deadline`](#pollcondition) | `Promise`, `Deadline` | | [`validateJsonValue`](#validatejsonvalue) | `void` | | [`rejectUnknownKeys`](#rejectunknownkeys) | `void` | @@ -1703,6 +1722,21 @@ export function siteOf(hostname: string): string; export function sameSite(url: string | URL, site: string): boolean; ``` +#### `isRefusedLinkScheme` + +Whether a link's scheme is one no app link may take: `file:`, `data:`, +`javascript:`, `view-source:`, `blob:`, or `filesystem:`. An engine that +opens app links (`session.openLink`) refuses these with `POLICY_DENIED`, as +`app.open` does with `app.linkBase`. + +```ts +export function isRefusedLinkScheme(protocol: string): boolean; +``` + +```ts +if (isRefusedLinkScheme(new URL(link).protocol)) throw new ConfigurationError('POLICY_DENIED', 'refused link scheme'); +``` + #### `pollCondition` Assertion polling and the negation grace window. diff --git a/docs/reference/errors.mdx b/docs/reference/errors.mdx index 8dcc3e9139..a45b16b4c6 100644 --- a/docs/reference/errors.mdx +++ b/docs/reference/errors.mdx @@ -95,17 +95,17 @@ category a blocked verdict reports as the obstacle. | --- | --- | --- | ---: | | `AUTH_CREDENTIAL_UNAVAILABLE` | A `Secret` or `credentials.user()` names a credential missing from `config.credentials`, the call ran outside an active runner, or the password's provider returned no string of at least 6 characters (code points) | configuration | 2 | | `AUTH_CREDENTIAL_INVALID` | The application rejected a configured credential (blocked: credentials) | configuration | 2 | -| `SECRET_UNAVAILABLE` | A `Secret` or `secrets.get()` during a run names a secret missing from `config.secrets`, the secret's provider returned no string of at least 6 characters (code points), or an engine resolved a secret it did not declare in its `secrets` | configuration | 2 | +| `SECRET_UNAVAILABLE` | A `Secret` or `secrets.get()` during a run names a secret missing from `config.secrets`, the secret's provider returned no string of at least 6 characters (code points), or an engine resolved a secret it did not declare in its `secrets`. Also `host.secret(name)` (`@e2e-dev/core`) naming a secret not passed to `launch` | configuration | 2 | | `ENVIRONMENT_UNAVAILABLE` | The environment under test is down or unreachable beyond the app itself (blocked: environment) | infrastructure | 3 | | `SEED_DATA_MISSING` | Data the step depends on does not exist in the environment (blocked: seed_data) | configuration | 2 | | `TEST_SETUP_FAILED` | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a `test.extend()` fixture that broke its contract: it returned without calling `use()`, called it twice, or redefined a fixture the target's engine contributes | configuration | 2 | | `AUTOMATION_UNSUPPORTED` | The step needs an interaction the available toolset cannot perform, such as a right-click on a device (blocked: automation) | test | 1 | | `MODEL_UNAVAILABLE` | No model configured: the built-in agent's `agents..model` holds no AI SDK model instance (the message names the selected agent), or GitHub Copilot does not serve the configured model to this account (`model_not_supported`; the message names the model). Recorded once as a run-level error, and the run stops. In a test run, a missing model is found when the `agent` fixture is first acquired, and an unserved model at its first call. `e2e explore` runs the built-in agent, so it checks for a missing model before anything starts, a custom executor's entry included. A custom executor without a model is not checked: its `act` and `assert` steps run, and `waitFor` or `extract` fails with this code on its first call. Evicts no [recording](/cache). | configuration | 2 | -| `POLICY_DENIED` | The security policy refused an operation: a denied destination, a forbidden fill, `vision: 'only'` without pixels, or a read from a secure field. [Policy denials](/reference/security#policy-denials) lists every case. An unknown tool or an undeclared field goes back to the model as the call's result, never as this code | configuration | 2 | +| `POLICY_DENIED` | The security policy refused an operation: a denied destination, a forbidden fill, `vision: 'only'` without pixels, or a read from a secure field. Also an `app.open` link with a refused scheme. [Policy denials](/reference/security#policy-denials) lists every case. An unknown tool or an undeclared field goes back to the model as the call's result, never as this code | configuration | 2 | | `APP_UNREACHABLE` | The app URL could not be reached, or an app command exited or exceeded `startupTimeout`. An exit after readiness during `e2e run` interrupts active tests and skips queued tests only for dependent targets with unfinished work. See [app commands](/reference/config#command) for diagnostics. Under the agent, also any engine fault with no more specific code. On a device, the message can identify a busy or wedged iOS automation runner and its recovery. See [Mobile](/mobile#troubleshooting). | infrastructure | 3 | | `APP_ALREADY_RUNNING` | `readyUrl` answered before the target's `app.command` was spawned, and `reuseExisting` was not in effect. Stop that process, or set `reuseExisting: true` locally | infrastructure | 3 | | `MODEL_PROVIDER_FAILED` | Provider transport failure (network error, 5xx, rate limit, or no response within 120 s) after the adapter's transport retries: up to 5 per model call, honoring `retry-after`, within the remaining step timeout. A stalled request is aborted at 120 s and sent again; an `agent.act` step's `--debug` transcript notes it. The 120 s covers the whole request, so a model that needs longer for one answer fails here too. An error body the provider package cannot read shows as the HTTP status and the body, secrets redacted, cut at 1 KB. Also a call the provider ended on an `error` finish, whatever text came with it, and an `e2e doctor --check-model` call with no answer within 1 minute. Evicts no [recording](/cache) | infrastructure | 3 | -| `CANCELLED` | The runner cancelled the operation without a process signal: an `e2e mcp` `open_session` cut short by a client cancel, a closed connection, or the server's shutdown, after it stopped what it had started, or a `run_tests` cancelled before it started or called while the server shuts down | infrastructure | 3 | +| `CANCELLED` | The runner cancelled the operation without a process signal: an `e2e mcp` `open_session` cut short by a client cancel, a closed connection, or the server's shutdown, after it stopped what it had started, or a `run_tests` cancelled before it started or called while the server shuts down. In `@e2e-dev/core`, an operation a `signal` aborted, or one on a closed session | infrastructure | 3 | | `MODEL_OUTPUT_INVALID` | Structured output still failed validation after the repair budget was spent. Under a [decision model](/decision-models): an answer without a valid choice distribution, a choice the request never offered, or a question the provider refused. Decisions are not repaired | test | 1 | | `APP_NOT_OPEN` | An operation needed an open app: `app.open()` had not run, or the engine reported invalid state | test | 1 | | `LOCATOR_NOT_FOUND` | No matching node within the deadline, or an agent action refers to a node that is no longer available. For a filtered locator the message can also list how many candidates each filter step left, counted after the failure, when that diagnostic pass can read the chain | test | 1 | @@ -130,9 +130,9 @@ These appear in terminal output and in the report as plain string codes. | `CONFIG_NOT_FOUND` | `--config ` names a file that does not exist, or no `e2e.config.ts`, `e2e.config.mts`, `e2e.config.js`, or `e2e.config.mjs` exists from the working directory up to the repository root | configuration | 2 | | `CONFIG_AMBIGUOUS` | Both `e2e.config.ts` and `e2e.config.mts` in one directory, or both `e2e.config.js` and `e2e.config.mjs` in one with no TypeScript config | configuration | 2 | | `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: options a factory the config calls refuses, or a `secrets.get()` handle turned into a string, is `INVALID_CONFIG` | configuration | 2 | -| `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`, `decisionFilter()`) refused its options, with the factory's own message. Or: an unknown key, out-of-range value, invalid enum, or missing `targets`; a target key outside `{ name, platform, engine, app, trace, video }`; a target `app` key that is unknown or malformed; `app` on a target without an engine; a `web()` target without `app.url` or with a device field; a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), with a link in `bundleId`, or with an unknown permission name or state (the message names the nearest name); an unknown key in a `web()` or `mobile()` option, `web({ connect })` and `web({ basicAuth })` included (the message names the nearest one); a `trace` or `video` that is not a recording mode; the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) or `artifacts.trace` block (the message names the `trace` mode they meant); the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option); the removed `specVersion` key; a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one); a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`), or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one); an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead); an unknown key in `app.command` (the message names the nearest one); an `engine` that is not a `defineEngine` handle; the `json` reporter combined with `list`, or in a run with `--step`; a duplicate or malformed target name; a secret or credential password shorter than 6 characters (code points); a credential that is not `{ username, password }`, whose `username` is not a string (only `password` may be a function), or that has no username in the config or `E2E_USER__USERNAME`; two credentials or two secrets whose names map to one `E2E_USER_*` or `E2E_SECRET_*` override variable (`api-key` and `api_key`); an oversized `agents..context`; an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds, or a `timeout` of 0 outside a test; a `tests` value that is not a glob or a list of globs, or a `tests` list of only `!` exclusions; the removed top-level `limits` key; an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`); an unknown agent key (the message names the nearest one); the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`); `system` or `tools` beside `executor`; a `tools` entry that `defineTool` did not build, or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `run_code`, `test_lines`, `start_recording`, `stop_recording`, or `report_finding`); `createToolLoopExecutor({ maxTurns })`; `reuseExisting` on an app whose `app.url` asks for a free port; a port-0 `app.url` with no `app.command` to serve it; a `filters` value that is not a list of `{ name, filter }` (for a bare function such as `decisionFilter`, the message says to call it); or a filter `name` that breaks the target name rules, is longer than 256 characters, or is used twice | configuration | 2 | +| `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`, `decisionFilter()`) refused its options, with the factory's own message. Or: an unknown key, out-of-range value, invalid enum, or missing `targets`; a target key outside `{ name, platform, engine, app, trace, video }`; a target `app` key that is unknown or malformed; `app` on a target without an engine; a `web()` target without `app.url` or with a device field; a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), an `app.linkBase` on a target whose engine opens no links, or one that is not an absolute URL or scheme, has a query or fragment, or names a refused scheme, with a link in `bundleId`, or with an unknown permission name or state (the message names the nearest name); an unknown key in a `web()` or `mobile()` option, `web({ connect })` and `web({ basicAuth })` included (the message names the nearest one); a `trace` or `video` that is not a recording mode; the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) or `artifacts.trace` block (the message names the `trace` mode they meant); the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option); the removed `specVersion` key; a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one); a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`), or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one); an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead); an unknown key in `app.command` (the message names the nearest one); an `engine` that is not a `defineEngine` handle; the `json` reporter combined with `list`, or in a run with `--step`; a duplicate or malformed target name; a secret or credential password shorter than 6 characters (code points); a credential that is not `{ username, password }`, whose `username` is not a string (only `password` may be a function), or that has no username in the config or `E2E_USER__USERNAME`; two credentials or two secrets whose names map to one `E2E_USER_*` or `E2E_SECRET_*` override variable (`api-key` and `api_key`); an oversized `agents..context`; an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds, or a `timeout` of 0 outside a test; a `tests` value that is not a glob or a list of globs, or a `tests` list of only `!` exclusions; the removed top-level `limits` key; an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`); an unknown agent key (the message names the nearest one); the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`); `system` or `tools` beside `executor`; a `tools` entry that `defineTool` did not build, or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `run_code`, `test_lines`, `start_recording`, `stop_recording`, or `report_finding`); `createToolLoopExecutor({ maxTurns })`; `reuseExisting` on an app whose `app.url` asks for a free port; a port-0 `app.url` with no `app.command` to serve it; a `filters` value that is not a list of `{ name, filter }` (for a bare function such as `decisionFilter`, the message says to call it); or a filter `name` that breaks the target name rules, is longer than 256 characters, or is used twice | configuration | 2 | | `CONFIG_NOT_DETERMINISTIC` | A worker re-resolved the config module and got a different digest than the runner | configuration | 2 | -| `APP_URL_REQUIRED` | A navigation or relative URL on a target that declares no `app.url`, or an `app.command` with no `app.url` or `app.readyUrl` to poll | configuration | 2 | +| `APP_URL_REQUIRED` | A navigation or relative URL on a target that declares no `app.url` (on a device target with no `app.linkBase`, `app.open(path)`), or an `app.command` with no `app.url` or `app.readyUrl` to poll | configuration | 2 | | `INVALID_APP_URL` | A base URL that is unparseable or has userinfo, a query, a fragment, a non-HTTP scheme, or plain HTTP on a non-loopback host; port 0 on a host other than a literal loopback address (`127.0.0.1`, `[::1]`); or a loopback address the run could not bind a free port on | configuration | 2 | | `INVALID_GLOB` | An empty pattern; a first segment that starts with `!` beyond the one leading `!` of a `tests` exclusion (`!!tests/**`, `./!tests/**`); a bare `!`; braces, character classes, or extglobs; a backslash separator; an absolute path or a `..` segment; a trailing `/` or `.`; a `tests` entry with no wildcard that names a directory (the message names the glob to write); a pattern that names only the project root; or a `**` that is not a whole segment | configuration | 2 | | `UNKNOWN_TARGET` | `--target`, `open_session.target`, or `openHost` (`e2e/runner`) named a target that is not configured | configuration | 2 | @@ -155,9 +155,9 @@ These appear in terminal output and in the report as plain string codes. | `INVALID_CHANGED_SINCE` | `--changed-since` (`changedSince`) is not a string, is empty, or starts with `-`. Or, when the run calls test filters or `e2e explore` explores the change, it names no commit in the checkout or shares no history with HEAD (in a shallow clone the message says to fetch more history, otherwise that the histories are unrelated); or the project is not in a git repository, git is missing, or git did not answer within 30 seconds | configuration | 2 | | `ONLY_IN_CI` | A focused test reached a CI run | configuration | 2 | | `COLLECTION_ERROR` | Registration outside collection; an invalid title; a non-function body; an async `describe` body; nested serial groups; an option forbidden inside a serial group; a setup test inside `describe`; an invalid session name; a malformed test option, such as `tags` that is not a list of distinct names; a test or `describe` option key the call does not take (the message names the nearest one); a test file that fails to import, in a run with no positionals or when the positionals selected it; a session that a selected test consumes, or `e2e explore --session` names, but no collected setup test produces (the message lists the declared sessions); a consumed session on a target whose engine has no `state` capability to save it; or a positional path outside the project root | configuration | 2 | -| `SESSION_MISMATCH` | A stored session does not match the current run, target, engine, platform, or app identity | configuration | 2 | -| `SESSION_EXPIRED` | A stored session is past its expiry at restore time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 | -| `SESSION_INVALID` | A stored session failed authentication. Tampered or corrupt. | configuration | 2 | +| `SESSION_MISMATCH` | A stored session does not match the current run, target, engine, platform, or app identity. Also an `@e2e-dev/core` `SessionState` passed to a host of another engine, platform, or app | configuration | 2 | +| `SESSION_EXPIRED` | A stored session, or an `@e2e-dev/core` `SessionState`, is past its expiry at restore or import time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 | +| `SESSION_INVALID` | A stored session failed authentication. Tampered or corrupt. Also `SessionState.import` (`@e2e-dev/core`) on bytes that are not a `session-state-1` document or that fail authentication under the key: another key, or a changed byte | configuration | 2 | | `SESSION_UNAVAILABLE` | A test requires a session no selected setup test produced | configuration | 2 | | `SESSION_CONTRACT` | A setup test saved a session twice, saved an undeclared name, or finished without saving a declared one | test | 1 | | `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout`. The body is not stopped: the harness operation it is in is cancelled, and its fixtures die with the attempt, so it cannot reach the next test. A test that blocks the event loop gets its worker killed once `cleanupTimeout` (at least 5 s) passes after the timeout. See [Hooks](/reference/test#hooks) | test | 1 | @@ -177,7 +177,7 @@ These appear in terminal output and in the report as plain string codes. | `TYPESCRIPT_COMPILER_UNAVAILABLE` | oxc, which compiles e2e's TypeScript, could not be loaded; the message gives the cause. Most often its native bindings for this platform (`@oxc-transform/binding--`, `@oxc-parser/binding--`) are not installed. npm leaves them out with `--omit=optional`, or when `package-lock.json` was written on another platform (npm/cli#4828). Install again without `--omit=optional`, or delete `package-lock.json` and `node_modules` and run `npm install` | infrastructure | 3 | | `IMAGE_CODEC_UNAVAILABLE` | `@napi-rs/image`, which reads and writes the PNGs of screenshots, could not be loaded; the message gives the cause. It is loaded when the first screenshot is read or written, so a run that takes none never needs it. Most often its native binding for this platform (`@napi-rs/image--*`) is not installed. npm leaves it out with `--omit=optional`, or when `package-lock.json` was written on another platform (npm/cli#4828). Install again without `--omit=optional`, or delete `package-lock.json` and `node_modules` and run `npm install`. The package publishes no binding for Linux on ppc64, s390x, or riscv64, where `toHaveScreenshot` cannot work, and neither can an agent screenshot the runner has to resize (one with a side over 768 px) | infrastructure | 3 | | `ENGINE_FAILURE` | Any engine fault with no more specific mapping | infrastructure | 3 | -| `WORKER_PROTOCOL` | A worker reported the wrong unit, an attempt overlapped a live attempt, `openHost` tried to acquire an engine instance another host owns, or `host.openSession` tried to reuse a closed host or overlap an opening, live, or closing session | infrastructure | 3 | +| `WORKER_PROTOCOL` | A worker reported the wrong unit, an attempt overlapped a live attempt, `openHost` tried to acquire an engine instance another host owns, or `host.openSession` tried to reuse a closed host or overlap an opening, live, or closing session. In `@e2e-dev/core`, `launch` on an engine instance another host owns, or `host.newSession` on a closed host or while a session is open | infrastructure | 3 | | `SESSION_KEY_INVALID` | A worker received a session key that is not 32 bytes. Packaging or IPC fault. | infrastructure | 3 | | `REPORT_WRITE_FAILED` | `report.json` or `ai-trace.json` could not be written. Recorded as a run error; the in-memory outcome is still complete. A `junit.xml` that could not be written is a line on the `junit` reporter's stderr instead | infrastructure | 3 | | `EVALUATE_FAILED` | `browser.evaluate` threw inside the page, or the page deleted or replaced a JavaScript built-in (such as `Map`) that the engine's in-page code needs, which fails later evaluates and locators (`@e2e-dev/web`) | test | 1 | diff --git a/docs/reference/mobile.mdx b/docs/reference/mobile.mdx index 75f3fbf9e8..101a45599a 100644 --- a/docs/reference/mobile.mdx +++ b/docs/reference/mobile.mdx @@ -295,8 +295,9 @@ export default { - A device target needs `app.bundleId` or `app.appPath`. - `app.open()` launches the target's app fresh. An attempt launches nothing on its own, so a test starts where the previous one left the app. -- `app.open` takes no path on a device: a link goes through - [`device.openLink`](#openlink). +- `app.open(path)` opens the path as a link in the app when the target + declares [`app.linkBase`](/reference/config#linkbase); without it, `app.open` + takes no path and a link goes through [`device.openLink`](#openlink). - `app.appPath` is the build the suite runs against. The engine installs nothing on its own: [`device.installApp()`](#installapp) installs it where the suite says so. A device provider installs it on the device it leases. @@ -411,7 +412,7 @@ them to finish. | `keyboard` | `type` and `press` into the focused field, the agent's `type` and `press` without a target and the fallback of `type_at` and `press_at`, and `dismiss`, the agent's `dismiss_keyboard`. | | `locate` | Every `screen` query. Text and label queries answer with the innermost match, since a device echoes both up the tree (a React Native input is a host view and the field inside it, both labelled). On Android a test id query, a string or a RegExp, also matches a resource id without its `:id/` prefix, for any package but the framework's `android`, when nothing on the screen matches the full id. `screen.locator('id=…')` is agent-device's selector and takes the full id. | | `validateApp` | Requires `app.bundleId` or `app.appPath` and refuses `app.url` for now, with a note on reaching this machine from the platform's device. A target with either is a `native-app` target, so `requires: ['native-app']` selects device targets. | -| `session` | `back`, `restart`, and `reset`. No `open`: `app.open()` takes no path and the runner serves it with `restart`, the fresh launch of the pinned app; the agent's `navigate` verb is unavailable, and `device.openLink(url)` opens a deep link or web link. | +| `session` | `back`, `restart`, `reset`, and `openLink`, behind `app.open(path)` on a target with `app.linkBase`. No `open`: `app.open()` with no path is served with `restart`, the fresh launch of the pinned app; the agent's `navigate` verb is unavailable, and `device.openLink(url)` opens a deep link or web link. | | `artifacts` | Redacted screenshots under `screenshots/NNN-