Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
14 changes: 12 additions & 2 deletions .changeset/config.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
{
"$schema": "https://unpkg.com/@changesets/config@4.0.0/schema.json",
"changelog": ["@changesets/changelog-github", { "repo": "e2e-inc/e2e" }],
"changelog": [
"@changesets/changelog-github",
{
"repo": "e2e-inc/e2e"
}
],
"commit": false,
"access": "public",
"baseBranch": "main",
"updateInternalDependencies": "patch",
"fixed": [],
"fixed": [
[
"e2e",
"@e2e-dev/core"
]
],
"linked": [],
"ignore": [],
"privatePackages": false,
Expand Down
8 changes: 8 additions & 0 deletions .changeset/e2e-core-package.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'e2e': patch
'@e2e-dev/core': patch
'@e2e-dev/web': patch
'@e2e-dev/mobile': patch
---

New package `@e2e-dev/core`: the engine contract, locators, agent, replay cache, and engine sessions that `e2e` is built on. `e2e` depends on it at the same version, and `e2e/engine` and `e2e/agent` re-export it unchanged. `@e2e-dev/web` and `@e2e-dev/mobile` peer on `@e2e-dev/core` beside `e2e`, and import the contract from `@e2e-dev/core/engine`, which a third-party engine can now import too. npm, pnpm and Bun install the new peer themselves. A Yarn project lists `@e2e-dev/core` beside the engine; `e2e init` and `e2e upgrade` add it.
2 changes: 1 addition & 1 deletion .fallowrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
"ignoreBinaries": ["eas"],
"ignoreExports": [
{
"file": "packages/e2e/src/types.ts",
"file": "packages/core/src/types.ts",
"exports": ["StandardSchemaV1"]
}
],
Expand Down
9 changes: 5 additions & 4 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,15 @@
* @okwasniewski

# Public contract: emitted types, type assertions, wire schemas.
/packages/e2e/src/types.ts @okwasniewski
/packages/e2e/src/engine/ @okwasniewski
/packages/core/src/types.ts @okwasniewski
/packages/core/src/engine/ @okwasniewski
/packages/e2e/src/run-types.ts @okwasniewski
/packages/e2e/tests/types/ @okwasniewski
/packages/e2e/schema/ @okwasniewski

# Agent, secrets, navigation policy.
/packages/e2e/src/agent/ @okwasniewski
/packages/e2e/src/internal/urls.ts @okwasniewski
/packages/core/src/agent/ @okwasniewski
/packages/core/src/internal/urls.ts @okwasniewski
/SECURITY.md @okwasniewski

# Engines.
Expand Down
4 changes: 3 additions & 1 deletion .github/actions/example-packages/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ description: >-
Installs an example's dependencies with e2e, @e2e-dev/web, and
@e2e-dev/mobile swapped for the tarballs the examples workflow packed from
main, so the example runs this checkout's packages instead of the npm
releases its package.json names.
releases its package.json names. @e2e-dev/core, which they all build on, comes
from its tarball through an override, since e2e names its exact snapshot.

inputs:
agent-device-version:
Expand Down Expand Up @@ -45,6 +46,7 @@ runs:
console.log(`${name}: ${manifest[field][name]}`);
}
}
manifest.overrides = { ...manifest.overrides, "@e2e-dev/core": `file:${process.env.PACKED}/core.tgz` };
fs.writeFileSync("package.json", `${JSON.stringify(manifest, null, 2)}\n`);
'
if [ -n "$AGENT_DEVICE_VERSION" ]; then
Expand Down
1 change: 1 addition & 0 deletions .github/filters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ runner: &runner
- '.github/actions/workspace-install/**'
- '.github/scripts/agent-device.cjs'
- 'packages/e2e/**'
- 'packages/core/**'
- '!packages/*/tests/**'
- '!**/*.md'
- 'package.json'
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Pack
run: |
pnpm --filter @e2e-dev/core pack --out "$RUNNER_TEMP/packed/core.tgz"
pnpm --filter e2e pack --out "$RUNNER_TEMP/packed/e2e.tgz"
pnpm --filter @e2e-dev/web pack --out "$RUNNER_TEMP/packed/web.tgz"
pnpm --filter @e2e-dev/mobile pack --out "$RUNNER_TEMP/packed/mobile.tgz"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ jobs:
- uses: ./.github/actions/workspace-install
with:
agent-device-version: ${{ inputs.agent-device-version }}
- run: pnpm --filter e2e run build
- run: pnpm --filter @e2e-dev/core run build && pnpm --filter e2e run build
- name: The CLI on Bun ${{ matrix.bun }}
run: pnpm --filter e2e exec vitest run tests/integration/bun-runtime.test.ts
env:
Expand Down
66 changes: 43 additions & 23 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ TypeScript 7.
There is no separate spec. The code is the contract, pinned in three places:

- The emitted `packages/e2e/dist/index.d.ts` (and `dist/engine/index.d.ts`,
`dist/runner.d.ts`, `dist/oauth/*.d.ts`)
is the public API. `packages/e2e/tests/types/sdk-types.ts` holds compile-time
`dist/runner.d.ts`, `dist/oauth/*.d.ts`), and `@e2e-dev/core`'s
`packages/core/dist/engine/index.d.ts`, are the public API. `packages/e2e/tests/types/sdk-types.ts` holds compile-time
assertions (`@ts-expect-error` lines) for the parts that are easy to loosen
by accident; it runs under the package `typecheck`, never under vitest.
- Wire formats live in `packages/e2e/schema/*.schema.json` (report-1,
Expand All @@ -34,8 +34,28 @@ descriptions and commit bodies; `git log` and `gh pr view` are the archive.
`packages/` holds what publishes to npm; `apps/` holds the private apps and
suites that consume the built packages the way a user would.

- `packages/core` — the published `@e2e-dev/core` package, which `e2e` is built
on and depends on at its exact version (a changesets `fixed` group keeps
the two versions equal): the engine contract (`@e2e-dev/core/engine`, and
`src/engine/protocol/`), locators (`src/locator/`), the agent
(`src/agent/`, the `act` executor socket plus the judgment methods),
the replay cache (`src/cache/`), secrets, and the session half of a
worker: `run/engine-sessions.ts` (engine init and dispose, each attempt's
session from launch to close, which the runner's `TargetExecutor`
extends), `run/standalone.ts` (the worker a runner host or MCP keeps
alive, with a `WorkerPolicy` per kind), and `run/standalone-attempt.ts`
(one attempt: reservation, launch, fixtures, deadline, close). Its one
public entry is `@e2e-dev/core/engine`. It imports nothing of `e2e`:
`tests/unit/package-imports.test.ts` in `packages/e2e` fails on a
dependency `@e2e-dev/core`'s manifest does not declare. `e2e` reaches its
modules as `@e2e-dev/core/lib/<path>` (`./lib/*` in its exports), which is
not public API; `@e2e-dev/core`'s behavior is tested by `packages/e2e`'s
suites, through those paths and through the runner.
- `packages/e2e` — the published `e2e` package: SDK surface, runner, CLI,
`e2e/engine` contract. Core knows the contract and never an engine's
and `e2e/engine` and `e2e/agent`, which re-export `@e2e-dev/core`'s. A
config's reporters and test filters resolve in `config/resolve-run.ts`
over `@e2e-dev/core`'s `resolveProjectConfig`, and their types live in
`src/run-types.ts`. Core knows the contract and never an engine's
internals: no engine package is imported from `src/` (grep for
`from '@e2e-dev/` and `import('@e2e-dev/`; the presets below are the only
hits). Engine words still appear as platform labels, capability names, and in
Expand All @@ -48,18 +68,15 @@ suites that consume the built packages the way a user would.
example, and run command; interactive choices derive from this list. These
presets never import engine implementations.
- `src/run/` runner core (scheduler, units, workers, retries, sessions;
`standalone.ts` is the worker a host or MCP's one-shot attempt keeps
alive, `standalone-attempt.ts` one attempt on it: reservation, launch,
fixtures, deadline, close, `host-trace.ts` its `trace.md`, and
`standalone-kinds.ts` the hosted and one-shot kinds of `@e2e-dev/core`'s
standalone worker, `host-trace.ts` a hosted attempt's `trace.md`, and
`select-target.ts` the target a session names, shared with MCP;
`open-session.ts` exposes `openHost` on `e2e/runner` for another test
runner: worker-lifetime provisioning, app processes, and engine init,
then `host.openSession({ name })` for each test's fixtures, deadline,
steps, artifacts, and trace at close. Only one hosted attempt may be
opening, live, or closing per process; close it before opening another),
`src/collect/` registration+selection, `src/locator/` locator AST/engine,
`src/agent/` the agent (the `act` executor socket plus the judgment
methods), `src/mcp/` the `e2e mcp` server (a live session that rides
`src/collect/` registration+selection, `src/mcp/` the `e2e mcp` server (a live session that rides
the `act` socket with a queue executor so every MCP call is a harness
action; in a directory with no config, `open_session {engine}` names the
engine as a package, whose default export is called, or as a factory
Expand All @@ -76,17 +93,19 @@ suites that consume the built packages the way a user would.
checks that need a stored login and are never part of `pnpm test`.
- `packages/web` — the published `@e2e-dev/web` package: the
browser engine, built with the public `defineEngine`, contributing the
`browser` fixture and `expect(browser)`. It depends on `e2e` (peer), never the
reverse; a target names it explicitly as `engine: web()`. There is
no default engine and no well-known id registry in core. It imports from
`e2e/engine` only: the semantics every engine must reproduce
(error taxonomy, text and URL matching, assertion polling, JSON-value rules)
are exported there, and there is no `e2e/internal` subpath.
`browser` fixture and `expect(browser)`. It depends on `@e2e-dev/core` and
`e2e` (peers), never the reverse; a target names it explicitly as
`engine: web()`. There is no default engine and no well-known id registry
in core. It imports from `@e2e-dev/core/engine` only: the semantics every
engine must reproduce (error taxonomy, text and URL matching, assertion
polling, JSON-value rules) are exported there, and an engine never imports
`@e2e-dev/core/lib/*`.
- `packages/mobile` — the published `@e2e-dev/mobile` package: the
iOS/Android engine on agent-device, built with the same public
`defineEngine`, contributing the `device` fixture. Like the web engine it
depends on `e2e` (peer), never the reverse; the engine implementation
imports from `e2e/engine` only, and a target names it as `engine: mobile()`.
depends on `@e2e-dev/core` and `e2e` (peers), never the reverse; the
engine implementation imports from `@e2e-dev/core/engine` only, and a target
names it as `engine: mobile()`.
The `@e2e-dev/mobile/tools` subpath holds the agent-side `open_app`, `swipe`,
and `alert` tools; focused-field typing uses the engine's keyboard grammar.
The main entry never loads the AI SDK.
Expand All @@ -97,7 +116,7 @@ suites that consume the built packages the way a user would.
device-cloud providers (`testmu`, `browserstack`) share: one daemon per
slot, heartbeat, and a release that fails unless the session end is
confirmed.
- `packages/e2e/src/engine/protocol/` - the engine protocol: the engine
- `packages/core/src/engine/protocol/` - the engine protocol: the engine
contract as LSP-style JSON-RPC (`connectEngine` client, `serveEngine`
server, spawn/socket/loopback transports), so an engine can run in
another process, machine, or language. The harness holds only protocol
Expand Down Expand Up @@ -337,8 +356,8 @@ pnpm test:web-benchmark # builds, then runs the real CLI against the benchmark
Focused work:

```bash
pnpm --filter e2e run build
pnpm --filter e2e run test:unit # unit only, no build
pnpm --filter @e2e-dev/core run build && pnpm --filter e2e run build
pnpm --filter e2e run test:unit # unit only; needs @e2e-dev/core built
pnpm --filter e2e exec vitest run tests/unit/scheduler.test.ts
pnpm --filter e2e exec vitest run -t 'name fragment'
pnpm --filter @e2e-dev/web run test
Expand All @@ -348,8 +367,9 @@ pnpm --filter @e2e-dev/testbed run test:headed
- Package `test` scripts do **not** build. Root `build` and `test` order the
packages explicitly rather than relying on topological sort, because `e2e`
devDepends on the web engine for its browser-backed integration tests
while the engine peer-depends on `e2e` — pnpm reports that cycle on every
install.
while the engine peer-depends on `e2e` — pnpm reports that cycle
on every install. `@e2e-dev/core` builds first: `e2e` and every engine compile
against its `dist`.
- `pnpm typecheck` runs `build` first, then per-package `typecheck`. The
package `typecheck` covers `tests/**`, which is what makes
`tests/types/sdk-types.ts` a test.
Expand Down Expand Up @@ -693,7 +713,7 @@ trees, on both platforms, without a device.
entry. On `0.x` the bump is `patch` for a feature or fix; never
pick `minor` on your own. A breaking change is a `minor` with the summary
opening `**Breaking:**`; a `minor` or `major` for a release the maintainer
calls opens `**Release:**` (`scripts/check-changeset-bumps.ts`, `pnpm check`). Peer ranges point one way only (engine -> `e2e`, integration ->
calls opens `**Release:**` (`scripts/check-changeset-bumps.ts`, `pnpm check`). Peer ranges point one way only (engine -> `@e2e-dev/core` and `e2e`, integration ->
engine) and read `>=<major.minor.patch> <major+1>`, the floor the oldest
sibling the package works with. Raise it when a package starts relying on a
newer sibling's export or contract field;
Expand Down
23 changes: 14 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,8 +289,9 @@ committed.
The following are the public contract. A change that breaks any of them is a
breaking change and follows the rules below:

- Public types exported from `e2e` and `e2e/engine`, as
emitted in `dist/index.d.ts` and `dist/engine/index.d.ts`. Removing an
- Public types exported from `e2e`, `e2e/engine`, and `@e2e-dev/core/engine`,
as emitted in `dist/index.d.ts` and `dist/engine/index.d.ts` of `e2e`,
and `dist/engine/index.d.ts` of `@e2e-dev/core`. Removing an
export, narrowing an accepted input, widening a returned type, or changing a
fixture's runtime behavior all count.
- Config keys in `e2e.config.ts`, their types, and their defaults.
Expand All @@ -300,7 +301,8 @@ breaking change and follows the rules below:
Adding an optional field is compatible; renaming, removing, or changing the
meaning of one is not.

Not covered: anything under `internal`, the testbed, the exact text of error
Not covered: anything under `internal`, `@e2e-dev/core/lib/*` (the modules `e2e`
reaches at the exact `@e2e-dev/core` version it depends on), the testbed, the exact text of error
messages and reporter output (error *codes* are covered), and the agent's
prompts and tool descriptions.

Expand All @@ -326,12 +328,15 @@ prompts and tool descriptions.

### Engine contract

`e2e/engine` is what `@e2e-dev/web` and `@e2e-dev/mobile`
build against, and what a third-party engine builds against too. A change to
that contract bumps all three packages together in one release, with a
changeset for each, so an engine and a runner from the same release always
match. Engines declare a peer range on `e2e` that points one way only
(engine to runner, `>=x <1`), and each integration (`@e2e-dev/kernel`, `@e2e-dev/lightpanda`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`, `@e2e-dev/browserstack`, `@e2e-dev/e2b`) does
`@e2e-dev/core/engine` (re-exported as `e2e/engine`) is what `@e2e-dev/web` and
`@e2e-dev/mobile` build against, and what a third-party engine builds against
too. A change to that contract bumps `@e2e-dev/core`, `e2e`, and both engines
together in one release, with a changeset for each, so an engine and a runner
from the same release always match. `e2e` and `@e2e-dev/core` are a changesets
`fixed` group: they always share a version, and `e2e` depends on that exact
`@e2e-dev/core`. Engines declare a peer range on `@e2e-dev/core` that points one way
only (engine to core, `>=x <1`), plus one on `e2e` for the main entry that
adds the engine's fixture to `test`, and each integration (`@e2e-dev/kernel`, `@e2e-dev/lightpanda`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`, `@e2e-dev/browserstack`, `@e2e-dev/e2b`) does
the same on the engines it plugs into; do not make it mutual or narrow it.

### Driver nightly compatibility
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1146,6 +1146,10 @@ Updated the agent skill in .agents/skills/e2e/ (12 files)
does. When none does, the command exits 2. A deprecated release is
never picked, and a prerelease only when `e2e` gets one. An integration
follows the engine it plugs into.
- **An added peer.** An `@e2e-dev/*` package those releases peer on, such as
`@e2e-dev/core` for an engine, is added beside the package that needs it
when `package.json` doesn't list it. Yarn installs no peer the project
doesn't list. It gets the version the new `e2e` depends on.
- **The package manager.** The one the project's lockfile or
`packageManager` field names, or pnpm at a pnpm workspace root. A
workspace member with neither uses its workspace root's. A lockfile above
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ implementation, derives its capabilities, and returns a handle for `targets`
in the config. The runner drives it over the
[engine protocol](/reference/engine-protocol), in its own process; see
[Engines](/engines). Use `test.extend` fixtures for resources outside the UI,
such as APIs or databases.
such as APIs or databases. `@e2e-dev/core/engine` is the same module, and a
published engine imports it from there and peers on `@e2e-dev/core`.

This sketch assumes a `connect` adapter for the platform:

Expand Down
12 changes: 7 additions & 5 deletions docs/writing-an-engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,18 @@ Return a handle from `defineEngine`. It validates the manifest and computes
the engine's capabilities when the config loads.

<Note>
Import types and helpers from `e2e/engine` only; there is no internal
subpath. Declare `e2e` as a peer dependency, so the engine and the runner
share one set of error classes.
Import types and helpers from `@e2e-dev/core/engine` only; there is no internal
subpath. Declare `@e2e-dev/core` as a peer dependency, so the engine and the runner
share one set of error classes. `e2e/engine`
re-exports the same module, so an engine that imports it and peers on `e2e`
still runs under the runner.
</Note>

This skeleton assumes a `createClient` adapter for your platform. Its nodes
must follow the [semantic tree rules](#nodes-and-refs).

```ts title="src/index.ts" expandable
import { ConfigurationError, defineEngine, resolveExpression, type EngineHandle } from 'e2e/engine';
import { ConfigurationError, defineEngine, resolveExpression, type EngineHandle } from '@e2e-dev/core/engine';

export function hyperdrive(options: { device: string }): EngineHandle {
const client = createClient(options);
Expand Down Expand Up @@ -215,7 +217,7 @@ An engine option can take a secret too, such as a password the engine sends
to the app:

1. Accept `string | Secret`. Tell them apart with `isSecret` from
`e2e/engine`.
`@e2e-dev/core/engine`.
2. List each handle in the engine's `secrets`. The config load rejects a
name that no secret has.
3. Read the value in `startAttempt` with `context.resolveSecret(secret)`.
Expand Down
Loading
Loading