Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/mcp-engine-session.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"e2e": patch
"@e2e-dev/web": patch
"@e2e-dev/mobile": patch
---

Open an MCP session in a directory with no config: `open_session {engine: "@e2e-dev/web", app: {url}}` names the engine to drive and `app` the app under test. `engine` is a package name, or the factory call a config would write as `{package, export, options}`, where an option can be another such call: `{package: "@e2e-dev/mobile", options: {platform: "ios", device: {package: "@e2e-dev/limrun", export: "limrun"}}}`. Each package is found in the project, beside e2e, or in `~/.e2e/engines`; when it is in none, `ENGINE_NOT_INSTALLED` names the install command. `@e2e-dev/web` and `@e2e-dev/mobile` export their factory as the default export for this, and `mobile()` called without a platform now says which platforms it takes.
6 changes: 5 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,11 @@ suites that consume the built packages the way a user would.
`src/agent/` the agent (the `act` executor socket plus the judgment
methods), `src/mcp/` the `e2e mcp` server (a live session that rides
the `act` socket with a queue executor so every MCP call is a harness
action), `src/oauth/` subscription sign-in (the `e2e login`, `logout`,
action; in a directory with no config, `open_session {engine}` names the
engine as a package, whose default export is called, or as a factory
call, a package with an export and options that may hold further such
calls, which `mcp/default-engine.ts` resolves in the project, beside e2e,
or in `~/.e2e/engines`, never by a specifier written in core), `src/oauth/` subscription sign-in (the `e2e login`, `logout`,
and `models` commands and the `e2e/oauth/*` model constructors; each
constructor subpath is the place its `@ai-sdk/*` optional peers are
imported, and the Copilot constructor loads `@ai-sdk/openai` lazily, only
Expand Down
13 changes: 13 additions & 0 deletions docs/mcp-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,19 @@ A session starts the target's app command, if it has one, and names it
with its URL and port. The config reads the server's environment, not the
agent's shell.

## Try it without a config

In a directory with no e2e config file, the agent names the engine when it
opens a session:

```txt
open_session {engine: "@e2e-dev/web", app: {url: "http://localhost:3000"}}
```

`run_tests` still needs a config. For engine options, device providers, and
installing an engine, see
[Without a config](/reference/mcp#without-a-config).

## Ask the agent

- "Add a Remember me checkbox to the login form. Check it in the app with e2e."
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/errors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ These appear in terminal output and in the report as plain string codes.
| `NO_SESSION` | `e2e mcp`: `call` or `tools` before `open_session` or after the session ended (the message names why), or `call`, `tools`, or `close_session` naming a `session` id that is not open or is closing | configuration | 2 |
| `SESSION_REQUIRED` | `e2e mcp`: `call`, `tools`, or `close_session` without a `session` while several sessions are open. The message lists them | configuration | 2 |
| `ENGINE_IN_USE` | `e2e mcp`: `open_session` on a target whose engine instance another open session drives, because the config gets it from a package every session shares. Create the engine in the config, or in a file it imports by path | configuration | 2 |
| `CONFIG_IN_USE` | `e2e mcp`: `open_session` on a config other than the one the open sessions use. Credentials and secrets resolve process-wide, so open on that config or close those sessions first | configuration | 2 |
| `ENGINE_NOT_INSTALLED` | `e2e mcp`: `open_session {engine}` named a package that is not installed in the project, beside e2e, or in `~/.e2e/engines`. The message names the install command | configuration | 2 |
| `CONFIG_IN_USE` | `e2e mcp`: `open_session` on a config other than the one the open sessions use, or on a config while sessions opened with `engine` are open. Credentials and secrets resolve process-wide, so open on that config or close those sessions first | configuration | 2 |
| `SESSION_OPEN` | `e2e mcp`: `open_session` while every session slot (`--max-sessions`, default 4) holds an open, opening, or closing session, or while the server shuts down. Close one first | configuration | 2 |
| `UNKNOWN_TOPIC` | `e2e mcp`: an `e2e://guide/{topic}` resource named a topic the skill does not have. The message lists them | configuration | 2 |
| `UNKNOWN_TOOL` | `e2e mcp`: `call` or `tools` named a tool the session's catalog does not have. The message lists the catalog. An action verb the engine does not declare is `UNSUPPORTED_CAPABILITY` instead | configuration | 2 |
Expand Down
68 changes: 67 additions & 1 deletion docs/reference/mcp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ open_session {target: "web", headed: true}
| `config` | string (path) | no | The config for this session, absolute or relative to the server's working directory (`cwd` in the client's MCP config). Otherwise `--config`, else discovered like `e2e run`. |
| `headed` | boolean | no | `true` shows this session's browser or simulator, `false` hides it. Without it the session follows `--headed`. |
| `test` | string | no | One test to run first, named like an `e2e run` positional. The session stops where the test stopped. See [Running a test](#running-a-test). |
| `engine` | string or object | no | The session's engine in a directory with no config file, as a package name or `{package, export, options}`. See [Without a config](#without-a-config). |
| `app` | object | no | The app under test for an `engine` session, as `url`, `bundleId`, or `appPath`. |

#### `tools`

Expand Down Expand Up @@ -534,6 +536,70 @@ Start the server in the project directory. Config discovery works like
`e2e run`. For another config, see [`--config`](#config) and
[`open_session`](#open_session).

## Without a config

In a directory with no config file, `open_session` needs an `engine`. The
session has one target on that engine. Every other setting is at its
default, so the session has no credentials, secrets, or project tools.

```txt
open_session {engine: "@e2e-dev/web", app: {url: "http://localhost:3000"}}
open_session {engine: {package: "@e2e-dev/mobile", options: {platform: "ios"}}, app: {bundleId: "com.example.app"}}
```

A package name is enough for an engine that needs no options. To pass
options, or to name an export, write `engine` as an object:

| Field | Type | Does |
| --- | --- | --- |
| `package` | string | The package to load. It can carry a subpath, such as `@e2e-dev/testmu/web`. |
| `export` | string | The export to call. Default: the default export. |
| `options` | object | The options the factory takes, as in an `e2e.config.ts`. |

- **A package name alone** calls the package's default export with no
options. `@e2e-dev/web` and `@e2e-dev/mobile` export their factory this
way. [Your own engine](/writing-an-engine#open-it-by-name) can do the same.
- **An option can be another factory call.** The server reads an option
value as a call when it has `package` and no key other than `export` and
`options`. It makes that call first and passes the result as the option.
Use it to give an engine a device or browser provider:

```txt
open_session {
engine: {package: "@e2e-dev/mobile", options: {
platform: "ios",
device: {package: "@e2e-dev/limrun", export: "limrun", options: {region: "eu"}}
}},
app: {bundleId: "com.example.app"}
}
```

The call above builds
`mobile({ platform: 'ios', device: limrun({ region: 'eu' }) })`. The
provider reads its API key from the server's environment.
- **Lookup.** The server looks for each package in the project, then beside
the running `e2e`, then in `~/.e2e/engines`. When it is in none of them,
the error names the command that installs it for every project:
`npm install --prefix ~/.e2e/engines @e2e-dev/web`.
- **`app`** takes `url`, `bundleId`, and `appPath`, as a target's
[`app`](/reference/config#the-app-under-test) does. `@e2e-dev/web` needs
`url`. `@e2e-dev/mobile` needs `bundleId` or `appPath`.
- **Files.** The project root is the server's working directory, and the
session writes `.e2e/` there.
- **Tests.** `run_tests` and `open_session {test}` need a config. To keep
the session's test lines as a test, write a config with
[`e2e init`](/reference/cli#e2e-init).
- **Several sessions.** Sessions opened with `engine` run side by side. A
session on a config doesn't open beside them and fails with
`CONFIG_IN_USE`.

| Throws | When |
| --- | --- |
| `CONFIG_NOT_FOUND` | `open_session` without `engine`, and the directory has no config. |
| `ENGINE_NOT_INSTALLED` | A package that `engine` names is in none of the three lookup places. |
| `INVALID_ARGUMENT` | `package` is not a package name, `export` is not an export name, `engine` comes with `target`, `config`, or `test`, or the directory has a config. Also `app` without `engine`. |
| `INVALID_CONFIG` | The named export is not a function, a factory refused its `options`, the engine's factory returned no engine, or the engine refused `app`. For an export that is not a function, the message lists the package's functions. |

## Output

- Operator output goes to stderr, prefixed `e2e mcp:`. Stdout carries only
Expand Down Expand Up @@ -661,7 +727,7 @@ into a scripted test drives its own session.
| `SESSION_REQUIRED` | A call without `session` while several are open. The message lists the open sessions. |
| `NO_SESSION` | A `session` id that is not open. For a session that is closing or ended, the message says why. |
| `SESSION_OPEN` | An `open_session` past the limit, or while the server shuts down. |
| `CONFIG_IN_USE` | An `open_session` on another config while sessions are open. |
| `CONFIG_IN_USE` | An `open_session` on another config while sessions are open, or on a config beside sessions opened with `engine`. |
| `ENGINE_IN_USE` | A second session on an engine created inside a package. |
| `CANCELLED` | An `open_session` the client cancelled or disconnected from, or the server shut down during, once what it started has stopped. |

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -350,5 +350,5 @@ plugins, and app can make more connections.

| Command | Rule |
| --- | --- |
| `e2e mcp` | Uses stdio and the same action authorization as a test run. Four open sessions by default ([`--max-sessions`](/reference/mcp), 1 through 16). Closes idle sessions. |
| `e2e mcp` | Uses stdio and the same action authorization as a test run. Four open sessions by default ([`--max-sessions`](/reference/mcp), 1 through 16). Closes idle sessions. `open_session {engine}` loads the installed packages the agent names and calls the exports it names. That code runs with the server's full authority, like a config's imports. `engine` names packages only, never a path or a URL, and the server installs nothing. |
| `e2e explore` | Offers every configured credential to every step. The explorer sees account names and usernames. Passwords stay handles and need an engine that supports sensitive fills; without one, the explorer skips them. With no credentials, it may sign in with made-up test credentials, typed as plain text. |
20 changes: 20 additions & 0 deletions docs/writing-an-engine.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,26 @@ For platform operations the action set lacks, ship
[project tools](/tools) built with `defineTool`, with `platforms` set so each
shows only on compatible targets, as `@e2e-dev/mobile/tools` does.

## Open it by name

Export the engine's factory as the package's default export. Then a coding
agent can open an [MCP session](/reference/mcp#without-a-config) on the
engine in a directory with no config:

```ts title="src/index.ts"
export default function kiosk(options?: KioskOptions) {
return defineEngine({ /* ... */ });
}
```

```txt
open_session {engine: {package: "kiosk-engine", options: {lane: 2}}}
```

The factory receives `options`, or `undefined` when the agent passes only
the package name. Return a new engine on every call, so each session gets
its own.

## Verify it

Run a test file against your engine. Check the lifecycle order,
Expand Down
7 changes: 4 additions & 3 deletions packages/e2e/src/config/load.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,16 +78,17 @@ function explicitConfigNotFound(cwd: string, explicitPath: string, absolute: str
* The failure for a run that found no config file: where the search looked,
* a lookalike in the working directory when one exists, and otherwise the
* command that creates a config. Thrown when a config file is discovered,
* never for a supplied config value.
* never for a supplied config value. `alternative` is what the caller can do
* instead of creating one: the CLI's flag by default.
*/
export function missingConfigError(cwd: string): ConfigurationError {
export function missingConfigError(cwd: string, alternative = 'pass --config <path>'): ConfigurationError {
const root = path.resolve(cwd);
const lookalike = CONFIG_LOOKALIKES.find((name) => existsSync(path.join(root, name)));
const names = `${CONFIG_NAMES.slice(0, -1).join(', ')}, or ${CONFIG_NAMES.at(-1)}`;
const searched = `no ${names} found in ${root} or its parent directories`;
const remedy =
lookalike === undefined
? 'run e2e init to create one, or pass --config <path>'
? `run e2e init to create one, or ${alternative}`
: `found ${lookalike}, but only ${names} is loaded: rename it and keep it an ES module`;
Comment thread
KrzysztofMoch marked this conversation as resolved.
Resolved
return new ConfigurationError('CONFIG_NOT_FOUND', `${searched}; ${remedy}`);
}
Expand Down
42 changes: 40 additions & 2 deletions packages/e2e/src/mcp/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,61 @@
import path from 'node:path';
import { discoverConfig, loadConfigModule, missingConfigError } from '../config/load.ts';
import { resolveConfig, type ResolvedConfig } from '../config/resolve.ts';
import { ConfigurationError } from '../internal/errors.ts';
import type { TargetApp } from '../types.ts';
import { loadEngine, type EngineLookup, type FactoryRef } from './default-engine.ts';

export interface ProjectOptions {
readonly cwd: string;
/** An explicit config file, relative to `cwd`; otherwise the nearest config file (`e2e.config.ts`, `.mts`, `.js`, or `.mjs`) from `cwd` upward. */
readonly configPath?: string | undefined;
}

/** A config loaded from a file, which a session always has: the file is what sessions share. */
/** A config loaded from a file: what sessions on a project share. A session opened with `engine` has none. */
export type LoadedConfig = ResolvedConfig & { readonly configPath: string };

/** The absolute path of a project's config, found without evaluating it. */
export function locateProjectConfig(options: ProjectOptions): string {
const { configPath } = discoverConfig(options.cwd, options.configPath);
if (configPath === undefined) throw missingConfigError(options.cwd);
if (configPath === undefined) {
throw missingConfigError(
options.cwd,
'pass engine to open_session to open a session without one: {engine: "@e2e-dev/web", app: {url: "<app URL>"}}, or {engine: {package: "@e2e-dev/mobile", options: {platform: "ios"}}, app: {bundleId: "<app>"}}',
);
}
return configPath;
}

/** What `open_session` names to open a session where the project has no config file: the engine's factory call, and the app. */
export interface EngineSession extends FactoryRef {
/** The target's app: the URL it opens, or the installed app or build a device launches. */
readonly app?: Pick<TargetApp, 'url' | 'bundleId' | 'appPath'> | undefined;
}

/**
* The config of a session opened with `engine`: one target on that engine
* package, with the app the call names and every other setting at its
* default. The project root is `cwd`. Refused where a config file exists,
* since that file is what a session on the project loads.
*/
export async function loadEngineConfig(
options: ProjectOptions & { readonly env: NodeJS.ProcessEnv } & Pick<EngineLookup, 'enginesDir'>,
session: EngineSession,
): Promise<ResolvedConfig> {
const { configPath } = discoverConfig(options.cwd, options.configPath);
if (configPath !== undefined) {
throw new ConfigurationError(
'INVALID_ARGUMENT',
`open_session: engine opens a session where there is no config file, and ${configPath} exists; leave engine out to open one of its targets`,
);
}
const engine = await loadEngine(session, { cwd: options.cwd, enginesDir: options.enginesDir });
return resolveConfig(
{ targets: [{ engine, ...(session.app === undefined ? {} : { app: session.app }) }] },
{ projectRoot: path.resolve(options.cwd), env: options.env },
);
}

/**
* Loads the config at `configPath` (absolute, from `locateProjectConfig`)
* fresh, down to the files it imports by path: an agent edits the config, a
Expand Down
Loading
Loading