Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/chromium-sandbox-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@e2e-dev/web": minor
---

**Release:** Chromium launches with its OS sandbox on by default, so a renderer exploit on a page the agent reaches stays contained instead of running as the user running `e2e run`. On a host where the sandbox cannot start (root, a container, or Ubuntu 23.10+ with the AppArmor user-namespace restriction), set `E2E_CHROMIUM_SANDBOX=0` in the run environment, or pass `web({ chromiumSandbox: false })`; otherwise the launch fails with a friendly error naming the opt-out that applies.
2 changes: 2 additions & 0 deletions .github/workflows/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ on:
# The repository's own runs are not usage: the CLI sends no telemetry from CI here.
env:
E2E_TELEMETRY_DISABLED: '1'
# The ubuntu-2404 runners block the user namespaces Chromium's sandbox needs, so CI opts out explicitly.
E2E_CHROMIUM_SANDBOX: '0'

permissions:
contents: read
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ on:
# The repository's own runs are not usage: the CLI sends no telemetry from CI here.
env:
E2E_TELEMETRY_DISABLED: '1'
# The ubuntu-2404 runners block the user namespaces Chromium's sandbox needs, so CI opts out explicitly.
E2E_CHROMIUM_SANDBOX: '0'

permissions:
contents: read
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ on:
env:
# The repository's own runs are not usage: the CLI sends no telemetry from CI here.
E2E_TELEMETRY_DISABLED: '1'
# The ubuntu-2404 runners block the user namespaces Chromium's sandbox needs, so CI opts out explicitly.
E2E_CHROMIUM_SANDBOX: '0'

permissions:
contents: read
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ on:
# The repository's own runs are not usage: the CLI sends no telemetry from CI here.
env:
E2E_TELEMETRY_DISABLED: '1'
# The ubuntu-2404 runners block the user namespaces Chromium's sandbox needs, so CI opts out explicitly.
E2E_CHROMIUM_SANDBOX: '0'

permissions:
contents: read
Expand Down
7 changes: 6 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,12 @@ against a model mistake at the cost of configuring every sign-in flow that
leaves the app's domain. Treat a target whose app can send the agent
elsewhere as one the agent may follow there, with the secrets the step was
given. If a threat model ever calls for an origin allowlist again, it comes
back as an opt-in.
back as an opt-in. Chromium launches with its OS sandbox on (see the
[security docs](https://e2e.com/docs/security) for why); `web({ chromiumSandbox: false })`
or `E2E_CHROMIUM_SANDBOX=0` turns it off on a host where the sandbox cannot
start. That covers a locally launched Chromium only: a connected browser
(`web({ connect })`) or a browser-provider browser is launched by its host,
so this setting does not reach it.

Child frames off the site of the target's `url`, its registrable domain, are
dropped from observations. The browser engine's `headers` and `basicAuth`
Expand Down
6 changes: 6 additions & 0 deletions docs/ci/github-actions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ jobs:
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
E2E_USER_ADMIN_USERNAME: ${{ secrets.E2E_USER_ADMIN_USERNAME }}
E2E_USER_ADMIN_PASSWORD: ${{ secrets.E2E_USER_ADMIN_PASSWORD }}
E2E_CHROMIUM_SANDBOX: '0'

- name: Upload report
if: ${{ !cancelled() }}
Expand Down Expand Up @@ -125,6 +126,11 @@ The step saves after the job passes. To save after a failed run too, use
[`workers`](/reference/config#workers) of 2 or more, and the workflow above
runs one.

Chromium launches with its sandbox on by default, and the `ubuntu-latest`
runners block the user namespaces it needs, so the workflow opts out with
`E2E_CHROMIUM_SANDBOX: '0'`. On a runner where the sandbox starts, leave it
out. See [Security](/security) for why it defaults on.

## iOS and Android

This workflow runs one job for each platform and follows the rules in
Expand Down
1 change: 1 addition & 0 deletions docs/reference/environment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ What is sent and how to read it is on the [Telemetry](/telemetry) page.
| --- | --- | --- | --- |
| `PLAYWRIGHT_BROWSERS_PATH` | `PLAYWRIGHT_BROWSERS_PATH=~/.cache/ms-playwright` | value | Playwright's browser cache, honored by the engine's `prepare` hook |
| `PLAYWRIGHT_SKIP_BROWSER_GC` | `PLAYWRIGHT_SKIP_BROWSER_GC=0` | value | Defaults to `1` for the engine's browser install, which keeps other browsers in the cache. `0` lets the install remove them |
| `E2E_CHROMIUM_SANDBOX` | `E2E_CHROMIUM_SANDBOX=0` | value | `0` turns the Chromium sandbox off unless `web({ chromiumSandbox })` is set |
| `KERNEL_API_KEY` | `KERNEL_API_KEY=<key>` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with. Unset, the browser lease fails. See [Kernel](/integrations/kernel) |
| `LIGHTPANDA_PATH` | `LIGHTPANDA_PATH=/opt/lightpanda` | value | The binary `lightpanda()` from `@e2e-dev/lightpanda` starts a CDP server with. Unset, `lightpanda` on `PATH`, `~/.lightpanda`, or `~/.local/bin`. See [Lightpanda](/integrations/lightpanda) |
| [`EXPO_TOKEN`](#expo_token) | `EXPO_TOKEN=<token>` | value | The Expo access token `expoSimulators()` from `@e2e-dev/eas` uses |
Expand Down
26 changes: 26 additions & 0 deletions docs/reference/web.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ web({ viewport: { width: 390, height: 844 }, locale: 'de-DE' });
| Option | Type | Default |
| --- | --- | --- |
| [`browser`](#browser) | `BrowserName \| BrowserProvider` | `'chromium'` |
| [`chromiumSandbox`](#chromiumsandbox) | `boolean` | `true` |
| [`viewport`](#viewport) | `ViewportSize \| null` | 1280 by 720 |
| [`screencast`](#screencast) | `WebScreencastOptions` | the viewport's size |
| [`connect`](#connect) | `WebConnectOptions` | launches locally |
Expand Down Expand Up @@ -98,6 +99,31 @@ web({ browser: 'firefox' }); // default: 'chromium'
| --- | --- |
| `INVALID_CONFIG` | `browser: provider` together with `connect`. |

#### `chromiumSandbox`

Whether a locally launched Chromium runs with its OS sandbox on. A
connected browser (`connect`) or a browser-provider browser is launched by
its host, so the setting does not reach it.

```ts
web({ chromiumSandbox: false }); // default: true
```

- The agent can follow the app to another origin; see [Security](/security)
for why the sandbox defaults on.
- Set `false` only on a host where the sandbox cannot start (root, a
container that blocks the required sandbox mechanisms, or Ubuntu 23.10+
with `kernel.apparmor_restrict_unprivileged_userns=1`):
leaving it on there fails the launch with an `ENGINE_FAILURE` naming
the opt-outs that apply. `E2E_CHROMIUM_SANDBOX=0` in the environment is the same
opt-out; an explicit option wins over the variable.
- Firefox and WebKit ignore it.

| Throws | When |
| --- | --- |
| `INVALID_CONFIG` | Any value that is not a boolean, at config load. |
| `ENGINE_FAILURE` | The launch fails because the sandbox cannot start on this host. |

#### `viewport`

The initial viewport of every attempt's page.
Expand Down
7 changes: 6 additions & 1 deletion docs/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,12 @@ The model acts only through tools. The runner checks each call against a
closed schema before it runs, and never evaluates model text as code. A call
that breaks a rule, such as a navigation to `file:`, fails with
`POLICY_DENIED`. There is no origin allowlist: the agent can follow the app
to another site and use the step's secrets there.
to another site and use the step's secrets there. Chromium launches with its
OS sandbox on, which helps contain a renderer exploit on a page the agent
reaches instead of letting it run as the user running `e2e run`, but it does
not prevent sandbox escapes or Chromium vulnerabilities;
`web({ chromiumSandbox: false })` or `E2E_CHROMIUM_SANDBOX=0` turns it off on
a host where the sandbox cannot start.

## What runs with your permissions

Expand Down
7 changes: 6 additions & 1 deletion docs/visual-testing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,10 @@ bunx e2e run --update-snapshots

Take the screenshots on the machine that renders them in CI: a Linux
runner for the web and Android, a macOS runner with the same simulator for
iOS.
iOS. Chromium launches with its sandbox on by default, and the `ubuntu-latest`
runners block the user namespaces it needs, so the job opts out with
`E2E_CHROMIUM_SANDBOX: '0'`. On a runner where the sandbox starts, leave it
out; see [Security](/security) for why it defaults on.

Without `--update-snapshots`, a CI run never writes into the project. A
missing screenshot fails the test, and the run saves it in the test's
Expand All @@ -85,6 +88,8 @@ jobs:
steps:
# the same checkout, setup, and install steps as the test job
- run: npx e2e run --update-snapshots
env:
E2E_CHROMIUM_SANDBOX: '0'
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: screenshots
Expand Down
2 changes: 1 addition & 1 deletion packages/e2e/tests/integration/mcp-test-run.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ describe('open_session {test}', { timeout: 120_000 }, () => {
configPath,
}),
loadEngineConfig: () => Promise.reject(new Error('these sessions open on the config')),
env: {},
env: { E2E_CHROMIUM_SANDBOX: process.env['E2E_CHROMIUM_SANDBOX'] },
headed: false,
log: () => undefined,
});
Expand Down
104 changes: 93 additions & 11 deletions packages/web/src/browser-connection.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
/** Shares one browser process connection across a worker's attempts. */

import type { Browser, BrowserType, ConnectOverCDPOptions } from 'playwright-core';
import { EngineError } from 'e2e/engine';
import { SELECTOR_ENGINES } from './selector-engines.ts';
import { message } from './support.ts';

/**
* Playwright, imported the first time a browser is looked up, launched, or
Expand Down Expand Up @@ -38,6 +40,54 @@ export function isBrowserName(name: string): name is BrowserName {
*/
const RUNNER_OWNS_SIGNALS = { handleSIGINT: false, handleSIGTERM: false } as const;

/**
* Whether a locally launched Chromium runs with its sandbox on, and where
* the value came from. One object carries both, so the launch and its
* error cannot disagree about which opt-out set the value.
*/
export interface ChromiumSandbox {
/** Whether a locally launched Chromium runs with its OS sandbox on. */
readonly enabled: boolean;
/**
* Whether the value came from the `web({ chromiumSandbox })` option
* rather than the run's environment. The launch error names the
* environment opt-out only when the option did not set the value, since
* an explicit option wins over the variable.
*/
readonly fromOption: boolean;
}

/**
* Whether a locally launched Chromium runs with its sandbox on: an explicit
* option wins, then the run's environment, then on. `0` and `false` turn it
* off, the same way `CI` is read; anything else leaves it on. The run's
* environment is the only environment read here: an in-process worker shares
* its process with the runner and every other target, so the process
* environment never decides a launch.
*/
export function resolveChromiumSandbox(
configured: boolean | undefined,
env: Readonly<Record<string, string | undefined>>,
): ChromiumSandbox {
if (configured !== undefined) return { enabled: configured, fromOption: true };
const value = env['E2E_CHROMIUM_SANDBOX']?.trim().toLowerCase();
return { enabled: value !== '0' && value !== 'false', fromOption: false };
}

/** What `BrowserConnection.acquire` launches or attaches to. */
export interface BrowserAcquireOptions {
/** Which browser to launch. */
readonly name: BrowserName;
/** Whether the browser opens headed. */
readonly headed: boolean;
/** How long a launch or attach may take. */
readonly timeoutMs: number;
/** Whether a Chromium launch keeps its OS sandbox on, and where the value came from; other browsers ignore it. */
readonly chromiumSandbox: ChromiumSandbox;
/** Attaches to a remote browser instead of launching one, when given. */
readonly connect?: () => Promise<Browser>;
}

/**
* Launching a browser process costs hundreds of milliseconds; per-attempt
* isolation lives in browser contexts, so every attempt a worker runs can
Expand All @@ -53,28 +103,26 @@ export class BrowserConnection {
* When `connect` is given the connection attaches to a remote browser through it
* instead of launching a local one; `connect` is re-invoked on a relaunch,
* so a caller that resolves a fresh per-run endpoint reconnects cleanly
* after a dropped session.
* after a dropped session. The caller passes the resolved sandbox value:
* the `web({ chromiumSandbox })` option, the run's environment, or on.
* Only a Chromium launch takes it, and a launch takes the value its own
* call passes, so a caller that passes the same value on every call keeps
* it across relaunches.
*/
async acquire(
name: BrowserName,
headed: boolean,
timeoutMs: number,
connect?: () => Promise<Browser>,
): Promise<Browser> {
async acquire(options: BrowserAcquireOptions): Promise<Browser> {
const { connect } = options;
const cached = this.pending;
if (cached !== undefined) {
const browser = await cached.catch(() => null);
if (browser !== null && browser.isConnected()) return browser;
if (this.pending !== cached) return this.acquire(name, headed, timeoutMs, connect);
if (this.pending !== cached) return this.acquire(options);
this.pending = undefined;
}
// Set before anything awaits: the first call loads Playwright, and a `dispose`
// that arrives meanwhile must close the browser this launch produces, while a
// second caller waits for it instead of launching another.
const launching = registerSelectorEngines().then(() =>
connect !== undefined
? connect()
: browserType(name).then((type) => type.launch({ headless: !headed, timeout: timeoutMs, ...RUNNER_OWNS_SIGNALS })),
connect !== undefined ? connect() : this.launchLocal(options),
);
this.pending = launching;
try {
Expand All @@ -85,6 +133,40 @@ export class BrowserConnection {
}
}

/**
* Launches a local browser. A Chromium launch whose sandbox cannot start
* fails with Playwright's "Chromium sandboxing failed!" error and the
* whole launch call log; the engine error names the opt-outs that apply
* instead. The environment opt-out is named only when the option did not
* set the value, since an explicit option wins over the variable. Any
* other launch failure passes through unchanged.
*/
private async launchLocal(options: BrowserAcquireOptions): Promise<Browser> {
const { name, headed, timeoutMs, chromiumSandbox } = options;
const type = await browserType(name);
try {
return await type.launch({
headless: !headed,
timeout: timeoutMs,
...RUNNER_OWNS_SIGNALS,
...(name === 'chromium' ? { chromiumSandbox: chromiumSandbox.enabled } : {}),
});
} catch (cause) {
if (name === 'chromium' && chromiumSandbox.enabled && message(cause).includes('Chromium sandboxing failed!')) {
// No `cause`: the runner follows the chain into the surfaced
// message, which would put the call log back.
throw new EngineError(
'ENGINE_FAILURE',
chromiumSandbox.fromOption
? 'Chromium failed to launch because its sandbox cannot start on this host. Pass web({ chromiumSandbox: false }) to launch without the sandbox.'
: 'Chromium failed to launch because its sandbox cannot start on this host. Set E2E_CHROMIUM_SANDBOX=0 in the run environment, or pass web({ chromiumSandbox: false }), to launch without the sandbox.',
{ retryable: false },
);
}
throw cause;
}
}

/** Closes the connection, including a launch still in flight; idempotent and best-effort. */
async dispose(): Promise<void> {
const pending = this.pending;
Expand Down
12 changes: 12 additions & 0 deletions packages/web/src/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ export function web(options: WebOptions = {}): EngineHandle {
if (options.basicAuth !== undefined) validateBasicAuth(options.basicAuth);
if (options.credentialOrigins !== undefined) validateCredentialOrigins(options);
if (options.testIdAttribute !== undefined) validateTestIdAttribute(options.testIdAttribute);
if (options.chromiumSandbox !== undefined) validateChromiumSandbox(options.chromiumSandbox);
if (options.userAgent !== undefined) validateUserAgent(options.userAgent, options.headers);
if (options.locale !== undefined) validateLocale(options.locale, options.headers);
if (options.timezoneId !== undefined) validateTimezoneId(options.timezoneId);
Expand Down Expand Up @@ -167,6 +168,7 @@ export function web(options: WebOptions = {}): EngineHandle {
/** Every option `web()` takes, kept equal to `WebOptions` by the compiler. */
const WEB_OPTION_KEYS: readonly string[] = Object.keys({
browser: true,
chromiumSandbox: true,
viewport: true,
screencast: true,
connect: true,
Expand Down Expand Up @@ -428,6 +430,16 @@ function validateTestIdAttribute(attribute: unknown): void {
}
}

/** Refuses a sandbox opt-out that is not a boolean, at config load. */
function validateChromiumSandbox(chromiumSandbox: unknown): void {
if (typeof chromiumSandbox !== 'boolean') {
throw new ConfigurationError(
'INVALID_CONFIG',
`web({ chromiumSandbox }) must be a boolean, got ${JSON.stringify(chromiumSandbox)}`,
);
}
}

/**
* A browser target opens a URL: without `app.url` there is nothing for
* `app.open()` to open, and the fields that describe an installed device app
Expand Down
Loading
Loading