Skip to content

fix(web): launch Chromium with its sandbox on by default - #1621

Open
Varshith-Kali wants to merge 11 commits into
e2e-inc:mainfrom
Varshith-Kali:fix-1613-chromium-sandbox
Open

Varshith-Kali wants to merge 11 commits into
e2e-inc:mainfrom
Varshith-Kali:fix-1613-chromium-sandbox

Conversation

@Varshith-Kali

@Varshith-Kali Varshith-Kali commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Issue

Fixes #1613.

  • @e2e-dev/web launched Chromium without chromiumSandbox, so Playwright added --no-sandbox to every browser process.
  • The agent can follow the app to another origin, so a renderer exploit on a page it reaches ran unsandboxed as the user running e2e run, with the checkout, .env, and model keys in reach.
  • There was no config option to turn the sandbox on.

What changed

  • Chromium launches with chromiumSandbox: true by default. Firefox and WebKit never receive the option.
  • web({ chromiumSandbox: false }) opts out, as does E2E_CHROMIUM_SANDBOX=0 in the run's environment. An explicit option wins over the variable.
  • The flag resolves from the option, then the run's environment, then on. The process environment never decides a launch, per the EngineInitInfo.env contract, so a host that exports the variable in its shell cannot silently turn the sandbox off for an in-process run.
  • BrowserConnection.acquire takes the resolved flag as a required argument; the two integration tests that launch a browser directly pass false explicitly, the way they ran before.
  • Docs updated next to the no-origin-allowlist note (docs/reference/web.mdx, docs/security.mdx, SECURITY.md, skills/e2e/references/setup.md), with a .changeset entry (@e2e-dev/web, patch).
  • CI opts out explicitly (E2E_CHROMIUM_SANDBOX: '0' in spec.yml, benchmark.yml, agent.yml, examples.yml): the ubuntu-2404 runners restrict unprivileged user namespaces, so a sandboxed launch fails there. Tests that launch a real browser forward the flag into the run's environment instead of reading the process environment.
  • The smol VM's --no-sandbox is unchanged: the VM is the boundary there, and the guest browser runs inside it.

Limits

  • The sandbox contains the renderer; it does not fix the known Chromium CVEs by itself. Playwright 1.65.0 and Chromium 158 cover those through chore(deps): bump workspace deps to latest, playwright-core to 1.65.0 #1609.
  • Test, config, and engine code still run with the runner's full OS authority; nothing here sandboxes them.
  • On a host where the sandbox cannot start, leaving it on fails the launch with Playwright's "Chromium sandboxing failed!" error; set false or E2E_CHROMIUM_SANDBOX=0 there.

Verified

Ran it locally: yes

  • pnpm check: green (lint, dead-code, typecheck, error codes, peer ranges, changeset bumps, plugins, mcp-server, docs).
  • pnpm --filter @e2e-dev/web exec vitest run --project unit: 354 passed.
  • pnpm --filter @e2e-dev/web exec vitest run --project integration with E2E_CHROMIUM_SANDBOX=0: 379 passed, 3 failed, 1 skipped. The 3 failures are the load-sensitive app-log, provider-downloads, and semantic-fallback tests; each passes in isolation, and all three fail intermittently under parallel load, including on a main checkout in this environment.
  • Real CLI on the testbed (apps/testbed, node node_modules/e2e/dist/cli/bin.js run tests/browser.e2e.ts):
    • default: 6 failed, every one with "Chromium sandboxing failed!"
    • E2E_CHROMIUM_SANDBOX=0: 6 passed
  • pnpm --filter e2e exec vitest run tests/integration/mcp-test-run.test.ts with E2E_CHROMIUM_SANDBOX=0: 13 passed.
main this branch
testbed browser suite passes: the launch is unsandboxed by default, the same launch the opt-out run exercises (6 passed) testbed browser suite fails by default (6 failed, "Chromium sandboxing failed!") and passes with the opt-out (6 passed)
new sandbox unit tests fail: no such option (6 of 7 failed before the fix) 13 new tests pass
web integration suite: same 3 load flakes on a main checkout same 3 load flakes, no new failures

View guided diff

@github-actions github-actions Bot added the size:L Up to 499 changed lines, not counting tests, fixtures, or the lockfile label Nov 7, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 48 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread SECURITY.md Resolved
Comment thread docs/reference/web.mdx Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 5 files (changes from recent commits).

View guided diff | Re-trigger cubic

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

CI on 869b879: the check job failed on one testbed web test, tests/screenshot.e2e.ts > page screenshot > should work while navigating (ENGINE_FAILURE: screenshot failed because the execution context was destroyed by a navigation; 508 of 511 passed). This head is docs-only (SECURITY.md, reference page, skill reference, one JSDoc comment), so it cannot cause a screenshot race; it reads as a timing flake in that navigation test. A re-run of the check job should clear it.

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review (agent-run triage).

Thanks, the code is correct and well tested:

  • One local launch path (browser-connection.ts:85-92), CDP attach/provider leases/smol take connect() and are untouched, Firefox/WebKit untouched.
  • Flag read from the run env (info.env), never process.env; option > env > default on.
  • Real launch on macOS: default has no --no-sandbox, E2E_CHROMIUM_SANDBOX=0 adds it. 50 integration tests pass sandboxed, 338 web unit tests pass.
  • 11 of 15 mutations killed (default flip, env drop, process.env read, precedence swap, chromium-only guard, validation, init resolution).

The blocker is what this does to users on Linux.

Must fix

  1. Docker/Linux CI users break after a patch bump. In the official mcr.microsoft.com/playwright:v1.65.0-noble image, a sandboxed launch fails both as root and as pwuser ("Chromium sandboxing failed!"); unsandboxed works. Same on ubuntu-2404 runners (the PR's own CI opt-outs show it), likely on most Linux containers. The changeset is a patch and says nothing about it. At minimum a minor bump with a line telling users to set E2E_CHROMIUM_SANDBOX=0.
  2. The failure is a raw Playwright error. It surfaces as ENGINE_FAILURE: browser launch failed: browserType.launch: Target page, context or browser has been closed ... Chromium sandboxing failed! plus a ~40-flag call log, and only hints at a chromiumSandbox: false option, never E2E_CHROMIUM_SANDBOX=0, which is what a CI user can actually use. Catch that one failure in acquireBrowser and throw an e2e error naming both opt-outs, without the call log.
  3. CI docs not updated. docs/ci/github-actions.mdx, the other docs/ci/*.mdx and docs/visual-testing.mdx don't mention the opt-out, and nobody has checked whether the documented ubuntu-latest recipe still launches.

Should fix (structure)

  • Parsing is split: readChromiumSandboxFlag (browser-connection.ts:40) is exported only to return a lowercased string, and surface.ts:293-300 decides what it means. One resolveChromiumSandbox(configured, env): boolean in one place, drop the export.
  • configuredChromiumSandbox plus a mutable chromiumSandbox = true (surface.ts:312-315): one field would do; the pre-init true is untested (mutation to false survives).
  • acquire(name, headed, timeoutMs, chromiumSandbox, connect?): four primitives in a row, switch to an options object.
  • surface.ts is already 1.3k lines; ~15 of the 43 added lines are a JSDoc repeating the paragraph that also appears in web.mdx, both setup.md copies, SECURITY.md, security.mdx and the changeset. One or two lines.

Nits

  • cdp-attach test edits are pointless (flag doesn't apply to attach). screenshot-caret and stalled-frame hardcode false, so they never run sandboxed even on macOS.
  • Shrink each 4-line workflow comment to one line; drop the comment lines in mcp-test-run.test.ts and artifact-path.test.ts.
  • Surviving mutations: .trim() / .toLowerCase() dropped, retry recursion passing !chromiumSandbox.
  • Docs explain the "why" in six places; keep it in security.mdx, link from the rest.
  • Red CI is screenshot.e2e.ts > should work while navigating, a known flake unrelated to this.

For maintainers: default policy. Always on (this PR), or on for macOS/Windows and off on Linux, which is what Playwright MCP does for the same agent-browsing threat model (bundled Chromium on Linux defaults to sandbox off). That keeps dev laptops (where the keys live) protected and drops the workflow edits, most of the test churn and the Docker breakage. Also: patch vs minor, whether the env var is worth it, smol's VM --no-sandbox stays.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. The screenshot.e2e.ts failure is the known flake, confirmed unrelated: the 869b879 head is docs-only, and the same test passed on the earlier head. On the must-fix items: I will add the friendly launch-failure error naming both opt-outs, update the CI docs (github-actions, the other ci pages, visual-testing) and verify the ubuntu-latest recipe, and bump the changeset to minor with a line telling Linux and Docker users to set E2E_CHROMIUM_SANDBOX=0. I will also do the should-fix restructuring (single resolveChromiumSandbox, one field, options object for acquire, shorter JSDoc) and the nits. The default-policy question (always on, or on for macOS and Windows and off on Linux) is yours to decide; I am keeping the always-on default unless you say otherwise, since the must-fix items assume it.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

All addressed in f9147f3 (two commits on the PR branch):

  • Must-fix 1: changeset is now minor (with the Release: opening the bump check requires) and tells Linux and Docker users to set E2E_CHROMIUM_SANDBOX=0.
  • Must-fix 2: acquireBrowser now catches a local-launch failure containing 'Chromium sandboxing failed!' and throws an ENGINE_FAILURE naming both opt-outs, with no call log and no cause attached (the runner would surface the call log back through the cause chain). Other launch failures keep the old wrapping.
  • Must-fix 3: docs/ci/github-actions.mdx (web job) and docs/visual-testing.mdx (screenshots job) now set E2E_CHROMIUM_SANDBOX: '0' on ubuntu-latest, matching this repo's own CI. The other docs/ci pages run mobile targets only, so there is nothing to add there.
  • Should-fix: one resolveChromiumSandbox(configured, env) in browser-connection.ts, readChromiumSandboxFlag export deleted; one chromiumSandbox field; acquire takes a BrowserAcquireOptions object; the JSDoc is two lines linking to /security. New tests pin .trim()/.toLowerCase() and the opt-out surviving a failed relaunch.
  • Nits: cdp-attach edits reverted, screenshot-caret and stalled-frame now run sandboxed by default, workflow comments shrunk to one line, comment lines dropped in mcp-test-run and artifact-path tests, and the 'why' lives in security.mdx only with links from the rest.
  • Verification: web unit suite 358/358 with E2E_CHROMIUM_SANDBOX=0; chromium-sandbox 8, chromium-sandbox-surface 9, browser-connection 2, surface-lifecycle 8; tsc clean for web and e2e, oxlint clean, docs check chain clean, plugin check clean. Without the opt-out on this Linux host the suite fails with the new friendly error, as expected (this host cannot sandbox).
  • Left for you: the default-policy question (always on, or on for macOS and Windows and off on Linux). The default is unchanged.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 21 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="skills/e2e/references/setup.md">

<violation number="1" location="skills/e2e/references/setup.md:223">
P2: This link targets `/docs/security`, which is neither the published `/security` page nor its `/docs/security.md` Markdown endpoint. Link to `https://e2e.com/security` so readers reach the guide.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

View guided diff | Re-trigger cubic

Comment thread docs/visual-testing.mdx Resolved
| `colorScheme`, `reducedMotion` | The `prefers-color-scheme` (`'light'`, `'dark'`, `'no-preference'`) and `prefers-reduced-motion` (`'reduce'`, `'no-preference'`) every page reports to its `@media` rules and `matchMedia`, from first paint. |
| `initScripts` | Scripts every document runs before the page's own: source, `{ path }`, or a function with no closures. |
| `testIdAttribute` | What `getByTestId` reads; default `data-testid`. |
| `chromiumSandbox` | Whether a locally launched Chromium runs with its OS sandbox on; default `true` (see [Security](https://e2e.com/docs/security) for why it defaults on). A connected browser (`connect`) or a browser-provider browser is launched by its host, so the setting does not reach it. Set `false` only on a host where the sandbox cannot start (root, a container that blocks the required sandbox mechanisms, Ubuntu 23.10+ with the AppArmor user-namespace restriction). `E2E_CHROMIUM_SANDBOX=0` in the run's environment is the same opt-out, but only when the option is omitted; an explicit option wins. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This link targets /docs/security, which is neither the published /security page nor its /docs/security.md Markdown endpoint. Link to https://e2e.com/security so readers reach the guide.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At skills/e2e/references/setup.md, line 223:

<comment>This link targets `/docs/security`, which is neither the published `/security` page nor its `/docs/security.md` Markdown endpoint. Link to `https://e2e.com/security` so readers reach the guide.</comment>

<file context>
@@ -220,7 +220,7 @@ start a script that brings them up and serves the app.
 | `initScripts` | Scripts every document runs before the page's own: source, `{ path }`, or a function with no closures. |
 | `testIdAttribute` | What `getByTestId` reads; default `data-testid`. |
-| `chromiumSandbox` | Whether a locally launched Chromium runs with its OS sandbox on; default `true`. A connected browser (`connect`) or a browser-provider browser is launched by its host, so the setting does not reach it. Set `false` only on a host where the sandbox cannot start (root, a container that blocks the required sandbox mechanisms, Ubuntu 23.10+ with the AppArmor user-namespace restriction). `E2E_CHROMIUM_SANDBOX=0` in the run's environment is the same opt-out, but only when the option is omitted; an explicit option wins. |
+| `chromiumSandbox` | Whether a locally launched Chromium runs with its OS sandbox on; default `true` (see [Security](https://e2e.com/docs/security) for why it defaults on). A connected browser (`connect`) or a browser-provider browser is launched by its host, so the setting does not reach it. Set `false` only on a host where the sandbox cannot start (root, a container that blocks the required sandbox mechanisms, Ubuntu 23.10+ with the AppArmor user-namespace restriction). `E2E_CHROMIUM_SANDBOX=0` in the run's environment is the same opt-out, but only when the option is omitted; an explicit option wins. |
 | `screencast` | `{ size?, quality? }` for the engine's own video: frame size (default the viewport's), JPEG quality 0 to 100. |
 
</file context>
Suggested change
| `chromiumSandbox` | Whether a locally launched Chromium runs with its OS sandbox on; default `true` (see [Security](https://e2e.com/docs/security) for why it defaults on). A connected browser (`connect`) or a browser-provider browser is launched by its host, so the setting does not reach it. Set `false` only on a host where the sandbox cannot start (root, a container that blocks the required sandbox mechanisms, Ubuntu 23.10+ with the AppArmor user-namespace restriction). `E2E_CHROMIUM_SANDBOX=0` in the run's environment is the same opt-out, but only when the option is omitted; an explicit option wins. |
| `chromiumSandbox` | Whether a locally launched Chromium runs with its OS sandbox on; default `true` (see [Security](https://e2e.com/security) for why it defaults on). A connected browser (`connect`) or a browser-provider browser is launched by its host, so the setting does not reach it. Set `false` only on a host where the sandbox cannot start (root, a container that blocks the required sandbox mechanisms, Ubuntu 23.10+ with the AppArmor user-namespace restriction). `E2E_CHROMIUM_SANDBOX=0` in the run's environment is the same opt-out, but only when the option is omitted; an explicit option wins. |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this live: https://e2e.com/docs/security renders the security guide, while https://e2e.com/security returns 404. The docs site serves the page under /docs/security, so the current link is the working one. Leaving it as is.

@okwasniewski
okwasniewski dismissed their stale review November 7, 2026 22:14

Superseded by re-review of f9147f3 (automated review)

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude-opus-5-5] automated review on behalf of @okwasniewski

Re-review of f9147f3. The minor bump, the friendly launch error, the CI docs and the single resolveChromiumSandbox are all done. You were right about the /docs/security link: it returns 200. One small round left.

Must fix

  1. skills/e2e/references/running.md:230-247 (and the plugins/e2e copy): the skill's GitHub Actions recipe runs ubuntu-latest without E2E_CHROMIUM_SANDBOX: '0'. This is the workflow agents copy into user repos, so every web job built from it fails at launch once this ships. Add the env line there.

Should fix
2. Move the friendly-error mapping out of surface.ts (around line 424, with verb === 'launch' and a string match) and into BrowserConnection's local launch (browser-connection.ts around line 101). There name === 'chromium' && chromiumSandbox is already known, so you can throw the EngineError directly. The surface already rethrows EngineError as is. This shrinks surface.ts, which is already 1278 lines, and removes the untested verb guard. Removing that guard currently survives the tests.
3. The message can name an opt-out that does nothing. With web({ chromiumSandbox: true }), the option wins, so E2E_CHROMIUM_SANDBOX=0 changes nothing. Only name the env var when the option is unset, or say "unless the config sets chromiumSandbox".
4. chromiumSandbox on the surface holds the configured option before init and the resolved value after. The ?? true fallback in acquireBrowser is said never to run, but startAttempt without init reaches it. That is why artifact-path.test.ts needed an init call. Use a readonly option field plus private chromiumSandbox = true, the same pattern as headed.
5. Test gap: the relaunch recursion passing !chromiumSandbox still survives. The "relaunch fails while another caller waits" test only uses false and recurses twice, so the flip cancels itself out. A case that starts with true would kill it.

Nits

  • resolveChromiumSandbox: value === undefined || value === '' || is dead code, because value !== '0' && value !== 'false' already covers both.
  • docs/reference/web.mdx:113-118 still describes Playwright's "Chromium sandboxing failed!" error. Point it to the new e2e error and add a launch ENGINE_FAILURE row to the Throws table.
  • docs/visual-testing.mdx:71: limit "the Linux runners block" to ubuntu-latest, the same way the github-actions page does.
  • The acquire JSDoc repeats the BrowserAcquireOptions.chromiumSandbox field doc.

Mutation run: 15 of 21 mutants were killed. The real survivors are items 2, 4 and 5 above. The web suite passes 740 tests on macOS.

Maintainer decisions (not for you)

  • Always on, or off on Linux like Playwright MCP. Off on Linux would make item 1 and most of the workflow churn unnecessary.
  • **Release:** or **Breaking:** as the changeset opener.
  • Whether an explicit chromiumSandbox: true beats E2E_CHROMIUM_SANDBOX=0.

This was an automated review.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Review round addressed as 3a0ee17 (plus 2b24558 for the visual-testing nit below).

Must fix:

  1. The skill GitHub Actions recipe now sets E2E_CHROMIUM_SANDBOX: 0 in the run step env, in both skills/ and the synced plugins/ copy.

Should fix:
2. The friendly-error mapping moved out of surface.ts into BrowserConnection.launchLocal, where the browser name and resolved sandbox flag are known. The untested verb guard is gone; any other launch failure passes through unchanged.
3. The launch error names the environment opt-out only when the option did not set the value. When web({ chromiumSandbox: true }) set it, the message names only web({ chromiumSandbox: false }).
4. The surface keeps the option as a readonly field and resolves it into a private field defaulting to true, the same pattern as headed. The ?? true fallback in acquireBrowser is gone.
5. Added a relaunch test starting with true, so the flag flip no longer survives mutation, plus tests for the friendly error in both message variants and the passthrough.

Nits:

  • resolveChromiumSandbox lost its dead branches.
  • docs/reference/web.mdx now points at the new ENGINE_FAILURE instead of Playwright error text, with a launch row in the Throws table.
  • The acquire JSDoc no longer repeats the field doc.
  • The visual-testing nit (limit "the Linux runners block" to ubuntu-latest) is fixed as 2b24558, replied in its thread.

Verified: web unit suite 362/363 pass and tsc --noEmit plus oxlint are clean. The one failure is artifact-path.test.ts, which fails identically on the base head here because a real Chromium cannot start sandboxed in this container; CI is the check that matters for it.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 8 files (changes from recent commits).

View guided diff | Re-trigger cubic

@okwasniewski
okwasniewski dismissed their stale review November 8, 2026 04:14

Superseded by new review

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude-opus-5-5] automated review on behalf of @okwasniewski

Re-review of 3a0ee17. CR 2 asks 1-4 and every nit are done:

  • the opt-out is in both skill copies
  • the friendly error now lives in BrowserConnection.launchLocal
  • the message names the env var only when the option is unset
  • the readonly option field is in
    The precedence is option, then run env, then on, and it's tested. I reproduced the matcher's Playwright text with a real root launch in playwright:v1.63.0-noble. CI is green and there are no em dashes.

Should fix

  1. The relaunch flip survivor (ask 5) is still alive. The new test that starts with true recurses twice, so the flip cancels out. Add a case where a waiter calls acquire while the relaunch's pending is unresolved: the second launch mock rejects after setTimeout(20), the waiter joins after setTimeout(0), and you assert the sandbox flags are [true, true, true]. That kills the mutant.
  2. The friendly-error test should also assert retryable === false. A retryable: true mutant survives today.
  3. docs/reference/environment.mdx: E2E_CHROMIUM_SANDBOX is missing from the env var reference. Add a row: 0 turns the Chromium sandbox off unless web({ chromiumSandbox }) is set.
  4. skills/e2e/references/running.md:248 (and the plugins copy) and packages/web/src/surface.ts:176 link [Security](/security). That link is dead outside the docs site, and in the skill it sits in a YAML comment users copy. Use https://e2e.com/docs/security, as setup.md and SECURITY.md do.
  5. browser-connection.ts ~55-73 and 129: two parallel fields, chromiumSandbox and an optional chromiumSandboxConfigured?, hold one resolved value, and the surface always passes the second. Have resolveChromiumSandbox return { enabled, fromOption } and pass that one object. Also pass options to launchLocal instead of 5 positional args.

Nits

  • The changeset says the error names "both opt-outs". That's no longer always true.
  • The acquire JSDoc says "a relaunch keeps the value the first launch used", but BrowserConnection doesn't enforce that. It holds only because the surface fixes the value in init.

Open for @okwasniewski

  • Always on, or off on Linux like Playwright MCP?
  • Should an explicit chromiumSandbox: true beat E2E_CHROMIUM_SANDBOX=0?
  • **Release:** or **Breaking:** in the changeset?
  • Our CI sets the opt-out everywhere, so no job ever launches with the sandbox on. Do we want one sandbox-on smoke job?

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Done in de15327, all five asks plus the two nits.

  1. New test in chromium-sandbox.test.ts: a waiter joins a relaunch that is still in flight (second launch rejects after 20ms, waiter joins after 0ms), flags assert [true, true, true]. Verified it kills a flip injected on the re-acquire; the old test passes with the mutant, this one fails on it.
  2. The friendly-error test now asserts retryable is false.
  3. E2E_CHROMIUM_SANDBOX has a row in docs/reference/environment.mdx.
  4. The Security links in running.md (both copies) and surface.ts now point at https://e2e.com/docs/security.
  5. resolveChromiumSandbox returns { enabled, fromOption } and that one object goes through acquire into launchLocal, which now takes the options object instead of five positional args.

Nits: the changeset says the error names the opt-out that applies, and the acquire JSDoc states the actual guarantee (a launch takes the value its own call passes).

Left for you: always on vs off on Linux, whether explicit chromiumSandbox: true beats E2E_CHROMIUM_SANDBOX=0, Release: vs Breaking:, and the sandbox-on smoke job. Web unit suite 364/364 green with E2E_CHROMIUM_SANDBOX=0.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread docs/reference/environment.mdx Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

View guided diff | Re-trigger cubic

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

CI note on head 86c6cf8: the only failing job is check, and it fails on an infra step, not the change. The mcp-publisher download step got HTTP 504 from the modelcontextprotocol/registry release four times in a row, then exited 22. Lint, typecheck, dead code, error codes, peer ranges, changesets, examples, and the plugin and MCP registry checks all passed, and the test job is green. A re-run of the check job should clear it.

@okwasniewski

Copy link
Copy Markdown
Member

Automated review.

Re-review of 86c6cf8: all five CR 3 asks are done and verified. Labeling Ready for Human Review.

  • The new in-flight relaunch waiter test kills the flip mutant. Overall, mutation testing killed 15 of 17. The two that survived: retryable: true, which is equivalent because EngineError forces ENGINE_FAILURE to non-retryable, and the enabled guard before the friendly error, which is nearly unreachable.
  • https://e2e.com/docs/security resolves. The env row is there, and { enabled, fromOption } flows through acquire into launchLocal.
  • Web unit tests pass 364/364 with and without E2E_CHROMIUM_SANDBOX=0. The integration tests (lifecycle, app-log, init-scripts) pass 37/37 with the sandbox on (macOS).
  • The check job fails only on the mcp-publisher download, which gets a 504 from the registry. It failed again on rerun, so this is an upstream outage and not this PR.

Optional nits:

  • surface.ts:287-289: the placeholder { enabled: true, fromOption: false } that init overwrites. Make the resolved value optional, or resolve it in init.
  • browser-connection.ts:100-104: the acquire JSDoc repeats the options doc. :154-156: the two error strings share their first sentence, and only the remedy needs to differ.
  • chromium-sandbox.test.ts: the options literal is repeated about 20 times (a helper would cut that), the :90 test is mostly superseded by :114, and :240 should also assert code: 'INVALID_CONFIG'.
  • environment.mdx:176: say that false is accepted too.

For maintainers to decide:

  • Always on, or off by default on Linux.
  • Explicit chromiumSandbox wins over E2E_CHROMIUM_SANDBOX (current behavior). Keep it?
  • Changeset uses **Release:**, but launches can now fail on root, container, and Ubuntu 23.10+ hosts, which reads as **Breaking:** per CONTRIBUTING.
  • A Linux smoke job with the sandbox on (CI runs sandbox-off only).

@okwasniewski
okwasniewski dismissed their stale review November 8, 2026 10:13

CR 3 asks addressed in de15327/86c6cf80

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Thanks for the re-review, all five CR 3 asks verified and the Ready for Human Review label noted. The optional nits are left as-is for now: the placeholder in surface.ts, the JSDoc repeats, and the test-literal repetition are cosmetic, and I did not want another push to strip the label. Happy to fold any of them in if you would like them done, and the open questions (always-on vs off-on-Linux, explicit option vs env var, Release vs Breaking, a Linux sandbox-on smoke job) are yours to decide.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (d8b694f). New head fbf2a4f.

All five CR 3 asks remain addressed, and the rebase kept all eight commits. The only conflicts were mechanical: upstream additions alongside our change in packages/e2e/tests/integration/mcp-test-run.test.ts and docs/ci/github-actions.mdx, resolved by keeping both sides.

Verified locally: web typecheck and lint clean, web unit 364/364 with E2E_CHROMIUM_SANDBOX=0, and the e2e package typecheck clean. The one unit test that launches a real browser fails without the opt-out on a host whose sandbox cannot start, which is the friendly error this PR adds, not a regression.

Ready for human review.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Follow-up: the rebase dropped the first half of the new Cache the timings history section in docs/ci/github-actions.mdx, and CI caught it (docs anchor check: docs/ci.mdx links to #cache-the-timings-history). Restored in full as f30af13; the docs check chain passes locally now. Nothing else changed; the rebase note above still stands.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 52 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread docs/security.mdx Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

View guided diff | Re-trigger cubic

@okwasniewski

Copy link
Copy Markdown
Member

Re-checked after the rebase onto d8b694f. git range-diff against the labeled head 86c6cf8: conflicts were mechanical (mcp-test-run.test.ts around the new loadEngineConfig, docs/ci/github-actions.mdx). f30af13 restores the timings-history section, so docs/ci/github-actions.mdx now only adds the E2E_CHROMIUM_SANDBOX: '0' env line and its paragraph against main. 1807223's softer docs/security.mdx wording is accurate. CI green. Re-labeling Ready for Human Review.

Nit: docs/security.mdx:58 now runs past the 80-column wrap.

Maintainer decisions unchanged: default on Linux, option vs env precedence, Release vs Breaking, a sandbox-on smoke job.

This was an automated review.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Fixed the nit as ac85d7f: docs/security.mdx:58 now wraps at 80 columns, no wording change. All five docs gate scripts pass locally (anchors 1075/1075, examples, logos, MDX compile, package managers). The Ready for Human Review label will drop on this push as usual; the re-review, CI green, and the four maintainer decisions in your comment above are unchanged.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

View guided diff | Re-trigger cubic

@okwasniewski

Copy link
Copy Markdown
Member

[claude-opus-5-5] automated review on behalf of @okwasniewski

Re-checked at ac85d7f. The only change since the labeled head 1807223 is the rewrap of docs/security.mdx:58 at 80 columns, with no wording change. CI is green. Re-adding Ready for Human Review.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Thanks for re-checking at ac85d7f and re-adding Ready for Human Review. CI is green on this head and the docs wording is unchanged from the labeled head, so nothing further from my side. Awaiting human review.

@okwasniewski

Copy link
Copy Markdown
Member

[claude-opus-5-5] automated review on behalf of @okwasniewski

Re-checked 123efd8 (rebase onto 046bd08). Range-diff is mechanical: the conflicts with #1393 (credentialOrigins, app.origin) resolve right in engine.ts, surface.ts, the integration tests, and the unknown-key message. The SECURITY.md sandbox paragraph still reads fine above the new origin-scoping paragraph. Nothing new to ask. Relabeling Ready for Human Review.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Thanks for re-checking the rebase. The range-diff being mechanical and the SECURITY.md paragraph still reading fine above the new origin-scoping paragraph matches what I verified locally (web unit suite 50/50 and integration 64/64 with E2E_CHROMIUM_SANDBOX=0 on the rebased head). Nothing further from my side; awaiting human review.

The web engine launched Chromium without chromiumSandbox, so Playwright added --no-sandbox to every browser process. The agent can follow the app to another origin, so a renderer exploit on a page it reaches ran unsandboxed as the user running e2e run, with the checkout, .env, and model keys in reach.

Chromium now launches with chromiumSandbox: true unless the config opts out with web({ chromiumSandbox: false }) or the run environment sets E2E_CHROMIUM_SANDBOX=0, for hosts where the sandbox cannot start (root, containers, Ubuntu 23.10+ with the AppArmor user-namespace restriction). Firefox and WebKit never receive the option. The flag resolves from the option, then the run's environment, then on; the process environment never decides a launch, per the EngineInitInfo.env contract. CI opts out explicitly because the ubuntu-2404 runners restrict unprivileged user namespaces.

Docs updated next to the no-origin-allowlist note (docs/reference/web.mdx, docs/security.mdx, SECURITY.md, skills/e2e/references/setup.md) with a .changeset entry. The smol VM's --no-sandbox is unchanged: the VM is the boundary there.

Verified: pnpm check green; web unit 354/354; web integration 379 passed with 3 known load flakes that pass in isolation; testbed CLI 6/6 fail with "Chromium sandboxing failed!" by default and 6/6 pass with the opt-out on this host.

Fixes e2e-inc#1613

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Cubic review on the sandbox-default PR found three doc overstatements: SECURITY.md presented the sandbox as universal while connect() and browser providers bypass launch, the skill reference omitted that E2E_CHROMIUM_SANDBOX=0 only applies when the option is omitted, and the reference page implied any container cannot run the sandbox. All three are corrected in the SECURITY note, the reference page, the option JSDoc, and both skill copies.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
…ution

A sandboxed launch on a host whose sandbox cannot start failed with Playwright's raw error and its full call log, which only hints at the option. acquireBrowser now catches that one failure on a local launch and throws an ENGINE_FAILURE naming both opt-outs, E2E_CHROMIUM_SANDBOX=0 and web({ chromiumSandbox: false }), with no call log and no cause chain to leak it back.

The sandbox flag resolves in one place now: resolveChromiumSandbox(configured, env) in browser-connection.ts replaces the split readChromiumSandboxFlag plus surface resolver, and the surface keeps a single field that init resolves. BrowserConnection.acquire takes an options object, so a relaunch retries with the same options. The option JSDoc shrinks to two lines pointing at the security docs.

Tests follow the restructure: the cdp-attach edits are gone (the flag never applied to attach), screenshot-caret and stalled-frame run sandboxed by default, resolved from the ambient environment so the CI opt-out still reaches a direct launch, and the workflow comments shrink to one line. New tests pin the trim and lowercase handling, the opt-out surviving a failed relaunch, and the friendly error text.

Verified: web unit 358/358 with the opt-out; a real launch on this host fails with the friendly error by default and passes with E2E_CHROMIUM_SANDBOX=0.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
…or the why

The sandbox default breaks Docker and Linux CI hosts, and the changeset said nothing about it: it is now a minor bump, opened as a maintainer-called release, telling users to set E2E_CHROMIUM_SANDBOX=0.

The CI docs name the opt-out where CI users find it: the GitHub Actions web workflow and the visual-testing screenshots workflow both set E2E_CHROMIUM_SANDBOX=0 on ubuntu-latest, matching this repo's own CI, with a note pointing at the security docs. The other CI pages run mobile targets only, so there is nothing to add there.

The why now lives in docs/security.mdx alone: the reference page, SECURITY.md, the option JSDoc, and the skill reference link to it instead of repeating it.

Verified: changeset-bumps, the docs check chain (anchors, MDX, examples, package managers, logos), and the skill tests all pass.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The visual-testing guide said Linux runners generally block the user
namespaces Chromium needs. The limitation is specific to the
ubuntu-latest runners used here, so name them and tell readers to leave
the sandbox on where it starts.
Move the sandbox launch error out of the surface and into
BrowserConnection.launchLocal, where the browser name and the resolved
sandbox flag are already known, and drop the untested verb guard. The
error now names the environment opt-out only when the option did not set
the value, since an explicit option wins over the variable. The surface
keeps the option as a readonly field and resolves it into a private
field defaulting to true, the same pattern as headed, so startAttempt
without init no longer needs the fallback. The skill GitHub Actions
recipe runs ubuntu-latest with E2E_CHROMIUM_SANDBOX 0, the web reference
points at the new ENGINE_FAILURE instead of Playwright error text, and
the resolve helper loses its dead branches.

Web unit suite: 362/363 pass; the one failure is artifact-path, which
fails identically on the base head because a real Chromium cannot start
sandboxed in this container.
Resolve the sandbox into one { enabled, fromOption } object and pass it through acquire into launchLocal, which now takes the options object instead of five positional args, so the launch and its error cannot disagree about which opt-out set the value. Add the waiter-joins-a-pending-relaunch case, where the re-acquire runs once and a flip cannot cancel out, and assert retryable is false on the friendly error. Document E2E_CHROMIUM_SANDBOX in the environment reference, point the dead [Security](/security) links at https://e2e.com/docs/security, and fix the changeset wording, since the error no longer always names both opt-outs. The acquire JSDoc now states the actual guarantee: a launch takes the value its own call passes.

Web unit suite: 364/364 pass with E2E_CHROMIUM_SANDBOX=0 (artifact-path fails without it, identically on the base head, because a real Chromium cannot start sandboxed in this container). Web typecheck clean, oxlint clean. The new waiter test fails against a flip mutant injected on the re-acquire and passes without it.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The entry was marked flag, but on this page 0 and false mean unset for flags, while 0 opts out of sandboxing here. Mark it value so readers do not apply the wrong flag semantics.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The rebase conflict resolution in docs/ci/github-actions.mdx kept only the tail of the new Cache the timings history section, dropping its heading, intro, and example, which broke the docs anchor check (docs/ci.mdx links to #cache-the-timings-history). Restored in full; the sandbox opt-out paragraph follows it unchanged.

Verified: the docs check chain passes locally (anchors, examples, package managers, logos, MDX).

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The security page stated a renderer exploit stays contained by the
sandbox, which reads as a guarantee. Say the sandbox helps contain it
and note it does not prevent sandbox escapes or Chromium
vulnerabilities, matching the review feedback on the Chromium sandbox
default.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
@okwasniewski

Copy link
Copy Markdown
Member

[claude-opus-5-5] automated review on behalf of @okwasniewski

Re-checked 491dba3 (rebase onto 5ff7481). The range-diff against the previous head d8fb454 is mechanical. The same 11 commits are there, 9 identical, and the other 2 differ only in context lines from main (#1634 CdpTarget type tests, ConnectOverCDPOptions import). Nothing changed in src, tests or docs. docs/ci/github-actions.mdx keeps main's timings-history section intact and only adds the sandbox env + paragraph. Merges clean, CI green. Relabeling.

@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Thanks for re-checking the rebase. The range-diff being mechanical and nothing changed in src, tests or docs matches what I verified locally. Nothing further from my side; awaiting human review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:L Up to 499 changed lines, not counting tests, fixtures, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

web: launch Chromium with the sandbox on (agent can navigate off-origin)

2 participants