Repository navigation
feat(mcp): open a session without a config by naming the engine - #1665
Conversation
open_session takes engine, a package name or the factory call a config
would write as {package, export, options}, and app. An option can be
another such call, which is how an engine gets a device or browser
provider. Packages are found in the project, beside e2e, or in
~/.e2e/engines; ENGINE_NOT_INSTALLED names the install command.
@e2e-dev/web and @e2e-dev/mobile default-export their factory.
Review fixes: resolve a package by its node_modules directory, so an installed package without a loadable entry is reported as such; fall back to the import condition; refuse Node builtin names; never read the options object itself as a factory call; mobile() names the platforms it takes when called without one.
🟢 e2e web: 91 passedAll 91 tests in 41 files
Tested by e2e 0.23.0 · 1m 2s · web · run artifacts |
🟢 e2e web agent: 33 passed34 agent steps · 26 replayed from cache · 20 model calls · 88.2k tokens (94% cached) · $0.0036 All 33 tests in 5 files
Tested by e2e 0.23.0 · 1m 24s · web · run artifacts |
There was a problem hiding this comment.
All reported issues were addressed across 28 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
… the __proto__ key Review fixes: a concrete app.url in the skill example, the string shorthand in AGENTS.md, the engine-only wording in the security note, and a test assertion that the __proto__ key survives as an own property.
🟢 e2e ios agent: 21 passed, 1 skipped25 agent steps · 11 replayed from cache · 68 model calls · 427.9k tokens (83% cached) · $0.0281 All 22 tests in 1 file
Tested by e2e 0.23.0 · 9m 33s · ios-simulator · run artifacts |
🟢 e2e android agent: 20 passed24 agent steps · 15 replayed from cache · 39 model calls · 215.0k tokens (86% cached) · $0.0126 All 20 tests in 1 file
Tested by e2e 0.23.0 · 5m 46s · android-emulator · run artifacts |
🟢 e2e android: 57 passed, 1 skippedAll 58 tests in 7 files
Tested by e2e 0.23.0 · 9m 0s · android-emulator · run artifacts |
🟢 e2e ios: 55 passed, 1 skippedAll 56 tests in 7 files
Tested by e2e 0.23.0 · 7m 15s · ios-simulator · run artifacts |
|
[claude-fable-5-2] responding on behalf of KrzysztofMoch Ready for review at 5835a98. Three things are left for a human:
|
|
ENG-1191 |
kdy1
left a comment
There was a problem hiding this comment.
No blocking findings. I checked out the branch at 5835a980 and went through the parts that carry risk.
Checked
- Two copies of
e2e.isEngineHandletests the brandSymbol.for('e2e.engine.v1')(internal/brands.ts:7), so a handle from an engine that installed its owne2einto~/.e2e/enginesstill passes.loadEnginealso checksspiVersion. __proto__test. I swappedObject.fromEntriesfor plain assignment inresolveEntries. The test failed, so it pins the own-property behavior. I restored the file afterward.- Unit tests.
mcp-default-engine.test.tspasses, 17 of 17. configSourcerefusals.enginewithtarget,config, ortest, andappwithoutengine, are each covered. So are an engine session next to a config session (CONFIG_IN_USE) andenginewhere a config file exists.- Secrets in
options.@e2e-dev/limrunreadsLIMRUN_API_KEYfrom the environment, and@e2e-dev/browserstackreadsBROWSERSTACK_USERNAMEandBROWSERSTACK_ACCESS_KEYthe same way. Neither takes a key as an option, so an agent has no reason to put a secret literal inoptions. A third-party provider that takes one would not reach the secret ledger. importEntry. It runs only after Node has parsedpackage.jsonand answeredERR_PACKAGE_PATH_NOT_EXPORTED, so theJSON.parsethere cannot see a broken manifest.- Security doc. The
security.mdxrow says the server loads installed packages the agent names, calls the exports it names, and runs that code with the server's full authority. That matches the code.run_codealready runs arbitrary test statements, so this does not widen what an agent can do.
Not checked
- I did not run the integration tests (
mcp-server,mcp-session-host) locally. I relied on CI, where every check passed. - Android, and an engine installed with pnpm or Bun into
~/.e2e/engines. The PR body lists both as unverified. - Two engine copies side by side. I read the brand check but did not reproduce the install.
Open decision
isFactoryRef reads an option value as a factory call when it has a string package and no key besides export and options. A third-party option with that exact shape would be called instead of passed through. The tests pin that {package, bundleId} is left alone and that the options object itself is never a call, and the ENGINE_NOT_INSTALLED message says how the value was read, so a misread does not fail quietly.
I would keep the plain shape. A reserved key such as {$call: ...} would be a breaking change later, so decide it now if you want one.
I did not approve, since AGENTS.md says agents never approve. This is a comment review.
kdy1
left a comment
There was a problem hiding this comment.
I verified this PR with help of Sonnet 5.5
I left the verification history above
* Document MCP open_session without a config (#1509) as unported Upstream's open_session {engine} (e2e-inc/e2e#1665) names an engine package to load. The port builds engines in code from a target's platform and loads no packages, so the feature is recorded as not ported in COMPATIBILITY.md, with the reason. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> * List every skipped upstream part of the MCP engine session (#1509) Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> * Gate the sloppy fixture's late rejections on the test's release (#1509) The rejection timers of Leaves_a_promise_behind (50 ms) and Leaves_a_promise_for_after_the_close (1 s) raced the run's own work: on a slow runner the 50 ms rejection landed inside the turn and failed the run. Each rejection now waits for the test to release it once its run has returned or its session has closed, so it lands at the same point every run. The assertions are unchanged. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
e2e mcponly worked next to ane2e.config.ts. A coding agent in a fresh directory gotCONFIG_NOT_FOUNDand had to set a project up before it could look at an app. Nowopen_sessioncan name the engine instead.Fixes ENG-1191.
open_session {engine: "@e2e-dev/web", app: {url: "http://localhost:3000"}} open_session { engine: {package: "@e2e-dev/mobile", options: { platform: "ios", device: {package: "@e2e-dev/limrun", export: "limrun", options: {region: "eu"}} }}, app: {bundleId: "com.example.app"} }engineis a package name, or the factory call a config would write as{package, export?, options?}. The second call above ismobile({ platform: 'ios', device: limrun({ region: 'eu' }) }).exportnames any export, so providers already on npm work without a release.apptakesurl,bundleId, andappPathonly.open_sessionwith no config still fails. The error now shows both shapes, which is how an agent finds the option.engineis refused where a config exists, and together withtarget,config, ortest.@e2e-dev/weband@e2e-dev/mobiledefault-export their factory.mobile()without a platform now says which platforms it takes.Where a package comes from
flowchart LR A[package name] --> B{in the project?} B -- yes --> L[load it] B -- no --> C{beside e2e?} C -- yes --> L C -- no --> D{in ~/.e2e/engines?} D -- yes --> L D -- no --> E[ENGINE_NOT_INSTALLED<br/>names the npm command]e2eworks. Only the handle's brand and contract version cross.packages/e2e/src/mcp/default-engine.tsimports a resolved file, never a specifier.Security
docs/reference/security.mdxsays so...segment, and a Node builtin name are refused withINVALID_ARGUMENT.__proto__key inoptionsstays an own property.Open for a decision
packageplus nothing butexportandoptions. A third-party engine option that looks like that would be read as a call. No shipped engine or provider has one. The fresh-context reviewer suggested a reserved key such as{$call: {...}}. I kept the plain shape, made the options object itself never a call, and made the not-installed error say how the value was read.Verified
Ran it locally: yes
Surface: the built
e2e mcpserver in directories with no config.Final build, driven over stdio (
limrunhere is the repo's, with #1663):Real clients calling the tools, on earlier builds of this branch:
codex mcp add. Codex showed the modelengine?: string | { package, export?, options? }. Sol opened e2e.com with the string form, tapped[ docs ], readtest_lines, then opened/docs/quickstartwith the object form. Every step passed.claude mcp addopened e2e.com, went to the docs, and returned a four-line draft test.e2e. The runner alone in one tree,@e2e-dev/webinstalled with the printed command into a separate~/.e2e/engines: a session opened,run_codepassed a tap and an assertion.@e2e-dev/mobiledrove Safari on a local simulator to e2e.com and the docs.Not verified: Android, and an engine installed with pnpm or Bun into
~/.e2e/engines.Review
import-only packages failed to load, an installed package with no built entry was reported as not installed,engine: "@e2e-dev/mobile"failed with a rawTypeError, a Node builtin name became a file import, two untested hunks, a stale comment. Kept with changes: the shape rule above.app.urlvalue, a__proto__test that would pass if the key were dropped, "no e2e.config.ts" where any config name counts, one debugging-table row covering two error codes, the string shorthand missing from AGENTS.md, and a security sentence that read as covering more thanengine. Declined two: offeringenginewhen a lookalike config file exists (that branch predates this PR and names the rename as the fix), and wildcard subpaths ofimport-only packages (the import fallback is deliberately narrow and fails with an explicit error).Docs pass:
docs/mcp-server.mdx199 to 212 lines, 1065 to 1105 prose words.docs/writing-an-engine.mdx306 to 326 lines, 1606 to 1665 words. Both gain one short section; nothing was cut.Written by Claude Fable 5.2 in T3 Code, on the Claude Code harness.