Skip to content

chore: version packages - #1590

Merged
okwasniewski merged 1 commit into
mainfrom
changeset-release/main
Nov 13, 2026
Merged

okwasniewski merged 1 commit into
mainfrom
changeset-release/main

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@e2e-dev/browser-run@0.1.0

Minor Changes

  • #1634 352b774 Thanks @irvinebroque! - Release: New package: @e2e-dev/browser-run. A Cloudflare Browser Run browser provider for hosted Chromium leases.

@e2e-dev/web@0.16.0

Minor Changes

  • #1393 046bd08 Thanks @KrzysztofMoch! - Breaking: web({ headers }) and web({ basicAuth }) go only to the app's origin, the scheme, host, and port of app.url. Before, headers went to every host on the app's site (another port on the same host, sibling subdomains, every host under a shared domain such as vercel.app), and basicAuth answered a 401 from any origin, so a third-party origin the app embeds could collect the credential.

    To migrate, list each other origin that needs the credentials in credentialOrigins: ['https://api.example.com']; it covers both. An app whose app.url redirects to another origin (http: to https:, the apex to www) needs the origin it lands on listed, or set app.url to it. headers still ride a request's own redirects to any origin, as before. WebKit can't scope basic auth to an origin on a default port or an IPv6 host, so there it still answers any origin. Update e2e with this release: the engine needs the app's origin from the runner.

Patch Changes

  • #1609 34a260c Thanks @okwasniewski! - Bump playwright-core to 1.65.0 and commander to 15.1.0. fill no longer rejects malformed date, time, color, and range values, and actions wait for an element to leave an inert subtree. Smol sandboxes use the Playwright 1.65.0 image.

  • #1595 12c6c01 Thanks @samanyudas! - check, uncheck, and tap click a checkbox or radio hidden under its own label (as in Element Plus) through that label. uncheck still refuses a native radio without clicking it. Anything else on top of the point the click lands on still fails ACTION_FAILED.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1595 12c6c01 Thanks @samanyudas! - The agent now sees a checkbox or radio that a component library shrinks to no size under its own label (as in Element Plus), with its checked state. Before, it saw only the label text, and a tap that ticked the box reported no change.

  • #1665 3c9de80 Thanks @KrzysztofMoch! - Open an MCP session in a directory with no config: open_session {engine: "@e2e-dev/web", app: {url}} names the engine to drive and app the app under test. engine is a package name, or the factory call a config would write as {package, export, options}, where an option can be another such call: {package: "@e2e-dev/mobile", options: {platform: "ios", device: {package: "@e2e-dev/limrun", export: "limrun"}}}. Each package is found in the project, beside e2e, or in ~/.e2e/engines; when it is in none, ENGINE_NOT_INSTALLED names the install command. @e2e-dev/web and @e2e-dev/mobile export their factory as the default export for this, and mobile() called without a platform now says which platforms it takes.

  • #1614 f6aba97 Thanks @okwasniewski! - A screenshot taken while the page navigates still captures the document showing when Playwright reports the destroyed context before the navigation event.

  • #1592 b43e846 Thanks @KrzysztofMoch! - expect(browser).toHaveURL failures print a relative expected URL resolved against the app URL, the same way it is compared, so it reads beside the absolute observed URL.

  • #1747 a94c765 Thanks @okwasniewski! - Send CDP handshake headers for hosted browser leases.

  • #1710 0920ef5 Thanks @kdy1! - The browser fixture drives tabs. A tab the app opens (a target="_blank" link, window.open, an OAuth popup) joins browser.tabs() in the background, and the active tab stays where it was. browser.waitForPopup(trigger) runs the trigger and makes the tab it opens active, browser.newTab() opens a tab in the same context, and each BrowserTab has url, title, isActive, isClosed, activate, and close. When the active tab closes, the tab that opened it is active again, or the tab active most recently when that one is closed too. The agent's switch_tab and close_tab verbs move between tabs and close them. The web engine needs e2e 0.23.1 or newer. Tabs

@e2e-dev/acp@0.1.2

Patch Changes

  • #1626 c9783a0 Thanks @Varshith-Kali! - Fix WORKER_CRASH (EPERM) on Windows when an ACP session ends: a failed session directory removal no longer crashes the worker and is retried after the agent exits.

  • #1568 1144df4 Thanks @bheemreddy-samsara! - Add opt-in scrolling exploration to agent.assert with bounded action and model-call budgets. Enforce read-only assertion tools for built-in and custom executors. ACP assertions now deny hover; mutating actions from custom executors fail with POLICY_DENIED during any assertion. createVerdictTool(kind) drops ASSERTION_INCONCLUSIVE from an act verdict, and createToolLoopExecutor({ inconclusiveOnOverflow: true }) fails a context overflow as ASSERTION_INCONCLUSIVE instead of shortening the history.

  • #1666 795f71f Thanks @okwasniewski! - Bump runtime dependencies: @clack/prompts 1.9.0, oxc-parser, oxc-transform, and @oxc-project/runtime 0.158.0, @ai-sdk/provider 4.0.37, and @agentclientprotocol/sdk 2.5.0 (ACP schema v1.30.0).

  • #1712 97093eb Thanks @okwasniewski! - Bump @agentclientprotocol/sdk to 2.6.0.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1715 be460ec Thanks @okwasniewski! - Redact project tool results and errors before the model or AI trace recorder reads them, and redact and bound skip reasons before reporting. Avoid repeated scanning of malformed stack frames and trailing slashes in provider API endpoints. Keep ACP forwarding errors out of HTTP responses while preserving the cause in the runner's redacted failure output.

@e2e-dev/browserstack@0.1.1

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1715 be460ec Thanks @okwasniewski! - Redact project tool results and errors before the model or AI trace recorder reads them, and redact and bound skip reasons before reporting. Avoid repeated scanning of malformed stack frames and trailing slashes in provider API endpoints. Keep ACP forwarding errors out of HTTP responses while preserving the cause in the runner's redacted failure output.

@e2e-dev/decision@0.3.3

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1587 34ccc5f Thanks @kdy1! - PNG decoding and encoding now run on @napi-rs/image, a native addon with a prebuilt binary per platform, in place of fast-png and pngjs: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a toHaveScreenshot capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. e2e loads it when the first screenshot is read or written, so --version and a run that takes none never need it, and a missing binding fails that call with IMAGE_CODEC_UNAVAILABLE; @e2e-dev/mobile and @e2e-dev/decision load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.

  • #1589 b5e4703 Thanks @kdy1! - A test filter can score the tests it keeps. relevance in its result gives each test a number from 0 to 1, and a run with --changed-since starts the files of the likeliest first in each target, in bands at 75%, 50%, and 25%, so a failure shows sooner and --max-failures stops earlier. Targets go by the band of their best file, ties in config order. Inside a band, a run with more than one worker still starts the file the previous run spent longest in first.

    decisionFilter() now scores every test it keeps: 100% for a test whose file, setup file, or stored screenshots changed, the model's answer for one it judged, and 50% for one it did not judge. When it fails, as without a key, it puts the scores the change alone gives on the error it throws, so the run still warns and runs every test, with the touched ones first. A filter of your own can do the same: an error it throws may carry relevance. decisionFilter() requires e2e 0.23.1 or newer, since an older runner refuses the new relevance key and drops the whole filter.

@e2e-dev/e2b@0.0.1

Patch Changes

  • #1735 16ff228 Thanks @gronxb! - Update agent-device to 0.21.37. Without device, the iOS pool no longer picks a paired iPhone that is not connected: agent-device now lists a phone whose CoreDevice tunnel is unavailable as not booted. A device that names such a phone fails in prepare with "iOS device is not ready for automation", before any test runs, instead of at app launch. A test's own swipe and scroll now skip agent-device's post-gesture wait, which 0.21.36 ignored.

  • #1730 7e309c1 Thanks @karolkulesza! - The setup guide's link to the command reference no longer returns 404. The e2b and HTML reporter READMEs link to e2e.com instead of e2e.tester.army.

  • #1598 ae5d121 Thanks @tomasvarga! - New package: e2b(), a device provider that runs Android 16 in one E2B sandbox per worker slot and installs the target's .apk, and e2bBrowsers() from @e2e-dev/e2b/web, a browser provider that runs Chromium in E2B sandboxes per worker slot or per attempt.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

e2e@0.23.1

Patch Changes

  • #1710 0920ef5 Thanks @kdy1! - The agent tells the model when an action opened a tab in the background (The app opened a new tab (tab 2 of 2). The screen still shows tab 1; use switch_tab to see it.), names the tab in every screen header while more than one is open (tab 1 of 2), and gets switch_tab and close_tab verbs on engines that declare the new session.switchTab and session.closeTab hooks. EngineSnapshot.tabs reports the open tabs. Action tools

  • #1600 f5c9099 Thanks @okwasniewski! - The built-in agent can try an upload when the step names no file. upload takes sample:photo.png, sample:photo.jpg, sample:document.pdf, sample:notes.txt, sample:table.csv, and sample:empty.txt, small files e2e ships, and the tool lists them. An e2e mcp upload of a sample prints no test line, since a test needs a project file.

  • #1541 11dea90 Thanks @samanyudas! - Fail e2e run when an app command exits unexpectedly after readiness.
    Report its exit code or signal and redacted log tail.
    Interrupt active tests and skip queued tests only for targets that depend on that command.
    Ignore command exits after those targets finish their work.
    An exit with code 0 also fails the run, so a command that starts the app in the background and then exits, such as docker compose up -d --wait, now fails.
    Keep the app in the foreground instead, for example docker compose up without -d.

  • #1577 9289ff4 Thanks @DeryFerd! - A startup failure now quotes the first lines of a long command.log as well as the last, with [startup output omitted] between them, so the error a service printed before it kept logging survives in the APP_UNREACHABLE message.

  • #1568 1144df4 Thanks @bheemreddy-samsara! - Add opt-in scrolling exploration to agent.assert with bounded action and model-call budgets. Enforce read-only assertion tools for built-in and custom executors. ACP assertions now deny hover; mutating actions from custom executors fail with POLICY_DENIED during any assertion. createVerdictTool(kind) drops ASSERTION_INCONCLUSIVE from an act verdict, and createToolLoopExecutor({ inconclusiveOnOverflow: true }) fails a context overflow as ASSERTION_INCONCLUSIVE instead of shortening the history.

  • #1609 34a260c Thanks @okwasniewski! - Bump playwright-core to 1.65.0 and commander to 15.1.0. fill no longer rejects malformed date, time, color, and range values, and actions wait for an element to leave an inert subtree. Smol sandboxes use the Playwright 1.65.0 image.

  • #1666 795f71f Thanks @okwasniewski! - Bump runtime dependencies: @clack/prompts 1.9.0, oxc-parser, oxc-transform, and @oxc-project/runtime 0.158.0, @ai-sdk/provider 4.0.37, and @agentclientprotocol/sdk 2.5.0 (ACP schema v1.30.0).

  • #1616 5847701 Thanks @okwasniewski! - The CLI runs on Bun 1.4.3 or newer without Node.js: bunx --bun e2e run. In a Bun project, e2e init registers the MCP server with bunx. An older Bun gets the version to upgrade to. What differs on Bun.

  • #1586 3c902b0 Thanks @kdy1! - Register the MCP server in Codex project settings during e2e init. Preserve existing Codex settings and comments when updating the server entry.

  • #1730 7e309c1 Thanks @karolkulesza! - The setup guide's link to the command reference no longer returns 404. The e2b and HTML reporter READMEs link to e2e.com instead of e2e.tester.army.

  • #1742 8077542 Thanks @bardrop! - Reject a config timeout above 2147483647 ms with INVALID_CONFIG instead of letting every test time out at once.
    This covers timeout, launchTimeout, actionTimeout, assertionTimeout, cleanupTimeout, app.command.startupTimeout and shutdownTimeout, and agents.<name>.judgmentTimeout.
    A test's or group's own timeout above that limit now fails collection too.

  • #1393 046bd08 Thanks @KrzysztofMoch! - Engines receive the app's origin as EngineAppInfo.origin in prepare and init, on the run's port when app.url asks for a free one (and left out until the run assigns it), so an engine can scope a credential it injects to the exact origin.

  • #1711 17a2e46 Thanks @KrzysztofMoch! - e2e explore --changed-since <ref> explores what a branch changed. The planner reads the commit messages and the committed diff since the merge base of the ref and HEAD, and plans its steps for the screens the change can affect. Only a defect the change can cause is an issue; one elsewhere is a warning. With no goal, the goal is the change. The report records the change as run.explore.change, and explore() on e2e/runner takes changedSince too.

    npx e2e explore --changed-since origin/main
  • #1711 17a2e46 Thanks @KrzysztofMoch! - e2e explore --changed-since can skip a change no user can see, such as docs or tests only. It skips only with no goal given, and only when the planner read the whole change on a clean working tree. The run passes with no step, run.explore.ended reads skipped, and the comment headline reads nothing to explore.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1526 ed9eb0e Thanks @Mohsintl! - The junit reporter links failure screenshots using GitLab CI's [[ATTACHMENT|<path>]] format in <system-out>, so screenshots show directly in pipeline test reports.

  • #1515 ed9e633 Thanks @two-pack! - agent.assert, agent.waitFor, and agent.extract now see the current location, so an instruction about the URL can hold or fail instead of ending inconclusive. agent.waitFor also judges again when only the location changed, except under vision: 'only', which does not see the location.

  • #1717 258ef9b Thanks @KrzysztofMoch! - ENGINE_NOT_INSTALLED from e2e mcp now tells the agent to ask the user to run the install command when it cannot run it itself.

  • #1665 3c9de80 Thanks @KrzysztofMoch! - Open an MCP session in a directory with no config: open_session {engine: "@e2e-dev/web", app: {url}} names the engine to drive and app the app under test. engine is a package name, or the factory call a config would write as {package, export, options}, where an option can be another such call: {package: "@e2e-dev/mobile", options: {platform: "ios", device: {package: "@e2e-dev/limrun", export: "limrun"}}}. Each package is found in the project, beside e2e, or in ~/.e2e/engines; when it is in none, ENGINE_NOT_INSTALLED names the install command. @e2e-dev/web and @e2e-dev/mobile export their factory as the default export for this, and mobile() called without a platform now says which platforms it takes.

  • #1696 de8ae3c Thanks @kacperkapusciak! - e2e mcp tells the agent to read the writing-tests guide before it writes or edits a test, in the server's instructions and in every open_session result.

  • #1208 cc94232 Thanks @piotrpasztor! - A step whose effect or click target is a node identified only by a generated test id (a uuid, hex id, cuid, or timestamp) no longer fails replay on every run.

  • #1587 34ccc5f Thanks @kdy1! - PNG decoding and encoding now run on @napi-rs/image, a native addon with a prebuilt binary per platform, in place of fast-png and pngjs: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a toHaveScreenshot capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. e2e loads it when the first screenshot is read or written, so --version and a run that takes none never need it, and a missing binding fails that call with IMAGE_CODEC_UNAVAILABLE; @e2e-dev/mobile and @e2e-dev/decision load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.

  • #1721 565e685 Thanks @KrzysztofMoch! - The bundled e2e skill's running guide now says --output must be a directory inside the project root, not the root itself.

  • #1589 b5e4703 Thanks @kdy1! - A test filter can score the tests it keeps. relevance in its result gives each test a number from 0 to 1, and a run with --changed-since starts the files of the likeliest first in each target, in bands at 75%, 50%, and 25%, so a failure shows sooner and --max-failures stops earlier. Targets go by the band of their best file, ties in config order. Inside a band, a run with more than one worker still starts the file the previous run spent longest in first.

    decisionFilter() now scores every test it keeps: 100% for a test whose file, setup file, or stored screenshots changed, the model's answer for one it judged, and 50% for one it did not judge. When it fails, as without a key, it puts the scores the change alone gives on the error it throws, so the run still warns and runs every test, with the touched ones first. A filter of your own can do the same: an error it throws may carry relevance. decisionFilter() requires e2e 0.23.1 or newer, since an older runner refuses the new relevance key and drops the whole filter.

  • #1431 382b8c4 Thanks @21prnv! - A replay no longer fails its end check on a value that changes every run, such as a clock or a date, when the other side of the step's delta already holds a stable result like a deleted row.

  • #1642 a133807 Thanks @KrzysztofMoch! - A reporter's onRunFinished gets run.redact(text), which masks the config's secret values the way a test filter's redact does, for text it adds from outside the report, such as lines of a test file.

  • #1662 d01bd6d Thanks @Akarshkushwaha! - When --target selects configured targets, the run header and the run-started event's targets name those targets in config order instead of every configured target. Without --target, or with an unknown target ID, they still name every configured target. The report's run.targets still lists every configured target.

  • #1540 93c93f5 Thanks @KrzysztofMoch! - e2e run --step runs the first step, then holds each later step until you press a key, with the keys of --watch: Enter runs it, n runs the rest of the test, a runs the rest of the run, h lists the keys, and q stops the run with exit 130. Add --headed to watch the app between steps. The time a step is held counts toward no timeout, so a test can wait at a prompt past its timeout and maxDuration. Setup tests, retries, teardown steps, and steps inside an expect.poll read run without stopping. --step runs one worker and is refused in CI, without a terminal, with --watch, with --workers above 1, and with the json reporter.

  • #1331 d1b0ff6 Thanks @fghpdf! - Add screen.doubleTapAt(point, options?) to double-tap a viewport point on engines that support the doubleTap pointer action.

  • #1570 34f85c3 Thanks @kdy1! - scroll_to({ text }) is faster: it reads each page once when the swipe visibly landed, instead of waiting for a second look to confirm it. A clock ticking beside a list no longer delays the end of the list: the call fails with LOCATOR_NOT_FOUND after a few swipes, not 800 screens later.

  • #1614 f6aba97 Thanks @okwasniewski! - scroll_to({ text }) keeps paging a list that appends when it reaches the end. An unchanged screen is re-read for a second before it counts as stopped, so that pause is not LOCATOR_NOT_FOUND.

  • #1715 be460ec Thanks @okwasniewski! - Redact project tool results and errors before the model or AI trace recorder reads them, and redact and bound skip reasons before reporting. Avoid repeated scanning of malformed stack frames and trailing slashes in provider API endpoints. Keep ACP forwarding errors out of HTTP responses while preserving the cause in the runner's redacted failure output.

  • #1601 0f158a3 Thanks @okwasniewski! - The bundled e2e skill's bug bash writes each repro test from the finding as agent.act steps and a closing agent.assert, so confirming a bug needs no locators. Each verifier runs its own repros into its own output directory, and the report runs them all once with --tag bugbash. A finding is confirmed when its repro fails on the reported defect, and the skill says how to read a failed act. Charters get about fifteen minutes each, a bash under a time limit keeps running new charters until near the end and reports each bug as its repro confirms it, and an account with a password under 6 characters is called out before the config fails.

  • #1591 937ea15 Thanks @kacperkapusciak! - The bundled e2e skill's GitHub Actions workflow now caches the Playwright browsers between runs.

  • #1664 14dc946 Thanks @Akarshkushwaha! - The bundled e2e skill's running reference now gives e2e doctor --json for JSON output, the flag the command accepts, instead of --reporter json.

  • #1659 00e19d6 Thanks @karolkulesza! - The bundled e2e skill's setup reference shows how to run an iOS app in another language with app.launchArguments, one target per language.

  • #1694 dc01aaa Thanks @kacperkapusciak! - The bundled e2e skill now writes tests with one top-level describe per feature, one behavior per test, and arrange, act, assert order with every assertion at the end, instead of a check after each agent.act.

  • #1627 b0e3012 Thanks @karolkulesza! - The bundled e2e skill now says what timeout: 0 and a negative timeout do on actions, waitFor, matchers, and expect.poll.

  • #1639 b285835 Thanks @kdy1! - e2e run keeps a small <output>/timings.json (schema timings-1) of how long each file took, smoothed over recent runs, and starts the longest files first from it when it runs with more than one worker. A CI cache can restore and save just that file, so a fresh checkout gets the ordering; --timings <file> moves it. A missing or invalid file is no history, and the previous report.json is still the fallback.

  • #1511 d784448 Thanks @Mohsintl! - A replay no longer skips text like Marketing 12 as a date when checking its end state.

@e2e-dev/eas@0.3.4

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

@e2e-dev/github@0.4.5

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

@e2e-dev/html-reporter@0.0.1

Patch Changes

  • #1730 7e309c1 Thanks @karolkulesza! - The setup guide's link to the command reference no longer returns 404. The e2b and HTML reporter READMEs link to e2e.com instead of e2e.tester.army.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1642 a133807 Thanks @KrzysztofMoch! - New package: htmlReporter() writes the run as one HTML page, .e2e/report.html, that opens from disk with no server. It lists the tests by file with status tabs and search, and shows each test's attempts, error, steps, screenshots, and video, with each step linked to its moment in the video. Agent steps show their explanation, the actions they took, the model, tokens and cost, and whether the step replayed from the cache. A failed assertion that reports expected and received values shows them with the difference marked. When the test file is in the project, the failure shows the code around the failing call, and each step opens to its code. copy prompt on a failed test copies its failure, steps, and file paths for a coding agent. The page has a light and a dark theme, following the system's by default. media: 'embed' puts screenshots and videos in the page so the one file is the whole report, and media: 'none' leaves them out.

@e2e-dev/kernel@0.2.2

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

@e2e-dev/lightpanda@0.0.2

Patch Changes

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

@e2e-dev/limrun@0.2.5

Patch Changes

  • #1735 16ff228 Thanks @gronxb! - Update agent-device to 0.21.37. Without device, the iOS pool no longer picks a paired iPhone that is not connected: agent-device now lists a phone whose CoreDevice tunnel is unavailable as not booted. A device that names such a phone fails in prepare with "iOS device is not ready for automation", before any test runs, instead of at app launch. A test's own swipe and scroll now skip agent-device's post-gesture wait, which 0.21.36 ignored.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1663 322f00e Thanks @KrzysztofMoch! - A Limrun device lease no longer names an agent-device session in its client configuration, which @e2e-dev/mobile rejected with ENGINE_FAILURE ("returned a lease without an id and a daemon baseUrl or JSON client configuration") before any test or session could start. The provider still allocates and renews the lease under its own session.

@e2e-dev/mobile@0.12.7

Patch Changes

  • #1735 16ff228 Thanks @gronxb! - Update agent-device to 0.21.37. Without device, the iOS pool no longer picks a paired iPhone that is not connected: agent-device now lists a phone whose CoreDevice tunnel is unavailable as not booted. A device that names such a phone fails in prepare with "iOS device is not ready for automation", before any test runs, instead of at app launch. A test's own swipe and scroll now skip agent-device's post-gesture wait, which 0.21.36 ignored.

  • #1702 461659e Thanks @karolkulesza! - Say which platform each biometric sensor works on in the setBiometrics and enrollBiometrics docs.

  • #1674 e534014 Thanks @KrzysztofMoch! - Fix ENGINE_FAILURE (launch) boot failed: N devices match this request equally when a target names no device and no simulator or emulator is booted. The engine now asks agent-device to boot a shut-down one: newest OS first, then name order.

  • #1624 1abd372 Thanks @karolkulesza! - device.dismissKeyboard() now documents in its type how it dismisses the keyboard and when it fails, and the videoTouches description names Expo Simulators as a slow hosted daemon.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1665 3c9de80 Thanks @KrzysztofMoch! - Open an MCP session in a directory with no config: open_session {engine: "@e2e-dev/web", app: {url}} names the engine to drive and app the app under test. engine is a package name, or the factory call a config would write as {package, export, options}, where an option can be another such call: {package: "@e2e-dev/mobile", options: {platform: "ios", device: {package: "@e2e-dev/limrun", export: "limrun"}}}. Each package is found in the project, beside e2e, or in ~/.e2e/engines; when it is in none, ENGINE_NOT_INSTALLED names the install command. @e2e-dev/web and @e2e-dev/mobile export their factory as the default export for this, and mobile() called without a platform now says which platforms it takes.

  • #1685 68407d2 Thanks @shamilovtim! - Cancel pending taps, double taps, focus, check and uncheck actions when the step stops waiting for them or the run is interrupted. Input already sent to the device may still finish.

  • #1686 368f638 Thanks @kdy1! - A node under the soft keyboard is hidden (the keyboard's own nodes and the focused field excepted): screen.scrollUntilVisible keeps scrolling to it, toBeVisible() fails on it, and getByRole skips it. A tap that agent-device refuses, without sending it, because the keyboard or an overlay covers the target now fails ACTION_FAILED with the hint that names the cover, not as an engine failure.

  • #1619 a016b3e Thanks @karolkulesza! - mobile({ device: null }) now fails with INVALID_CONFIG instead of a raw TypeError.

  • #1640 6dbb8f9 Thanks @kdy1! - Log how long each device's prepare steps took: a line such as iPhone 17 Pro: boot 8.1s, iOS runner 120.3s, open Settings 3.2s follows each booting line, also when a step fails or times out, so a slow engine prepare in the run output shows where the time went.

  • #1584 984f139 Thanks @mikolajadamowicz! - Custom tools can use unstable_deviceSession(...engines) from @e2e-dev/mobile/tools to access the running attempt's raw driver session for capabilities e2e does not wrap yet. It shares DeviceSession with device.unstable_raw. The raw client is typed as DeviceSessionClient, exported from @e2e-dev/mobile. Both escape hatches can change in any release.

  • #1564 f45d9f8 Thanks @shamilovtim! - Wait out the remaining transition budget before a test scrolls the screen, including after dismissing a native alert.

  • #1668 2b727fe Thanks @kdy1! - Send postGestureStabilization: false with a test's own swipe and scroll, so an agent-device release that accepts the option stops holding the next read for up to 3.5 s after a gesture the test then checks with expect. The first action on a control within 2 s of such a gesture re-reads the control until two reads agree on where it is, so a tap does not land on a list still coasting and get swallowed. The agent's gestures keep the wait.

  • #1587 34ccc5f Thanks @kdy1! - PNG decoding and encoding now run on @napi-rs/image, a native addon with a prebuilt binary per platform, in place of fast-png and pngjs: decoding is 8 to 15 times faster, encoding 5 to 8 times, and a toHaveScreenshot capture 4 to 12 times. Stored screenshots decode to the same pixels as before. A screenshot the runner writes (a new baseline, or one rewritten because the screen changed) has the same pixels in different bytes: a Chromium capture comes out about 15% larger and a device screenshot about 20% smaller. The binary adds about 15 to 21 MB to an install, depending on the platform, and the package publishes none for Linux on ppc64, s390x, or riscv64. e2e loads it when the first screenshot is read or written, so --version and a run that takes none never need it, and a missing binding fails that call with IMAGE_CODEC_UNAVAILABLE; @e2e-dev/mobile and @e2e-dev/decision load it when they are imported. Gray PNGs with 1, 2, or 4 bits per sample now decode, and a 16-bit gray or RGB PNG with a transparent color now compares pixels to it at 16 bits, so a pixel that only rounds to it at 8 bits stays opaque.

  • #1605 62d3c66 Thanks @karolkulesza! - device.setPermission rejects an unknown permission name or state with INVALID_ARGUMENT, naming the nearest permission for a typo.

  • #1679 a801a52 Thanks @karolkulesza! - The settle option's documentation now says the agent's actions settle and a test's steps do not, except device.back().

@e2e-dev/smol@0.1.3

Patch Changes

  • #1609 34a260c Thanks @okwasniewski! - Bump playwright-core to 1.65.0 and commander to 15.1.0. fill no longer rejects malformed date, time, color, and range values, and actions wait for an element to leave an inert subtree. Smol sandboxes use the Playwright 1.65.0 image.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

@e2e-dev/testmu@0.0.5

Patch Changes

  • #1735 16ff228 Thanks @gronxb! - Update agent-device to 0.21.37. Without device, the iOS pool no longer picks a paired iPhone that is not connected: agent-device now lists a phone whose CoreDevice tunnel is unavailable as not booted. A device that names such a phone fails in prepare with "iOS device is not ready for automation", before any test runs, instead of at app launch. A test's own swipe and scroll now skip agent-device's post-gesture wait, which 0.21.36 ignored.

  • #1672 3735887 Thanks @okwasniewski! - Move GitHub repository metadata and CLI links to e2e-inc/e2e after the organization rename. Register the MCP server as io.github.e2e-inc/e2e.

  • #1715 be460ec Thanks @okwasniewski! - Redact project tool results and errors before the model or AI trace recorder reads them, and redact and bound skip reasons before reporting. Avoid repeated scanning of malformed stack frames and trailing slashes in provider API endpoints. Keep ACP forwarding errors out of HTTP responses while preserving the cause in the runner's redacted failure output.

@github-actions
github-actions Bot force-pushed the changeset-release/main branch 2 times, most recently from 561a094 to 5ce7c5a Compare November 6, 2026 09:20
@github-actions
github-actions Bot force-pushed the changeset-release/main branch 26 times, most recently from 6f64284 to c3a4239 Compare November 9, 2026 13:45
@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e web: 93 passed

All 93 tests in 42 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 643ms
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 474ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 169ms
🟢 tests/async-states.e2e.ts · 2 passed 7.1s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.3s
🟢 async states › a wrong code is rejected after the toast has gone 4.8s
🟢 tests/canvas-only.e2e.ts · 2 passed 237ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 126ms
🟢 canvas only › a shape out of order paints the error and resets progress 111ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 387ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 236ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 151ms
🟢 tests/cart-totals.e2e.ts · 2 passed 339ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 191ms
🟢 cart totals › a quantity never drops below one 148ms
🟢 tests/checkout-review.e2e.ts · 1 passed 165ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 165ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 515ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 267ms
🟢 context menu trap › the decoy menu items refuse and keep the file 248ms
🟢 tests/control-inventory.e2e.ts · 8 passed 3.4s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 247ms
🟢 control inventory › check picks a radio the pick replaces with its summary 139ms
🟢 control inventory › a long press and a double click are told apart from a tap 976ms
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 239ms
🟢 control inventory › setInputFiles attaches files named from the project root 127ms
🟢 control inventory › the details view is left through the browser history 147ms
🟢 control inventory › scrollIntoView brings the footnote into view 237ms
🟢 control inventory › every exercise done shows the success message 1.3s
🟢 tests/css-content-ui.e2e.ts · 2 passed 299ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 164ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 135ms
🟢 tests/date-picker.e2e.ts · 2 passed 546ms
🟢 date picker › navigates three months ahead and books the required date 313ms
🟢 date picker › confirming without the exact date shows the error 233ms
🟢 tests/debounced-search.e2e.ts · 2 passed 2.8s
🟢 debounced search › a decoy one character off is refused 1.4s
🟢 debounced search › picking the exact item adds it to the cart 1.4s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.1s
🟢 one submit opens one confirm, even when double-clicked 1.1s
🟢 tests/div-soup.e2e.ts · 2 passed 607ms
🟢 div soup › a wrong code is rejected 275ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 332ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 597ms
🟢 drag and drop › an out-of-order item resets the dropzone 297ms
🟢 drag and drop › dropping Banana then Cherry passes 300ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 198ms
🟢 filter deep link › filtering writes the query to the URL 109ms
🟢 filter deep link › opening a shared link restores the filter 89ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 482ms
🟢 gift card purchase › refuses to buy without an amount 134ms
🟢 gift card purchase › refuses a recipient without an at sign 180ms
🟢 gift card purchase › sends the gift card 168ms
🟢 tests/hover-menu.e2e.ts · 2 passed 398ms
🟢 hover menu › a decoy leaf is refused 181ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 217ms
🟢 tests/iframe-form.e2e.ts · 2 passed 579ms
🟢 iframe form › a wrong code is refused by the host page 328ms
🟢 iframe form › the code read from one frame applies through the other 251ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 403ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 99ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 138ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 166ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 2.6s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 2.6s
🟢 tests/kanban-board.e2e.ts · 3 passed 703ms
🟢 kanban board › pointer drags move both cards into the goal layout 350ms
🟢 kanban board › a swipe along a path drags a card as well 135ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 218ms
🟢 tests/login-form.e2e.ts · 4 passed 712ms
🟢 login form › rejects a malformed email 182ms
🟢 login form › rejects a short password 165ms
🟢 login form › rejects an unknown account 168ms
🟢 login form › signs in and logs out 197ms
🟢 tests/lying-labels.e2e.ts · 4 passed 597ms
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 133ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 91ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 189ms
🟢 lying labels › visible text drives the payment through 184ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 435ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 170ms
🟢 native dialogs › dismissing the confirm cancels the order 131ms
🟢 native dialogs › a wrong gift code is rejected 134ms
🟢 tests/new-tab-flow.e2e.ts · 2 passed 379ms
🟢 new tab flow › reads the code in the tab the app opens and enters it back in the original tab 193ms
🟢 new tab flow › a tap that opens a tab leaves the screen on the original tab, and activate shows the new one 186ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.2s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.2s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 4.2s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 244ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 325ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.1s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 1.2s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 1.3s
🟢 tests/order-history.e2e.ts · 2 passed 1.4s
🟢 order history › overview and settings render their panels 216ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.2s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.3s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 444ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 124ms
🟢 otp auto-advance › a wrong code clears every box and starts over 745ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 697ms
🟢 continues only after the banner and the overlay are gone 697ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 411ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 213ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 198ms
🟢 tests/price-sorting.e2e.ts · 3 passed 336ms
🟢 price sorting › opens in the featured order with sale prices struck through 97ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 112ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 127ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.1s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.1s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.4s
🟢 buys a deal once the promo chrome is out of the way 1.4s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 429ms
🟢 rich text editor › bolding exactly the required word passes the check 212ms
🟢 rich text editor › bold that spills into the previous word fails the check 217ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 501ms
🟢 saved addresses › adds an address and clears the form 171ms
🟢 saved addresses › ignores an add with an empty field 147ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 183ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 497ms
🟢 shadow DOM form › rejects a wrong access code 247ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 250ms
🟢 tests/smoke.e2e.ts · 1 passed 393ms
🟢 home lists the scenarios and opens one 393ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.1s
🟢 taps the moving target three times across rebuilds of the list 2.1s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 202ms
🟢 accepts the terms once the button clears the sticky footer 202ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 14.6s
🟢 scrolls the windowed table to the Golden Row and claims it 14.6s
🟢 tests/wishlist.e2e.ts · 2 passed 1.5s
🟢 wishlist › saving toggles the button and the tab badge 216ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.2s

Tested by e2e 0.23.1 · 60.0s · web · run artifacts

@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e web agent: 34 passed

35 agent steps · 26 replayed from cache · 25 model calls · 117.7k tokens (87% cached) · $0.0068

All 34 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 7.5s
🟢 control inventory › check sets the checkbox and the radio 1 step 686ms
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 316ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 934ms
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.5s
🟢 control inventory › upload attaches the named project files 1 step 2.4s
🟢 control inventory › back returns from the details view to the inventory 1 step 249ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 345ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 354ms
🟢 act signs in with the benchmark account 1 step 354ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 17.0s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 10.0s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 1.6s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 5.4s
🟢 tests-agent/scenarios.e2e.ts · 22 passed 22 steps · 17 calls 1m 20s
🟢 act completes shadow-dom-form 1 step · 3 calls 4.0s
🟢 act completes lying-labels 1 step · 3 calls 4.9s
🟢 act completes iframe-form 1 step · 3 calls 4.6s
🟢 act completes infinite-scroll 1 step 3.4s
🟢 act completes virtualized-table 1 step 20.8s
🟢 act completes overlay-trap 1 step 841ms
🟢 act completes sticky-chrome 1 step 927ms
🟢 act completes async-states 1 step · 3 calls 6.8s
🟢 act completes debounced-search 1 step 2.2s
🟢 act completes hover-menu 1 step 448ms
🟢 act completes drag-and-drop 1 step 417ms
🟢 act completes kanban-board 1 step 565ms
🟢 act completes context-menu-trap 1 step 2.6s
🟢 act completes stale-dom 1 step 432ms
🟢 act completes otp-auto-advance 1 step 719ms
🟢 act completes date-picker 1 step 866ms
🟢 act completes rich-text-editor 1 step 10.0s
🟢 act completes playbook-cleanup 1 step 510ms
🟢 act completes promo-storefront 1 step 2.3s
🟢 act completes new-tab-flow 1 step · 5 calls 9.7s
🟢 act completes gift-card-purchase 1 step 2.5s
🟢 act filters the list and the URL carries the filter 1 step 215ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 267ms
🟢 act adds an address with a run-unique label 1 step 267ms

Tested by e2e 0.23.1 · 1m 45s · web · run artifacts

@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e android agent: 20 passed

24 agent steps · 15 replayed from cache · 46 model calls · 300.8k tokens (78% cached) · $0.0234

All 20 tests in 1 file
Test Agent Time
🟢 tests-agent/scenarios.e2e.ts · 20 passed 24 steps · 46 calls 6m 0s
🟢 act completes Login Form 1 step 23.7s
🟢 act completes Infinite Scroll List 1 step 21.5s
🟢 act completes Modal Flow 1 step 9.9s
🟢 act completes Bottom Tabs 1 step 7.9s
🟢 act completes Text Input Variations 1 step 6.8s
🟢 act completes Broken Accessibility 2 steps · 5 calls 14.6s
🟢 act completes Vision Only 2 steps · 1 call 6.0s
🟢 act completes Gestures 1 step · 9 calls 22.9s
🟢 act completes Async States 1 step · 19 calls 43.3s
🟢 act completes Debounced Search 1 step 6.6s
🟢 act completes Huge Virtualized List 1 step 1m 49s
🟢 act completes Flattened Registration Form 2 steps · 1 call 8.3s
🟢 act completes Flattened Login 2 steps · 6 calls 14.8s
🟢 act completes Sticky Chrome Target 1 step 8.8s
🟢 act completes WebView Accessibility 1 step · 5 calls 15.5s
🟢 act completes Permission Prompt 1 step 6.3s
🟢 act completes Product Catalog 1 step 8.1s
🟢 act completes Error Recovery 1 step 8.9s
🟢 act completes Choice Controls 1 step 8.2s
🟢 act completes Sequential Onboarding 1 step 9.4s

Tested by e2e 0.23.1 · 6m 1s · android-emulator · run artifacts

@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 3m 3s
🟢 control inventory › a label query resolves a labeled text field 11.2s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 16.6s
🟢 control inventory › a placeholder query and attribute reads find the field 5.4s
🟢 control inventory › a secure field takes a fill and denies value reads 8.4s
🟢 control inventory › press sends a character and Enter through the soft keyboard 9.8s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 10.5s
🟢 control inventory › toBeFocused follows the focus 8.8s
🟢 control inventory › a switch checks, unchecks, and reports its state 10.7s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 8.6s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 13.1s
🟢 control inventory › a radio group reports the checked option 9.3s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 9.6s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.3s
🟢 control inventory › a header answers its accessible name 4.4s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.4s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 6.7s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.2s
🟢 control inventory › pointer verbs land at the point asked for 9.1s
🟢 control inventory › an ambiguous locator fails at once 5.8s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 11.4s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 515ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 53s
🟢 device fixture › setAppearance flips the color scheme the app reads 11.5s
🟢 device fixture › setOrientation rotates the window the app measures 10.7s
🟢 device fixture › the clipboard reads back what was written 12.7s
🟢 device fixture › home leaves the app, openApp brings it back where it was 7.1s
🟢 device fixture › closeApp then openApp relaunches on the home list 9.4s
🟢 device fixture › closeApp leaves the next commands on the same device 22.9s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 6.0s
🟢 device fixture › device.back pops the scenario 6.8s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 4.9s
🟢 device fixture › setLocation and clearLocation change what the app reads 10.5s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 10.5s
🟢 tests/device.e2e.ts · 3 passed 50.1s
🟢 sequential onboarding echoes the exact values 29.0s
🟢 onboarding rejects a malformed email before advancing 10.1s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.1s
🟢 tests/flows-lists.e2e.ts · 7 passed 4m 1s
🟢 infinite scroll reaches the target item 34.5s
🟢 product catalog adds exactly two of the right variant 10.0s
🟢 gestures: long-press, then swipe left 9.1s
🟢 sticky chrome: accept below the fold, then continue 12.8s
🟢 async states: load, pull to refresh, claim through the toast 11.7s
🟢 huge virtualized list reaches row 512 2m 30s
🟢 web view coupon form applies the code shown on the page 12.9s
🟢 tests/flows.e2e.ts · 7 passed 1m 9s
🟢 text inputs unlock the submit button 12.1s
🟢 filled text inputs report their values 10.3s
🟢 modal flow confirms through the native alert 10.2s
🟢 bottom tabs act inside the last tab 7.4s
🟢 error recovery retries, then confirms the delete 9.4s
🟢 choice controls place the exact order 7.5s
🟢 debounced search selects the target once results arrive 11.7s
🟢 tests/login-form.e2e.ts · 3 passed 44.9s
🟢 login form › rejects a malformed email 12.8s
🟢 login form › rejects an unknown account 13.1s
🟢 login form › signs in and logs out 19.0s
🟢 tests/smoke.e2e.ts · 1 passed 9.1s
🟢 home lists the scenarios and opens one 9.1s

Tested by e2e 0.23.1 · 6m 45s · ios-simulator · run artifacts

@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e android: 57 passed, 1 skipped

All 58 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 24 passed, 1 skipped 2m 38s
🟢 control inventory › a label query resolves a labeled text field 21.6s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 11.8s
🟢 control inventory › a placeholder query and attribute reads find the field 5.2s
🟢 control inventory › a secure field takes a fill and denies value reads 6.1s
🟢 control inventory › press sends a character and Enter through the soft keyboard 7.7s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 7.5s
🟢 control inventory › toBeFocused follows the focus 7.7s
🟢 control inventory › a switch checks, unchecks, and reports its state 7.8s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 6.6s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 9.8s
🟢 control inventory › a tab role answers getByRole 3.4s
🟢 control inventory › a radio group reports the checked option 7.3s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 7.0s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.2s
🟢 control inventory › a header answers its accessible name 3.6s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.2s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.3s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.4s
🟢 control inventory › a double tap lands inside the 300 ms window 6.5s
🟢 control inventory › an ambiguous locator fails at once 4.4s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 7.0s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.6s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 12s
🟢 device fixture › setOrientation rotates the window the app measures 5.0s
🟢 device fixture › home leaves the app, openApp brings it back where it was 4.3s
🟢 device fixture › foregroundApp sees the home screen after home() 4.0s
🟢 device fixture › closeApp then openApp relaunches on the home list 6.3s
🟢 device fixture › closeApp leaves the next commands on the same device 6.0s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 3.9s
🟢 device fixture › device.back pops the scenario 4.2s
🟢 device fixture › unstable_raw reaches the test session through the raw driver 3.5s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 15.3s
🟢 device fixture › setLocation and clearLocation change what the app reads 19.8s
🟢 tests/device.e2e.ts · 5 passed 38.8s
🟢 permission prompt › allowing the system dialog enables the microphone 5.8s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 12.3s
🟢 sequential onboarding echoes the exact values 9.5s
🟢 onboarding rejects a malformed email before advancing 5.2s
🟢 dismissing the keyboard over choice controls leaves them as they were 6.1s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 5s
🟢 infinite scroll reaches the target item 27.4s
🟢 product catalog adds exactly two of the right variant 8.3s
🟢 gestures: long-press, then swipe left 7.5s
🟢 gestures: double-tap completes the flow 9.2s
🟢 sticky chrome: accept below the fold, then continue 10.3s
🟢 async states: load, pull to refresh, claim through the toast 11.9s
🟢 huge virtualized list reaches row 512 1m 51s
🟢 tests/flows.e2e.ts · 7 passed 47.3s
🟢 text inputs unlock the submit button 6.0s
🟢 filled text inputs report their values 5.4s
🟢 modal flow confirms through the native alert 8.4s
🟢 bottom tabs act inside the last tab 6.6s
🟢 error recovery retries, then confirms the delete 8.2s
🟢 choice controls place the exact order 4.9s
🟢 debounced search selects the target once results arrive 7.9s
🟢 tests/login-form.e2e.ts · 3 passed 25.5s
🟢 login form › rejects a malformed email 6.8s
🟢 login form › rejects an unknown account 6.8s
🟢 login form › signs in and logs out 11.8s
🟢 tests/smoke.e2e.ts · 1 passed 4.8s
🟢 home lists the scenarios and opens one 4.8s

Tested by e2e 0.23.1 · 8m 53s · android-emulator · run artifacts

@github-actions

Copy link
Copy Markdown
Contributor Author

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 12 replayed from cache · 66 model calls · 422.0k tokens (82% cached) · $0.0285

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 66 calls 9m 25s
🟢 act completes Login Form 1 step 38.7s
🟢 act completes Infinite Scroll List 1 step 39.6s
🟢 act completes Modal Flow 1 step 9.6s
🟢 act completes Bottom Tabs 1 step 7.8s
🟢 act completes Text Input Variations 1 step 13.1s
🟢 act completes Broken Accessibility 2 steps · 8 calls 44.0s
🟢 act completes Vision Only 2 steps · 6 calls 30.0s
🟢 act completes Gestures 1 step · 9 calls 41.1s
🟢 act completes Async States 1 step · 14 calls 1m 0s
🟢 act completes Debounced Search 1 step · 1 call 27.7s
🟢 act completes Flattened Registration Form 2 steps · 1 call 27.2s
🟢 act completes Flattened Login 2 steps · 10 calls 54.5s
🟢 act completes Sticky Chrome Target 1 step 13.1s
🟢 act completes WebView Accessibility 1 step 12.7s
🟢 act completes Permission Prompt 1 step 12.2s
🟢 act completes Photo Picker 1 step · 2 calls 31.3s
🟢 act completes Product Catalog 1 step 10.7s
🟢 act completes Error Recovery 1 step 9.8s
🟢 act completes Choice Controls 1 step · 7 calls 38.6s
🟢 act completes Stripe PaymentSheet 1 step · 8 calls 31.6s
🟢 act completes Apple Pay 1 step 11.9s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.1 · 9m 26s · ios-simulator · run artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant