diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 7b87c7f3c..0d155b834 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -320,7 +320,7 @@ Upstream's `4982d8d` (#1333, ported with #1224) makes the browser engine's seman `skills/migrate-to-e2e/` ports upstream's third skill (`e6885dc`, `feat(skills): add migrate-to-e2e (#1109)`, and `6609428`). Install it with `npx skills add hardkoded/e2e-dotnet --skill migrate-to-e2e`; like upstream, it is not bundled in `E2E.Cli`. Differences: - The source runners are Playwright .NET, Selenium .NET, and Appium (the .NET client or WebdriverIO), with upstream's Node ones (Playwright, Cypress, Detox, Maestro) kept. The `Appium` row of the detection table and its `appium` guide are upstream's (tester-army/e2e#1597); the port adds `Appium.WebDriver` in a `.csproj`. The e2e side is an NUnit or xUnit v3 test project run with `dotnet test`, in place of `e2e.config.ts`, `*.e2e.ts` files, `npx e2e init` (the port's `e2e init` creates the test project), `e2e run --repeat-each 3`, and `e2e list`. -- The migration guides are ported: see [Migration guides](#migration-guides). The skill reads the guide for the source runner first, as upstream's does, and treats "Not yet", "No equivalent", and `missing` as gap markers (tester-army/e2e#1611): a step with a workaround takes it, one without stays in the old runner. The TesterArmy section of `SKILL.md` keeps its inlined MCP commands and agent-call names in C#, and points to the `testerarmy` guide's "Port the project" and "Map each step" tables. The hand-over offers an issue at `hardkoded/e2e-dotnet` in place of `e2e feedback`. +- The migration guides are ported: see [Migration guides](#migration-guides). The skill reads the guide for the source runner first, as upstream's does, and treats the guide's "What e2e doesn't have yet" section as the gap list (tester-army/e2e#1611, #1714): a step with a workaround takes it, one without stays in the old runner. The TesterArmy section of `SKILL.md` keeps its inlined MCP commands and agent-call names in C#, and points to the `testerarmy` guide's "Port the project" and "Map each step" tables. The hand-over offers an issue at `hardkoded/e2e-dotnet` in place of `e2e feedback`. - Not ported: the plugin copy (`plugins/e2e/skills/migrate-to-e2e`), since the port has no plugin; the `AGENTS.md` and `CONTRIBUTING.md` lines about it map to `CONTRIBUTING.md` here. - The harness step runs `e2e doctor` and fixes its checks, then `--check-model` when a model is configured (upstream's `npx e2e doctor`, tester-army/e2e#1653). Upstream's note that the doctor needs e2e 0.23.0 or newer is an npm version, so the step names none. - Upstream added no tests for this skill, so none are ported. @@ -328,14 +328,14 @@ Upstream's `4982d8d` (#1333, ported with #1224) makes the browser engine's seman ## Migration guides -`docs-site/src/content/docs/migrate/` ports upstream's `docs/migrate/*.mdx` (`index`, `playwright`, `selenium`, `cypress`, `appium`, `detox`, `maestro`, `testerarmy`) as read at `7b1b8b27`, with `index`, `detox`, and `maestro` brought to `a016b3e4` (tester-army/e2e#1597, #1579, #1602, #1610, #1618; `8537abc7`, tester-army/e2e#1441; `b81b7d75`, tester-army/e2e#1411; `d52e9b01`, tester-army/e2e#1468; `a60310f9`, tester-army/e2e#1508; `f6d0679f`, tester-army/e2e#1361), and `appium` to `e83fdf9` (tester-army/e2e#1651). They keep upstream's file names, headings, and order. Each guide's tables use the port's C# names, and the sidebar has a Migrate group. Differences: +`docs-site/src/content/docs/migrate/` ports upstream's `docs/migrate/*.mdx` (`index`, `playwright`, `selenium`, `cypress`, `appium`, `detox`, `maestro`, `testerarmy`) as read at `7b1b8b27`, with `index`, `detox`, and `maestro` brought to `a016b3e4` (tester-army/e2e#1597, #1579, #1602, #1610, #1618; `8537abc7`, tester-army/e2e#1441; `b81b7d75`, tester-army/e2e#1411; `d52e9b01`, tester-army/e2e#1468; `a60310f9`, tester-army/e2e#1508; `f6d0679f`, tester-army/e2e#1361), and `appium` to `e83fdf9` (tester-army/e2e#1651). Every guide then takes the "What e2e doesn't have yet" rewrite of `9c9c01f` (tester-army/e2e#1714): Planned, Not yet, and Not planned tables with a workaround, "What differs on purpose", and `Defaults that differ` where upstream has them. `skills/migrate-to-e2e/SKILL.md` points to that section. A row's status follows the port where it differs from upstream: `--shard`, `--watch`, `--step`, the trace page, and `describe` options (`serial`, `.only`) are Not yet in the port, so the rows that use them name the framework's equivalent. They keep upstream's file names, headings, and order. Each guide's tables use the port's C# names, and the sidebar has a Migrate group. Differences: - The `playwright` guide maps Playwright for .NET (`PageTest`, `ContextOptions()`, `Page.GetByRole(AriaRole...)`), where upstream maps Node Playwright. Playwright .NET has no `playwright.config.ts`, so the config tables map `BrowserNewContextOptions`, `.runsettings`, and the environment variables instead of `testDir`, `webServer`, `projects`, and `reporter`. The `selenium` guide maps Selenium for .NET, where upstream's example is Java. The `cypress`, `detox`, `maestro`, and `testerarmy` guides keep their source runners and write the target in C#, in a new .NET test project beside the old suite. The `appium` guide maps the `mobile:` commands and WebdriverIO calls upstream lists, and shows the Appium .NET client (`Appium.WebDriver`) in its source examples where upstream shows WebdriverIO. Upstream's example project `docs/examples/appium/` (checked by `scripts/check-docs-examples.ts`) has no counterpart: the port's guides hold their snippets inline. - Every code sample has an NUnit form, and an xUnit v3 form where the two differ. A short NUnit and xUnit table in the overview replaces upstream's `test`, `describe`, hook, tag, and retry names. - Upstream's `SetupPrompt` card (a copyable prompt) is a `text` block named "Agent prompt". The prompts use `e2e init --yes`, `e2e guide`, `e2e login`, and `dotnet test --filter` in place of `npx e2e init --yes`, `npx e2e guide`, `npx e2e login`, and `npx e2e run`. - The Detox, Maestro, and Appium prompts have a step that runs `e2e doctor` (upstream's `npx e2e doctor`). Upstream's note that it needs e2e 0.23.0 or newer (tester-army/e2e#1615) is an npm version of upstream's package, so the port's prompts name none. - Upstream's `CardGroup` and `Card` components are lists of links, and `AccordionGroup` is a heading for each table. -- No counterpart in the port, marked `missing` or `by design` in the tables: `e2e run` and `e2e list` (use `dotnet test`), the `tests`, `workers`, `output`, `--shard`, `--watch`, `--changed-since`, `--repeat-each`, `--max-failures`, and `--update-snapshots` flags and keys (the framework owns selection, retries, and parallelism; `--last-failed` is the `E2E_LAST_FAILED` variable and the `LastFailed` member of the run fixture, see [Test filters](#test-filters)), `describe` options (`serial`, `skip`, `retries`, `timeout`), `test.extend` fixtures and `base.extend` page objects as fixtures (a page object takes `Screen` and `Browser` in a `[SetUp]` or `InitializeAsync`), several targets in one run (the port runs one), the HTML and markdown run summaries, `unique()` as a free function (`Values.Unique`), `credentials` entries for TOTP (`secrets` hold values, not code), and the `inbox` credential. A row names a feature with `Not yet` where the port has no way to do it, such as `.tapBackspaceKey()`, a pinch, or `device.shake()`. The tap `position`, `Screen.TapAtAsync`, and the screen-level `SwipeAsync` are ported and the device engine declares their pointer actions, so the Detox, Maestro, and Appium rows map to them. +- No counterpart in the port, listed under "What e2e doesn't have yet" in each guide: `e2e run` and `e2e list` (use `dotnet test`), the `tests`, `workers`, `output`, `--shard`, `--watch`, `--changed-since`, `--repeat-each`, `--max-failures`, and `--update-snapshots` flags and keys (the framework owns selection, retries, and parallelism; `--last-failed` is the `E2E_LAST_FAILED` variable and the `LastFailed` member of the run fixture, see [Test filters](#test-filters)), `describe` options (`serial`, `skip`, `retries`, `timeout`), `test.extend` fixtures and `base.extend` page objects as fixtures (a page object takes `Screen` and `Browser` in a `[SetUp]` or `InitializeAsync`), several targets in one run (the port runs one), the HTML and markdown run summaries, `unique()` as a free function (`Values.Unique`), `credentials` entries for TOTP (`secrets` hold values, not code), and the `inbox` credential. A row names a feature with `Not yet` where the port has no way to do it, such as `.tapBackspaceKey()`, a pinch, or `device.shake()`. The tap `position`, `Screen.TapAtAsync`, and the screen-level `SwipeAsync` are ported and the device engine declares their pointer actions, so the Detox, Maestro, and Appium rows map to them. - The `detox`, `maestro`, and `testerarmy` rows that use `ToHaveScreenshotAsync` on a device say the port documents the matcher for the web engine only. Nothing in the port runs it on a device. - The prompt's step to port `.detoxrc.js` and `config.yaml` into one device target relies on `E2E_MOBILE_PLATFORM` and `E2E_MOBILE_APP_PATH` to switch between iOS and Android, since `e2e.config.json` holds one target (see [MobileAppTest](#mobileapptest-not-in-upstream)). - The snippets were written against the port's public API and compiled by reading, not run. The docs site build checks the pages, not the C#. diff --git a/docs-site/src/content/docs/migrate/appium.mdx b/docs-site/src/content/docs/migrate/appium.mdx index ad019c6a1..ceb397512 100644 --- a/docs-site/src/content/docs/migrate/appium.mdx +++ b/docs-site/src/content/docs/migrate/appium.mdx @@ -5,23 +5,23 @@ description: Move Appium mobile tests, from the .NET client or WebdriverIO, to C This guide ports upstream's `docs/migrate/appium.mdx` (tester-army/e2e at `e83fdf9`, from tester-army/e2e#1597, #1579, #1602, and #1651). Upstream's examples are WebdriverIO and its target is TypeScript. The port's target is a .NET test project with a `MobileAppTest` base class, and the source examples use the Appium .NET client (`Appium.WebDriver`). The tables name the WebdriverIO calls and the `mobile:` commands every client sends. A .NET call maps the same way: `driver.FindElement(AppiumBy.AccessibilityId("x")).Click()` is `$('~x').click()`, `SendKeys` is `setValue`, `driver.ActivateApp(id)` is `driver.activateApp(id)`, and `driver.ExecuteScript("mobile: ...")` is `driver.execute('mobile: ...')`. -e2e replaces the Appium server, its drivers, and the client library with the `E2E.Mobile.NUnit`, `E2E.Mobile.XUnit.V3`, or `E2E.Mobile.TUnit` package. It drives iOS simulators, Android emulators, and connected phones through [agent-device](https://github.com/callstack/agent-device), which it starts through `npx`, so Node 22.12 or newer must be on `PATH`. Your app build and its accessibility ids stay as they are. +e2e replaces the Appium server, its drivers, and the client library with the `E2E.Mobile.NUnit`, `E2E.Mobile.XUnit.V3`, or `E2E.Mobile.TUnit` package. It drives iOS simulators, Android emulators, and connected phones through [agent-device](https://github.com/callstack/agent-device), which it starts through `npx`, so Node 22.12 or newer must be on `PATH`. Capabilities become a target in `e2e.config.json`, and `driver` commands become the `Device` and `App` members. Your app build and its accessibility ids stay as they are. -This guide is for suites that test a mobile app through Appium's XCUITest or UiAutomator2 driver, from the .NET client, WebdriverIO, or another client. +The source examples use the Appium .NET client. A WebdriverIO, Java, or Python client maps the same way. ## Agent prompt ```text Migrate this project's Appium tests to e2e, one test at a time. Docs: the "From Appium" page of the e2e-dotnet docs site (docs-site/src/content/docs/migrate/appium.mdx). -1. Read the migration guide. It maps Appium capabilities, selectors, and commands to e2e and marks what has no equivalent. +1. Read the migration guide. It maps Appium capabilities, selectors, and commands to e2e, and lists what e2e doesn't have yet. 2. Ask me where the .NET test project should live and whether to use NUnit or xUnit v3. Run `e2e init --yes --engine mobile` (add `--framework xunit` for xUnit v3). It creates the project with E2E.Mobile.NUnit (or E2E.Mobile.XUnit.V3) and an example test. Rewrite the example for your app or delete it. 3. Read the e2e skill (`e2e guide` prints it, `e2e guide mobile` prints the mobile topic) and follow it for every step below. 4. If you ran init, ask me which model to use for agent steps. If it needs a sign-in, give me the `e2e login ` command and wait until I confirm. 5. Port the capabilities into e2e.config.json with the guide's table: one target for the platform, with app.bundleId, app.appPath, app.launchArguments, and app.permissions. Keep passwords and tokens out of the config. 6. Run `e2e doctor` and fix what it reports. 7. List the Appium tests and ask me which one to move first. Suggest one with XPath selectors or explicit waits that often break. -8. Rewrite that test as a test class that derives from MobileAppTest, with the guide's tables. Follow "Rewrite selectors": XPath has no port, and e2e fails on several matches. Where the guide says "Not yet" or "No equivalent", use the workaround it gives, or leave that test in Appium and tell me why. +8. Rewrite that test as a test class that derives from MobileAppTest, with the guide's tables. Follow "Rewrite selectors": XPath has no port, and e2e fails on several matches. Before you port a test, check the guide's "What e2e doesn't have yet" section for each capability and command the test uses. Use the "Use today" workaround, or leave that test in Appium and tell me why. 9. Keep the final assertions deterministic, with Expect.That. Where a run of scripted taps breaks often, propose an Agent.ActAsync goal for it instead. 10. Run the new class on a booted simulator or emulator with `dotnet test --filter` and fix it until it passes. Keep the Appium test until I confirm it can go. 11. Summarize which tests moved, which stayed in Appium, and why. @@ -29,46 +29,24 @@ Migrate this project's Appium tests to e2e, one test at a time. Docs: the "From For a migration that spans several sessions, the `migrate-to-e2e` skill records what moved in a progress file and keeps the old suite running until you retire each test. See [Coding agents](/coding-agents/#migrate-an-existing-suite). -## What replaces each Appium part - -| Appium | e2e | -| --- | --- | -| The Appium server, started before the run | No server. The engine starts agent-device's daemon | -| Drivers (`appium driver install xcuitest`) | `platform` on the target: `ios` or `android`. Nothing else to install | -| Capabilities | A target in `e2e.config.json`: the platform, plus `app` | -| A session (`remote()`, `deleteSession()`, `new AndroidDriver(...)`, `Quit()`) | One device session per worker, which `MobileAppTest` opens and closes | -| Selector strategies (accessibility id, id, XPath) | `Screen.GetByTestId`, `GetByLabel`, `GetByText`, `GetByRole` | -| Element commands (`click`, `setValue`) | Locator actions (`TapAsync`, `FillAsync`), which wait for one actionable match | -| `driver` and `mobile:` commands | `Device` and `App` | -| expect-webdriverio, implicit and explicit waits | `Expect.That(locator)` matchers, which retry until their timeout | -| The WebdriverIO testrunner, Mocha, TestNG, pytest, NUnit | An NUnit or xUnit v3 class and `dotnet test` | -| Nothing in Appium | Agent steps: `Agent.ActAsync`, `AssertAsync`, and `ExtractAsync` take a goal in plain words | - ## Port the capabilities Set up a target with the [mobile guide](/get-started/mobile/) and a model with the [web guide](/get-started/web/). Only agent steps need a model. -On this page, "Not yet" marks a capability or command e2e doesn't have today, and "No equivalent" one it leaves out on purpose. - -| Capability | e2e | +| Appium | e2e | | --- | --- | -| `platformName` | `platform` on the target (`ios` or `android`), or `E2E_MOBILE_PLATFORM`. The port runs one device target, so the variable switches it | -| `appium:automationName` | Nothing to set | -| `appium:deviceName`, `appium:udid` | `MobileOptions.Device`, or `E2E_MOBILE_DEVICE`: a name, a simulator UDID, or an `emulator-5554` serial. `MobileOptions.Devices` is a pool, one worker per device | -| `appium:platformVersion` | Not yet. Name a simulator or emulator that runs that version | -| `appium:app` | `app.appPath` on the target. `MobileAppTest` installs the build once per run with `Device.InstallAppAsync()` | +| `platformName`, `appium:automationName` | `platform` on the target (`ios` or `android`), or `E2E_MOBILE_PLATFORM`. The port runs one device target, so the variable switches it. Nothing to set for the automation name | +| `appium:deviceName`, `appium:udid` | `MobileOptions.Device`, or `E2E_MOBILE_DEVICE`: a name, a simulator UDID, or an `emulator-5554` serial. A phone goes by its name. `MobileOptions.Devices` is a pool, one worker per device | +| `appium:app` | `app.appPath` on the target. `MobileAppTest` installs the build once per run with `Device.InstallAppAsync()`, see [Point at your app](/get-started/mobile/#2-point-at-your-app) | | `appium:bundleId`, `appium:appPackage` | `app.bundleId` on the target, the app `App.OpenAsync()` launches | -| `appium:appActivity`, `appium:appWaitActivity` | Not yet. e2e opens the app by its package | -| `appium:processArguments`, `appium:optionalIntentArguments` | `app.launchArguments` on the target, or `OpenAppOptions.LaunchArguments` for one launch: a list of strings, process arguments on iOS, `am start` arguments on Android. `processArguments.env` is not yet supported | -| `appium:autoGrantPermissions`, `appium:permissions` | `app.permissions` on the target, `{ "camera": "grant" }`, set before every fresh launch; nothing is granted unless named | -| `appium:autoAcceptAlerts`, `appium:autoDismissAlerts` | Not yet. Set `app.permissions` so the prompt never shows, or call `Device.AlertAsync(AlertAction.Accept)` | -| `appium:noReset: true` | The default. Nothing clears the app's data unless a test asks | -| `appium:noReset: false` | `App.ClearStateAsync()` at the start of the test | +| `appium:processArguments`, `appium:optionalIntentArguments` | `app.launchArguments` on the target, or `OpenAppOptions.LaunchArguments` for one launch, a list of strings | +| `appium:permissions` | `app.permissions` on the target, `{ "camera": "grant" }`, set before every fresh launch. Each `DevicePermission` name takes `grant`, `deny`, or `reset` | +| `appium:noReset` | `true` is the default, so nothing clears the app's data. For `false`, call `App.ClearStateAsync()` at the start of the test | | `appium:fullReset` | `Device.InstallAppAsync(options: new InstallAppOptions { Reinstall = true })` | | `appium:newCommandTimeout`, WebDriverAgent timeouts | Not needed. Actions use `actionTimeout`, assertions `assertionTimeout`, and the test its own timeout | -| `appium:language`, `appium:locale` | Not yet | +| `appium:platformVersion`, `appium:language`, `appium:locale` on a hosted device | `OsVersion`, `Language`, and `Locale` on `BrowserStackDevicesOptions`; `Android.OsVersion` on `LimrunSimulatorsOptions`. `E2E.TestMu.Mobile` has none of them | | `bstack:options`, `lt:options` | A device provider: `E2E.BrowserStack`, see [BrowserStack](/integrations/browserstack/), or `E2E.TestMu.Mobile` | -| Several capability sets, `maxInstances` | One target, switched with `E2E_MOBILE_PLATFORM`, and a `MobileOptions.Devices` pool | +| Several capability sets, `maxInstances` | One target, switched with `E2E_MOBILE_PLATFORM`, and a `MobileOptions.Devices` list with one worker per device | ```json title="e2e.config.json" { @@ -130,7 +108,7 @@ public sealed class LoginTests : MobileAppTest For xUnit v3, use `E2E.Mobile.XUnit.V3` and `[Fact]`. -Where a run of scripted taps breaks often, one goal can replace it. The agent reads the screen and finds the fields. The replay cache records its actions once a later check passes, and the next run replays them without a model call, see [Caching](/features/caching/). A password belongs in [configured credentials](/features/signing-in/), which the model never sees: +Where a run of scripted taps breaks often, one `Agent.ActAsync` goal can replace it. The [replay cache](/features/caching/) records its actions once a later check passes, and the next run replays them without a model call. A password belongs in [configured credentials](/features/signing-in/), which the model never sees: ```csharp var ada = Credentials.User("ada"); @@ -149,117 +127,79 @@ await Agent.ActAsync("sign in as {username}", new ActOptions | `$('id=com.example:id/email')`, `AppiumBy.Id`, `resourceId` | `Screen.GetByTestId("email")`; the `com.example:id/` prefix is optional | | `-ios predicate string:label == "Sign in"`, `android=new UiSelector().text("Sign in")` | `Screen.GetByText("Sign in")` or `GetByLabel("Sign in")` | | `-ios class chain`, class names (`XCUIElementTypeButton`, `android.widget.Button`) | `Screen.GetByRole("button", "Sign in")` | -| XPath | No equivalent. Chain queries (`Screen.GetByTestId("row").GetByText("Delete")`), or use `Filter(has: ..., hasText: ...)` | | `$$(selector)[2]`, `FindElements(...)[2]` | `.Nth(2)`; `AllAsync()` returns every match | -| `-image` selectors | No equivalent. Use a goal, or `Screen.TapAtAsync(new Point(x, y))` | | A selector the queries can't express | `Device.Locator("role=text label=\"Total\"")`, agent-device's selector syntax | -Three differences from Appium cause most edits: - - `GetByText`, `GetByLabel`, and `GetByTestId` match the whole string, case included. Pass a `Regex`, or `exact: false` for a case-insensitive substring of text or label. -- Appium's `$()` and `FindElement` return the first match. An e2e action or single-element read needs exactly one, and fails with `LOCATOR_AMBIGUOUS` on several. Narrow it with `Filter(...)`, `First()`, or `Nth(n)`. -- Appium reads the tree in each platform's own terms, as the iOS class in the XPath above shows. e2e maps native controls to shared roles such as `button`, `textbox`, and `switch`, so one test runs on both platforms. The [roles table](/get-started/mobile/#roles) lists the mapping. +- Appium's `$()` and `FindElement` return the first match. An e2e action needs exactly one and fails with `LOCATOR_AMBIGUOUS` on several. Narrow it with `Filter(...)`, `First()`, or `Nth(n)`. +- Roles such as `button`, `textbox`, and `switch` are the same on iOS and Android, so one test runs on both. See [Roles](/get-started/mobile/#roles). ## Map each command Calls on `Device` run only on a device. Calls on `Screen`, `App`, `Agent`, and `Expect.That(locator)` run on every target. -### Element commands +### Elements and gestures -| Appium (WebdriverIO) | e2e | +| Appium | e2e | | --- | --- | -| `.click()` | `.TapAsync()` | -| `.doubleClick()` | `.DoubleTapAsync()` | -| `mobile: longClickGesture`, `mobile: touchAndHold` | `.LongPressAsync(new LongPressOptions { Duration = ... })` | -| `.setValue(t)` | `.FillAsync(t)`, which replaces the content | -| `.addValue(t)` | `.PressSequentiallyAsync(t)`, which taps the field and types key by key at the cursor | -| `.clearValue()` | `.ClearAsync()` | -| `driver.keys('Enter')`, `pressKeyCode(66)` | `.PressAsync("Enter")` on the field; a device takes only `Enter`, `Space`, or one character | -| `.getText()` | `.TextContentAsync()` | -| `.getValue()`, `.getAttribute('value')` | `.InputValueAsync()` | -| `.getAttribute(name)` | `.GetAttributeAsync("placeholder")`; `placeholder` is the only attribute on a device | -| `.isDisplayed()`, `.isEnabled()` | `.IsVisibleAsync()`, `.IsEnabledAsync()` | -| `.isSelected()` | `await Expect.That(locator).ToBeSelectedAsync()`; there is no selected read | -| `.getAttribute('checked')` | `.IsCheckedAsync()` | +| `.click()`, `.doubleClick()` | `.TapAsync()`, `.DoubleTapAsync()` | +| `mobile: longClickGesture`, `mobile: touchAndHold` on an element | `.LongPressAsync(new LongPressOptions { Duration = ... })` | +| `.setValue(t)`, `.clearValue()` | `.FillAsync(t)`, which replaces the content, and `.ClearAsync()` | +| `.addValue(t)` | `.PressSequentiallyAsync(t)`, which types key by key at the cursor | +| `driver.keys('Enter')`, `pressKeyCode(66)` | `.PressAsync("Enter")` on the field | +| `.getText()`, `.getValue()`, `.getAttribute('value')` | `.TextContentAsync()`, `.InputValueAsync()` | +| `.isDisplayed()`, `.isEnabled()`, `.getAttribute('checked')` | `.IsVisibleAsync()`, `.IsEnabledAsync()`, `.IsCheckedAsync()` | | `.isExisting()` | `await locator.CountAsync() > 0` | | `.getLocation()`, `.getSize()` | `.BoundingBoxAsync()` | | `.waitForDisplayed({ timeout })` | `await Expect.That(locator).ToBeVisibleAsync(timeout: ...)`, or `locator.WaitForAsync()` | | `.waitForDisplayed({ reverse: true })` | `await Expect.That(locator).ToBeHiddenAsync()` | - -### Gestures - -| Appium (WebdriverIO) | e2e | -| --- | --- | -| `mobile: swipeGesture`, `mobile: swipe`, `driver.swipe({ direction: 'up' })` | `locator.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })` or `Screen.SwipeAsync(new SwipeOptions { ... })`; Appium names the finger's direction, e2e the content's | +| `mobile: swipeGesture`, `mobile: swipe`, `driver.swipe({ direction: 'up' })` | `locator.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })` or `Screen.SwipeAsync(new SwipeOptions { ... })`. e2e names the content's direction, not the finger's | | `mobile: scrollGesture`, `mobile: scroll` to an element, `.scrollIntoView()` | `Screen.ScrollUntilVisibleAsync(locator)`, or `list.ScrollUntilVisibleAsync(locator)` on the scroll view | -| `mobile: dragGesture`, `mobile: dragFromToForDuration`, `.dragAndDrop(target)` | `locator.DragToAsync(target)` | +| `.dragAndDrop(target)`, a drag that ends on an element | `locator.DragToAsync(target)` | | A `performActions` swipe between two points | `Screen.SwipeAsync(new SwipePathOptions { From = new Point(x1, y1), To = new Point(x2, y2) })` | | A tap at coordinates | `Screen.TapAtAsync(new Point(x, y))`; inside an element, `.TapAsync(new TapOptions { Position = new Point(x, y) })` | -| `mobile: pinchOpenGesture`, `mobile: pinch`, multi-finger `performActions` | Not yet | -### Assertions +### App and device -| expect-webdriverio | e2e | -| --- | --- | -| `toBeDisplayed()`, `not.toBeDisplayed()` | `ToBeVisibleAsync()`, `ToBeHiddenAsync()` | -| `toExist()` | `ToBeAttachedAsync()` | -| `toHaveText(t)`, `toHaveText(t, { containing: true })` | `ToHaveTextAsync(t)`, `ToContainTextAsync(t)` | -| `toHaveValue(v)` | `ToHaveValueAsync(v)` | -| `toBeEnabled()`, `toBeDisabled()` | `ToBeEnabledAsync()`, `ToBeDisabledAsync()` | -| `toBeSelected()`, `toBeChecked()` | `ToBeSelectedAsync()`, `ToBeCheckedAsync()` | -| `toBeElementsArrayOfSize(n)` | `ToHaveCountAsync(n)` | -| `@wdio/visual-service` screen comparison | `Expect.That(Screen).ToHaveScreenshotAsync("name.png")`. The port documents it for the web engine only and has no device run of it, so keep the comparison until you have checked it on your device | - -### App lifecycle - -| Appium (WebdriverIO) | e2e | +| Appium | e2e | | --- | --- | | The launch at session start | `MobileAppTest` launches the target's app fresh before each test; `App.OpenAsync()` launches it again | | `driver.activateApp(id)` | `Device.OpenAppAsync(id)`; `new OpenAppOptions { Relaunch = true }` restarts it | -| `driver.terminateApp(id)` | `Device.CloseAppAsync()`, for the session's app only. It ends the session, so open an app next | +| `driver.terminateApp(id)` on the app under test | `Device.CloseAppAsync()`. It ends the session, so open an app next | | `terminateApp`, then `activateApp` | `App.RestartAsync()` | -| `mobile: clearApp` | `App.ClearStateAsync()` | -| `driver.installApp(path)` | `Device.InstallAppAsync(path)` | -| `driver.removeApp(id)` | Not yet. `Device.UnstableRaw.Client.CallAsync(DeviceMethods.AppsUninstall, ...)` with `app` set to the id removes it, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes); `Device.InstallAppAsync(options: new InstallAppOptions { Reinstall = true })` removes the app first and installs it with no data (not on TestMu) | -| `driver.isAppInstalled(id)` | Not yet. `Device.UnstableRaw.Client.CallAsync(DeviceMethods.AppsList, Device.UnstableRaw.Selection.ToArgs(), CancellationToken.None)` lists the installed user apps as `Name (id)`; on Android, only apps with a launcher icon | -| `queryAppState(id)` | Not yet. On Android, `Device.ForegroundAppAsync()` names the app on screen | +| `mobile: clearApp`, `driver.installApp(path)` | `App.ClearStateAsync()`, `Device.InstallAppAsync(path)` | | `driver.background(seconds)` | `Device.HomeAsync()`, `await Task.Delay(TimeSpan.FromSeconds(seconds))` for a positive `seconds`, then `Device.OpenAppAsync(id)` | | `driver.url('myapp://x')`, `mobile: deepLink` | `Device.OpenLinkAsync("myapp://x")` | -| `driver.back()` | `App.BackAsync()` | -| `driver.hideKeyboard()` | `Device.DismissKeyboardAsync()` | +| `driver.back()`, `driver.hideKeyboard()`, `pressKeyCode` for Home | `App.BackAsync()`, `Device.DismissKeyboardAsync()`, `Device.HomeAsync()` | | `driver.getPageSource()` | `Screen.TreeAsync()`, or `Screen.DebugAsync()` to print it | | `driver.takeScreenshot()` | `App.ScreenshotAsync("label")`, secure fields masked | | `startRecordingScreen()`, `stopRecordingScreen()` | `App.StartRecordingAsync()`, `App.StopRecordingAsync()`, or `E2E_VIDEO=on dotnet test` | - -### Device - -| Appium (WebdriverIO) | e2e | -| --- | --- | | `mobile: setPermission`, `mobile: changePermissions` | `Device.SetPermissionAsync(DevicePermission.Camera, PermissionState.Grant)`, one per call, on the target's app, which need not be running; call it before the app launches | | `driver.acceptAlert()`, `driver.dismissAlert()` | `Device.AlertAsync(AlertAction.Accept)`, `Device.AlertAsync(AlertAction.Dismiss)` | | `driver.setGeoLocation({ latitude, longitude })` | `Device.SetLocationAsync(new Coordinates(latitude, longitude))` | -| `driver.setOrientation('LANDSCAPE')` | `Device.SetOrientationAsync(DeviceOrientation.LandscapeLeft)` | -| `mobile: setAppearance` | `Device.SetAppearanceAsync(Appearance.Dark)` | -| `driver.setClipboard(t)`, `driver.getClipboard()` | `Device.SetClipboardAsync(t)`, `Device.ClipboardAsync()`, plain text where Appium uses base64. Neither works on an Android build with no clipboard shell command | -| `driver.toggleWiFi(false)`, `mobile: setConnectivity` with `wifi: false` | `Device.SetNetworkAsync(NetworkState.Offline)`, which turns off Wi-Fi only; Android only | +| `driver.setOrientation('LANDSCAPE')`, `mobile: setAppearance` | `Device.SetOrientationAsync(DeviceOrientation.LandscapeLeft)`, `Device.SetAppearanceAsync(Appearance.Dark)` | +| `driver.setClipboard(t)`, `driver.getClipboard()` | `Device.SetClipboardAsync(t)`, `Device.ClipboardAsync()`, plain text where Appium uses base64 | +| `driver.toggleWiFi(false)`, `mobile: setConnectivity` with `wifi: false` | `Device.SetNetworkAsync(NetworkState.Offline)`, Wi-Fi only; Android only | | `driver.toggleAirplaneMode()`, `mobile: setConnectivity` with `airplaneMode: true` | `Device.SetAirplaneModeAsync(true)`, Wi-Fi and cellular; Android 11 or later | -| `mobile: enrollBiometric` | `Device.EnrollBiometricsAsync(BiometricSensor.FaceId, true)` | -| `mobile: sendBiometricMatch`, `driver.fingerPrint(1)` | `Device.SetBiometricsAsync(BiometricSensor.FaceId, BiometricResult.Match)`, `Device.SetBiometricsAsync(BiometricSensor.Fingerprint, BiometricResult.Match)` | -| `pressKeyCode` for Home | `Device.HomeAsync()` | -| `driver.lock()`, `pressKeyCode` for power or volume, `mobile: pressButton` for volume | Not yet. `Device.UnstableRaw.Client.CallAsync(DeviceMethods.CommandHardwareButton, ...)` with `button` set to `"power"`, `"volume-up"`, or `"volume-down"` presses one; on iOS only the volume keys, on a physical device, not a simulator. See [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | +| `mobile: enrollBiometric`, `mobile: sendBiometricMatch`, `driver.fingerPrint(1)` | `Device.EnrollBiometricsAsync(BiometricSensor.FaceId, true)`, `Device.SetBiometricsAsync(BiometricSensor.FaceId, BiometricResult.Match)`, or `BiometricSensor.Fingerprint` | | `mobile: clearKeychains` | `Device.ClearKeychainAsync()`; iOS simulator only | | `driver.isIOS`, `driver.isAndroid` | the `Platform` property of `MobileAppTest`, `MobilePlatform.Ios` or `Android`, with `Assume.That` (NUnit) | -### Runner and tooling +### Assertions and runner -| Appium (WebdriverIO) | e2e | +| Appium | e2e | | --- | --- | +| `toBeDisplayed()`, `not.toBeDisplayed()` | `ToBeVisibleAsync()`, `ToBeHiddenAsync()` | +| `toExist()`, `toBeElementsArrayOfSize(n)` | `ToBeAttachedAsync()`, `ToHaveCountAsync(n)` | +| `toHaveText(t)`, `toHaveText(t, { containing: true })` | `ToHaveTextAsync(t)`, `ToContainTextAsync(t)` | +| `toHaveValue`, `toBeEnabled`, `toBeDisabled`, `toBeSelected`, `toBeChecked` | The same, with the `Async` suffix | +| `@wdio/visual-service` screen comparison | `Expect.That(Screen).ToHaveScreenshotAsync("name.png")`. The port documents it for the web engine only and has no device run of it, so keep the comparison until you have checked it on your device | +| `remote()`, `deleteSession()`, `new AndroidDriver(...)`, `Quit()` | Nothing to call. `MobileAppTest` opens and closes one device session per worker | | `specs` in `wdio.conf.ts` | the test project's own tests; `dotnet test` collects the classes in the project | | `specFileRetries`, Mocha `retries` | NUnit `[Retry(n)]`, where `n` counts total attempts (two Mocha retries are `[Retry(3)]`) and a retry reruns the method, not the file; xUnit v3 has no retry attribute | | Mocha `--grep`, tags in titles | `dotnet test --filter "FullyQualifiedName~Name"`, or `[Category("smoke")]` (NUnit) and `--filter "TestCategory=smoke"` | | `before`, `beforeEach` hooks | `[OneTimeSetUp]`, `[SetUp]` (NUnit); the constructor and `InitializeAsync` (xUnit v3) | | JUnit, Allure, spec reporters | the `dotnet test` loggers, such as `--logger trx` (`--report-trx` on Microsoft.Testing.Platform; see [Testing frameworks](/testing-frameworks/#vstest-and-microsofttestingplatform)), or a JUnit logger package | -| Appium Inspector | `Screen.DebugAsync()`, or `e2e mcp` for a coding agent's live session, see [Coding agents](/coding-agents/) | ## Permissions and deep links @@ -292,11 +232,11 @@ public sealed class OrderLinkTests : MobileAppTest } ``` -Set a permission before `App.OpenAsync()`: a change terminates a running app on iOS. Permissions every test needs go in `app.permissions`, and so does one the app reads at startup, since `MobileAppTest` has launched the app before the test body runs. See [Mobile](/get-started/mobile/). +Set a permission before `App.OpenAsync()`, since a change stops a running app on iOS. Permissions every test needs go in `app.permissions`, and so do those the app reads at startup, since `MobileAppTest` has launched the app before the test body runs. ## Run in CI -An Appium job starts the server with `appium &`, then the test run. An e2e job has no server. `MobileAppTest` installs the build from `app.appPath`, and `dotnet test` runs the suite on the booted simulator or emulator: +With no server to start, an e2e job lets `MobileAppTest` install the build from `app.appPath`, and `dotnet test` runs the suite on the booted simulator or emulator: ```bash xcrun simctl boot "iPhone 17" @@ -306,23 +246,84 @@ dotnet test Each platform needs its own job: iOS on macOS, Android on Linux with KVM. Boot exactly one device, or name it with `E2E_MOBILE_DEVICE`, see [Mobile](/get-started/mobile/#4-run). -## Migration checklist +## What e2e doesn't have yet + +Planned rows are on the roadmap, and Not yet rows have no date. Not planned rows are left out on purpose. A call through `Device.UnstableRaw` is not a step, so it skips the report and secret redaction, and it takes only the calls the engine makes, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes). -1. Install e2e in a new test project beside the Appium project, or in an `e2e/` folder for a Java or Python suite. Run `e2e doctor`. -2. Port the capabilities to the target. -3. Rewrite one flaky test. Replace XPath with test ids, labels, and roles, and add an id where the app has none. -4. Run it with `dotnet test --filter` on each platform until it passes. -5. Add the e2e job to CI beside the Appium job. -6. Delete the Appium test, and repeat. After the last one, remove the Appium server and drivers from CI. +### Planned -## What has no direct equivalent +#### Capabilities and devices -The tables above mark single commands. These gaps cut across a whole suite: +| Appium | Use today | +| --- | --- | +| `appium:autoGrantPermissions`, Android permissions off the list, iOS location `always` | List each permission in `app.permissions`, or run `adb shell pm grant` before the run. For iOS location `always`, tap the prompt's button | +| Device settings and alerts on a BrowserStack or TestMu device | Tap the prompt as UI. Run those tests on a local device | +| `bstack:options.local`, `lt:options.tunnel` | Point the app at a backend the hosted device can reach | +| `appium:appActivity`, `appWaitActivity`, `startActivity()` | `Device.OpenLinkAsync(url)`, or `DeviceMethods.AppsOpen` through `Device.UnstableRaw` | +| `appium:processArguments.env` | Pass a launch argument and read it in the app | +| `appium:language`, `appium:locale` on a local device | On iOS, `LaunchArguments = ["-AppleLanguages", "(fr)", "-AppleLocale", "fr_FR"]`, see [Run the app in another language](/get-started/mobile/#run-the-app-in-another-language). On Android, set the device language before the run | +| `appium:disableWindowAnimation`, `reduceMotion` | On Android, `DeviceMethods.SettingsUpdate` with `setting` `animations` through `Device.UnstableRaw`, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | + +#### Elements and gestures + +| Appium | Use today | +| --- | --- | +| `.getAttribute()` for a label, id, or class, `.isSelected()`, `.getTagName()` | `ToHaveAccessibleNameAsync` for a label, `ToBeSelectedAsync` for the selected state. No read for an id, class, or tag name | +| `pressKeyCode(code)`, `driver.keys('Backspace')`, `isKeyboardShown()` | `.PressAsync()` takes `Enter`, `Space`, or one character. Use `.ClearAsync()` and `.FillAsync()` | +| `driver.getWindowRect()`, `getWindowSize()`, `mobile: deviceInfo` | Swipe by direction, or read `.BoundingBoxAsync()` of a full-screen element | +| `UiScrollable` `scrollToEnd()`, `scrollToBeginning()` | Loop `locator.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })` until `AllTextContentsAsync()` stops changing; `Up` for the beginning | +| `mobile: selectPickerWheelValue`, date and time pickers, slider values | Swipe the wheel and read `.InputValueAsync()`, or use an `Agent.ActAsync` goal | +| `mobile: dragGesture` to coordinates, `mobile: dragFromToForDuration` | `DeviceMethods.InteractionsDrag` through `Device.UnstableRaw`, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | + +#### App, device, and runner -- **Webview contexts.** Not yet. There is no `getContexts` or `switchContext`, and no script runs in a webview. -- **Device shell and files.** Not yet. No `mobile: shell`, `pushFile`, `pullFile`, `startActivity`, or performance data. For what agent-device supports, `Device.UnstableRaw` exposes its client, but it takes only the calls the engine makes, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes). Its calls aren't steps, so they skip the report and secret redaction. -- **Sauce Labs and other Appium grids.** Not yet. BrowserStack, TestMu AI, Limrun, and Expo Simulators have providers. -- **Session reuse.** No equivalent. A device has no saved sessions. Sign in inside the test or in `[SetUp]`, see [Mobile](/get-started/mobile/). +| Appium | Use today | +| --- | --- | +| `driver.getLogs('logcat')`, `'syslog'`, `'crashlog'` | Run `adb logcat` or `xcrun simctl spawn log stream` from a `[SetUp]` with `System.Diagnostics.Process` | +| `driver.removeApp(id)`, `isAppInstalled(id)` | `DeviceMethods.AppsUninstall` or `AppsList` through `Device.UnstableRaw`, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes). `Device.InstallAppAsync(options: new InstallAppOptions { Reinstall = true })` removes the app first and installs it with no data (not on TestMu) | +| `queryAppState(id)` | On Android, `Device.ForegroundAppAsync()` names the app on screen | +| `driver.lock()`, `unlock()`, power and volume buttons | `DeviceMethods.CommandHardwareButton` through `Device.UnstableRaw` with `button` set to `"power"`, `"volume-up"`, or `"volume-down"`; on iOS only the volume keys, on a physical device, not a simulator. Not on TestMu | +| `driver.openNotifications()`, `mobile: pushNotification` | `Screen.SwipeAsync(new SwipePathOptions { From = ..., To = ... })` from the top edge; `xcrun simctl push` from .NET | +| `driver.pushFile()`, `pullFile()`, `pullFolder()` | `adb push`, `adb pull`, or `xcrun simctl addmedia` from .NET; local devices only | +| `driver.getAlertText()` | `Screen.GetByRole("alert").TextContentAsync()` on iOS, `GetByText` on Android | +| `wdio.conf.ts` hooks such as `onPrepare` and `afterTest` | `[OneTimeSetUp]`, `[TearDown]` (NUnit); the constructor and `DisposeAsync` (xUnit v3) | + +### Not yet + +| Appium | Use today | +| --- | --- | +| `appium:autoAcceptAlerts`, `appium:autoDismissAlerts` | Set `app.permissions`, or call `Device.AlertAsync(AlertAction.Accept)` | +| Webview contexts: `getContexts()`, `switchContext()`, CSS selectors, `execute(js)` | Query the webview's accessibility nodes with `GetByText` and `GetByRole` | +| Mobile web: `browserName: 'Safari'` or `'Chrome'` | A `WebEngine` target with a phone `WebEngineOptions.Viewport`, layout only | +| `appium:unlockType`, `unlockKey` | Keep the device unlocked | +| `driver.getPageSource()` as XML | No XML export. `Screen.TreeAsync()` returns the tree as text, and `DeviceMethods.CaptureSnapshot` through `Device.UnstableRaw` returns its nodes. Keep XML consumers in Appium | +| `mobile: pinchOpenGesture`, `mobile: pinch`, `performActions()` with several fingers or waypoints | Single gestures. The raw client takes no pinch or gesture call. Keep the test in Appium | +| `mobile: shell`, `getPerformanceData()` | Run `adb` or `simctl` from .NET with `Device.DeviceId`; local devices only | +| `mobile: setSimulatedLocation`, `mobile: setAppearance`, `mobile: setPasteboard` on an iPhone | Run those tests on a simulator | +| Other `bstack:options` keys, such as `gpsLocation` and `uploadMedia` | - | +| `appium:platformVersion` on a local device, a phone's `appium:udid` | Name a simulator or emulator that runs the version. Put the phone's name in `MobileOptions.Device` | +| `appium:avd`, `avdArgs`, `isHeadless` | Boot the emulator before the run and name its serial | +| An HTML report | Use a `dotnet test` logger such as `--logger html`. The port has no `htmlReporter()` | +| Tests in Java, Python, Ruby, or JavaScript | e2e tests are C#. Rewrite them in a test project beside the app | + +### Not planned + +| Appium | Use instead | +| --- | --- | +| XPath | Chain queries (`Screen.GetByTestId("row").GetByText("Delete")`) or use `Filter(has: ..., hasText: ...)`. XPath ties a test to one platform's tree | +| `-image` selectors, `findImage` | `Screen.TapAtAsync(new Point(x, y))`, or an `Agent.ActAsync` goal | +| `$(selector)` acting on the first of several matches | `.First()`, see [Rewrite selectors](#rewrite-selectors) | +| WebDriverAgent capabilities such as `usePrebuiltWDA` | Nothing to set. agent-device builds and signs its own runner | +| Compound `-ios predicate string` logic, `UiSelector` flags and `fromParent()` | A `GetBy*` query with `Filter(has: ...)` or a `Regex` | +| Gherkin feature files | One test per scenario. A step can be an `Agent.ActAsync` goal | +| Saved sessions on a device | Sign in per test or in `[SetUp]`, or seed the app. A device has no session to save | +| A device grid or hub, other device clouds | A hosted provider, or an `IDeviceProvider` you write, see [Hosted devices](/get-started/mobile/#hosted-devices) | +| `sauce:options` | Write an `IDeviceProvider` | +| Appium Inspector | `Screen.DebugAsync()`, or `e2e mcp` for a coding agent's live session, see [Coding agents](/coding-agents/) | +| `driver.updateSettings()`, `appium:settings[...]` | `Snapshot`, `Settle`, and `Transition` on `MobileOptions` | +| `.getAttribute('clickable')`, `.isClickable()` | Act on the element. The action waits until it is actionable | +| A remote Appium server (`hostname`, `port`) | Run e2e on the machine that has the devices | +| Allure | Feed it the `dotnet test` logger's output | ## Next diff --git a/docs-site/src/content/docs/migrate/cypress.mdx b/docs-site/src/content/docs/migrate/cypress.mdx index c6afa4cc4..a350e73f7 100644 --- a/docs-site/src/content/docs/migrate/cypress.mdx +++ b/docs-site/src/content/docs/migrate/cypress.mdx @@ -12,13 +12,13 @@ e2e replaces Cypress command chains with async C# tests that run with `dotnet te ```text Migrate this project's Cypress tests to e2e, one spec at a time. Docs: the "From Cypress" page of the e2e-dotnet docs site (docs-site/src/content/docs/migrate/cypress.mdx). -1. Read the migration guide. It maps Cypress config, commands, and assertions to e2e. +1. Read the migration guide. It maps Cypress config, commands, and assertions to e2e, and lists what e2e doesn't have yet. 2. Ask me where the .NET test project should live and whether to use NUnit or xUnit v3. If the repository has none, run `e2e init --yes` (add `--framework xunit` for xUnit v3). It creates the test project and example test, writes e2e.config.json, and installs the skill. In an existing test project it adds the missing package references to the .csproj. 3. Read the e2e skill (`e2e guide` prints it, `e2e guide ` prints a topic) and follow it for every step below. 4. If you ran init, ask me which model to use for agent steps. If it needs a sign-in, give me the `e2e login ` command and wait until I confirm. 5. Port cypress.config.* into e2e.config.json: baseUrl and the dev server command become the target's app, and if the specs select by data-cy, set WebEngineOptions.TestIdAttribute = "data-cy" so Screen.GetByTestId reads it. Keep Cypress installed while both suites run. 6. List the Cypress end-to-end specs and ask me which one to move first. Suggest one that breaks often or waits with cy.wait(ms). Component tests have no e2e equivalent; leave them in Cypress. -7. Rewrite that spec as a test class that derives from E2ETest with the guide's tables, one awaited call per line. Where cy.contains matched a fragment, pass exact: false. Replace each cy.wait(ms) with an assertion on the state the test waits for. +7. Rewrite that spec as a test class that derives from E2ETest with the guide's tables, one awaited call per line. Before you port a test, check "What e2e doesn't have yet": use the "Use today" workaround, or leave the test in Cypress and say why. Where cy.contains matched a fragment, pass exact: false. Replace each cy.wait(ms) with an assertion on the state the test waits for. 8. Keep navigation and the final assertions deterministic, with App.OpenAsync and Expect.That. Where a run of scripted interactions breaks often, propose an Agent.ActAsync goal for it instead. 9. Run the new class with `dotnet test --filter` and fix it until it passes. Keep the Cypress spec until I confirm it can go. 10. Summarize which tests moved, which stayed in Cypress, and why. @@ -30,19 +30,14 @@ For a migration that spans several sessions, the `migrate-to-e2e` skill records Install e2e and configure a model with the [web guide](/get-started/web/). The example below uses an agent call. -Keep your `data-cy` attributes. The engine option below reads them as test ids. - | Cypress | e2e | | --- | --- | -| `baseUrl` | `app.url` on the target | -| `start-server-and-test` | `app.url` with `app.command`, see [Web](/get-started/web/) | -| `viewportWidth`, `viewportHeight` | `WebEngineOptions.Viewport` | +| `baseUrl`, `start-server-and-test` | `app.url` with `app.command` on the target, see [Web](/get-started/web/) | | `data-cy` selectors | `WebEngineOptions.TestIdAttribute = "data-cy"`, then `Screen.GetByTestId` | -| `retries` | NUnit `[Retry]`; xUnit v3 has no retry attribute | -| `env`, `Cypress.env()` | `Environment.GetEnvironmentVariable` | -| `cypress.env.json` secrets | `credentials` and `secrets`, see [Authentication](/features/signing-in/) | -| `cypress run --browser firefox` | A second target with `WebEngineOptions.BrowserName = BrowserName.Firefox` | -| Component testing | No equivalent | +| `viewportWidth`, `viewportHeight`, `cy.visit(url, { auth, headers })` | `WebEngineOptions.Viewport`, `BasicAuth`, `Headers` | +| `retries: { runMode: 2, openMode: 0 }` | NUnit `[Retry(3)]`, which counts attempts; xUnit v3 has no retry attribute | +| `env`, `Cypress.env()`, `cypress.env.json` | `Environment.GetEnvironmentVariable`; sign-in values go in `credentials` and `secrets`, see [Authentication](/features/signing-in/) | +| `cypress run --browser firefox` | A target with `WebEngineOptions.BrowserName = BrowserName.Firefox` | ```json title="e2e.config.json" { @@ -105,42 +100,83 @@ For xUnit v3, use `E2E.XUnit.V3`, `[Fact]`, and `[Trait("TestCategory", "RealMod | Cypress | e2e | | --- | --- | -| `cy.visit(path)` | `App.OpenAsync(path)` | -| `cy.get('[data-cy=x]')` | `Screen.GetByTestId("x")` | -| `cy.get('css')` | `Browser.Locator("css")`, not portable; prefer a role or label | -| `cy.contains('text')` | `Screen.GetByText("text", exact: false)` | -| `cy.contains('li', 'text')` | `Screen.GetByRole("listitem").Filter("text")` | -| `.find()`, `.within()` | Chain a query off the locator: `row.GetByRole(...)` | -| `.first()`, `.eq(n)` | `.First()`, `.Nth(n)` | -| `.click()`, `.dblclick()`, `.rightclick()` | `.TapAsync()`, `.DoubleTapAsync()` (or `.DblClickAsync()`), `.SecondaryTapAsync()` | -| `.type('text')` | `.FillAsync("text")` replaces the content | -| `.type('{enter}')` | `.PressAsync("Enter")` | -| `.clear()`, `.check()`, `.select()` | `.ClearAsync()`, `.CheckAsync()`, `.SelectOptionAsync()` | -| `.selectFile()` | `.SetInputFilesAsync()` | -| `.should('be.visible')` | `await Expect.That(locator).ToBeVisibleAsync()` | -| `.should('have.text', t)`, `.should('contain', t)` | `.ToHaveTextAsync(t)`, `.ToContainTextAsync(t)` | -| `.should('have.length', n)` | `.ToHaveCountAsync(n)` | -| `.should('have.value', v)` | `.ToHaveValueAsync(v)` | -| `.should('not.exist')` | `.ToHaveCountAsync(0)`; `ToBeHiddenAsync()` also accepts a node that is still in the DOM | -| `cy.url().should('include', p)` | `Expect.That(Browser).ToHaveURLAsync(new Regex("p"))` | -| `cy.title()` | `Expect.That(Browser).ToHaveTitleAsync(...)` | -| `cy.intercept(url, body)` | `Browser.RouteAsync(url, route => route.FulfillAsync(new RouteFulfillResponse { Json = body }))` | -| `cy.wait('@alias')` | `Browser.WaitForResponseAsync(url)` | -| `cy.on('window:confirm')` | `Browser.OnDialogAsync(DialogHandler.Accept)` | -| `cy.window()`, `cy.document()` | `Browser.EvaluateAsync(...)` | -| `cy.setCookie` | `Browser.SetCookiesAsync` | -| `cy.viewport` | `Browser.SetViewportAsync` | +| `cy.visit(path)`, `cy.reload()`, `cy.viewport(w, h)`, `cy.setCookie()` | `App.OpenAsync(path)`, `Browser.ReloadAsync()`, `Browser.SetViewportAsync(...)`, `Browser.SetCookiesAsync(...)` | +| `cy.get('[data-cy=x]')`, `.find('[data-cy=x]')` | `Screen.GetByTestId("x")`, `row.GetByTestId("x")` | +| `cy.get('css')` | `Browser.Locator("css")`, web only; prefer a role or label | +| `cy.contains('text')`, `cy.contains('li', 'text')` | `Screen.GetByText("text", exact: false)`, `Screen.GetByRole("listitem").Filter("text")` | +| `.click()`, `.dblclick()`, `.clear()`, `.check()`, `.uncheck()`, `.focus()`, `.first()`, `.last()`, `.scrollIntoView()` | `.TapAsync()` (or `.ClickAsync()`), `.DoubleTapAsync()` (or `.DblClickAsync()`), `.ClearAsync()`, `.CheckAsync()`, `.UncheckAsync()`, `.FocusAsync()`, `.First()`, `.Last()`, `.ScrollIntoViewAsync()` | +| `.rightclick()`, `.eq(n)`, `.selectFile(path)` | `.SecondaryTapAsync()`, `.Nth(n)`, `.SetInputFilesAsync(path)` | +| `.type('text')`, `.type('{enter}')`, `.clear().type('text')` | `.PressSequentiallyAsync("text")` appends with key events, `.PressAsync("Enter")`, `.FillAsync("text")` | +| `.select('Pro')` | `.SelectOptionAsync("Pro")` matches the label; a value or an index needs the option object | +| `.should('be.visible')`, `be.checked`, `be.disabled`, `be.focused`, `exist` | `await Expect.That(locator).ToBeVisibleAsync()`, `ToBeCheckedAsync()`, `ToBeDisabledAsync()`, `ToBeFocusedAsync()`, `ToBeAttachedAsync()` | +| `have.text`, `contain`, `have.value`, `have.attr`, `have.length`, `not.exist` | `ToHaveTextAsync`, `ToContainTextAsync`, `ToHaveValueAsync`, `ToHaveAttributeAsync`, `ToHaveCountAsync(n)`, `ToHaveCountAsync(0)` | +| `cy.url().should('include', p)`, `cy.title()` | `Expect.That(Browser).ToHaveURLAsync(new Regex("p"))`, `Expect.That(Browser).ToHaveTitleAsync(t)` | +| `cy.intercept('/api/users', body)` | `Browser.RouteAsync("**/api/users", route => route.FulfillAsync(new RouteFulfillResponse { Json = body }))`, a glob on the full URL | +| `cy.on('window:confirm')`, `cy.on('window:alert')` | `await Browser.OnDialogAsync(DialogHandler.Accept)` before the action, `DialogHandler.Dismiss` to cancel | | `cy.session` | `[E2ESetup]` and `[E2ESession]` | -| `cy.task`, `cy.exec`, `cy.request` | .NET code: `HttpClient`, `File`, your database client | -| `cy.fixture` | a file read in the test, such as `File.ReadAllTextAsync` | +| `cy.task`, `cy.exec`, `cy.request`, `cy.fixture` | .NET code: `HttpClient`, `File`, your database client, a file read such as `File.ReadAllTextAsync` | | `Cypress.Commands.add` | A method on the test class or a base class | -| `cy.wait(ms)` | Use a locator assertion for the condition you need | | `cypress open` | `E2E_HEADLESS=0 dotnet test`, plus `E2E_DEBUG=1` for the step table | -Two things differ from Cypress: +## What e2e doesn't have yet + +Planned gaps are on the roadmap, and Not yet gaps have no date. Not planned gaps are left out on purpose. + +### Planned + +| Cypress | Use today | +| --- | --- | +| `.find('css')`, `.filter('css')`, `.not('css')` | Chain a `GetBy*` query, or write one selector: `Browser.Locator("li.active >> .title")` | +| `.invoke()`, `.its()`, `.then($el => ...)`, `.should($el => ...)` | `GetAttributeAsync()`, `TextContentAsync()`, `InputValueAsync()`, else `Browser.EvaluateAsync` | +| `have.class`, `have.css`, `have.prop`, `be.empty` | `Expect.That(Browser).ToHaveClassAsync(locator, new Regex("..."))`, `Expect.That(locator).ToHaveTextAsync("")`; else read the value with `Browser.EvaluateAsync` | +| `.trigger()`, `cy.scrollTo()`, `.blur()` | `HoverAsync()`, `DragToAsync()`, `ScrollIntoViewAsync()`, `PressAsync("Tab")`; else `Browser.EvaluateAsync` | +| `cy.intercept('POST', url)` and other method or header matchers | Check `route.Request.Method` in the handler, else call `route.FallbackAsync()` | +| `cy.wait('@alias')`, `.its('request.body')`, `@alias.all` | Start `Browser.WaitForResponseAsync(pattern)` before the action; read the request in a `Browser.RouteAsync` handler | +| `Cypress.on('uncaught:exception')` | An app error doesn't fail a test. To fail on one, collect `window.onerror` in `Browser.AddInitScriptAsync`, then assert the list is empty with `Browser.EvaluateAsync` | +| `--browser chrome`, `edge`, `electron`, `before:browser:launch` | `BrowserName.Chromium`, or attach to your own Chrome with `WebEngineOptions.Connect` | +| `cypress open`, time-travel snapshots | `E2E_HEADLESS=0 dotnet test` with `E2E_DEBUG=1`. The port has no trace page, `--watch`, or `--step` | +| `before()` and `after()` that drive the app, `describe.only` | UI setup in `[SetUp]` (NUnit) or `InitializeAsync` (xUnit), cleanup in `[TearDown]` or `DisposeAsync`; sign-in in an `[E2ESetup]` that saves a session; `[Explicit]` or a `--filter` on the test | +| `cy.clock()`, `cy.tick()` | A fake-timers script loaded with `Browser.AddInitScriptAsync`, driven by `Browser.EvaluateAsync` | +| `cy.request()` with the browser's cookies | Send the cookies from `Browser.CookiesAsync()` with `HttpClient` | + +### Not yet + +| Cypress | Use today | +| --- | --- | +| `supportFile`, `blockHosts` | A base class that every test class derives from; abort hosts there with `Browser.RouteAsync` | +| `cypress run --record --parallel`, Cypress Cloud | The framework's parallelism and its CI test logger; the port has no `--shard` | +| `cy.visit()` failing on a 4xx or 5xx page | `App.OpenAsync` ignores the status. Assert on the page content | +| `cy.stub()`, `cy.spy()`, `cy.clearLocalStorage()` | Wrap the function in `Browser.AddInitScriptAsync`; read calls or clear storage with `Browser.EvaluateAsync` | -- There is no chain. Await each call on its own line. It returns a value: `var text = await locator.TextContentAsync();`. -- Actions and single-node reads and assertions need one match. Several fail with `LOCATOR_AMBIGUOUS`. Narrow with `Filter(...)`, `First()`, or `Nth(n)`. +### Not planned + +| Cypress | Use instead | +| --- | --- | +| `cy.wait(ms)` | A locator assertion on the state the test waits for | +| `it`, `context`, `specify`, `xit` | `[Test]` or `[Fact]`, a test class, `[Ignore]` or `[Fact(Skip = "...")]` | +| chai `expect(x).to...`, `assert.*` | Your framework's assertions (`Assert.That`, `Assert.Equal`) on the value | +| `cy.window()`, `cy.document()`, `onBeforeLoad(win)` | `Browser.EvaluateAsync` and `Browser.AddInitScriptAsync` with JSON values. Tests run in .NET, not in the page | +| Mocha reporters, `reporterOptions` | The test framework's loggers and result files | +| Cucumber `.feature` files, `file:preprocessor` | Rewrite features as tests | +| Component testing | Keep it in Cypress. e2e tests a running app | +| `.parent()`, `.closest()`, `.siblings()`, `.next()` | Locate the ancestor and narrow it: `card.Filter(child)` | +| `{ force: true }` | Act on the visible label, or `TapAsync` with a position, which skips the actionability checks | + +## What differs on purpose + +- There is no command chain. Await each call and keep the locator or value in a variable, where Cypress uses `.as()`, `.within()`, or `.then()`. +- Locators are strict. An action or single-node assertion on several matches fails with `LOCATOR_AMBIGUOUS`. Narrow with `Filter(...)`, `First()`, or `Nth(n)`, or loop over `await locator.AllAsync()`. +- A string in a query matches the whole text. `exact: false` matches a fragment in any case. `cy.contains` is case-sensitive and takes the first match, so pass a Regex or add `.First()`. +- Text matchers compare the rendered text. CSS `text-transform` applies and hidden children are left out. +- Dialogs are not accepted for you. An `alert` or `confirm` with no handler fails the next step, where Cypress accepts it. + +## Defaults that differ + +| Setting | Cypress | e2e | +| --- | --- | --- | +| Viewport | 1000 by 660 | 1280 by 720 | +| Action and assertion timeouts | `defaultCommandTimeout`, 4 s | `actionTimeout` 30 s, `assertionTimeout` 5 s | +| Retries in CI | `0` | The framework's retry attribute, none by default | ## Next diff --git a/docs-site/src/content/docs/migrate/detox.mdx b/docs-site/src/content/docs/migrate/detox.mdx index 9e8fb2255..fb9bf5a79 100644 --- a/docs-site/src/content/docs/migrate/detox.mdx +++ b/docs-site/src/content/docs/migrate/detox.mdx @@ -12,14 +12,14 @@ e2e tests a React Native app on a simulator or emulator with your normal debug o ```text Migrate this project's Detox tests to e2e, one spec at a time. Docs: the "From Detox" page of the e2e-dotnet docs site (docs-site/src/content/docs/migrate/detox.mdx). -1. Read the migration guide. It maps Detox config and calls to e2e. +1. Read the migration guide. It maps Detox config and calls to e2e, and lists what e2e doesn't have yet. 2. Ask me where the .NET test project should live and whether to use NUnit or xUnit v3. Run `e2e init --yes --engine mobile` (add `--framework xunit` for xUnit v3). It creates the project with E2E.Mobile.NUnit (or E2E.Mobile.XUnit.V3) and an example test. 3. Read the e2e skill (`e2e guide` prints it, `e2e guide mobile` prints the mobile topic) and follow it for every step below. 4. If you ran init, ask me which model to use for agent steps. If it needs a sign-in, give me the `e2e login ` command and wait until I confirm. 5. Port .detoxrc.js into e2e.config.json with the guide's config table: one target per platform, with app.bundleId and app.appPath. 6. Run `e2e doctor` and fix what it reports. 7. List the Detox specs and ask me which one to move first. Suggest a flow that needs frequent maintenance. -8. Rewrite that spec as a test class that derives from MobileAppTest. Map each element(by.*) call to Screen with the guide. Replace each waitFor with an assertion that has a timeout. Where the guide says "Not yet" or "No equivalent", use the workaround it gives, or leave that spec in Detox and tell me why. +8. Rewrite that spec as a test class that derives from MobileAppTest. Map each element(by.*) call to Screen with the guide. Replace each waitFor with an assertion that has a timeout. Check each Detox call in that spec against "What e2e doesn't have yet". Use the "Use today" or "Use instead" entry, or leave the spec in Detox and say why. 9. Keep the final assertions deterministic, with Expect.That. Where a run of scripted interactions breaks often, propose an Agent.ActAsync goal for it instead. 10. Run the new class on a booted simulator with `dotnet test --filter` and fix it until it passes. Keep the Detox spec until I confirm it can go. 11. Summarize which tests moved, which stayed in Detox, and why. @@ -98,7 +98,7 @@ public sealed class LoginTests : MobileAppTest ## Map each Detox call -"Not yet" marks a call e2e doesn't have today, and "No equivalent" one it leaves out on purpose. +Calls the port lacks are under [What e2e doesn't have yet](#what-e2e-doesnt-have-yet). | Detox | e2e | | --- | --- | @@ -113,14 +113,9 @@ public sealed class LoginTests : MobileAppTest | `.typeText(t)`, `.replaceText(t)` | `.FillAsync(t)` replaces the content | | `.clearText()` | `.ClearAsync()` | | `.tapReturnKey()` | `.PressAsync("Enter")` | -| `.tapBackspaceKey()` | Not yet. `.PressAsync("Backspace")` is `UNSUPPORTED_CAPABILITY` on a device. `.FillAsync(t)` with the shorter text replaces the value and sends no Backspace, so keep a test that depends on the key in Detox | | `.swipe('up', 'fast')` | `.SwipeAsync(new SwipeOptions { Direction = ..., Momentum = ... })`: Detox names the finger's direction, e2e the content's | | `.scroll(200, 'down')` | `Screen.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })`, or `Locator.SwipeAsync` on the scroll view. It does not keep the 200-point distance: for a test that scrolls to reach a control, use `list.ScrollUntilVisibleAsync(target)`; check a distance-sensitive test on your device before dropping the Detox call | | `waitFor(el).toBeVisible().whileElement(list).scroll(...)` | `list.ScrollUntilVisibleAsync(el)`, with `new ScrollUntilVisibleOptions { Direction = ScrollDirection.Up }` when Detox scrolls up | -| `list.scrollTo('bottom')` | `list.ScrollUntilVisibleAsync(lastRow)`; for `'top'`, `list.ScrollUntilVisibleAsync(firstRow, new ScrollUntilVisibleOptions { Direction = ScrollDirection.Up })` | -| `.adjustSliderToPosition()` | Not yet. Drag with `Screen.SwipeAsync(new SwipePathOptions { From = new Point(x1, y1), To = new Point(x2, y2) })`, the points taken from the slider's `.BoundingBoxAsync()`; on Android, `.TapAsync(new TapOptions { Position = ... })` on the track also works | -| `.setDatePickerDate()` | Not yet | -| `.getAttributes()` | One call per field: `.TextContentAsync()`, `.InputValueAsync()`, `.IsEnabledAsync()`, `.IsVisibleAsync()`, `.IsCheckedAsync()`, `.BoundingBoxAsync()`, and `.GetAttributeAsync("placeholder")`, the only attribute a device node has | | `waitFor(el).toBeVisible().withTimeout(ms)` | `await Expect.That(el).ToBeVisibleAsync(timeout: TimeSpan.FromMilliseconds(ms))` | | `expect(el).toBeVisible()`, `.toExist()` | `.ToBeVisibleAsync()` for visibility; `.ToHaveCountAsync(1)` for existence | | `expect(el).not.toBeVisible()` | `.ToBeHiddenAsync()` | @@ -133,26 +128,72 @@ public sealed class LoginTests : MobileAppTest | `device.launchApp({ newInstance: false })` | `Device.OpenAppAsync(id)` without `Relaunch` | | `device.launchApp({ url })`, `device.openURL({ url })` | `Device.OpenLinkAsync(url)` | | `device.terminateApp()` | `Device.CloseAppAsync()` | -| `device.uninstallApp()` | Not yet. `Device.UnstableRaw.Client.CallAsync(DeviceMethods.AppsUninstall, ...)` with `app` set to the id removes it, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes); `Device.InstallAppAsync(options: new InstallAppOptions { Reinstall = true })` installs with no data (not on TestMu) | -| `device.resetContentAndSettings()` | Not yet. `App.ClearStateAsync()` resets the app; on iOS, `Device.ClearKeychainAsync()` clears the simulator keychain for every app | -| `device.reloadReactNative()` | No equivalent. `App.RestartAsync()` is the nearest | | `device.sendToHome()`, `device.pressBack()` | `Device.HomeAsync()`, `Device.BackAsync()` | | `device.setLocation(lat, lon)` | `Device.SetLocationAsync(new Coordinates(latitude, longitude))` | | `device.setOrientation('landscape')` | `Device.SetOrientationAsync(DeviceOrientation.LandscapeLeft)` | | `device.setBiometricEnrollment(true)` | `Device.EnrollBiometricsAsync(BiometricSensor.FaceId, true)` | | `device.matchFace()`, `device.unmatchFace()` | `Device.SetBiometricsAsync(BiometricSensor.FaceId, BiometricResult.Match)` or `NonMatch` | -| `device.setStatusBar` | No equivalent. The device engine's masked screenshot holds the status bar fixed, except on TestMu and .NET MAUI DevFlow, which show the real one | -| `device.setURLBlacklist` | No equivalent | -| `device.shake()`, `device.sendUserNotification()` | Not yet | -| `device.disableSynchronization()` | No equivalent. Actions wait and assertions retry | | `device.takeScreenshot()` | `App.ScreenshotAsync("label")`, secure fields masked | | `device.takeScreenshot()` compared with an image snapshot library | `Expect.That(Screen).ToHaveScreenshotAsync("name.png")`. The port documents it for the web engine only and has no device run of it, so keep the Detox comparison until you have checked it on your device | | `:ios:` / `:android:` in a test name | `Assume.That` on the platform, such as `E2E_MOBILE_PLATFORM`, or a separate run per platform | | `device.getPlatform()` | the `Platform` property of `MobileAppTest`, `MobilePlatform.Ios` or `Android` | -Two things differ from Detox: +## What e2e doesn't have yet -- Actions and single-node reads and assertions need one match. Several fail with `LOCATOR_AMBIGUOUS`. Narrow with `Filter(...)`, `First()`, or `Nth(n)`. +Planned is on the roadmap, and Not yet has no date. Not planned is left out on purpose. Use the workaround, or keep the spec in Detox. + +### Planned + +| Detox | Use today | +| --- | --- | +| `beforeAll` that launches the app | `App.OpenAsync()` in `[SetUp]`; `MobileAppTest` already launches the app fresh before each test | +| `by.id('x').and(by.text('y'))` | `.Filter(...)`, or `Device.Locator("id=x label=\"y\"")` | +| `system.element(by.system.label('Allow'))` | `Device.AlertAsync(AlertAction.Accept)` or `Dismiss`, or `app.permissions` so no prompt shows | +| `.longPressAndDrag()`, `.swipe()` offset, `.multiTap(3)` | `.DragToAsync(target)`, `Screen.SwipeAsync(new SwipePathOptions { From = ..., To = ... })`, repeated `.TapAsync()` | +| `.scrollTo('bottom')` | `list.ScrollUntilVisibleAsync(lastRow)`, or the first row with `new ScrollUntilVisibleOptions { Direction = ScrollDirection.Up }` | +| `.tapBackspaceKey()` | `.FillAsync(t)` with the shorter text. `.PressAsync("Backspace")` is `UNSUPPORTED_CAPABILITY` on a device | +| `.setDatePickerDate()`, `.setColumnToValue()`, `.adjustSliderToPosition()` | An `Agent.ActAsync` goal, or a drag with `Screen.SwipeAsync(new SwipePathOptions { From = ..., To = ... })` on a slider, the points taken from `.BoundingBoxAsync()` | +| `.getAttributes()` | One call per field: `.TextContentAsync()`, `.InputValueAsync()`, `.IsEnabledAsync()`, `.IsVisibleAsync()`, `.IsCheckedAsync()`, `.BoundingBoxAsync()`, `.GetAttributeAsync("placeholder")` | +| `permissions` for names and values beyond `PermissionState.Grant`, `Deny`, `Reset` | Tap the prompt's button, or `Device.AlertAsync(AlertAction.Accept)` | +| `device.sendUserNotification()`, `device.uninstallApp()` | `Device.UnstableRaw.Client.CallAsync(...)`, see [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | +| `launchApp({ languageAndLocale })` | On iOS, `app.launchArguments` of `["-AppleLanguages", "(fr)", "-AppleLocale", "fr_FR"]`, see [Run the app in another language](/get-started/mobile/#run-the-app-in-another-language) | +| Jest `globalSetup`, `globalTeardown` | Start dependencies before the run, or in a framework-level one-time setup | +| `device.reverseTcpPort()` | Run `adb reverse` in a CI step | +| `device.shake()` | - | +| `launchArgs` as one key-value map | `["-key", "value"]` on iOS, `["--es", "key", "value"]` on Android | +| `device.setStatusBar()` | `xcrun simctl status_bar`. The masked screenshot holds the status bar fixed, except on TestMu and .NET MAUI DevFlow | + +### Not yet + +| Detox | Use today | +| --- | --- | +| `.pinch()` | The raw client has no pinch. Keep the test in Detox | +| `.toBeVisible(50)`, and the 75% default | `.ToBeVisibleAsync()` checks the platform's flag, not the area. Compare `.BoundingBoxAsync()` | +| `web.element(by.web.cssSelector('x'))` | The WebView's accessibility text and labels, or an `Agent.ActAsync` goal | +| `device.resetContentAndSettings()` | `App.ClearStateAsync()`, plus `Device.ClearKeychainAsync()` on iOS, which clears the simulator keychain for every app | +| `device: { type, os }`, `headless`, `bootArgs` | Create and boot the device in a CI step | +| `device.takeScreenshot()` compared with an image snapshot library | `Expect.That(Screen).ToHaveScreenshotAsync("name.png")` is documented for the web engine only, with no device run. Keep the Detox comparison until you have checked it on your device | +| `:ios:` / `:android:` in a test name | `Assume.That` on `Platform`, or a separate run per platform. The port has no per-test platform option | + +### Not planned + +| Detox | Use instead | +| --- | --- | +| `device.reloadReactNative()` | `App.RestartAsync()`. A reload needs the Detox framework in the app | +| `device.disableSynchronization()`, `enableSynchronization()` | Delete the calls. There is no synchronization to switch off | +| `device.setURLBlacklist()` | Delete the call. Network activity never blocks an action | +| `behavior.init.reinstallApp` | `Device.InstallAppAsync(options: new InstallAppOptions { Reinstall = true })` in `[SetUp]`, then `App.OpenAsync()` in the test | +| `--reuse` | Omit `app.appPath` (and unset `E2E_MOBILE_APP_PATH`) and keep `app.bundleId` set for the preinstalled app. With `app.appPath` set, `MobileAppTest` installs the build once per run, replacing the installed binary | +| `apps..build`, `detox build` | Your normal build, before `dotnet test` | +| `.scroll()` by an exact offset | `Screen.SwipeAsync(new SwipePathOptions { From = ..., To = ... })` | +| `by.type('RCTImageView')` | `Device.Locator("role=image")`, or a `testID` | +| Jest reporters, Allure | The framework's reporters, or the TRX or JUnit logger of `dotnet test` | + +## What differs on purpose + +- e2e doesn't wait for the app to be idle. Actions wait and assertions retry. For a control that is still arriving, raise `MobileOptions.TransitionMilliseconds`; for agent actions, `SettleMilliseconds`. +- `MobileAppTest` installs the build once per run. Call `Device.InstallAppAsync()` for anything else. +- Locators are strict. An action or single-node assertion on several matches fails with `LOCATOR_AMBIGUOUS`. Narrow with `Filter(...)`, `First()`, or `Nth(n)`. - A device has no saved sessions. Sign in inside the test or in `[SetUp]`. See [Mobile](/get-started/mobile/). ## Next diff --git a/docs-site/src/content/docs/migrate/index.mdx b/docs-site/src/content/docs/migrate/index.mdx index 892b87c99..bd3da6354 100644 --- a/docs-site/src/content/docs/migrate/index.mdx +++ b/docs-site/src/content/docs/migrate/index.mdx @@ -3,7 +3,7 @@ title: Migrating to e2e description: Move a Playwright .NET, Selenium .NET, Cypress, Appium, Detox, Maestro, or TesterArmy suite to e2e one test at a time. --- -This page ports upstream's migration overview (`docs/migrate/index.mdx` in tester-army/e2e at `a016b3e4`, from tester-army/e2e#1441 and #1597). A migration guide maps one test runner's config, API, and commands to e2e. It also marks what e2e does not have yet, with a workaround where there is one. A test with no workaround stays in the old runner. Keep the old suite installed while you move one test at a time, so both suites stay green. +This page ports upstream's migration overview (`docs/migrate/index.mdx` in tester-army/e2e at `a016b3e4`, from tester-army/e2e#1441 and #1597). A migration guide maps one test runner's config, API, and commands to e2e. It also lists what e2e does not have yet: what is planned, what has no date, and what e2e leaves out on purpose, with a workaround where there is one. A test with no workaround stays in the old runner. Keep the old suite installed while you move one test at a time, so both suites stay green. ## Pick your runner @@ -58,7 +58,7 @@ WebdriverIO for the web has no guide. Map its WebDriver calls with the [Selenium ## What the port does not have -Upstream's guides name `e2e run`, `e2e list`, `e2e feedback`, `e2e.config.ts`, and a TypeScript project. The port has none of them. The guides here use `dotnet test`, `e2e.config.json`, and C#. Each guide lists the rest as `missing` or `by design`. [COMPATIBILITY.md](/compatibility/#migration-guides) names every difference. +Upstream's guides name `e2e run`, `e2e list`, `e2e feedback`, `e2e.config.ts`, and a TypeScript project. The port has none of them. The guides here use `dotnet test`, `e2e.config.json`, and C#. Each guide lists the rest under "What e2e doesn't have yet". [COMPATIBILITY.md](/compatibility/#migration-guides) names every difference. ## Next diff --git a/docs-site/src/content/docs/migrate/maestro.mdx b/docs-site/src/content/docs/migrate/maestro.mdx index df445e821..657d5e0d7 100644 --- a/docs-site/src/content/docs/migrate/maestro.mdx +++ b/docs-site/src/content/docs/migrate/maestro.mdx @@ -12,14 +12,14 @@ e2e replaces Maestro's YAML flows with C# tests. Each command becomes an awaited ```text Migrate this project's Maestro flows to e2e, one flow at a time. Docs: the "From Maestro" page of the e2e-dotnet docs site (docs-site/src/content/docs/migrate/maestro.mdx). -1. Read the migration guide. It maps Maestro config and commands to e2e. +1. Read the migration guide. It maps the Maestro config options and commands e2e has, and lists what e2e does not have yet with what to use today. 2. Ask me where the .NET test project should live and whether to use NUnit or xUnit v3. Run `e2e init --yes --engine mobile` (add `--framework xunit` for xUnit v3). It creates the project with E2E.Mobile.NUnit (or E2E.Mobile.XUnit.V3) and an example test. 3. Read the e2e skill (`e2e guide` prints it, `e2e guide mobile` prints the mobile topic) and follow it for every step below. 4. If you ran init, ask me which model to use for agent steps. If it needs a sign-in, give me the `e2e login ` command and wait until I confirm. 5. Port the flows' appId into the target's app.bundleId and the build path into app.appPath, with the guide's config table. 6. Run `e2e doctor` and fix what it reports. 7. List the flow files and ask me which one to move first. Suggest a flow whose optional steps and runFlow branches often need updating. -8. Rewrite that flow as a test class that derives from MobileAppTest, one awaited call per command, with the guide's tables. Apply "Rewrite selectors" to every selector. Where the guide says "Not yet" or "No equivalent", use the workaround it gives, or leave that flow in Maestro and tell me why. +8. Rewrite that flow as a test class that derives from MobileAppTest, one awaited call per command, with the guide's tables. Follow the guide's "Rewrite selectors" and "What differs on purpose" sections: e2e matches the whole string, fails on several matches, and grants no permission the config does not name. Check "What e2e doesn't have yet" before you port a flow: for a command listed there, use its "Use today" workaround, or leave that flow in Maestro and tell me why. 9. Keep the final assertions deterministic, with Expect.That. Where a run of commands breaks often, propose an Agent.ActAsync goal for it instead. 10. Run the new class on a booted simulator with `dotnet test --filter` and fix it until it passes. Keep the Maestro flow until I confirm it can go. 11. Summarize which flows moved, which stayed in Maestro, and why. @@ -34,23 +34,16 @@ Set up a target with the [mobile guide](/get-started/mobile/) and a model with t | Maestro | e2e | | --- | --- | | `appId` at the top of every flow | `app.bundleId` on the target, the app `MobileAppTest` launches before each test | -| a different `appId` per flow, or a second one mid-flow | `Device.OpenAppAsync(id)` inside the test, or separate app-specific configs and runs. The agent's `open_app` tool switches apps inside a goal, see [Mobile](/get-started/mobile/) | -| `--platform ios` | `platform` on the target, or `E2E_MOBILE_PLATFORM`. The port runs one device target | -| `--device` | `MobileOptions.Device`, or `E2E_MOBILE_DEVICE`: a UDID, or a name only one simulator has | -| `.maestro/config.yaml` `flows` | the test project's own tests; `dotnet test` collects the classes in the project | -| `flows: ['!helper.yaml']` | a helper that is not a test method, or a class that is not collected | -| `tags` and `--include-tags` | `[Category("smoke")]` (NUnit) or `[Trait("TestCategory", "smoke")]` (xUnit); `dotnet test --filter "TestCategory=smoke"` | -| `--exclude-tags` | `dotnet test --filter "TestCategory!=smoke"` | +| a different `appId` per flow, or a second one mid-flow | `Device.OpenAppAsync(id)` in the test, or one config per app | +| `--platform`, `--device` | `platform` on the target (or `E2E_MOBILE_PLATFORM`) and `MobileOptions.Device` (or `E2E_MOBILE_DEVICE`): a UDID, or a name only one simulator has. The port runs one device target | +| `.maestro/config.yaml` `flows`, `'!helper.yaml'` | the test project's own tests; `dotnet test` collects the classes in the project. A helper is a method that is not a test, or a class that is not collected | +| `tags`, `--include-tags`, `--exclude-tags` | `[Category("smoke")]` (NUnit) or `[Trait("TestCategory", "smoke")]` (xUnit); `dotnet test --filter "TestCategory=smoke"`, `"TestCategory!=smoke"` | | `env` and `${VAR}` | constants in the test, and `Params` in a goal: `Agent.ActAsync("sign in as {email}", new ActOptions { Params = ... })` | | `MAESTRO_*` variables | `Environment.GetEnvironmentVariable`; a password or token goes under `credentials` or `secrets`, which the model never sees, see [Authentication](/features/signing-in/) | -| `--format junit`, `--output` | the `dotnet test` loggers, such as `--logger trx` (`--report-trx` on Microsoft.Testing.Platform; see [Testing frameworks](/testing-frameworks/#vstest-and-microsofttestingplatform)), or a JUnit logger package | -| `--format html` | Not yet. Each failed test gets a `trace.md` page under `.e2e/results/` | -| `properties` (JUnit ``) | Not yet | +| `--format junit` | the `dotnet test` loggers, such as `--logger trx` (`--report-trx` on Microsoft.Testing.Platform; see [Testing frameworks](/testing-frameworks/#vstest-and-microsofttestingplatform)), or a JUnit logger package | | `--shards`, `--shard-split` | The framework's parallelism on one machine; the port has no `--shard` | -| `maestro record` | `E2E_VIDEO=on dotnet test`, one MP4 of the device per attempt, or `video` in the config | -| `maestro test -c`, Maestro Studio | Not yet. `e2e mcp` gives a coding agent a live session on the device, see [Coding agents](/coding-agents/) | -| `maestro hierarchy` | Not yet. `await Device.UnstableRaw.Client.CallAsync(DeviceMethods.CaptureSnapshot, new JsonObject(), CancellationToken.None)` (`DeviceMethods` is in `E2E.Mobile`, `JsonObject` in `System.Text.Json.Nodes`) returns agent-device's node tree as a `JsonNode`, the call the engine's own snapshot makes, see [`UnstableRaw`](/get-started/mobile/#3-write-a-test) and [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes); `Device.Locator(selector)` takes agent-device selectors for nodes `Screen` cannot name | -| Maestro Cloud | your CI | +| `maestro record` | `E2E_VIDEO=on dotnet test`, one MP4 of the device per attempt, or a hosted provider's own recording | +| `maestro hierarchy` | `await Screen.DebugAsync()` in a test prints the screen's tree; `Screen.TreeAsync()` returns it | ## Rewrite one flow @@ -98,109 +91,108 @@ For xUnit v3, use `E2E.Mobile.XUnit.V3`, `[Fact]`, and `[Trait("TestCategory", " ## Map each command -Calls on `Device` run only on a device. Calls on `Screen`, `App`, `Agent`, and `Expect.That(locator)` run on every target. "Not yet" marks a command e2e doesn't have today, and "No equivalent" one it leaves out on purpose. +Calls on `Device` run only on a device. Calls on `Screen`, `App`, `Agent`, and `Expect.That(locator)` run on every target. The reference pages hold the signatures. -### App lifecycle +### App and device | Maestro | e2e | | --- | --- | | `launchApp` | `MobileAppTest` launches the target's `app.bundleId` fresh before each test; `App.OpenAsync()` launches it again | -| `launchApp: { clearState: true }`, `clearState` | `App.ClearStateAsync()` | +| `clearState`, `launchApp: { clearState: true }` | `App.ClearStateAsync()` | | `launchApp: { stopApp: false }` | `Device.OpenAppAsync(id)` without `Relaunch` | -| `launchApp: { arguments }` | `app.launchArguments` on the target, or `OpenAppOptions.LaunchArguments` for one launch; a list of strings, not a map | -| `launchApp: { permissions }` | `app.permissions` on the target, set before every fresh launch, or `OpenAppOptions.Permissions` for one launch; `PermissionState.Grant`, `Deny`, `Reset` for `allow`, `deny`, `unset`; no `all`, nothing is granted unless named | -| `stopApp`, `killApp` | `Device.CloseAppAsync()` | -| `stopApp`, then `launchApp` | `App.RestartAsync()` | -| `clearKeychain` | `Device.ClearKeychainAsync()`, the whole simulator's keychain; iOS only | -| `openLink` | `Device.OpenLinkAsync(url, app)`; on iOS the link opens into `app`, default the pinned app | -| `addMedia` | Not yet. Add the photo from the host before `App.OpenAsync()`: `xcrun simctl addmedia` on `Device.UnstableRaw.DeviceId` (the simulator's UDID) on an iOS simulator; Android is unchecked. `OpenAsync` restarts the app, so import the media before the flow navigates to the picker, or navigate back to it afterwards. See [Photos for a picker test](/get-started/mobile/#photos-for-a-picker-test) | -| `back` | `App.BackAsync()`, or `Device.BackAsync()` | +| `launchApp: { arguments }` | `app.launchArguments` on the target, a list of strings, not a map; `OpenAppOptions.LaunchArguments` for one launch | +| `launchApp: { permissions }`, `setPermissions` | `app.permissions` on the target, or `OpenAppOptions.Permissions` for one launch; `PermissionState.Grant`, `Deny`, `Reset` for `allow`, `deny`, `unset`. `Device.SetPermissionAsync(permission, PermissionState.Grant)` before the app launches, one permission per call. A permission or state outside `DevicePermission` and `PermissionState` (a cast number) throws `INVALID_ARGUMENT` before any device command. A name read from outside the code can go in as a string: `Device.SetPermissionAsync("camera", "grant")` takes the wire names, and a misspelled one is refused with a `did you mean` hint before any device command | +| `stopApp`, `killApp` | `Device.CloseAppAsync()`, a force stop; `App.RestartAsync()` to stop the app and launch it again | +| `clearKeychain`, `openLink`, `setLocation`, `setAirplaneMode`, `setOrientation`, `setClipboard` | the same on `Device`, such as `Device.OpenLinkAsync(url, app)`; on iOS the link opens into `app`, default the pinned app. `ClearKeychainAsync()` is iOS simulator only, `SetAirplaneModeAsync(true)` Android only. `SetLocationAsync(new Coordinates(latitude, longitude))`, `SetOrientationAsync(DeviceOrientation.LandscapeLeft)`; `Device.ClipboardAsync()` reads the clipboard, and neither works on an Android build with no clipboard shell command. For `travel`, loop `Device.SetLocationAsync` | +| `back`, `pressKey: Home`, `hideKeyboard`, `setDarkMode` | `App.BackAsync()` or `Device.BackAsync()`, `Device.HomeAsync()`, `Device.DismissKeyboardAsync()`, `Device.SetAppearanceAsync(Appearance.Dark)` | +| platform-specific selectors | `Device.Locator("role=text label=\"...\"")`, agent-device's selector terms | -### Taps and input +### Taps, input, and swipes | Maestro | e2e | | --- | --- | -| `tapOn: "Text"` | `Screen.GetByText("Text").TapAsync()`; exact, see [Rewrite selectors](#rewrite-selectors) | -| `tapOn: { id: x }` | `Screen.GetByTestId("x").TapAsync()` | -| `tapOn: { text, index: 2 }` | `.Nth(2)` | -| `tapOn: { below: "Email" }`, `above`, `leftOf`, `rightOf`, `containsDescendants` | No equivalent, see [Rewrite selectors](#rewrite-selectors) | +| `tapOn: "Text"`, `tapOn: { id: x }` | `Screen.GetByText("Text").TapAsync()`, `Screen.GetByTestId("x").TapAsync()`, see [Rewrite selectors](#rewrite-selectors) | +| `index: 2`, `containsDescendants`, `childOf` | `.Nth(2)`, `.Filter(has: other)`, `parent.GetByText("Text")` | | `tapOn: { point: "x,y" }` | `Screen.TapAtAsync(new Point(x, y))`, in device points: convert a percentage point (`50%,50%`) with the screen size first; inside a node, `.TapAsync(new TapOptions { Position = new Point(x, y) })` | | `retryTapIfNoChange`, `waitToSettleTimeoutMs` | `MobileOptions.SettleMilliseconds` and `TransitionMilliseconds`, engine-wide; no per-call knob | | `doubleTapOn`, `longPressOn` | `.DoubleTapAsync()`, `.LongPressAsync(new LongPressOptions { Duration = ... })` | -| `inputText` | `Screen.GetByTestId("field").FillAsync("text")` | +| `inputText`, `eraseText`, `pressKey: Enter` after a `tapOn` | `.FillAsync("text")`, `.ClearAsync()`, `.PressAsync("Enter")` on the field's locator | | `inputRandomEmail`, `inputRandomText`, `faker` | `Values.Unique(...)` around a value in `Params`, so the cache still replays; any .NET faker library for the rest | -| `eraseText` | `.ClearAsync()` | -| `hideKeyboard` | `Device.DismissKeyboardAsync()` | -| `pressKey: Enter` | `.PressAsync("Enter")` on the field | -| `pressKey: Home`, `back` | `Device.HomeAsync()`, `Device.BackAsync()` | -| `pressKey: lock`, `power`, `volume up`, `volume down` | Not yet. `Device.UnstableRaw.Client.CallAsync(DeviceMethods.CommandHardwareButton, ...)` with `button` set to `"power"`, `"volume-up"`, or `"volume-down"` presses one (no `lock`); on iOS only the volume keys, on a physical device, not a simulator. See [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | -| TV remote keys | Not yet | | `copyTextFrom`, `pasteText` | `await locator.TextContentAsync()`, then `.FillAsync(text)` | -| `setClipboard` | `Device.SetClipboardAsync(text)`; `Device.ClipboardAsync()` reads it. Neither works on an Android build with no clipboard shell command | -| `label: "..."` on a command | Not yet. Every `Screen`, `Device`, and `Agent` call is a report step named by the call | - -### Gestures - -| Maestro | e2e | -| --- | --- | -| `swipe: { direction: UP }` | `Screen.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })`, or `locator.SwipeAsync(...)` inside one node. Maestro names the finger's direction, e2e the content's | -| `swipe: { start, end }` | `Screen.SwipeAsync(new SwipePathOptions { From = new Point(x1, y1), To = new Point(x2, y2) })`, the two endpoints in device points | -| `scroll` | `Screen.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })` | -| `scrollUntilVisible: { element }` | `Screen.ScrollUntilVisibleAsync(locator, new ScrollUntilVisibleOptions { Direction = ... })` | -| `waitForAnimationToEnd` | an assertion on the settled state; `MobileOptions.SettleMilliseconds` and `TransitionMilliseconds` for the engine | +| `swipe: { direction: UP }`, `scroll` | `Screen.SwipeAsync(new SwipeOptions { Direction = ScrollDirection.Down })`, or `locator.SwipeAsync(...)` inside one node. Maestro names the finger's direction, e2e the content's | +| `swipe: { start, end }`, `scrollUntilVisible` | `Screen.SwipeAsync(new SwipePathOptions { From = new Point(x1, y1), To = new Point(x2, y2) })` in device points, `Screen.ScrollUntilVisibleAsync(locator, new ScrollUntilVisibleOptions { Direction = ... })` | -### Assertions and waits +### Assertions and flows | Maestro | e2e | | --- | --- | -| `assertVisible`, `assertNotVisible` | `await Expect.That(locator).ToBeVisibleAsync()`, `.ToBeHiddenAsync()` | -| `assertVisible: { text, enabled: true }` | `.ToBeVisibleAsync()` and `.ToBeEnabledAsync()` | -| `assertVisible: { checked, selected, focused }` | `.ToBeCheckedAsync()`, `.ToBeSelectedAsync()`, `.ToBeFocusedAsync()` | +| `assertVisible`, `assertNotVisible`, `extendedWaitUntil` | `await Expect.That(locator).ToBeVisibleAsync(timeout: ...)`, `.ToBeHiddenAsync()` | +| `assertVisible: { enabled, checked, selected, focused }` | `.ToBeVisibleAsync()`, then `.ToBeEnabledAsync()`, `.ToBeCheckedAsync()`, `.ToBeSelectedAsync()`, or `.ToBeFocusedAsync()` | | `assertTrue: ${...}` | `Assert.That(value, Is.True)`, or `Expect.Poll(read)` for a value that is still changing | -| `extendedWaitUntil: { visible, timeout }` | `.ToBeVisibleAsync(timeout: ...)` | | `assertScreenshot: { path, thresholdPercentage }` | `Expect.That(Screen).ToHaveScreenshotAsync(name, new ScreenshotOptions { MaxDiffPixelRatio = ... })`. The port documents it for the web engine only and has no device run of it, so keep the Maestro assertion until you have checked it on your device | -| `assertDarkMode`, `assertLightMode` | Not yet. Assert on a themed control | -| `retry: { maxRetries, commands }` | `Expect.Poll` for a value, NUnit `[Retry]` for the flow | -| `takeScreenshot` | `App.ScreenshotAsync("label")`, secure fields masked | +| `takeScreenshot`, `startRecording`, `stopRecording` | `App.ScreenshotAsync("label")`, `App.StartRecordingAsync(new RecordingOptions { Name = "..." })`, `App.StopRecordingAsync()` | +| `assertWithAI`, `assertNoDefectsWithAI`, `extractTextWithAI` | `Agent.AssertAsync("...", new AssertOptions { Vision = VisionMode.On })`, `Agent.ExtractAsync("...")`; the type is the schema | +| `runFlow: file.yaml`, `onFlowStart`, `onFlowComplete` | a method on the test class or a base class; `[SetUp]`, `[TearDown]` (NUnit), `InitializeAsync`, `DisposeAsync` (xUnit v3) | +| `runFlow: { when: { visible } }` | `if (await locator.IsVisibleAsync()) { ... }`, see [Rewrite selectors](#rewrite-selectors) | +| `repeat`, `runScript`, `evalScript`, `${output.x}`, `http.get` | C# in the test: loops, variables, and `HttpClient` | -### Device +## Rewrite selectors -| Maestro | e2e | -| --- | --- | -| `setLocation` | `Device.SetLocationAsync(new Coordinates(latitude, longitude))` | -| `travel: { points }` | `Device.SetLocationAsync` in a loop | -| `setAirplaneMode`, `toggleAirplaneMode` | `Device.SetAirplaneModeAsync(true)` or `false`, the state you want; `true` is Android only | -| `setDarkMode`, `toggleDarkMode` | `Device.SetAppearanceAsync(Appearance.Dark)` or `Appearance.Light` | -| `setOrientation` | `Device.SetOrientationAsync(DeviceOrientation.LandscapeLeft)` | -| `setPermissions` | `Device.SetPermissionAsync(permission, PermissionState.Grant)`, one permission per call, on the target's app, which need not be running; call it before the app launches. A permission or state outside `DevicePermission` and `PermissionState` (a cast number) throws `INVALID_ARGUMENT` before any device command. A name read from outside the code can go in as a string: `Device.SetPermissionAsync("camera", "grant")` takes the wire names, and a misspelled one is refused with a `did you mean` hint before any device command | -| platform-specific selectors | `Device.Locator("role=text label=\"...\"")`, agent-device's selector terms | +- Maestro reads text and id as case-insensitive regular expressions. `GetByText` and `GetByTestId` match the whole string, case included. Write `GetByText(new Regex("^mute account$", RegexOptions.IgnoreCase))`, or `exact: false` for a substring of text. +- Maestro taps the first of several matches. e2e fails with `LOCATOR_AMBIGUOUS`, so add `.First()` when the first match is what you mean. +- `IsVisibleAsync()` replaces `optional` and `runFlow: { when }`. It reads the screen one time and doesn't wait. It fails on several matches, so add `Visible = true` or `.First()` when a hidden twin may exist. +- `below`, `above`, `leftOf`, and `rightOf` have no port, see [Not planned](#not-planned). `Filter(has:)` and chaining match nothing on a plain React Native `View` on iOS, see [Known device limitations](/get-started/mobile/#known-device-limitations). -### AI commands and flow control +## What e2e doesn't have yet -| Maestro | e2e | +Planned is on the roadmap, and Not yet has no date. Not planned is left out on purpose. + +### Planned + +| Maestro | Use today | | --- | --- | -| `assertWithAI`, `assertNoDefectsWithAI` | `Agent.AssertAsync("...")`, `Agent.AssertAsync("...", new AssertOptions { Vision = VisionMode.On })` | -| `extractTextWithAI` | `Agent.ExtractAsync("...")`; the type is the schema | -| `runFlow: file.yaml` | a method on the test class or a base class | -| `runFlow: { when: { visible } }` | `if (await locator.IsVisibleAsync()) { ... }`, see [Rewrite selectors](#rewrite-selectors) | -| `optional: true` on a command | the same `IsVisibleAsync()` check | -| `repeat: { times: n }`, `repeat: { while }` | a `for` or `while` loop | -| `runScript`, `evalScript`, `${output.x}` | C# in the test | -| `http.get`, `http.post`, `json()` | `HttpClient` | -| `onFlowStart`, `onFlowComplete` | `[SetUp]`, `[TearDown]` (NUnit), `InitializeAsync`, `DisposeAsync` (xUnit v3) | -| `startRecording`, `stopRecording` | `App.StartRecordingAsync(new RecordingOptions { Name = "..." })`, `App.StopRecordingAsync()` | +| `permissions: { all }`, the grant-all default, names such as `bluetooth` and `storage` | Name each permission in `app.permissions`, or accept the prompt with `Device.AlertAsync(AlertAction.Accept)` | +| `point: "50%,50%"`, percent `start` and `end` on `swipe` | Compute pixels from a full-screen node's `BoundingBoxAsync()` | +| `inputText` and `pressKey: Enter` with no field, `eraseText: N`, `pressKey: backspace`, `tab` | `FillAsync` or `PressAsync` on the field's locator; `FillAsync` a shortened `InputValueAsync()` to erase; tap the next field for Tab | +| `addMedia` | On an iOS simulator, `xcrun simctl addmedia` on `Device.UnstableRaw.DeviceId` (the simulator's UDID) before `App.OpenAsync()`; Android is unchecked. `OpenAsync` restarts the app, so import the media before the flow navigates to the picker, or navigate back to it afterwards. See [Photos for a picker test](/get-started/mobile/#photos-for-a-picker-test) | +| `label` on a command | `-`. The call names each step | +| `scrollUntilVisible: { centerElement, visibilityPercentage }` | No exact match. After it resolves, read `BoundingBoxAsync()`, then `Screen.SwipeAsync` with `Momentum.Slow` if the node is off center | +| Device logs and crash reports beside a failure | `adb logcat` or `xcrun simctl spawn log stream`, collected in a `[SetUp]` or `InitializeAsync` | +| `maestro hierarchy` as a terminal command | A one-line test that calls `Screen.DebugAsync()` | +| `--device-locale` | `app.launchArguments` of `-AppleLanguages (de) -AppleLocale de_DE` on iOS, see [Run the app in another language](/get-started/mobile/#run-the-app-in-another-language); on Android, set the device language before the run | +| Maestro Cloud | Your CI with a [hosted device provider](/get-started/mobile/#hosted-devices) | +| `pressKey: lock`, `power`, `volume up`, `volume down` | `Device.UnstableRaw.Client.CallAsync(DeviceMethods.CommandHardwareButton, ...)` with `button` set to `"power"`, `"volume-up"`, or `"volume-down"` (no `lock`): power and volume on Android, volume on a physical iPhone. See [what the raw client takes](/get-started/mobile/#what-the-raw-client-takes) | +| JUnit `properties`, `--format html`, a JUnit path with `--output` | Ids in categories or the title; each failed test gets a `trace.md` page under `.e2e/results/`; copy the logger's output after the run | +| `--shards`, `--shard-split` | The framework's parallelism on one machine; the port has no `--shard` | +| `--device` on the command line | `E2E_MOBILE_DEVICE` | -## Rewrite selectors +### Not yet + +| Maestro | Use today | +| --- | --- | +| `optional: true` | `if (await locator.IsVisibleAsync())` or try/catch; the report shows no warning | +| `executionOrder` | One test method that calls each flow as a method; the port has no `describe` option for serial order | +| `retry: { maxRetries, commands }` | A `for` loop with try/catch; `Expect.Poll` for a value, NUnit `[Retry]` for the whole test (xUnit v3 has none) | +| Android process death with `killApp` | `Device.HomeAsync()`, then `adb shell am kill `, then `Device.OpenAppAsync(id)` | +| `assertDarkMode`, `assertLightMode` | Assert on a themed control | +| `maestro test -c` | `e2e mcp` gives a coding agent a live session on the device, see [Coding agents](/coding-agents/); the port has no `--watch` | +| TV remote keys | `-` | -Four differences from Maestro's selectors cause most edits. +### Not planned -- Maestro reads text and id as case-insensitive regular expressions, so `tapOn: ".*Save.*"` matches any text with Save in it, and `"Mute Account"` matches `Mute account`. `GetByText` and `GetByTestId` match the whole string, case included. Write `GetByText(new Regex("^mute account$", RegexOptions.IgnoreCase))` or `GetByTestId(new Regex("^row-"))` for a pattern, or `exact: false` for a case-insensitive substring of text. -- Maestro taps the first of several matches. e2e fails with `LOCATOR_AMBIGUOUS`, so write `.First()` when the first match is what you mean. -- `IsVisibleAsync()`, the check that replaces `optional` and `runFlow: { when }`, is `false` on zero matches and `LOCATOR_AMBIGUOUS` on several. Add `Visible = true` to the query or `.First()` when a hidden twin may exist. -- `below`, `above`, `leftOf`, `rightOf`, and `containsDescendants` have no port. e2e has `Filter(has:, hasText:)` and chaining for structure, and nothing spatial. Use `GetByLabel`, a test id, or a goal: `Agent.ActAsync("enter {email} in the email field", ...)` finds the field. +| Maestro | Use instead | +| --- | --- | +| Text and id as regex, first match wins | A `Regex` and `.First()`, see [Rewrite selectors](#rewrite-selectors) | +| `retryTapIfNoChange` | Assert the result, then tap again in the test. A blind second tap can fire a control twice | +| `waitToSettleTimeoutMs` | An assertion with its own `timeout` after the action; that assertion is the wait | +| `toggleAirplaneMode`, `toggleDarkMode` | Set the state you want: `Device.SetAirplaneModeAsync(true)`, `Device.SetAppearanceAsync(Appearance.Dark)` | +| Maestro Studio | A coding agent over `e2e mcp`, or `Screen.DebugAsync()` | +| `waitForAnimationToEnd` | `await Expect.That(locator).ToBeVisibleAsync()` on what the animation ends with; `MobileOptions.SettleMilliseconds` and `TransitionMilliseconds` for the engine | +| `below`, `above`, `leftOf`, `rightOf` | `GetByLabel`, a test id, `Filter(has:)` on a shared container, or an `Agent.ActAsync` goal | ## What differs on purpose +- Maestro grants every permission when it launches the app. e2e grants none unless you name it, so a ported flow meets system prompts Maestro never showed. Name each one in `app.permissions`, with `grant`, `deny`, and `reset` for Maestro's `allow`, `deny`, and `unset`. - Tests get no agent-device client. Every call goes through `Screen`, `App`, `Device`, or the agent, so e2e records and redacts it. `Device.UnstableRaw` is the unstable exception: its calls are not steps, so the report and trace leave them out, the agent cache does not replay them, nothing redacts their arguments, and nothing waits for the screen to settle after one. - A device has no saved sessions. Sign in inside the test or in `[SetUp]`. diff --git a/docs-site/src/content/docs/migrate/playwright.mdx b/docs-site/src/content/docs/migrate/playwright.mdx index e986bd7b9..19d019668 100644 --- a/docs-site/src/content/docs/migrate/playwright.mdx +++ b/docs-site/src/content/docs/migrate/playwright.mdx @@ -12,13 +12,13 @@ This guide ports upstream's `docs/migrate/playwright.mdx` (tester-army/e2e at `7 ```text Migrate this project's Playwright .NET tests to e2e, one test class at a time. Docs: the "From Playwright .NET" page of the e2e-dotnet docs site (docs-site/src/content/docs/migrate/playwright.mdx). -1. Read the migration guide. It maps every Playwright .NET context option, Page call, and test attribute to e2e and marks what e2e does not have yet. +1. Read the migration guide. It maps the Playwright .NET context options, Page calls, and test attributes that work in e2e, and lists what e2e doesn't have yet. 2. If the solution has no e2e test project, run `e2e init --yes` beside the Playwright project (add `--framework xunit` for xUnit v3). It creates the test project and example test, writes e2e.config.json, and installs the skill. In an existing test project it adds the missing package references to the .csproj. 3. Read the e2e skill (`e2e guide` prints it, `e2e guide ` prints a topic) and follow it for every step below. 4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. If it needs a sign-in, give me the `e2e login ` command and wait until I confirm. 5. Port the context options and the base URL into e2e.config.json and the WebEngineOptions with the guide's config tables. Keep Microsoft.Playwright installed in the old project. 6. List the Playwright test classes and ask me which one to move first. Suggest a flow whose steps change often, such as a wizard or a checkout. -7. Rewrite that class with the same framework (NUnit or xUnit v3), deriving from E2ETest. Map each Page call to Screen, App, or Browser with the guide. Where a Playwright query matched a fragment, add exact: false or a Regex. Where the guide marks a feature missing, use the workaround it gives, or leave that test in Playwright and tell me why. +7. Check each Playwright .NET API in that class against "What e2e doesn't have yet". Use the "Use today" or "Use instead" entry, or leave that test in Playwright and tell me why. Then rewrite the class with the same framework (NUnit or xUnit v3), deriving from E2ETest. Map each Page call to Screen, App, or Browser with the guide. Where a Playwright query matched a fragment, add exact: false or a Regex. 8. Keep navigation and the final assertions deterministic, with App.OpenAsync and Expect.That. Where a run of scripted interactions breaks often, propose an Agent.ActAsync goal for it instead. 9. Run the new class with `dotnet test --filter "FullyQualifiedName~"` and fix it until it passes. Keep the Playwright test until I confirm it can go. 10. Summarize which tests moved, which stayed in Playwright, and why. @@ -34,40 +34,36 @@ Playwright .NET has no config file: a test class overrides `ContextOptions()`, a ### App, targets, and output -| Playwright .NET | e2e | Status | -| --- | --- | --- | -| `ContextOptions().BaseURL` | `app.url` on the target, or an override of `BaseUrl` on the test class | renamed | -| a web server you start yourself or from a `[OneTimeSetUp]` | `app: { url, command }` on the target; targets that declare the same command share one process | renamed | -| a `[OneTimeSetUp]` or `SetUpFixture` that seeds data | the framework's setup, a `[E2ESetup]` test for sign-in | same | -| the `BROWSER` environment variable, `Playwright.BrowserName` in `.runsettings` | `WebEngineOptions.BrowserName`: `BrowserName.Chromium`, `Firefox`, or `Webkit`. Two browsers on one app are two targets | renamed | -| `ExpectTimeout` in `.runsettings`, `Expect.SetDefaultExpectTimeout` | `assertionTimeout` in the config, or the `AssertionTimeout` override | renamed | -| `Page.SetDefaultTimeout` | `actionTimeout` in the config, or the `ActionTimeout` override | renamed | -| `Page.SetDefaultNavigationTimeout` | `timeout`, the test budget; `Browser.GotoAsync(url, new GotoOptions { Timeout })` overrides one call | renamed | -| NUnit's `[Timeout]`, `Retry`, `NumberOfTestWorkers` | the same attributes and settings: the framework owns them, and the config `retries` key is checked and not used. See [defaults that differ](#defaults-that-differ) | by design | -| the HTML report, `playwright.ps1 show-report` | `report.json` and `.e2e/results//trace.md` | missing | -| the `OutputDir` of a `PageTest` | `.e2e`, holding `results/` and the replay cache `cache/` | renamed | +| Playwright .NET | e2e | +| --- | --- | +| `ContextOptions().BaseURL` | `app.url` on the target, or an override of `BaseUrl` on the test class | +| a web server you start yourself or from a `[OneTimeSetUp]` | `app: { url, command }` on the target; targets that declare the same command share one process | +| a `[OneTimeSetUp]` or `SetUpFixture` that seeds data | the framework's setup, a `[E2ESetup]` test for sign-in | +| the `BROWSER` environment variable, `Playwright.BrowserName` in `.runsettings` | `WebEngineOptions.BrowserName`: `BrowserName.Chromium`, `Firefox`, or `Webkit`. Two browsers on one app are two targets | +| `ExpectTimeout` in `.runsettings`, `Expect.SetDefaultExpectTimeout` | `assertionTimeout` in the config, or the `AssertionTimeout` override | +| `Page.SetDefaultTimeout` | `actionTimeout` in the config, or the `ActionTimeout` override | +| `Page.SetDefaultNavigationTimeout` | `timeout`, the test budget; `Browser.GotoAsync(url, new GotoOptions { Timeout })` overrides one call | +| NUnit's `[Timeout]`, `Retry`, `NumberOfTestWorkers` | the same attributes and settings: the framework owns them, and the config `retries` key is checked and not used. See [defaults that differ](#defaults-that-differ) | +| the `OutputDir` of a `PageTest` | `.e2e`, holding `results/` and the replay cache `cache/` | ### Browser, timeouts, and artifacts -| Playwright .NET | e2e | Status | -| --- | --- | --- | -| `ViewportSize` | `WebEngineOptions.Viewport`, default 1280 by 720 | renamed | -| `Playwright.Selectors.SetTestIdAttribute` | `WebEngineOptions.TestIdAttribute` | renamed | -| `HttpCredentials` | `WebEngineOptions.BasicAuth = new WebBasicAuth(username, secret)`, answered for the app's origin only; `CredentialOrigins` adds more | renamed | -| `ExtraHTTPHeaders` | `WebEngineOptions.Headers`, sent on requests that start at the app's origin; `CredentialOrigins` adds more | renamed | -| `BrowserType.ConnectOverCDPAsync(url)` | `WebEngineOptions.Connect = new WebConnectOptions { CdpEndpoint = ct => ... }`, Chromium only | renamed | -| `BrowserType.ConnectAsync(wsEndpoint)` | `Connect` attaches over CDP only | missing | -| `StorageState`, a setup test that saves it | `[E2ESetup("title", Saves = ["admin"])]` with `SaveSessionAsync("admin")`, then `[E2ESession("admin")]` on a test; no file | by design | -| `StorageState` with fixed cookies | `Browser.SetCookiesAsync(cookies)` in a `[SetUp]` | renamed | -| `Tracing.StartAsync` and `StopAsync` | `trace`: `off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries`, in the config, per target, or `E2E_TRACE`. A trace is a markdown page, `trace.md`, not a zip, with no viewer. No `retain-on-first-failure`, `retain-on-failure-and-retries`, or object form. The default differs, see [below](#defaults-that-differ) | renamed | -| `RecordVideoDir` | `video`, with the modes of `trace` and `E2E_VIDEO`; no `RecordVideoSize` | renamed | -| `Headless = false`, `HEADED=1` | `E2E_HEADLESS=0`, or `new WebEngineOptions { Headless = false }` | renamed | -| `Locale`, `TimezoneId` | `WebEngineOptions.Locale`, `TimezoneId` | renamed | -| `ColorScheme`, `ReducedMotion` | `WebEngineOptions.ColorScheme`, `ReducedMotion` | renamed | -| `UserAgent` | `WebEngineOptions.UserAgent` | renamed | -| `Playwright.Devices["iPhone 15"]`, `DeviceScaleFactor`, `IsMobile`, `HasTouch` | `Viewport` sets the size only | missing | -| `Permissions` | `app.permissions` on the target, `{ "clipboard-read": "grant" }`: `grant` only, held by every page on any origin. See [Granting browser permissions](/get-started/web/#granting-browser-permissions) in the web guide | renamed | -| `Geolocation` (the position; the permission is `app.permissions` above), `IgnoreHTTPSErrors`, `BypassCSP`, `Offline`, `Proxy`, `LaunchOptions` | not yet | missing | +| Playwright .NET | e2e | +| --- | --- | +| `ViewportSize` | `WebEngineOptions.Viewport`, default 1280 by 720 | +| `Playwright.Selectors.SetTestIdAttribute` | `WebEngineOptions.TestIdAttribute` | +| `HttpCredentials` | `WebEngineOptions.BasicAuth = new WebBasicAuth(username, secret)`, answered for the app's origin only; `CredentialOrigins` adds more | +| `ExtraHTTPHeaders` | `WebEngineOptions.Headers`, sent on requests that start at the app's origin; `CredentialOrigins` adds more | +| `BrowserType.ConnectOverCDPAsync(url)` | `WebEngineOptions.Connect = new WebConnectOptions { CdpEndpoint = ct => ... }`, Chromium only | +| `StorageState`, a setup test that saves it | `[E2ESetup("title", Saves = ["admin"])]` with `SaveSessionAsync("admin")`, then `[E2ESession("admin")]` on a test; no file | +| `StorageState` with fixed cookies | `Browser.SetCookiesAsync(cookies)` in a `[SetUp]` | +| `Tracing.StartAsync` and `StopAsync` | `trace`: `off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries`, in the config, per target, or `E2E_TRACE`. A trace is a markdown page, `trace.md`, not a zip, with no viewer. No `retain-on-first-failure`, `retain-on-failure-and-retries`, or object form. The default differs, see [below](#defaults-that-differ) | +| `RecordVideoDir` | `video`, with the modes of `trace` and `E2E_VIDEO`; no `RecordVideoSize` | +| `Headless = false`, `HEADED=1` | `E2E_HEADLESS=0`, or `new WebEngineOptions { Headless = false }` | +| `Locale`, `TimezoneId` | `WebEngineOptions.Locale`, `TimezoneId` | +| `ColorScheme`, `ReducedMotion` | `WebEngineOptions.ColorScheme`, `ReducedMotion` | +| `UserAgent` | `WebEngineOptions.UserAgent` | +| `Permissions` | `app.permissions` on the target, `{ "clipboard-read": "grant" }`: `grant` only, held by every page on any origin. See [Granting browser permissions](/get-started/web/#granting-browser-permissions) in the web guide | ## Rewrite one test @@ -127,182 +123,141 @@ The agent does the checkout steps and adapts when the form changes. A test can a ## Map each API -`Status` is `same`, `renamed`, `web only`, `by design`, or `missing` with what to do today. See [What differs on purpose](#what-differs-on-purpose) for the `by design` rows. Calls on `Browser` run only in a browser, so a device or document target fails them with `UNSUPPORTED_CAPABILITY`. Calls on `Screen`, `App`, `Agent`, and `Expect.That(locator)` run on every target. +Every row works today. For an API that is not here, check [What e2e doesn't have yet](#what-e2e-doesnt-have-yet). Calls on `Browser` run only in a browser, so a device or document target fails them with `UNSUPPORTED_CAPABILITY`. Calls on `Screen`, `App`, `Agent`, and `Expect.That(locator)` run on every target. ### Locators -| Playwright .NET | e2e | Status | -| --- | --- | --- | -| `Page.GetByRole(AriaRole.Button, new() { Name = "Save" })` | `Screen.GetByRole("button", "Save")`; the role is a string and the name is positional | renamed | -| `GetByLabel`, `GetByText`, `GetByPlaceholder`, `GetByTestId` | the same on `Screen` | same | -| `GetByRole`, `GetByLabel`, `GetByPlaceholder`, `GetByText` with a string | an exact, case-sensitive match, see [What differs on purpose](#what-differs-on-purpose) | by design | -| `GetByRole(AriaRole.Img)` | the same; `img` is an alias of `image` | same | -| `GetByAltText(alt)` | `GetByRole("image", alt)` | renamed | -| `GetByTitle` | the node carries no title | missing | -| `GetByRole(role, new() { IncludeHidden = true })` | a role query never matches a hidden node; `Visible = true` on the other queries drops hidden twins | by design | -| `Filter(new() { HasText, HasNotText, Has, HasNot })`, `First`, `Last`, `Nth(i)`, `CountAsync()` | `Filter(hasText:, hasNotText:, has:, hasNot:)`, `First()`, `Last()`, `Nth(i)`, `CountAsync()` | same | -| `Locator.Or(other)` | the same; a locator is strict, so add `First()` when both match | same | -| `Locator.And(other)` | not yet | missing | -| `AllAsync()`, `AllTextContentsAsync()` | the same; `AllInnerTextsAsync()` is `AllTextContentsAsync()` | same | -| `Page.Locator("css")`, `Page.Locator("xpath=...")` | `Browser.Locator("css")`: the string is a Playwright selector, so `xpath=` (or a leading `//`) and `>>` chains work too | web only | -| `Page.WaitForSelectorAsync(selector)` | `Browser.Locator(selector).WaitForAsync()`; a locator is strict | renamed | -| `Page.ClickAsync(selector)`, `FillAsync(selector, value)`, `GetAttributeAsync(selector, name)`, and the other `Page` methods that take a selector | `Browser.Locator(selector).TapAsync()`, `.FillAsync(value)`, `.GetAttributeAsync(name)`. Playwright acts on the first match; a locator is strict, so add `First()` for the old behavior | renamed | -| `Locator.Locator("css")` | a semantic query inside the scope (`cell.GetByRole("textbox")` for `cell.Locator("input")`), `Browser.Locator("css").GetByRole(...)` the other way round, or one `Browser.Locator` selector for the whole path | missing | -| `Page.FrameLocator("#frame")` | `Browser.FrameLocator("#frame")`, then `Screen` queries inside it. A plain query never enters an iframe, as in Playwright | web only | +| Playwright .NET | e2e | +| --- | --- | +| `Page.GetByRole(AriaRole.Button, new() { Name = "Save" })` | `Screen.GetByRole("button", "Save")`; the role is a string and the name is positional | +| `GetByLabel`, `GetByText`, `GetByPlaceholder`, `GetByTestId` | the same on `Screen` | +| `GetByRole`, `GetByLabel`, `GetByPlaceholder`, `GetByText` with a string | an exact, case-sensitive match, see [What differs on purpose](#what-differs-on-purpose) | +| `GetByRole(AriaRole.Img)` | the same; `img` is an alias of `image` | +| `GetByAltText(alt)` | `GetByRole("image", alt)` | +| `Filter(new() { HasText, HasNotText, Has, HasNot })`, `First`, `Last`, `Nth(i)`, `CountAsync()` | `Filter(hasText:, hasNotText:, has:, hasNot:)`, `First()`, `Last()`, `Nth(i)`, `CountAsync()` | +| `Locator.Or(other)` | the same; a locator is strict, so add `First()` when both match | +| `AllAsync()`, `AllTextContentsAsync()` | the same; `AllInnerTextsAsync()` is `AllTextContentsAsync()` | +| `Page.Locator("css")`, `Page.Locator("xpath=...")` | `Browser.Locator("css")`: the string is a Playwright selector, so `xpath=` (or a leading `//`) and `>>` chains work too | +| `Page.WaitForSelectorAsync(selector)` | `Browser.Locator(selector).WaitForAsync()`; a locator is strict | +| `Page.ClickAsync(selector)`, `FillAsync(selector, value)`, `GetAttributeAsync(selector, name)`, and the other `Page` methods that take a selector | `Browser.Locator(selector).TapAsync()`, `.FillAsync(value)`, `.GetAttributeAsync(name)`. Playwright acts on the first match; a locator is strict, so add `First()` for the old behavior | +| `Page.FrameLocator("#frame")` | `Browser.FrameLocator("#frame")`, then `Screen` queries inside it. A plain query never enters an iframe, as in Playwright | ### Actions -| Playwright .NET | e2e | Status | -| --- | --- | --- | -| `ClickAsync()` | `TapAsync()`; `ClickAsync()` is an alias | same | -| `ClickAsync(new() { Button = MouseButton.Right })` | `SecondaryTapAsync()` | renamed | -| `DblClickAsync()` | `DoubleTapAsync()`; `DblClickAsync()` is an alias | same | -| `ClickAsync(new() { Position })` | `TapAsync(new TapOptions { Position })`, an offset from the node's top-left corner inside its border | renamed | -| `ClickAsync(new() { Force })` | not yet | missing | -| `ClickAsync(new() { Modifiers })`, `DblClickAsync(new() { Modifiers })` | `TapAsync(new ClickOptions { Modifiers = [...] })`, `DoubleTapAsync`, `SecondaryTapAsync`; web only | renamed | -| `ClickAsync(new() { ClickCount, Delay, Trial })` | not yet | missing | -| `Trial`, `Force`, `NoWaitAfter` on `CheckAsync`, `FillAsync`, and the other actions | no such option: the options type has no member for them | missing | -| `FillAsync(value)` | the same; also takes a `Secret` | same | -| `PressSequentiallyAsync(text, new() { Delay })` | `PressSequentiallyAsync(text, new PressSequentiallyOptions { Delay })`; focuses the field, then types through the platform keyboard, one call for the whole text or one per code point with a `Delay`; the call shares one `Timeout` and fails with `ACTION_FAILED` when a pause reaches it; a `Secret` has no overload, so it goes through `FillAsync` | same | -| `PressAsync(key)` | the same; modifiers join with `+` | same | -| `ClearAsync()`, `FocusAsync()`, `HoverAsync()`, `SetInputFilesAsync()`, `DragToAsync()` | the same verbs, each with `Timeout` as the only option (`ActionOptions`): no `Force`, `Trial`, `Position`, `Modifiers`, or `SourcePosition`. `SetInputFilesAsync([])` clears the input | renamed | -| `CheckAsync()`, `UncheckAsync()` | the same, except when the state updates late or the click removes the control, see [What differs on purpose](#what-differs-on-purpose). `UncheckAsync()` on a checked native radio is `ACTION_FAILED` (`NOT_ACTIONABLE`); a `role=radio` is clicked, as in Playwright | by design | -| `SelectOptionAsync("pro")` | `SelectOptionAsync("Pro")` selects the option whose value or label is the string | same | -| `SelectOptionAsync(new[] { "a", "b" })`, `SelectOptionAsync(new SelectOptionValue { Index })` | one value or label per call | missing | -| `SetInputFilesAsync(new FilePayload { ... })`, `SetInputFilesAsync(Array.Empty())` | paths only, resolved from the project root; an empty list throws `ArgumentException`, so there is no clear | missing | -| `ScrollIntoViewIfNeededAsync()` | `ScrollIntoViewAsync()` | renamed | -| `InnerTextAsync()` | `TextContentAsync()`, whitespace collapsed to single spaces | renamed | -| `TextContentAsync()` | the rendered text, what `InnerTextAsync()` reads, see [What differs on purpose](#what-differs-on-purpose) | by design | -| `GetAttributeAsync()`, `IsVisibleAsync()`, `IsHiddenAsync()`, `IsEnabledAsync()`, `IsDisabledAsync()`, `IsCheckedAsync()`, `BoundingBoxAsync()` | the same | same | -| `InputValueAsync()` | the same, and reads a `contenteditable` editor, where Playwright throws. A checkbox or radio reads its value attribute, `on` when it has none | by design | -| `InputValueAsync()`, `IsCheckedAsync()` on a `