Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ The port installs the skill with `npx skills add hardkoded/e2e-dotnet` and regis
- Upstream's `device.unstable_raw` (`feat(mobile): device.unstable_raw exposes the test's agent-device session`, `c3c4238`, tester-army/e2e#1380) is `Device.UnstableRaw`. Upstream's `DeviceSession` interface (`device.ts`) is `RawDeviceSession`, a .NET-only name, because the port's `DeviceSession` is the surface (upstream's `AgentDeviceSurface`) and a public type of that name exists. Its `client` is the session's `IDeviceClient` (the port's stand-in for `AgentDeviceClient`), `selection` is `DeviceSelection`, and `platform` is `MobilePlatform`. `BoundDevice` (`surface.ts`) is internal, set by the boot from the `id` and `device` of the `devices.boot` answer (`DeviceSession.BoundDevice`, upstream's `boundDevice`), and cleared when the session closes. Reading `UnstableRaw`, or `Client` of a handle taken earlier, after the session closed is `INVALID_STATE`. The client is `DeviceCommands.Translate`, which refuses a method it does not list, so `DeviceMethods.AppsList` (`apps.list`, the daemon's `apps` command) is added for upstream's reference example; the other methods of upstream's client stay unsupported until code calls them. Tests: the three `it` of `describe('unstable_raw')` in `Engine/DeviceFixtureUnstableRawTests`; the first is `Partial_`, since the port's fixture has no map of declared operations for `expect(declared).not.toHaveProperty('unstable_raw')`. Not ported: `unstable_raw reaches the test session through agent-device` (`apps/mobile-benchmark/tests/device-fixture.e2e.ts`), which needs the mobile benchmark app, a React Native app the port lacks.
- Upstream's known-viewport fix (`fix(mobile): keep the known viewport when the app reads smaller than the screen`, `7674a2c`, tester-army/e2e#1291) is ported. `DeviceSession.Project` learns a snapshot's viewport only when it is plainly the screen's: the first one the session reads, the one it knows, or that one rotated (`SameScreen`, upstream's `sameScreen`; two sizes within a point are one screen, since rects from the tree carry fractions). `ViewportForAsync` (upstream's `viewportFor`) checks a tree of any other size against the device's screenshot, in either orientation, before it replaces the known one: a screenshot reports the portrait size even in landscape, so it confirms a size but cannot pick the orientation. The size it checks against is `DeviceSupport.LogicalScreenSize` (upstream's `logicalScreenSize`): the screenshot's logical dimensions, else its pixel dimensions over its density. A simulator's screenshot reports those; a real device's reports neither, so there the tree's size is taken as it comes. A screenshot that fails during that check fails the observation, as upstream's does. `TraceScreen` and `MaskedScreenshotAsync` size the trace screen and the secure-field masks by the known viewport, not the tree's. On iOS, a swipe from a sheet's grabber can pull down Notification Center, and the app's tree behind it reported a third of the screen (130x281 on a 390x844 device), which misplaced every rect and tap after it. Upstream records this only in its changeset; its skill references and docs say nothing about it, so `SKILL.md` and `docs/` have nothing to mirror. Tests: in `Engine/ObservationTests`, 'keeps the known viewport for an app drawn smaller than the screen, and follows a rotation' (new) and 'measures the screen in the viewport the snapshot publishes, and in its tree when it publishes none' (split into two harnesses, as upstream's); in `Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests`, 'drags at the centre of a screen rotated since the last observation' (new), since the keyboard drag measures its start through `ViewportForAsync` too.
- Upstream's input budget (`fix(mobile): bound each input by its budget`, `14b77d3`, tester-army/e2e#1073) is ported. An input (a tap or other node action, a point action, typing, a key, `back`, a keyboard dismissal, and the node `LongPressAsync`, `HoverAsync`, `DragToAsync`, and `SwipeAsync` the port has in place of upstream's `longPress`, `hover`, `dragTo`, and `swipe` actions) runs its device commands under `InputBudget` (`DeviceSupport.cs`, upstream's `support.ts`): with no budget left the input is not sent and fails with `OPERATION_TIMEOUT`, an input that does not answer in time fails with `ACTION_MAY_HAVE_COMMITTED`, one sent with almost no budget left still gets 250 ms to answer, a hold (a long press or timed swipe) at least as long as the budget left is refused with `NOT_ACTIONABLE`, and each command of a multi-command input (a key on an unfocused field) is sent on its own. Reads, launches, artifacts, and the `Device` fixture calls (including `device.dismissKeyboard`, which passes no budget, as upstream's `dismissKeyboard(signal)` does) are unbounded. Differences: upstream reads the budget per call, from `operation.timeoutMs`, which is the call's `timeout` or `actionTimeout`. The port's session calls carry no timeout, so every input gets `DeviceSession.ActionTimeout`, which `MobileStartInfo.ActionTimeout` sets (`EngineStartOptions.ActionTimeout` fills it), as the web engine's operations get the action timeout. `DeviceSession.InputBudget` is upstream's `inputBudget` and `DeviceSession.SendInputAsync` its `sendInput`. `InputBudget`'s `track` is an `Action<Task>` that registers the task (upstream's returns the promise it is given); see [In-flight commands](#in-flight-commands). The `OperationContext.signal` doc change in `docs/reference/engine.mdx` and `contract.ts` has no counterpart: the port has no `OperationContext`. Tests: in `Engine/ObservationTests`, 'ends %s, hung on %s, at its budget as possibly committed' (a theory of the same seven entries; the screen swipe is a `perform` on the root), 'sends nothing once a settling target spent the budget, and no keystroke once a focus tap used it up', and 'refuses a long press that holds longer than its timeout leaves, before sending it' (also on the node `LongPressAsync`); in `Support/SupportHelpersTests`, 'sends an input with what is left of its budget, and reports one cut off as possibly committed'. These run on real timers of a few hundred milliseconds, where upstream's fake timers do not wait. 'lets a locate outlive its budget' was already ported.
- Upstream's Compose Multiplatform text fields on iOS (`mobile reads Compose Multiplatform text fields on iOS`, `75ae2ab`, tester-army/e2e#1427) are ported in part. `SnapshotProjection.ProjectSnapshot` reads the placeholder an empty iOS field folds into its label (upstream's `foldedPlaceholder` in `nodes.ts`, `FoldedPlaceholder` here): for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out of the tree, and the name is the label before `, ` (a non-empty `ContentDescription` stays the name, and that label is then the `Text`). A field of any other shape keeps its label and children as reported. The private record `Folded` is a .NET-only name for upstream's inline `{ label, placeholder, child }` object. Test: 'reads the placeholder an empty Compose Multiplatform field folds into its label' in `Nodes/SnapshotProjectionTests`. Not ported: `mayTakeText` and `NodeBinding.takesText` (`nodes.ts`, `surface.ts`), with the three `it`s that use them in `describe('input freshness')` of `engine.test.ts` ('acts on an unclassified iOS field', 'rejects an unclassified iOS element', 'rejects an unclassified iOS element that gained a control before typing'). The port's `fill` and `clear` have no check of the node's role and send the text to any node with a ref, so an iOS `other` is already typed into; the check upstream widens does not exist here. Adding it would be a new refusal, not this change; it is tracked in #850.
- Upstream's Compose Multiplatform text fields on iOS (`mobile reads Compose Multiplatform text fields on iOS`, `75ae2ab`, tester-army/e2e#1427) are ported in part. `SnapshotProjection.ProjectSnapshot` reads the placeholder an empty iOS field folds into its label (upstream's `foldedPlaceholder` in `nodes.ts`, `FoldedPlaceholder` here): for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out of the tree, and the name is the label before `, ` (a non-empty `ContentDescription` stays the name, and that label is then the `Text`). A field of any other shape keeps its label and children as reported. The private record `Folded` is a .NET-only name for upstream's inline `{ label, placeholder, child }` object. Test: 'reads the placeholder an empty Compose Multiplatform field folds into its label' in `Nodes/SnapshotProjectionTests`. `mayTakeText` and `NodeBinding.takesText` (`nodes.ts`, `surface.ts`) are ported (#850): `SnapshotProjection.MayTakeText` and `NodeBinding.TakesText`, set in `Bind` from the node with its children. `DeviceActions.PerformAsync` refuses a `fill` or `clear` whose node does not take text with `UNSUPPORTED_CAPABILITY` (the message of upstream's `refuseFill`, `RefuseFill` here), before the action and again on the binding found by a fresh snapshot taken just before typing (upstream's `relocated`; `DeviceSession.RelocatedAsync`, with the transition wait split out as `WaitForTransitionAsync`, upstream's `waitForTransition`). Tests: 'Acts_on_an_unclassified_iOS_field', 'Rejects_an_unclassified_iOS_element', and 'Rejects_an_unclassified_iOS_element_that_gained_a_control_before_typing' in `Engine/InputFreshnessTests`. Differences: a fill or clear now reads the field again, so it costs one more `capture.snapshot` than before. Upstream's `relocated` also refuses, as `NODE_STALE`, a truncated capture or a changed input purpose, disabled, secure, or hidden state before typing; the port has no such check yet, so only the `takesText` re-check of this change is ported. The other `it`s of upstream's `describe('input freshness')` are not part of this change.
- Upstream's WebDriver element-type fix (`fix(mobile): read XCUIElementType class names as their XCTest types`, `3f9a7d8`, tester-army/e2e#1133) is ported. `SnapshotProjection.NormalizeKind` is upstream's `normalizeKind` in `nodes.ts`: it drops `^XCUIElementType(?=[A-Z])` (`XcuiElementTypePrefix`, a .NET `[GeneratedRegex]` for upstream's inline regex) before it maps the type, so the `XCUIElementTypeButton` that agent-device's WebDriver runtime reports on hosted Appium clouds reads as `Button`. `DeviceSession.KeyboardShowing` is upstream's `keyboardShowing` in `surface.ts`: it routes node types through `NormalizeKind`, so an `XCUIElementTypeKeyboard` counts as a shown keyboard. Upstream records this only in its changeset; its skill references and docs say nothing about it, so `SKILL.md` and `docs/` have nothing to mirror. Tests: in `Nodes/SnapshotProjectionTests`, 'reads the XCUIElementType class names a WebDriver session reports as the native XCTest types'; in `Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests`, 'reads an XCUIElementTypeKeyboard from a WebDriver session as a shown keyboard'.
- Upstream's held-device skip (`mobile: skip booted devices another agent-device session holds`, `9e53c40`, tester-army/e2e#1379) is ported. `BootedDevices` (`DevicePool.cs`, upstream's class in `pool.ts`) skips a booted device whose inventory entry has a `claimedBy.session` that is not one of this target's own slot sessions, logs the skipped ones, and fails `prepare` with a non-retryable `ENGINE_FAILURE` when every booted device is held. `BootedDevice` is upstream's interface of that name, as a record. `LifecycleTests.Skips_a_booted_device_another_agent_device_session_holds_keeps_one_its_own_slot_session_holds_and_fails_when_every_booted_one_is_held` ports the `lifecycle` `it` of `engine.test.ts`, whole.
- Not ported: the tool-pixels test (needs the full runner and a scripted model), and the `device.openApp` check of an unknown permission name.
Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/get-started/mobile.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ public sealed class GreetingTests : MobileAppTest
}
```

`GetByTestId` finds a control by its accessibility identifier. A field's hint is its `Placeholder`, shown or not, so `GetByPlaceholder` finds it. Compose Multiplatform reports no placeholder on iOS: an empty field appends the text it draws to its label (`Name, Ada Lovelace`) over one `StaticText` child. For an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text (`Name, Ada Lovelace`), that text is the field's `Placeholder`, the child is left out of the tree, and the name is the label before `, ` (a non-empty `ContentDescription` stays the name, and that label is then the `Text`). A field of any other shape keeps its label and children as reported. So `GetByPlaceholder("Ada Lovelace")` finds such a field and `GetByText("Ada Lovelace")` does not. `Device` is the device fixture: `HomeAsync`, `OpenAppAsync`, `ForegroundAppAsync`, and `SetAppearanceAsync(Appearance.Dark)`.
`GetByTestId` finds a control by its accessibility identifier. A field's hint is its `Placeholder`, shown or not, so `GetByPlaceholder` finds it. Compose Multiplatform reports no placeholder on iOS: an empty field appends the text it draws to its label (`Name, Ada Lovelace`) over one `StaticText` child. For an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text (`Name, Ada Lovelace`), that text is the field's `Placeholder`, the child is left out of the tree, and the name is the label before `, ` (a non-empty `ContentDescription` stays the name, and that label is then the `Text`). A field of any other shape keeps its label and children as reported. So `GetByPlaceholder("Ada Lovelace")` finds such a field and `GetByText("Ada Lovelace")` does not. `FillAsync` and `ClearAsync` read the field again before typing into it, and refuse anything that is not an editable field with `UNSUPPORTED_CAPABILITY`, a button or a text view included. An iOS `other`, the type of an element the platform did not classify, is typed into when it holds a value or only text, since a custom-drawn field (Compose Multiplatform, Flutter) can read that way; an `other` with no value and no children, such as a React Native `Pressable`, or one that holds a control, stays refused. `Device` is the device fixture: `HomeAsync`, `OpenAppAsync`, `ForegroundAppAsync`, and `SetAppearanceAsync(Appearance.Dark)`.

```csharp
await Device.HomeAsync();
Expand Down
2 changes: 1 addition & 1 deletion skills/e2e-dotnet/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ trip. The installed tool prints the same text with `e2e guide <topic>`
| Running tests: `dotnet test` filters, the `e2e` tool, `report.json` for scripts, exit codes, CI, test filters (not wired into `dotnet test`: the `ITestFilter` seam and the `run.changes` and `run.filters` report fields) | `running` | [references/running.md](references/running.md) |
| Handling a failed run: the trace page, `report.json` for scripts, `E2ESession.Report()` for a session you run yourself, error codes and their fixes, a visible browser, replay misses, phase timings, the agent step table, every model call | `debugging` | [references/debugging.md](references/debugging.md) |
| Exploring an app toward a goal without a test file, with `e2e explore` or `Explore.ExploreAsync` | `explore` | [references/explore.md](references/explore.md) |
| Testing an iOS or Android app: `MobileEngine`, the `Device` fixture, picking a simulator (without `MobileOptions.Device`, `MobileEngine` skips booted devices another agent-device session holds, and `prepare` fails with `ENGINE_FAILURE` when every booted one is held), video of the device screen, `Device.UnstableRaw` (the test's agent-device session: name, device id and name, selection, and a raw `IDeviceClient` whose calls are not steps, replayed, or redacted; unstable; `INVALID_STATE` after the session is disposed), Limrun hosted devices, a MAUI app through the DevFlow agent, known limits, `Locator.PressAsync("Enter")` tapping an unfocused field first, and the agent's `open_app` returning to the app under test when it names no app, a Compose Multiplatform field on iOS (for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out, and the name is the label before `, `; a non-empty `ContentDescription` stays the name, and that label is then the `Text`; any other field keeps its label and children) | `mobile` | [references/mobile.md](references/mobile.md) |
| Testing an iOS or Android app: `MobileEngine`, the `Device` fixture, picking a simulator (without `MobileOptions.Device`, `MobileEngine` skips booted devices another agent-device session holds, and `prepare` fails with `ENGINE_FAILURE` when every booted one is held), video of the device screen, `Device.UnstableRaw` (the test's agent-device session: name, device id and name, selection, and a raw `IDeviceClient` whose calls are not steps, replayed, or redacted; unstable; `INVALID_STATE` after the session is disposed), Limrun hosted devices, a MAUI app through the DevFlow agent, known limits, `Locator.PressAsync("Enter")` tapping an unfocused field first, and the agent's `open_app` returning to the app under test when it names no app, a Compose Multiplatform field on iOS (for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out, and the name is the label before `, `; a non-empty `ContentDescription` stays the name, and that label is then the `Text`; any other field keeps its label and children), and `FillAsync` and `ClearAsync` refusing a node that is not an editable field with `UNSUPPORTED_CAPABILITY` (an iOS `other` that holds a value or only text is typed into) | `mobile` | [references/mobile.md](references/mobile.md) |
| Testing a Windows desktop app: `WinAppEngine`, `WinAppTest`, the `Window` fixture, the `windows` target in `e2e explore` and `e2e mcp`, the interactive-desktop requirement, known limits | `windows` | [references/windows.md](references/windows.md) |
| Asked to bug bash or hunt for bugs: proving each bug with a repro test | `bug-bash` | [references/bug-bash.md](references/bug-bash.md) |

Expand Down
Loading
Loading