diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 33b22fe3..e3216aeb 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -410,6 +410,7 @@ The port installs the skill with `npx skills add hardkoded/e2e-dotnet` and regis - Upstream's known-viewport fix (`fix(mobile): keep the known viewport when the app reads smaller than the screen`, `7674a2c`, tester-army/e2e#1291) is ported. `DeviceSession.Project` learns a snapshot's viewport only when it is plainly the screen's: the first one the session reads, the one it knows, or that one rotated (`SameScreen`, upstream's `sameScreen`; two sizes within a point are one screen, since rects from the tree carry fractions). `ViewportForAsync` (upstream's `viewportFor`) checks a tree of any other size against the device's screenshot, in either orientation, before it replaces the known one: a screenshot reports the portrait size even in landscape, so it confirms a size but cannot pick the orientation. The size it checks against is `DeviceSupport.LogicalScreenSize` (upstream's `logicalScreenSize`): the screenshot's logical dimensions, else its pixel dimensions over its density. A simulator's screenshot reports those; a real device's reports neither, so there the tree's size is taken as it comes. A screenshot that fails during that check fails the observation, as upstream's does. `TraceScreen` and `MaskedScreenshotAsync` size the trace screen and the secure-field masks by the known viewport, not the tree's. On iOS, a swipe from a sheet's grabber can pull down Notification Center, and the app's tree behind it reported a third of the screen (130x281 on a 390x844 device), which misplaced every rect and tap after it. Upstream records this only in its changeset; its skill references and docs say nothing about it, so `SKILL.md` and `docs/` have nothing to mirror. Tests: in `Engine/ObservationTests`, 'keeps the known viewport for an app drawn smaller than the screen, and follows a rotation' (new) and 'measures the screen in the viewport the snapshot publishes, and in its tree when it publishes none' (split into two harnesses, as upstream's); in `Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests`, 'drags at the centre of a screen rotated since the last observation' (new), since the keyboard drag measures its start through `ViewportForAsync` too. - Upstream's input budget (`fix(mobile): bound each input by its budget`, `14b77d3`, tester-army/e2e#1073) is ported. An input (a tap or other node action, a point action, typing, a key, `back`, a keyboard dismissal, and the node `LongPressAsync`, `HoverAsync`, `DragToAsync`, and `SwipeAsync` the port has in place of upstream's `longPress`, `hover`, `dragTo`, and `swipe` actions) runs its device commands under `InputBudget` (`DeviceSupport.cs`, upstream's `support.ts`): with no budget left the input is not sent and fails with `OPERATION_TIMEOUT`, an input that does not answer in time fails with `ACTION_MAY_HAVE_COMMITTED`, one sent with almost no budget left still gets 250 ms to answer, a hold (a long press or timed swipe) at least as long as the budget left is refused with `NOT_ACTIONABLE`, and each command of a multi-command input (a key on an unfocused field) is sent on its own. Launches and artifacts are unbounded. Snapshots are bounded since #869 (see the next bullet). Differences: upstream reads the budget per call, from `operation.timeoutMs`, which is the call's `timeout` or `actionTimeout`. The port's session calls carry no timeout, so every input gets `DeviceSession.ActionTimeout`, which `MobileStartInfo.ActionTimeout` sets (`EngineStartOptions.ActionTimeout` fills it), as the web engine's operations get the action timeout. `DeviceSession.InputBudget` is upstream's `inputBudget` and `DeviceSession.SendInputAsync` its `sendInput`. `InputBudget`'s `track` is an `Action` that registers the task (upstream's returns the promise it is given); see [In-flight commands](#in-flight-commands). The `OperationContext.signal` doc change in `docs/reference/engine.mdx` and `contract.ts` has no counterpart: the port has no `OperationContext`. Tests: in `Engine/ObservationTests`, 'ends %s, hung on %s, at its budget as possibly committed' (a theory of the same seven entries; the screen swipe is a `perform` on the root), 'sends nothing once a settling target spent the budget, and no keystroke once a focus tap used it up', and 'refuses a long press that holds longer than its timeout leaves, before sending it' (also on the node `LongPressAsync`); in `Support/SupportHelpersTests`, 'sends an input with what is left of its budget, and reports one cut off as possibly committed'. These run on real timers of a few hundred milliseconds, where upstream's fake timers do not wait. 'lets a locate outlive its budget' was replaced in #869. - Upstream's Compose Multiplatform text fields on iOS (`mobile reads Compose Multiplatform text fields on iOS`, `75ae2ab`, tester-army/e2e#1427) are ported in part. `SnapshotProjection.ProjectSnapshot` reads the placeholder an empty iOS field folds into its label (upstream's `foldedPlaceholder` in `nodes.ts`, `FoldedPlaceholder` here): for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out of the tree, and the name is the label before `, ` (a non-empty `ContentDescription` stays the name, and that label is then the `Text`). A field of any other shape keeps its label and children as reported. The private record `Folded` is a .NET-only name for upstream's inline `{ label, placeholder, child }` object. Test: 'reads the placeholder an empty Compose Multiplatform field folds into its label' in `Nodes/SnapshotProjectionTests`. `mayTakeText` and `NodeBinding.takesText` (`nodes.ts`, `surface.ts`) are ported (#850): `SnapshotProjection.MayTakeText` and `NodeBinding.TakesText`, set in `Bind` from the node with its children. `DeviceActions.PerformAsync` refuses a `fill` or `clear` whose node does not take text with `UNSUPPORTED_CAPABILITY` (the message of upstream's `refuseFill`, `RefuseFill` here), before the action and again on the binding found by a fresh snapshot taken just before typing (upstream's `relocated`; `DeviceSession.RelocatedAsync`, with the transition wait split out as `WaitForTransitionAsync`, upstream's `waitForTransition`). Tests: 'Acts_on_an_unclassified_iOS_field', 'Rejects_an_unclassified_iOS_element', and 'Rejects_an_unclassified_iOS_element_that_gained_a_control_before_typing' in `Engine/InputFreshnessTests`. Differences: a fill or clear now reads the field again, so it costs one more `capture.snapshot` than before. Upstream's `relocated` `input` flag is ported with #751. +- Upstream's screen swipe transition wait (`fix(mobile): wait for transitions before screen swipes`, `f45d9f8`, tester-army/e2e#1564) is ported. A test's screen swipe (`OperationContext.Test`) waits out the remaining `TransitionMilliseconds` budget before it scrolls, through `DeviceSession.WaitOutTransitionAsync` (upstream's `waitOutTransition`), which is capped by the input's own budget and throws `OPERATION_TIMEOUT` rather than send a scroll the transition cannot finish for. `WaitForTransitionAsync` and `SettledAsync` take that budget too, so an arriving control's wait is capped the same way. Tests: the screen-swipe `it`s in `Engine/DeterministicActionsTests` (the tight-budget ones observe no screen root, since the first look of a session is real-time and can outlast a sub-frame budget; they use the screen root's ref directly, which the swipe needs), and 'gives a screen swipe the remaining transition budget after an alert is dismissed' in `Engine/DeviceFixtureTests`. One upstream `it` is partial: 'does not send a 20 ms or 20.5 ms capped screen swipe when its timer wakes before the wall-clock deadline' depends on a `Date.now` mock that lags the timer, which the port's fake clock has no equivalent of, so the port's `Partial_Does_not_send_a_capped_screen_swipe_the_transition_outlasts` checks only that the capped swipe is not sent. - Upstream's mobile input revalidation (`fix(mobile): revalidate input bindings before fill and clear`, `b309bcd`, tester-army/e2e#1410) and keyboard transitions (`fix(mobile): reobserve targets after keyboard transitions`, `bdd7315`, tester-army/e2e#1297) are ported. `DeviceSession.RelocatedAsync` takes `forInput` (upstream's `input` flag of `relocated`; `input` is the `InputBudget` here). With it, a truncated or sparse capture, a node the capture no longer lists, or a changed `InputPurpose`, `Disabled`, `Secure`, or `Hidden` is a retryable `NODE_STALE` before anything is typed, and `DeviceActions.PerformAsync` sets it for `fill` and `clear`. `ProjectedSnapshot.KeyboardVisible` (set in `Project` from `KeyboardShowing`) and `NodeBinding.KeyboardVisible` stand for upstream's `ProjectedScreen` type (`ProjectedSnapshot & { keyboardVisible }`), which C# has no intersection type for. `LatestScreen` and `MarkAction(ProjectedSnapshot?)` replace `LatestIndex` and the index-based `MarkAction` (upstream's `latestScreen` and `markAction(before: ProjectedScreen)`), and `WaitForTransitionAsync` acts at once on an unmoved control only when the keyboard state of the screen before the action equals the control's. `DeviceErrors.StaleOr` also maps `target_still_moving` with `details.dispatched: no` to a retryable `NODE_STALE`. The `agent-input-stale.test.ts` file of #1410 is already ported in part (`AgentInputStale/SecretFillStaleRetryTests`); its replay half still needs a project runner, `credentials`, and the `secret.fill` policy event, which the port lacks. Tests: the `input freshness` `it`s in `Engine/InputFreshnessTests`, the three keyboard `it`s in `Engine/DeterministicActionsTests`, and 'retries a moving target only when no input was dispatched' in `Errors/ErrorTranslationTests`. - Upstream's WebDriver element-type fix (`fix(mobile): read XCUIElementType class names as their XCTest types`, `3f9a7d8`, tester-army/e2e#1133) is ported. `SnapshotProjection.NormalizeKind` is upstream's `normalizeKind` in `nodes.ts`: it drops `^XCUIElementType(?=[A-Z])` (`XcuiElementTypePrefix`, a .NET `[GeneratedRegex]` for upstream's inline regex) before it maps the type, so the `XCUIElementTypeButton` that agent-device's WebDriver runtime reports on hosted Appium clouds reads as `Button`. `DeviceSession.KeyboardShowing` is upstream's `keyboardShowing` in `surface.ts`: it routes node types through `NormalizeKind`, so an `XCUIElementTypeKeyboard` counts as a shown keyboard. Upstream records this only in its changeset; its skill references and docs say nothing about it, so `SKILL.md` and `docs/` have nothing to mirror. Tests: in `Nodes/SnapshotProjectionTests`, 'reads the XCUIElementType class names a WebDriver session reports as the native XCTest types'; in `Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests`, 'reads an XCUIElementTypeKeyboard from a WebDriver session as a shown keyboard'. - Upstream's held-device skip (`mobile: skip booted devices another agent-device session holds`, `9e53c40`, tester-army/e2e#1379) is ported. `BootedDevices` (`DevicePool.cs`, upstream's class in `pool.ts`) skips a booted device whose inventory entry has a `claimedBy.session` that is not one of this target's own slot sessions, logs the skipped ones, and fails `prepare` with a non-retryable `ENGINE_FAILURE` when every booted device is held. `BootedDevice` is upstream's interface of that name, as a record. `LifecycleTests.Skips_a_booted_device_another_agent_device_session_holds_keeps_one_its_own_slot_session_holds_and_fails_when_every_booted_one_is_held` ports the `lifecycle` `it` of `engine.test.ts`, whole. diff --git a/docs-site/src/content/docs/get-started/mobile.mdx b/docs-site/src/content/docs/get-started/mobile.mdx index a3051b24..46635475 100644 --- a/docs-site/src/content/docs/get-started/mobile.mdx +++ b/docs-site/src/content/docs/get-started/mobile.mdx @@ -100,9 +100,9 @@ await Expect.That(Device.Locator("role=navigation-bar id=About")).ToBeVisibleAsy await Device.Locator("id=SW_VERSION_SPECIFIER role=button").TapAsync(); ``` -Every action on a node pins its ref to the capture that issued it, so a ref a newer capture replaced is never sent to a different element. If the control changed during the `TransitionMilliseconds` wait (500 ms by default), the action fails with a retryable `NODE_STALE` and a locator resolves it again within its timeout. A direct `Device` call such as `DragToAsync` does not retry, so it surfaces the `NODE_STALE`. Unmoved controls are acted on immediately only when keyboard visibility is unchanged: opening or closing the keyboard also uses the remaining `TransitionMilliseconds` budget. A `target_still_moving` refusal from agent-device that reports `dispatched: no` is a retryable `NODE_STALE` as well. +Every action on a node pins its ref to the capture that issued it, so a ref a newer capture replaced is never sent to a different element. If the control changed during the `TransitionMilliseconds` wait (500 ms by default), the action fails with a retryable `NODE_STALE` and a locator resolves it again within its timeout. When the action's own timeout runs out before the transition does, the action fails with a non-retryable `OPERATION_TIMEOUT` and sends nothing. A direct `Device` call such as `DragToAsync` does not retry, so it surfaces the `NODE_STALE`. Unmoved controls are acted on immediately only when keyboard visibility is unchanged: opening or closing the keyboard also uses the remaining `TransitionMilliseconds` budget. A `target_still_moving` refusal from agent-device that reports `dispatched: no` is a retryable `NODE_STALE` as well. -A step of your test does not wait for the screen to settle after it acts: it checks its outcome with `Expect`, and it only waits out `TransitionMilliseconds` for a control that came with the last action. The agent's actions settle first, because it reads the screen right after acting. `Device.BackAsync` settles too, as `MobileOptions.Settle` says. The engine tells the two apart by the `OperationContext` of the call (`OperationContext.Test` or `OperationContext.Agent`). `DeviceSession.Origin` and `ActionOrigin` are obsolete and go in the next release: pass an `OperationContext` to the `PerformAsync`, `PerformAtAsync`, `SwipeAsync`, or `BackAsync` overload instead. +A step of your test does not wait for the screen to settle after it acts: it checks its outcome with `Expect`, and it only waits out `TransitionMilliseconds` for a control that came with the last action. A screen swipe in a test waits out the remaining `TransitionMilliseconds` budget before it scrolls. The agent's actions settle first, because it reads the screen right after acting. `Device.BackAsync` settles too, as `MobileOptions.Settle` says. The engine tells the two apart by the `OperationContext` of the call (`OperationContext.Test` or `OperationContext.Agent`). `DeviceSession.Origin` and `ActionOrigin` are obsolete and go in the next release: pass an `OperationContext` to the `PerformAsync`, `PerformAtAsync`, `SwipeAsync`, or `BackAsync` overload instead. `FillAsync` and `ClearAsync` read the field again before typing. A field the fresh capture no longer lists, a changed input purpose or secure, disabled, or hidden state, or an incomplete capture fails with a retryable `NODE_STALE` before anything is typed, and the locator resolves it again. A node that cannot take text fails with `UNSUPPORTED_CAPABILITY`. diff --git a/skills/e2e-dotnet/SKILL.md b/skills/e2e-dotnet/SKILL.md index b6ad8910..fcf9e6cf 100644 --- a/skills/e2e-dotnet/SKILL.md +++ b/skills/e2e-dotnet/SKILL.md @@ -36,7 +36,7 @@ trip. The installed tool prints the same text with `e2e guide ` | Handling a failed run: the trace page, `report.json` for scripts, `E2ESession.Report()` for a session you run yourself, error codes and their fixes, a visible browser, replay misses, phase timings, the agent step table, every model call | `debugging` | [references/debugging.md](references/debugging.md) | | Exploring an app toward a goal without a test file, with `e2e explore` or `Explore.ExploreAsync` | `explore` | [references/explore.md](references/explore.md) | | Running isolated per-test sessions from your own runner: `E2EHost.OpenAsync`, `OpenHostOptions` (config value or file, target, `EngineFactory`), `host.OpenSessionAsync`, `E2ESession.CloseAsync` and the failures it returns, `Steps()` | `host` | [references/host.md](references/host.md) | -| Testing an iOS or Android app: `MobileEngine`, the `Device` fixture, picking a simulator (without `MobileOptions.Device`, `MobileEngine` skips booted devices another agent-device session holds, and `prepare` fails with `ENGINE_FAILURE` when every booted one is held), video of the device screen, `Device.SetPermissionAsync` (the enums, or wire-name strings such as `"camera"` and `"grant"`; an unknown name or state is `INVALID_ARGUMENT` before any device command), `Device.UnstableRaw` (the test's agent-device session: name, device id and name, selection, and a raw `IDeviceClient` whose calls are not steps, replayed, or redacted, and that throws `NotSupportedException` for the `text-size` and `animations` settings, `apps.uninstall`, and `command.hardwareButton`; unstable; `INVALID_STATE` after the session is disposed), a NativeScript `testID` or `accessibilityIdentifier` as the test id `GetByTestId` reads (an Android `accessibilityLabel` is put before the text), Limrun and BrowserStack hosted devices, a MAUI app through the DevFlow agent, known limits, `Locator.PressAsync("Enter")` tapping an unfocused field first, and the agent's `open_app` returning to the app under test when it names no app, a Compose Multiplatform field on iOS (for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out, and the name is the label before `, `; a non-empty `ContentDescription` stays the name, and that label is then the `Text`; any other field keeps its label and children), and `FillAsync` and `ClearAsync` refusing a node that is not an editable field with `UNSUPPORTED_CAPABILITY` (an iOS `other` that holds a value or only text is typed into), reading the field again before typing (a field the new capture no longer lists, an incomplete capture, or a changed input purpose or secure, disabled, or hidden state is a retryable `NODE_STALE` before anything is typed), looking at the screen again after the keyboard opens or closes (an unmoved control is acted on at once only when keyboard visibility is unchanged, else it waits out `TransitionMilliseconds`), and a `target_still_moving` refusal with `dispatched: no` being a retryable `NODE_STALE` | `mobile` | [references/mobile.md](references/mobile.md) | +| Testing an iOS or Android app: `MobileEngine`, the `Device` fixture, picking a simulator (without `MobileOptions.Device`, `MobileEngine` skips booted devices another agent-device session holds, and `prepare` fails with `ENGINE_FAILURE` when every booted one is held), video of the device screen, `Device.SetPermissionAsync` (the enums, or wire-name strings such as `"camera"` and `"grant"`; an unknown name or state is `INVALID_ARGUMENT` before any device command), `Device.UnstableRaw` (the test's agent-device session: name, device id and name, selection, and a raw `IDeviceClient` whose calls are not steps, replayed, or redacted, and that throws `NotSupportedException` for the `text-size` and `animations` settings, `apps.uninstall`, and `command.hardwareButton`; unstable; `INVALID_STATE` after the session is disposed), a NativeScript `testID` or `accessibilityIdentifier` as the test id `GetByTestId` reads (an Android `accessibilityLabel` is put before the text), Limrun and BrowserStack hosted devices, a MAUI app through the DevFlow agent, known limits, `Locator.PressAsync("Enter")` tapping an unfocused field first, and the agent's `open_app` returning to the app under test when it names no app, a Compose Multiplatform field on iOS (for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out, and the name is the label before `, `; a non-empty `ContentDescription` stays the name, and that label is then the `Text`; any other field keeps its label and children), and `FillAsync` and `ClearAsync` refusing a node that is not an editable field with `UNSUPPORTED_CAPABILITY` (an iOS `other` that holds a value or only text is typed into), reading the field again before typing (a field the new capture no longer lists, an incomplete capture, or a changed input purpose or secure, disabled, or hidden state is a retryable `NODE_STALE` before anything is typed), looking at the screen again after the keyboard opens or closes (an unmoved control is acted on at once only when keyboard visibility is unchanged, else it waits out `TransitionMilliseconds`; a test's screen swipe waits out the remaining budget before it scrolls), and a `target_still_moving` refusal with `dispatched: no` being a retryable `NODE_STALE` | `mobile` | [references/mobile.md](references/mobile.md) | | Testing a Windows desktop app: `WinAppEngine`, `WinAppTest`, the `Window` fixture, the `windows` target in `e2e explore` and `e2e mcp`, the interactive-desktop requirement, known limits | `windows` | [references/windows.md](references/windows.md) | | Asked to bug bash or hunt for bugs: the bug-bash config, many `e2e explore` runs at once, proving each bug with a repro test | `bug-bash` | [references/bug-bash.md](references/bug-bash.md) | diff --git a/skills/e2e-dotnet/references/mobile.md b/skills/e2e-dotnet/references/mobile.md index faae92f6..88288c30 100644 --- a/skills/e2e-dotnet/references/mobile.md +++ b/skills/e2e-dotnet/references/mobile.md @@ -261,6 +261,6 @@ Limits of the preview: the tests attach to the running app, so `Device.InstallAp - Agent steps on a device are slower than in a browser. Raise `StepTimeout` to 90 seconds. The 30 second default timed out a two-field step. - A ref belongs to the agent-device capture it came from, and every action pins it there (`@e4~s7`), so agent-device refuses a ref once a newer capture supersedes it. An action on a node that a newer capture replaced never acts on a different element: a test's action waits out `TransitionMilliseconds` for a control that arrived with the last action, and a node the fresh capture no longer lists is a retryable `NODE_STALE`, so the locator resolves again within its timeout. A direct `Device` call (`DragToAsync`, `LongPressAsync`) does not retry and surfaces it. Unmoved controls are acted on immediately only when keyboard visibility is unchanged: opening or closing the keyboard also uses the remaining `TransitionMilliseconds` budget. A `target_still_moving` refusal that reports `dispatched: no` is a retryable `NODE_STALE`. `FillAsync` and `ClearAsync` read the field again before typing: a field the fresh capture no longer lists, a changed input purpose or secure, disabled, or hidden state, or an incomplete capture is a retryable `NODE_STALE` before anything is typed; a node that cannot take text is `UNSUPPORTED_CAPABILITY`. `Device.DragToAsync` between nodes located in separate captures takes both refs from the newest one. - An input that outlives its timeout. A tap, fill, key, swipe, or `Back` gets the `actionTimeout` (30 seconds by default). If the device doesn't answer in time, the step fails with `ACTION_MAY_HAVE_COMMITTED` and isn't retried: the input may have landed. An input the timeout ran out before (its target was still settling) isn't sent and fails with `OPERATION_TIMEOUT`. A long press that holds at least as long as the time left is refused before it's sent, as `NOT_ACTIONABLE`: raise the `actionTimeout`. A locate (the snapshot a locator or `Expect` takes), an observation, and a screenshot run within the timeout of the call that takes them, so a device that stops answering fails the step with `ACTION_FAILED` (the engine's `OPERATION_TIMEOUT` as its `InnerException`) instead of holding it until the test timeout. `WaitForAsync`, `Expect`, and `ToHaveScreenshotAsync` give their last read extra time, up to their slowest read so far and within the test timeout, so a screen that shows up during it still passes. An element that never shows up still fails as a locator timeout. A command a timeout or a cancel abandoned keeps running on the device, so the next attempt on that device waits for it to settle before it starts (within the launch timeout, which fails the start as cancelled when it runs out). -- A step of your test does not wait for the screen to settle after it acts: it checks its outcome with `Expect`, and it only waits out `TransitionMilliseconds` for a control that came with the last action. The agent's actions settle first, because it reads the screen right after acting. `Device.BackAsync` settles too, as `MobileOptions.Settle` says. The engine tells the two apart by the `OperationContext` of the call (`OperationContext.Test` or `OperationContext.Agent`). `DeviceSession.Origin` and `ActionOrigin` are obsolete and go in the next release: pass an `OperationContext` to the `PerformAsync`, `PerformAtAsync`, `SwipeAsync`, or `BackAsync` overload instead. +- A step of your test does not wait for the screen to settle after it acts: it checks its outcome with `Expect`, and it only waits out `TransitionMilliseconds` for a control that came with the last action. A screen swipe in a test waits out the remaining `TransitionMilliseconds` budget before it scrolls. The agent's actions settle first, because it reads the screen right after acting. `Device.BackAsync` settles too, as `MobileOptions.Settle` says. The engine tells the two apart by the `OperationContext` of the call (`OperationContext.Test` or `OperationContext.Agent`). `DeviceSession.Origin` and `ActionOrigin` are obsolete and go in the next release: pass an `OperationContext` to the `PerformAsync`, `PerformAtAsync`, `SwipeAsync`, or `BackAsync` overload instead. - Device tests need a device. Keep them out of CI, for example with the `RealModel` category (`dotnet test --filter "TestCategory!=RealModel"`). - Android can report `Android snapshot helper returned insufficient foreground app content`, and iOS can report that the app is not running: agent-device could not read the screen yet, often an app still loading after a cold start. On iOS this only counts until the first readable screen after `OpenAppAsync`, `RestartAsync`, or `OpenLinkAsync`, as on a physical iPhone that is still launching the app; an app that exits later fails at once. A waiting locator action or `Expect` waits through it, as for an element not there yet. If the screen is still unreadable when the wait runs out, the step fails with this message. Immediate reads such as `CountAsync` and `IsVisibleAsync` see no matches instead, so `Not.ToBeVisibleAsync` can pass while the screen is unreadable, including on an app that crashed before it rendered. An observation, such as the agent's next turn, fails on it at once. diff --git a/src/E2E.Mobile/DeviceActions.cs b/src/E2E.Mobile/DeviceActions.cs index f7341454..69c810a5 100644 --- a/src/E2E.Mobile/DeviceActions.cs +++ b/src/E2E.Mobile/DeviceActions.cs @@ -98,7 +98,7 @@ await session.ActAsync( // The field is read again before it is typed into, and what it reads as may have changed. if (context.Origin == OperationOrigin.Test) { - await session.WaitForTransitionAsync(entry, token).ConfigureAwait(false); + await session.WaitForTransitionAsync(entry, input, token).ConfigureAwait(false); } target = await session.RelocatedAsync(entry, input, token, forInput: true).ConfigureAwait(false); @@ -243,6 +243,12 @@ private static async Task SwipeNodeAsync(IDeviceClient client, NodeBinding targe private static async Task ScrollScreenAsync(DeviceSession session, ScrollDirection direction, OperationContext context, InputBudget? input, CancellationToken cancellationToken) { + // A test's swipe of the screen waits out the transition the last action started, as a test's tap waits for its control. + if (context.Origin == OperationOrigin.Test) + { + await session.WaitOutTransitionAsync(input, cancellationToken).ConfigureAwait(false); + } + var before = session.LatestScreen; await session.CommandAsync( "swipe", diff --git a/src/E2E.Mobile/DeviceSession.Screen.cs b/src/E2E.Mobile/DeviceSession.Screen.cs index 211cb989..94dbe1cf 100644 --- a/src/E2E.Mobile/DeviceSession.Screen.cs +++ b/src/E2E.Mobile/DeviceSession.Screen.cs @@ -349,18 +349,20 @@ a is not null && b is not null /// again in a fresh snapshot. The budget is the only signal that the /// transition is over, since the tree cannot tell a sliding control from a landed one. The fresh snapshot is where /// the control landed: Android reports a window's frames while it slides in, so a ref resolved mid-flight would - /// send the action where the control was. + /// send the action where the control was. When the input's budget cannot cover the rest of the transition, it throws + /// OPERATION_TIMEOUT and sends nothing. /// internal async Task SettledAsync(NodeBinding entry, InputBudget input, CancellationToken cancellationToken) => - await WaitForTransitionAsync(entry, cancellationToken).ConfigureAwait(false) + await WaitForTransitionAsync(entry, input, cancellationToken).ConfigureAwait(false) ? await RelocatedAsync(entry, input, cancellationToken).ConfigureAwait(false) : entry; /// /// Waits out the remainder of the transition budget when the control came with the last action. True when it waited, - /// so the binding may be stale; false when the control was already in place. + /// so the binding may be stale; false when the control was already in place. It throws OPERATION_TIMEOUT when the + /// input's budget cannot cover the rest of the transition, and sends nothing in that case. /// - internal async Task WaitForTransitionAsync(NodeBinding entry, CancellationToken cancellationToken) + internal async Task WaitForTransitionAsync(NodeBinding entry, InputBudget? input, CancellationToken cancellationToken) { var since = _engine.Clock.GetUtcNow() - _lastActionAt; var remaining = TimeSpan.FromMilliseconds(Options.TransitionMilliseconds) - since; @@ -374,23 +376,54 @@ internal async Task WaitForTransitionAsync(NodeBinding entry, Cancellation return false; } - try + await WaitOutTransitionAsync(input, cancellationToken).ConfigureAwait(false); + return true; + } + + /// + /// Waits out the remainder of the transition budget without outlasting the input's own budget. When the input's budget + /// cannot cover the rest of the transition, it throws OPERATION_TIMEOUT rather than sending an input the + /// transition has not finished for. Upstream's waitOutTransition. + /// + internal async Task WaitOutTransitionAsync(InputBudget? input, CancellationToken cancellationToken) + { + // The wait resolves on cancellation, and the clock can move before a cancelled wait is noticed. Nothing waits and + // nothing goes out once the caller has already been told the operation was cancelled. + if (cancellationToken.IsCancellationRequested) { - await _engine.Delay(remaining, cancellationToken).ConfigureAwait(false); + throw DeviceSupport.Cancelled("transition wait cancelled"); } - catch (OperationCanceledException) + + var since = _engine.Clock.GetUtcNow() - _lastActionAt; + var remaining = TimeSpan.FromMilliseconds(Options.TransitionMilliseconds) - since; + if (remaining <= TimeSpan.Zero) { - throw DeviceSupport.Cancelled("transition wait cancelled"); + return; + } + + var available = input?.Remaining ?? remaining; + if (available > TimeSpan.Zero) + { + try + { + await _engine.Delay(available < remaining ? available : remaining, cancellationToken).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + throw DeviceSupport.Cancelled("transition wait cancelled"); + } } - // The wait resolves on cancellation. The action behind it must not go out once the caller has already been told - // the operation was cancelled. if (cancellationToken.IsCancellationRequested) { throw DeviceSupport.Cancelled("transition wait cancelled"); } - return true; + // A wait the budget capped leaves the transition unfinished, so the input it was for is not sent. + if (available < remaining) + { + throw new EngineException(EngineErrorCodes.OperationTimeout, "transition could not finish within the action budget", retryable: false); + } } /// diff --git a/src/E2E.Mobile/MobileOptions.cs b/src/E2E.Mobile/MobileOptions.cs index 31421f9c..d78c0ded 100644 --- a/src/E2E.Mobile/MobileOptions.cs +++ b/src/E2E.Mobile/MobileOptions.cs @@ -152,7 +152,7 @@ public sealed record MobileOptions /// acts on it, in milliseconds. Default 500, the length of a modal or screen transition. If the control changes during the /// wait, the action fails with a retryable NODE_STALE and a locator resolves it again. Unmoved controls are acted /// on immediately only when keyboard visibility is unchanged. Opening or closing the keyboard also uses the remaining - /// budget. Raise it for slower transitions. + /// budget. A test's screen swipe waits out the remaining budget before it scrolls. Raise it for slower transitions. /// public int TransitionMilliseconds { get; init; } = DefaultTransitionMilliseconds; diff --git a/tests/E2E.Mobile.Tests/Engine/DeterministicActionsTests.cs b/tests/E2E.Mobile.Tests/Engine/DeterministicActionsTests.cs index 363f6954..797f2369 100644 --- a/tests/E2E.Mobile.Tests/Engine/DeterministicActionsTests.cs +++ b/tests/E2E.Mobile.Tests/Engine/DeterministicActionsTests.cs @@ -373,4 +373,162 @@ public async Task Does_not_count_a_toggle_already_in_the_wanted_state_as_an_acti h.Fake.Respond(DeviceMethods.CaptureSnapshot, _ => FakeDeviceClient.SettingsSnapshotWith(Submit("@e11", 600, "Save"))); await TapsAtOnce(h, session, await Observed(session, "Save"), """{"ref":"@e11"}"""); } + + private static LocatorAction Swipe(ScrollDirection direction = ScrollDirection.Down) => new LocatorAction.Swipe(direction); + + /// The screen root as a swipe names it, without a look: a swipe of the screen needs only its ref. + private static SemanticNode ScreenRoot() => new() { Ref = SnapshotProjection.RootId }; + + private static int Scrolls(Harness h) => h.Fake.Calls.Count(call => call.Method == "interactions.scroll"); + + [Fact] + public async Task Keeps_agent_origin_screen_swipes_immediate_with_driver_settling() + { + var h = WithTransition(120); + var session = await OpenAttempt(h); + var screen = await RootOf(session); + var startedAt = h.Clock.GetUtcNow(); + + await session.PerformAsync(screen, Swipe(), OperationContext.Agent, CancellationToken.None); + + Assert.Equal(startedAt, h.Clock.GetUtcNow()); + JsonAssert.Args("""{"direction":"down","settle":true,"settleQuietMs":150}""", h.Fake.LastArgs("interactions.scroll")); + } + + [Fact] + public async Task Counts_captures_between_consecutive_screen_swipes_toward_the_transition_budget() + { + var h = WithTransition(120); + var session = await OpenAttempt(h); + h.Clock.Advance(TimeSpan.FromMilliseconds(150)); + var screen = await RootOf(session); + await session.PerformAsync(screen, Swipe(), OperationContext.Test, CancellationToken.None); + var firstSwipeAt = h.Clock.GetUtcNow(); + h.Fake.Respond(DeviceMethods.CaptureSnapshot, _ => + { + h.Clock.Advance(TimeSpan.FromMilliseconds(50)); + return FakeDeviceClient.SettingsSnapshot(); + }); + var nextScreen = await RootOf(session); + + await session.PerformAsync(nextScreen, Swipe(), OperationContext.Test, CancellationToken.None); + + Assert.Equal(TimeSpan.FromMilliseconds(120), h.Clock.GetUtcNow() - firstSwipeAt); + Assert.Equal(2, Scrolls(h)); + } + + [Fact] + public async Task Does_not_dispatch_a_screen_swipe_whose_transition_wait_was_cancelled() + { + var h = WithTransition(300); + h.Waits.Wait = (_, token) => Task.Delay(Timeout.Infinite, token); + var session = await OpenAttempt(h); + var screen = await RootOf(session); + using var controller = new CancellationTokenSource(); + controller.CancelAfter(TimeSpan.FromMilliseconds(20)); + + var cancelled = await Assert.ThrowsAsync(() => session.PerformAsync(screen, Swipe(), OperationContext.Test, controller.Token)); + + Assert.Equal("CANCELLED", cancelled.Code); + Assert.DoesNotContain("interactions.scroll", h.Fake.Methods()); + } + + [Fact] + public async Task Rejects_an_already_cancelled_screen_swipe_without_starting_its_transition_wait() + { + var h = WithTransition(300); + var session = await OpenAttempt(h); + var screen = await RootOf(session); + using var controller = new CancellationTokenSource(); + controller.Cancel(); + var startedAt = h.Clock.GetUtcNow(); + + var cancelled = await Assert.ThrowsAsync(() => session.PerformAsync(screen, Swipe(), OperationContext.Test, controller.Token)); + + Assert.Equal("CANCELLED", cancelled.Code); + Assert.Equal(startedAt, h.Clock.GetUtcNow()); + Assert.DoesNotContain("interactions.scroll", h.Fake.Methods()); + } + + [Fact] + public async Task Ends_a_screen_swipe_transition_at_its_action_budget_without_sending_a_gesture() + { + var h = WithTransition(160); + var session = await OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(20)); + var screen = ScreenRoot(); + var startedAt = h.Clock.GetUtcNow(); + + var timeout = await Assert.ThrowsAsync(() => session.PerformAsync(screen, Swipe(), OperationContext.Test, CancellationToken.None)); + + Assert.Equal("OPERATION_TIMEOUT", timeout.Code); + Assert.Equal(TimeSpan.FromMilliseconds(20), h.Clock.GetUtcNow() - startedAt); + Assert.DoesNotContain("interactions.scroll", h.Fake.Methods()); + } + + // Partial: upstream's `it` also has the timer wake a sub-millisecond early against a lagging Date.now. The fake clock + // cannot lag its own delays, so this checks only that a capped swipe the transition outlasts is not sent. + [Theory] + [InlineData(20)] + [InlineData(20.5)] + public async Task Partial_Does_not_send_a_capped_screen_swipe_the_transition_outlasts(double timeoutMs) + { + var h = WithTransition(160); + var session = await OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(timeoutMs)); + + // The screen root needs no look: a look is a read the same sub-frame budget bounds, and its first call can outlast it. + var screen = ScreenRoot(); + var startedAt = h.Clock.GetUtcNow(); + + var timeout = await Assert.ThrowsAsync(() => session.PerformAsync(screen, Swipe(), OperationContext.Test, CancellationToken.None)); + + Assert.Equal("OPERATION_TIMEOUT", timeout.Code); + Assert.False(timeout.Retryable); + Assert.True(h.Clock.GetUtcNow() - startedAt <= TimeSpan.FromMilliseconds(timeoutMs)); + Assert.DoesNotContain("interactions.scroll", h.Fake.Methods()); + } + + [Fact] + public async Task Sends_a_screen_swipe_when_its_action_budget_can_accommodate_the_full_transition() + { + var h = WithTransition(160); + var session = await OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(200)); + var screen = await RootOf(session); + var startedAt = h.Clock.GetUtcNow(); + + await session.PerformAsync(screen, Swipe(), OperationContext.Test, CancellationToken.None); + + Assert.Equal(TimeSpan.FromMilliseconds(160), h.Clock.GetUtcNow() - startedAt); + Assert.Equal(1, Scrolls(h)); + } + + [Fact] + public async Task Reports_cancellation_when_the_caller_aborts_a_capped_transition_wait() + { + var h = WithTransition(160); + h.Waits.Wait = (_, token) => Task.Delay(Timeout.Infinite, token); + var session = await OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(20)); + var screen = ScreenRoot(); + using var controller = new CancellationTokenSource(); + controller.CancelAfter(TimeSpan.FromMilliseconds(10)); + + var cancelled = await Assert.ThrowsAsync(() => session.PerformAsync(screen, Swipe(), OperationContext.Test, controller.Token)); + + Assert.Equal("CANCELLED", cancelled.Code); + Assert.DoesNotContain("interactions.scroll", h.Fake.Methods()); + } + + [Fact] + public async Task Ends_an_arriving_control_transition_at_its_action_budget_without_tapping() + { + var h = WithTransition(160); + var session = await OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(20)); + var about = await Observed(session, "About"); + var startedAt = h.Clock.GetUtcNow(); + + var timeout = await Assert.ThrowsAsync(() => session.PerformAsync(about, new LocatorAction.Tap(), OperationContext.Test, CancellationToken.None)); + + Assert.Equal("OPERATION_TIMEOUT", timeout.Code); + Assert.Equal(TimeSpan.FromMilliseconds(20), h.Clock.GetUtcNow() - startedAt); + Assert.DoesNotContain("interactions.press", h.Fake.Methods()); + } } diff --git a/tests/E2E.Mobile.Tests/Engine/DeviceFixtureTests.cs b/tests/E2E.Mobile.Tests/Engine/DeviceFixtureTests.cs index 9087e4a1..81471ea4 100644 --- a/tests/E2E.Mobile.Tests/Engine/DeviceFixtureTests.cs +++ b/tests/E2E.Mobile.Tests/Engine/DeviceFixtureTests.cs @@ -45,6 +45,24 @@ public async Task Mints_a_locator_from_an_agent_device_selector_on_the_attempts_ JsonAssert.Args("""{"ref":"@e4","settle":true,"settleQuietMs":150}""", h.Fake.LastArgs("interactions.press")); } + [Fact] + public async Task Gives_a_screen_swipe_the_remaining_transition_budget_after_an_alert_is_dismissed() + { + var h = Harness.Create(options => options with { TransitionMilliseconds = 120 }); + var session = await OpenAttempt(h); + h.Clock.Advance(TimeSpan.FromMilliseconds(150)); + await RootOf(session); + await session.Device.AlertAsync(AlertAction.Accept); + var dismissedAt = h.Clock.GetUtcNow(); + var screen = await RootOf(session); + h.Clock.Advance(TimeSpan.FromMilliseconds(30)); + + await session.PerformAsync(screen, new LocatorAction.Swipe(ScrollDirection.Down), OperationContext.Test, CancellationToken.None); + + Assert.Equal(TimeSpan.FromMilliseconds(120), h.Clock.GetUtcNow() - dismissedAt); + JsonAssert.Args("""{"direction":"down"}""", h.Fake.LastArgs("interactions.scroll")); + } + [Fact] public async Task Counts_a_back_home_alert_keyboard_rotation_or_fold_as_an_action_so_a_control_that_arrives_with_it_waits_out_the_transition_budget() { diff --git a/tests/E2E.Mobile.Tests/Engine/ObservationTests.cs b/tests/E2E.Mobile.Tests/Engine/ObservationTests.cs index 8d154cb4..f0f8c909 100644 --- a/tests/E2E.Mobile.Tests/Engine/ObservationTests.cs +++ b/tests/E2E.Mobile.Tests/Engine/ObservationTests.cs @@ -380,6 +380,8 @@ public async Task Ends_an_input_hung_on_its_device_command_at_its_budget_as_poss // observation when a loaded runner stretches it. h.Clock.ManualTimers = true; var session = await Screens.OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(200)); + // The launch's transition is over before the input: a budget shorter than the transition is refused, so the input is only sent once it is not in flight. + h.Clock.Advance(TimeSpan.FromMilliseconds(600)); var root = await Screens.RootOf(session); var about = Named(root, "About"); var sent = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); @@ -415,6 +417,9 @@ public async Task Sends_nothing_once_a_settling_target_spent_the_budget_and_no_k var session = await Screens.OpenAttempt(h, actionTimeout: TimeSpan.FromMilliseconds(100)); var about = await Screens.Observed(session, "About"); + // The launch's transition is over before the first tap: a budget shorter than the transition is refused, so the tap goes out once it is not in flight. + h.Clock.Advance(TimeSpan.FromMilliseconds(150)); + // About moves with the tap, so the next tap relocates it first; a slow relocation spends the whole budget. await session.PerformAsync(about, new LocatorAction.Tap(), OperationContext.Test, CancellationToken.None); h.Fake.Respond(DeviceMethods.CaptureSnapshot, _ => FakeDeviceClient.SnapshotOf(