Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ The port installs the skill with `npx skills add hardkoded/e2e-dotnet` and regis
- `DeviceSession.BackAsync` always settles (upstream's agent origin), since a test step has no origin here.
- The core session has no attempt hooks, pixel observations, or artifact directory. So `DeviceSession` is one attempt (it registers on the engine until it closes), and `ScreenshotAsync` returns the masked, status-bar-normalized PNG for `toHaveScreenshot` instead of writing numbered files. A secure field that cannot be masked still withholds the image with `ENGINE_FAILURE`.
- `DeviceSession.LocateAsync(LocatorExpression, TimeSpan timeout, CancellationToken)` is upstream's `locate` (`surface.ts`), so a locator or `Expect` on a device resolves in the engine. It snapshots with `interactiveOnly: false`, runs `DeviceLocator.ResolveExpression`, and binds the matches in a separate map capped at 2048 (the oldest goes first; an older ref is `NODE_STALE`). It hands the capture it matched against to `EngineStartOptions.Screen`, and tells `EngineStartOptions.Environment` the platform and device once, when the session starts. Differences: `timeout` is the budget left (upstream's `operation.timeoutMs`), since the core session has no operation context; the expression is one flat record (`LocatorExpression`) where upstream's is a union of `query`, `filter`, `index`, `union`, `selector`, and `frame`; and the port's refs carry no revision, so upstream's `rejectSupersededLocate` has no port (a device mints a new id for every projection, so a later locate never reuses a ref). Core retries a retryable `EngineException` of a locate until the wait's deadline, as `resolveOnce` does.
- Upstream's operation budget for reads (`fix(mobile): bound reads by the operation budget`, `57f9ed9`, tester-army/e2e#1464) is ported for the snapshots a locate and a keyboard dismissal take. `InputBudget.ReadAsync` (upstream's `read`) runs a snapshot with what is left of the budget, and `InputBudget.Remaining` is `remainingMs`. A locate runs under a budget of its own `timeout` (`DeviceSession.InputBudget(label, timeout)`, upstream's `inputBudget(label, timeoutMs)`), so a device that stops answering fails the locate with `OPERATION_TIMEOUT`, which a locator reports as `ACTION_FAILED`, where it held the step until the test timeout. A sparse capture's retry the budget cannot fit returns the sparse capture, and a capture the budget abandoned is not taken again. A relocation before an action, and the snapshots of the keyboard dismissal, run under the input's budget. `device.dismissKeyboard` now has a budget too (`actionTimeout`), as upstream's has. The observation and the screenshots are bounded too (#892). `IEngineSession.ObserveAsync(TimeSpan timeout, CancellationToken)` and `ScreenshotAsync(TimeSpan timeout, CancellationToken)` carry upstream's `operation.timeoutMs` (a public API change, like `LocateAsync`'s `timeout`): they have a default that drops the budget and calls the overload without it, so an engine that bounds nothing keeps compiling (.NET-only). `DeviceSession` overrides them. `ObserveAsync` runs its snapshot and its viewport probe under one `InputBudget('observe')`, `ScreenshotAsync` its snapshot and its capture under one `InputBudget('screenshot')`, so a device that stops answering fails with `OPERATION_TIMEOUT` and the screenshot cannot take a fresh budget. The core callers pass what they have: `Screen.TreeAsync` its timeout, a locator resolve over an observation and `ToHaveScreenshotAsync` the budget of the read window. Calls with no timeout (the agent's looks, failure evidence, `ObserveAsync(CancellationToken)`) get `ActionTimeout`. `IDeviceActions.ScreenshotAsync(DeviceSession, TimeSpan, CancellationToken)` takes the timeout too; the overload without it stays, and the one with it defaults to it (.NET-only), so a caller-provided `MobileEngine.Actions` that defines only the old member keeps compiling. An infinite timeout (`Screen.TreeAsync` with none) gives an `InputBudget` that never expires and bounds no wait (.NET-only: upstream's budgets are numbers of milliseconds). `ScreenshotArtifactAsync` runs under `ActionTimeout`, where upstream's `artifacts.screenshot` takes `operation.timeoutMs`. A read the caller cancels is `CANCELLED` (`InputBudget.ReadAsync`), as the command reports it. The `InputBudget` parameter of `SnapshotAsync` and `SnapshotOrEmptyAsync` stays nullable, where upstream's is required. The port's observe takes no `pixels`, so upstream's `capturePixels` has no port: the screenshot an observation takes is the viewport probe, which rethrows `OPERATION_TIMEOUT` too. Tests, in `Engine/ObservationTests`: 'Partial_Bounds_screenshot_by_operation_budget' (the port's `ScreenshotAsync`, not `artifacts.screenshot`), 'Partial_Rethrows_OPERATION_TIMEOUT_from_capturePixels_during_observe' and 'Partial_Shares_one_budget_across_observe_steps_so_screenshot_cannot_take_a_fresh_budget' (the viewport probe, not the observation's pixels). Not ported: the positioned tap (`performAt`) `lateAction` check ('never sends a positioned tap whose target became visible past the deadline'), which needs `performWithin` (#861). Tests: in `Engine/ObservationTests`, 'bounds a snapshot by the operation budget', 'returns a sparse capture when the budget cannot fit a retry, instead of timing out', 'bounds the snapshots of a keyboard dismissal by its budget, and reports one cut off after the drag as possibly committed', 'lets a slow snapshot run for the whole timeout the locate gets', and 'takes no second snapshot for a capture the budget abandoned'; in `Support/SupportHelpersTests`, 'runs a read with what is left of its budget, and reports one cut off after an input as possibly committed'. These run on real timers of a few hundred milliseconds.
- Upstream's operation budget for reads (`fix(mobile): bound reads by the operation budget`, `57f9ed9`, tester-army/e2e#1464) is ported for the snapshots a locate and a keyboard dismissal take. `InputBudget.ReadAsync` (upstream's `read`) runs a snapshot with what is left of the budget, and `InputBudget.Remaining` is `remainingMs`. A locate runs under a budget of its own `timeout` (`DeviceSession.InputBudget(label, timeout)`, upstream's `inputBudget(label, timeoutMs)`), so a device that stops answering fails the locate with `OPERATION_TIMEOUT`, which a locator reports as `ACTION_FAILED`, where it held the step until the test timeout. A sparse capture's retry the budget cannot fit returns the sparse capture, and a capture the budget abandoned is not taken again. A relocation before an action, and the snapshots of the keyboard dismissal, run under the input's budget. `device.dismissKeyboard` now has a budget too (`actionTimeout`), as upstream's has. The observation and the screenshots are bounded too (#892). `IEngineSession.ObserveAsync(TimeSpan timeout, CancellationToken)` and `ScreenshotAsync(TimeSpan timeout, CancellationToken)` carry upstream's `operation.timeoutMs` (a public API change, like `LocateAsync`'s `timeout`): they have a default that drops the budget and calls the overload without it, so an engine that bounds nothing keeps compiling (.NET-only). `DeviceSession` overrides them. `ObserveAsync` runs its snapshot and its viewport probe under one `InputBudget('observe')`, `ScreenshotAsync` its snapshot and its capture under one `InputBudget('screenshot')`, so a device that stops answering fails with `OPERATION_TIMEOUT` and the screenshot cannot take a fresh budget. The core callers pass what they have: `Screen.TreeAsync` its timeout, a locator resolve over an observation and `ToHaveScreenshotAsync` the budget of the read window. Calls with no timeout (the agent's looks, failure evidence, `ObserveAsync(CancellationToken)`) get `ActionTimeout`. `IDeviceActions.ScreenshotAsync(DeviceSession, TimeSpan, CancellationToken)` takes the timeout too; the overload without it stays, and the one with it defaults to it (.NET-only), so a caller-provided `MobileEngine.Actions` that defines only the old member keeps compiling. An infinite timeout (`Screen.TreeAsync` with none) gives an `InputBudget` that never expires and bounds no wait (.NET-only: upstream's budgets are numbers of milliseconds). `ScreenshotArtifactAsync(label, TimeSpan timeout, ct)` is `artifacts.screenshot(label, operation)`; the overload without a timeout runs under `ActionTimeout`. `IEngineSession.ObserveAsync(ObserveOptions? options, TimeSpan timeout, ct)` is upstream's `observe(operation, { pixels, comparable })` (`ObserveOptions { Pixels, Comparable }` is `EngineObserveOptions`; a public API change). With `Pixels`, `Observation.Pixels` (an `EngineScreenshot`: upstream's `pixels` and `maskedRegionCount`) carries the screen's image, with every secure field covered and the status bar normalized when `Comparable`; the default drops the options and returns the tree alone (.NET-only), as an engine that takes none does. `DeviceSession.CapturePixelsAsync` is `capturePixels`: it takes the screenshot under the observation's budget, rethrows `OPERATION_TIMEOUT`, and costs the observation its image (not the step) for any other failure or a secure field it cannot prove covered. The agent still takes its screenshot through `Capture`, so `ObservationFeed` does not use `Observation.Pixels` yet. A read the caller cancels is `CANCELLED` (`InputBudget.ReadAsync`), as the command reports it. The `InputBudget` parameter of `SnapshotAsync` and `SnapshotOrEmptyAsync` stays nullable, where upstream's is required. Tests, in `Engine/ObservationTests`: 'bounds screenshot by operation budget', 'rethrows OPERATION_TIMEOUT from capturePixels during observe', 'shares one budget across observe steps so screenshot cannot take a fresh budget', 'captures pixels on request, masks every secure node, and degrades to tree-only when it cannot', and 'holds the status bar in a fixed state for a screenshot taken for comparison, and only then' and 'withholds the pixels of a screen with a secure field when the image cannot be read or decoded' (now run through `ObserveAsync` with `Pixels`). Not ported: the positioned tap (`performAt`) `lateAction` check ('never sends a positioned tap whose target became visible past the deadline'), which needs `performWithin` (#861). Tests: in `Engine/ObservationTests`, 'bounds a snapshot by the operation budget', 'returns a sparse capture when the budget cannot fit a retry, instead of timing out', 'bounds the snapshots of a keyboard dismissal by its budget, and reports one cut off after the drag as possibly committed', 'lets a slow snapshot run for the whole timeout the locate gets', and 'takes no second snapshot for a capture the budget abandoned'; in `Support/SupportHelpersTests`, 'runs a read with what is left of its budget, and reports one cut off after an input as possibly committed'. These run on real timers of a few hundred milliseconds.
- Upstream's Android locate (`fix(mobile): keep locator polling through unreadable android snapshots`, `4fe6baf`) is ported. A snapshot agent-device cannot read yet (`details.androidSnapshotHelperFailureReason` of `empty-helper-output`, `system-window-only`, or `content-poor-app-window`, the message `Android snapshot helper returned insufficient foreground app content`) matches nothing in a locate, so a locator or `Expect` keeps polling until its wait runs out, and a locate whose capture leaves no budget fails with `ENGINE_FAILURE` and that message. `DeviceErrors.IsUnreadableCapture` reads the reason off the `AgentDeviceError` that the translated `EngineException` keeps as its `InnerException`. An observation, such as the agent's next turn, still fails on it at once, as upstream's.
- Upstream's iOS locate (`2b2b845`, tester-army/e2e#1281) is ported. Just after `OpenAppAsync`, `RestartAsync` or `OpenLinkAsync`, a snapshot that iOS refuses with `details.runnerErrorCode` of `APP_NOT_RUNNING` (`DeviceErrors.IsAppNotRunning`) matches nothing in a locate (locator waits and `Expect` only; an observation still fails at once), like an unreadable Android capture, until a snapshot succeeds (`DeviceSession._launching`, upstream's `launching`). A wait that runs out while the app is still down fails with `ENGINE_FAILURE` and that reason. An app that exits after its first readable screen still fails at once.
- Upstream's Android test id match without the `<package>:id/` prefix (`feat(mobile): match Android test ids without their package prefix`, `2791786`) is in `DeviceLocator`, which `DeviceSession.LocateAsync` now calls, so `Screen.GetByTestId("greet")` matches `<package>:id/greet` on Android when nothing matches the full id.
Expand Down
2 changes: 1 addition & 1 deletion docs-site/public/api/E2E.Engine.IEngineSession.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html lang="en">
<head>
<meta charset="utf-8">
<link rel="canonical" href="https://hardkoded.github.io/e2e-dotnet/api/E2E.Engine/IEngineSession/">
<link rel="canonical" href="https://www.hardkoded.com/e2e-dotnet/api/E2E.Engine/IEngineSession/">
<meta http-equiv="refresh" content="0; url=/e2e-dotnet/api/E2E.Engine/IEngineSession/">
<title>Redirecting…</title>
<script>location.replace("/e2e-dotnet/api/E2E.Engine/IEngineSession/" + (location.hash || ""));</script>
Expand Down
13 changes: 13 additions & 0 deletions docs-site/public/api/E2E.Engine.ObserveOptions.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<link rel="canonical" href="https://www.hardkoded.com/e2e-dotnet/api/E2E.Engine/ObserveOptions/">
<meta http-equiv="refresh" content="0; url=/e2e-dotnet/api/E2E.Engine/ObserveOptions/">
<title>Redirecting…</title>
<script>location.replace("/e2e-dotnet/api/E2E.Engine/ObserveOptions/" + (location.hash || ""));</script>
</head>
<body>
<p><a href="/e2e-dotnet/api/E2E.Engine/ObserveOptions/">Continue</a></p>
</body>
</html>
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ public interface IEngineSession : IAsyncDisposable

One attempt's session with the app. Disposed at the end of the attempt, after `afterEach`.

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/IEngine.cs#L179)
[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/IEngine.cs#L185)
19 changes: 19 additions & 0 deletions docs-site/src/content/docs/api/E2E.Engine/Observation.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,25 @@ null only when it did not measure it.

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/Observation.cs#L128)

<a id="E2E_Engine_Observation_Pixels_"></a>
<a id="E2E_Engine_Observation_Pixels"></a>

### Pixels


```csharp
public EngineScreenshot? Pixels { get; init; }
```


The screen's pixels, with every secure field covered, when [Pixels](/api/E2E.Engine/ObserveOptions/#E2E_Engine_ObserveOptions_Pixels) asked for them and the engine could
take them (upstream's `EngineSnapshot.pixels` and `maskedRegionCount`, as [EngineScreenshot](/api/E2E.Engine/EngineScreenshot/)). Null when it did
not ask, or when the image could not be taken or proven redacted: the observation is then the tree alone.

**Value:** [EngineScreenshot](/api/E2E.Engine/EngineScreenshot/)

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/Observation.cs#L154)

<a id="E2E_Engine_Observation_Revision_"></a>
<a id="E2E_Engine_Observation_Revision"></a>

Expand Down
64 changes: 64 additions & 0 deletions docs-site/src/content/docs/api/E2E.Engine/ObserveOptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: "ObserveOptions"
description: "What an observation takes besides the tree. Upstream's EngineObserveOptions."
editUrl: false
prev: false
next: false
sidebar:
label: "ObserveOptions"
hidden: true
---

<a id="E2E_Engine_ObserveOptions"></a>

Namespace: [E2E.Engine](/api/E2E.Engine/)


```csharp
public sealed record ObserveOptions : IEquatable<ObserveOptions>
```


What an observation takes besides the tree. Upstream's `EngineObserveOptions`.

**Inheritance:** [object](https://learn.microsoft.com/dotnet/api/system.object)

**Implements:** [IEquatable&lt;ObserveOptions&gt;](https://learn.microsoft.com/dotnet/api/system.iequatable-1)

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/Observation.cs#L158)

## Properties

<a id="E2E_Engine_ObserveOptions_Comparable_"></a>
<a id="E2E_Engine_ObserveOptions_Comparable"></a>

### Comparable


```csharp
public bool Comparable { get; init; }
```


Take the pixels in a state fit for comparing against a stored one: the status bar normalized.

**Value:** [bool](https://learn.microsoft.com/dotnet/api/system.boolean)

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/Observation.cs#L164)

<a id="E2E_Engine_ObserveOptions_Pixels_"></a>
<a id="E2E_Engine_ObserveOptions_Pixels"></a>

### Pixels


```csharp
public bool Pixels { get; init; }
```


Take the screen's pixels with the tree ([Pixels](/api/E2E.Engine/Observation/#E2E_Engine_Observation_Pixels)).

**Value:** [bool](https://learn.microsoft.com/dotnet/api/system.boolean)

[View source](https://github.com/hardkoded/e2e-dotnet/blob/main/src/E2E/Engine/Observation.cs#L161)
1 change: 1 addition & 0 deletions docs-site/src/content/docs/api/E2E.Engine/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ Public types in the `E2E.Engine` namespace.
| [NodeStates](/api/E2E.Engine/NodeStates/) | Boolean states a platform can report for one node. |
| [Observation](/api/E2E.Engine/Observation/) | A redacted semantic snapshot of the current screen. |
| [ObservationLimits](/api/E2E.Engine/ObservationLimits/) | Limits from the engine contract. Engines cut names and text before they reach the model. |
| [ObserveOptions](/api/E2E.Engine/ObserveOptions/) | What an observation takes besides the tree. Upstream's EngineObserveOptions. |
| [ProviderRecordContext](/api/E2E.Engine/ProviderRecordContext/) | Handed to a provider's RecordAsync, once per attempt that records video. |
| [ProviderRecording](/api/E2E.Engine/ProviderRecording/) | A recording a provider's service makes of the browser or device it leased, as RecordAsync started it. |
| [ProviderRecordingResult](/api/E2E.Engine/ProviderRecordingResult/) | What a stopped recording became: a file the provider wrote into the directory it was given, by name (File), or a recording its service keeps, by http(s) URL and media type (Url and MediaType). |
Expand Down
8 changes: 7 additions & 1 deletion skills/e2e-dotnet/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,13 @@ lines you changed:
with `ACTION_FAILED` (`InnerException` `OPERATION_TIMEOUT`) instead of holding
it until the test timeout. `WaitForAsync`, `Expect`, and `ToHaveScreenshotAsync` give their
last read extra time, up to their slowest read so far and within the test
timeout, so a screen that shows up during it still passes.
timeout, so a screen that shows up during it still passes. An engine session
takes the screen's pixels with the tree through
`ObserveAsync(new ObserveOptions { Pixels = true, Comparable = ... }, timeout, ct)`
(`Observation.Pixels`, secure fields covered); the screenshot shares the
observation's budget, and a device that stops answering fails the call with
`OPERATION_TIMEOUT`. Any other failure of the screenshot leaves the
observation with the tree alone.
- Secrets never appear in test code. Declare them under `secrets` in the
config and read them with `Secrets.Get(name)`; accounts come from
`Credentials.User(name)`. Hand the `Secret` only to `FillAsync` or
Expand Down
Loading
Loading