Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,11 +168,11 @@ Differences:
Differences:

- A session is one `Session` class with `Screen`, `App`, `Platform`, `Browser`, `SaveStateAsync`, `SetDefaultTimeout`, `CloseAsync`, and `DisposeAsync`, where upstream's is a flat object typed by the engine's fixtures (`Host<Fixtures>`). There is no `Device` member on it yet (#1678). It has no `agent`: `createAgent` is not ported (#1676).
- `LaunchOptions` has no `Headed` (the web engine takes its window mode through its own options), no `Signal` or `HandleSignals` (the token passed to `LaunchAsync` and `NewSessionAsync` is the signal; closing on SIGINT, SIGTERM, and SIGHUP is #1677), and `Timeout` is a `TimeSpan`. `Env` is a dictionary and `OnNotice` takes the line only.
- `LaunchOptions` has no `Headed` (the web engine takes its window mode through its own options), no `Signal` (the token passed to `LaunchAsync` and `NewSessionAsync` is the signal). `HandleSignals` closes the host on SIGINT, SIGTERM, and SIGHUP (`Core/Signals.cs` is `core/signals.ts`, `Signals.CloseOnSignals` is `closeOnSignals`); `engineListeners` is not ported, because .NET cannot list a process's signal registrations (#1693), so a program's listener is told from an engine's by whether it cancels the signal raised again, and on Windows the process exits with 128 plus the signal number without asking the program, and `Timeout` is a `TimeSpan`. `Env` is a dictionary and `OnNotice` takes the line only.
- `RedactThrown` rebuilds an error that carries a secret value (type and code kept, causes redacted) where upstream edits the thrown error in place, which a .NET exception's read-only message does not allow. An error with no secret is thrown as it is.
- `SessionState.ExportAsync` and `ImportAsync` are `Task`-returning like upstream's `async` methods, so a bad key rejects instead of throwing. The key is `SessionStateKeyOptions.Key`, a `byte[]` of 32 bytes; there is no unknown-option check (`plaintext: true` cannot be passed).
- `app.linkBase` (`AppConfig.LinkBase`, `ResolvedApp.LinkBase`, `LaunchApp.LinkBase`, checked by `TargetApps.CheckLinkBase`, upstream's `checkLinkBase`) and the engine `openLink` hook (`IEngineSession.OpenLinkAsync`, `EngineSessionSpec.OpenLink`, `ISessionApp.OpenLinkAsync`, `session/openLink` in the manifest schema) are ported, with `App.OpenAsync(path)` joining and opening the link as upstream's `launchPinnedApp` does. `EngineCapabilities.OpenLink` is .NET-only: C# has no optional member to test, so an engine declares the hook the way it declares `Resize`. `TargetApps.JoinLink`, `TargetApps.RefusesLink`, and `Urls.IsRefusedLinkScheme` (upstream's `joinLink`, `refusesLink`, `isRefusedLinkScheme`) are shared with `DeviceLinks`. Not ported: the unknown-key message of a `defineEngine` session, which a typed `EngineSessionSpec` cannot receive (`engine.test.ts`).
- Tests: `Core/CoreSessionStateTests` is `core-state.test.ts`; `Core/CoreTests` is the ported part of `core.test.ts` (partial, and renamed `Partial_...`: 'opens sessions one at a time' (no `gadget` fixture or key list), 'refuses what is not an engine' (no non-engine or unknown app key to pass), 'says a closed session is closed' (the redaction half only), 'bounds each operation by its default timeout' (no agent step, which needs `createAgent`, #1676)); `AppLinkBase/` is `app-link-base.test.ts`; `Core/CoreTests` also holds 'opens a path as a link in the pinned app on a device target with a link base' (`Opens_a_path_as_a_link_...`); `SchemaFixtures/SessionStateV1SchemaTests`; `StepVerification/StepRecorderLastVerifiedStepIndexTests` has the two new `it` (`RedactThrown` is partial: the caught error is checked, not the original in place); `TraceCache/Trace1EntryTests` has the space-press `it`. Not ported: the `it` of `createAgent`, the link-base `it` of `core.test.ts`, `core-signals.test.ts`, and the `engine.test.ts` fixture-name check (#1676 to #1678).
- Tests: `Core/CoreSessionStateTests` is `core-state.test.ts`; `Core/CoreTests` is the ported part of `core.test.ts` (partial, and renamed `Partial_...`: 'opens sessions one at a time' (no `gadget` fixture or key list), 'refuses what is not an engine' (no non-engine or unknown app key to pass), 'says a closed session is closed' (the redaction half only), 'bounds each operation by its default timeout' (no agent step, which needs `createAgent`, #1676)); `AppLinkBase/` is `app-link-base.test.ts`; `Core/CoreTests` also holds 'opens a path as a link in the pinned app on a device target with a link base' (`Opens_a_path_as_a_link_...`); `SchemaFixtures/SessionStateV1SchemaTests`; `StepVerification/StepRecorderLastVerifiedStepIndexTests` has the two new `it` (`RedactThrown` is partial: the caught error is checked, not the original in place); `TraceCache/Trace1EntryTests` has the space-press `it`. Not ported: the `it` of `createAgent`, the link-base `it` of `core.test.ts`, and the `engine.test.ts` fixture-name check (#1676 to #1678). `CoreSignals/CoreOnASignalTests` ports the three `it` of `core-signals.test.ts`; the second is partial (`Partial_EndsBySighupThoughTheEngineListensForItAndLeavesAProgramThatListensItselfToDecide`): it does not check that an engine's own listener is removed before the signal is raised again, which .NET cannot tell from a program's (#1693).

## Not ported

Expand Down
15 changes: 15 additions & 0 deletions docs-site/src/content/docs/api/E2E/LaunchOptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,21 @@ The environment the engine reads; default the process environment.

**Value:** [IReadOnlyDictionary&lt;string, string&gt;](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlydictionary-2)

<a id="E2E_LaunchOptions_HandleSignals_"></a>
<a id="E2E_LaunchOptions_HandleSignals"></a>

### HandleSignals


```csharp
public bool HandleSignals { get; init; }
```


Closes the host on SIGINT, SIGTERM, and SIGHUP, so the browsers and devices it launched do not outlive the program; the signal is raised again afterwards unless the program listens for it itself. Default `true`. A closing that fails is said on stderr. Upstream's `handleSignals`.

**Value:** [bool](https://learn.microsoft.com/dotnet/api/system.boolean)

<a id="E2E_LaunchOptions_Headed_"></a>
<a id="E2E_LaunchOptions_Headed"></a>

Expand Down
3 changes: 2 additions & 1 deletion docs-site/src/content/docs/docs/core.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,14 @@ foreach (var sku in new[] { "A-1001", "B-2040" })
| Sessions | One session at a time on a host. Close it before `NewSessionAsync()`. For parallel work, launch more hosts. |
| Timeouts | An action, a wait, or a read gets 30 seconds. `App.OpenAsync()` and `App.RestartAsync()` get 120 seconds. `LaunchOptions.Timeout` or `session.SetDefaultTimeout(timeout)` sets one value for all of them, for the operations that start from then on. There is no session deadline. |
| Cancellation | The token passed to `LaunchAsync` and `NewSessionAsync` stops what is running. |
| Ctrl-C | The host closes on SIGINT, SIGTERM, and SIGHUP, so its browser or device session does not stay behind. Then the program ends by the signal, unless it listens for the signal itself (`PosixSignalRegistration`), which then sees it again and decides. Turn it off with `LaunchOptions.HandleSignals = false`. A close that fails is said on stderr first. |
| App links | On a device, `LaunchApp.LinkBase` (an absolute URL or a scheme such as `shopapp://`, with no query or fragment) makes `session.App.OpenAsync("/orders/42")` open `shopapp://orders/42` in the pinned app. An absolute link opens as given; a `file:`, `data:`, `javascript:`, `view-source:`, `blob:`, or `filesystem:` link is `POLICY_DENIED`. An engine with no link hook, such as `WebEngine`, refuses a link base with `INVALID_CONFIG`. |
| Window | `LaunchOptions.Headed` shows the browser window; the default is hidden. A device ignores it. |
| Files | Screenshots and downloads go to a temp directory that is deleted when the host closes. Set `ArtifactsDir` to keep them; a relative path resolves from `Cwd`. |
| Closing | `CloseAsync` is idempotent and throws an `AggregateException` of the failures closing found. |

<Aside type="note">
Not ported yet: closing the host on SIGINT, SIGTERM, and SIGHUP, and `createAgent`. Issues [#1676](https://github.com/hardkoded/e2e-dotnet/issues/1676) and [#1677](https://github.com/hardkoded/e2e-dotnet/issues/1677) track them.
Not ported yet: `createAgent` and the marking of an engine's own signal listeners (upstream's `engineListeners`). Issues [#1676](https://github.com/hardkoded/e2e-dotnet/issues/1676) and [#1693](https://github.com/hardkoded/e2e-dotnet/issues/1693) track them.
</Aside>

## Keep a signed-in state
Expand Down
2 changes: 1 addition & 1 deletion skills/e2e-dotnet/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ trip. The installed tool prints the same text with `e2e guide <topic>`
| Running tests: `dotnet test` filters, the `e2e` tool (`e2e doctor` checks that `e2e explore` and `e2e mcp` can start; `--json` prints the `doctor-1` report for a host, `--check-model` calls each model once; `e2e cache ls|stats|clear` lists, sizes, or empties the replay cache), `report.json` for scripts, exit codes, CI, test filters (NUnit, xUnit v3, and TUnit, mobile and Windows adapters included: `E2ERunFixture.Filters` and `ChangedSince`, or `E2E_CHANGED_SINCE`; `run.changes`, `run.filters`, and `filtered` skips in `report.json`), `--last-failed` reruns (`E2ERunFixture.LastFailed` or `E2E_LAST_FAILED=1`; `run.carried` in `report.json`, `FinishedRun.LastRun` for reporters, folded into the GitHub comment) | `running` | [references/running.md](references/running.md) |
| Handling a failed run: the trace page, `report.json` for scripts, `E2ESession.Report()` for a session you run yourself, error codes and their fixes, a visible browser, replay misses, phase timings, the agent step table, every model call | `debugging` | [references/debugging.md](references/debugging.md) |
| Exploring an app toward a goal without a test file, with `e2e explore` or `Explore.ExploreAsync` | `explore` | [references/explore.md](references/explore.md) |
| Driving an app from plain code with no test framework (`Launch.LaunchAsync(engine, new LaunchOptions { App, Secrets, Headed, Timeout, ArtifactsDir, Cwd })`, `Host.NewSessionAsync`, `Host.Secret`, `Session.SaveStateAsync`, and `SessionState.ExportAsync`/`ImportAsync` with a 32-byte key you keep), or running isolated per-test sessions from your own runner: `E2EHost.OpenAsync`, `OpenHostOptions` (config value or file, target, `EngineFactory`), `host.OpenSessionAsync`, `E2ESession.CloseAsync` and the failures it returns, `Steps()` | `host` | [references/host.md](references/host.md) |
| Driving an app from plain code with no test framework (`Launch.LaunchAsync(engine, new LaunchOptions { App, Secrets, Headed, Timeout, HandleSignals, ArtifactsDir, Cwd })`, `Host.NewSessionAsync`, `Host.Secret`, `Session.SaveStateAsync`, and `SessionState.ExportAsync`/`ImportAsync` with a 32-byte key you keep), or running isolated per-test sessions from your own runner: `E2EHost.OpenAsync`, `OpenHostOptions` (config value or file, target, `EngineFactory`), `host.OpenSessionAsync`, `E2ESession.CloseAsync` and the failures it returns, `Steps()` | `host` | [references/host.md](references/host.md) |
| Testing an iOS or Android app: `MobileEngine`, the `Device` fixture, picking a simulator (without `MobileOptions.Device`, `MobileEngine` skips booted devices another agent-device session holds, and `prepare` fails with `ENGINE_FAILURE` when every booted one is held, and with none booted agent-device boots a shut-down one: on iOS an iPhone over an iPad over an Apple TV, on Android a phone or tablet over a TV, then newest OS, then name order, and a reachable paired iPhone or iPad counts as booted; `prepare` logs a per-device timing line), video of the device screen, `Device.SetPermissionAsync` (the enums, or wire-name strings such as `"camera"` and `"grant"`; an unknown name or state is `INVALID_ARGUMENT` before any device command), `Device.UnstableRaw` (the test's agent-device session: name, device id and name, selection, and a raw `IDeviceClient` whose calls are not steps, replayed, or redacted, and that throws `NotSupportedException` for the `text-size` and `animations` settings, `apps.uninstall`, and `command.hardwareButton`; unstable; `INVALID_STATE` after the session is disposed; a custom agent tool gets the running attempt's from `MobileTools.UnstableDeviceSession(engines)`), a NativeScript `testID` or `accessibilityIdentifier` as the test id `GetByTestId` reads (an Android `accessibilityLabel` is put before the text), Limrun and BrowserStack hosted devices, a MAUI app through the DevFlow agent, known limits, `Locator.PressAsync("Enter")` tapping an unfocused field first, and the agent's `open_app` returning to the app under test when it names no app, a Compose Multiplatform field on iOS (for an empty (no value), editable, non-secure iOS field that reports no placeholder of its own, whose only child is a `StaticText` with no children, and whose label is a non-empty name, then `, `, then that child's text, that text is the field's `Placeholder`, the child is left out, and the name is the label before `, `; a non-empty `ContentDescription` stays the name, and that label is then the `Text`; any other field keeps its label and children), and `FillAsync` and `ClearAsync` refusing a node that is not an editable field with `UNSUPPORTED_CAPABILITY` (an iOS `other` that holds a value or only text is typed into), reading the field again before typing (a field the new capture no longer lists, an incomplete capture, or a changed input purpose or secure, disabled, or hidden state is a retryable `NODE_STALE` before anything is typed), looking at the screen again after the keyboard opens or closes (an unmoved control is acted on at once only when keyboard visibility is unchanged, else it waits out `TransitionMilliseconds`; a test's screen swipe waits out the remaining budget before it scrolls), and a `target_still_moving` refusal with `dispatched: no` being a retryable `NODE_STALE` | `mobile` | [references/mobile.md](references/mobile.md) |
| Testing a Windows desktop app: `WinAppEngine`, `WinAppTest`, the `Window` fixture, the `windows` target in `e2e explore` and `e2e mcp`, the interactive-desktop requirement, known limits | `windows` | [references/windows.md](references/windows.md) |
| Asked to bug bash or hunt for bugs: the bug-bash config, many `e2e explore` runs at once, proving each bug with a repro test | `bug-bash` | [references/bug-bash.md](references/bug-bash.md) |
Expand Down
4 changes: 2 additions & 2 deletions skills/e2e-dotnet/references/host.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ await session.Screen.GetByLabel("Password").FillAsync(shop.Secret("SHOP_PASSWORD

- A second `LaunchAsync` on an engine instance another host owns, and a second open session on a host, fail with `WORKER_PROTOCOL`. Parallel work is more hosts.
- A secret value is at least 6 characters, and is redacted from the errors a session throws and from `Screen.TreeAsync()`. `Host.Secret(name)` with another name is `SECRET_UNAVAILABLE`.
- `LaunchOptions.Timeout` bounds an action, a wait, a read, and `App.OpenAsync()` or `App.RestartAsync()` (30 seconds for the first three and 120 seconds for the last two without it). `session.SetDefaultTimeout(timeout)` moves all of them for the operations that start from then on; a zero or infinite span is `INVALID_ARGUMENT`. There is no session deadline. `Headed` shows the browser window (a device ignores it). An operation on a closed session fails with `CANCELLED` and the message `the session is closed`. `ArtifactsDir` keeps screenshots and downloads (a relative path resolves from `Cwd`); without it they go to a temp directory deleted when the host closes.
- `LaunchOptions.Timeout` bounds an action, a wait, a read, and `App.OpenAsync()` or `App.RestartAsync()` (30 seconds for the first three and 120 seconds for the last two without it). `session.SetDefaultTimeout(timeout)` moves all of them for the operations that start from then on; a zero or infinite span is `INVALID_ARGUMENT`. There is no session deadline. `Headed` shows the browser window (a device ignores it). An operation on a closed session fails with `CANCELLED` and the message `the session is closed`. On SIGINT, SIGTERM, and SIGHUP the host closes, then the program ends by the signal unless it listens for it itself with `PosixSignalRegistration` (it sees the signal again and decides); `LaunchOptions.HandleSignals = false` turns that off, and a failed close is said on stderr. `ArtifactsDir` keeps screenshots and downloads (a relative path resolves from `Cwd`); without it they go to a temp directory deleted when the host closes.
- `await session.SaveStateAsync()` captures cookies and storage in memory. Pass it to `shop.NewSessionAsync(new NewSessionOptions { State = state })`. To keep it between runs, `state.ExportAsync(new SessionStateKeyOptions { Key = key })` with a 32-byte key you keep, and `SessionState.ImportAsync(bytes, options)`. A state restores only on the same engine, platform, and app (`SESSION_MISMATCH`; give two addresses of one app the same `LaunchApp.Identity`), a wrong key or changed byte is `SESSION_INVALID`, and an old one `SESSION_EXPIRED`.
- `LaunchApp.LinkBase` (an absolute URL or a scheme such as `shopapp://`, with no query or fragment) makes `session.App.OpenAsync("/orders/42")` open `shopapp://orders/42` in the pinned app of a device engine. An absolute link opens as given; `file:`, `data:`, `javascript:`, `view-source:`, `blob:`, and `filesystem:` are `POLICY_DENIED`; an engine without a link hook, or a bad value, is `INVALID_CONFIG` at launch.
- Not ported yet: closing on signals and `createAgent` (issues 1676 to 1678).
- Not ported yet: `createAgent`, and an engine's own signal listeners (issues 1676, 1678, and 1693). On Windows, the host closes on a signal and the process then exits with 128 plus the signal number, whether or not the program listens for it.
Loading
Loading