diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 27defa3e8..21c5e24b7 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -56,8 +56,8 @@ Names are C# versions of the JavaScript API: `agent.act` is `ActAsync`, `screen. - The `browser` fixture subset: `goto`, `reload`, `back`, `forward`, `url`, `title`, `waitForURL`, `evaluate`, `waitForResponse`, `route`, `unroute`, `unrouteAll`, `cookies`, `setCookies`, `clearCookies`, `setViewport`, `emulateMedia`, `addInitScript`, `keyboard.press`, `keyboard.type`, `keyboard.down`, `keyboard.up`, and `mouse.move`, `wheel`, `down`, `up` - `press` keys are checked in core, as upstream's `requireKey` (`internal/keys.ts`, `Keys.RequireKey`) does: `Locator.PressAsync` checks the key before the node is resolved (in `LocatorEngine.CheckAction`, upstream's `checkAction`), and the agent's `press` and `pressKey` check it in the dispatcher before the action is committed. A key outside the grammar (`KeyGrammar.Parse`, upstream's `parseKey`: zero or more modifiers and one key joined by `+`) is `INVALID_ARGUMENT` on every engine, where the mobile and Windows engines alone refused it with `UNSUPPORTED_CAPABILITY` (those checks stay, as upstream's engines keep theirs). `Locator.PressAsync` no longer throws an `ArgumentException` for an empty or blank key: `""` is `INVALID_ARGUMENT` like any key outside the grammar, and `" "` is a space character key. `Browser.Keyboard.PressAsync` takes any Playwright key (`Alt+KeyA`), as upstream's `browser.keyboard` does, and is not checked. The agent's `press` still refuses an empty or blank key first with `press requires a key` (a .NET-only pre-check). - Agent budgets: `ActOptions.MaxSteps`, `ActOptions.MaxModelCalls`, and `WaitForOptions.MaxModelCalls`. A per-call budget can only lower the agent's configured one. A higher or non-positive value throws `INVALID_ARGUMENT` -- An `act` past its action budget ends `STEP_BUDGET_EXHAUSTED`, blocked. The model is told and may still conclude: a passing verdict, or a failure without a code, becomes the budget error. Replayed actions draw on the same budget. `navigate`, `back`, `scroll`, and `scroll_to` take a slot; `observe` does not. An `act` whose tool loop uses every model turn without a `done` call ends `STEP_NO_CONCLUSION`, a failed verdict, as upstream's does (see the tool loop bullet); an executor that records one model call more than the budget allows still ends `STEP_BUDGET_EXHAUSTED`, blocked (`agent.act exhausted its model-call budget of N`) -- As upstream, an `act` guards against a model that repeats itself. The same call with the same input 3 times in a row, or a sequence of 2 to 4 calls twice, adds a warning. 5 identical calls, or a sequence 3 times, offers the model only `done`, so it still writes its own verdict. The `done` call never counts. The failure streak is upstream's `checkFailureStreak` (`LoopGuards.CheckFailureStreak`, tester-army/e2e `loop-guards.ts` at `f4be1c4`): 3 failed calls in a row add a notice, 5 offer the model only `done`, and a call that worked starts the count over. The loop reads each tool result's text as upstream does: `LoopGuards.ExtractToolResults` (upstream's `extractToolResults`) over the tool messages, and `LoopGuards.IsFailedResult` (upstream's `isFailedResult`) for the failure shape. A call the SDK refused or a tool that threw is an `error-text` result (`ModelMessage.ErrorText`, `ToolOutcome.ErrorText`), so it counts as failed whatever its text says. Differences: a project tool's structured result has no port counterpart (the port's tool results are text, so the `it` that reads a structured result skips it), and the grammar's failure leads are the port's own text: `ActionNames.GrammarToolNames` also holds the port's `fill`, `fill_secret`, `uncheck`, and `clear`, which upstream does not have, so their failures count too. The partial read of a structured result is `Partial_Reads_text_results_...` in `LoopGuards/FailureStreakTests`. The thresholds are upstream's `LoopGuardThresholds` and `DEFAULT_LOOP_GUARD_THRESHOLDS` (`LoopGuards.DefaultLoopGuardThresholds`), and `CheckLoopGuards` and `CheckFailureStreak` take an optional `thresholds` argument; the loop passes none, and the type is internal, so no public option exposes it. A turn with no tool call counts as a failure. An agent step's transcript is kept only under `E2E_DEBUG=1`, as upstream keeps it under `--debug`: the loop attaches it, and it is a step-attributed `log` artifact (see Custom executors). +- An `act` past its action budget ends `STEP_BUDGET_EXHAUSTED`, blocked. The model is told and may still conclude: a passing verdict, or a failure without a code, becomes the budget error. Replayed actions draw on the same budget. `navigate`, `back`, `scroll`, and `scroll_to` take a slot; `observe` does not. An `act` whose tool loop uses every model turn without a `complete_step` call ends `STEP_NO_CONCLUSION`, a failed verdict, as upstream's does (see the tool loop bullet); an executor that records one model call more than the budget allows still ends `STEP_BUDGET_EXHAUSTED`, blocked (`agent.act exhausted its model-call budget of N`) +- As upstream, an `act` guards against a model that repeats itself. The same call with the same input 3 times in a row, or a sequence of 2 to 4 calls twice, adds a warning. 5 identical calls, or a sequence 3 times, offers the model only `complete_step`, so it still writes its own verdict. The `complete_step` call never counts. The failure streak is upstream's `checkFailureStreak` (`LoopGuards.CheckFailureStreak`, tester-army/e2e `loop-guards.ts` at `f4be1c4`): 3 failed calls in a row add a notice, 5 offer the model only `complete_step`, and a call that worked starts the count over. The loop reads each tool result's text as upstream does: `LoopGuards.ExtractToolResults` (upstream's `extractToolResults`) over the tool messages, and `LoopGuards.IsFailedResult` (upstream's `isFailedResult`) for the failure shape. A call the SDK refused or a tool that threw is an `error-text` result (`ModelMessage.ErrorText`, `ToolOutcome.ErrorText`), so it counts as failed whatever its text says. Differences: a project tool's structured result has no port counterpart (the port's tool results are text, so the `it` that reads a structured result skips it), and the grammar's failure leads are the port's own text: `ActionNames.GrammarToolNames` is upstream's `GRAMMAR_TOOL_NAMES`. The partial read of a structured result is `Partial_Reads_text_results_...` in `LoopGuards/FailureStreakTests`. The thresholds are upstream's `LoopGuardThresholds` and `DEFAULT_LOOP_GUARD_THRESHOLDS` (`LoopGuards.DefaultLoopGuardThresholds`), and `CheckLoopGuards` and `CheckFailureStreak` take an optional `thresholds` argument; the loop passes none, and the type is internal, so no public option exposes it. A turn with no tool call counts as a failure. An agent step's transcript is kept only under `E2E_DEBUG=1`, as upstream keeps it under `--debug`: the loop attaches it, and it is a step-attributed `log` artifact (see Custom executors). - `ActResult.ModelCalls` (0 for a full replay) and `ActResult.Actions` (replayed and live actions, counting failed attempts) - `AgentException.Blocked` and `AgentException.Explanation` (the same text as `Message`) - OpenAI-compatible tool calling @@ -171,7 +171,7 @@ Differences: - Model reasoning on step events, #1196 (tester-army/e2e `agent/executor.ts`, `agent/usage.ts`): the reasoning a turn returns rides its `model` event as `Reasoning`, for an executor turn (`ExecutorModelCall.Reasoning`, set by `ToolLoop` and `StepRecorder.AddModelCall`) and for a judgment call alike, and the list reporter prints its excerpt (`ListSteps`). `ModelResponse.ReasoningText` is upstream's `StepResult.reasoningText`: each provider fills it from the reasoning text it returned (Anthropic `thinking` blocks, Responses reasoning summaries, Gemini thought parts, Bedrock `reasoningText`, chat `reasoning_content`/`reasoning`), so an unsigned Anthropic thinking block shows in the report although it is never replayed. `ModelUsage.ReasoningField` is upstream's `reasoningField`: the text is trimmed, redacted, and cut to `ModelUsage.MaxReasoningChars` (600, upstream's `MAX_REASONING_CHARS`); a turn with no reasoning leaves the field out of the report. The AI trace keeps the full text. The port has no `ModelCallTracker`; `ToolLoop` and the judgment call pass `ReasoningText` on directly. - The `dismiss_keyboard` tool and the `dismissKeyboard` executor verb (#1014, tester-army/e2e `f4be1c4`): `dismiss_keyboard` in `AgentTools.GrammarToolDefinitions` (upstream's `dismiss_keyboard` in `createGrammarTools`, `agent/primitives.ts`), run by `ActTools`; `IExecutorActions.DismissKeyboardAsync` and `ExecutorVerb.DismissKeyboard` (upstream's `dismissKeyboard` of `ExecutorActions` and `target.verbs`), run by `ActionDispatcher.DismissKeyboardAsync` (upstream's `dismissKeyboard`). The trace records `dismissKeyboard` (`RecordedAction.Kind`), `Replay` replays it as given, `SettlePolicy` settles it like a key, and `ActionName.DismissKeyboard` reports it to `OnAction` (upstream's `actions.ts` summary is `dismiss the keyboard`). Upstream offers the verb when the engine defines the optional `keyboard.dismiss` hook; C# has no optional member to test, so `IEngineSession.CanDismissKeyboard` declares it (.NET-only, false by default, like `CanLocate`) and `IEngineSession.DismissKeyboardAsync` is the hook (a default member that refuses with `UNSUPPORTED_CAPABILITY`, so no engine breaks). `DeviceSession` is the one engine that says true, and its hook runs the dismissal under the `keyboard dismiss` input budget, as upstream's `keyboard.dismiss` does. `IExecutorActions.DismissKeyboardAsync` is a default member that refuses with `UNSUPPORTED_CAPABILITY` (.NET-only), so a custom executor context written before it keeps compiling. A session over MCP lists `dismiss_keyboard` after `back` where the engine can dismiss, and it answers with the reason `a test hides the keyboard only when it covers the next control` instead of a test line (upstream's `test-lines.ts`). Like upstream's, the tool tells the model nothing about the keyboard leaving and does not expect a change. Not ported: the decision executor's `dismiss_keyboard` (`packages/decision`, offered while `keyboardVisible`), which the port does not have. Tests: the keyboard lines of `engine.test.ts`'s verbs `it` are `Engine/CreateEngineSessionTests.Partial_Derives_the_keyboard_and_pointer_verbs_from_the_declared_hooks` (partial: the port has no `createEngineSession` or `open` verb, so it checks `ActContext.VerbsOf`); the `dismiss_keyboard` input of `agent-grammar-tools.test.ts` is checked in `AgentGrammarTools/TheGrammarToolsHaveClosedSchemasTests` (the port's schemas are not closed); `mcp-test-lines.test.ts` in `McpTestLines/SessionTestLinesTests`; the `dismissKeyboard` of `trace-replay.test.ts`'s host in `TraceReplay/TestHost`; the `keyboard dismiss` row of the mobile `engine.test.ts` input-budget `it` now calls the engine's hook (`IEngineSession.DismissKeyboardAsync`); and `agent-act-pixel-tools.test.ts` already checks that no `dismiss_keyboard` is offered without the hook. - The `resize_viewport` tool and the `resize` executor verb (#1035, tester-army/e2e#1524 `cba5e0e`): `resize_viewport` in `AgentTools.GrammarToolDefinitions` (upstream's `resize_viewport` in `createGrammarTools`, `agent/primitives.ts`), run by `ActTools` with `ExpectChange: false` as upstream's `acting`; `IExecutorActions.ResizeAsync` and `ExecutorVerb.Resize` (upstream's `resize` of `ExecutorActions` and `target.verbs`), run by `ActionDispatcher.ResizeAsync` (upstream's `resize`). The size is whole CSS pixels from 200 to 4096 per side (`Geometry.ViewportLimits`, upstream's `VIEWPORT_LIMITS`; `Geometry.RequireViewportSize`, upstream's `requireViewportSize`); a size outside them is `INVALID_ARGUMENT` and takes no action slot. The step's viewport becomes the new size (upstream's `attachViewport`, with scale 1). The trace records `resize` with its `Size` (`RecordedAction.Size`, upstream's `ResizeAction.size`; an entry whose size is outside the limits is a miss), `Replay` replays it as given, `Decide.OpensWithNavigate` passes over a leading resize, `SettlePolicy` settles it like a scroll, and `ActionName.Resize` reports it to `OnAction` (summary `resize the viewport to 390x844`). Upstream offers the verb when the engine defines the optional `session.resize` hook; C# has no optional member to test, so `EngineCapabilities.Resize` declares it (.NET-only, as `History` is for `session.back`) and `IEngineSession.ResizeAsync` is the hook (a default member that refuses with `UNSUPPORTED_CAPABILITY`). `WebEngine` declares it and sets the viewport as `Browser.SetViewportAsync` does, then hands the trace the screen. `IExecutorActions.ResizeAsync` is a default member that refuses with `UNSUPPORTED_CAPABILITY` (.NET-only), so a custom executor context written before it keeps compiling. A session over MCP lists `resize_viewport` after `back` where the engine can resize, and answers with the reason `set 390x844 with the engine fixture's viewport call` instead of a test line. `ExecutorObservation.Viewport` (internal) is now set on the executor's observation, as upstream's `viewport` is. The document engine and the device engines do not resize, so the port's tool list of the default-executor test (`AgentActWithTheDefaultToolLoopAgentExecutorTests`, which runs on the document engine) stays as it was. Not ported: the engine protocol's `session/resize` method and the manifest schema (the port has no engine protocol, #873), the decision package's test helper, and the `engine.test.ts` defineEngine key check (the port has no `defineEngine`). Tests: `AgentActVerbs/` (the resize flow and its `it`; the engine-name line of the record-then-replay `it` is not ported, as the port's replay records no engine events), `AgentAct/AgentActResizeTests`, `AgentGrammarTools/TheGrammarToolsHaveClosedSchemasTests` (partial: the port's schemas are not closed), `Engine/CreateEngineSessionTests`, `TraceCache/` (the leading resize in `DecideTraceReplayTests`; the limits in `Trace1EntryTests`), `TraceRecorder/TraceRecorderTests` (the recording, and the stored size check as a partial), `TraceReplay/ReplayTraceTests`, `McpTestLines/SessionTestLinesTests`, and the tool list and count in `McpServer/E2EMcpTests`. -- Point tools (#632, tester-army/e2e#1284 `90e6fb8`, with `drag_at` of upstream main): `tap_at`, `hover_at`, `drag_at`, `type_at`, `press_at`, and `select_at` in `AgentTools.CreatePointTools` (upstream's `createPointTools` and `POINT_TOOLS`, `agent/primitives.ts`), run by `ActTools.PointToolAsync`. A point is read off the latest screenshot (`ScreenPresenter.LatestScreenshot`, upstream's `latestScreenshot`) and scaled by `PointTap.ImagePointToViewport`; `PointTap.HitTest` is `hitTest`, and `DescribePointAction`, `DescribePointDrag`, and `DescribePointHit` are upstream's. The executor gets `TapAtAsync`, `HoverAtAsync`, `DragAtAsync`, and `HitTestAsync` (`ActionDispatcher.Points.cs`: `pointVerb`, `pointDrag`, `hitTest`); the bare point goes through `IEngineSession.PerformAtAsync` when the engine declares the pointer kind (`tap`, `hover`, `dragTo`), else a tap acts on the control by id. The trace records `tapAt`, `hoverAt`, and `dragAt` with the point, the viewport (`RecordedAction.Viewport`), and the place inside the node under it (`RecordedAction.Under`, `RecordedPointPlace`: upstream's `within`); `Replay.PlacePoint` and `PlaceWithin` are `placePoint` and `placeWithin`, and a viewport of another size hands off with `viewport-changed`. `tap_at`, `hover_at`, and `drag_at` are offered when the engine takes the bare point or lists the node verb (`tap`, `hover`, `dragTo` in `ActionKinds`), and `type_at`, `press_at`, and `select_at` with the node verb they act through (see the Agent verbs entry), as upstream's `POINT_VERBS` pairing decides; on an engine without the bare point a hover acts on the listed control by id, and a drag between points on two different listed controls is joined by id (`ActionDispatcher.DragOntoAsync`, upstream's `dragOnto`, shared with the `drag` verb), while a drag inside one control or from a point on nothing listed is `UNSUPPORTED_CAPABILITY`. Differences: a port tree lists the page's top-level nodes with no root for the page, so a point inside a named top-level node is placed in it, where upstream skips its root; there is no twin position among look-alikes (the port records none), so an ambiguous place hands off; the failure lead of a point tool carries the error code (`select_at (30, 30) failed: ACTION_FAILED: ...`), where upstream's code-less error reads without one; the replay of a recorded point emits no step engine event, as no other replayed verb does. Tests: `AgentPointTap/` (the 12 its of `agent-point-tap`, one partial), `TraceReplay/ReplayTraceBarePointTapsTests` and `ReplayTraceBarePointDragsTests` (the bare-point hover `it` of `replayTrace` is in `ReplayTraceTests`), `TraceRecorder/TraceRecorderBarePointTapsTests` and `TraceRecorderBarePointDragsTests` (the eight its of `trace-recorder.test.ts`, all `Partial_`: the port records no summary (hardkoded/e2e-dotnet#1283) and no truncation flag (#1187), and its typed entry cannot hold a field of the wrong JSON type; `TraceRecorder.Record(kind, PlacedPoint, viewport)` and `Record(from, to, viewport)` are upstream's `record` of a point action, with `PlacedPoint` and the recorder's `withinOf`; a stored place whose `fx` or `fy` is outside 0 to 1 is a miss, as upstream's `isFraction`), and the point its of `AgentActPixelTools/` and `AgentTraceCache/` on the canvas and repeats pages. `AgentActPixelTools/` has all the its of `agent-act-pixel-tools`; `AgentActVerbs/` has the repeated `press` its and the bare-point hover of `agent-act-verbs`. Keyboard (#632, `IEngineSession.TypeTextAsync`, upstream's `keyboard.type`, with `PressAsync` for keys): `ActionDispatcher.TypeTextAsync` and `PressKeyAsync` are upstream's `typeText` and `pressKey`, recorded as `typeText` (`Value`, `Replace`) and `pressKey` (`Key`) and replayed by `Replay` (`ReplayHost`'s `ActReplayHost.TypeTextAsync` and `PressKeyAsync`). `type_at` and `press_at` on a point on nothing listed tap it to focus it and go through the keyboard (`ActTools.FocusAtAsync`, upstream's `focusAt`); a listed input under the point is filled by id, and a node the engine cannot fill takes the value through the keyboard (`ActTools.TypeIntoNodeAsync`, upstream's `typeIntoNode`, `TypedThroughKeyboard`). A fill with no target types into the focused field. Differences: the port's tool is `fill`, where upstream's is `type`, so the descriptions say `fill`; `IEngineSession.TypeTextAsync` is a default member that refuses with `UNSUPPORTED_CAPABILITY` (an engine that declares `Keyboard` implements it; the default keeps custom engines compiling); the options of `typeText` are `TypeTextOptions` (C# has no inline types); `dismissKeyboard` is not ported (#1014). The `press` tool takes `times` (`AgentTools.MaxPressTimes`, upstream's `MAX_PRESS_TIMES`, 20): the first press goes to the node and the rest to the focused field through `PressKeyAsync`, one action each. The repaint report (#1012, `screen-update.ts` at `f4be1c4`): once the step shows pixels, the built-in agent compares each fresh screenshot with the one the model held (`ScreenPresenter.Attach` returns upstream's `PixelEvidence`, the digest of the image bytes, as `pixelDigest` does) and the listing with the one it read (by the shape `AgentObservation.Shape` gives). A listing that stood still under a moved screenshot opens the screen with `listed nodes unchanged; the screenshot changed. Read the effect off the image ...`, and one that stood still under the same image after an action that should have changed it says `had no visible effect` (`ScreenPresenter.RenderUnchanged`, upstream's `renderUnchanged`). With no listed control to act on, no control is blamed. Point verbs expect a change by default, as upstream's `acting`, except `type_at` on a node typed through the keyboard. The note replaces the screen as upstream's does; the `none` and `unavailable` texts are the presenter's. `Reports_a_moved_screenshot_under_an_unchanged_listing_and_blames_the_control_only_when_the_image_stood_still_too` is a whole port: the first miss reads as the `changed #n status "Hit" ...` diff line. +- Point tools (#632, tester-army/e2e#1284 `90e6fb8`, with `drag_at` of upstream main): `tap_at`, `hover_at`, `drag_at`, `type_at`, `press_at`, and `select_at` in `AgentTools.CreatePointTools` (upstream's `createPointTools` and `POINT_TOOLS`, `agent/primitives.ts`), run by `ActTools.PointToolAsync`. A point is read off the latest screenshot (`ScreenPresenter.LatestScreenshot`, upstream's `latestScreenshot`) and scaled by `PointTap.ImagePointToViewport`; `PointTap.HitTest` is `hitTest`, and `DescribePointAction`, `DescribePointDrag`, and `DescribePointHit` are upstream's. The executor gets `TapAtAsync`, `HoverAtAsync`, `DragAtAsync`, and `HitTestAsync` (`ActionDispatcher.Points.cs`: `pointVerb`, `pointDrag`, `hitTest`); the bare point goes through `IEngineSession.PerformAtAsync` when the engine declares the pointer kind (`tap`, `hover`, `dragTo`), else a tap acts on the control by id. The trace records `tapAt`, `hoverAt`, and `dragAt` with the point, the viewport (`RecordedAction.Viewport`), and the place inside the node under it (`RecordedAction.Under`, `RecordedPointPlace`: upstream's `within`); `Replay.PlacePoint` and `PlaceWithin` are `placePoint` and `placeWithin`, and a viewport of another size hands off with `viewport-changed`. `tap_at`, `hover_at`, and `drag_at` are offered when the engine takes the bare point or lists the node verb (`tap`, `hover`, `dragTo` in `ActionKinds`), and `type_at`, `press_at`, and `select_at` with the node verb they act through (see the Agent verbs entry), as upstream's `POINT_VERBS` pairing decides; on an engine without the bare point a hover acts on the listed control by id, and a drag between points on two different listed controls is joined by id (`ActionDispatcher.DragOntoAsync`, upstream's `dragOnto`, shared with the `drag` verb), while a drag inside one control or from a point on nothing listed is `UNSUPPORTED_CAPABILITY`. Differences: a port tree lists the page's top-level nodes with no root for the page, so a point inside a named top-level node is placed in it, where upstream skips its root; there is no twin position among look-alikes (the port records none), so an ambiguous place hands off; the failure lead of a point tool carries the error code (`select_at (30, 30) failed: ACTION_FAILED: ...`), where upstream's code-less error reads without one; the replay of a recorded point emits no step engine event, as no other replayed verb does. Tests: `AgentPointTap/` (the 12 its of `agent-point-tap`, one partial), `TraceReplay/ReplayTraceBarePointTapsTests` and `ReplayTraceBarePointDragsTests` (the bare-point hover `it` of `replayTrace` is in `ReplayTraceTests`), `TraceRecorder/TraceRecorderBarePointTapsTests` and `TraceRecorderBarePointDragsTests` (the eight its of `trace-recorder.test.ts`, all `Partial_`: the port records no summary (hardkoded/e2e-dotnet#1283) and no truncation flag (#1187), and its typed entry cannot hold a field of the wrong JSON type; `TraceRecorder.Record(kind, PlacedPoint, viewport)` and `Record(from, to, viewport)` are upstream's `record` of a point action, with `PlacedPoint` and the recorder's `withinOf`; a stored place whose `fx` or `fy` is outside 0 to 1 is a miss, as upstream's `isFraction`), and the point its of `AgentActPixelTools/` and `AgentTraceCache/` on the canvas and repeats pages. `AgentActPixelTools/` has all the its of `agent-act-pixel-tools`; `AgentActVerbs/` has the repeated `press` its and the bare-point hover of `agent-act-verbs`. Keyboard (#632, `IEngineSession.TypeTextAsync`, upstream's `keyboard.type`, with `PressAsync` for keys): `ActionDispatcher.TypeTextAsync` and `PressKeyAsync` are upstream's `typeText` and `pressKey`, recorded as `typeText` (`Value`, `Replace`) and `pressKey` (`Key`) and replayed by `Replay` (`ReplayHost`'s `ActReplayHost.TypeTextAsync` and `PressKeyAsync`). `type_at` and `press_at` on a point on nothing listed tap it to focus it and go through the keyboard (`ActTools.FocusAtAsync`, upstream's `focusAt`); a listed input under the point is filled by id, and a node the engine cannot fill takes the value through the keyboard (`ActTools.TypeIntoNodeAsync`, upstream's `typeIntoNode`, `TypedThroughKeyboard`). A fill with no target types into the focused field. Differences: `IEngineSession.TypeTextAsync` is a default member that refuses with `UNSUPPORTED_CAPABILITY` (an engine that declares `Keyboard` implements it; the default keeps custom engines compiling); the options of `typeText` are `TypeTextOptions` (C# has no inline types); `dismissKeyboard` is not ported (#1014). The `press` tool takes `times` (`AgentTools.MaxPressTimes`, upstream's `MAX_PRESS_TIMES`, 20): the first press goes to the node and the rest to the focused field through `PressKeyAsync`, one action each. The repaint report (#1012, `screen-update.ts` at `f4be1c4`): once the step shows pixels, the built-in agent compares each fresh screenshot with the one the model held (`ScreenPresenter.Attach` returns upstream's `PixelEvidence`, the digest of the image bytes, as `pixelDigest` does) and the listing with the one it read (by the shape `AgentObservation.Shape` gives). A listing that stood still under a moved screenshot opens the screen with `listed nodes unchanged; the screenshot changed. Read the effect off the image ...`, and one that stood still under the same image after an action that should have changed it says `had no visible effect` (`ScreenPresenter.RenderUnchanged`, upstream's `renderUnchanged`). With no listed control to act on, no control is blamed. Point verbs expect a change by default, as upstream's `acting`, except `type_at` on a node typed through the keyboard. The note replaces the screen as upstream's does; the `none` and `unavailable` texts are the presenter's. `Reports_a_moved_screenshot_under_an_unchanged_listing_and_blames_the_control_only_when_the_image_stood_still_too` is a whole port: the first miss reads as the `changed #n status "Hit" ...` diff line. - Upstream's docs site pages that have no counterpart in the port: the page objects page (`docs/page-objects.mdx`, moved to `/page-object-model` in `d30114a`), `e2e feedback` and the `docs/reference/cli.mdx` text about its opt-out variables (`4c1e2e8`), and `examples/` with its registry (`b3ae3a4`, `38b802d`, `a29edcb`, `b54646f`). The migration guides (`docs/migrate/*.mdx`, `8537abc`) are ported: see [Migration guides](#migration-guides). The docs site has .NET-only guides for upstream's five example apps of `caf51eb` (tester-army/e2e#1497: i18n, file upload, location, biometrics, deep links), under [Examples](docs-site/src/content/docs/examples/index.mdx). They are not ports: that commit only adds registry entries that link to separate repositories, so there was no guide text. Each page writes C# tests for the upstream app. The self-healing deep dive (`6fa99ea`) is ported as the [Self-healing tests](README.md#self-healing-tests) section of the guide, since the port has no docs pages for the cache, the trace page, or executors to link to. Known gap: only `e2e explore` prints a run summary, since it is the one command that feeds a run to `ListReporter` (see [List reporter](#list-reporter)): the `Cache 4 replayed · 1 handed off · 1 missed` row is `Format.CacheText` over `Format.StepsCacheTally` (`E2E.Cli`, upstream's `cacheText` and `stepsCacheTally` of `report/format.ts`), which an exploration, with its cache off, never fills, and a test run does not print ([#799](https://github.com/hardkoded/e2e-dotnet/issues/799)) - Parallel workers and serial suites - `test.fail` and the `fail` test option (tester-army/e2e#1539, `732ea8a`), partly ported. A test is marked expected to fail with NUnit's own property `E2E.Fail` (`[Property("E2E.Fail", "reason")]` on the test, or set in its body through `TestExecutionContext.CurrentContext.CurrentTest.Properties`), which needs no public API. `open_session {test}` reads the mark (`SessionTestOutcome.ExpectedFailure`, `NUnitSessionTestRunner`): a body that fails reads `failed as expected` with `Expected to fail: `, a body that passes reads `UNEXPECTED_PASS: test is marked test.fail ("") but passed`, and a `beforeAll` failure under a mark reads `failed`, as upstream. Not ported: the verdict of the run. NUnit and xUnit v3 have no expected-failure outcome, so `dotnet test` still reports a marked test that fails as failed, and a marked test that passes as passed. The port has none of upstream's registry, scheduler, retry, or reporter parts (`settleExpectedFailure`). xUnit v3 has no session runner, so the mark is read for NUnit only. @@ -191,8 +191,8 @@ The tests of upstream commits `adcfc2ba`, `c9445ce6`, `a13e119c`, `1731e0d8`, `f Ported in part (the test name carries `Partial_` or a `PARTIAL` suffix): -- Agent verbs follow the engine's declared action kinds (#1278, tester-army/e2e `engine/session.ts`, `declaredVerbs` and `KINDS_BY_VERB`): `ActContext.VerbsOf` adds `tap`, `double_tap`, `type` and `type_secret` (`fill`), `press`, `select` (`selectOption`), and `check` (both `check` and `uncheck`) only for the kinds the engine declares in `IEngineSession.ActionKinds` (`ActContext.KindsByVerb`), and `AgentTools.GrammarToolDefinitions` and `SessionCatalog.Create` offer a tool only when `AgentTools.OffersVerb` finds a verb that unlocks it, as `verbs.has(...)` does at each tool's site in `createGrammarTools` and `createPointTools` (`agent/primitives.ts`). `type` and `press` are also offered with the keyboard alone, with no target field when the engine declares no `fill` or `press`. `ActTools` refuses a call to a tool the engine did not unlock as `Unknown tool`, and `ActionDispatcher.TakesNodeVerb` reads `tap` from the kinds, so `tap_at` on an engine that declares no bare tap and no `tap` kind is not offered. `type_at`, `press_at`, and `select_at` were already offered; they now follow `fill` or the keyboard, `press` or the keyboard, and `selectOption`. Differences: the port's `clear` verb (upstream has none) follows the `clear` kind, and `uncheck` follows `check`. An engine that declares no kind gets none of these tools (a null kind list is none, as upstream's `engine?.actions ?? []`). Not ported: the scroll, `scroll_to`, `upload`, and `navigate` verbs still follow `EngineCapabilities` (the tools are shaped by the verbs they give) where upstream derives them from `swipe`, `scrollIntoView`, `setInputFiles`, and the `open` hook. Tests: `Offers_each_node_verb_only_for_the_action_kinds_the_engine_declares` (.NET-only, `AgentGrammarTools/`), `Catalogs_a_node_verb_only_for_the_action_kinds_the_engine_declares` (.NET-only, MCP catalog), and the kind lines in `Engine/CreateEngineSessionTests` (`Partial_Derives_the_keyboard_and_pointer_verbs_from_the_declared_hooks`: the lines for `tap`, `doubleTap`, `fill`, `press`, `selectOption`, `check`, `uncheck`, and `clear` of upstream's `derives the grammar verbs from the declared action kinds and hooks`) -- The ported `describe('the grammar tools have closed schemas')` and `describe('the schema a provider receives')` run in the port's names (`fill` for `type`, `done` for `complete_step`) and validate against the JSON Schema a provider receives, with `JsonSchemaCheck`, where upstream parses with zod. The rejection of a blocked verdict without a blocking code reads `failed: a blocked verdict requires errorCode ...` where upstream's reads `Rejected: a blocked verdict requires errorCode ...`. The port's tool schemas carry no `$schema` keyword (zod's JSON Schema output does), so the plain-object check lists `additionalProperties`, `properties`, `required`, and `type` +- Agent verbs follow the engine's declared action kinds (#1278, tester-army/e2e `engine/session.ts`, `declaredVerbs` and `KINDS_BY_VERB`): `ActContext.VerbsOf` adds `tap`, `double_tap`, `type` and `type_secret` (`fill`), `press`, `select` (`selectOption`), and `check` (both `check` and `uncheck`) only for the kinds the engine declares in `IEngineSession.ActionKinds` (`ActContext.KindsByVerb`), and `AgentTools.GrammarToolDefinitions` and `SessionCatalog.Create` offer a tool only when `AgentTools.OffersVerb` finds a verb that unlocks it, as `verbs.has(...)` does at each tool's site in `createGrammarTools` and `createPointTools` (`agent/primitives.ts`). `type` and `press` are also offered with the keyboard alone, with no target field when the engine declares no `fill` or `press`. `ActTools` refuses a call to a tool the engine did not unlock as `Unknown tool`, and `ActionDispatcher.TakesNodeVerb` reads `tap` from the kinds, so `tap_at` on an engine that declares no bare tap and no `tap` kind is not offered. `type_at`, `press_at`, and `select_at` were already offered; they now follow `fill` or the keyboard, `press` or the keyboard, and `selectOption`. Differences: the agent has no `clear` or `uncheck` tool, as upstream; an MCP session still serves them (`clear` on the `clear` kind, run as `ActTools`' session-only `clear`; `uncheck` as `check` with `checked: false`). An engine that declares no kind gets none of these tools (a null kind list is none, as upstream's `engine?.actions ?? []`). Not ported: the scroll, `scroll_to`, `upload`, and `navigate` verbs still follow `EngineCapabilities` (the tools are shaped by the verbs they give) where upstream derives them from `swipe`, `scrollIntoView`, `setInputFiles`, and the `open` hook. Tests: `Offers_each_node_verb_only_for_the_action_kinds_the_engine_declares` (.NET-only, `AgentGrammarTools/`), `Catalogs_a_node_verb_only_for_the_action_kinds_the_engine_declares` (.NET-only, MCP catalog), and the kind lines in `Engine/CreateEngineSessionTests` (`Partial_Derives_the_keyboard_and_pointer_verbs_from_the_declared_hooks`: the lines for `tap`, `doubleTap`, `fill`, `press`, `selectOption`, `check`, `uncheck`, and `clear` of upstream's `derives the grammar verbs from the declared action kinds and hooks`) +- The ported `describe('the grammar tools have closed schemas')` and `describe('the schema a provider receives')` run over the port's tool schemas and validate against the JSON Schema a provider receives, with `JsonSchemaCheck`, where upstream parses with zod. The rejection of a blocked verdict without a blocking code reads `failed: a blocked verdict requires a code ...` where upstream's reads `Rejected: a blocked verdict requires errorCode ...`. The port's tool schemas carry no `$schema` keyword (zod's JSON Schema output does), so the plain-object check lists `additionalProperties`, `properties`, `required`, and `type` - `replays the typing, hands off at the renamed button, and tells the model what already ran` (`Partial_Hands_off_at_the_renamed_button_after_the_typing_and_tells_the_model_what_already_ran`): the port does not check that the entry the lenient run leaves keeps the renamed button's old name, because it rewrites the entry after a hand-off - `refuses $sink before reading the plaintext`: the port has no plaintext resolver to count reads and records no policy decisions - `never sends a secret value to the model`: runs on a scripted model, not a real one @@ -333,7 +333,7 @@ Upstream's bug bash (`skills/e2e/references/bug-bash.md`, last changed in `94e6e - The config is `e2e.bugbash.config.json`. The port's config is JSON, so it cannot spread the project's config (`...base`), cannot import a `gateway(...)` model, and is a copy with the keys a bug bash changes: a target without `app.command`, `credentials`, and the three agents (`default`, `skeptic`, `fuzzer`) with `maxSteps` and `maxModelCalls` of 40, a `context`, and a `system`. An agent inherits nothing from `default`, so each repeats the model and the `context`. The credentials leave the password to `E2E_USER__PASSWORD`. - There is no `tests` key. The repro tests live in the test project's `BugBash` folder, tagged `bugbash` (`[Category("bugbash")]` or `[Trait("TestCategory", "bugbash")]`), and `dotnet test --filter "TestCategory!=bugbash"` replaces `e2e run --exclude-tag bugbash`. A repro test runs with `dotnet test --filter "FullyQualifiedName~BugBash."`. -- `e2e explore --output ` puts `report.json` beside ``, so the fan-out passes `--output .e2e/bugbash//results`. `--reporter` exits 2, so there is no `summary.md`; the log holds the `Findings`. `--session` exits 2: a charter that needs an account names its credential and the explorer signs in with `fill_secret`. +- `e2e explore --output ` puts `report.json` beside ``, so the fan-out passes `--output .e2e/bugbash//results`. `--reporter` exits 2, so there is no `summary.md`; the log holds the `Findings`. `--session` exits 2: a charter that needs an account names its credential and the explorer signs in with `type_secret`. - The docs page drops upstream's demo video, `ShowMore`, and `CodeGroup` blocks (one package manager, `dotnet`), and the card links to pages the docs site does not have. `e2e list` is not ported, so no command checks `e2e.bugbash.config.json`. ## Coding agents @@ -503,15 +503,15 @@ Not ported: upstream's `describe('explicit fixture operations')` in `packages/e2 `AcpExecutor` (the `E2E.Acp` package, namespace `E2E.Acp`) ports `@e2e-dev/acp` (tester-army/e2e#960, `0410fc9`): an `IStepExecutor` that runs `agent.act` and `agent.assert` steps on a coding agent that speaks the Agent Client Protocol on stdio. It starts the agent, opens one session per attempt, and serves the built-in action tools and the verdict tool to it as an MCP server on `127.0.0.1`. Every action goes through `ctx.Actions`, so the runner authorizes, counts, and records it, and cached steps replay as the built-in agent's do. -- Upstream's `acp({ command, args, ... })` is `AcpExecutor.Create(command, args, options)`; `claudeCode(options)` and `codex(options)` are `AcpExecutor.ClaudeCode(options)` and `AcpExecutor.Codex(options)` (`presets.ts` is the internal `Presets`). `AcpAgentOptions` (`Env`, `Model`, `System`, `Name`) and `AcpOptions` (adds `Mode` and `SessionMeta`) are records. The step is `done` with an `errorCode`, as the port's verdict tool, not upstream's `complete_step`. The tools are the port's `AgentTools.CreateGrammarTools`, so the agent gets `fill` and `fill_secret`, not `type` and `type_secret`. -- Upstream uses the ACP and MCP TypeScript SDKs. There is no .NET ACP SDK, so `E2E.Acp` has an internal JSON-RPC peer over newline-delimited JSON (`JsonRpcConnection`), an internal ACP session (`AcpSession`, `AgentLaunch`), and an internal MCP server over `HttpListener` (`McpToolServer`, stateless Streamable HTTP, a random path segment, a request with an `Origin` header is refused). `StepTools` builds the served tools from upstream's `tools.ts`; its per-step gate runs batched action calls one at a time, so a call queued behind `done` is skipped. These are .NET-only. +- Upstream's `acp({ command, args, ... })` is `AcpExecutor.Create(command, args, options)`; `claudeCode(options)` and `codex(options)` are `AcpExecutor.ClaudeCode(options)` and `AcpExecutor.Codex(options)` (`presets.ts` is the internal `Presets`). `AcpAgentOptions` (`Env`, `Model`, `System`, `Name`) and `AcpOptions` (adds `Mode` and `SessionMeta`) are records. The step is `complete_step` with an `errorCode`, as upstream's. The tools are the port's `AgentTools.CreateGrammarTools`, so the agent gets `type` and `type_secret`. +- Upstream uses the ACP and MCP TypeScript SDKs. There is no .NET ACP SDK, so `E2E.Acp` has an internal JSON-RPC peer over newline-delimited JSON (`JsonRpcConnection`), an internal ACP session (`AcpSession`, `AgentLaunch`), and an internal MCP server over `HttpListener` (`McpToolServer`, stateless Streamable HTTP, a random path segment, a request with an `Origin` header is refused). `StepTools` builds the served tools from upstream's `tools.ts`; its per-step gate runs batched action calls one at a time, so a call queued behind `complete_step` is skipped. These are .NET-only. - Permission requests fail closed: an unknown request is answered `cancelled`. A tool of the agent's own (read from the adapter's marks and the tool kind, never from the agent's text) ends the step with `POLICY_DENIED`. - `InternalsVisibleTo E2E.Acp` on `E2E` reaches `ctx.Host.Scope.Redactor` and `ReplayHandOffReasons`, which upstream's package imports from `e2e`. .NET-only. - Node is required for the presets. Like upstream, they run the adapter from the install they find in `node_modules`, walking up from the working directory (the `bin` of its `package.json`, with a link resolved at the package folder), never `npx `. Upstream starts it with the running Node (`process.execPath`); the port runs `node` from the `PATH`, since a .NET process has no Node of its own. The adapters are npm packages, so a .NET project needs a `package.json` with `npm install -D @agentclientprotocol/claude-agent-acp` or `@agentclientprotocol/codex-acp`. The Codex preset lists your MCP servers with `codex mcp list --json` (30 s limit, process tree killed on a timeout or cancel) from the `@openai/codex` the adapter depends on, or from `CODEX_PATH`. A pnpm-style link in a folder above the package, not at the package, is not resolved. - .NET-only in `Presets.cs`: `CodexListTimeout` (upstream's inline `30_000`), `ResolveCodexBin` (upstream's inline `createRequire(...).resolve`), `RunAsync` (upstream's `execFileAsync` with `timeout`; it also kills the process tree on a timeout or cancel, as the repo rules require), and the local `StartAsync` (upstream's `.then` chain). The Codex list is shared by every caller, and each caller waits on it with its own cancellation token (upstream has no token). The error for a failed list adds the process's stderr. - The agent's environment has no variable named `E2E_SECRET_*` or `E2E_USER_*`, in any casing, from this process or from `Env`; the `codex mcp list` call gets the same environment. Secrets in any other variable reach the agent, as upstream warns. - On Windows, a command for `AcpExecutor.Create` must be an executable, not a `.cmd` shim: nothing goes through a shell. -- The tool names the agent sees differ from upstream: `fill` and `fill_secret` for `type` and `type_secret`, `done` for `complete_step`, with `errorCode` as upstream. `fill_secret` is offered only when the step declares a secret (`AgentTools.GrammarToolDefinitions` takes the step's secrets), and its description lists each as `"name" (purpose)` (upstream's `type_secret` in `createGrammarTools`, `agent/primitives.ts`). It takes the secret as `secret`, where upstream's takes `name`: the port's target is `role`, `name`, `testId`, or `ref`, so `name` is taken (#1642 follow-up). +- The agent's tool names are upstream's (`type`, `type_secret`, `check` with `checked`, `complete_step`; #1630). `type_secret` is offered only when the step declares a secret (`AgentTools.GrammarToolDefinitions` takes the step's secrets), and its description lists each as `"name" (purpose)` (upstream's `type_secret` in `createGrammarTools`, `agent/primitives.ts`). `type_secret` takes `target` and `name`, and the other tools address a node by `target`, as upstream (#1649, #1650). The ACP session lists `type_secret` for every step, as upstream's `listing` does, through the .NET-only `GrammarToolsOptions.Secrets` (the port has no `Object.create` of a context); a step that declares no secrets is told so. - Upstream moved the ACP guide into `subscriptions.mdx` (#1488), a page the port has no counterpart for. The guide is `docs-site/src/content/docs/features/acp.mdx`. - `tests/E2E.Acp.Tests` ports `packages/acp/tests/executor.test.ts` into `Executor/`, all 31 of its `it`s (the secret-stripping `it` included), against `tests/E2E.Acp.Tests.Agent`, a scripted fake agent (a console app, so no Node). Four tests are `Partial_`: the serial-group test (the port's attempt memory is per attempt, not shared by serial members), the read-only assertion test (the document engine has no `hover`, so it uses `scroll`), the tool-list-once test (no step lacks a tool), and the attempt-ends-while-starting test. The runtime-error test is `Partial_` too: the port's action budget answers the agent in text, so the error comes from an `OnAction` hook, and the runner does not count the cancelled turn as a model call. - `tests/E2E.Acp.Tests/Presets/AgentPresetsTests.cs` ports the three `it`s of `presets.test.ts`. The adapter in its tests is a Node script that starts the scripted agent, so the tests need Node on the `PATH`, as the presets do. The tests change the working directory and the environment, so they run in one serial collection (`WorkingDirectoryCollection`, .NET-only). @@ -621,11 +621,11 @@ The catalog is what the testing agent's tools do, so a coding agent acts as the Differences: - Node ids and diff results (tester-army/e2e#231, `b9f032b`, added `ScreenPresenter`; the session has shared it since #181, `afde198`): `observe` lists `#id role "name" ...` lines under `Current screen (revision b3, path /, 12 nodes):`, and an action answers with its lead, the test line, and what changed since the screen the agent last received (`Screen changes since revision b3 (now revision b4, 12 nodes): 1 added, 1 changed.`, then `added`, `changed … (was: …)`, `removed` lines), as upstream's `ScreenPresenter` (`agent/screen-update.ts`) does. `ScreenPresenter` is `Agent/ScreenUpdate.cs`: the first screen whole, a later one as the difference keyed by node id, focus left out of the comparison, the whole screen again past 60 changed lines or half of it, no removals against a truncated listing, and the note when an action closed the on-screen keyboard. An unchanged screen reads `Screen unchanged since revision …`, or `No listed node changed within the wait after this action …` after an action that should have changed it (`expectChange` is false for `scroll`, `scroll_to`, `type_secret`, and a `press` that only moves focus, as upstream). A failed action reports the changes after its `tap n9 failed: LOCATOR_NOT_FOUND: node n9 is not on the current screen; re-observe for the newest ids` lead. The built-in agent uses it too (#1022): `DefaultAgent.PresenterFor` makes one presenter per step, shared by the opening prompt (`Open`) and the tools (`Present`), with the pixel half (`Attach`, `PixelEvidence`, the screenshot the model holds). Transcript compaction and the overflow shrink find a whole screen by `Current screen (revision `, as upstream's `FULL_SCREEN_PATTERN`. Differences: the session has no screenshots on `act` of its own, so its presenter has no pixel half (the "No screenshot can be taken" text, the unchanged-screen cases that compare images); the lines are `FailureEvidence.Listing`'s (`SnapshotText.FormatNode`: states each in its own bracket, a secure field as `secure`), and the listing is cut at the default observation limit (262144 bytes), where upstream cuts at the agent's `maxObservationBytes`. A custom executor's `ExecutorObservation.Text` is the same lines (see Custom executors (#1022) below). Node ids are the engine's refs (`SemanticNode.Ref`), unchanged. Custom executors (#1022): `ExecutorObservation.Text`, which an executor reads from `ObserveAsync` and `Start`, is the `#id role "name" ...` listing, one node per line, cut at the observation byte budget (see [Agents](#agents)), as upstream's `text`; `ExecutorTarget` takes the id with or without the `#`. A custom executor that parsed the earlier `Screen (route):` text with `[ref=…]` lines breaks and reads the id after `#` now. The listing has no `Screen (` heading: the route is `ExecutorObservation.Path`. The presenter (the `Current screen` and diff results) stays the built-in agent's and the session's, as upstream's `screen-update.ts` does; a custom executor gets no diff results. .NET-only: the judgment path (`assert`, `waitFor`, `extract`) prints a `Screen (route):` line before the same listing, where upstream wraps its text in ``; `OverflowShrink.FullScreenPattern` still matches that line. -- The grammar is the port's: `type` and `type_secret` take `text` and `secret`; `uncheck` and `clear` are tools of their own. Upstream's point tools (`tap_at`, `hover_at`, `drag_at`, `type_at`, `press_at`, `select_at`) are served as upstream's grammar offers them, from `AgentTools.CreateGrammarTools`: the session's `screenshot` notes the shot and the viewport on the session's `ScreenPresenter` (`Attach`), which the point tools read their points against (before a screenshot they answer with the line that says to take one). On an engine with a keyboard `type` takes an optional target and `replace`, and a `type` with no target has no test line (`the value went to the focused field`). `long_press`, `right_click`, `hover`, and `drag` are, and a coding agent over MCP gets each one only where the engine declares its action kind (`longPress`, `secondaryTap`, `hover`, `dragTo`), as `act` does. `dismiss_keyboard` is a tool only where the engine can hide its keyboard (`IEngineSession.CanDismissKeyboard`: the mobile engine). +- The grammar is the port's: `type` takes `text`, and `type_secret` takes `target` and `name`; `uncheck` and `clear` are tools of the MCP catalog only (the agent has none, as upstream). Upstream's point tools (`tap_at`, `hover_at`, `drag_at`, `type_at`, `press_at`, `select_at`) are served as upstream's grammar offers them, from `AgentTools.CreateGrammarTools`: the session's `screenshot` notes the shot and the viewport on the session's `ScreenPresenter` (`Attach`), which the point tools read their points against (before a screenshot they answer with the line that says to take one). On an engine with a keyboard `type` takes an optional target and `replace`, and a `type` with no target has no test line (`the value went to the focused field`). `long_press`, `right_click`, `hover`, and `drag` are, and a coding agent over MCP gets each one only where the engine declares its action kind (`longPress`, `secondaryTap`, `hover`, `dragTo`), as `act` does. `dismiss_keyboard` is a tool only where the engine can hide its keyboard (`IEngineSession.CanDismissKeyboard`: the mobile engine). - `locate` writes the `Screen` call a test uses (`Screen.GetByRole("button", "Save")`) and upstream's codes `LOCATOR_NOT_FOUND` and `LOCATOR_AMBIGUOUS`. - As upstream, a `locate` with a `status`, `alert`, or `listitem` name that matches nothing points at `Screen.GetByRole(role).Filter(text)` and `Screen.GetByText` (spelled as `GetByRole(role).filter({ hasText })` upstream), and an `exact: false` query is not told to loosen the match again (upstream #1147, `5e9597c`): the advice says "an" before a role that starts with a vowel (`an alert`, `a status`), and for an `exact: false` query it keeps the option in the text call, `Screen.GetByText("Saved.", exact: false)` (`getByText(q, { exact: false })` upstream). The web engine names those three roles from their content, so there the name also matches; the advice still holds, since the text matches either way. - Project tools (#1272): `agents..tools` declares command tools (`CommandTool`, `E2E/Agent/CommandTool.cs`, .NET-only: upstream's config holds `defineTool` calls, and a JSON file has no code). An entry has `description`, `mutates`, `command` (the program and its arguments, started with `ArgumentList`), and optionally `inputSchema` and `platforms`. The command runs in the project root with the input as JSON on stdin; stdout is the answer, stderr is printed line by line (on the server it reaches the operator's stderr redacted), a non-zero exit fails the call (`TOOL_FAILED`), and a command still running after two minutes is killed with its children. The session catalog serves each tool after the built-in ones (`SessionCatalog.Create` takes the agent's `Tools`, the target's platform, and a warning sink, as upstream's `createSessionCatalog` takes `tools`); a read-only one joins the read-only set, a tool whose platforms leave the target out is skipped, and a name a built-in tool holds is not served and logs `project tool "x" is not served over MCP: the name belongs to a built-in session tool`. The same tools reach `AgentConfig.CreateOptions().Tools`, so an act offers them. Still not ported: a config that runs code when it loads, so `mcp-user-output.test.ts` stays partial: `E2EMcpRedactsWhatUserCodePrintsTests` has `Partial_Withholds_what_a_config_printed_before_it_failed_to_load` (the output layer in process) and `Partial_Keeps_the_protocol_working_and_redacts_the_config_and_a_project_tool_on_stdout_and_stderr` (spawns the server; the tool prints on stderr, and the config's top-level print, the tool's stdout and the stray rejection are not checked). The in-process test `Redacts_what_a_config_and_a_tool_print_and_logs_an_uncaught_error_through_the_output_layer` is .NET-only and keeps those three checks on `McpOutput`. -- Test lines (upstream #1317, `9ceb071`): every action answers with the `Screen` or `App` call that repeats it, after the first line of its result, and the read-only `test_lines` tool returns them as a draft test body. The code is C# (`await Screen.GetByRole("button", "Save").TapAsync();`, `Secrets.Get(name)`, `Filter(text)`, `Nth(i)`, `new RoleOptions { Visible = true }`, `App.OpenAsync(path)`), and `SessionTestLines` (`Mcp/TestLines.cs`) is upstream's `SessionTestLines` (`mcp/test-lines.ts`). Differences: the port's tree carries a node's test id, so `GetByTestId` is the last query tried for a node, where upstream reads the id off the live node; the port resolves each locator through the engine's `LocateAsync` (2 s each, binding only the matches, so a device keeps the agent's node ids valid) when the engine has one, else on one fresh look at the screen per node with `LocatorResolver`, where upstream spends one engine resolve on each (the cap of 10 tries stays); the live look of an engine with no locate comes in the port's resolver order, so the twin of a shadow root is told apart by its place in that order and never by the box it kept (the web engine's is a locate, in Playwright's order: the light DOM first, then a closed root, as upstream's); there are no credentials, so a fill names `Secrets.Get(name)` only; a `type` that fell back to the keyboard does not exist, because the port's `type` always fills; a locator tap takes no `modifiers`, so a `tap` or `double_tap` with `modifiers` prints `No test line:`; a `press` has no `times`. A web locator is Playwright's, which names no list item from its content, so a row is found by `Filter(text)`, as upstream's. The draft ends with a line that tells the agent to add an `Expect` after each step, where upstream also names the `credentials` and `secrets` imports. The session catalog reads the newest observation from the agent's observation feed, where upstream wraps the step context. Not ported with it: the reserved tool name (the port has no project tools), the telemetry event of an MCP session. The drafted lines run as a test class of a project built in a temporary directory with `dotnet test` (`DraftedTestRun`), where upstream runs them with its `runProject`. +- Test lines (upstream #1317, `9ceb071`): every action answers with the `Screen` or `App` call that repeats it, after the first line of its result, and the read-only `test_lines` tool returns them as a draft test body. The code is C# (`await Screen.GetByRole("button", "Save").TapAsync();`, `Secrets.Get(name)`, `Filter(text)`, `Nth(i)`, `new RoleOptions { Visible = true }`, `App.OpenAsync(path)`), and `SessionTestLines` (`Mcp/TestLines.cs`) is upstream's `SessionTestLines` (`mcp/test-lines.ts`). Differences: the port's tree carries a node's test id, so `GetByTestId` is the last query tried for a node, where upstream reads the id off the live node; the port resolves each locator through the engine's `LocateAsync` (2 s each, binding only the matches, so a device keeps the agent's node ids valid) when the engine has one, else on one fresh look at the screen per node with `LocatorResolver`, where upstream spends one engine resolve on each (the cap of 10 tries stays); the live look of an engine with no locate comes in the port's resolver order, so the twin of a shadow root is told apart by its place in that order and never by the box it kept (the web engine's is a locate, in Playwright's order: the light DOM first, then a closed root, as upstream's); a typed credential username reads `Credentials.User(name).Username` and a credential's password secret reads `Credentials.User(name).Password` (as upstream's `TestLineFills.credentials`), and any other secret `Secrets.Get(name)`; a `type` with no target (the keyboard) has no test line (`the value went to the focused field`); a locator tap takes no `modifiers`, so a `tap` or `double_tap` with `modifiers` prints `No test line:`; a `press` has no `times`. A web locator is Playwright's, which names no list item from its content, so a row is found by `Filter(text)`, as upstream's. The draft ends with a line that tells the agent to add an `Expect` after each step, where upstream also names the `credentials` and `secrets` imports. The session catalog reads the newest observation from the agent's observation feed, where upstream wraps the step context. Not ported with it: the reserved tool name (the port has no project tools), the telemetry event of an MCP session. The drafted lines run as a test class of a project built in a temporary directory with `dotnet test` (`DraftedTestRun`), where upstream runs them with its `runProject`. - The fixed tools check their arguments with a strict check of the JSON Schema subset tool schemas use, since the .NET MCP SDK does not validate them. The messages are zod's, so an unknown argument reads `Invalid arguments for tool : Unrecognized key: ""`, as upstream. - `run_code` (upstream #1316, `6b41857`) is not ported. It parses and runs TypeScript test statements (`mcp/code-parse.ts`, `code-surface.ts`, `run-code.ts`) with a surface that matches the test API. C# cannot be evaluated without a compiler or scripting host, so the tool, its `LEFT_OUT` reasons, and the `mcp-code-parse` tests have no counterpart. The agent writes the `Test line:` an action printed into the test and runs it with `dotnet test`. - `run_tests` (upstream #1520) is not ported. Upstream's fifth fixed tool runs `e2e run` in its own process and returns the command, the exit code and its meaning, and what the run printed; it takes `files` (files, directories, globs, or `file:line`; without it, every test the config discovers) and the `e2e run` flags (`config`, `target`, `grep`, `grepInvert`, `tag`, `excludeTag`, `lastFailed`, `changedSince`, `headed`, `workers`, `retries`, `repeatEach`, `maxFailures`, `actionTimeout`, `noCache`, `updateSnapshots`), and fails `RUNNER_BUSY` while a run is in progress (`mcp/run-tests.ts`, `mcp/test-outcome.ts`, `mcp/output.ts`, and the "Run tests next to a session" section of `docs/mcp-server.mdx`). The port has no `e2e run`: a suite is run by `dotnet test`, which the agent runs in its own shell, and a `dotnet test` wrapper would be a tool upstream does not have (documented, not built). The upstream tests #1520 added or changed have no counterpart. `tests/integration/mcp-run-tests.test.ts`, `describe('e2e mcp run_tests')`: "lists run_tests beside the session tools", "runs one file as e2e run does, with progress per line", "runs one test by file:line", "runs the whole suite and reports a failure as the run result, with the secret redacted", "keeps the end of an output too long for one result, and bounds each progress line", "returns a selection that matches nothing as an error result", "refuses a file that reads as a flag, an empty target list, and raw flags", "runs beside an open session and leaves it open". `tests/unit/mcp-run-tests.test.ts`, `describe('runArguments')`: "runs every test with no arguments", "puts each value in its option word and the files after --", "takes the server's config, target, and headed unless the call names its own". `tests/integration/mcp-server.test.ts`: "serves five fixed tools and the guide resources" (the port serves four). `tests/unit/cli-signals.test.ts`, which cover the run's interrupt: "climbs the whole ladder on its own once the parent closes its IPC channel", "climbs at once when the parent was gone before it armed". @@ -669,10 +669,10 @@ Not ported: - `providerOptions` is a dictionary of JSON objects by provider. It rides every `ModelRequest`, and `OpenAiCompatibleModel` adds the fields under its `Provider` key (`openai` by default) to the chat-completions body as given, so write the wire names (`reasoning_effort`, not `reasoningEffort`). It cannot replace `model`, `messages`, or `tools`. Every other client does the same with its own key (`anthropic`, `google`, `bedrock`, `azure`, `xai`, `openrouter`, `gateway`) and refuses its own core fields. As upstream, requests to OpenAI (and Azure over Responses) carry `store: false` and a prompt cache key per system prompt, and Anthropic requests carry cache breakpoints on the system prompt and the newest message; provider options win over both - `assert` and `extract` make one model call and one repair round. `waitFor` judges at once, then again only after `Interval` and on a changed screen, and every call, repairs included, counts against `MaxModelCalls`. It ends `STEP_TIMEOUT` (`waitFor timed out; last judgment: ...`) or `STEP_BUDGET_EXHAUSTED` (`waitFor exhausted its model-call budget; last judgment: ...`) - As upstream, the judge prompt says a value shown in more than one place holds only when every place agrees, unless the statement names the place or is about some item among several. The wording follows upstream, with "instruction" as "statement". The judge answers with upstream's `agent-judgment-2` object (see below) -- **Judgment grammar** (`Agent/AgentProtocol.cs`, internal; upstream's `agent/protocol.ts`). `assert` and `waitFor` judge through structured output, as upstream does: the model answers one `agent-judgment-2` object (`protocolVersion`, `explanation`, then `verdict`: `holds`, `fails`, or `inconclusive`) that `AgentProtocol.ValidateJudgmentResponse` checks. `holds` passes, `fails` is `ASSERTION_FAILED`, `inconclusive` is `ASSERTION_INCONCLUSIVE`. The judge has no `done` tool, no tool at all. An answer that is not JSON or does not match gets one repair round (a `` turn with the validation issue), then `MODEL_OUTPUT_INVALID` (`The judge did not answer with an agent-judgment-2 object: ...`). `AgentProtocol.JudgmentSchema` is upstream's `JUDGMENT_SCHEMA`, with `explanation` before `verdict`. `ModelRequest.Schema` and `SchemaName` are upstream's `ModelCall.schema` and `schemaName`, .NET-only in their place: the port's models take a `ModelRequest`, not a `ModelCall`. **Breaking for a custom `IAgentModel`**, which must honor `Schema` (return the JSON object as `ModelResponse.Content`; `Tools` is empty). The providers: chat completions send `response_format` `json_schema`, Responses `text.format`, Google `generationConfig.responseMimeType` and `responseJsonSchema`; Anthropic and Bedrock, which have no such field, offer one `json` tool and force it (the AI SDK's own fallback) unless the request thinks, when the tool is only offered. `OpenAiCompatibleModelOptions.JsonSchemaResponseFormat` (.NET-only, default on) leaves `response_format` out for a server that rejects it; the judge's prompt still asks for the object. Chat completions and Responses send `strict: false`, because strict mode refuses `maxLength`; the agent checks the answer itself. `ModelResponses.Judgment` and `ModelHttp.StructuredAnswer`/`JsonToolName` are .NET-only (a scripted judge answer; the forced-tool answer as content). Not ported: upstream's `extractSchema` and `validateExtractResponse` (the `agent-extract-2` envelope). `extract` still answers through its `extract` and `done` tools, so its inconclusive outlet is the `done` tool. -- **Closed grammar tool inputs** (`AgentTools.SchemaTool`, upstream's `schemaTool` in `agent/primitives.ts`). A grammar tool call that carries a field its schema does not declare is refused and never runs; the refusal is the call's result (`failed: Invalid input for tool tap: Type validation failed: Value: {...}.` then `Error message: [{"code":"unrecognized_keys","keys":[...],...}]`, the AI SDK's text over a zod issue), it counts toward the failure streak, and in a batched turn the other calls still run. The schema sent to the model carries `additionalProperties: false` for every grammar tool and for `done` (upstream's `schemaTool`), at the top level only (the `to` of `drag` is an object in the port, a node id in upstream, and stays open). Differences: only undeclared fields are refused by the runtime (the tools read their own values, so a wrong type is the tool's own failure); project tools (`AgentTool`) and the extract tools are not closed (upstream's `defineTool` closes nothing); `done` drops a code a passed verdict carries, as upstream's verdict tool does, instead of refusing it, and refuses an undeclared field like every grammar tool. Tests: `AgentPolicy/ToolCallsOutsideTheVocabularyTests`, `AgentGrammarTools/TheGrammarToolsHaveClosedSchemasTests`, and `AgentGrammarTools/TheSchemaAProviderReceivesTests`. +- **Judgment grammar** (`Agent/AgentProtocol.cs`, internal; upstream's `agent/protocol.ts`). `assert` and `waitFor` judge through structured output, as upstream does: the model answers one `agent-judgment-2` object (`protocolVersion`, `explanation`, then `verdict`: `holds`, `fails`, or `inconclusive`) that `AgentProtocol.ValidateJudgmentResponse` checks. `holds` passes, `fails` is `ASSERTION_FAILED`, `inconclusive` is `ASSERTION_INCONCLUSIVE`. The judge has no `complete_step` tool, no tool at all. An answer that is not JSON or does not match gets one repair round (a `` turn with the validation issue), then `MODEL_OUTPUT_INVALID` (`The judge did not answer with an agent-judgment-2 object: ...`). `AgentProtocol.JudgmentSchema` is upstream's `JUDGMENT_SCHEMA`, with `explanation` before `verdict`. `ModelRequest.Schema` and `SchemaName` are upstream's `ModelCall.schema` and `schemaName`, .NET-only in their place: the port's models take a `ModelRequest`, not a `ModelCall`. **Breaking for a custom `IAgentModel`**, which must honor `Schema` (return the JSON object as `ModelResponse.Content`; `Tools` is empty). The providers: chat completions send `response_format` `json_schema`, Responses `text.format`, Google `generationConfig.responseMimeType` and `responseJsonSchema`; Anthropic and Bedrock, which have no such field, offer one `json` tool and force it (the AI SDK's own fallback) unless the request thinks, when the tool is only offered. `OpenAiCompatibleModelOptions.JsonSchemaResponseFormat` (.NET-only, default on) leaves `response_format` out for a server that rejects it; the judge's prompt still asks for the object. Chat completions and Responses send `strict: false`, because strict mode refuses `maxLength`; the agent checks the answer itself. `ModelResponses.Judgment` and `ModelHttp.StructuredAnswer`/`JsonToolName` are .NET-only (a scripted judge answer; the forced-tool answer as content). Not ported: upstream's `extractSchema` and `validateExtractResponse` (the `agent-extract-2` envelope). `extract` still answers through its `extract` and `complete_step` tools, so its inconclusive outlet is the `complete_step` tool. +- **Closed grammar tool inputs** (`AgentTools.SchemaTool`, upstream's `schemaTool` in `agent/primitives.ts`). A grammar tool call that carries a field its schema does not declare is refused and never runs; the refusal is the call's result (`failed: Invalid input for tool tap: Type validation failed: Value: {...}.` then `Error message: [{"code":"unrecognized_keys","keys":[...],...}]`, the AI SDK's text over a zod issue), it counts toward the failure streak, and in a batched turn the other calls still run. The schema sent to the model carries `additionalProperties: false` for every grammar tool and for `complete_step` (upstream's `schemaTool`), at the top level only (the `to` of `drag` is an object in the port, a node id in upstream, and stays open). Differences: only undeclared fields are refused by the runtime (the tools read their own values, so a wrong type is the tool's own failure); project tools (`AgentTool`) and the extract tools are not closed (upstream's `defineTool` closes nothing); `complete_step` drops a code a passed verdict carries, as upstream's verdict tool does, instead of refusing it, and refuses an undeclared field like every grammar tool. Tests: `AgentPolicy/ToolCallsOutsideTheVocabularyTests`, `AgentGrammarTools/TheGrammarToolsHaveClosedSchemasTests`, and `AgentGrammarTools/TheSchemaAProviderReceivesTests`. - `ExtractAsync`: the type argument is the schema, in place of a Standard Schema. The judge gets the JSON schema of `T` (`JsonSchemaExporter`), and the answer must deserialize into `T` with required members and nullable annotations respected. A failure gets one repair round with the validation error, then `MODEL_OUTPUT_INVALID` (`extracted data failed schema validation: ...`). A judge that says the data is not shown ends `ASSERTION_INCONCLUSIVE` (`nothing to extract: ...`) -- `tools` (`AgentOptions.Tools`, `AgentTools` on the default agent) are project tools the model is offered beside the agent's own. As upstream's `defineTool`, an `AgentTool` declares `mutates` and `platforms`: a tool is offered only on the platforms it names, and a mutating tool takes an action slot. Its output is redacted and bounded to 400 lines and 16 KB (16384 UTF-8 bytes), whichever is reached first: the cut never splits a line and ends with a notice that says how much was left out (`ToolOutput.BoundToolOutput`, `ToolOutput.MaxToolOutputLines`, and `ToolOutput.MaxToolOutputBytes`, upstream's `boundToolOutput`, `MAX_TOOL_OUTPUT_LINES`, and `MAX_TOOL_OUTPUT_BYTES` in `agent/tool-output.ts`, tester-army/e2e at `f185ca8`). A first line longer than the whole byte budget keeps its head. The port redacts the result before it bounds it, so a cut cannot split a secret in two; upstream bounds first. Earlier versions of this port cut at 16384 characters with `[output cut]`. A name the agent already uses (`observe`, `done`, `navigate`, `back`, `scroll`, `scroll_to`, or a verb) is refused with `INVALID_CONFIG`. Differences: the tool is an `AgentTool` built with its schema as a `JsonObject` and a delegate, since the port has no `defineTool`, AI SDK tool, or `toModelOutput`; a tool that wants upstream's `getToolContext` takes the second constructor, whose delegate gets an `AgentToolContext` (`ObserveAsync`, `AttachScreenshotAsync`), and the first constructor keeps working; and upstream's reserved names for tools it adds elsewhere are reserved as upstream's `HARNESS_TOOL_NAMES` has them (`AgentTools.HarnessToolNames`): `complete_step` (every agent step), `locate`, `run_code`, `test_lines`, `start_recording`, `stop_recording` (an e2e mcp session), and `report_finding` (explore). A project tool of one of those names is `INVALID_CONFIG`. The port's verdict tool is `done`, which is reserved with the grammar's names, and `run_code` has no port. Upstream records a mutating tool as a gap in the trace, and so does the port: a recording holds an action of kind `tool` with the tool name, a replay runs the actions before it and then hands the step to the model, and the step is never replayed whole. Older versions of this port cannot read that action and run the step live. As upstream, a recording that stops at such a gap (reason `gap`) is not stale: a strict cache run replays what it can and runs the rest of the step live. The recording keeps the tool's name only, never its arguments +- `tools` (`AgentOptions.Tools`, `AgentTools` on the default agent) are project tools the model is offered beside the agent's own. As upstream's `defineTool`, an `AgentTool` declares `mutates` and `platforms`: a tool is offered only on the platforms it names, and a mutating tool takes an action slot. Its output is redacted and bounded to 400 lines and 16 KB (16384 UTF-8 bytes), whichever is reached first: the cut never splits a line and ends with a notice that says how much was left out (`ToolOutput.BoundToolOutput`, `ToolOutput.MaxToolOutputLines`, and `ToolOutput.MaxToolOutputBytes`, upstream's `boundToolOutput`, `MAX_TOOL_OUTPUT_LINES`, and `MAX_TOOL_OUTPUT_BYTES` in `agent/tool-output.ts`, tester-army/e2e at `f185ca8`). A first line longer than the whole byte budget keeps its head. The port redacts the result before it bounds it, so a cut cannot split a secret in two; upstream bounds first. Earlier versions of this port cut at 16384 characters with `[output cut]`. A name the agent already uses (`observe`, `complete_step`, `navigate`, `back`, `scroll`, `scroll_to`, or a verb) is refused with `INVALID_CONFIG`. Differences: the tool is an `AgentTool` built with its schema as a `JsonObject` and a delegate, since the port has no `defineTool`, AI SDK tool, or `toModelOutput`; a tool that wants upstream's `getToolContext` takes the second constructor, whose delegate gets an `AgentToolContext` (`ObserveAsync`, `AttachScreenshotAsync`), and the first constructor keeps working; and upstream's reserved names for tools it adds elsewhere are reserved as upstream's `HARNESS_TOOL_NAMES` has them (`AgentTools.HarnessToolNames`): `complete_step` (every agent step), `locate`, `run_code`, `test_lines`, `start_recording`, `stop_recording` (an e2e mcp session), and `report_finding` (explore). A project tool of one of those names is `INVALID_CONFIG`. `complete_step` is reserved as upstream's `HARNESS_TOOL_NAMES` has it, and `run_code` has no port. Upstream records a mutating tool as a gap in the trace, and so does the port: a recording holds an action of kind `tool` with the tool name, a replay runs the actions before it and then hands the step to the model, and the step is never replayed whole. Older versions of this port cannot read that action and run the step live. As upstream, a recording that stops at such a gap (reason `gap`) is not stale: a strict cache run replays what it can and runs the rest of the step live. The recording keeps the tool's name only, never its arguments - `tap`, `double_tap`, and `right_click` take an optional `modifiers` list (`Shift`, `Control`, `Alt`, `Meta`, `ControlOrMeta`) on an engine that declares `EngineCapabilities.TapModifiers`, as upstream's `tap`, `double_tap`, and `right_click` do on an engine that declares `tapModifiers`. The web engine declares it. The device engines and the document engine do not, and refuse the field with `UNSUPPORTED_CAPABILITY`. The held keys are recorded as `modifiers` of the `tap`, `doubleTap`, or `secondaryTap` action and replayed from the cache. An entry whose `modifiers` is empty, repeats a key, or sits on another action is a miss (`invalid-entry`). The grammar tools close their schemas (see Closed grammar tool inputs), so a tool called with a field its engine does not declare is refused by the schema as `unrecognized_keys`, as upstream's closed schema does, and `UNSUPPORTED_CAPABILITY` is left to a field the schema declares. - A typed value the run derived is recorded as a gap, not replayed (upstream's `derivedReason`, `agent/derived.ts`; `Derived.DerivedReason`, `Agent/Derived.cs`). `ActionDispatcher` checks the value of a `TypeAsync` fill and of a `TypeTextAsync` against the step's instruction, its params (as the executor reads them, a `Unique` as its value), and what the step's screens showed (`ObservationFeed.ShownText`, upstream's `shownText`, kept redacted as upstream keeps the prepared capture; `PixelsShown` says a screenshot went to the model). A value the instruction or params spell out replays. Otherwise the first rule that fits names the gap's `RecordedAction.Derived` (`DerivedReason`, upstream's type of that name in `cache/trace.ts`, spelled `pixels`, `whole-node`, `minted-token`, or `date` in the entry): `Pixels` once the model was shown a screenshot, `WholeNode` when a node's name or text is exactly the value, `MintedToken` when a one-token value with a digit appears as a word of its own in shown text, `Date` for a date or clock time. `TraceRecorder.RecordDerivedGap` is upstream's `recordDerivedGap`: the gap is a `tool` action whose text is `type (run-time value)`, and a replay stops at it like any project-tool gap (reason `gap`). A value the model composed itself replays. A replay types the recording's own values, so it never records a gap (the port commits a replay through `ActReplayHost`, so upstream's `replayingTrace` check has no counterpart). `E2ESession`'s check for an unchanged flow ignores `derived`, as upstream's `flowOf` does. The MCP session tool path (`Agent.RunToolAsync`) builds its dispatcher with no instruction or params and records no trace, as upstream's interactive step records none. Tests: `AgentDerived/DerivedReasonTests` (the ten `it`s of `agent-derived.test.ts`) and, .NET-only, `AgentDerived/DerivedReasonWireTests` (the enum's wire spelling) and `AgentObservation/ObservationFeedTextTests` (the feed's redacted shown text, .NET-only). A replay that stops at the gap reports the rule: `ReplayOutcome.Derived` (upstream's `derived`, set from the gap's `RecordedAction.Derived`), `CacheInfo.Derived` (upstream's `StepCacheInfo.derived`, set on a hand-off or a miss that stopped at the gap, by `StepCache.HandOff` and `Missed`), and `derived` in the step's `cache` of `report.json` (`ReportWire`; the schema and fixture already hold it). Tests: the two `it`s of `trace-replay.test.ts` that assert `derived` (`ReplayTraceTests`), and the `step-cache.test.ts` `it` "leaves an entry untouched when only the rule that flagged a typed value differs" (`FlushPipelineTests`); `schema-fixtures.test.ts`'s `it` was ported earlier (`ReportV1SchemaTests`). - `act` also offers a `double_tap` tool, which upstream does not have. Without it, the agent cannot open a control that reacts only to a double-click, such as a TodoMVC label that opens its editor. It runs the locator `doubleTap`, and the replay cache records it as `doubleTap`. Upstream and older versions of this port cannot replay that entry, so they run the step live @@ -684,16 +684,16 @@ Not ported: - `executor` (`AgentOptions.Executor`, `AgentExecutor` on `E2ESessionOptions` and the fixtures) replaces the built-in agent for `act` and `assert`. The socket is `IStepExecutor`: `Name`, and `Version`, `Cache`, `Model`, and `Judge` with defaults, so a hand-rolled executor writes `Name` and `RunStepAsync(context, cancellationToken)`. The token is upstream's `signal`; the context's `Attempt.CancellationToken` is upstream's `attempt.signal`. `StepExecutorContext` is a class a test can build and override: `Step`, `Attempt`, `Target`, `ReplayedPrefix`, `Model`, `ProviderOptions`, `Ledger`, `AgentContext`, `Budgets`, `Actions`, `ObserveAsync`, `AttachTranscript`, and `AttachScreenshotAsync`. An executor is code, so `agents..executor` in `e2e.config.json` stays `INVALID_CONFIG` and names `AgentOptions.Executor`. As upstream, `System` and `Tools` are refused beside an executor, and the `Model` and `Judge` an executor brought are the agent's, for the judgments too. An entry that names another is `INVALID_CONFIG` (`agents.default.model (gpt-5.4) and the executor's own model (gpt-5.4-mini) differ; configure the model in one place`). Two models are the same when they are one instance or share a `Name`, where upstream compares the provider and the id - The `Ledger` text an act reads is upstream's `serializeLedger` over the attempt's step records: `1. app.open passed :: /` per completed step, then ` observed: ` when it has one (` recorded in an earlier run: ` for a step the cache replayed whole, upstream's `PriorStep.replayed`, #1434), up to 8192 bytes, newest kept, with `[n earlier step(s) omitted]` first. A failed step is in it, with its status. Each label and handoff is stripped of ANSI sequences and control characters and cut to 256 and 700 bytes. A step is recorded for `agent.act`, `agent.assert`, `agent.waitFor`, `agent.extract`, a locator action and `locator.waitFor`, an `expect` matcher on a locator, the browser, or the screen, and `app.open`, `app.back`, `app.restart`, and `app.clearState`. The `Browser` and `Device` fixture calls are steps (see App and browser fixtures); `expect.poll` is not a step yet. The records are internal, and the report writes them as the attempt's `steps`. A replayed act answers upstream's `replayed N recorded action(s) zero-turn from the trace cache; recorded verdict: ...` and hands later steps the recorded verdict alone (`StepTraceSession.RecordedVerdictOf`, upstream's `recordedVerdictOf`), so the line reads ` recorded in an earlier run: ` -- The tool loop's turn policy is upstream's (`agent/tool-loop.ts`): with five turns left (`WindDownTurns`) it adds a wind-down notice, and with two or fewer (`ForcedConclusionTurns`), after a loop-guard stop, or when the remaining step time falls under a quarter of the step's budget, capped at 60 s (`ClockWindDownMs`, `ClockWindDownFraction`, never on the first turn), it offers only `done` and, on a low clock, asks for a verdict. A call to another tool on such a turn is answered ` did not run: only done is available now. Call done with your verdict.` and costs the turn. `ToolLoopExecutor.ToolCallsOnlyRule` (upstream's `TOOL_CALLS_ONLY_RULE`) is in the instructions from the first turn, and `TextReplyNotice` (`TEXT_REPLY_NOTICE`) answers a reply with no tool call. A step that runs out of turns returns a failed verdict with `STEP_NO_CONCLUSION` (`the agent used N of M turn(s) without calling done`), where the port used to throw `STEP_BUDGET_EXHAUSTED`; the model-call budget still throws it for an executor that records past `MaxModelCalls`, and the action budget (`MaxSteps`) is unchanged. Differences: the tool is `done`, not `complete_step` (deliberate, see the tool names above), so the notices and the rule name `done`. The loop sends `ModelRequest.ToolChoice` (`ModelToolChoice`, the AI SDK's `ToolChoice`; port-only name for the SDK type): the loop's mode (`toolChoice`) decides it, as upstream's `prepareStep` returns `toolChoice: this.toolChoice`, or `{ type: 'tool', toolName: 'complete_step' }` on a forced turn when the mode is `required`. The loop starts in `required`, as upstream does. The wire forms are the AI SDK's (Anthropic `any` or `{ type: 'tool', name }`, OpenAI `required` or `{ type: 'function', ... }`, Gemini `ANY` with `allowedFunctionNames`, Bedrock `any` or `{ tool: { name } }`); a request with no choice still sends `auto`. A provider option can no longer replace `tool_choice` (Anthropic, chat completions, Responses) or `toolConfig` (Google; Bedrock already reserved it): it fails with `INVALID_CONFIG`, like the other fields the client sets (port-only guard, since the port builds the body by hand). The AiTrace records the choice as asked (`{ type: 'required' }`). `ModelResponse.Warnings` (`ModelWarning`, upstream's `CallWarning`) carries what a provider changed: `AnthropicModel` and `BedrockModel` send `auto` and warn `unsupported`/`toolChoice` for a forced choice while the model thinks (both providers refuse a forced tool then; the AI SDK's provider decides this per model). A model that refuses a forced choice is asked again with `auto` (`ToolChoice.IsForcedToolChoiceRejected`, upstream's `isForcedToolChoiceRejected` in `agent/model/tool-choice.ts`: an HTTP 4xx whose message matches one of upstream's patterns, such as Claude Fable 5.1's `tool_choice: type "tool" and "any" are not supported for this model`, DeepSeek's thinking mode, or Meta Muse Spark's auto-only message, read through the exception chain), on the same turn budget and the same instructions (`FreeToolChoiceRetry`, upstream's `freeToolChoiceRetry`); the refused request is not a model call. So is a forced turn that came back as a finished reply without a tool call (`IsForcedToolCallSkipped`, upstream's `isForcedToolCallSkipped`; the port reads it off the response through `ToolChoiceViolationException`, where the AI SDK throws `AI_ToolChoiceViolationError`), and a provider that reports a `toolChoice` warning (`IsForcedToolChoiceDowngraded`, `NoticeToolChoiceDowngrade`) continues the step with `auto`. Each of the three remembers the model (`ToolLoopExecutor.FreeToolChoiceModels`, upstream's `FREE_TOOL_CHOICE_MODELS`), so later steps on it start in `auto`. Upstream's `WeakSet` is keyed by the SDK model object, which a worker builds once and shares; the port builds a provider model per session, so a model object the caller owns (`ScriptedModel`, a custom `IAgentModel`) is held weakly by object, and a built-in provider model (any disposable `IAgentModel`) is remembered for the process by its type and model id, whatever the endpoint. Deliberate .NET-only difference. Differences: the port's failure message carries the response body (`HTTP 400: {body}`), where upstream reads `responseBody` apart, and the AI SDK's `RetryError` (`Errors`, `LastError`) is carried as the `InnerException` of the `AgentException` that `ModelHttp.SendAsync` throws once its retries are spent (#1311; the `AgentException` keeps its code, `ProviderStatus`, and message, which are the last attempt's, because it is sealed and public and callers match it), and `ToolChoice.IsForcedToolChoiceRejected` follows `LastError` as upstream follows `cause ?? lastError`; upstream's `isForcedToolCallMismatched` and `recordThrownTurn` have no counterpart, since a model returns the wrong-tool or prose reply as a response the loop already records as a turn and answers (` did not run: only done is available now`), where the SDK throws on it. The loop's notices to the model carry upstream's prefixes and wording with `done` for `complete_step` and `[SYSTEM NOTICE]` for the going-in-circles warning, the failure-streak warning and the wind-down notice, `[SYSTEM]` for the loop-guard stop and the low-clock notice, and the loop records upstream's `loop guard warned before turn N: ` and `the step was stopped before turn N: ` notes on the turn. The loop-guard stop keeps its per-turn `PrepareMessages` stop, which does not persist across turns. Tests: `ToolLoopToolChoice` (`ToolLoopForcedToolChoiceTests`, `ToolLoopDowngradedToolChoiceTests`, `ToolLoopForcedConclusionAnsweredWithAnotherToolTests` (whole: a tool call carries `ModelToolCall.ProviderMetadata`, upstream's `providerMetadata` of a tool call part, and the refused call goes back with it), `IsForcedToolChoiceRejectedTests`, `IsForcedToolCallSkippedTests`). -- `ToolLoop.CreateToolLoopExecutor` and `DefaultAgent.CreateBuiltInAgent` are public. Differences: `ToolLoopExecutorOptions.Tools` returns a list of `ToolLoopTool` (a `ModelTool` and a delegate that answers a `ToolLoopResult`), where upstream's returns an AI SDK `ToolSet`; `BuildPrompt` answers a string, where upstream's can answer a whole message history; the chassis adds `done`, where upstream adds `complete_step`; there is no `maxTurns` to remove. `ToolLoopHelpers` has `GuardAsync`, `Concluding`, and `ReportHardStop`, and the loop guards every tool call itself. The built-in agent drives the context the runtime builds, so it runs as a step's executor or inside a custom executor that passes its context on, and not on a context built by hand +- The tool loop's turn policy is upstream's (`agent/tool-loop.ts`): with five turns left (`WindDownTurns`) it adds a wind-down notice, and with two or fewer (`ForcedConclusionTurns`), after a loop-guard stop, or when the remaining step time falls under a quarter of the step's budget, capped at 60 s (`ClockWindDownMs`, `ClockWindDownFraction`, never on the first turn), it offers only `complete_step` and, on a low clock, asks for a verdict. A call to another tool on such a turn is answered ` did not run: only complete_step is available now. Call complete_step with your verdict.` and costs the turn. `ToolLoopExecutor.ToolCallsOnlyRule` (upstream's `TOOL_CALLS_ONLY_RULE`) is in the instructions from the first turn, and `TextReplyNotice` (`TEXT_REPLY_NOTICE`) answers a reply with no tool call. A step that runs out of turns returns a failed verdict with `STEP_NO_CONCLUSION` (`the agent used N of M turn(s) without calling complete_step`), where the port used to throw `STEP_BUDGET_EXHAUSTED`; the model-call budget still throws it for an executor that records past `MaxModelCalls`, and the action budget (`MaxSteps`) is unchanged. The tool is `complete_step`, as upstream's, and the notices and the rule name it. The loop sends `ModelRequest.ToolChoice` (`ModelToolChoice`, the AI SDK's `ToolChoice`; port-only name for the SDK type): the loop's mode (`toolChoice`) decides it, as upstream's `prepareStep` returns `toolChoice: this.toolChoice`, or `{ type: 'tool', toolName: 'complete_step' }` on a forced turn when the mode is `required`. The loop starts in `required`, as upstream does. The wire forms are the AI SDK's (Anthropic `any` or `{ type: 'tool', name }`, OpenAI `required` or `{ type: 'function', ... }`, Gemini `ANY` with `allowedFunctionNames`, Bedrock `any` or `{ tool: { name } }`); a request with no choice still sends `auto`. A provider option can no longer replace `tool_choice` (Anthropic, chat completions, Responses) or `toolConfig` (Google; Bedrock already reserved it): it fails with `INVALID_CONFIG`, like the other fields the client sets (port-only guard, since the port builds the body by hand). The AiTrace records the choice as asked (`{ type: 'required' }`). `ModelResponse.Warnings` (`ModelWarning`, upstream's `CallWarning`) carries what a provider changed: `AnthropicModel` and `BedrockModel` send `auto` and warn `unsupported`/`toolChoice` for a forced choice while the model thinks (both providers refuse a forced tool then; the AI SDK's provider decides this per model). A model that refuses a forced choice is asked again with `auto` (`ToolChoice.IsForcedToolChoiceRejected`, upstream's `isForcedToolChoiceRejected` in `agent/model/tool-choice.ts`: an HTTP 4xx whose message matches one of upstream's patterns, such as Claude Fable 5.1's `tool_choice: type "tool" and "any" are not supported for this model`, DeepSeek's thinking mode, or Meta Muse Spark's auto-only message, read through the exception chain), on the same turn budget and the same instructions (`FreeToolChoiceRetry`, upstream's `freeToolChoiceRetry`); the refused request is not a model call. So is a forced turn that came back as a finished reply without a tool call (`IsForcedToolCallSkipped`, upstream's `isForcedToolCallSkipped`; the port reads it off the response through `ToolChoiceViolationException`, where the AI SDK throws `AI_ToolChoiceViolationError`), and a provider that reports a `toolChoice` warning (`IsForcedToolChoiceDowngraded`, `NoticeToolChoiceDowngrade`) continues the step with `auto`. Each of the three remembers the model (`ToolLoopExecutor.FreeToolChoiceModels`, upstream's `FREE_TOOL_CHOICE_MODELS`), so later steps on it start in `auto`. Upstream's `WeakSet` is keyed by the SDK model object, which a worker builds once and shares; the port builds a provider model per session, so a model object the caller owns (`ScriptedModel`, a custom `IAgentModel`) is held weakly by object, and a built-in provider model (any disposable `IAgentModel`) is remembered for the process by its type and model id, whatever the endpoint. Deliberate .NET-only difference. Differences: the port's failure message carries the response body (`HTTP 400: {body}`), where upstream reads `responseBody` apart, and the AI SDK's `RetryError` (`Errors`, `LastError`) is carried as the `InnerException` of the `AgentException` that `ModelHttp.SendAsync` throws once its retries are spent (#1311; the `AgentException` keeps its code, `ProviderStatus`, and message, which are the last attempt's, because it is sealed and public and callers match it), and `ToolChoice.IsForcedToolChoiceRejected` follows `LastError` as upstream follows `cause ?? lastError`; upstream's `isForcedToolCallMismatched` and `recordThrownTurn` have no counterpart, since a model returns the wrong-tool or prose reply as a response the loop already records as a turn and answers (` did not run: only complete_step is available now`), where the SDK throws on it. The loop's notices to the model carry upstream's prefixes and wording: `[SYSTEM NOTICE]` for the going-in-circles warning, the failure-streak warning and the wind-down notice, `[SYSTEM]` for the loop-guard stop and the low-clock notice, and the loop records upstream's `loop guard warned before turn N: ` and `the step was stopped before turn N: ` notes on the turn. The loop-guard stop keeps its per-turn `PrepareMessages` stop, which does not persist across turns. Tests: `ToolLoopToolChoice` (`ToolLoopForcedToolChoiceTests`, `ToolLoopDowngradedToolChoiceTests`, `ToolLoopForcedConclusionAnsweredWithAnotherToolTests` (whole: a tool call carries `ModelToolCall.ProviderMetadata`, upstream's `providerMetadata` of a tool call part, and the refused call goes back with it), `IsForcedToolChoiceRejectedTests`, `IsForcedToolCallSkippedTests`). +- `ToolLoop.CreateToolLoopExecutor` and `DefaultAgent.CreateBuiltInAgent` are public. Differences: `ToolLoopExecutorOptions.Tools` returns a list of `ToolLoopTool` (a `ModelTool` and a delegate that answers a `ToolLoopResult`), where upstream's returns an AI SDK `ToolSet`; `BuildPrompt` answers a string, where upstream's can answer a whole message history; there is no `maxTurns` to remove. `ToolLoopHelpers` has `GuardAsync`, `Concluding`, and `ReportHardStop`, and the loop guards every tool call itself. The built-in agent drives the context the runtime builds, so it runs as a step's executor or inside a custom executor that passes its context on, and not on a context built by hand - `ActOptions.OnBeforeAction` is upstream's `onBeforeAction` (#1142, tester-army/e2e#1387 `4461e9d`): a hook that runs before each action, replayed ones included. `BeforeActionHook` (upstream's `BeforeActionHook`) is a delegate returning a `Task`, and `BeforeActionEvent(Index, Source)` is upstream's `BeforeActionEvent` (`Source` is the `ActionSource` enum, as for `ActionEvent`). `ActionDispatcher.BeforeAsync` is upstream's `before`, called from `RunNowAsync` (upstream's `runActionNow`) once the action's slot is taken, and from `StepAccounting.RunToolAsync` for a mutating project tool (upstream's `runTool`, which sits in the dispatcher); a read-only tool does not call it. `InvokeHookAsync` is upstream's `invokeHook`, shared with `NotifyAsync`, so a throw fails the step as `ACTION_FAILED` with `agent.act onBeforeAction failed: ...` and leaves the step's recording alone (`AgentException.FromActionHook`, upstream's `ActionHookError`). The event's `Index` is the action's slot, the one its `OnAction` event carries. Differences, all .NET-only: the port's replay takes each slot in `Replay` (`TryReserve`) rather than through the dispatcher's verbs, so the replay calls the host's `BeforeAsync` right after each successful `TryReserve` (`ReserveAsync` in `Replay.cs`; `IReplayHost.BeforeAsync` is a default member, .NET-only, which `ActReplayHost` forwards to the dispatcher with `ActionSource.Replay`, where upstream reads `accounting.replayingTrace`); `StepAccounting.Attach` takes the dispatcher's hook as `beforeTool`. A replayed upload that is refused after its slot is taken has already called the hook, where upstream authorizes the upload before it takes the slot. Upstream's check that `onBeforeAction` is a function has no counterpart, since the delegate type says it. Tests: `agent-act-before-action.test.ts` is `AgentActBeforeAction/` (6 of 6). The first is `Partial_`: the hooks record their labels in a list where upstream takes an `app.screenshot` step (the document engine keeps no screenshots), so it does not check that each hook step holds one screenshot artifact. The last is `Partial_`: it makes its refused dispatch (a tap of node `n99999`) from a hand-rolled executor, because the port's tool loop turns a target that names no control away before the dispatcher takes a slot. The `act-validation` `it` "takes an onBeforeAction function and refuses any other value" is skipped (a delegate cannot be anything but a function). The `sdk-types.ts` lines are `SdkTypes/SdkTypesTests.Partial_the_before_action_observer_sees_an_index_and_a_source_and_acts_take_it_but_assertions_do_not` (reflection checks; partial: the `onBeforeAction: true` line has no counterpart, since a delegate cannot be anything else). Known difference: the port's tool loop (`ActTools`) turns a target that names no control away before the dispatcher takes a slot, so the built-in loop does not call the hook for that refused attempt, where upstream's `tap n99999` reaches the dispatcher and gets a capture; that sits in the tool loop, not in this change. -- Upstream's `grammar-tools-export` changeset (`ffed34e`, tester-army/e2e#960) exports the built-in agent's tools. The port has `AgentTools.CreateGrammarTools(StepExecutorContext, GrammarToolsOptions?)` (upstream's `createGrammarTools(ctx, { guard })`), `AgentTools.CreateVerdictTool()` with `VerdictTool` (`Tool`, `Verdict`, `Concluded`, `Execute`), `AgentTools.VerdictRules` (`VERDICT_RULES`), `DefaultAgent.BaseRules` (`BASE_RULES`), and the existing `StepExecutors.IsRuntimeHardStop`. Differences: the grammar tools are `ToolLoopTool`s (a `ModelTool` and a delegate), where upstream's are AI SDK tools, and `AgentTools` and `VerdictTool` were internal before; `GrammarToolsOptions.Guard` takes the body as `Func>` and answers a string (`body => helpers.GuardAsync(body)` runs a tool under the loop's policy), where upstream's is generic; the built-in agent passes no guard, since the loop already ends the step on a hard stop. The `done` tool stays `done` (not `complete_step`) and takes `errorCode`, so `VerdictRules` names `done`. The built-in rules were one text; they are now `BaseRules` (the action guidance) and `VerdictRules`, and `ToolLoop` appends `VerdictRules` after the executor's guidance and the project context, as upstream's chassis appends `VERDICT_RULES`, so every loop executor, the built-in one included, sends the verdict rules last. `VerdictTool.Execute` keeps the first accepted verdict, answers `Step concluded.` or `The step already concluded.`, drops a code sent beside `passed` and says so, and refuses a blocked verdict without a blockable code; `ToolLoop` uses the same policy through a stateless internal member, so the loop's own `done` answers the same text (`Step concluded.`, or the note that a code sent beside `passed` was ignored, which names the code only when it is from the closed set, and says a code was ignored without echoing any other). A call to a tool the step does not offer is answered `failed: Model tried to call unavailable tool 'click'. Available tools: observe, tap, fill, done.` (the list is the step's own tools, so it varies with the engine and the project tools), the AI SDK's `NoSuchToolError` text that upstream's `agent-policy` and `loop-guards` tests read, with `done` for `complete_step`; the turn records it as `[click] failed: ...`. `ToolOutcome.Finish` takes the dropped code (`droppedCode`, .NET-only: upstream's tool answers the text itself, and the port's loop reads the text off the stateless outcome). `ScreenOutput` has no counterpart: the port's tool answer, `ToolLoopResult` (`Content`, `Failed`, `Pixels`), is public and carries the text and the screenshot. The screen presenter and `onActionFailed` stay internal (`ScreenPresenter`, `ActTools`). `AgentTools` members the port had made public by mistake (`MaxScrollTimes`, `ModelErrorCodes`, `IsReserved`, `Project`, `ExtractFor`, and the others) are internal now. The internal definition builder `CreateGrammarTools(EngineCapabilities, ...)` is `GrammarToolDefinitions` (.NET-only, since the public name belongs to upstream's). Tests: `SdkTypes/SdkTypesTests` is the new lines of `tests/types/sdk-types.ts` as reflection checks (the four `Partial_` ones check a shape the type checker checks upstream); `AgentGrammarTools/CreateGrammarToolsTests` is .NET-only (a loop that spreads the grammar tools beside its own, the guard, the verdict tool). +- Upstream's `grammar-tools-export` changeset (`ffed34e`, tester-army/e2e#960) exports the built-in agent's tools. The port has `AgentTools.CreateGrammarTools(StepExecutorContext, GrammarToolsOptions?)` (upstream's `createGrammarTools(ctx, { guard })`), `AgentTools.CreateVerdictTool()` with `VerdictTool` (`Tool`, `Verdict`, `Concluded`, `Execute`), `AgentTools.VerdictRules` (`VERDICT_RULES`), `DefaultAgent.BaseRules` (`BASE_RULES`), and the existing `StepExecutors.IsRuntimeHardStop`. Differences: the grammar tools are `ToolLoopTool`s (a `ModelTool` and a delegate), where upstream's are AI SDK tools, and `AgentTools` and `VerdictTool` were internal before; `GrammarToolsOptions.Guard` takes the body as `Func>` and answers a string (`body => helpers.GuardAsync(body)` runs a tool under the loop's policy), where upstream's is generic; the built-in agent passes no guard, since the loop already ends the step on a hard stop. The `complete_step` tool takes `errorCode`, as upstream's, and `VerdictRules` names `complete_step`. The built-in rules were one text; they are now `BaseRules` (the action guidance) and `VerdictRules`, and `ToolLoop` appends `VerdictRules` after the executor's guidance and the project context, as upstream's chassis appends `VERDICT_RULES`, so every loop executor, the built-in one included, sends the verdict rules last. `VerdictTool.Execute` keeps the first accepted verdict, answers `Step concluded.` or `The step already concluded.`, drops a code sent beside `passed` and says so, and refuses a blocked verdict without a blockable code; `ToolLoop` uses the same policy through a stateless internal member, so the loop's own `complete_step` answers the same text (`Step concluded.`, or the note that a code sent beside `passed` was ignored, which names the code only when it is from the closed set, and says a code was ignored without echoing any other). A call to a tool the step does not offer is answered `failed: Model tried to call unavailable tool 'click'. Available tools: observe, tap, type, complete_step.` (the list is the step's own tools, so it varies with the engine and the project tools), the AI SDK's `NoSuchToolError` text that upstream's `agent-policy` and `loop-guards` tests read. The turn records it as `[click] failed: ...`. `ToolOutcome.Finish` takes the dropped code (`droppedCode`, .NET-only: upstream's tool answers the text itself, and the port's loop reads the text off the stateless outcome). `ScreenOutput` has no counterpart: the port's tool answer, `ToolLoopResult` (`Content`, `Failed`, `Pixels`), is public and carries the text and the screenshot. The screen presenter and `onActionFailed` stay internal (`ScreenPresenter`, `ActTools`). `AgentTools` members the port had made public by mistake (`MaxScrollTimes`, `ModelErrorCodes`, `IsReserved`, `Project`, `ExtractFor`, and the others) are internal now. The internal definition builder `CreateGrammarTools(EngineCapabilities, ...)` is `GrammarToolDefinitions` (.NET-only, since the public name belongs to upstream's). Tests: `SdkTypes/SdkTypesTests` is the new lines of `tests/types/sdk-types.ts` as reflection checks (the four `Partial_` ones check a shape the type checker checks upstream); `AgentGrammarTools/CreateGrammarToolsTests` is .NET-only (a loop that spreads the grammar tools beside its own, the guard, the verdict tool). - The verbs are the ones the port's engines have: `TapAsync`, `DoubleTapAsync`, and `SecondaryTapAsync` (all take `ClickOptions`, whose `Modifiers` need `Target.TapModifiers`, as upstream's `tap(target, { modifiers })` needs `target.tapModifiers`), `LongPressAsync`, `HoverAsync`, `DragAsync` (upstream's `longPress`, `secondaryTap`, `hover`, and `drag`; `Target.Verbs` lists them by the action kinds the engine declares in `IEngineSession.ActionKinds`: `longPress`, `secondaryTap`, `hover`, `dragTo`), `TypeAsync`, `TypeSecretAsync`, `ClearAsync` (upstream clears with `type`), `PressAsync`, `SelectAsync`, `CheckAsync`, `UploadAsync` (upstream's `upload`), `ScrollToAsync`, `ScrollUntilAsync`, `ScrollAsync`, `NavigateAsync`, and `BackAsync`. `TypeTextAsync` (with `TypeTextOptions.Replace`) and `PressKeyAsync` are upstream's `typeText` and `pressKey`, listed when the engine declares `Keyboard`. `DismissKeyboardAsync` is upstream's `dismissKeyboard`, listed when the engine can hide its keyboard (`IEngineSession.CanDismissKeyboard`, with `Keyboard`); it is a default member of `IExecutorActions` that refuses with `UNSUPPORTED_CAPABILITY`, so an existing implementation keeps compiling. `Target.Verbs` follows the engine's capabilities: the scroll verbs need `Scroll`, `Back` needs `History`, `Upload` needs `EngineCapabilities.Upload` (.NET-only name for the `upload` verb an upstream engine lists; the web engine declares it). A node is named by its ref (`new ExecutorTarget("e12")`, the `[ref=e12]` of a line), where upstream names `n42`. Actions and looks run in call order, mutating project tools included, and a mutating `RunToolAsync` body that calls an action or a look waits behind itself, as upstream's does. A verb checks its argument before it takes a slot (`INVALID_ARGUMENT`), but the port has no key grammar, so `PressAsync` takes any non-blank key - `ObserveAsync` answers `Text`, `Truncated`, `KeyboardVisible`, `Path`, and `Revision`, and with `ExecutorObserveOptions` `Tree` (a list of root `ExecutorNode`s, redacted, with no value or selection on a secure field, a link target absolute and cut as a line cuts it) and `Pixels` (the masked screenshot sized for the model, or `PixelsWithheld`, a `VisionDegradation`). `Revision` is upstream's `revision` (`Observation.Revision`, null when the capture carries none); it has no `Viewport` and no `framePath`, since the port has none. `ReplayedPrefix` carries the actions as the text the hand-off notice lists, `ReplayHandOffReason` adds the port's `ActionBudget` and `InvalidEntry`, and has no `viewport-changed`. A replay that meets `ACTION_MAY_HAVE_COMMITTED` stops as `action-uncertain` and names the action, which the built-in agent's notice repeats. `KeyboardVisible` is upstream's `keyboardVisible` (commit `5fce7b2`): `Observation.KeyboardVisible` carries the engine's measured on-screen keyboard state to `ExecutorObservation.KeyboardVisible`, and the mobile `DeviceSession` sets it from the capture's keyboard fact (`visible` is true, `absent` is false, and an unmeasured capture leaves it null). The web engine measures none, so it is always null there. Not ported: the part of that commit in `@e2e-dev/decision` (`dismiss_keyboard` and the page fingerprint), and the `sdk-types.ts` checks, which the C# types make. Tests: `FixtureRuntime/ProjectToolDispatchTests` and `E2E.Mobile.Tests`' `Engine/ObservationTests`. - The runtime owns the budgets and the clock. `RecordModelCall` throws `STEP_BUDGET_EXHAUSTED` past `MaxModelCalls`, and a verdict never outranks a hard stop: a passed one over a spent budget, a timeout, or a code-less failure after one ends with the runtime's code. An executor that ignores its token gets one second to unwind after the step is stopped, and then the step settles without it. The closed verdict grammar is upstream's: an empty summary, an unknown code, a passed verdict with a code, or a blocked one without a blockable code is `MODEL_OUTPUT_INVALID`, and so is a runtime code the runtime did not record. A thrown error that is no agent error becomes `ACTION_FAILED`, as upstream's `toAgentError` does, and a cancel by the caller stays an `OperationCanceledException`. The message of a failed or blocked verdict is upstream's `agent.act failed: ` or `agent.act blocked: ` (`agent.assert` for an assert). The step keeps the bare summary as its explanation, as upstream does. A hard stop (`Step timed out`, a spent budget) that ends the step with no verdict, or over a passed or code-less one, has a message with no prefix (the reason, then the verdict's summary if there is one); a coded failed or blocked verdict gets the prefix, also after a hard stop - `AssertAsync` on an agent with an executor goes to it as a step of kind `Assert`, with the agent's budgets and `JudgmentTimeout`. An assert is never cached, and a passing one verifies the acts before it. `Vision` is refused with `UNSUPPORTED_CAPABILITY`, since the executor decides what its model sees, and `Screenshot` keeps its evidence after the verdict, bounded by the action timeout: a capture that outlives it is dropped and the verdict stands. `WaitForAsync` and `ExtractAsync` keep the judge, which is the executor's `Judge` or `Model`. `ExecutorCache.Off` never replays or records, and takes no look before the step. A step with the cache on that cannot read its first screen (no app opened) still runs, and records nothing -- `TypeSecretAsync` fills a secret the step's params declared, and refuses with `POLICY_DENIED` any other name, a disabled or non-editable field, and a field whose purpose does not fit a password secret. A secret the attempt does not know is `AUTH_CREDENTIAL_UNAVAILABLE`. The built-in agent's `fill_secret` goes through the same check, so its refusals read as the dispatcher's messages and not as `Secret fill refused: ...`. `ExecutorStep.Params` is JSON: a `Secret` is its name and, when set, its purpose, and every string and key that holds a secret value the attempt knows is redacted. `ExecutorAttempt` has the session's `TestTitle` as `TestId`, a fresh id per session, and `Index` as the attempt number minus one; `ExecutorStep.Index` counts the agent steps of the attempt, where upstream's counts every step. `AttachTranscript` writes `transcript-.txt` into the attempt's artifact directory as a `log` artifact of the step, as upstream does, and only under `E2E_DEBUG=1`; `AttachScreenshotAsync` writes `screenshots/-