Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
33ba409
Name the agent tools as upstream: complete_step, type, type_secret, a…
claude Nov 14, 2026
e0244c1
List type_secret once for the ACP session and gate it per step; resto…
claude Nov 14, 2026
a070261
Address review: complete_step in the remaining prompts, session-only …
claude Nov 14, 2026
5a459b7
Merge origin/main into fix/1630-full-toolset-names
claude Nov 14, 2026
e0fb39d
Fix the KindsByVerb comment: MCP clear runs through the session-only …
claude Nov 14, 2026
1560c6b
Reserve type_secret for project tools and fix stale tool names in COM…
claude Nov 14, 2026
7801166
COMPATIBILITY: name complete_step in the forced-conclusion notice, dr…
claude Nov 14, 2026
2955a5a
Merge origin/main into fix/1630-full-toolset-names
claude Nov 14, 2026
ad5985d
Name complete_step in the ToolChoice docs, the ACP README, and the cl…
claude Nov 14, 2026
c3b0643
Document clearing a field the engine cannot fill through the keyboard…
claude Nov 14, 2026
896c30c
Merge origin/main into fix/1630-full-toolset-names
claude Nov 15, 2026
cf83ce1
Mark the ACP tools-listing test partial: its screenshot check needs a…
claude Nov 15, 2026
cb8e6b7
Name complete_step and type in the custom-executor and web guides (#1…
claude Nov 15, 2026
76c3076
Merge origin/main into fix/1630-full-toolset-names
claude Nov 15, 2026
fbe9475
Merge main into fix/1630-full-toolset-names
claude Nov 15, 2026
f7f6d56
check rejects a non-boolean checked; drop the port-only secret-fill test
claude Nov 15, 2026
1f063bf
Read the transcript through its registered path; fix the MCP test-lin…
claude Nov 15, 2026
87e62e6
Drop the same-name complete_step comparison from the tool loop bullet
claude Nov 15, 2026
009015e
Merge remote-tracking branch 'origin/main' into fix/1630-full-toolset…
claude Nov 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 23 additions & 23 deletions COMPATIBILITY.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions docs-site/src/content/docs/api/E2E/AgentTools.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,13 @@ The built-in agent's tool vocabulary (upstream's `agent/primitives.ts`). `Create


```csharp
public const string VerdictRules = "Finish with done when the step's goal is achieved, or you are certain it cannot be. done status is passed, failed, or blocked. blocked means credentials, the environment, or test setup prevented a verdict, and requires an errorCode."
public const string VerdictRules = "Finish with complete_step when the step's goal is achieved, or you are certain it cannot be. complete_step status is passed, failed, or blocked. blocked means credentials, the environment, or test setup prevented a verdict, and requires an errorCode."
```


Upstream's `VERDICT_RULES`: the guidance for [VerdictTool](/api/E2E/VerdictTool/). The tool loop appends it to the instructions of every
executor it builds, so an executor outside the loop gives it to its own model beside the tool. The port's tool is `done`
where upstream's is `complete_step`, and takes an `errorCode` as upstream's does.
executor it builds, so an executor outside the loop gives it to its own model beside the tool.
The tool is `complete_step`, and it takes an `errorCode`, as upstream's.

**Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)

Expand Down Expand Up @@ -83,7 +83,7 @@ public static VerdictTool CreateVerdictTool(ExecutorStepKind? kind = null)
```


Upstream's `createVerdictTool`: the `done` tool with the closed blocked-code policy, and the verdict it collected. Make a
Upstream's `createVerdictTool`: the `complete_step` tool with the closed blocked-code policy, and the verdict it collected. Make a
fresh one for each step: the first accepted verdict is final.

**Parameters**
Expand Down
6 changes: 3 additions & 3 deletions docs-site/src/content/docs/api/E2E/DefaultAgent.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@ Upstream's `createBuiltInAgent` (`agent/default-agent.ts`): the tool loop with t


```csharp
public const string AssertRules = "You are judging exactly one assertion against the current application screen.\n- Use only the observation, scrolling, and read-only project tools offered to you. Do not tap, type, navigate, or change application state.\n- Combine evidence from every observation in this step. Items need not fit in one viewport. Scroll only to inspect the assertion's subject, and stop when you have enough evidence.\n- Node ids belong to the observed tree. Never invent ids; use scroll_to for a listed node or scroll to inspect more of the current screen.\n- Each scroll result contains a fresh observation. When screenshots are shown, each result includes fresh pixels too.\n- Conclude passed when the collected evidence establishes the assertion, failed with code ASSERTION_FAILED when it contradicts the assertion, and failed with code ASSERTION_INCONCLUSIVE when the available evidence or remaining budget cannot establish an answer."
public const string AssertRules = "You are judging exactly one assertion against the current application screen.\n- Use only the observation, scrolling, and read-only project tools offered to you. Do not tap, type, navigate, or change application state.\n- Combine evidence from every observation in this step. Items need not fit in one viewport. Scroll only to inspect the assertion's subject, and stop when you have enough evidence.\n- Node ids belong to the observed tree. Never invent ids; use scroll_to for a listed node or scroll to inspect more of the current screen.\n- Each scroll result contains a fresh observation. When screenshots are shown, each result includes fresh pixels too.\n- Conclude passed when the collected evidence establishes the assertion, failed with errorCode ASSERTION_FAILED when it contradicts the assertion, and failed with errorCode ASSERTION_INCONCLUSIVE when the available evidence or remaining budget cannot establish an answer."
```


Assertion-specific rules (upstream's `ASSERT_RULES`): they allow evidence gathered across several viewports. The port's verdict tool is `done`,
Assertion-specific rules (upstream's `ASSERT_RULES`): they allow evidence gathered across several viewports. The verdict tool is `complete_step`,
which takes `errorCode`.

**Value:** [string](https://learn.microsoft.com/dotnet/api/system.string)
Expand All @@ -48,7 +48,7 @@ which takes `errorCode`.


```csharp
public const string BaseRules = "You are an e2e test agent. Act only by calling tools. The screen is a tree of nodes with stable ids like \"n42\", listed one per line as #id role \"name\": a node keeps its id for as long as it exists, across every screen and change in this conversation. Target a node by its id (ref), with or without the #. Never invent ids. The first screen is sent whole. Every action result and every observe then reports what changed since the screen you last received, one line per node: \"added\" (a new node), \"changed\" (with what it read before), or \"removed\" (the node is gone; never target it again). A node not listed as removed is still there under the id you have. When most of the screen changed, the result sends the whole new screen instead, headed \"Current screen\", and it replaces what you had: target only the ids it lists. Every action result already waited for the effect and contains the changes, so do not call observe after an action. Call observe only after waiting for something the last result showed in progress. Conclude passed only when the latest result shows the outcome the step asked for. A result that reports no change, or only the keyboard closing, means the decisive action did not land: act again or conclude failed. A result that says only that no listed node changed has not ruled out a drawn change: take a screenshot, when offered, before deciding. Never describe an unchanged screen as the outcome. Never type a secret value. Call fill_secret with the secret name. A secret looks like <secret:name>. If what you need is not on the screen, scroll to it. screenshot attaches the viewport's pixels when the tree lacks what you need (a shape on a canvas, a pin on a map, a region of an image, a control inside a system sheet) or contradicts what you expect; once you have one, every action result carries a fresh screenshot so you can see what the action did. tap_at, hover_at, drag_at, type_at, press_at, and select_at act at a point in the latest screenshot's pixel coordinates. tap_at, hover_at, and drag_at act on the bare points where the engine can, and on the listed controls under them by id where it cannot; type_at, press_at, and select_at act on a listed control under the point by its id. Prefer ids: tap, fill, press, and select by id whenever the screen lists the target, use the point tools only for a target the screen does not list, and take a screenshot before guessing a point rather than aiming from memory or from what you expect to be drawn."
public const string BaseRules = "You are an e2e test agent. Act only by calling tools. The screen is a tree of nodes with stable ids like \"n42\", listed one per line as #id role \"name\": a node keeps its id for as long as it exists, across every screen and change in this conversation. Target a node by its id (ref), with or without the #. Never invent ids. The first screen is sent whole. Every action result and every observe then reports what changed since the screen you last received, one line per node: \"added\" (a new node), \"changed\" (with what it read before), or \"removed\" (the node is gone; never target it again). A node not listed as removed is still there under the id you have. When most of the screen changed, the result sends the whole new screen instead, headed \"Current screen\", and it replaces what you had: target only the ids it lists. Every action result already waited for the effect and contains the changes, so do not call observe after an action. Call observe only after waiting for something the last result showed in progress. Conclude passed only when the latest result shows the outcome the step asked for. A result that reports no change, or only the keyboard closing, means the decisive action did not land: act again or conclude failed. A result that says only that no listed node changed has not ruled out a drawn change: take a screenshot, when offered, before deciding. Never describe an unchanged screen as the outcome. Never type a secret value. Call type_secret with the secret name. A secret looks like <secret:name>. If what you need is not on the screen, scroll to it. screenshot attaches the viewport's pixels when the tree lacks what you need (a shape on a canvas, a pin on a map, a region of an image, a control inside a system sheet) or contradicts what you expect; once you have one, every action result carries a fresh screenshot so you can see what the action did. tap_at, hover_at, drag_at, type_at, press_at, and select_at act at a point in the latest screenshot's pixel coordinates. tap_at, hover_at, and drag_at act on the bare points where the engine can, and on the listed controls under them by id where it cannot; type_at, press_at, and select_at act on a listed control under the point by its id. Prefer ids: tap, type, press, and select by id whenever the screen lists the target, use the point tools only for a target the screen does not list, and take a screenshot before guessing a point rather than aiming from memory or from what you expect to be drawn."
```


Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/api/E2E/ExploreOptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ public IReadOnlyDictionary<string, UserCredential>? Credentials { get; init; }


The accounts the explorer can sign in with, by name. Each travels into every step as the username and a password secret
named `<name>.password`, which the explorer fills with `fill_secret`; the password never reaches a model.
named `<name>.password`, which the explorer fills with `type_secret`; the password never reaches a model.
The config's `credentials` are always accounts, as upstream carries them; this adds accounts the config does not declare
(.NET-only: `e2e explore` passes the `E2E_USER_*` pairs of the environment).

Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/api/E2E/ModelRequest.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ public ModelToolChoice? ToolChoice { get; init; }

Upstream's `ModelCall.toolChoice` (the AI SDK's `toolChoice`): how the model must use the tools it is offered. Null leaves
the choice to the provider, which is [Auto](/api/E2E/ModelToolChoice/#E2E_ModelToolChoice_Auto). The act tool loop asks for [Required](/api/E2E/ModelToolChoice/#E2E_ModelToolChoice_Required)
and names `done` on its forced-conclusion turns. A custom [IAgentModel](/api/E2E/IAgentModel/) that cannot force a tool may ignore it.
and names `complete_step` on its forced-conclusion turns. A custom [IAgentModel](/api/E2E/IAgentModel/) that cannot force a tool may ignore it.

**Value:** [ModelToolChoice](/api/E2E/ModelToolChoice/)

Expand Down
4 changes: 2 additions & 2 deletions docs-site/src/content/docs/api/E2E/ModelResponses.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ public static ModelResponse Call(string name, object arguments)


```csharp
public static ModelResponse Done(string status, string summary, string? code = null)
public static ModelResponse Done(string status, string summary, string? errorCode = null)
```


Expand All @@ -64,7 +64,7 @@ public static ModelResponse Done(string status, string summary, string? code = n
| --- | --- | --- |
| `status` | [string](https://learn.microsoft.com/dotnet/api/system.string) | |
| `summary` | [string](https://learn.microsoft.com/dotnet/api/system.string) | |
| `code` | [string](https://learn.microsoft.com/dotnet/api/system.string) | |
| `errorCode` | [string](https://learn.microsoft.com/dotnet/api/system.string) | |

**Returns:** [ModelResponse](/api/E2E/ModelResponse/)

Expand Down
4 changes: 2 additions & 2 deletions docs-site/src/content/docs/api/E2E/PreparedMessages.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ public sealed record PreparedMessages : IEquatable<PreparedMessages>


What [PrepareMessages](/api/E2E/ToolLoopExecutorOptions/#E2E_ToolLoopExecutorOptions_PrepareMessages) hands back: the history for the next turn, and optionally a reason to
stop. The executor has evidence the loop guards cannot see that the step is not progressing. A stop leaves the model only `done`, like a loop guard.
stop. The executor has evidence the loop guards cannot see that the step is not progressing. A stop leaves the model only `complete_step`, like a loop guard.

**Inheritance:** [object](https://learn.microsoft.com/dotnet/api/system.object)

Expand All @@ -42,7 +42,7 @@ public PreparedMessages(IReadOnlyList<ModelMessage> Messages, string? Stop = nul


What [PrepareMessages](/api/E2E/ToolLoopExecutorOptions/#E2E_ToolLoopExecutorOptions_PrepareMessages) hands back: the history for the next turn, and optionally a reason to
stop. The executor has evidence the loop guards cannot see that the step is not progressing. A stop leaves the model only `done`, like a loop guard.
stop. The executor has evidence the loop guards cannot see that the step is not progressing. A stop leaves the model only `complete_step`, like a loop guard.

**Parameters**

Expand Down
3 changes: 2 additions & 1 deletion docs-site/src/content/docs/api/E2E/ScriptedModel.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ public sealed class ScriptedModel : IAgentModel
Returns whatever the callback decides for each request. Records every request. A callback that serves both the act loop
and the judge may answer a judgment (a request with a [Schema](/api/E2E/ModelRequest/#E2E_ModelRequest_Schema)) with `Done`:
`passed` reads as `holds`, a `failed` without the code `ASSERTION_INCONCLUSIVE` as `fails`, and a
`blocked` or inconclusive one as `inconclusive`. `Judgment` answers it directly.
`blocked` or inconclusive one as `inconclusive`. `Judgment` answers it directly. A tool call that names its node by
`role` and `name` (or `testId`) reaches the agent as a `target` id, the one the screens in the request list for it.

**Inheritance:** [object](https://learn.microsoft.com/dotnet/api/system.object)

Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/api/E2E/ToolLoop.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ public static class ToolLoop


Upstream's `createToolLoopExecutor` (`agent/tool-loop.ts`): the chassis of an executor that thinks in a loop of model
turns and tool calls. It owns everything but the tool vocabulary: the `done` verdict tool and its closed code policy, the
turns and tool calls. It owns everything but the tool vocabulary: the `complete_step` verdict tool and its closed code policy, the
hard stops (a spent budget, a timeout, or a cancel ends the loop, never the model), the loop guards, the shrink of a history
that overflowed the model, and the count of model calls. The built-in agent is this chassis plus the grammar tools
([CreateBuiltInAgent(BuiltInAgentOptions)](/api/E2E/DefaultAgent/#E2E_DefaultAgent_CreateBuiltInAgent_E2E_BuiltInAgentOptions_)). An executor for a device or an API brings other tools and inherits the same discipline.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ public required Func<StepExecutorContext, ToolLoopHelpers, IReadOnlyList<ToolLoo
```


Builds the step's tool vocabulary. The chassis adds `done`.
Builds the step's tool vocabulary. The chassis adds `complete_step`.

**Value:** [Func&lt;StepExecutorContext, ToolLoopHelpers, IReadOnlyList&lt;ToolLoopTool&gt;&gt;](https://learn.microsoft.com/dotnet/api/system.func-3)

Expand Down
4 changes: 2 additions & 2 deletions docs-site/src/content/docs/api/E2E/VerdictTool.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ drives a model outside the loop is the only caller of [Execute(ModelToolCall)](#


```csharp
public const string Name = "done"
public const string Name = "complete_step"
```


Expand Down Expand Up @@ -71,7 +71,7 @@ public ModelTool Tool { get; }
```


The `done` tool; offer it beside the grammar tools.
The `complete_step` tool; offer it beside the grammar tools.

**Value:** [ModelTool](/api/E2E/ModelTool/)

Expand Down
Loading
Loading