Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/adr/0012-interactive-replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,8 @@ The report carries a bounded `screen`, ranked `suggestions`, one `repairHint` en
- Capture failure never replaces or conceals the original replay cause.
- Serialized ceilings are 8/24/64 KiB for the compact, default, and full response levels. Overflow goes to a redacted session-scoped artifact.
- Entered text is never serialized. Native `.ad` replay replaces expanded variable values with `<var:NAME>` in human-readable divergence fields.
- Every surface (daemon text, JSON, Node client `AppError`, MCP `structuredContent`) preserves the structured error. No surface may reduce divergence to text and discard repair data.
- Every surface (daemon text, JSON, Node client `AppError`, MCP text, and MCP `structuredContent` on a structured server) preserves the repair data. No surface may reduce divergence to text that discards it. The default MCP server returns text only, so its error text carries the step, the repair hint (with the concrete resume command when `resume.allowed` is true), the screen, and the suggestions.
Comment thread
thymikee marked this conversation as resolved.
Resolved
Why: clients that receive `structuredContent` show the model that JSON and drop the text, so a default `structuredContent` error replaced the compact report with the larger JSON. A client that reads errors as data opts in with `agent-device mcp --structured`.

Resume is `replay --from N --plan-digest <sha>`.
`N` is a 1-based index into the fully expanded executable plan.
Expand Down Expand Up @@ -183,6 +184,7 @@ The [divergence construction contract](../../packages/replay-port/docs/replay-di
- **An absolute rect tolerance in identity.** No measured drift distribution justifies any constant.
- **A publish lock that protects only complete healed files.** It adds lock, lease, and reclaim races for a partial file. Exclusive `linkSync` decides the winner instead.
- **Infer a corrective read from its position relative to the later `--from`.** The daemon records actions immediately and cannot know their future resume role, so the inference would retrospectively reinterpret intent. `--record` marks the corrective read explicitly.
- **Keep `structuredContent` on default MCP errors.** Clients that receive it show the model the JSON instead of the compact repair report; data clients opt in with `agent-device mcp --structured`.
- **One flag helper with an `allowRecord` policy argument.** A changed literal could let a mutating reader forward `--record` with no schema change. Two named helpers make the capability the helper's name.

## Consequences
Expand Down
12 changes: 8 additions & 4 deletions packages/ad-replay/docs/divergence-and-resume.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The daemon computes it under ADR 0012 decision 6, R3.
Every divergence includes it, with `manual` when no safer route is proven.
It is never absent or null.
Its small fixed size permits inclusion at every response level, including `--level digest`.
It must survive these four projections unchanged: daemon text summary, JSON, Node client `AppError`, and MCP `structuredContent`.
It must survive these projections unchanged: daemon text summary, JSON, Node client `AppError`, MCP text, and MCP `structuredContent` where MCP returns it.

`screen` has two discriminated forms.
`{ state: "available", refsGeneration, refs, truncated }` is a fresh healthy snapshot digest.
Expand Down Expand Up @@ -96,7 +96,10 @@ All surfaces preserve the daemon error.
The Node client rejects with `AppError` and retains `details.divergence`.
CLI exits nonzero.
Text shows a compact report; JSON includes the complete structured error.
MCP returns `isError: true`, the object in `structuredContent`, and the same compact text summary.
MCP returns `isError: true` and the same compact text summary.
The default MCP server returns that text only, so the text carries the repair fields: step, repair hint, screen refs, and suggestions.
The repair hint includes the concrete resume command when `resume.allowed` is true.
Only `agent-device mcp --structured` and data hosts, such as the AI SDK adapter, also return the object in `structuredContent`.
MCP treats divergence as a ref-issuing result, including on the error path.
Before return, it merges and pins every `screen` ref with `refsGeneration`.
CLI and direct clients receive the daemon error's unpinned refs and generation.
Expand Down Expand Up @@ -232,8 +235,9 @@ Successful text replay prints one line with replayed count and wall time.
- Divergence reports: compact/default/full field and byte ceilings; digest output includes `suggestionCount` without entries; redaction, overflow artifacts, artifact-write failure, available/sparse/capture-failed screens, and original-cause preservation.
- Resume: digest emission and rejection after script/include/expansion changes; `resume.allowed` reasons, `--from` indexing, variable-output rejection, control-flow rejection, and `test --from` rejection.
- Surfaces: daemon/client/CLI/MCP tests prove typed divergence survives failure.
JSON and MCP structured output retain it.
JSON and MCP structured output (`--structured` and data hosts) retain it.
Default MCP error text carries the repair fields, and MCP pins its screen refs in both modes.
MCP pins only actionable error-path refs.
No text-only path discards the report.
`repairHint` is present and identical in text, JSON, client `AppError`, and MCP `structuredContent`, including compact `--level digest`.
`repairHint` is present and identical in text, JSON, client `AppError`, MCP text, and MCP `structuredContent` where returned, including compact `--level digest`.
- `--update` retirement: no source-file rewrite; suggestions remain bounded, ranked, and deduplicated under the suggestion total order.
2 changes: 1 addition & 1 deletion packages/command-output/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export {
} from './output-common.ts';
export { colorize, supportsColor } from './output/color.ts';
export type { ColorStream } from './output/color.ts';
export { formatErrorCandidateViews, formatHumanError } from './output/error.ts';
export { formatHumanError } from './output/error.ts';
export { formatDeliveredArtifactLines, serializeDevice } from './output/result-serialization.ts';

/** Renders a command's result through the family formatters; the formatter tables load on first call. */
Expand Down
2 changes: 1 addition & 1 deletion packages/command-output/src/output/error.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ function withFullStop(text: string): string {
return /[.!?…]$/.test(text) ? text : `${text}.`;
}

export function formatErrorCandidateViews(views: ErrorCandidateView[]): string[] {
function formatErrorCandidateViews(views: ErrorCandidateView[]): string[] {
return views.flatMap((view) => {
if (view.kind === 'element-match') {
const remaining = view.matches - view.candidates.length;
Expand Down
10 changes: 10 additions & 0 deletions packages/command-registry/src/flag-definitions-connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,16 @@ export const CONNECTION_FLAG_DEFINITIONS: readonly FlagDefinition[] = [
projectConfig: false,
recorded: false,
},
{
key: 'mcpStructured',
names: ['--structured'],
type: 'boolean',
usageLabel: '--structured',
usageDescription:
'MCP: advertise tool output schemas and return structuredContent; results are text only by default',
projectConfig: false,
recorded: false,
},
{
key: 'tenant',
names: ['--tenant'],
Expand Down
3 changes: 2 additions & 1 deletion packages/command-surface/src/schema/command-overrides.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,9 @@ const SCHEMA_ONLY_CLI_COMMAND_SCHEMAS = {
text: {
summary: 'Start MCP server',
description:
'Start the official stdio MCP server. It exposes structured command tools backed by the agent-device client. It accepts no flags or arguments other than --help and --version; configure it with AGENT_DEVICE_* environment variables or AGENT_DEVICE_CONFIG.',
'Start the official stdio MCP server. It exposes command tools backed by the agent-device client. Tool results are compact text by default; mcpOutputFormat: "json", a non-default responseLevel, or includeCost returns JSON text. With --structured, tools also advertise output schemas and return structuredContent, including errors, for clients that read results as data. It accepts no other flags or arguments besides --structured, --help, and --version; configure it with AGENT_DEVICE_* environment variables or AGENT_DEVICE_CONFIG.',
},
allowedFlags: ['mcpStructured'],
},
proxy: {
text: {
Expand Down
1 change: 1 addition & 0 deletions packages/contracts/src/cli-flags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export type CliFlags = CloudProviderProfileFields &
daemonServerMode?: DaemonServerMode;
proxyHost?: string;
proxyPort?: number;
mcpStructured?: boolean;
tenant?: string;
sessionIsolation?: SessionIsolationMode;
runId?: string;
Expand Down
2 changes: 1 addition & 1 deletion packages/kernel/src/errors-code-policy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ const EXPECTED_HINT_BY_CODE: Record<KnownAppErrorCode, string> = {
UNAUTHORIZED: 'Refresh daemon metadata and retry the command.',
AMBIGUOUS_MATCH: 'Multiple candidates matched. Narrow the query or pass an exact identifier.',
REPLAY_DIVERGENCE:
'Read the divergence report (details.divergence) for the screen and selector suggestions to repair the step.',
'Repair the step from the divergence report: its screen and selector suggestions.',
REPAIR_SESSION_EXPIRED:
'The --save-script repair session was reaped before it was finalized; re-run replay <script> --save-script from the start.',
REPAIR_COMMIT_FAILED:
Expand Down
2 changes: 1 addition & 1 deletion packages/kernel/src/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -675,7 +675,7 @@ function codeRecoveryHint(code: string): string | undefined {
case 'DEVICE_IN_USE':
return 'The device is busy with another agent-device request; retry once it frees up.';
case 'REPLAY_DIVERGENCE':
return 'Read the divergence report (details.divergence) for the screen and selector suggestions to repair the step.';
return 'Repair the step from the divergence report: its screen and selector suggestions.';
case 'REPAIR_SESSION_EXPIRED':
return 'The --save-script repair session was reaped before it was finalized; re-run replay <script> --save-script from the start.';
case 'REPAIR_COMMIT_FAILED':
Expand Down
1 change: 1 addition & 0 deletions scripts/integration-progress-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,7 @@ function summarizeProviderScenarioFlagExclusions() {
'textStdin',
'proxyHost',
'proxyPort',
'mcpStructured',
'stale',
],
},
Expand Down
4 changes: 3 additions & 1 deletion src/__tests__/cli-exit-paths.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ import { runCli } from '../cli.ts';
import { runWebCommand } from '../cli/commands/web.ts';
import { exitAfterFlush } from '../cli/process-exit.ts';

const MCP_ARGS_MESSAGE = 'mcp accepts no flags or arguments other than --help and --version.';
const MCP_ARGS_MESSAGE =
'mcp accepts no flags or arguments other than --structured, --help, and --version.';
import { installIsolatedCliTestEnv } from './cli-test-env.ts';
import { resolveDaemonPaths } from '@agent-device/daemon-contracts/daemon-resolution';
import type { DaemonResponse } from '@agent-device/daemon-client';
Expand Down Expand Up @@ -278,6 +279,7 @@ test('#1596: a failing `test` suite exits 1 through the flushed exit path', asyn
test.each([
{ argv: ['mcp', '--json'], stream: 'stdout', message: MCP_ARGS_MESSAGE },
{ argv: ['mcp', '--debug'], stream: 'stderr', message: MCP_ARGS_MESSAGE },
{ argv: ['mcp', '--structured', '--debug'], stream: 'stderr', message: MCP_ARGS_MESSAGE },
{
argv: ['mcp', 'extra'],
stream: 'stderr',
Expand Down
10 changes: 10 additions & 0 deletions src/ai-sdk/__tests__/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,16 @@ test('asks the executor for structuredContent from every tool, because execute r
assert.deepEqual(executorDeps, [{ structuredContent: 'always' }]);
});

test('typed tools keep their output schema for the data host; untyped tools declare none', async () => {
const { tools } = await createAgentDeviceTools({ session: 'my-session', set: 'all' });
const outputSchemaOf = (name: string) =>
(tools[name] as unknown as { outputSchema?: { jsonSchema: { required?: string[] } } })
.outputSchema;

assert.deepEqual(outputSchemaOf('devices')?.jsonSchema.required, ['devices']);
assert.equal(outputSchemaOf('snapshot'), undefined);
});

test("the AI SDK call's abortSignal reaches the executor", async () => {
const { tools } = await createAgentDeviceTools({ session: 'my-session' });
const controller = new AbortController();
Expand Down
4 changes: 2 additions & 2 deletions src/ai-sdk/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ type Json7Schema = Parameters<typeof jsonSchema>[0];

/**
* Builds an AI SDK tool set from the same command registry the MCP server
* exposes (`listCommandTools()` / `createCommandToolExecutor()`), so the two
* exposes (`listCommandTools({ outputSchemas })` / `createCommandToolExecutor()`), so the two
* integrations stay in lockstep without a hand-maintained tool list. Uses
* `jsonSchema()` against the registry's JSON Schema directly — no zod layer.
*
Expand Down Expand Up @@ -100,7 +100,7 @@ export async function createAgentDeviceTools(
if (platform) hiddenFields.add('platform');

const tools: ToolSet = {};
for (const definition of listCommandTools()) {
for (const definition of listCommandTools({ outputSchemas: true })) {
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
if (set === 'core' && resolveCommandFrameworkTier(definition.name) !== 'core') continue;

tools[definition.name] = tool({
Expand Down
4 changes: 2 additions & 2 deletions src/bin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ runEntry(
mcp: async () => {
const { runAgentDeviceMcpServer } = await import('./mcp/server.ts');
return {
runAgentDeviceMcpServer: async () =>
runAgentDeviceMcpServer: async (options) =>
await withCommandHost(
{ host, requestEnv: undefined },
async () => await runAgentDeviceMcpServer(),
async () => await runAgentDeviceMcpServer(options),
),
};
},
Expand Down
4 changes: 2 additions & 2 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -311,9 +311,9 @@ async function parseCliInputOrExit(
return await exitOnParseFailure(
new AppError(
'INVALID_ARGS',
'mcp accepts no flags or arguments other than --help and --version.',
'mcp accepts no flags or arguments other than --structured, --help, and --version.',
{
hint: 'Run agent-device mcp alone. Configure the server with AGENT_DEVICE_* environment variables or AGENT_DEVICE_CONFIG.',
hint: 'Run agent-device mcp or agent-device mcp --structured. Configure the server with AGENT_DEVICE_* environment variables or AGENT_DEVICE_CONFIG.',
},
),
argv,
Expand Down
16 changes: 12 additions & 4 deletions src/cli/entry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ function harness(options: { bundledVersion?: string; cli?: EntryModules['cli'] }
const stderr: string[] = [];
const cliArgv: string[][] = [];
const exits: number[] = [];
const mcpServers: Array<{ structured: boolean }> = [];
const modules: EntryModules = {
help: async () => {
loaded.push('help');
Expand All @@ -32,7 +33,11 @@ function harness(options: { bundledVersion?: string; cli?: EntryModules['cli'] }
}),
mcp: async () => {
loaded.push('mcp');
return { runAgentDeviceMcpServer: async () => {} };
return {
runAgentDeviceMcpServer: async (options) => {
mcpServers.push(options);
},
};
},
version: async () => {
loaded.push('version');
Expand All @@ -50,7 +55,7 @@ function harness(options: { bundledVersion?: string; cli?: EntryModules['cli'] }
stdout: (text) => stdout.push(text),
stderr: (text) => stderr.push(text),
});
return { run, loaded, stdout, stderr, cliArgv, exits };
return { run, loaded, stdout, stderr, cliArgv, exits, mcpServers };
}

test('every registered alias prints its canonical help without loading the CLI', async () => {
Expand Down Expand Up @@ -88,13 +93,16 @@ test('no command prints usage and exits 1 after flushing', async () => {
assert.deepEqual(entry.exits, [1]);
});

test('only a bare mcp starts the server; help prints usage and other arguments reach the CLI', async () => {
test('mcp and mcp --structured start the server; help prints usage and other arguments reach the CLI', async () => {
const server = harness();
await server.run(['mcp']);
assert.deepEqual(server.loaded, ['mcp']);
await server.run(['mcp', '--structured']);
assert.deepEqual(server.loaded, ['mcp', 'mcp']);
assert.deepEqual(server.mcpServers, [{ structured: false }, { structured: true }]);

for (const argv of [
['mcp', '--state-dir', './sd'],
['mcp', '--structured', '--json'],
['mcp', 'extra'],
['--json', 'mcp'],
]) {
Expand Down
18 changes: 13 additions & 5 deletions src/cli/entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ export type EntryModules = Readonly<{
resolveHelpTargetUsageText(target: string): string | null;
}>;
cli: () => Promise<{ runCliProcess(argv: string[]): Promise<unknown> }>;
mcp: () => Promise<{ runAgentDeviceMcpServer(): Promise<void> }>;
mcp: () => Promise<{ runAgentDeviceMcpServer(options: { structured: boolean }): Promise<void> }>;
version: () => Promise<{ readVersion(): string }>;
processExit: () => Promise<{ exitAfterFlush(code: number): Promise<void> }>;
}>;
Expand Down Expand Up @@ -33,7 +33,8 @@ async function dispatch(argv: string[], modules: EntryModules, io: EntryIo): Pro
if (isVersionRequest(argv)) return await printVersion(modules, io);
const helpTarget = simpleHelpTarget(argv);
if (helpTarget !== undefined) return await printHelp(helpTarget, argv, modules, io);
if (isMcpServerRequest(argv)) return await (await modules.mcp()).runAgentDeviceMcpServer();
const mcpServer = mcpServerRequest(argv);
if (mcpServer) return await (await modules.mcp()).runAgentDeviceMcpServer(mcpServer);
await runCli(argv, modules);
}

Expand Down Expand Up @@ -68,9 +69,16 @@ function isVersionRequest(argv: string[]): boolean {
return argv.length === 1 && (argv[0] === '--version' || argv[0] === '-V');
}

/** Only a bare `mcp` starts the server; the full CLI rejects any flag or argument beside it. */
function isMcpServerRequest(argv: string[]): boolean {
return argv.length === 1 && argv[0] === 'mcp';
/**
* Only `mcp` and `mcp --structured` start the server; the full CLI rejects any other argument.
* This is the one runtime read of `--structured`: its `mcpStructured` flag declaration serves help
* and that rejection only, so the server starts without loading the CLI module.
*/
function mcpServerRequest(argv: string[]): { structured: boolean } | undefined {
if (argv[0] !== 'mcp') return undefined;
if (argv.length === 1) return { structured: false };
if (argv.length === 2 && argv[1] === '--structured') return { structured: true };
return undefined;
}

/** `null` = general usage, a string = one command's help, `undefined` = not a help request. */
Expand Down
51 changes: 51 additions & 0 deletions src/mcp/__tests__/command-tools-divergence-text.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import assert from 'node:assert/strict';
import { test } from 'vitest';
import type { AgentDeviceClient } from '@agent-device/command-surface';
import { AppError } from '@agent-device/kernel/errors';
import { createCommandToolExecutor } from '../command-tools.ts';

// The default MCP server returns text only, so a replay divergence must reach the model through
// the error text with its repair data, and its screen refs must still be pinned.
test('a default-mode divergence is error text carrying the repair data, and its refs are pinned', async () => {
const runCalls: Array<{ name: string; input: unknown }> = [];
const executor = createCommandToolExecutor({
createClient: () => ({}) as AgentDeviceClient,
runCommand: async (_client, name, input) => {
runCalls.push({ name, input });
if (name !== 'replay') return {};
throw new AppError('REPLAY_DIVERGENCE', 'Replay failed at step 2 (click "Save")', {
divergence: {
version: 1,
kind: 'action-failure',
step: { index: 2, source: { path: '/tmp/flow.ad', line: 2 } },
action: 'click "Save"',
screen: {
state: 'available',
refsGeneration: 12,
refs: [{ ref: 'e5', role: 'button', label: 'Save' }],
},
suggestions: [
{ selector: 'id="save"', basis: 'id', ref: 'e5', role: 'button', label: 'Save' },
],
suggestionCount: 1,
resume: { allowed: true, from: 2, planDigest: 'digest-abc' },
repairHint: 'state-repair',
},
});
},
});

const result = await executor.execute('replay', { path: '/tmp/flow.ad' });

assert.equal(result.isError, true);
assert.equal('structuredContent' in result, false);
const text = result.content[0]?.text ?? '';
assert.match(text, /^Error \(REPLAY_DIVERGENCE\)/);
assert.match(text, /Divergence at step 2 \(\/tmp\/flow\.ad:2\)/);
assert.match(text, /Repair hint: state-repair — .*--from 2 --plan-digest digest-abc/);
assert.match(text, /@e5 \[button\] "Save"/);
assert.match(text, /\[id\] "Save" id="save"/);

await executor.execute('press', { ref: '@e5' });
assert.deepEqual(runCalls[1]?.input, { ref: '@e5~s12' });
});
Loading
Loading