Skip to content

feat(mcp)!: return text-only tool results by default; add mcp --structured - #4466

Merged
thymikee merged 10 commits into
mainfrom
feat/mcp-text-only-results
Nov 12, 2026
Merged

thymikee merged 10 commits into
mainfrom
feat/mcp-text-only-results

Conversation

@thymikee

@thymikee thymikee commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Default agent-device mcp returns compact text without output schemas or structuredContent, so clients do not replace that text with the larger JSON result. Errors use the shared CLI renderer, preserving dispatch disclosure, observe-first advice and diagnostic paths. ADR 0012 records the revised contract and rationale.

  • agent-device mcp --structured restores output schemas and structured results.
  • AI SDK data tools retain structured results and schemas.
  • mcpOutputFormat: "json", non-default responseLevel, and includeCost retain their JSON-text opt-ins.

42 files. Includes the #4458 output-schema guard adjustment; no new command family.

Validation

Current head: 5a5995e9adf2525a20e25e107e729077778f001a. Validation: CI. Hosted checks are green; Apex Sentinel approves with no findings. All review threads are resolved. Maintainer sign-off remains pending.

At a4bc1e6: frozen install/build and 227 MCP/AI SDK tests pass. Two new error regressions fail before the fix and pass afterward. Its exact affected gate failed at production-exports (run-FstHFp, clean checkout); prior stages passed. At 5a5995e9, the repaired production-exports guard passes locally and in hosted CI. CI tested merge checkout 8fbce857d8 (this head plus main e54dc4d8).

Review evidence: corrected error fixture sizes are 212→143 and 235→166 bytes. Earlier measurements belong to their earlier SHAs. The prior headless-client run did not capture a settled press success. No device-facing behavior changed in this review fix.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff Turn on auto-fix

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.22 MB 6.22 MB +607 B
Package (unpacked) 6.22 MB 6.22 MB +607 B
Package (download) 1.89 MB 1.89 MB +214 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 29.6 ms 37.8 ms +8.1 ms
CLI --help 56.8 ms 60.6 ms +3.9 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 issues found across 29 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/mcp/__tests__/command-tools-structured-content.test.ts">

<violation number="1" location="src/mcp/__tests__/command-tools-structured-content.test.ts:29">
P2: This test uses the retired `target` shape, so the executor returns an `INVALID_ARGS` error before the mocked command runs and the next assertion fails. Use the current top-level target form, such as `{ ref: '@e2' }`.</violation>
</file>

<file name="docs/adr/0012-interactive-replay.md">

<violation number="1" location="docs/adr/0012-interactive-replay.md:94">
P2: This ADR bullet states as an unconditional guarantee that default MCP error text carries the concrete resume command. The renderer emits that command only when `resume.allowed` is true, and `alternateFrom` only when present. Qualify the sentence so it does not overstate the contract.</violation>
</file>

<file name="src/mcp/__tests__/command-tools-timeout-note.test.ts">

<violation number="1" location="src/mcp/__tests__/command-tools-timeout-note.test.ts:12">
P3: This test now exercises only the structured tool listing, while the PR changes the default listing behavior. Call `listCommandTools({ outputSchemas: false })` here (or test both modes) so the default MCP surface remains covered.</violation>
</file>

<file name="src/ai-sdk/index.ts">

<violation number="1" location="src/ai-sdk/index.ts:103">
P3: The `outputSchemas: true` flag that keeps AI SDK tools structured has no test. `src/ai-sdk/__tests__/index.test.ts` checks the executor's `structuredContent: 'always'` but never asserts that typed tools expose `outputSchema`. Add a test that the `devices` tool from `createAgentDeviceTools` has `outputSchema` defined.</violation>
</file>

<file name="src/mcp/router.ts">

<violation number="1" location="src/mcp/router.ts:152">
P2: `--structured` drops `structuredContent` for tool-call failures handled by `callTool`: unknown command names and malformed arguments return `textToolResult` instead. Thread the mode through this catch and include the normalized error.

(Based on your team's feedback about failed-call structuredContent.)</violation>
</file>

<file name="packages/command-surface/src/schema/command-overrides.ts">

<violation number="1" location="packages/command-surface/src/schema/command-overrides.ts:155">
P3: The MCP help text contradicts its own `allowedFlags`: `--structured` is accepted, but the description says only `--help` and `--version` are allowed. Include `--structured` in that sentence and document the JSON-text opt-ins (`mcpOutputFormat: "json"`, non-default `responseLevel`, and `includeCost`) plus structured error results.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0012-interactive-replay.md Resolved
Comment thread src/mcp/router.ts Resolved
Comment thread src/ai-sdk/index.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 13 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0012-interactive-replay.md Resolved
@apex-sentinel-staging

apex-sentinel-staging Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Apex Sentinel

Verdict: approve

Since the last review: 0 new · 0 still open · 0 no longer raised

No findings.

Reviewed with this repository's own Sentinel configuration (e54dc4d87bbc).

Feedback goes in the review comment threads, as a reply under the inline comment of a finding:

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Comment @apex-sentinel-staging review on this conversation to run a full review.

Earlier reviews

Review of 4562bde7 · 12.11.2026 10:51 UTC

Apex Sentinel

Verdict: approve

Since the last review: 0 new · 0 still open · 1 no longer raised (1 marked fixed)

No findings.

1 finding below minor not shown (user_settings.min_severity).

1 new finding away from what changed since 518ea29 not shown. Comment @apex-sentinel-staging review for a full review.

Reviewed with this repository's own Sentinel configuration (9cbbae2c857d).

Review of 518ea29d · 12.11.2026 10:33 UTC

Apex Sentinel

Verdict: approve

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
src/mcp/router.ts:60 AGENTS.md bans fallback behavior without approval, but superseded MCP paths are kept

Reviewed with this repository's own Sentinel configuration (9cbbae2c857d).

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 518ea29. The verdict and the full list are in the summary comment.

Comment thread src/mcp/router.ts Resolved
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for this change. At 4562bde I found one problem that should be fixed before merge: the default text-only MCP error drops facts that clients used to get. This review comes from reading the code and tests at that commit. I did not run tests or tsc, and I did not reproduce the byte counts, the #4458 guard mutation result, or the headless Claude Code run in the PR body.

In default mode, formatToolErrorText is now the only error surface. It renders code, message, cause, hint, warnings, candidates, supportedOn and divergence. It does not render details.dispatched, logPath/logPathUnavailable, retriable or details.reason, which the old default structuredContent carried. For example, a press that fails after reaching the device with details.dispatched='unknown' now gives the client only Error (CODE): msg and a code hint. It gets no "may have reached the device" fact and no observe-first step, because the kernel hint leaves that step to the printer. A hint such as DEVICE_IN_USE's "retry once it frees up" then reads as a blind retry, and "Read the diagnostics log" names no path. The 212 to 142 byte drop in the DEVICE_IN_USE fixture is retriable and reason disappearing. The rule: every typed fact the CLI human error renders, including the dispatch disclosure with its observe-first step and the Log line, must also appear in the text-only MCP error. formatHumanError in @agent-device/command-output already owns that rendering, so please build the MCP text from it and keep the two renderers from drifting. Please also add a default-mode executor test with details.dispatched='unknown' and a logPath that asserts both lines appear in the text.

Not blocking, and you can take or leave these: open PR #4468 (read at 5d4ac57, so it may change) calls listCommandTools() with no argument in its budget test, rewrites the same field descriptions, and keeps the old "no flags or arguments other than --help and --version" sentence, so whichever merges second should rebase onto the { outputSchemas } signature and the --structured wording; mcpStructured in cli-flags.ts is parsed but never read (entry.ts re-parses argv), so it could be marked help-only; the comments at ai-sdk/index.ts:73 and router.ts:140 still cite listCommandTools() and the REPLAY_DIVERGENCE hint still points at details.divergence, which default MCP text no longer exposes; ADR 0012 narrows the MCP structuredContent guarantee without recording why, which FORMAT.md asks for; outputSchemas and the executor mode are two knobs set in pairs, so a mismatched pair is representable and one mode value could drive both; and the PR body says 29 files while the diff touches 35.

The open inline thread on the default error text still stands only through the finding above. These threads look fixed at head, so you can resolve them: the retired target input in the structured-content test (#4466 (comment)), the resume command now conditioned on resume.allowed in ADR 0012 (#4466 (comment)), executor.errorResult in router.ts (#4466 (comment)), the timeout-note test (#4466 (comment)), the ai-sdk schema assertions (#4466 (comment)), the --structured help text (#4466 (comment)), the divergence-and-resume doc (#4466 (comment)), and the removed handleMcpMessage/handleMcpPayload callers (#4466 (comment)).

Checks are still running: 21 passed at this commit, and one review check is still in progress. There are no conflicts. Before merge, the default MCP error text must show the dispatch disclosure with its observe-first step and the log line, ideally through formatHumanError, with a default-mode test for details.dispatched='unknown'.

thymikee and others added 6 commits November 12, 2026 12:11
…tured

Clients that receive structuredContent show the model that JSON and drop the
compact text, so typed tools paid for the larger form. The MCP server now
advertises no outputSchema and returns text only, on success and on error.
`agent-device mcp --structured` restores output schemas and structuredContent;
the AI SDK data host keeps always-structured results. JSON text renders
compactly, and includeCost renders JSON text so the cost reaches text-only
clients. The output-schema guard validates the structured server's schemas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… tool-list comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…review notes

A structured server now returns structuredContent for an unknown tool or
malformed arguments too: the router's catch builds its result through the
executor, the one owner of the mode. The mcp help names --structured and the
JSON-text opt-ins, ADR 0012 qualifies the resume command, input-schema tests
read the default listing, and the AI SDK keeps a test for its output schemas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The default MCP server returns divergence errors as text that carries the
repair fields; only --structured and data hosts add structuredContent. Cover
the default-mode text, its resume command, and its ref pinning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n caller

runAgentDeviceMcpServer builds its own payload handler, so the module-level
handleMcpMessage and handleMcpPayload only served tests. Tests now build the
handler for the mode they exercise, and the stdio session takes its handler
explicitly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t guard

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee
thymikee force-pushed the feat/mcp-text-only-results branch from 4562bde to 41cb575 Compare November 12, 2026 11:12
…-only

ADR 0012 records why default MCP errors drop structuredContent. The
REPLAY_DIVERGENCE hint names the report, not details.divergence, which
default MCP text does not expose. mcpStructured is declared for help and
argument rejection; the process entry's argv check is its one runtime read.
Two comments cite the current listCommandTools signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Non-blocking notes, at 791c722:

  • mcpStructured is now documented as help-only. Its CliFlags declaration and entry.ts both say that the process entry's argv check is the flag's one runtime read. The parsed value only serves mcp --help and the rejection of other arguments. I kept the entry check rather than moving the server start into the CLI parse, because that would load the CLI module and run the server inside the CLI's diagnostics and config resolution.
  • The comments at ai-sdk/index.ts and router.ts now cite listCommandTools({ outputSchemas }).
  • The REPLAY_DIVERGENCE hint no longer names details.divergence. It now reads "Repair the step from the divergence report: its screen and selector suggestions."
  • ADR 0012 now says why default MCP errors drop structuredContent, and records keeping structuredContent on default errors as a rejected alternative.
  • The PR body file count now says 38.

Not in this push:

  • The blocking default-error finding (formatHumanError, the dispatch/log test, the error-byte table) is being handled in a separate push.
  • Collapsing outputSchemas and the executor mode into one value touches command-tools-structured-content.test.ts, which is part of that separate work. It will follow once that work lands.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/contracts/src/cli-flags.ts Resolved
Comment thread docs/adr/0012-interactive-replay.md Resolved
…ault structuredContent

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the default-error finding in a4bc1e6fc52741a526976d2d243b08f8bcc3bb6c, preserved by the subsequent owner push.

formatToolErrorText now uses formatHumanError for its summary and body. Possible dispatch includes “May have reached the device” and the observe-first step; proven non-dispatch and local/unavailable logs are retained. Causes, warnings, candidates and divergence use that same CLI owner. MCP's existing Supported-on line remains.

The two new cases fail on the old formatter; the corrected MCP/AI SDK selection passes 227 tests. The default-mode executor case supplies dispatched: unknown and a log path. Another covers dispatched: no with an unavailable remote log. Example-call tests still submit every suggested input through its real parser; their extraction now follows the shared Next: summary instead of the removed separate Hint: line.

Remeasured UTF-8 output for the two error fixtures:

Fixture Previous normalized JSON Corrected text
DEVICE_IN_USE 212 143
policy-normalized 235 166

These fixtures do not contain dispatch or log facts, so the routed regression supplies those separately. The earlier 142/165 figures describe the old formatter. Exact affected validation is running at a4bc1e6; it is not yet a pass. The existing owner continues the other review cleanup and final-head CI.

… only formatHumanError uses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

The local affected run at a4bc1e6fc52741a526976d2d243b08f8bcc3bb6c finished failed, receipt run-FstHFp, exactCommit=true, checkoutChanged=false. The failure is the unused formatErrorCandidateViews barrel export. The same fault caused the earlier hosted Repo Guards failure. All preceding stages passed, including 19,351 Vitest tests (three skipped), 76 Node tests (six skipped), and provider integration. Later selected stages did not run.

The existing owner fixed that export in 5a5995e9adf2525a20e25e107e729077778f001a, preserving the error-renderer fix. At that head, pnpm gate production-exports passes with no suppressions added. Full current-head hosted CI and review remain the readiness evidence; the older failed receipt is not a pass. The PR body now records the corrected 42-file count and error-byte measurements.

@thymikee

Copy link
Copy Markdown
Member Author

5a5995e fixes the unused formatErrorCandidateViews re-export from that run: it is now file-private in output/error.ts, used only by formatHumanError. All 22 checks pass at 5a5995e, Apex Sentinel approves with no findings, 0 unresolved threads. The blocking default-error finding is addressed by a4bc1e6 (default MCP errors render through formatHumanError, with dispatch/log regressions).

@thymikee

Copy link
Copy Markdown
Member Author

At 5a5995e9adf2525a20e25e107e729077778f001a, the check rollup is green, Apex Sentinel approves with no findings, and no review threads remain open. iOS and macOS smoke lanes passed. The unused-export failure is fixed: production-exports passes both locally at this head and in hosted Repo Guards.

Hosted checks used merge checkout 8fbce857d8 (head plus main e54dc4d8). The older a4bc1e6 local receipt remains failed as recorded; it is not being relabeled as a pass. No source change is needed for the new approval. Maintainer sign-off remains pending; I have not changed ready-for-human or merged the PR.

@thymikee

Copy link
Copy Markdown
Member Author

I found no code problems at 5a5995e. Both findings from the earlier review (#4466 (comment)) are fixed, and nothing from that review remains open.

All CI checks pass (21 pass, 1 skipped, none failing), and there are no conflicts. I did not run the MCP/AI SDK tests, tsc or the production-exports gate locally, and I did not reproduce the remeasured error byte counts or the headless MCP client run. I judged the regression coverage by reading the pre-change formatter. Maintainer sign-off is the only step left before merge.

Two open review threads are now fixed at this commit, so the author can resolve them: the mcpStructured flag comment in cli-flags.ts (#4466 (comment)) and the rejected-alternative entry in ADR 0012 (#4466 (comment)).

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 12, 2026
@thymikee
thymikee merged commit b6f4617 into main Nov 12, 2026
22 checks passed
@thymikee
thymikee deleted the feat/mcp-text-only-results branch November 12, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant