Skip to content

feat(mcp)!: take UI targets as top-level ref, selector, and x/y - #4459

Merged
thymikee merged 4 commits into
mainfrom
t3/flatten-mcp-target-args
Nov 12, 2026
Merged

thymikee merged 4 commits into
mainfrom
t3/flatten-mcp-target-args

Conversation

@thymikee

@thymikee thymikee commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

An agent without the schema loaded guesses the CLI's vocabulary, such as press {"ref":"@e6"}. The interaction tools wanted {"target":{"kind":"ref","ref":"@e6"}}, so every guess cost an INVALID_ARGS turn.

  • The shared command input in command-surface is now flat. press, click, fill, longpress, and hover take ref (with optional label), selector, or x+y. get takes ref or selector. kind is gone: the key that is present decides the form.
  • The CLI readers spread the positional parser's target into that same object. The MCP schema stays derived from the field metadata.
  • A nested target now fails with the top-level spelling.
  • Breaking: batch steps carry the same input. Saved batch --steps files and client.batch.run steps that use the nested target now fail with that migration message. The typed client.interactions.* API was already flat and is unchanged. client-api.md already said step input uses the client command's fields.
  • An unknown argument now answers with a call example the tool accepts, for example Hint: Example call: press {"ref":"@e12"}. Inside a batch, it answers with an example step.
  • 39 files.
press {"ref":"@e6"}        click {"selector":"id=\"home/products\""}        press {"x":201,"y":160}

Validation

  • validation: CI (head 39190fb). Local runs were limited to focused vitest: command-surface, src/mcp, cli-grammar, src/commands, and the client index tests, all green.
  • Guard: in command-tools-parity.test.ts, CLI argv and MCP input reach runCommand as the identical object, for 6 commands across ref, ref with label, selector, point, and pinned ref.
  • Contract: every tool's hint example is accepted on retry.
  • Headless claude -p with only the local agent-device mcp, iPhone 17 Pro Max simulator, Settings app. press {"ref":"@e6"} and click {"selector":"label=\"Accessibility\""} both landed. 0 INVALID_ARGS, both with schemas loaded and with ToolSearch blocked.
  • CI owns unit-ci and the device lanes.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff Turn on auto-fix

An agent without the schema loaded guesses the CLI's vocabulary as flat
arguments, such as press {"ref":"@e6"} or click {"selector":"id=..."}.
The interaction tools wanted {"target":{"kind":"ref","ref":"@e6"}}, so
each guess cost an INVALID_ARGS turn.

The shared command input in command-surface is now flat: press, click,
fill, longpress, and hover take ref (with optional label), selector, or
x with y as top-level keys; get takes ref or selector. The CLI readers
spread the positional parser's target into that same object, the MCP
schema stays derived from the field metadata, and kind is gone: the
present key decides the form. The nested target is retired and answers
with the top-level spelling. Batch steps carry the same input, which now
matches the client-api statement that step input uses the client
command's fields. A parity test pins CLI argv and MCP input to the
identical object reaching runCommand for every targeted command.

An argument a tool does not take now answers with a call example the
tool accepts, derived from its required properties plus a declared
example where a placeholder cannot state one (the UI target, fold,
telephony, gesture, batch). A batch step answers with a step example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.21 MB 6.22 MB +674 B
Package (unpacked) 6.21 MB 6.21 MB +674 B
Package (download) 1.88 MB 1.88 MB +664 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 20.1 ms 23.3 ms +3.2 ms
CLI --help 46.4 ms 50.0 ms +3.6 ms

…at target

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/mcp/server-guide.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 40 files

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/quick-start.md Resolved
Comment thread src/mcp/server-guide.ts Resolved
thymikee and others added 2 commits November 12, 2026 10:31
- The nested-admission fixtures used `tap`, which batch does not resolve,
  so they checked nothing; they now use the registered `press`.
- The batch retry example names `snapshot` only when that batch's
  allowlist includes it.
- Target field descriptions state "exactly one of ref, selector, or x+y";
  the reader's refusals of no target, two targets, and a lone x are
  pinned with their example hint. The schema keeps no top-level oneOf,
  which some model clients reject in a tool input schema.
- The MCP help preamble and batch help name label in the mapping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The batch doc examples drop the redundant wait kind, so no step reads
  like the removed target discriminator.
- A top-level kind is retired beside target: it answers with the same
  migration INVALID_ARGS instead of being dropped.
- The target read tests assert the exact projection.
- The help preamble shows MCP calls (press {ref, settle: true}) instead
  of CLI flag spellings; batch help says "for example" for its field
  list; a test comment names the keys help leaves out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/mcp/server-guide.ts Resolved
@apex-sentinel-staging

Copy link
Copy Markdown

Apex Sentinel

Verdict: approve

No findings.

Reviewed with this repository's own Sentinel configuration (799e9c027625).

Feedback goes in the review comment threads, as a reply under the inline comment of a finding:

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Comment @apex-sentinel-staging review on this conversation to run a full review.

@thymikee

Copy link
Copy Markdown
Member Author

This PR is ready at 3038008. CI is green: 22 checks, none failing. There are no conflicts, and nothing else needs to happen before merge. I did not run the tests locally, and I could not verify the live claude -p simulator run in the PR body because it has no artifact. That run is not required, since the daemon receives the same positionals as before. Not blocking, and you can take or leave these: the client writers no longer unwrap an undeclared nested target after readTargetRecord was removed in common.ts, so an untyped caller sending { target: { kind: 'selector', ... } } gets the generic "requires @ref, selector, or point target" error instead of the migration message, and a note in the PR body would help. REF_ARGUMENT_TOOLS in tool-ref-pins.ts could be derived from the command metadata that declares ref, so the list cannot drift. The PR body says 39 files, but the diff touches 40.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 12, 2026
@thymikee
thymikee merged commit 0702d70 into main Nov 12, 2026
22 checks passed
@thymikee
thymikee deleted the t3/flatten-mcp-target-args branch November 12, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant