Skip to content

fix(mcp): add tool titles, effect hints, and descriptive help - #4480

Merged
thymikee merged 6 commits into
mainfrom
t3/mcp-directory-metadata
Nov 13, 2026
Merged

thymikee merged 6 commits into
mainfrom
t3/mcp-directory-metadata

Conversation

@thymikee

@thymikee thymikee commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Address the MCPVetter report with titles and separate read-only/destructive hints for all 62 tools.

Command facets own required effects; titles reuse existing summaries. Read changes no device, app, session, or host state: no boot, foreground change, or artifact write. apps and appstate are writes because their runtime declarations include readiness operations. Arbitrary app actions and forced script replacement remain destructive. Idempotence stays at MCP’s default.

Example tools/list entry:

{"name":"apps","title":"List installed apps or deferred provider app assets","annotations":{"readOnlyHint":false,"destructiveHint":false}}

Descriptions state behavior rather than workflows. 36 files; declarations, MCP projection, shared help, docs, and tests only. No linked GitHub issue. Rebase preserves main’s relaunch, orientation, and verification guidance.

Validation

Head: 7824e9914398e76814e0550de61bd0b985553444; base: efe6bb9eece4794e4cc7f4e62ea22f3e75569b56.

pnpm check:affected --run --head HEAD: passed, exactCommit=true, checkoutChanged=false. Unit: 19,705 passed, 3 skipped. Related: 372 passed. Readiness regression: prior head a18f192512 plus test → 2 failures; current head → 2 passes.

Read-only guards inspect required/preferred/conditional runtime uses for readiness/boot operations and reject recorded app mutations. Focused tests: 551 passed. Stdio smoke: both protocols/output modes.

iOS smoke rerun passed at prior head a18f192512. Current-head hosted CI remains separate and owns provider/platform/coverage lanes. No device execution changes.

@apex-sentinel-staging

Copy link
Copy Markdown

Apex Sentinel could not review commit 6baf06aa. No review ran, so there are no findings; this is not a verdict on the change.

Apex Sentinel hit an internal error and could not finish this review. This is a problem on Sentinel's side, not with the change; if it keeps happening, tell whoever runs Sentinel for this repository. Push a new commit or comment @apex-sentinel-staging review to try again.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.25 MB 6.25 MB +453 B
Package (unpacked) 6.25 MB 6.25 MB +453 B
Package (download) 1.90 MB 1.90 MB -177 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 23.0 ms 21.7 ms -1.3 ms
CLI --help 46.5 ms 46.1 ms -0.3 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 26 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/mcp/command-tools.ts Resolved
@thymikee

Copy link
Copy Markdown
Member Author

The change looks good overall at 6baf06a, with one problem to fix: src/mcp/command-tools.ts:152 sets destructiveHint to !readOnlyHint, so snapshot, get, is, wait, screenshot, doctor and find are advertised as destructive. MCP defines destructiveHint as destructive versus additive updates, so hosts that confirm destructive tools will gate pure observations. Could each command declare its destructive effect directly, so only uninstall, reinstall, shutdown, clipboard write, settings and similar commands carry it?

Could CommandMetadata (or the registry descriptor next to recordingEffect) own a title and an effect of read, write or destructive, with listCommandTools deriving the annotations from it? That would replace the parallel table in src/mcp/tool-metadata.ts and fix the inversion in one place. First decide whether MCP read-only means "observes the app" (a registry trait) or "writes no artifacts and does not foreground"; if the latter, the registry needs a declared trait for that rather than an MCP-side override.

Not blocking: no tool advertises idempotentHint (devices, apps and help are idempotent, and fill replaces text), so either derive it from the same effect declaration or note in the PR body that it stays at the default; you can take or leave this.

On the open inline threads: these still apply and are wording or hint fixes: #4480 (comment) (destructiveHint inversion, the finding above), #4480 (comment) (error overlay wording), #4480 (comment) (platform identifier wording), #4480 (comment) (alert timeout sentence). These do not apply and can be resolved: #4480 (comment) (snapshot refs text is unchanged base claim), #4480 (comment) (doctor simulator text is unchanged base), #4480 (comment) (longpress help scope expansion), #4480 (comment) (target_still_moving text is byte-identical to base), #4480 (comment) (the type already makes the table exhaustive; a check against the registry recordingEffect is the better test), #4480 (comment) (pose text unchanged base).

All 21 build and test checks pass. The only non-passing check is an external review check that runs no build or test lane, so it looks unrelated. There are no conflicts. I read the diff and the PR body but did not run the tests or the stdio smoke, did not confirm the advertised tool count (62 or 63 with help), and did not trace the iOS runner to confirm that get, is, snapshot and wait can foreground the app. Before merge, replace the destructiveHint inversion with a declared per-command effect and answer the four open wording threads.

@thymikee

Copy link
Copy Markdown
Member Author

This branch now conflicts with main at 6baf06a. Please rebase, and while rebasing address the earlier review: derive the MCP annotations from one per-command effect declaration so read-only tools are not advertised as destructive.

@thymikee
thymikee force-pushed the t3/mcp-directory-metadata branch from 6baf06a to cf0a96a Compare November 13, 2026 00:15
@apex-sentinel-staging

apex-sentinel-staging Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Apex Sentinel · ✅ Approve

Since the last review: 1 new · 0 still open · 2 no longer raised (1 marked fixed)

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
packages/command-surface/src/observability/index.ts:156 🆕 logs marked destructive though it is an observation tool; only read/delete-free subcommands

Reviewed with this repository's own Sentinel configuration (efe6bb9eece4).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command What it does
@apex-sentinel-staging note <context> Adds context; Sentinel re-checks the cited code
@apex-sentinel-staging pivot <why> Says the finding is wrong; Sentinel re-checks the cited code
@apex-sentinel-staging like Marks the finding as useful
@apex-sentinel-staging dislike <why> Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

Earlier reviews

Review of a18f1925 · 13.11.2026 00:33 UTC

Apex Sentinel

Verdict: approve

Since the last review: 2 new · 0 still open · 0 no longer raised

2 finding(s): 2 minor.

🟡 Minor (2)

Where Finding
packages/command-surface/src/interaction/index.ts:584 🆕 get/is/wait/snapshot advertised non-read-only though their text says they change nothing
packages/command-surface/src/metro/index.ts:131 🆕 metro reload marked destructive but cannot replace data; inconsistent with hardware button press

1 finding below minor not shown (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (b7ded9cfb269).

Review of cf0a96af · 13.11.2026 00:22 UTC

Apex Sentinel

Verdict: approve

No findings.

Reviewed with this repository's own Sentinel configuration (b7ded9cfb269).

@thymikee
thymikee force-pushed the t3/mcp-directory-metadata branch from cf0a96a to a18f192 Compare November 13, 2026 00:26
@thymikee

Copy link
Copy Markdown
Member Author

Addressed in a18f192512e26be103ab8e77d37336547ff93757 and rebased onto b7ded9cfb2.

  • Command metadata now owns required read, write, or destructive effects and titles derived from existing summaries. The parallel MCP table and destructive/read-only inversion are removed.
  • Observation/evidence tools are non-destructive writes. find retains a destructive hint for its default click action; session retains one because save-script --force replaces existing scripts. Read-only excludes artifact writes and foreground changes. Idempotence stays at MCP’s default.
  • Corrected warning/error-overlay, platform/name/available-identifier, and alert absence/wait/pre-activation/post-press wording.
  • Replied to and resolved all 10 inline threads. The remaining scope/completeness requests have individual dispositions; the completeness check now includes the registry mutation invariant without duplicating a classification table.

The rebase preserves main’s newer wait-text, snapshot, and verified iOS back-swipe contracts. A fresh merge-tree check is clean.

Validation at the published head: affected gate passed with exactCommit=true, checkoutChanged=false; 19,557 unit tests passed (3 skipped), 369 related tests passed, and all 550 focused tests passed. Built stdio smoke passes both protocol versions and output modes. The isolated destructive-hint regression fails twice on the previous published head and passes twice here. Hosted CI remains separate; no device execution changed.

@blacksmith-sh

This comment has been minimized.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 2 new findings on a18f192. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. The conflict from the earlier review (#4480 (comment)) is gone and no conflicts remain, but one defect remains at a18f192.

apps is declared effect: 'read' in management/app.ts, so tools/list advertises readOnlyHint: true. On the production route, apps resolves a device with androidAvdSelection: 'include-stopped' (registry.ts:363) and then calls ensureAppsRuntimeReady (inventory.ts:382). On Android this is ensureAndroidReady, the same function as bootTarget (runtime.ts:499-511), and it launches a stopped emulator. Apple ensureReady is used the same way for simulators. An MCP host that auto-approves read-only tools would run apps without confirmation and boot an emulator or simulator, which changes the device's environment. That contradicts MCP's readOnlyHint. The new test locks in the wrong hint, because apps is in its read-only list.

Please declare apps as 'write' and remove it from that list. Please also state the rule in the CommandEffect doc comment and in agent-setup.md: read means the command changes no device, app, session or host state (no boot, no foreground change, no artifact write). Then enforce it in the router test from the registry: a read tool's platformExecution uses must not include ensureReady or boot operations. The current cross-check, recordingEffect !== 'mutates-app', cannot see a device boot. That is how this shipped.

Not blocking: the read check in router-tool-metadata.test.ts:27 can also require deviceClaimPolicy: 'observe', as part of the same change. Take it or leave it.

On the open inline threads: two review threads do not apply, so please resolve them. The readOnlyHint: false on get/is/wait/snapshot is the conservative MCP default, and activating the app is a foreground change that agent-setup.md already excludes from read (#4480 (comment)). A destructive hint on metro reload is defensible because it discards connected apps' JS state, and hardware-button write is a judgment call (#4480 (comment)). The other review threads are fixed or benign, so please resolve those too. The exception is the required-effect-field thread (#4480 (comment)), whose test cross-check is the one the fix above strengthens.

Smoke Tests (ios) fails at agent-device back with BACK_SWIPE_NO_CHANGE. This diff touches only command-surface metadata, help text, the MCP tools/list projection, docs and tests, so it does not overlap the back route. #4501 and #4502 passed iOS smoke after #4460, which points to flakiness. A rerun should confirm it.

I did not run the router test or the stdio smoke. I confirmed the Android boot path for apps in source and did not trace the Apple one. Before merge, apps needs the write fix and the registry-based test check, plus a green iOS smoke rerun.

@thymikee

Copy link
Copy Markdown
Member Author

This branch now conflicts with main after today's merges. The finding from the review at a18f192 still applies (#4480 (comment)): apps advertises readOnlyHint: true but can boot a stopped emulator or simulator. Could you rebase onto main and fix that in the same push? I will re-check the rebased head.

@thymikee
thymikee force-pushed the t3/mcp-directory-metadata branch from a18f192 to 7824e99 Compare November 13, 2026 09:53
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the latest review and conflicts together in 7824e9914398e76814e0550de61bd0b985553444, rebased onto efe6bb9eec.

  • apps is now write. The same registry invariant exposed appstate’s readiness operation, so it is also write. Both advertise readOnlyHint: false, destructiveHint: false.
  • CommandEffect and agent-setup.md define read as changing no device, app, session, or host state: no boot, foreground change, or artifact write.
  • The router test rejects ensureReady, bootTarget, and bootTargetHeadless in required, preferred, or conditional runtime uses of a read-only tool. The existing recorded-app-mutation cross-check stays. Isolated regression: the current test fails twice on a18f192512 for apps’ readiness operation and passes twice at head. During the fix, it also caught appstate after apps was corrected.
  • Replied to/resolved the observation and metro threads; added the strengthened invariant evidence to the earlier completeness thread.

The rebase preserves main’s relaunch, orientation, and verification guidance. A fresh merge-tree check was clean.

Exact-head affected gate: passed (exactCommit=true, checkoutChanged=false); 19,705 unit tests passed (3 skipped), 372 related tests passed, 551 focused tests passed. Built stdio smoke passes both protocols/output modes with all 62 tools.

The requested iOS smoke rerun passed on unchanged a18f192512 (Apple workflow attempt 2, including the aggregate). That is diagnostic evidence for the earlier failure; fresh CI for the new head remains separate.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 7824e99. The verdict and the full list are in the summary comment.

Comment on lines 156 to 161
export const logsCommandFacet = defineCommandFacet({
name: LOGS_COMMAND_NAME,
effect: 'destructive',
text: {
summary: 'Manage session app logs',
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆕 🟡 Minor · logs marked destructive though it is an observation tool; only read/delete-free subcommands

logs is published with destructiveHint: true, so a client gating on that hint refuses plain log reads via logs path, logs doctor, and logs mark—all read-only/diagnostic subcommands. The sibling evidence tools in the same file use write/read effects, and the PR's own rule in agent-setup.md limits destructive to tools that "replace or delete existing data, or dispatch arbitrary app actions." The new router-tool-metadata test does not cover logs, so nothing pins either choice.

⚠️ Check this patch before you commit it: the review doubted it would be right as written.

Suggested change

Either mark logs as 'write' like the other evidence tools, or justify the destructive classification in agent-setup.md and add 'logs' to the expected-annotations list in router-tool-metadata.test.ts.

export const logsCommandFacet = defineCommandFacet({
  name: LOGS_COMMAND_NAME,
  effect: 'write',
  text: {
    summary: 'Manage session app logs',
Details

Also raised here

Also raised at this location by persona javascript: logs, a log/evidence tool, advertises destructiveHint: true, contradicting the new doc contract — A client that gates destructive calls asks the user to approve logs ("Session app log info, start/stop streaming, diagnostics, and markers"), i.e. reading the session app log, while network, perf, record and trace — the same evidence class — do not. The facet declares effect: 'destructive', which listCommandTools projects to destructiveHint: true. This contradicts both the PR description ("Observation/evidence tools are non-destructive writes") and the sentence this PR adds to website/docs/docs/agent-setup.md: "Observation and evidence tools that can cause these changes declare a write effect without a destructive hint." Either the effect is wrong for logs, or the doc's rule needs the logs exception stated.

persona: senior-developer, javascript

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

@thymikee

Copy link
Copy Markdown
Member Author

No code blockers found at 7824e99: apps and appstate now advertise write effects, and the registry-based test checks read hints against declared runtime operations. CI has no failures (20 successful checks, one skipped and one neutral). The earlier conflict is resolved.

The open logs thread does not apply: logs clear [--restart] deletes the session app log, so the destructive hint is accurate; the earlier registry test finding is fixed at this head.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 13, 2026
@thymikee
thymikee merged commit 38b40be into main Nov 13, 2026
22 checks passed
@thymikee
thymikee deleted the t3/mcp-directory-metadata branch November 13, 2026 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant