From f29738e16f8070cf5b92302a37f9c88a622191a7 Mon Sep 17 00:00:00 2001 From: dadachi Date: Sun, 24 May 2026 17:18:33 +0900 Subject: [PATCH] =?UTF-8?q?feat(plugin):=20phase=202=20=E2=80=94=20walk-ap?= =?UTF-8?q?p=20skill=20(mobile-mcp=20UI=20walkthrough)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second skill on the plugin epic: launch a generated app on a booted iOS sim / Android emulator and walk its UI conversationally via mobile-mcp — capture the home screen, list elements, tap through a flow, screenshots inline. - plugin/skills/walk-app/SKILL.md — resolve out/ + platform; get the app onto a device (recommended: a NATIVEAPPTEMPLATE_VISUAL=1 generate run does the build+install+launch; manual xcodebuild/gradlew path documented); connect mobile-mcp; capture + walk; stay interactive; diagnose the known device-layer failure modes. Honest about the booted-device + installed-build prerequisites (the flakiest link). - plugin/.mcp.json — bundle @mobilenext/mobile-mcp alongside the generator server. - generate-app SKILL.md — hand-off now points at walk-app instead of "not wired". - plugin/README.md — document both skills + both MCP servers. - plugin.json — 0.1.0 → 0.2.0. Android emulator guidance follows the house rule: boot via Android Studio Device Manager, not CLI `emulator -avd` (Studio owns adb here). Device-dependent paths are untested from here by design — needs a booted sim/emulator on the dev machine. Co-Authored-By: Claude Opus 4.7 (1M context) --- plugin/.claude-plugin/plugin.json | 4 +- plugin/.mcp.json | 4 ++ plugin/README.md | 33 +++++---- plugin/skills/generate-app/SKILL.md | 6 +- plugin/skills/walk-app/SKILL.md | 100 ++++++++++++++++++++++++++++ 5 files changed, 129 insertions(+), 18 deletions(-) create mode 100644 plugin/skills/walk-app/SKILL.md diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 191ea53..429f074 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "nativeapptemplate-agent", - "version": "0.1.0", - "description": "Generate a validated three-platform SaaS app (Rails 8.1 API + SwiftUI iOS + Jetpack Compose Android) from a one-sentence spec, then read the validation report and explain what passed and failed — without leaving Claude Code.", + "version": "0.2.0", + "description": "Generate a validated three-platform SaaS app (Rails 8.1 API + SwiftUI iOS + Jetpack Compose Android) from a one-sentence spec, explain the validation report, and walk the running app on a simulator/emulator via mobile-mcp — without leaving Claude Code.", "author": { "name": "Daisuke" }, "homepage": "https://github.com/nativeapptemplate/nativeapptemplate-agent", "repository": "https://github.com/nativeapptemplate/nativeapptemplate-agent", diff --git a/plugin/.mcp.json b/plugin/.mcp.json index 93f16aa..1ca2db7 100644 --- a/plugin/.mcp.json +++ b/plugin/.mcp.json @@ -3,6 +3,10 @@ "nativeapptemplate-agent": { "command": "npx", "args": ["-y", "-p", "nativeapptemplate-agent", "nativeapptemplate-agent-mcp"] + }, + "mobile-mcp": { + "command": "npx", + "args": ["-y", "@mobilenext/mobile-mcp@latest"] } } } diff --git a/plugin/README.md b/plugin/README.md index a6fdda8..77c6e87 100644 --- a/plugin/README.md +++ b/plugin/README.md @@ -6,30 +6,35 @@ generates a validated three-platform implementation (Rails 8.1 API + SwiftUI iOS + Jetpack Compose Android), then reads the validation report back to you in plain language. -This is the **first cut** — generate → validate → explain. A live home-screen -walk-through via `mobile-mcp` is planned as a second phase (see the repo -[roadmap](../ROADMAP.md#post-v01-backlog)). +Two skills: **generate → validate → explain**, and an interactive **walk the +running app** layer over `mobile-mcp`. ## What's in here ``` plugin/ ├── .claude-plugin/plugin.json # manifest -├── .mcp.json # bundles the nativeapptemplate-agent MCP server +├── .mcp.json # bundles the generator MCP server + mobile-mcp ├── skills/ -│ └── generate-app/SKILL.md # the orchestration skill +│ ├── generate-app/SKILL.md # generate → validate → explain +│ └── walk-app/SKILL.md # launch on a device, walk the UI with mobile-mcp └── README.md ``` -- **`/nativeapptemplate-agent:generate-app `** — the skill. Runs the - generator on your spec, parses `out//report.json`, and summarizes - per-platform / per-layer results, the domain mapping, and any failures with - the specific evidence and the next move. -- **Bundled MCP server** (`nativeapptemplate-agent`) — exposes `generate_app` as - a tool for direct/tool-call use; check it with `/mcp`. Wired as - `npx -y -p nativeapptemplate-agent nativeapptemplate-agent-mcp` (the MCP entry - point is a **bin** of the `nativeapptemplate-agent` package, not its own - package — hence `-p`). +- **`/nativeapptemplate-agent:generate-app `** — runs the generator on your + spec, parses `out//report.json`, and summarizes per-platform / per-layer + results, the domain mapping, and any failures with the specific evidence and the + next move. +- **`/nativeapptemplate-agent:walk-app ios|android`** — launches a generated + app on a **booted simulator/emulator** and drives `mobile-mcp` to capture the + home screen and walk the UI conversationally, surfacing screenshots inline. + Needs a booted device + an installed build (easiest via a + `NATIVEAPPTEMPLATE_VISUAL=1` generate run — see the skill for details). +- **Bundled MCP servers** (`/mcp` to check): the generator server + (`nativeapptemplate-agent`, wired as + `npx -y -p nativeapptemplate-agent nativeapptemplate-agent-mcp` — the MCP entry + point is a **bin** of the package, not its own package, hence `-p`) and + `mobile-mcp` (`@mobilenext/mobile-mcp`) for device automation. ## Requirements diff --git a/plugin/skills/generate-app/SKILL.md b/plugin/skills/generate-app/SKILL.md index 609d605..5a081fb 100644 --- a/plugin/skills/generate-app/SKILL.md +++ b/plugin/skills/generate-app/SKILL.md @@ -122,5 +122,7 @@ Don't just say "it failed." Pull the exact evidence and propose a fix: The generated projects are real, git-initialized, and buildable. Offer concrete follow-ups: open the report, tweak generated code, commit a project, or re-run -with overrides. (A live home-screen walk-through via `mobile-mcp` is planned as a -second phase of this skill — not wired yet.) +with overrides. If the user wants to **see the app running** — capture the home +screen, walk the UI, screenshots inline — hand off to the `walk-app` skill +(`/nativeapptemplate-agent:walk-app ios|android`), which drives `mobile-mcp` +against a booted simulator/emulator. diff --git a/plugin/skills/walk-app/SKILL.md b/plugin/skills/walk-app/SKILL.md new file mode 100644 index 0000000..e797933 --- /dev/null +++ b/plugin/skills/walk-app/SKILL.md @@ -0,0 +1,100 @@ +--- +name: walk-app +description: >- + Launch a generated app on a booted iOS Simulator or Android emulator and walk + its UI with mobile-mcp — capture the home screen, show screenshots inline, list + on-screen elements, and optionally drive a guided tap-through. Use when the + user wants to see, preview, screenshot, or interact with the running generated + app, or asks to "walk the home screen" / "show me the UI" after a generation. +--- + +# Walk a generated app's UI with mobile-mcp + +This is the interactive companion to `generate-app`. Where `generate-app` produces +and validates the projects, this skill puts a generated app **on screen** and lets +you explore it conversationally — capture the current screen, list its elements, +and tap through a flow, surfacing screenshots inline as you go. + +It depends on `mobile-mcp` (bundled with this plugin — check `/mcp`) and on a real +booted device + an installed build. Those parts are environment-dependent and the +flakiest link in the chain; be explicit with the user about what's required rather +than failing silently. + +## 1. Resolve the target + +- **Project:** a slug from `$ARGUMENTS`, else the most recently modified directory + under `./out/`. Confirm the chosen `out//` exists. +- **Platform:** `ios` or `android` from `$ARGUMENTS`. If unspecified, ask — they + need different devices and you can only drive one at a time. + +## 2. Get the app onto a device (the heavy prerequisite) + +mobile-mcp drives an app that is already **installed and running** on a **booted** +device. Two routes: + +**Recommended — let the generator build + install it.** The agent's visual mode +already does the full build → install → launch on both platforms and is the +tested path. Re-run the generator for this spec with `NATIVEAPPTEMPLATE_VISUAL=1` +(home-screen build) or `=2` (full scripted walk). After it finishes the app is +installed and launched on the booted device, and you can pick up here for +interactive exploration. + +**Manual — build it yourself** from `out//`: +- **iOS:** the scheme is the project's PascalCase name (run `xcodebuild -list` in + `out//ios/` to confirm it). Build for the iPhone 17 Pro simulator + (`xcodebuild -scheme -destination 'platform=iOS Simulator,name=iPhone 17 Pro,OS=26.2' build`), + then install/launch on the booted sim. +- **Android:** from `out//android/`, `./gradlew assembleDebug && ./gradlew installDebug`. + +Device readiness: +- **iOS:** ensure a simulator is booted (`xcrun simctl list devices | grep Booted`; + boot iPhone 17 Pro + `open -a Simulator` if none). +- **Android:** ensure an emulator is running. Boot it from **Android Studio's + Device Manager** — don't start it with the CLI `emulator -avd` on this machine + (Android Studio owns adb here). Confirm with `adb devices`. + +If the app needs live data past the welcome screen, the generated Rails API must +be running: in `out//rails/`, `mise exec -- bin/dev` (after `bundle install` ++ `db:prepare` + `db:seed_fu`). + +## 3. Connect mobile-mcp to the device + +Use mobile-mcp to list available devices and select the booted simulator/emulator. +Then list installed apps and launch the generated one (match on the project's +display name / PascalCase name) — launching by name via mobile-mcp avoids hunting +for the bundle id / package name. + +## 4. Capture and walk + +The point of this skill is showing, not narrating: +1. Take a screenshot of the current screen and **surface it inline** to the user. +2. List on-screen elements (mobile-mcp prefers the accessibility tree — more + reliable than coordinate taps) so you and the user can see what's actionable. +3. Briefly describe what's on screen in domain terms (the renamed entities, not + substrate tokens — e.g. "Barbershop list", "Add Ticket"). + +Then offer to walk a short flow. A sensible default for these apps mirrors the +validated scenario: Welcome → Sign Up → Sign In → drill into the seeded sample. +Take a fresh screenshot after each meaningful step and show it. Keep steps small +and confirm before destructive actions. + +## 5. Stay interactive + +This is exploratory, not a fixed script. After the home-screen capture, let the +user drive: "tap Sign Up", "go back", "what's on this screen?". Drive mobile-mcp +per request and return a screenshot each time. That conversational loop — generate, +then walk the running app together — is the whole reason this skill exists. + +## 6. When it fails + +The device layer fails in known ways; diagnose, don't just retry: +- **No device / mobile-mcp finds nothing** → the sim/emulator isn't booted, or + (Android) Studio isn't running and adb sees no device. Point the user at §2. +- **App not installed** → the build/install step didn't run or failed; fall back to + the recommended `NATIVEAPPTEMPLATE_VISUAL=1` route. +- **App launches but screens are blank / error** → the Rails API likely isn't + running or the app can't reach it; start it (§2) and check the + `NATIVEAPPTEMPLATE_API_*` env the app was built against. +- **Build fails** → surface the compiler output; Jetpack Compose + Hilt are the + known-cryptic ones. Offer a repair re-run of `generate-app` with + `NATIVEAPPTEMPLATE_REPAIR=on`.