Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions scripts/help-conformance-cases.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,10 @@ export const CASES = [
id: 'capturesStrongIssueEvidence',
pattern: /\b(?:screenshot\b[^\n]*--overlay-refs|record\s+start\b|logs\s+mark\b)/i,
},
// help dogfood "Platform checks": the plan must carry an `agent-device
// audit` line anywhere, so an accessibility pass that only eyeballs
// screenshots fails the case. The matcher checks presence, not order.
{ id: 'runsPlatformAudit', pattern: /(?:^|\n)agent-device\s+audit\b/i },
],
},
{
Expand Down
2 changes: 2 additions & 0 deletions skills/agent-device/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ That starts the session and returns the initial interactive snapshot with `@refs

Loop: act with `press|click|fill|longpress <target> ... --settle`, `scroll <direction> --settle`, or `back --settle`; continue from the printed diff, verify the named expectation with a read-only command (`wait text "..."`, `is`, or `get`; bare `find` clicks, so check with `find text "..." exists`), then run `agent-device close`, or `agent-device close --shutdown` if your session booted a local simulator or emulator, unless the user asks to keep it running.

On the first screen, and after a screen changes meaningfully, run `agent-device audit` when the target supports snapshots. It reports advisory findings and prints each finding's ref pinned (`@eN~sG`), so press that ref.

Reaching an off-screen target is one command, not a scroll-and-check loop: `scroll down --until <selector>` scrolls until that element is on screen, and `scroll bottom` runs to the end of the content. Repeated bare `scroll down` calls are the slow way to find something.

Copy refs byte-for-byte: `@e12`, `@e12~s4` — keep the `@` and any `~sN`. Prefer current refs, then `id`/`label`/`role` selectors; coordinates are a last resort. If snapshot reports sparse/AX-unavailable, its refs and selectors are invalid: run `agent-device screenshot`, inspect the image, use coordinates, then retry `snapshot -i` after navigating. Otherwise run `snapshot -i` only when the diff lacks the next target.
Expand Down
2 changes: 2 additions & 0 deletions skills/dogfood/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,6 @@ agent-device help dogfood

Loop: open app -> snapshot -i + screenshot -> explore flows -> capture evidence per issue -> close (`close --shutdown` if your session booted a local simulator or emulator, unless the user asks to keep it running).

On the first screen, and after a screen changes meaningfully, run `agent-device audit` when the target supports snapshots. Report its findings as accessibility findings.

Target app is required; infer platform or ask. Findings must come from runtime behavior, not source reads. Let `help dogfood` provide exact report shape, evidence commands, and current workflow guidance.
2 changes: 2 additions & 0 deletions src/commands/schema/cli-help-first-screen.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ const FIRST_SCREEN_RULES: Record<string, readonly string[]> = {
'Find user-visible issues from runtime behavior. Do not read app source or invent findings from code',
'1. Identify target app/platform; ask only if missing',
'Keep stateful commands serial within the same session',
'On the first screen, and after a screen changes meaningfully, run audit, then snapshot -i',
'Follow Apple',
'React Native warning/error overlays can be real findings or test blockers. Capture them, use react-native dismiss-overlay',
],
validate: [
Expand Down
9 changes: 9 additions & 0 deletions src/commands/schema/cli-help-workflows.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,15 @@ Loop:
6. Close the session and reconcile the report summary.
Keep stateful commands serial within the same session. Parallel runs can pollute text fields, focus, alerts, and navigation state.

Platform checks:
On the first screen, and after a screen changes meaningfully, run audit, then snapshot -i. Audit captures the screen itself, so it goes first. Act on the refs from that snapshot, or on the pinned refs (@eN~sG) audit prints. Report audit's findings as accessibility findings.
On iOS, test Back and the edge-back swipe where a screen was pushed. On Android, test system Back and applicable sheet dismissal.
Where a sheet is dismissible, swipe it down with no edits and expect dismiss. Dismiss it after an edit and expect confirm or save. Then press explicit Cancel.
When the app autosaves, discards explicitly, requires the flow, or presents the surface as nondismissible, skip that probe.
Check that each destructive action confirms or offers undo, and that every swipe action is reachable another way.
Check each field's return key against its task, for example in a screenshot with the keyboard up. Exercise each control once with press <ref-or-selector> --settle, using reversible test data.
Follow Apple (https://developer.apple.com/design/human-interface-guidelines/) and Material (https://m3.material.io/foundations/designing/structure) guidance for further platform conventions.

Coverage:
Navigation, forms, empty/error/loading states, offline or retry behavior, permissions, settings, accessibility labels, orientation/keyboard, and obvious performance stalls.
React Native warning/error overlays can be real findings or test blockers. Capture them, use react-native dismiss-overlay if unrelated, re-snapshot, and report them.
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/agent-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Add this rule to your agent client's project rules or custom instructions. [Set
Use agent-device only for app/device automation tasks.
For a normal app-driving task, start immediately. Do not probe first with `--help`, `--version`, `devices`, `appstate`, `snapshot`, or `screenshot`; open the requested app in the foreground and continue from its initial interactive snapshot.
For TV, Fire TV, or Vega OS tasks, read `agent-device help tv`.
For exploratory QA, read `agent-device help dogfood`.
For exploratory QA, read `agent-device help dogfood`; its platform checks run `agent-device audit` on the first screen and after a screen changes meaningfully, when the target supports snapshots. Report audit's findings as accessibility findings.
For logs, network, audio, traces, or runtime failures, read `agent-device help debugging`.
For React Native component internals (tree, props, state, hooks) and performance profiling (slow renders, rerenders, and their causes), read `agent-device help react-devtools`.
For React Native JavaScript heap growth, heap snapshots, allocation hotspots, or retained-object leaks, read `agent-device help cdp`.
Expand Down
Loading