diff --git a/CLAUDE.md b/CLAUDE.md index 48034223..04a7ae0b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,8 +1,10 @@ # CLAUDE.md -Mobile app automation for AI agents and humans: native apps on Android -emulators and phones, iOS simulators and iPhones, WebViews included — one Go -binary, driven from a CLI, an MCP server, or five language clients. +Mobile app automation for AI agents and humans: native, hybrid and +cross-platform apps, and pages in a mobile browser or installed as a PWA, on +Android emulators and phones, iOS simulators and iPhones — one Go binary, +driven from a CLI, an MCP server, or five language clients. docs/APP-TYPES.md +says which app types were driven and how; a claim about one goes there too. ## Key Docs @@ -11,7 +13,7 @@ binary, driven from a CLI, an MCP server, or five language clients. - docs/ARCHITECTURE.md — every layer, and one call traced end to end - docs/API.md and docs/FLAGS.md — every tool and argument, generated - docs/CHALLENGES.md — every defect and what it taught. Read before asserting platform behavior -- docs/decisions/ — the design records; docs/guides/ — how to use each surface +- docs/guides/ — how to use each surface; examples/drivers/PROTOCOL.md — the driver protocol - docs/ROADMAP.md — what is next ## Tech Stack diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 779d14a8..00d84240 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,7 +16,7 @@ Anderson's "The Three-Cornered Wheel" — and the rule applied across the codeba - `internal/mobiumdriver` is the only platform-specific layer. Everything above it is written once. A third party adds a backend as a **driver process** - ([decisions/0003](docs/decisions/0003-drivers-are-processes-not-plugins.md)), + ([the driver protocol](examples/drivers/PROTOCOL.md)), not by importing this package. - `internal/device` finds, launches and readies the thing being automated. - The CLI parses flags and calls a tool by name. Behavior lives in @@ -60,7 +60,7 @@ old binary. - **Every error has a code.** Create one with `mobiumerr.New(mobiumerr., ...)`, never a bare `fmt.Errorf` or `errors.New`. The codes are public API, the same in every client and in the CLI's exit status - ([decisions/0005](docs/decisions/0005-errors.md)). + ([the codes](docs/guides/cli.md#6-when-a-command-fails)). - A tool that fails returns `isError` with an explanation, never a JSON-RPC error. An error that suggests a remedy must suggest one that can work. diff --git a/README.md b/README.md index 6a651a57..6587e59d 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ *Mutatis mutandis.* -**Native app automation for AI agents and humans.** Android emulators, Android +**Mobile app automation for AI agents and humans.** Android emulators, Android phones, iOS simulators and iPhones, driven through one tool layer from a single Go binary with no runtime dependencies. @@ -12,7 +12,7 @@ Go binary with no runtime dependencies. mobium launch com.example.shop && mobium map && mobium tap @e5 && mobium map ``` -Mobium drives native mobile apps the way [Vibium](https://github.com/VibiumDev/vibium) +Mobium drives mobile apps the way [Vibium](https://github.com/VibiumDev/vibium) drives browsers: a `map` → `@ref` → act loop that an agent can follow without learning a new model. It is Vibium's architecture with the browser swapped for a device. @@ -25,6 +25,7 @@ person driving an emulator for it. People get the same commands. - [Quick start](#quick-start) - [How it works](#how-it-works) - [Platforms](#platforms) +- [App types](#app-types) - [Hybrid apps and WebViews](#hybrid-apps-and-webviews) - [Front doors](#front-doors) - [Drivers for other platforms](#drivers-for-other-platforms) @@ -188,13 +189,32 @@ simulated location — are refused on a phone with the reason; accessibility settings go through the phone's own Settings app. [docs/SETUP.md](docs/SETUP.md#ios-real-device) has the steps. -**Windows is not supported yet.** Everything cross-compiles for Windows, and -the daemon transport is written but not yet verified on a Windows machine; see +**Windows is not supported yet.** Everything that needs no device, the +named-pipe daemon transport included, passes in CI on a Windows runner; no +emulator or phone has been driven from Windows yet. See [docs/WINDOWS.md](docs/WINDOWS.md). Both device-side agents are pinned by version, verified against checksums compiled into the binary, and refused on a mismatch. +## App types + +What an app is built with decides where its elements are, so it decides what +Mobium reaches. Each type has been driven on a device: + +| Type | How Mobium reaches it | Driven here | +| --- | --- | --- | +| **Native** — Android SDK, Jetpack Compose, UIKit, SwiftUI | the platform's accessibility tree | Settings, Calculator, Clock, Wikipedia, F-Droid, Aegis, Seal (Compose), NetNewsWire | +| **Hybrid** — a native shell around WebViews | the shell through the tree; each WebView as its own context, [below](#hybrid-apps-and-webviews) | Wikipedia's articles, MobiumApp's web screens | +| **Mobile web and PWAs** — a page in a browser, or installed to the home screen | the browser's page as a context; on iOS, taps through the native tree | Safari, Chrome, Squoosh as a PWA — on Android a PWA's taps only while Chrome reports its WebView | +| **Cross-platform, native once removed** — React Native | an ordinary native app: it renders real native views | MobiumApp, on both platforms and on real phones | +| **Cross-platform that paints** — Flutter | not through the tree; a job for a [driver](#drivers-for-other-platforms) | not yet | + +A browser is reachable but not managed — no tabs, no browser sessions; testing +a website is [Vibium](https://github.com/VibiumDev/vibium)'s job. +[docs/APP-TYPES.md](docs/APP-TYPES.md) has what was measured for each type, +what is checked in and what was driven only once. + ## Hybrid apps and WebViews A hybrid app's web content is not usable through the accessibility tree, so @@ -218,9 +238,8 @@ tapped @e2 at (462, 397) in WEBVIEW_com.example mapped by a script, its CSS coordinates are converted to device pixels using the WebView's on-screen frame and the page's visual viewport, and the tap is delivered by the native driver like any other. Android WebViews speak CDP; iOS -WKWebViews speak WebKit's Remote Web Inspector -([decisions/0001](docs/decisions/0001-cdp-not-webdriver-bidi.md), -[0002](docs/decisions/0002-ios-webviews-are-reachable.md)). +WKWebViews speak WebKit's Remote Web Inspector, over a Unix socket on a +simulator and through usbmuxd and lockdown on a phone. A WebView is reachable only if the app opted in — `WebView.setWebContentsDebuggingEnabled(true)` on Android, `isInspectable` on @@ -323,8 +342,8 @@ A driver needs three methods to be useful — `snapshot`, `screenshot` and `tap` — and inherits locators, `@ref`s, waiting, scrolling, the CLI, MCP and all five clients. [examples/drivers/](examples/drivers/) has the guide and a complete driver in dependency-free Python that produces the same map as the -built-in backend; [decisions/0003](docs/decisions/0003-drivers-are-processes-not-plugins.md) -is the protocol. +built-in backend; [PROTOCOL.md](examples/drivers/PROTOCOL.md) is the +protocol. ## Documentation @@ -348,7 +367,7 @@ is the protocol. | [docs/WINDOWS.md](docs/WINDOWS.md) | the state of Windows support | | [docs/ROADMAP.md](docs/ROADMAP.md) | what is next | | [docs/RELEASE-CHECKLIST.md](docs/RELEASE-CHECKLIST.md) | the device checks CI cannot run | -| [docs/decisions/](docs/decisions/) | architecture decision records | +| [examples/drivers/PROTOCOL.md](examples/drivers/PROTOCOL.md) | the protocol a third-party driver speaks | | [docs/checks/](docs/checks/) | end-to-end scripts that drive a real device | | [examples/drivers/](examples/drivers/) | writing a driver | | [CONTRIBUTING.md](CONTRIBUTING.md) | building, testing and the project's rules | @@ -356,8 +375,8 @@ is the protocol. Several checks drive **[MobiumApp](https://github.com/mobiumdev/mobium-app)**, a React Native app built to be driven: each screen is a control for a case that can go wrong, and on iOS it is the only way to reach an app's WebView, -since one that has not opted into inspection is invisible to any debugger -([decisions/0004](docs/decisions/0004-an-app-under-test-of-our-own.md)). +since one that has not opted into inspection (`isInspectable`) is invisible +to any debugger, and that cannot be forced from outside. ## License diff --git a/assets/branding/README.md b/assets/branding/README.md index bd121b46..8946fc27 100644 --- a/assets/branding/README.md +++ b/assets/branding/README.md @@ -134,7 +134,7 @@ Wheel", is in [docs/PHILOSOPHY.md](../../docs/PHILOSOPHY.md). Set it in italics, capitalized as a sentence — *Mutatis mutandis* — and do not translate it inline in display use; the gloss belongs in body text, as above. It pairs with the mark rather than replacing the descriptive line: the mark -says what it is called, "native app automation for AI agents and humans" says +says what it is called, "mobile app automation for AI agents and humans" says what it does, and the motto says how it is built. ## Namespace diff --git a/clients/dotnet/Mobium/Device.cs b/clients/dotnet/Mobium/Device.cs index a38c6f31..a102df85 100644 --- a/clients/dotnet/Mobium/Device.cs +++ b/clients/dotnet/Mobium/Device.cs @@ -5,7 +5,7 @@ namespace Mobium { /// - /// Drives native apps on Android emulators, Android phones, iOS + /// Drives mobile apps on Android emulators, Android phones, iOS /// simulators and iPhones. /// /// diff --git a/clients/dotnet/Mobium/Mobium.csproj b/clients/dotnet/Mobium/Mobium.csproj index 10f3fc7c..164be8a1 100644 --- a/clients/dotnet/Mobium/Mobium.csproj +++ b/clients/dotnet/Mobium/Mobium.csproj @@ -29,7 +29,7 @@ Mobium 0.1.0 Lana Begunova - Drives native apps on Android emulators, Android phones, iOS simulators and iPhones, over the mobium CLI. No dependencies. + Drives mobile apps on Android emulators, Android phones, iOS simulators and iPhones, over the mobium CLI. No dependencies. Copyright (c) 2026 Lana Begunova MIT https://github.com/mobiumdev/mobium diff --git a/clients/dotnet/Mobium/MobiumException.cs b/clients/dotnet/Mobium/MobiumException.cs index cd88c965..c2929ac1 100644 --- a/clients/dotnet/Mobium/MobiumException.cs +++ b/clients/dotnet/Mobium/MobiumException.cs @@ -9,7 +9,8 @@ namespace Mobium /// /// /// Every tool failure carries a stable , the same in every - /// client and on the wire (docs/decisions/0005 in the mobium repository), + /// client and on the wire (the error codes in the mobium repository's + /// docs/guides/cli.md), /// and each code has a subclass — , /// and so on — so a test catches the kind /// it can handle and lets the rest through. says what diff --git a/clients/dotnet/README.md b/clients/dotnet/README.md index 7b714576..5543069c 100644 --- a/clients/dotnet/README.md +++ b/clients/dotnet/README.md @@ -1,6 +1,6 @@ # Mobium for .NET -Drives native apps on Android emulators, Android phones, iOS simulators and +Drives mobile apps on Android emulators, Android phones, iOS simulators and iPhones from C#. ```csharp @@ -81,7 +81,7 @@ code, `Remedy` what to do about it, `Retryable` whether a retry can help, and Each code has its own subclass — `NoSuchElementException`, `AmbiguousLocatorException`, `UnsupportedException`, `NotConfirmedException`, `TimedOutException` (not `TimeoutException`, which is .NET's) and the rest, -the same set in every client (`docs/decisions/0005-errors.md`) — so a test +the same set in every client ([the codes](../../docs/guides/cli.md#6-when-a-command-fails)) — so a test catches the kind it can handle: ```csharp diff --git a/clients/dotnet/Tests/Program.cs b/clients/dotnet/Tests/Program.cs index 6b00379e..ff582b02 100644 --- a/clients/dotnet/Tests/Program.cs +++ b/clients/dotnet/Tests/Program.cs @@ -738,7 +738,7 @@ private static void TraceSendsOnlyWhatWasGiven() // -- the harness ---------------------------------------------------- - // -- error codes: docs/decisions/0005 ----------------------------- + // -- error codes: docs/guides/cli.md ------------------------------- private static readonly string[] Codes = {"no_device", "device_not_ready", "toolchain_missing", "no_such_element", "ambiguous_locator", "element_not_reachable", "no_such_context", diff --git a/clients/go/README.md b/clients/go/README.md index 2c2282c4..f04c5290 100644 --- a/clients/go/README.md +++ b/clients/go/README.md @@ -1,6 +1,6 @@ # Mobium for Go -Drive native apps on Android emulators, Android phones, iOS simulators and +Drive mobile apps on Android emulators, Android phones, iOS simulators and iPhones — map the screen, tap, type, wait and screenshot — through the same tools as the [mobium](https://github.com/mobiumdev/mobium) command line and MCP server. Standard library only. diff --git a/clients/go/errors.go b/clients/go/errors.go index 20a990a7..cbc49b40 100644 --- a/clients/go/errors.go +++ b/clients/go/errors.go @@ -6,8 +6,8 @@ import ( ) // Code classifies a failure. The strings are Mobium's stable error codes, -// the same in every client and on the wire; see docs/decisions/0005 in the -// mobium repository. A client is not required to know every code: one it +// the same in every client and on the wire; see the error codes in the +// mobium repository's docs/guides/cli.md. A client is not required to know every code: one it // does not recognize is still an *Error, with Code set. type Code string @@ -43,8 +43,7 @@ var ( ErrUnsupported = &Error{Code: CodeUnsupported} ErrNotConfirmed = &Error{Code: CodeNotConfirmed} // TimedOut, not Timeout, as in every client: Python's TimeoutError and - // Java's TimeoutException are builtins a Timeout name would shadow - // (docs/decisions/0005). + // Java's TimeoutException are builtins a Timeout name would shadow. ErrTimedOut = &Error{Code: CodeTimeout} ErrInvalidArgument = &Error{Code: CodeInvalidArgument} ErrDeviceServer = &Error{Code: CodeDeviceServer} diff --git a/clients/go/mobium.go b/clients/go/mobium.go index e780202f..bed874a4 100644 --- a/clients/go/mobium.go +++ b/clients/go/mobium.go @@ -1,4 +1,4 @@ -// Package mobium drives native apps on Android emulators, Android phones, iOS +// Package mobium drives mobile apps on Android emulators, Android phones, iOS // simulators and iPhones. // // It speaks to the same tool layer the CLI and the MCP server use, over diff --git a/clients/java/README.md b/clients/java/README.md index 563b90b5..f7ea25ed 100644 --- a/clients/java/README.md +++ b/clients/java/README.md @@ -1,6 +1,6 @@ # Mobium Java client -Drives native apps on Android emulators, Android phones, iOS simulators and +Drives mobile apps on Android emulators, Android phones, iOS simulators and iPhones, over the same tool layer the CLI and the MCP server use. ```java @@ -174,8 +174,8 @@ try { `TimedOutException` (not `TimeoutException`, which is Java's), `InvalidArgumentException`, `DeviceServerException`, `InternalException`. Import `dev.mobium.NoSuchElementException` by name: it shares its simple name -with `java.util`'s, as Selenium's does. The codes are the same in every client; -see `docs/decisions/0005-errors.md`. +with `java.util`'s. The codes are the same in every client; +see [the codes](../../docs/guides/cli.md#6-when-a-command-fails). ### Threads, timeouts and closing diff --git a/clients/java/pom.xml b/clients/java/pom.xml index fddcdfc3..329985b0 100644 --- a/clients/java/pom.xml +++ b/clients/java/pom.xml @@ -16,7 +16,7 @@ jar Mobium Java client - Drives native apps on Android emulators, Android phones, iOS simulators and iPhones, over the mobium CLI. No dependencies. + Drives mobile apps on Android emulators, Android phones, iOS simulators and iPhones, over the mobium CLI. No dependencies. https://github.com/mobiumdev/mobium 2026 diff --git a/clients/java/src/main/java/dev/mobium/Mobium.java b/clients/java/src/main/java/dev/mobium/Mobium.java index 77ca49b7..e9316007 100644 --- a/clients/java/src/main/java/dev/mobium/Mobium.java +++ b/clients/java/src/main/java/dev/mobium/Mobium.java @@ -11,7 +11,7 @@ import java.util.Map; /** - * Drives native apps on Android emulators, Android phones, iOS simulators + * Drives mobile apps on Android emulators, Android phones, iOS simulators * and iPhones. * *

Speaks to the same tool layer the CLI and the MCP server use, over diff --git a/clients/java/src/main/java/dev/mobium/MobiumException.java b/clients/java/src/main/java/dev/mobium/MobiumException.java index 1ab8f5d9..f76f784b 100644 --- a/clients/java/src/main/java/dev/mobium/MobiumException.java +++ b/clients/java/src/main/java/dev/mobium/MobiumException.java @@ -7,8 +7,8 @@ * Mobium failed. * *

Every tool failure carries a stable {@link #code() code}, the same in - * every client and on the wire (docs/decisions/0005 in the mobium - * repository), and each code has a subclass — {@link NoSuchElementException}, + * every client and on the wire (the error codes in the mobium + * repository's docs/guides/cli.md), and each code has a subclass — {@link NoSuchElementException}, * {@link UnsupportedException} and so on — so a test catches the kind it can * handle and lets the rest through. {@link #remedy()} says what to do about * it, {@link #retryable()} whether the same call can succeed if made again. diff --git a/clients/java/src/main/java/dev/mobium/NoSuchElementException.java b/clients/java/src/main/java/dev/mobium/NoSuchElementException.java index cad2a4b5..4a5541ad 100644 --- a/clients/java/src/main/java/dev/mobium/NoSuchElementException.java +++ b/clients/java/src/main/java/dev/mobium/NoSuchElementException.java @@ -5,8 +5,8 @@ /** * A locator or ref matched nothing on screen. Worth scrolling for. * - *

Shares its simple name with {@code java.util.NoSuchElementException}, as - * Selenium's does; import this one by name rather than through a wildcard. + *

Shares its simple name with {@code java.util.NoSuchElementException}; + * import this one by name rather than through a wildcard. * *

Error code {@code no_such_element}. See {@link MobiumException}. */ diff --git a/clients/java/src/test/java/dev/mobium/Tests.java b/clients/java/src/test/java/dev/mobium/Tests.java index 178f9e6a..c313f837 100644 --- a/clients/java/src/test/java/dev/mobium/Tests.java +++ b/clients/java/src/test/java/dev/mobium/Tests.java @@ -203,7 +203,7 @@ static void pathSearchSkipsRelativeDirectories() { } } - // -- error codes: docs/decisions/0005 ----------------------------------- + // -- error codes: docs/guides/cli.md ------------------------------------- static final String[] CODES = {"no_device", "device_not_ready", "toolchain_missing", "no_such_element", "ambiguous_locator", "element_not_reachable", "no_such_context", diff --git a/clients/javascript/README.md b/clients/javascript/README.md index 11a1b04b..d1452e77 100644 --- a/clients/javascript/README.md +++ b/clients/javascript/README.md @@ -1,6 +1,6 @@ # Mobium for JavaScript -Drive native apps on Android emulators, Android phones, iOS simulators and +Drive mobile apps on Android emulators, Android phones, iOS simulators and iPhones — map the screen, tap, type, wait and screenshot — through the same tools as the [mobium](https://github.com/mobiumdev/mobium) command line and MCP server. No dependencies; TypeScript declarations included. diff --git a/clients/javascript/index.js b/clients/javascript/index.js index fe9b2426..ab6e809f 100644 --- a/clients/javascript/index.js +++ b/clients/javascript/index.js @@ -1,5 +1,5 @@ /** - * Mobium — native mobile app automation on emulators, simulators and phones. + * Mobium — mobile app automation on emulators, simulators and phones. * * import { start } from 'mobium' * @@ -16,8 +16,8 @@ import { createInterface } from 'node:readline' /** * A tool reported that it could not do what was asked. Every failure carries - * a stable code, the same in every client and on the wire (docs/decisions/0005 - * in the mobium repository), and each code has a subclass below, so a script + * a stable code, the same in every client and on the wire (the error codes + * in the mobium repository's docs/guides/cli.md), and each code has a subclass below, so a script * catches the kind it can handle: * * try { await device.tap('text=Continue') } diff --git a/clients/javascript/package.json b/clients/javascript/package.json index d0c3834b..0be11e13 100644 --- a/clients/javascript/package.json +++ b/clients/javascript/package.json @@ -1,7 +1,7 @@ { "name": "mobium", "version": "0.1.0", - "description": "Drive native apps on Android emulators, Android phones, iOS simulators and iPhones. No dependencies.", + "description": "Drive mobile apps on Android emulators, Android phones, iOS simulators and iPhones. No dependencies.", "keywords": ["mobile", "automation", "testing", "android", "ios", "uiautomator2", "webdriveragent", "mcp"], "homepage": "https://github.com/mobiumdev/mobium/blob/main/docs/quickstart/javascript.md", "bugs": "https://github.com/mobiumdev/mobium/issues", diff --git a/clients/python/README.md b/clients/python/README.md index 156fc3b1..8792516e 100644 --- a/clients/python/README.md +++ b/clients/python/README.md @@ -1,6 +1,6 @@ # Mobium for Python -Drive native apps on Android emulators, Android phones, iOS simulators and +Drive mobile apps on Android emulators, Android phones, iOS simulators and iPhones — map the screen, tap, type, wait and screenshot — through the same tools as the [mobium](https://github.com/mobiumdev/mobium) command line and MCP server. No dependencies. diff --git a/clients/python/mobium/__init__.py b/clients/python/mobium/__init__.py index f4f8976c..6b11dc4b 100644 --- a/clients/python/mobium/__init__.py +++ b/clients/python/mobium/__init__.py @@ -1,4 +1,4 @@ -"""Mobium — native mobile app automation on emulators, simulators and phones. +"""Mobium — mobile app automation on emulators, simulators and phones. from mobium import start diff --git a/clients/python/mobium/_errors.py b/clients/python/mobium/_errors.py index f17cbe98..a1f66c2d 100644 --- a/clients/python/mobium/_errors.py +++ b/clients/python/mobium/_errors.py @@ -1,7 +1,8 @@ """Mobium's failures, one exception per error code. Every failure a tool reports carries a stable code — the same in every client -and on the wire (docs/decisions/0005 in the mobium repository). Each code has +and on the wire (the error codes in the mobium repository's +docs/guides/cli.md). Each code has an exception here, all subclasses of MobiumError, so a script catches the kind it can handle and lets the rest through:: diff --git a/clients/python/pyproject.toml b/clients/python/pyproject.toml index 7fe3d142..4a7c70ea 100644 --- a/clients/python/pyproject.toml +++ b/clients/python/pyproject.toml @@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta" [project] name = "mobium" version = "0.1.0" -description = "Drive native apps on Android emulators, Android phones, iOS simulators and iPhones. No dependencies." +description = "Drive mobile apps on Android emulators, Android phones, iOS simulators and iPhones. No dependencies." readme = "README.md" requires-python = ">=3.9" license = "MIT" diff --git a/cmd/mobium/grid.go b/cmd/mobium/grid.go index 97ebc5b0..aa67c4bb 100644 --- a/cmd/mobium/grid.go +++ b/cmd/mobium/grid.go @@ -20,7 +20,7 @@ import ( "github.com/spf13/cobra" ) -// A grid is Selenium Grid's shape without its hub: MOBIUM_GRID names the +// A grid has no hub: MOBIUM_GRID names the // nodes, the caller's own mobium routes each run to a free device that // matches, and the lease that keeps a device to one run lives on the node, so // callers on different machines cannot take the same one. Nothing new listens diff --git a/cmd/mobium/inspect.go b/cmd/mobium/inspect.go index 6d7db77b..d6ab8284 100644 --- a/cmd/mobium/inspect.go +++ b/cmd/mobium/inspect.go @@ -29,7 +29,7 @@ func newInspectCmd() *cobra.Command { "daemon, so a terminal and the page drive one session.\n\n" + "It listens on 127.0.0.1 only, and answers only requests carrying the token in\n" + "the URL it prints — any other page in the browser can reach 127.0.0.1. See\n" + - "docs/decisions/0007-an-inspector.md.", + "docs/guides/inspector.md.", Example: ` mobium inspect # the only device running mobium inspect --device emulator-5554 --open mobium inspect --driver wda --device `, diff --git a/cmd/mobium/lifecycle.go b/cmd/mobium/lifecycle.go index 6f96ef35..ebfcd0c2 100644 --- a/cmd/mobium/lifecycle.go +++ b/cmd/mobium/lifecycle.go @@ -20,7 +20,7 @@ func newLaunchCmd() *cobra.Command { }, } cmd.Flags().BoolVar(&hitTest, "hit-test", false, "On an iOS simulator, load the hit probe into the app as it "+ - "launches, so every tap on an element in it asks UIKit where the touch goes first (docs/decisions/0008)") + "launches, so every tap on an element in it asks UIKit where the touch goes first (docs/guides/autowait.md)") return cmd } diff --git a/cmd/mobium/main.go b/cmd/mobium/main.go index 90c20638..fbc39639 100644 --- a/cmd/mobium/main.go +++ b/cmd/mobium/main.go @@ -1,4 +1,4 @@ -// Command mobium automates native apps on virtual devices. +// Command mobium automates mobile apps on devices, virtual and real. // // Step 1 scope: Android emulators, via adb and uiautomator, with the map/@ref // model that the rest of the tool will be built on. @@ -57,7 +57,8 @@ func main() { gridActive = len(gridNodes()) > 0 return nil }, - Long: "Mobium automates native apps on Android emulators and phones, iOS simulators\n" + + Long: "Mobium automates mobile apps — native, hybrid, cross-platform and pages in a\n" + + "mobile browser — on Android emulators and phones, iOS simulators\n" + "and iPhones, using the same map/@ref workflow as vibium:\n\n" + " mobium map && mobium tap @e1 && mobium map", } diff --git a/cmd/mobium/test.go b/cmd/mobium/test.go index a410e1b5..a7a84a39 100644 --- a/cmd/mobium/test.go +++ b/cmd/mobium/test.go @@ -52,7 +52,7 @@ func newTestCmd() *cobra.Command { "testDir. With MOBIUM_GRID set, each project leases a device of its own from\n" + "the grid — by its platform, or its device — for the whole run. Exits 0 when\n" + "every test passed — a flaky one passes and is listed — and 1 when any failed.\n" + - "See docs/decisions/0006-a-test-runner.md.", + "See docs/guides/test-runner.md.", Example: ` mobium test # everything, every project mobium test tests/login.test.json mobium test -g "wrong password" @@ -245,7 +245,7 @@ func newTestCmd() *cobra.Command { f.StringVar(&outDir, "output", "", "Where reports go (default: mobium-report beside the config)") f.StringVar(&trace, "trace", "off", "Keep a screenshot and the map after every step, and the test as a recording in Vibium's record format: on, off, or retain-on-failure") f.BoolVar(&debug, "debug", false, "Stop before each step, show it and the screen, and wait: Enter steps, c continues, q quits") - f.BoolVar(&ui, "ui", false, "Serve a page on this machine to pick tests, run them and watch each step (docs/decisions/0009)") + f.BoolVar(&ui, "ui", false, "Serve a page on this machine to pick tests, run them and watch each step (docs/guides/test-runner.md)") f.IntVar(&uiPort, "ui-port", 0, "With --ui, the port on 127.0.0.1 (default: any free one)") f.BoolVar(&uiOpen, "open", false, "With --ui, open the page in the browser") return cmd @@ -368,7 +368,7 @@ func newRunID() string { } // serveTestUI is --ui: a page that runs the run this command would, any -// part of it at a time (docs/decisions/0009). +// part of it at a time. func serveTestUI(cfg *testrun.Config, args []string, opts testrun.Options, call testrun.Caller, port int, open bool) error { var projects []string for _, p := range cfg.Projects { diff --git a/docs/APP-TYPES.md b/docs/APP-TYPES.md index 57b886d2..453f91a2 100644 --- a/docs/APP-TYPES.md +++ b/docs/APP-TYPES.md @@ -19,7 +19,23 @@ category reaches a different part of Mobium. | Hybrid | **yes** | native shell through the tree; web content through CDP or Remote Web Inspector | | Cross-platform | **depends, and not on the vendor's claim** | see below — the answer splits the category in half | | Mobile web | **not supported, and partly reachable** | a browser is [Vibium's](https://github.com/VibiumDev/vibium) job; what Mobium reaches anyway is below | -| Progressive web app | **read on both, acted on in full only on Android** | the browser's web context; on iOS, taps go through the native tree | +| Progressive web app | **read on both; tapped from the native tree on iOS, and on Android only while Chrome reports its WebView** | the browser's web context; on iOS, taps go through the native tree | + +## What has been driven + +Every type has been driven here; not every framework within a type has. +"Checked in" means a script in [checks/](checks/) holds it; "measured" means +it was driven once, with the result written down below, and nothing re-runs it. + +| Type | Driven | Where | +| --- | --- | --- | +| Native | Android: Settings, Calculator, Clock, Wikipedia, F-Droid, Aegis, and Seal (Jetpack Compose). iOS: Settings, Wikipedia and NetNewsWire from the App Store | checked in — most of [checks/](checks/); `third-party-app.sh`, `compose-app.sh`, `netnewswire-ios.sh` for the apps nobody at Google or Apple wrote | +| Hybrid | Wikipedia's articles, and MobiumApp's WebView screens, on both platforms and on real phones | checked in — `third-party-app.sh`, `mobium-app.sh`, `web-type.sh`, `web-actionability.sh`, `web-storage.sh` | +| Mobile web | Safari on iOS; Chrome on Android | checked in — `chrome.sh` (read, a tap counted by the page, a link followed), `ios-webview.sh`, `orientation.sh`, `shake.sh` | +| Progressive web app | Squoosh, installed to the home screen on both platforms | checked in — `pwa.sh`: installed if absent, launched from its icon, standalone, and a tap counted by the page or refused with the reason | +| Cross-platform | React Native: MobiumApp, on both platforms and on real phones | checked in — `mobium-app.sh`, `login.sh`, `otp.sh`, `dialogs.sh` and every other MobiumApp check | +| | Flutter, Xamarin/.NET MAUI | **never driven** — see below for why Flutter is expected to need a driver | +| Hybrid frameworks | Cordova, Ionic | **never driven**; a WebView inside them is the hybrid case above | ## Native @@ -59,9 +75,12 @@ What that line does *not* mean is that Mobium cannot see a browser, and until WebView spans the chrome and the page cannot see its own inset ([CHALLENGES 47](CHALLENGES.md)). -So "not supported" is a statement about scope — no browser management, no -tabs, no cookies, no network, and nothing checked in that drives a browser — -not a wall Mobium puts up. What is reachable is reachable because a browser +So "not supported" is a statement about scope — no browser management and +no tabs — not a wall Mobium puts up. A browser's page is driven by checks all +the same: Safari's in `ios-webview.sh` and `orientation.sh`, Chrome's in +`chrome.sh` and `shake.sh`. A web context's cookies (`app_cookies`) and the device's network +conditions (`app_network`) are reachable, but they belong to the page and the +device, not to managing a browser. What is reachable is reachable because a browser is, underneath, the hybrid case below. ## Progressive web apps @@ -77,11 +96,11 @@ on both devices above. | --- | --- | --- | | What runs it | Chrome's `WebappActivity`, in `com.android.chrome` | `com.apple.webapp`, with its own bundle id `com.apple.WebKit.PushBundle.` | | Launched by id | no — use the home screen icon | no — FrontBoard does not know the bundle id; use the icon | -| The page says | `display-mode: standalone`, a service worker in control | the same, and `navigator.standalone` | +| The page says | `display-mode: standalone`, a service worker registered and active | the same, and `navigator.standalone` | | Its context | `WEBVIEW_com.android.chrome`, named for Chrome | `WEBVIEW_com.apple.SafariViewService`, named for neither the app nor Safari | | `map`, `text`, `eval` | work | work | -| A tap in the web context | lands | refused: the host is 874 points and the viewport 812, the difference being the status bar | -| A tap from `NATIVE_APP` | works | works — WebKit puts the page's controls in the accessibility tree | +| A tap in the web context | lands while Chrome reports the WebView; opened while Chrome is already running, Chrome stops reporting it within about five seconds and the tap is refused ([CHALLENGES 200](CHALLENGES.md)) | refused: the host is 874 points and the viewport 812, the difference being the status bar | +| A tap from `NATIVE_APP` | works while Chrome reports the page; once it stops, its tree is empty there too | works — WebKit puts the page's controls in the accessibility tree | Three things follow. @@ -99,9 +118,19 @@ Three things follow. and `app_storage`, once the page is on an http or https origin; an app's inline HTML has none, and is refused as that. - **Installing one is not something Mobium does.** On Android it needs - Chrome's menu, on iOS the share sheet. On this emulator Chrome pinned a - legacy web-app shortcut rather than minting a WebAPK, which needs Play - services; a WebAPK — a real APK, with its own package — is unmeasured. + Chrome's menu, on iOS the share sheet; `checks/pwa.sh` drives both. On this + emulator Chrome pinned a legacy web-app shortcut rather than minting a + WebAPK, which needs Play services; a WebAPK — a real APK, with its own + package — is unmeasured. Those shortcuts are fragile: after a cold boot, + and once after force-stopping Chrome, the launcher showed none of them + while Chrome still listed six, and Chrome then offered only a plain + shortcut, which opens a tab. +- **On Android, when the tap matters, launch the web app with Chrome not + running.** Opened from its icon while Chrome was running, Chrome stopped + reporting the app's WebView within about five seconds, three launches out + of three; opened after Chrome's process had ended, it kept reporting it, + two out of two. Mobium refuses the tap it can no longer place and says + why. ## Hybrid @@ -130,12 +159,11 @@ area — it spans the window, and no element exposes the content rectangle. That is the difference between a description that is true and one you can compute against. -Web content is reached over CDP on Android -([decisions/0001](decisions/0001-cdp-not-webdriver-bidi.md)) and Remote Web -Inspector on iOS ([decisions/0002](decisions/0002-ios-webviews-are-reachable.md)), -and on iOS the app must have opted in with `isInspectable` — which cannot be -forced from outside, and is why the app under test had to be one we control -([decisions/0004](decisions/0004-an-app-under-test-of-our-own.md)). +Web content is reached over CDP on Android, which needs no chromedriver to +match the device's Chrome, and Remote Web Inspector on iOS, and on iOS the +app must have opted in with `isInspectable` — which cannot be forced from +outside, and is why the app under test had to be one we control: +[MobiumApp](https://github.com/mobiumdev/mobium-app). Cordova and Ionic sit here. Neither has been driven. @@ -162,7 +190,7 @@ special support — and the hierarchy it produces was different enough to find engine, so there are no per-widget native views; what reaches the accessibility tree is a *synthesized* semantics tree over a single surface. This is the same shape as the Chrome case above, which is why Flutter appears in -[decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md) as +[the driver protocol](../examples/drivers/PROTOCOL.md) as something a third party would add **as a driver process**, rather than as an app type that happens to work. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index b09155f4..dcf82e93 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -13,6 +13,8 @@ flowchart TB cli["CLI
mobium map, mobium tap @e5"] clients["Language clients
Python · JavaScript · Go · Java · .NET"] mcp["MCP client
an agent"] + test["mobium test
JSON test files · --ui page"] + inspect["mobium inspect
a page showing the screen"] end pipe["mobium pipe
JSON-RPC on stdio"] @@ -22,6 +24,11 @@ flowchart TB cli --> daemon clients --> pipe --> daemon mcp --> mcpcmd + test --> daemon + test -- "a pipe per project on a grid" --> pipe + inspect --> daemon + remote["another machine's daemon
--remote, MOBIUM_GRID"] + cli & pipe -.->|"SSH socket forward"| remote subgraph core["internal/agent — the tool layer, written once"] tools["Tools and schemas
map · tap · type · wait · scroll-to · alert · …"] @@ -75,7 +82,7 @@ flowchart TB the ordinary driver. - **A third-party backend is a process**, not a Go plugin: any executable named `mobium-driver-` on `PATH`, speaking JSON-RPC - ([decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md)). + ([the driver protocol](../examples/drivers/PROTOCOL.md)). ## Packages @@ -92,25 +99,36 @@ flowchart LR driver["internal/mobiumdriver
backends"] webview["internal/webview
CDP, RWI"] formflux["internal/formflux
screen profiles"] + testrun["internal/testrun
mobium test"] + testui["internal/testui
mobium test --ui"] + inspect["internal/inspect
mobium inspect"] + grid["internal/grid
leases on a node"] + trace["internal/trace
session recording"] uitree["internal/uitree
nodes, locators, map"] device["internal/device
adb, simctl, devicectl, lockdown"] plist["internal/plist
property lists"] paths["internal/paths"] err["internal/mobiumerr
error codes"] - cmd --> agent & daemon & paths + cmd --> agent & daemon & testrun & testui & inspect & grid & device & paths daemon --> agent & paths apisurface --> agent - agent --> driver & webview & formflux & uitree & device & paths + testrun --> agent + testui --> testrun + inspect --> agent & testrun & uitree + agent --> driver & webview & formflux & uitree & device & grid & trace & paths + grid --> paths driver --> device & uitree webview --> device & uitree & plist formflux --> uitree device --> plist & paths ``` -Every package also imports `internal/mobiumerr`, left out above for -legibility: every failure carries one of its codes, which are public API -([decisions/0005](decisions/0005-errors.md)). +Nearly every package also imports `internal/mobiumerr`, left out above for +legibility; only `internal/trace`, `internal/testui` and the test helper +`internal/fakecmd` do not. Every failure carries one of its codes, which are +public API +([the codes](guides/cli.md#6-when-a-command-fails)). | Package | Responsibility | | --- | --- | @@ -123,6 +141,11 @@ legibility: every failure carries one of its codes, which are public API | `internal/device` | adb, simctl, devicectl, usbmuxd and lockdown; installing the pinned device agents | | `internal/formflux` | one device impersonating many screens ([FORMFLUX.md](FORMFLUX.md)) | | `internal/plist` | the property-list codec iOS's inspector and lockdown speak | +| `internal/testrun` | `mobium test`: JSON test files run through the tools, with retries, workers and reports | +| `internal/testui` | `mobium test --ui`: a page that runs the suite and shows each step | +| `internal/inspect` | `mobium inspect`: a page showing the screen and its elements, recording a test file | +| `internal/grid` | a grid node's device leases | +| `internal/trace` | recording a session in Vibium's record format | | `internal/apisurface` | build-time checks that every tool and flag is reachable from every front door | | `internal/paths` | socket, PID and session paths, and the OS length limit on them | | `internal/mobiumerr` | the error codes | diff --git a/docs/CHALLENGES.md b/docs/CHALLENGES.md index d92bf013..146926d5 100644 --- a/docs/CHALLENGES.md +++ b/docs/CHALLENGES.md @@ -7,7 +7,7 @@ the source. The pattern across them is the reason the document exists: **almost nothing here was found by reading code or by a test written from imagination.** Of -199 defects, 163 were found only by running against a real device. The other +201 defects, 165 were found only by running against a real device. The other thirty-six — 4, 5, 6, 14, 23, 24, 32, 35, 36, 39, 44, 50, 53, 54, 57, 66, 89, 99, 100, 121, 123, 129, 130, 133, 134, 139, 140, 141, 142, 144, 150, 158, 162, 164, 165 and 172 — came from reading code, the compiler, a test, a linter, @@ -61,8 +61,8 @@ with the evidence. | What you are seeing | What it usually is | | --- | --- | | A command "succeeded" and nothing changed | Nothing here reports failure by exit code. `pm grant`, `simctl privacy`, `am start`, `adb uninstall` and `uiautomator dump` each fail differently — 25, 26, 27, 30 | -| An element is on screen but a locator says it is not | It matched more than once and the message said "no element matches" — 33; or it resolved but sits outside the container being judged — 33 | -| A tap lands slightly wrong | The element was still moving; actions now wait for stable bounds — the settling section, and 17 for iOS points against pixels | +| An element is on screen but a locator says it is not | It matched more than once and the message said "no element matches" — 29; or it resolved but sits outside the container being judged — 33 | +| A tap lands slightly wrong | The element was still moving; actions now wait for stable bounds — 109 and 112, and 17 for iOS points against pixels | | `map` shows nothing useful, or every icon is a "link" | Role inference — 1, 4, 15, 16 | | Scrolling swipes forever, or refuses to scroll | A ticking clock defeating the progress check — 22; or a container the element is not inside — 33 | | The dump backend cannot read a screen at all | It waits for idle, and some screens never are — 25 | @@ -77,7 +77,7 @@ with the evidence. | `map` offers something that is not on screen | Visibility. Android never reports it, iOS does, and a driver may — 39 | | A measurement comes back "zero" or "no difference" | Show it can report non-zero before believing it. A probe reported 0 for four apps and had never left the launcher — 39 | | An app "launched" and the screen did not change | `monkey` is a fuzzer, not a launcher: chatter on stdout, exit 0 for a package that does not exist — 37. Resolve the activity and `am start -n` it | -| A third-party driver's answer is ignored | Only advertised capabilities are wired up; `mobium doctor` lists drivers found, and the refusal names the backend — [decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md) | +| A third-party driver's answer is ignored | Only advertised capabilities are wired up; `mobium doctor` lists drivers found, and the refusal names the backend — [the driver protocol](../examples/drivers/PROTOCOL.md) | --- @@ -753,7 +753,7 @@ Mobium** — an independent implementation of the same approach hit the same wall in the same place. And **the protocol carried the diagnosis intact**: the driver's own sentence, including its advice to use `--driver uiautomator2`, reached the user verbatim with the driver's name in front of it, which is what -the pass-through rule in [decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md) +the pass-through rule in [the driver protocol](../examples/drivers/PROTOCOL.md) is for. ### 39. A visibility field that nothing consulted @@ -778,7 +778,7 @@ the code: invisible scroll view or switch, whose `Scrollable`/`Checkable` come from the element type rather than from visibility, would still have been offered. - **An external driver may send it**, and after - [decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md) the + [the driver protocol](../examples/drivers/PROTOCOL.md) the protocol documents `displayed:false` as meaningful. It was not: a driver could have said false and been mapped anyway. A protocol that documents a field which changes nothing is worse than one that omits it. @@ -2174,7 +2174,7 @@ The code never reached them. Clients spawn `mobium pipe`, and `pipe` turned a failed call into a result with its text and no `structuredContent`, where the MCP server built the same result with the structured half. Two front doors, two copies of one answer, and the copy every client used was the -incomplete one. Nothing had caught it: decision 0005's measurements went +incomplete one. Nothing had caught it: the error codes' measurements went through the CLI, which reads the daemon's error itself, and each client's error tests feed a hand-built result to its mapper, which is exactly the half that works. @@ -2709,7 +2709,7 @@ an accessible Other. So, before a tap, long press or `check`: a control over all of the target is waited for within the implicit wait, which is what makes the toast work, and then refused as `element_not_reachable`, naming the cover; a control over the center only is aimed around, at the clear point nearest -the center, as EarlGrey does; and something over the point that is not a +the center; and something over the point that is not a control is tapped through and reported in the result, `cover` in the structured half. Measured again after the change, every row went the way the table says it should, on both platforms, except one: on iOS the overlay @@ -2724,7 +2724,7 @@ the target, naming it and saying it is hidden from accessibility. Opt-in, since the attach stops the app for about two seconds; a real iPhone is still blind. `docs/checks/hit-test.sh` holds it to what a touch at the same point really reached, case by case, and all seven agreed -([decisions/0008](decisions/0008-a-hit-test-below-accessibility.md)). +([the hit test](guides/autowait.md#what-it-does-not-see)). The first version reported a non-control cover over nearly every target on every captured screen — iOS puts a later, transparent XCUIElementTypeWindow @@ -4084,7 +4084,7 @@ after Settings has been is a walk that left. ### 181. mobium drove a WebDriverAgent it had not installed, believing it its own **Found by:** another tool's session on the same simulator, during the work -on [decisions/0008](decisions/0008-a-hit-test-below-accessibility.md). A +on [the hit test](guides/autowait.md#what-it-does-not-see). A WebDriverAgent built from source elsewhere is installed under the bundle id every build of it shares, `com.facebook.WebDriverAgentRunner.xctrunner`. On a simulator mobium checked only that an app with that id was installed, and @@ -4416,6 +4416,59 @@ own path waits for that and does not exit first. Killed mid-run with SIGTERM, `--ui` left no daemon; interrupted mid-run three times, `mobium test` left none. +### 200. A refused tap in Chrome's web app said the app had navigated away + +**Found by:** `docs/checks/pwa.sh`, written on 2026-10-01 to hold a +progressive web app the way APP-TYPES described it, on its first runs on +the Pixel 7 AVD. Squoosh, opened from its home-screen icon, mapped, read +and evaluated, and a tap on a ref in its page was refused: "no WebView is +visible in the native hierarchy — the app may have navigated away; switch +back with `app_context NATIVE_APP`". The app had not moved: Squoosh was in +front, in Chrome's `WebappActivity`. Chrome had stopped putting its +`android.webkit.WebView` in the accessibility tree, leaving a +`FrameLayout` described as "Web View" with no children, and nothing native +said where the page was. + +Measured by reading the hierarchy every second or two after a launch from +the icon: opened while Chrome was already running, the WebView was there +for about five seconds and then gone for good, three launches out of three; +opened after Chrome's process had ended, it stayed for the half minute +sampled, two out of two. Once gone, Chrome's tabs lost theirs too, until +Chrome was restarted. The "Web View" frame is not a stand-in: when the +WebView is present the frame is not, and the one sample of it started 147 +pixels higher than the WebView had. + +The refusal itself was right — a tap with no host to measure from would +land wherever the arithmetic put it. Its remedy was not: it sent the caller +to `NATIVE_APP`, where Chrome's tree is just as empty, and a remedy that +cannot work is obeyed anyway. Now it says the app is in front, names both +causes, gives the remedy only for the one that has it, and says reading the +page still works. `pwa.sh` launches the app with Chrome running, as an +install leaves it, and holds Mobium to landing the tap or refusing it with +that explanation, with the page counting nothing; `chrome.sh` starts Chrome +fresh. What would let a tap be placed again is open (ROADMAP). + +### 201. On iOS an async expression came back as `{}` + +**Found by:** the same check, asking a home-screen web app's page whether a +service worker was registered: `navigator.serviceWorker.getRegistrations()` +is a promise, and on the iPhone 17 Pro simulator `app_eval` answered `{}` +where the same expression on Android answered `true`. Any promise did: +`Promise.resolve(42)` was `{}` too. + +Both transports send `awaitPromise: true` with `Runtime.evaluate`. CDP +honors it. WebKit's `Runtime.evaluate` has no such parameter, ignores it, +and with `returnByValue` serialized the pending promise itself. Nothing +failed — the call succeeded with a value — so a script that awaited +anything on iOS was answered before it ran. Now the expression is evaluated +to a handle; a promise is awaited with WebKit's own `Runtime.awaitPromise`, +anything else is read back by value, and the handles are released. The +simulator test asks for a resolved promise, an async function, a delayed +promise, an object and a rejection: against the old code the four promise +cases failed, against the new one all pass, and `ios-webview.sh`, +`pwa.sh`, `web-storage.sh`, `web-type.sh` and `web-actionability.sh` passed +on the simulator after it. + ## Findings that were not defects Worth recording because each one closed off an approach that looked obvious. @@ -4668,8 +4721,8 @@ Worth recording because each one closed off an approach that looked obvious. - **A shipped app's WebView stays shut on a real phone too.** With a phone's WebViews reachable, Wikipedia's App Store build still published no page: - it does not set `isInspectable`. decisions/0004 said so from a simulator - measurement of another app; this is the same answer from a shipped app on + it does not set `isInspectable`. That had been measured on a simulator + with another app; this is the same answer from a shipped app on real hardware, and why the Wikipedia check reads articles through the native tree. @@ -4710,7 +4763,7 @@ Worth recording because each one closed off an approach that looked obvious. debugger client could solve, and it is not a simulator limitation either: a shipped App Store app that has not opted in is just as unreachable on real hardware. It is why the app under test had to be one we control - ([decisions/0004](decisions/0004-an-app-under-test-of-our-own.md)). + ([MobiumApp](https://github.com/mobiumdev/mobium-app)). The measurement took three attempts to become valid, each failing the same way this project keeps documenting: the first read "no contexts" on a screen @@ -4765,7 +4818,7 @@ Worth recording because each one closed off an approach that looked obvious. recorded without anyone checking what that is, and the phrase did the work of a decision for months. It is a Unix socket. Ten minutes of probing turned a wall into a specification — see - [decisions/0002](decisions/0002-ios-webviews-are-reachable.md). The lesson is + [SETUP, "WebViews and Safari"](SETUP.md#webviews-and-safari). The lesson is the same one as the rest of this document, applied to an assumption rather than to code: **a name written down is not a finding until someone has run it.** @@ -4780,8 +4833,8 @@ Worth recording because each one closed off an approach that looked obvious. ## Environment traps -Neither is Mobium's fault; both cost real time and neither error names its -cause. +None of these is a Mobium defect; each cost real time and no error named +its cause. - **`platform-tools` must be installed into the SDK root** with `sdkmanager`, not only as the Homebrew cask. The emulator validates the SDK root by looking diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 28f8b4d6..3b8ca347 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -52,24 +52,11 @@ make ci ``` It is exactly what the CI workflow runs, so a green local run means a green -build. On a fresh clone it printed, among its other lines: - -``` -gofmt clean -clients parse -javascript errors: 14 codes mapped, all checks passed -javascript connection: 27 checks passed -javascript types: 20 exports and 77 Device methods declared -python errors: 14 codes mapped, all checks passed -python connection: 28 checks passed -148 checks, 0 failed -license copies match -docs: spelling, anchors and quick-start pages clean -169 checks, 0 failed -``` - -The `148` is the Java client's suite and the second `169` the .NET client's. -It also covers `go vet`, the linter, both Go modules' tests, cross-compilation +build. Among its other lines it prints `gofmt clean`, `clients parse`, a +line per JavaScript and Python client check, a `checks, 0 failed` total for +each of the Java and .NET suites, `license copies match` and `docs: spelling, +anchors and quick-start pages clean`; the counts in them grow with the API, +so none is quoted here. It also covers `go vet`, the linter, both Go modules' tests, cross-compilation for six targets, and the checks that the generated docs are current. `make test` alone is faster and skips most of that. `clients/go` is its own @@ -85,8 +72,9 @@ Behavior on a device is checked by the scripts in [checks/](checks/), each of which drives a real emulator, simulator or phone and asserts on what happened rather than on an exit code. -Start one device ([SETUP.md](SETUP.md) has every kind, and the traps), then -run the check for what you changed. The five clients' end-to-end flows are +Start one device — `mobium boot mobium-test` for an emulator, `mobium boot +"iPhone 17 Pro"` for a simulator; [SETUP.md](SETUP.md) has every kind, and +the traps — then run the check for what you changed. The five clients' end-to-end flows are one script: ```sh @@ -109,6 +97,9 @@ release. When you are done, stop cleanly — the daemon before the device: sh docs/checks/clean-stop.sh --quit ``` +or, for one device, `mobium shutdown emulator-5554`, which ends its daemon's +session and then shuts the device down ([SHUTDOWN.md](SHUTDOWN.md)). + ## 5. Work on a client Each client's tests start a fake `mobium` and need no device: @@ -146,8 +137,10 @@ before running `make ci`. ## 7. Send a pull request `main` is protected. A change reaches it through a pull request that has been -reviewed and whose **Build, vet and test** check passes; a second job runs the -.NET client's tests on Windows. History stays linear: pull requests are +reviewed and whose **Build, vet and test** check passes. Two more jobs run on +Windows: one runs the .NET client's tests, and `go-windows` runs both Go +modules' tests, the named-pipe daemon transport's, and a smoke test of the +built `mobium.exe`. History stays linear: pull requests are squash-merged, merge commits are off, force-pushes to `main` are refused, and a merged branch is deleted. diff --git a/docs/FORMFLUX.md b/docs/FORMFLUX.md index c2947aba..ae7b552b 100644 --- a/docs/FORMFLUX.md +++ b/docs/FORMFLUX.md @@ -30,7 +30,7 @@ element counts above are the positive control, not a demo. **On Android a screen is a setting.** `wm size` and `wm density` override the display for every app, both report their physical value alongside the override, and `reset` puts them back. Cost: about half a second per profile, -measured across all six. +measured across the first six; there are eight Android profiles now. **On iOS a screen is a device.** A simulator's geometry is fixed when it is created and nothing resizes a booted one. Covering four screens means creating @@ -179,6 +179,15 @@ MOBIUM_DEVICE_TESTS=1 MOBIUM_DEVICE=emulator-5554 \ `TestDeviceCatchesThePlantedTargets` needs MobiumApp installed, and skips without it. +The iOS test has its own switch, because it may create and delete +simulators, which is more to consent to than resizing one. It defaults to +`iphone-16e`, a type that is usually not booted; `MOBIUM_SIM_PROFILE` names +another: + +```sh +MOBIUM_SIM_TESTS=1 go test ./internal/formflux/ -run DeviceEnsures -v +``` + It applies every Android profile, checks each against a readback, and restores the physical screen **in a deferred call, so it runs even when an assertion fails**. A half-applied profile is a state left on somebody's phone; @@ -208,7 +217,7 @@ written against a page somebody controls is a check that agrees with itself. That last point is the one formflux should eventually assert. A page serving a 2× image to a 3× screen is a real, machine-checkable defect of exactly the kind a resolution factory exists to find, and it is invisible to a human -looking at a screenshot. Nothing asserts it yet — see below. +looking at a screenshot. Nothing asserts it yet. It is a third-party page and it will change without telling us, so a check built on it must fail loudly rather than silently pass when the markup moves, @@ -238,7 +247,9 @@ Two things it learned by being run: - Findings carry the element's path, because without it two problems on two unlabeled containers print identically and neither can be acted on. -**iOS.** `Ensure` finds a simulator of a profile's device type, boots it, or +**iOS.** `Ensure` and `Release` are called only by the device-backed test; +nothing in the tool layer calls them, so `app_screen` never boots or creates +a simulator. `Ensure` finds a simulator of a profile's device type, boots it, or creates one if none exists; `Release` shuts down only what it booted and deletes only what it created. Verified both ways: against an already-booted simulator, which it correctly left alone, and against a shut-down type, which @@ -272,4 +283,7 @@ warns about: a control's tap area can be larger than its bounds. - **Overlap** is unimplemented. Two controls on top of each other is a real defect, but a container legitimately contains its children, so it needs ancestry-aware comparison to avoid reporting every list. -- **Font scale.** Display size is driveable; `font_scale` is not. +- **Font scale in a profile.** `font_scale` is settable on its own — + `mobium accessibility text_scale 1.3` (`app_accessibility`) — but no profile + sets it, so `app_screen` cannot check a layout at a screen size and a text + scale together. diff --git a/docs/GESTURES.md b/docs/GESTURES.md index 07ef78ca..7f204fcd 100644 --- a/docs/GESTURES.md +++ b/docs/GESTURES.md @@ -39,6 +39,7 @@ multi-finger taps. | **Two-, three-finger tap** | `tap --fingers N` | Multi-Touch — each finger's landing, lifting and travel | yes | yes | | **Press and tap** | `press-tap` | Multi-Touch | Android 15: yes. 16 and later: **refused** | **refused** — see below | | **Press and drag** | `press-drag` | Multi-Touch | Android 15: yes. 16 and later: **refused** | **refused** | +| **Swipe an element** (a list row's swipe actions) | `swipe @e5 left` | none in MobiumApp: NetNewsWire's article rows, `docs/checks/netnewswire-ios.sh`, not `gestures.sh` | not yet driven | yes, 2026-10-01: eight of eight revealed the actions and none performed one | Not here, deliberately: **system gestures.** iOS reserves three-finger swipes and pinches for undo, redo, copy and paste, four-finger swipes on iPad, and diff --git a/docs/PHILOSOPHY.md b/docs/PHILOSOPHY.md index 89da2e9f..ff92e5e3 100644 --- a/docs/PHILOSOPHY.md +++ b/docs/PHILOSOPHY.md @@ -67,8 +67,8 @@ does exactly what a wheel does for the load on top of it; only its outline is different. That is the relation Mobium keeps between its platforms. Android, iOS and a third-party device are three differently shaped rollers underneath, and what rides on top — the tools, `map` and its `@ref`s, the locators, the -error codes, `start` and `quit` — stays at the same height on every one of -them. +error codes, a client's `start` and `quit` (the CLI's `session start` and +`session end`) — stays at the same height on every one of them. **Change only what must change.** The traders did not reinvent transport; they changed the one property the law cared about and kept everything else. Mobium @@ -93,7 +93,7 @@ necessary changes made: | One tool layer | the CLI, the MCP server and five language clients | only the front door; the CLI parses flags and calls a tool by name | | One way to read a web page | the WebView protocols — CDP on Android, WebKit's Remote Web Inspector on iOS simulators and iPhones — under one `Page` | only the wire; `map` and `text` are the same scripts | | One locator vocabulary | every platform's element tree | compiled per platform, instead of a dialect per platform | -| The built-in drivers | third-party drivers, as separate processes | only the platform layer; waiting, scrolling and refs come for free ([decision 0003](decisions/0003-drivers-are-processes-not-plugins.md)) | +| The built-in drivers | third-party drivers, as separate processes | only the platform layer; waiting, scrolling and refs come for free ([the driver protocol](../examples/drivers/PROTOCOL.md)) | | The Go client's connection | the Python, JavaScript, Java and .NET clients | each language's idiom: a lock, a timeout, the same failure rules | The rule's negative form is the one that catches mistakes: **a proposal that diff --git a/docs/RELEASE-CHECKLIST.md b/docs/RELEASE-CHECKLIST.md index 706b8fbc..c37aeeac 100644 --- a/docs/RELEASE-CHECKLIST.md +++ b/docs/RELEASE-CHECKLIST.md @@ -4,7 +4,7 @@ [CI workflow](../.github/workflows/ci.yml) runs on every push — formatting, vet, both modules' tests, the four non-Go clients' own tests, and six cross-compile targets. This file is the rest, and the rest is where most -serious defects in this project have come from: 154 of 190 were found only by +serious defects in this project have come from: 165 of 201 were found only by running against a real device. Work through it before tagging a release, on three substrates: an Android @@ -132,7 +132,8 @@ first command builds and signs WebDriverAgent, which takes a minute or two. the 61-second stall (defect 71), rows with no bounds, typing read back, and every refusal a phone owes - [ ] `./docs/checks/third-party-app-ios.sh ` passes — Wikipedia from the - App Store, the only third-party app driven on iOS + App Store, one of two third-party apps driven on iOS; the other, + NetNewsWire, is `netnewswire-ios.sh`, below - [ ] `MOBIUMAPP_BUNDLE=/MobiumApp.app ./docs/checks/mobium-app.sh ` passes, WebViews included — the phone's reach them over lockdown, not the simulator's socket @@ -154,9 +155,10 @@ first command builds and signs WebDriverAgent, which takes a minute or two. - [ ] `./docs/checks/device-state.sh ` passes — all four orientations read back, the hierarchy follows the rotation, a tap lands in landscape, and an app pinned to `ja-JP` renders in Japanese with no split characters -- [ ] `./docs/checks/third-party-app.sh wikipedia.apk` passes — the - only check that drives an app nobody at Google wrote. If a - labeling change is in this release, this is the one that catches it +- [ ] `./docs/checks/third-party-app.sh wikipedia.apk` passes — + Wikipedia, the first app driven here that nobody at Google wrote; F-Droid + and Aegis followed, and Seal is `compose-app.sh`, below. If a labeling + change is in this release, these are the checks that catch it - [ ] `./docs/checks/gestures.sh` and `./docs/checks/zoom.sh` pass on the emulator, the simulator and the phone — each gesture asserted against the one it could be mistaken for, and the refusals where a platform @@ -165,6 +167,32 @@ first command builds and signs WebDriverAgent, which takes a minute or two. - [ ] `./docs/checks/crashes.sh` passes on the emulator and the simulator — device logs and crash reports, each against a crash caused on purpose +### Not in the last full run + +These checks are in [checks/](checks/) but were not on this list for the +last full run (2026-09-25), and most were written after it, so that run +vouches for none of them. What each proves, and the devices it runs on, is in +[checks/README.md](checks/README.md). + +- [ ] Third-party apps: `netnewswire-ios.sh ` (NetNewsWire, iOS) and + `compose-app.sh ` (Seal, the first Jetpack Compose app) +- [ ] MobiumApp's demos: `login.sh`, `keyboard.sh`, `otp.sh`, `dialogs.sh`, + `autowait.sh`, `obstruction.sh`, `hit-test.sh`, `wait-states.sh`, + `map-diff.sh`, `source.sh`, `clear-data.sh`, `files.sh`, `battery.sh`, + `biometric.sh` and `accessibility.sh` +- [ ] Inside a WebView: `web-actionability.sh`, `web-type.sh` and + `web-storage.sh`; and `ios-webview-probe.sh `, the + platform assumption with no Mobium code +- [ ] A mobile browser and a PWA: `chrome.sh ` and + `pwa.sh `, both with the network +- [ ] Device state: `orientation.sh`, `network.sh `, + `shake.sh`, and on iOS `locale-ios.sh`, `timezone-ios.sh`, + `notifications-ios.sh` and `audit.sh` +- [ ] Recording: `record.sh` and `trace.sh` +- [ ] Devices started and stopped: `boot.sh [simulator-udid]` +- [ ] The test runner: `test-runner.sh`, `test-ui.sh` (needs Vibium), and + `MOBIUM_GRID= test-grid.sh` + ## Third-party drivers The extension point is a published protocol, so a break here is a break in @@ -173,7 +201,7 @@ somebody else's code that we cannot see. - [ ] `examples/drivers/mobium-driver-adb` still imports nothing but the Python standard library. If a reference driver needs anything from this repository, the extension point is not open -- [ ] `docs/decisions/0003`'s method table matches what `internal/mobiumdriver` +- [ ] `examples/drivers/PROTOCOL.md`'s method table matches what `internal/mobiumdriver` actually sends, field for field - [ ] Every capability in `mobiumdriver.KnownCapabilities` has a matching `As*` helper, and nothing above `internal/mobiumdriver` asserts a capability @@ -233,8 +261,11 @@ other known gaps are written and cross-compile, and [WINDOWS.md](WINDOWS.md), supported when those pass, and only then. - [ ] `make crosscompile` passes (CI does this) -- [ ] On Windows: `go test ./internal/daemon/ ./internal/paths/` passes, and - the CLI auto-starts a daemon that survives its terminal closing +- [ ] On Windows: `go test ./internal/daemon/ ./internal/paths/` passes (CI + does this, five times over, in the `go-windows` job), and the CLI + auto-starts a daemon (CI does this too, and reads its status back) that + survives its terminal closing (still by hand: no runner has a terminal + to close) ## The MCP Registry diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 9d159d0b..b112eb67 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -6,8 +6,8 @@ this is what is not. ## Next -- **A test runner — `mobium test`, on the phones.** Iterations 1 and 2 are - built and checked (`docs/checks/test-runner.sh`): JSON test files of +- ~~**A test runner — `mobium test`, on the phones.**~~ Done: iterations 1 + and 2 are built and checked (`docs/checks/test-runner.sh`): JSON test files of `app_batch` steps, `app_wait_for` and `expect` assertions, projects as devices, workers, retries with flaky reported, `-g`, `--last-failed`, list, JSON, JUnit and HTML reports, `show-report` — and since 2026-09-29 @@ -18,14 +18,14 @@ this is what is not. stays on emulators and simulators. Getting there found CHALLENGES 170–172. On an iPhone a password still needs a keyboard with its letters up (159). Parameters since 2026-10-01: `"each"` runs a test once per case, - `${key}` filled in (decisions/0006, "Parameters"). An interactive mode + `${key}` filled in (the test runner guide, section 9). An interactive mode since 2026-10-01: `mobium test --ui` serves a page that lists the suite, runs a test, a file or all of it on its projects, shows each step with its screen as it happens, and re-runs what failed, the files read again - for every run ([decisions/0009](decisions/0009-a-test-ui.md), + for every run ([the test runner guide, --ui](guides/test-runner.md#10-a-page-to-run-tests-from---ui), `docs/checks/test-ui.sh`). Recording a test from what a person does is - `mobium inspect`, since 2026-09-29 ([decisions/0007](decisions/0007-an-inspector.md)). - [decisions/0006](decisions/0006-a-test-runner.md). + `mobium inspect`, since 2026-09-29 ([the inspector guide](guides/inspector.md)). + The format is in [the test runner guide](guides/test-runner.md). - ~~**A second third-party app on iOS.**~~ Done 2026-10-01: NetNewsWire, an RSS reader, built from its MIT source for a simulator and from the App Store (7.1.4) on the iPhone 15 Plus, so the same app runs on both, as @@ -43,6 +43,18 @@ this is what is not. the iPhone its center was at y 2553 pixels, under the toolbar from 2538, and `main` tapped it there and opened nothing, while this scrolls it out and the article opens. +- **A tap in Chrome's web app after Chrome stops reporting its WebView.** + Opened while Chrome is running, an installed web app's WebView leaves the + accessibility tree within about five seconds, and Mobium refuses taps in + it with the reason (CHALLENGES 200). What could place one is unmeasured: + CDP can say where the page's viewport is in Chrome's own coordinates, and + whether those can be tied to the screen without a native host is the + question. Seen alongside it on 2026-10-01, each once and not reproduced: + switching into a long-backgrounded Chrome tab hung until the client gave + up after two minutes, where an `eval` in the same tab timed out at thirty + seconds with an error; and the emulator and the booted simulators went + away mid-session with nothing in Mobium stopping them and nothing in + their logs saying why. - **Windows.** Everything that needs no device passes on a GitHub-hosted Windows runner, every run: both modules' tests, the named-pipe daemon transport's acceptance tests five times over, and the built `mobium.exe` — @@ -68,8 +80,9 @@ this is what is not. a decision, not a step; a Homebrew tap; and notarizing the macOS binaries, which needs an Apple Developer Program membership — until then [SETUP.md](SETUP.md#installing-a-release) says to download with `curl`. -- **Video walkthroughs** of the quick start, one per client, once `start` and - `quit` have settled. The pages' examples and captured output are the script. +- **Video walkthroughs** of the quick start, one per client, once the clients' + `start` and `quit` (the CLI's `session start` and `session end`) have + settled. The pages' examples and captured output are the script. - **Auto-wait, the rest of the actionability checks.** Actions already wait for a target to exist, be in view, stop moving and be enabled; refuse one under a dialog or the keyboard; and aim around, wait out or refuse a control the @@ -82,17 +95,16 @@ this is what is not. does not contain, so a tap under one still lands on it. **On a simulator, `mobium hit-test` sees it** since 2026-09-29: UIKit's own hit test, asked through lldb, opt-in because the attach stops the app for - about two seconds ([decisions/0008](decisions/0008-a-hit-test-below-accessibility.md)). + about two seconds ([the hit test](guides/autowait.md#what-it-does-not-see)). On a real iPhone since 2026-09-30: the probe is evaluated as an lldb expression, through `xcrun lldb`'s own Python, and all seven cases of the Obstruction Demo agreed with where a raw touch went on the iPhone - 15 Plus, about nine seconds a case (decisions/0008, "On a real - iPhone"). It needs the app built for development. **Before every + 15 Plus, about nine seconds a case. It needs the app built for development. **Before every action, on a simulator**, since 2026-09-30: `launch --hit-test` loads the probe as the app starts and it answers on a Unix socket on the Mac's own disk in under a millisecond, so every action on an element asks it first; the seven cases agreed with a raw touch, and a tap cost - the same with it as without (decisions/0008, "Loaded at launch"). Not + the same with it as without. Not on a phone, where Mobium could reach a probe only over the phone's network; it refuses, and `hit-test` there stays the debugger's. - ~~**Session recording and `diff map`.**~~ Done 2026-09-29. `mobium trace @@ -296,7 +308,10 @@ this is what is not. scrolled. - **Switching between apps** and more than one window. - **Seeding a device** with photos, files and other data before a flow. -- **Uploads and downloads** through the system file pickers. +- **Uploads and downloads** through the system file pickers. Copying a file + to and from the device is done (`upload` and `download`, above, verified + on a real iPhone on 2026-09-29); choosing it in the picker an app opens + is not. - **Frames and iframes** inside a WebView. - **Accessibility checks** as a side effect of the actions already being taken. An explicit one exists since 2026-09-30: `mobium audit` (`app_audit`) @@ -423,7 +438,7 @@ this is what is not. become D-pad presses too; `press` already has back, home and the media keys. That makes it the existing backend with a focus strategy, not a new driver. Amazon's newer Linux-based OS, Vega, is not Android, and would be - the case for a driver process ([decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md)). + the case for a driver process ([the driver protocol](../examples/drivers/PROTOCOL.md)). First, without code: does `mobium devices` list a TV after `adb connect`; does `map` work with `--driver uiautomator`, and does Fire OS let the UiAutomator2 server install; does the hierarchy report `focused` reliably @@ -433,8 +448,10 @@ this is what is not. ## Not planned -- **A test runner.** Mobium is a tool an agent or a test framework calls; it - does not want to be the framework. Test code generation is the same decision. +- **Test code generation.** Mobium is a tool an agent or a test framework + calls. Its own runner, `mobium test`, runs JSON test files of tool calls, + and `mobium inspect` records one, but nothing generates test code in a + client's language. - **Device-cloud allocation.** Mobium drives devices you can reach; renting them is a separate concern. - **Anything that needs a runtime on the user's machine.** One static binary is diff --git a/docs/SETUP.md b/docs/SETUP.md index 94d78dee..ece64275 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -8,12 +8,16 @@ most of them report something that names the wrong cause. device, and exits non-zero if it found something: ``` -ok adb Android Debug Bridge version 1.0.41 at /opt/... -ok android sdk root /opt/homebrew/share/android-commandlinetools -ok android devices emulator-5554 (device) -note ios simulators 11 available, none booted -ok daemon socket /Users/you/.mobium/daemon/mobium.sock (46 of ~104 bytes) +ok adb Android Debug Bridge version 1.0.41 at /opt/homebrew/bin/adb +note android sdk root ANDROID_SDK_ROOT is not set — only needed to start emulators, not to drive them +note android devices none running — start an emulator or plug in a phone +ok xcode /usr/bin/xcrun +note ios simulators 11 available, none booted — `xcrun simctl boot ` +ok daemon socket ~/.mobium/daemon/mobium.sock (46 bytes of ~104) ok device agents cached: uiautomator2, webdriveragent +note third-party drivers none on PATH — a driver is an executable named mobium-driver-, used with --driver + +everything mobium needs is here ``` `note` means something is missing that is only needed for part of the job — @@ -79,6 +83,10 @@ emulator -avd mobium-test -no-snapshot-save -no-boot-anim & adb wait-for-device ``` +Or let Mobium start it: `mobium boot mobium-test` cold-boots the AVD headless +(`--window` to see it) and answers with its serial once it has booted, and +`mobium shutdown emulator-5554` ends the session on it and shuts it down. + `sdkmanager` and `avdmanager` need a JDK. If they fail with a Java error: ```sh @@ -166,13 +174,14 @@ mid-session and three things changed at once. The screen is the same; the is a restart, in the order `docs/SHUTDOWN.md` gives: ```sh -mobium daemon stop # every session on this device, first -adb -s emulator-5554 emu kill -emulator -avd mobium-test -no-snapshot-load -no-boot-anim & # headed -emulator -avd mobium-test -no-snapshot-load -no-boot-anim -no-window & # headless -adb -s emulator-5554 wait-for-device +mobium shutdown emulator-5554 && mobium boot mobium-test --window # headed +mobium shutdown emulator-5554 && mobium boot mobium-test # headless ``` +`mobium boot` cold-boots, passing `-no-snapshot-load`. `mobium shutdown` ends +only its own daemon's session, so stop any other session on the device first +(`docs/SHUTDOWN.md`). + **`-no-snapshot-load`, not `-no-snapshot-save`.** `-no-snapshot-save` still *loads* the quick-boot snapshot on the next start, and everything installed since that snapshot was taken is gone — MobiumApp came back as a build from @@ -245,8 +254,8 @@ emulator -avd mobium-test -no-snapshot-save -no-boot-anim & # 5554 emulator -avd mobium-test-17 -no-snapshot-save -no-boot-anim -port 5556 & # 5556 ``` -Then pass `--device emulator-5556` to every command, or set it once with -`mobium daemon start --device`. +Then pass `--device emulator-5556` to every command, or run each one under +its own `MOBIUM_SESSION`, or give a client the device option. ### What the images do *not* have @@ -365,6 +374,9 @@ open -a Simulator # only this puts it on screen mobium --device map ``` +`mobium boot "iPhone 17 Pro"` boots one by name or UDID and waits until it +has, and `mobium shutdown "iPhone 17 Pro"` ends the session and shuts it down. + **`simctl boot` shows you nothing.** It starts the runtime with no window, so a simulator can be fully booted and driveable while the screen stays empty. `open -a Simulator` attaches the GUI to whatever is already booted. Mobium @@ -387,8 +399,8 @@ reports progress. It is cached under `~/.mobium` afterwards. ### WebViews on a simulator -WKWebViews work, over WebKit's Remote Web Inspector rather than CDP — see -[decisions/0002](decisions/0002-ios-webviews-are-reachable.md). Two +WKWebViews work, over WebKit's Remote Web Inspector rather than CDP — a +Unix socket on a simulator, carrying binary property lists. Two preconditions, neither of which mobium can arrange: ```sh @@ -425,9 +437,9 @@ from outside. The capability runs in opposite directions on the two platforms. **The binary you ship.** `simctl install` takes a `.app` built against the simulator SDK. The `.ipa` that goes to the App Store is device-signed and for a different architecture, so nothing about signing, entitlements or store -packaging is exercised on a simulator. This is the reason -[decisions/0004](decisions/0004-an-app-under-test-of-our-own.md) could not -close the hybrid-app gap by installing somebody's shipped app. +packaging is exercised on a simulator. This is also why somebody's shipped +app could not close the hybrid-app gap here, and +[MobiumApp](https://github.com/mobiumdev/mobium-app) was built instead. The Android emulator has its own list — no Bluetooth, NFC, SD card insert/eject, attached headphones or USB — and it is a different list, which is @@ -507,7 +519,7 @@ removing `~/.mobium/webdriveragent-device/` makes Mobium build a fresh one. These are done by `simctl` on a simulator and `devicectl` has no equivalent, so on a phone they are refused, each with the reason, rather than approximated: **permissions, appearance, the clipboard, simulated location -and routes.** +and routes, and notifications** — reading or posting them. **Clearing an app's data takes its bundle on a phone.** Nothing can delete from an app's container there, so `mobium clear-data --bundle @@ -525,8 +537,7 @@ MP4, and none has been tried. ### WebViews and Safari They work on a phone, reached through the phone's lockdown service and -speaking the same protocol as a simulator's -([decisions/0002](decisions/0002-ios-webviews-are-reachable.md)). It needs +speaking the same protocol as a simulator's. It needs the phone connected by cable — usbmuxd, which carries it, is USB — and: - **an app's WebView** is inspectable only if the app sets @@ -537,8 +548,8 @@ the phone connected by cable — usbmuxd, which carries it, is USB — and: — Apple's documented switch for remote inspection, and on a simulator the measured one (its equivalent `defaults` key). On the phone this was verified with it on; it has not yet been measured with it off. Remote - Automation, on the same page, is Apple's switch for `safaridriver`, which - Mobium does not use. + Automation, on the same page, is Apple's switch for WebDriver automation + of Safari, which Mobium does not use. Tapping inside Safari's page is refused for the reason it is on a simulator: the page cannot say where it sits under Safari's own chrome (CHALLENGES 47). @@ -639,8 +650,8 @@ recordings on the node's daemon, every file landing on the caller's side. ### A grid -**`MOBIUM_GRID=node1,node2` spreads runs over several machines' devices**, as -Selenium Grid does, without a hub. At a run's first call — which is where it +**`MOBIUM_GRID=node1,node2` spreads runs over several machines' devices**, +with no hub. At a run's first call — which is where it says what it wants: a serial, or `platform` when it starts a session — mobium asks every node over SSH for its devices and which of them are held, takes the first free one that matches, and connects to its node as `--remote` diff --git a/docs/SHUTDOWN.md b/docs/SHUTDOWN.md index b329ea35..e76f5541 100644 --- a/docs/SHUTDOWN.md +++ b/docs/SHUTDOWN.md @@ -44,8 +44,8 @@ cannot be closed from outside and come back. returns holds the sessions, and shutdown used to wait for it indefinitely — SIGTERM and `daemon stop` both hung, with the socket already gone, so the daemon looked dead while its PID file refused a replacement (CHALLENGES 89). -Now it waits 10 seconds for calls to finish and 20 more for the sessions to -close, then exits and says on stderr that a session was left open. A second +Now it waits 10 seconds for calls to finish and up to 75 more for the +sessions to close, then exits and says on stderr that a session was left open. A second Ctrl-C or SIGTERM exits at once. Either way, what a session left behind is still there: run `docs/checks/clean-stop.sh --quit`, or the commands under "On the device" below. @@ -59,8 +59,9 @@ adb -s emulator-5554 emu kill `mobium shutdown ` does this for one device in the same order — its session ended, then the device — and -returns once the device is gone. Other daemons' sessions on it are theirs -to end first: it ends only its own. +returns once the device is gone. It ends only its own daemon's session and +does not check for other daemons on the device, so stop those first +(`MOBIUM_SESSION= mobium daemon stop` for each). A **headless** emulator (`-no-window`) is stopped exactly the same way, and this is where the check below earns its keep: there is no window whose absence diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 77a18cb9..c225664f 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -28,10 +28,10 @@ usually wrong (see [CHALLENGES](CHALLENGES.md)). | --- | --- | --- | --- | | **LLM01 Prompt injection** | Screen text is the classic indirect path: an app or page shows "ignore your task and uninstall…", and the agent reads it in `map` or `text` | Labels are bounded and cleaned, so one label cannot carry a page of instructions | Results mark which content came from the device, so an agent or its harness can treat it as data. An agent evaluation: a MobiumApp screen showing injection text, driven by a real agent, counting how often the agent obeys the screen over its task — a measurement that has been seen to come back non-zero before its zero means anything | | **LLM02 Sensitive information disclosure** | On a real phone, screenshots, `source`, logs, notifications (message text), the clipboard and crash reports are a person's data | Passwords are redacted everywhere a node's text surfaces (`uitree.Redact`, CHALLENGES 43). Checks assert *whether*, never *what*, on a real phone (CHALLENGES 111) | One redaction path for everything device-derived, not only password fields. A way to run with personal surfaces off | -| **LLM03 Supply chain** | The UiAutomator2 server APKs, WebDriverAgent's source, and third-party drivers | Device-side artifacts are pinned by version and verified by checksum. A third-party driver is a separate process, never code loaded into Mobium ([decisions/0003](decisions/0003-drivers-are-processes-not-plugins.md)) | A driver is still a stranger's executable, run with the user's rights; the docs should say so where drivers are installed | +| **LLM03 Supply chain** | The UiAutomator2 server APKs, WebDriverAgent's source, and third-party drivers | Device-side artifacts are pinned by version and verified by checksum. A third-party driver is a separate process, never code loaded into Mobium ([the driver protocol](../examples/drivers/PROTOCOL.md)) | A driver is still a stranger's executable, run with the user's rights; the docs should say so where drivers are installed | | **LLM04 Data and model poisoning** | — | Mobium trains nothing | — | | **LLM05 Improper output handling** | An agent's output becomes device input: text typed into fields, JavaScript run by `eval`, arguments that reach `adb shell` on the device | Everything passed to `adb shell` is quoted (`shellQuote`, CHALLENGES 52). `type` refuses what is certainly not a text field | An audit of every argument an agent sets that reaches a shell, a page or the file system, checked by a test per path | -| **LLM06 Excessive agency** | An agent can uninstall, clear data, reset every app's permissions, lock the device, and drive other machines' devices | Destructive actions confirm by reading back, and name what they changed. A grid node lends physical phones only with `MOBIUM_GRID_PHONES=1` in its own environment | A read-only mode and a tool allowlist, so a session meant to look cannot act. An app allowlist, so actions refuse outside the app under test | +| **LLM06 Excessive agency** | An agent can uninstall, clear data, reset every app's permissions, lock the device, shut a device down (`app_shutdown`), and drive other machines' devices | Destructive actions confirm by reading back, and name what they changed. A grid node lends physical phones only with `MOBIUM_GRID_PHONES=1` in its own environment. Every MCP tool carries `readOnlyHint`, true only for a tool that never acts (`internal/agent/readonly.go`), so a harness can allow only reads | A read-only mode Mobium enforces itself, and a tool allowlist, so a session meant to look cannot act. An app allowlist, so actions refuse outside the app under test | | **LLM07 System prompt leakage** | — | Mobium holds no prompts | — | | **LLM08 Vector and embedding weaknesses** | — | Mobium keeps no index | — | | **LLM09 Misinformation** | An agent reports "tapped Allow" and a person believes it | Every answer says what was checked and what was only sent — "report only what was checked" is a project rule; a tap that did not land is reported, not assumed | — | diff --git a/docs/WINDOWS.md b/docs/WINDOWS.md index 0a028136..ebf94283 100644 --- a/docs/WINDOWS.md +++ b/docs/WINDOWS.md @@ -41,24 +41,26 @@ over is CoreDevice's, which is Xcode's. A Mac-built `.ipa` installed from Window route other projects take and is not this project's architecture today; it would be a driver, not a flag. -Nothing here has been verified on Windows yet, only written — and everything -this project has learned says that unverified means wrong. The list below is -ordered so the first hour on a Windows machine is spent finding out what is -broken rather than setting up. +Everything that needs no device is verified in CI on Windows, `mobium mcp` +answering `tools/list` included (the `go-windows` job). What has never run is +a tool call against a device — and everything this project has learned says +that unverified means wrong. The list below is ordered so the first hour on a +Windows machine is spent finding out what is broken rather than setting up. **The first thing to do on Windows is not to write code.** It is to run `mobium mcp` and a client against a connected Android device and find out -whether the claim above — that the MCP path already works on Windows — is true. -It has never been run. If it is true, Windows users have a working tool today -and the daemon is an improvement rather than a rescue; if it is false, the gap -is much bigger than a transport and the plan changes. +whether the claim above — that the MCP path already works on Windows — holds +for a tool call that reaches a device. That has never been run. If it holds, +Windows users have a working tool today and the daemon is an improvement +rather than a rescue; if it does not, the gap is much bigger than a transport +and the plan changes. --- ## What is expected to work already -Never run on Windows — that is the point of this exercise — but these have no -platform-specific code: +Never run against a device on Windows — that is the point of this exercise — +but these have no platform-specific code: - the tool layer, the driver interface, the tree parser, locators and refs - `adb` discovery and every Android command, including port forwarding @@ -68,17 +70,18 @@ platform-specific code: touches the daemon socket The last one matters: **`mobium mcp` should be fully usable on Windows today.** -If it is, Windows users have a working tool via MCP and the language clients -are the only thing blocked. +CI shows it starts and answers `tools/list` there; whether a tool call reaches +a device is the unverified part. If it does, Windows users have a working tool +via MCP. -## The five gaps — written, not yet run +## The five gaps — verified in CI, without a device -Found by reading rather than running, and **since 2026-09-25 all five are -written**. Everything below compiles and vets for `windows/amd64` and -`windows/arm64`, and the Windows tests compile — none of it has run on -Windows. Read "written" as "unverified", which this project's record says -means wrong until shown otherwise. The list is still a starting point, not a -complete inventory. +Found by reading rather than running, **written on 2026-09-25**, and since +2026-09-27 run on a GitHub-hosted Windows runner: everything below compiles +and vets for `windows/amd64` and `windows/arm64`, and its tests pass on +Windows. None of it has yet carried a session to a device, which is the part +this project's record says is wrong until shown otherwise. The list is still a +starting point, not a complete inventory. ### 1. The daemon transport — `internal/daemon/listener_windows.go`, `dial_windows.go` A named pipe through `github.com/Microsoft/go-winio`, as Vibium does it. It @@ -120,7 +123,7 @@ exited process stays openable while anything holds a handle to it. Access denied answers false, as EPERM does on Unix: a PID taken by another user's process was reused. -### 4. `setDetached` — `cmd/mobium/detach_windows.go` +### 4. `device.Detach` — `internal/device/detach_windows.go` (and `detach_unix.go`) `CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS`: a Ctrl-C does not reach the daemon, and with no console it does not die when the terminal window closes. @@ -231,10 +234,13 @@ git clone https://github.com/mobiumdev/mobium.git cd mobium go build -o bin\mobium.exe .\cmd\mobium go test ./... +cd clients/go; go test ./...; cd ../.. ``` -There is a `Makefile` but no `make` on Windows by default; the two commands -above are all it does. +There is a `Makefile` but no `make` on Windows by default. These are the +build and test it runs, near enough: `make build` also stamps the version +through `-ldflags`, and `make test` runs `clients/go` because it is a module +of its own. --- @@ -248,23 +254,23 @@ The failures are the useful part. ```powershell go test ./... ``` -445 Go tests, no device needed. Any failure here is a portability bug in code +No device needed. Any failure here is a portability bug in code nobody suspected, which is the most valuable thing this exercise can find. **2. Does the MCP server work?** This is the path that should already be fine. ```powershell '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | .\bin\mobium.exe mcp ``` -Expect 43 tools — [API.md](API.md) is generated from the source and is the -live number if this one has drifted again. It has, twice. +Expect 71 tools — [API.md](API.md) is generated from the source and is the +live number if this one has drifted again. It has before. Then, with an emulator running: ```powershell '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"app_map","arguments":{}}}' | .\bin\mobium.exe mcp ``` -**3. What exactly does the CLI do?** It goes through the daemon, which now -has a named-pipe transport that has never run. Expect it to work; a hang, a +**3. What exactly does the CLI do?** It goes through the daemon, whose +named-pipe transport passes in CI but has never carried a device session. Expect it to work; a hang, a panic, or an error that names the wrong cause is a bug, and the exact text is what to report. ```powershell @@ -289,8 +295,9 @@ of the above is meaningful. ## Doing the work -Written; see "The five gaps" above. What is left is to run it. `internal/daemon`'s -tests are the acceptance criteria: they start a real daemon, call it over +Written and verified in CI without a device; see "The five gaps" above. What +is left is to run it against one. `internal/daemon`'s tests are the +acceptance criteria: they start a real daemon, call it over the real transport, check the pipe exists and is owner-only while it runs and is gone after, and check that a second home reaches no daemon. If those pass on Windows, the transport works: @@ -300,7 +307,8 @@ go test ./internal/daemon/ ./internal/paths/ -v ``` Then the CLI end to end, which the tests do not cover — auto-start in -particular, which goes through `setDetached` and a real child process: +particular, which goes through `device.Detach` and a real child process. The +`go-windows` CI job runs all of this except closing the terminal window: ```powershell .\bin\mobium.exe daemon status # "not running" @@ -316,7 +324,7 @@ From [CONTRIBUTING.md](../CONTRIBUTING.md): - **Verify by outcome, never exit code.** Especially here: a named pipe that accepts a connection and never answers looks identical to success. -- Run `go test ./...` before committing, and `mobium daemon stop` after +- Run `make ci` before committing, and `mobium daemon stop` after rebuilding, or you will debug a binary you are not running. - Never name a Go file `*_windows.go` unless you intend the build constraint — though here, you do. diff --git a/docs/checks/README.md b/docs/checks/README.md index cfcccb41..68face91 100644 --- a/docs/checks/README.md +++ b/docs/checks/README.md @@ -12,18 +12,25 @@ down so nobody has to invent them again. Getting a device attached in the first place is [../SETUP.md](../SETUP.md). -Each script takes a device serial and exits non-zero on failure. +Most scripts take a device serial or UDID as their first argument, and every +one exits non-zero on failure. The exceptions say so in their headers: +`clean-stop.sh` takes no device, `boot.sh` takes an AVD name and an optional +simulator UDID, `ios-webview.sh` defaults to the booted simulator, +`test-runner.sh` and `test-ui.sh` run on `emulator-5554` and read +`MOBIUM_IOS_DEVICE` for iOS, and `test-grid.sh` reads `MOBIUM_GRID`. | Script | What it proves | | --- | --- | | [calculator.sh](calculator.sh) | A full type-and-read loop: 1 + 1 = 2 in Google Calculator, every step asserted against a named element | | [clock-timer.sh](clock-timer.sh) | Tab navigation, a keypad whose keys have no accessibility label, and one app in two versions whose screens differ | | [clean-stop.sh](clean-stop.sh) | That nothing is left running afterwards, **on the machine and on the device** — instrumentation, `/data/local/tmp`, `/sdcard`, port forwards. `--quit` stops it all first; see [../SHUTDOWN.md](../SHUTDOWN.md) | -| [ios-webview-probe.sh](ios-webview-probe.sh) | That the iOS WebView path still works, without any Mobium code. Evidence for [../decisions/0002](../decisions/0002-ios-webviews-are-reachable.md); it verifies a *platform* assumption, so it failing means the plan needs revisiting | -| [external-driver.sh](external-driver.sh) | That a third-party driver process can drive a device as well as a built-in backend. Evidence for [../decisions/0003](../decisions/0003-drivers-are-processes-not-plugins.md) | -| [ios-webview.sh](ios-webview.sh) | That mobium drives a WKWebView end to end — discovery, switching, reading, and refusing coordinates it cannot compute. Evidence for [../decisions/0002](../decisions/0002-ios-webviews-are-reachable.md) | -| [device-state.sh](device-state.sh) | Rotation, per-app language, timezone, notifications and **geolocation**, each read back — and the only test of non-Latin text on a real screen | -| [third-party-app.sh](third-party-app.sh) | A genuine third-party **hybrid** app — install, feed, search, article, WebView. The only check driving an app nobody at Google or Apple wrote. **Given an iPhone's UDID it runs [third-party-app-ios.sh](third-party-app-ios.sh)**: Wikipedia from the App Store, which cannot run on a simulator — onboarding, the feed's cards, search, an article's links read natively, and a link followed by tapping it, since a phone's WebViews cannot be attached yet | +| [ios-webview-probe.sh](ios-webview-probe.sh) | That the iOS WebView path still works, without any Mobium code. Evidence that iOS WebViews are reachable ([SETUP](../SETUP.md#webviews-and-safari)); it verifies a *platform* assumption, so it failing means the plan needs revisiting | +| [external-driver.sh](external-driver.sh) | That a third-party driver process can drive a device as well as a built-in backend. Evidence for [the driver protocol](../../examples/drivers/PROTOCOL.md) | +| [chrome.sh](chrome.sh) | **A page in Chrome on Android**, on an emulator, with the network: listed by its title and URL, `text`, `eval` and `map` answered by the page, a tap on a ref counted by a button put on the page, and the page's own link followed to iana.org. Starts Chrome fresh first ([CHALLENGES 200](../CHALLENGES.md)); a phone refused | +| [pwa.sh](pwa.sh) | **A progressive web app**, Squoosh, on an emulator or a simulator, with the network: installed through Chrome's menu or Safari's share sheet when its icon is absent, launched from the icon, running standalone with an active service worker, its context found by its start URL. iOS: a tap in the web context refused with the reason, and from `NATIVE_APP` counted by the page. Android: the tap counted by the page, or refused because Chrome stopped reporting its WebView, and the page counted nothing ([CHALLENGES 200](../CHALLENGES.md)); a phone refused | +| [ios-webview.sh](ios-webview.sh) | That mobium drives a WKWebView end to end — discovery, switching, reading, and refusing coordinates it cannot compute. Evidence that iOS WebViews are reachable ([SETUP](../SETUP.md#webviews-and-safari)) | +| [device-state.sh](device-state.sh) | Rotation, per-app language, lock, a call and an SMS (on an emulator; a phone's refusal otherwise), timezone, notifications and **geolocation**, each read back — and the only test of non-Latin text on a real screen | +| [third-party-app.sh](third-party-app.sh) | A genuine third-party **hybrid** app — install, feed, search, article, WebView. The only check driving an app nobody at Google or Apple wrote. **Given an iPhone's UDID it runs [third-party-app-ios.sh](third-party-app-ios.sh)**: Wikipedia from the App Store, which cannot run on a simulator — onboarding, the feed's cards, search, an article's links read natively, and a link followed by tapping it. A phone's WebViews can be attached, but the App Store build does not opt into inspection, so the check asserts it publishes nothing | | [gestures.sh](gestures.sh) | That a long press is a long press, that a rotation is a rotation, and that a drag and a double tap are not a swipe and two taps. Both had shipped with no evidence behind them: a long press delivered as a tap is indistinguishable from outside, so the check asserts the *difference* and that `--duration` reaches the gesture; and a two-finger turn is measured by a WebView that computes its own angle from `touchmove`, in both directions, with the refusals. **The drag is asserted against a swipe over the same two points** — same landing, same travel, and holds of 715ms against 8ms, so an assertion that only said "it moved and landed" would pass on a swipe. The double tap is asserted against the browser's own `dblclick` on Android, and on iOS against a React Native control that counts two presses inside the double-tap window where two separate taps land outside it ([CHALLENGES 149](../CHALLENGES.md)) | | [ios-device.sh](ios-device.sh) | A **real iPhone**, through Apple's Settings: app switches in both directions and through Home, **each timed** — the first read after a switch once stalled for 61 seconds ([CHALLENGES 71](../CHALLENGES.md)) and a 40-second limit is what catches it coming back; scrolling onto rows the phone reports with no bounds; typing confirmed by read-back; a screenshot; and every refusal a phone owes, with its reason. Changes no setting | | [record.sh](record.sh) | **Screen recording**: something moving recorded, saved where the command ran, and judged by the video's own header — frames and duration — then a still screen, fewer frames and still a valid file (on a real iPhone, whose stream sends frames at a steady rate, a steady rate of them), the refusals, and nothing left recording or on the device | @@ -32,7 +39,7 @@ Each script takes a device serial and exits non-zero on failure. | [timezone-ios.sh](timezone-ios.sh) | **The time zone on iOS**, read back from Calendar, which marks the hour in progress: the device's hour, then Asia/Tokyo's once set — the app in front relaunched, and a later launch too — and the device's again once set back; an unknown zone refused. Only hour rows are read, so a phone's events are never printed | | [audit.sh](audit.sh) | **Apple's accessibility audit**, held to two known screens: Settings, at least one finding, each with a type and summary, bounds on screen where it names an element and a locator that resolves (a phone's contrast findings name none, and a phone prints only counts); and MobiumApp's Layout Demo, no findings, while `screen --inspect` names its 24pt target — Apple's hit-region rule is not the 44pt guideline. Android refuses, naming what it has instead | | [orientation.sh](orientation.sh) | **Orientation**, read back: an app that turns — Settings on Android, Safari on iOS — turned to landscape, landscape-reverse and portrait; MobiumApp, which is portrait only, refused and left upright; on iOS, portrait-reverse and `auto` refused with the reason; and the device left in portrait, Android's rotation lock handed back if it was found free | -| [autowait.sh](autowait.sh) | **Auto-wait**, judged by what MobiumApp received: a tap on a target sliding in waits for the slide to end (the Motion Demo's own stopwatch), and with Reduce Motion on the honoring target is tapped at once while the ignoring one still waits; a still button beside a confetti burst is tapped, and a piece that never holds still is refused; a second tap on Log In waits out "Signing in…" and lands; typing into a button, a checkbox and a read-only field is refused. Simulators and emulators: it switches Reduce Motion from outside, and puts it back | +| [autowait.sh](autowait.sh) | **Auto-wait**, judged by what MobiumApp received: a tap on a target sliding in waits for the slide to end (the Motion Demo's own stopwatch), and with Reduce Motion on the honoring target is tapped at once while the ignoring one still waits; a still button beside a confetti burst is tapped, and a piece that never holds still is refused; a second tap on Log In waits out "Signing in…" and lands; typing into a button, a checkbox and a read-only field is refused. Simulators and emulators: it switches Reduce Motion from outside, and puts it back. On a real iPhone, which does not allow that, the half its own setting selects is checked and the other reported NOT CHECKED | | [dialogs.sh](dialogs.sh) | **Every kind of dialog** on MobiumApp's Dialog Demo, each judged by what the app says it received: alerts with one, two and three buttons, one that arrives late, the iOS action sheet, the share sheet, camera and location permission, App Tracking Transparency on iOS, and a paste — iOS's Allow Paste prompt refused and granted, Android asking nothing. `accept` and `dismiss` are held to what they were measured to press on each platform (CHALLENGES 106), and answering by caption to meaning the same thing on both. Then a declared rule answering a dialog on the way to a tap, a rule for a button the dialog lacks refused, and the refusals under a dialog and under the keyboard, with each remedy followed. On a real iPhone it needs `MOBIUMAPP_BUNDLE` and reinstalls the app, the only permission reset a phone has, and paste is not checked | | [obstruction.sh](obstruction.sh) | **Receives events**, on MobiumApp's Obstruction Demo, judged by what the app says each tap touched: a control over all of a target is refused and named, one over its center is aimed around, a toast is waited out; a pass-through view, the negative control, lets the tap reach its target, and a plain view that swallows it is reported. On iOS an overlay hidden from accessibility is asserted as the known blind spot. Emulators, simulators and phones ([CHALLENGES 115](../CHALLENGES.md)) | | [web-actionability.sh](web-actionability.sh) | **Actionability inside a WebView**, on MobiumApp's Actionability page, judged by what the page says each tap reached: a sliding target is tapped once it stops, disabled and `aria-disabled` ones are refused, one enabled late is waited for, a full cover and a plain div are refused by the page's own hit test, a covered center is aimed around, the `pointer-events: none` layer — the negative control — lets the tap through, and a target below the fold is scrolled into view. Emulators, simulators and phones ([CHALLENGES 118](../CHALLENGES.md)) | @@ -50,8 +57,8 @@ Each script takes a device serial and exits non-zero on failure. | [otp.sh](otp.sh) | **A one-time code**, on MobiumApp's OTP Demo: the code read where it arrives — the notification shade on Android, the banner on iOS, read through rather than landing on SpringBoard (CHALLENGES 155) — six one-digit boxes that move focus on, filled digit by digit and then as one string into the first box, which either arrives whole or names the digit that never did (156); no code, verified, already used, wrong twice, locked, and expired after `background 62`; and Resend refused while the screen disables it | | [wait-states.sh](wait-states.sh) | **Waiting for a state**, on MobiumApp's Form and Login Demos: `wait --for` checked and unchecked on a custom checkbox, a radio and a real Switch; focused, arriving on a tap and moving to the next field — on iOS by asking WebDriverAgent which element is active, since the tree says every field is unfocused; and value, the whole of it where `text` is a part, with a field showing its placeholder holding "". Each timed out, saying what it saw, before it succeeded. A checked state on a text field and a password's value are refused at once, and a refusal's `details.check` is read from a real one (CHALLENGES 158) | | [network.sh](network.sh) | **Network conditions**, judged by the traffic: on an emulator, latency by ping and download and upload by timing 2MB and 1MB against servers the script runs on the Mac, each before, shaped and after; offline by a connection that must fail; the download limit airplane mode removes, put back on coming online; and a session ended offline and shaped leaving the device online and unshaped. On a phone, offline and back, and shaping refused. The emulator console's own `network speed`/`delay` read back as set and change nothing — see CHALLENGES, findings | -| [test-runner.sh](test-runner.sh) | **`mobium test`**, held to its controls: a project whose device is unset refused before any test runs; the suite in `tests/` passing on every project, the projects at once and faster than one after another; every test in `tests/controls/must-fail.test.json` failing with its step, code and a screenshot, exit 1, and the JUnit file read by Python's XML parser agreeing; `--last-failed` running exactly those; and the flaky control failing with no retry and reported flaky with one. Emulators and simulators (the flaky control clears app data). [decisions/0006](../decisions/0006-a-test-runner.md) | -| [test-ui.sh](test-ui.sh) | **`mobium test --ui`, in a browser**, driven through Vibium as a person drives it: the suite listed on every project, ▶ on one test showing its steps while it runs and every step's screen loaded, a test file changed on disk under the open page running as changed and failing on every project, and "Re-run failed" running exactly those once the file is put back. Emulators and simulators; needs Vibium ([decisions/0009](../decisions/0009-a-test-ui.md)) | +| [test-runner.sh](test-runner.sh) | **`mobium test`**, held to its controls: a project whose device is unset refused before any test runs; the suite in `tests/mobiumapp` passing on every project, the projects at once and faster than one after another; every test in `tests/controls/must-fail.test.json` failing with its step, code and a screenshot, exit 1, and the JUnit file read by Python's XML parser agreeing; `--last-failed` running exactly those; and the flaky control failing with no retry and reported flaky with one. Iteration 2: the soft control reporting both soft failures and still reaching its last step, a retain-on-failure trace kept for a failed test and not a passing one, with its recording in Vibium's record format, and `--debug` stopping before each step and quitting. The suite's login test uses `each`; `--ui` is `test-ui.sh`'s. Emulators and simulators (the flaky control clears app data). [the test runner guide](../guides/test-runner.md) | +| [test-ui.sh](test-ui.sh) | **`mobium test --ui`, in a browser**, driven through Vibium as a person drives it: the suite listed on every project, ▶ on one test showing its steps while it runs and every step's screen loaded, a test file changed on disk under the open page running as changed and failing on every project, and "Re-run failed" running exactly those once the file is put back. Emulators and simulators; needs Vibium ([the test runner guide, --ui](../guides/test-runner.md#10-a-page-to-run-tests-from---ui)) | | [test-grid.sh](test-grid.sh) | **`mobium test` through a grid** (`MOBIUM_GRID=`): two projects asking only for a platform lease two different devices and run at once, every result naming its device, both leases held mid-run and none after; and a third project the grid cannot serve refuses the run with exit 3 before any test, releasing what the others took. Needs a node with two Android emulators. [the grid guide](../guides/grid.md#4-tests-on-a-grid) | | [source.sh](source.sh) | **The raw source** on MobiumApp: the native hierarchy well-formed and in the platform's units — px on Android, pt with a scale on iOS — with a typed password nowhere in it; then a WebView's markup with a password planted in the page's own markup, hidden in the answer and still in the page. The native redaction's positive control is `internal/uitree`'s test on a captured Aegis screen | | [clear-data.sh](clear-data.sh) | **Clearing an app's data**, with MobiumApp's Storage Demo as the witness: a count raised and shown to survive a relaunch, so that 0 afterwards means something; cleared; and 0 and no file read back from the app itself. On Android, a location grant read back as granted and then as revoked. An app that is not installed is refused. A real iPhone, which cannot clear in place, refuses without a bundle and names `--bundle`; with `MOBIUMAPP_BUNDLE` set, the reset itself — uninstalled and installed again from the bundle — is held to the same witness, its unreadable permissions must be listed apart, and a bundle for another app is refused with MobiumApp left installed | @@ -59,12 +66,10 @@ Each script takes a device serial and exits non-zero on failure. | [clients.sh](clients.sh) | **All five clients driven**, not only compiled: Python, JavaScript, Go, Java and .NET each run one flow through their own API against a booted Android device, from a test that sits beside the client's unit tests and skips itself unless `MOBIUM_E2E_DEVICE` is set. A client whose toolchain is missing is reported as skipped, never as passed | | [crashes.sh](crashes.sh) | **Device logs, crash reports and ANRs** — the ANR caused on purpose where adb can become root, by freezing MobiumApp and touching it, and checked through its dialog, its line in the app-filtered log and its report. **Device logs and crash reports**, on an Android device, an iOS simulator or a real iPhone, each against a positive control: a line logged between two reads is the next read's, a burst past the limit is counted rather than hidden, and a crash caused on purpose — `am crash` on Android, an injected `abort()` on a simulator, MobiumApp's Crash Demo on a phone, and on Android too when MobiumApp is installed — appears scoped to its app, reads in full with its cause, and is listed again, since a crash is a record and not a stream | | [shake.sh](shake.sh) | **Shake**, each platform against a detector, before and after: on a simulator iOS's own Undo Typing after typing, and none with nothing typed; on an emulator a page in Chrome listening to `devicemotion` (so it needs the network), still before and quiet after; a real phone refused, saying why | -| [hit-test.sh](hit-test.sh) | **The hit test below accessibility**, held to what a touch really reached: for each case on MobiumApp's Obstruction Demo, `hit-test`'s verdict, then a touch at that same point by coordinates, and the app's word for what it reached — all seven must agree, the overlay hidden from accessibility named as hidden. Then, on a simulator, the same seven with the probe loaded at launch (`launch --hit-test`): a plain `tap` must tap where the touch reaches the target and refuse, touching nothing, everywhere else. On a simulator and a phone (about nine seconds a case there, where `--hit-test` is refused, saying why); an emulator is refused too ([decisions/0008](../decisions/0008-a-hit-test-below-accessibility.md)) | +| [hit-test.sh](hit-test.sh) | **The hit test below accessibility**, held to what a touch really reached: for each case on MobiumApp's Obstruction Demo, `hit-test`'s verdict, then a touch at that same point by coordinates, and the app's word for what it reached — all seven must agree, the overlay hidden from accessibility named as hidden. Then, on a simulator, the same seven with the probe loaded at launch (`launch --hit-test`): a plain `tap` must tap where the touch reaches the target and refuse, touching nothing, everywhere else. On a simulator and a phone (about nine seconds a case there, where `--hit-test` is refused, saying why); an emulator is refused too ([the hit test](../guides/autowait.md#what-it-does-not-see)) | | [biometric.sh](biometric.sh) | **Biometrics**, held to MobiumApp's Biometrics Demo, which says what the platform told it: enrollment reaching the running app, a match refused with nothing enrolled and with no prompt up, the enrolled face or finger signing in, and a stranger not recognized. On a Face ID simulator a second stranger to Not Recognized is refused and Cancel reaches the app; on a Touch ID one the third closes the prompt as `authentication_failed`; on an emulator — enrolled through Settings, with PIN 1111 — strangers lock the sensor out, the app hears `lockout`, and the finger signs in once it ends. The enrollment is put back. A real phone is refused, status included | | [zoom.sh](zoom.sh) | Pinch-to-zoom, the first two-finger gesture — and the only thing that can confirm one. Neither platform reports a zoom level in the hierarchy, so it pinches a WebView and asks the page for its own `visualViewport.scale` | -| [mobium-app.sh](mobium-app.sh) | A genuine hybrid app on **both** platforms: an embedded WebView that opts into inspection, two live contexts in one app, a password field, **a GPX route watched from inside the app**, and **an -interruption**: a real system permission dialog, and whether the app still -holds what was typed before it. One script, backend chosen from the serial. Evidence for [../decisions/0004](../decisions/0004-an-app-under-test-of-our-own.md) | +| [mobium-app.sh](mobium-app.sh) | A genuine hybrid app on **both** platforms: an embedded WebView that opts into inspection, two live contexts in one app, a password field, **a GPX route watched from inside the app**, and **an interruption**: a real system permission dialog, and whether the app still holds what was typed before it. One script, backend chosen from the serial. Evidence for why the app under test is our own, [MobiumApp](https://github.com/mobiumdev/mobium-app) | ## Results @@ -89,7 +94,8 @@ way of a native tap on a web link. Its first run found defects 77–80. disabled it failed on its second launch, at 64.8s — the run that makes its time limit a check rather than a guess. -**Everything added since has run on the iPhone 15 Plus too, on 2026-09-27**, +**The checks that existed by 2026-09-27 and can run on a phone ran on the +iPhone 15 Plus that day**, with MobiumApp rebuilt from `mobiumdev/mobium-app` and signed for the phone. All passed: `login.sh` (2m45s), `keyboard.sh`, `autowait.sh`, `obstruction.sh`, `web-actionability.sh`, `web-type.sh`, `source.sh`, @@ -111,8 +117,8 @@ alerts, the action and share sheets, camera, location and App Tracking Transparency prompts, rules and refusals — except paste, **NOT CHECKED** because a phone's clipboard cannot be seeded from outside. -**Every check that runs on Android passed on the Pixel 8 Pro, Android 17, on -2026-09-27**, with MobiumApp rebuilt from `mobiumdev/mobium-app`: the six in +**Every check that ran on Android by then passed on the Pixel 8 Pro, Android +17, on 2026-09-27**, with MobiumApp rebuilt from `mobiumdev/mobium-app`: the five in the table, and `login.sh`, `keyboard.sh`, `autowait.sh`, `obstruction.sh`, `web-actionability.sh`, `web-type.sh`, `source.sh`, `crashes.sh`, `accessibility.sh`, `clear-data.sh`, `dialogs.sh`, `gestures.sh`, `zoom.sh`, @@ -183,8 +189,8 @@ a server running. `MOBIUM_NETWORK_TESTS=1` follows it, and then asserts the host, the path and the title. It was verified on both platforms when the link pointed at the -author's GitHub profile; the move to `github.com/mobiumdev` has not been -followed on a device yet. The title is matched on the **handle** rather than the +author's GitHub profile, and followed to `github.com/mobiumdev` on the iPhone +15 Plus and the Pixel 8 Pro on 2026-09-27, after the move. The title is matched on the **handle** rather than the display name: one is part of the URL and stable, the other is a profile field anybody can edit. diff --git a/docs/checks/autowait.sh b/docs/checks/autowait.sh index 8a735698..14f70521 100755 --- a/docs/checks/autowait.sh +++ b/docs/checks/autowait.sh @@ -4,7 +4,7 @@ # and refuses, saying why, what it cannot wait out. Judged by what MobiumApp # says it received, never by the tool's own report. # -# docs/checks/autowait.sh +# docs/checks/autowait.sh # # Stable, on the Motion Demo: a tap on a target sliding in waits for the slide # to end — the app's own stopwatch says how long after Replay the tap came — diff --git a/docs/checks/calculator.sh b/docs/checks/calculator.sh index 8b83493c..9145af77 100755 --- a/docs/checks/calculator.sh +++ b/docs/checks/calculator.sh @@ -14,6 +14,7 @@ # assertion names which one it means. set -e DEV="$1"; M="./bin/mobium --device $DEV" +if [ -z "$DEV" ]; then echo "usage: $0 " >&2; exit 2; fi ID=com.google.android.calculator:id # The formula id appears on three nodes — the container and its children all diff --git a/docs/checks/chrome.sh b/docs/checks/chrome.sh new file mode 100755 index 00000000..95356863 --- /dev/null +++ b/docs/checks/chrome.sh @@ -0,0 +1,80 @@ +#!/bin/sh +# Chrome on Android, end to end: a page in a mobile browser, reached as a web +# context and held to what the page itself says. +# +# docs/checks/chrome.sh +# +# Needs the network: the page is https://example.com, opened in a new tab +# with a query string of its own so that its context is found by URL — the +# tabs a browser keeps are numbered in listing order, so a name from one +# listing is not a name for the next. +# +# - The page opens in com.android.chrome, and its context is listed with the +# page's title and URL. +# - text, map and eval answer from the page. +# - A tap on a ref in the web context lands where the page sees it: a button +# put on the page counts its clicks, and the count is read from the page. +# - A tap on the page's own link navigates: the page's host is then iana.org. +# +# Safari's page on iOS is ios-webview.sh, where a tap in the web context is +# refused; this is the platform where it lands. A real phone is refused: the +# check opens tabs in somebody's browser. +set -e +DEV="$1" +if [ -z "$DEV" ]; then echo "usage: $0 " >&2; exit 2; fi +case "$DEV" in + emulator-*) ;; + *) echo "$DEV is not an emulator: this check opens tabs in the device's browser, and a phone's is somebody's" >&2; exit 2 ;; +esac +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +M="$ROOT/bin/mobium --device $DEV" +fail() { $M context NATIVE_APP >/dev/null 2>&1 || true; echo "FAIL: $*" >&2; exit 1; } +row() { printf ' %-12s %-58s ok\n' "$1" "$2"; } +echo "--- $DEV" + +T=$(date +%s) +URL="https://example.com/?chrome=$T" +$M context NATIVE_APP >/dev/null 2>&1 || true +# Chrome started fresh: once an installed web app has been opened while +# Chrome was running, Chrome stops putting its WebView in the accessibility +# tree — for its tabs too — and a tap has nowhere to be measured from. +# CHALLENGES 200; pwa.sh holds that case. +$M terminate com.android.chrome >/dev/null 2>&1 || true +$M open "$URL" >/dev/null +sleep 3 +[ "$($M current)" = com.android.chrome ] || fail "Chrome is not in front after opening $URL" +CTX=$($M contexts | grep -F "$URL" | awk '{print $1}' | head -1) +[ -n "$CTX" ] || fail "the page did not appear as a context (is the emulator online?): $($M contexts | head -3)" +$M contexts | grep -F "$URL" | grep -q "Example Domain" || fail "the listing does not carry the page's title" +row "listed" "$CTX, with the page's title and URL" + +$M context "$CTX" >/dev/null +$M text | grep -q "This domain is for use in" || fail "the page's own text did not come back" +[ "$($M eval 'document.title')" = "Example Domain" ] || fail "eval did not answer from the page" +LINK=$($M map | grep '(link)' | head -1 | awk '{print $1}') || true +[ -n "$LINK" ] || fail "the page's link is not in the map: $($M map)" +row "read" "text, eval and map answer from the page" + +$M eval '(() => { document.querySelectorAll("button").forEach(e => { if (e.textContent === "Mobium probe") e.remove() }); + const b = document.createElement("button"); b.id = "mobium-probe"; b.textContent = "Mobium probe"; + b.style.cssText = "position:fixed;left:30%;top:60%;width:40%;height:60px;font-size:20px;z-index:9"; + window.__taps = 0; b.onclick = () => { window.__taps++ }; document.body.append(b); return "ok" })()' >/dev/null +PROBE=$($M map | grep 'Mobium probe' | awk '{print $1}') || true +[ -n "$PROBE" ] || fail "the button put on the page is not in the map" +out=$($M tap "$PROBE" 2>&1) || fail "a tap in the web context was refused: $out" +[ "$($M eval 'window.__taps')" = 1 ] || fail "a tap in the web context did not reach the page: it counted $($M eval 'window.__taps')" +row "tap" "a tap on $PROBE landed: the page counted one click" + +LINK=$($M map | grep '(link)' | head -1 | awk '{print $1}') +$M tap "$LINK" >/dev/null +host="" +for _ in 1 2 3 4 5 6 7 8 9 10; do + sleep 1 + host=$($M eval 'location.host' 2>/dev/null || true) + case "$host" in *iana.org) break ;; esac +done +case "$host" in *iana.org) ;; *) fail "the page's link did not navigate: the page is at \"$host\"" ;; esac +row "navigate" "a tap on the page's link took it to $host" + +$M context NATIVE_APP >/dev/null +echo PASS diff --git a/docs/checks/clock-timer.sh b/docs/checks/clock-timer.sh index 1d15c1dd..211cab83 100755 --- a/docs/checks/clock-timer.sh +++ b/docs/checks/clock-timer.sh @@ -20,6 +20,7 @@ # phone whose saved timers all have "Timer" in the name. set -e DEV="$1"; M="./bin/mobium --device $DEV" +if [ -z "$DEV" ]; then echo "usage: $0 " >&2; exit 2; fi ID=com.google.android.deskclock:id # show normalizes both layouts to HH:MM:SS. diff --git a/docs/checks/crashes.sh b/docs/checks/crashes.sh index b9194fa6..d0958c13 100755 --- a/docs/checks/crashes.sh +++ b/docs/checks/crashes.sh @@ -1,6 +1,6 @@ #!/bin/sh -# Device logs and crash reports, end to end, on an Android device or an iOS -# simulator. +# Device logs and crash reports, end to end, on an Android device, an iOS +# simulator or a real iPhone. # # Each half needs a positive control — something known to have happened — or # an empty answer proves nothing. On Android that is `am crash`, which crashes diff --git a/docs/checks/device-state.sh b/docs/checks/device-state.sh index 6e523251..6805e7c8 100755 --- a/docs/checks/device-state.sh +++ b/docs/checks/device-state.sh @@ -1,10 +1,12 @@ #!/bin/sh -# Rotation and per-app language, end to end. +# Rotation and per-app language, end to end — and lock, a call and an SMS +# (an emulator's; a real phone's refusal), the timezone, notifications and +# geolocation, each read back. # -# Both exist because they change how every screen is laid out, and both are -# only worth having because they can be read back — a rotation that the sensor -# undoes a moment later, or a language the device did not store, is the shape -# of defects 26 and 27. +# The first two exist because they change how every screen is laid out, and +# both are only worth having because they can be read back — a rotation that +# the sensor undoes a moment later, or a language the device did not store, +# is the shape of defects 26 and 27. # # The language half doubles as the only test of non-Latin text on a real # screen. Every label rule mobium has — truncation, markup stripping, diff --git a/docs/checks/dialogs.sh b/docs/checks/dialogs.sh index 049f4135..6536f8e0 100755 --- a/docs/checks/dialogs.sh +++ b/docs/checks/dialogs.sh @@ -4,6 +4,7 @@ # dialog going away. # # docs/checks/dialogs.sh +# MOBIUMAPP_BUNDLE= docs/checks/dialogs.sh # # What it holds each platform to is what was measured (CHALLENGES 106): # accept and dismiss press a button the platform picks, and it is not the diff --git a/docs/checks/external-driver.sh b/docs/checks/external-driver.sh index 0f070a55..18b8e54d 100755 --- a/docs/checks/external-driver.sh +++ b/docs/checks/external-driver.sh @@ -2,7 +2,7 @@ # The third-party driver protocol, end to end against a real Android device. # # What this proves is not "the code runs". It is that a driver written from -# docs/decisions/0003 alone, in a different language, importing nothing from +# examples/drivers/PROTOCOL.md alone, in a different language, importing nothing from # this repository, sees the same screen mobium's own backend sees. The # load-bearing step is the diff: `examples/drivers/mobium-driver-adb` covers # deliberately the same ground as --driver uiautomator, so the two maps can be diff --git a/docs/checks/hit-test.sh b/docs/checks/hit-test.sh index 5e9272cb..f6ea32ae 100755 --- a/docs/checks/hit-test.sh +++ b/docs/checks/hit-test.sh @@ -7,7 +7,7 @@ # app's outcome line says what it reached. The two must agree: "reaches" # only where the target got the touch, and a named receiver where something # else did. The overlay hidden from accessibility is the case this exists -# for (CHALLENGES 115, docs/decisions/0008); the pass-through view is the +# for (CHALLENGES 115); the pass-through view is the # negative control, reached although something is drawn over it. # # docs/checks/hit-test.sh diff --git a/docs/checks/ios-webview.sh b/docs/checks/ios-webview.sh index 21dfb9e6..2d1340a4 100755 --- a/docs/checks/ios-webview.sh +++ b/docs/checks/ios-webview.sh @@ -53,7 +53,11 @@ sleep 3 [ "$($M current 2>/dev/null | tail -1)" = "com.apple.mobilesafari" ] \ || fail "Safari is not in the foreground; the native WebView element will not be there" -ctx=$(appContexts com.apple.mobilesafari | head -1) +# By URL: Safari keeps every tab it has open as a context, numbered in +# listing order, so the first one is whatever tab happens to list first — a +# Squoosh tab left by pwa.sh, once, and this read the wrong page. +ctx=$(for c in $(appContexts com.apple.mobilesafari); do + $M contexts | awk -v id="$c" '$1 == id' | grep -q "https://example.com/" && echo "$c"; done | head -1) [ -n "$ctx" ] || fail "no WebView context on a simulator showing a web page: $($M contexts 2>&1 | grep -m1 "^error:" || echo "mobium listed none for this app")" echo " contexts $ctx" @@ -86,7 +90,9 @@ echo " geometry refused to tap, and said why ok" # target once per connection. $M context NATIVE_APP >/dev/null $M context "$ctx" >/dev/null -$M text | grep -q "Example Domain" || fail "re-attaching to the same page did not work" +# The body text, not the heading: example.com has dropped its "Example Domain" +# heading, and the title is all that still says it. +$M text | grep -q "This domain is for use in" || fail "re-attaching to the same page did not work" echo " re-attach detached and attached again cleanly ok" $M context NATIVE_APP >/dev/null diff --git a/docs/checks/mobium-app.sh b/docs/checks/mobium-app.sh index 3c876487..9dada850 100755 --- a/docs/checks/mobium-app.sh +++ b/docs/checks/mobium-app.sh @@ -24,7 +24,7 @@ # MobiumApp is that app: React Native, `webviewDebuggingEnabled` on every # WebView, pages shipped inline rather than fetched. It lives outside this # repository because it needs Node, Gradle and CocoaPods, and mobium's -# no-runtime-dependencies property is load-bearing. See docs/decisions/0004. +# no-runtime-dependencies property is load-bearing. # # The same script drives both platforms. Only the backend differs, which is the # whole point of where the driver seam was put: `map`, `contexts`, `context` diff --git a/docs/checks/pwa.sh b/docs/checks/pwa.sh new file mode 100755 index 00000000..12f726dc --- /dev/null +++ b/docs/checks/pwa.sh @@ -0,0 +1,184 @@ +#!/bin/sh +# A progressive web app, end to end: Squoosh (https://squoosh.app), installed +# to the home screen and launched from its icon, held to what the page itself +# says. +# +# docs/checks/pwa.sh +# +# Needs the network. If no Squoosh icon is on the home screen the check +# installs one — through Chrome's menu on Android, Safari's share sheet on +# iOS, since Mobium does not install a PWA itself — and leaves it there, so a +# second run launches the same icon. +# +# - Launched from its icon, the app runs standalone: Chrome's WebappActivity +# on Android, com.apple.webapp on iOS; the page reports display-mode +# standalone and an active service worker, and on iOS +# navigator.standalone. +# - Its context is found by URL, the manifest's start_url with +# utm_source=launcher — on Android it is named for Chrome, on iOS for +# SafariViewService, and on neither for the app. +# - Android: launched while Chrome is running, Chrome stops reporting the +# app's WebView to accessibility within about five seconds (CHALLENGES +# 200), and nothing native can then say where the page is. So a tap on a +# ref in the web context either lands — a button put on the page counts it +# — or is refused with that explanation and the page counts nothing; never +# a tap reported as made that went somewhere else. Which one is printed. +# - iOS: the same tap is refused, because the page's position in its host +# cannot be known; from NATIVE_APP the button is in the tree, and a tap +# there lands — the page counts it. docs/APP-TYPES.md, "Progressive web +# apps". +# +# A real phone is refused: the check installs an app on it. +set -e +DEV="$1" +if [ -z "$DEV" ]; then echo "usage: $0 " >&2; exit 2; fi +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +case "$DEV" in + emulator-*) PLATFORM=android; M="$ROOT/bin/mobium --device $DEV" ;; + ????????-????-????-????-????????????) + xcrun simctl list devices 2>/dev/null | grep -q "$DEV" || + { echo "$DEV is not a simulator: this check installs an app, and a phone is somebody's" >&2; exit 2; } + PLATFORM=ios; M="$ROOT/bin/mobium --driver wda --device $DEV" ;; + *) echo "$DEV is not an emulator or a simulator: this check installs an app, and a phone is somebody's" >&2; exit 2 ;; +esac +fail() { $M context NATIVE_APP >/dev/null 2>&1 || true; echo "FAIL: $*" >&2; exit 1; } +row() { printf ' %-12s %-58s ok\n' "$1" "$2"; } +echo "--- $DEV ($PLATFORM)" +$M context NATIVE_APP >/dev/null 2>&1 || true + +# icon prints the ref of the first Squoosh icon on the home screen, if any. +# The launcher is read more than once: just after a boot, or straight after +# leaving Chrome, its first map can come before its icons — a miss there +# installed a second copy, and then a third. +icon() { + $M press home >/dev/null + r="" + for _ in 1 2 3; do + sleep 2 + r=$($M map | grep -m1 'Squoosh (button)' | awk '{print $1}') || true + [ -n "$r" ] && break + if [ "$PLATFORM" = ios ]; then + $M swipe left >/dev/null; sleep 2 + r=$($M map | grep -m1 'Squoosh (button)' | awk '{print $1}') || true + [ -n "$r" ] && break + $M press home >/dev/null + fi + done + echo "$r" +} + +# On Android the launcher shows Chrome's web-app icons only while Chrome is +# running — after a boot they are hidden, and the install path would find +# an installed app and offer a plain shortcut instead — so Chrome is started +# first, with a tab. That is also how the launch below is a warm one. +if [ "$PLATFORM" = android ]; then + $M open "https://example.com/?pwa=$(date +%s)" >/dev/null; sleep 3 +fi +ICON=$(icon) +if [ -z "$ICON" ]; then + $M open https://squoosh.app >/dev/null; sleep 5 + if [ "$PLATFORM" = android ]; then + $M tap testid=com.android.chrome:id/menu_button >/dev/null; sleep 2 + $M tap "text=Add to Home screen" >/dev/null 2>&1 || $M tap "text=Install app" >/dev/null + sleep 3 + if $M map | grep -q "Create shortcut"; then + # Chrome has the app installed and offers only a plain shortcut, which + # opens a tab. Without Play services Chrome installs a web app as a + # launcher shortcut, not an APK, and after a cold boot the launcher + # showed none of them while Chrome still listed six. + $M tap "text=Cancel" >/dev/null 2>&1 || true + fail "Chrome has Squoosh installed but the home screen shows no icon for it, and Chrome will only add a plain shortcut — wipe the emulator's data (emulator -avd -wipe-data) to install it again" + fi + $M tap "text=Install" >/dev/null; sleep 3 + $M tap "text=Add to home screen" >/dev/null 2>&1 || true # the launcher's own confirmation, when it asks + else + # The menu is asked for until it opens: on a simulator that had never + # opened the page, a tap on More while it was still loading opened + # nothing. + for _ in 1 2 3; do + $M tap 'label=More,role=button' >/dev/null + $M wait 'label=Share,role=button' --timeout 5s >/dev/null 2>&1 && break + done + $M tap 'label=Share,role=button' >/dev/null || fail "Safari's menu did not open"; sleep 3 + $M tap 'label=Add to Home Screen,role=button' >/dev/null 2>&1 || + { $M tap 'label=View More,role=button' >/dev/null; sleep 2; $M tap 'label=Add to Home Screen,role=button' >/dev/null; } + sleep 3 + $M map | grep -q 'Open as Web App (switch, checked)' || fail "the sheet would not add Squoosh as a web app" + $M tap 'label=Add,role=button' >/dev/null; sleep 3 + fi + ICON=$(icon) + [ -n "$ICON" ] || fail "Squoosh is not on the home screen after installing it" + row "installed" "Squoosh added to the home screen" +fi + +# On Android, Chrome is running when the icon is tapped, which is how an +# install leaves it and the case CHALLENGES 200 is about. A launch with +# Chrome's process gone cannot be arranged from here: the launcher hides +# Chrome's web-app icons while Chrome is not running. +$M tap "$ICON" >/dev/null; sleep 5 +front=$($M current) +if [ "$PLATFORM" = android ]; then + [ "$front" = com.android.chrome ] || fail "the icon opened $front, not Chrome" + adb -s "$DEV" shell dumpsys activity activities | grep -m1 topResumedActivity | grep -q WebappActivity || + fail "the icon opened Chrome in a tab, not as a web app" + row "launched" "from its icon, in Chrome's WebappActivity" +else + [ "$front" = com.apple.webapp ] || fail "the icon opened $front, not a web app" + row "launched" "from its icon, as com.apple.webapp" +fi + +# Several launches leave several pages at the start URL; the app in front is +# the one that is visible. +CTX="" +for c in $($M contexts | grep -F 'squoosh.app/?utm_medium=PWA&utm_source=launcher' | awk '{print $1}'); do + $M context "$c" >/dev/null 2>&1 || continue + if [ "$($M eval 'document.visibilityState' 2>/dev/null)" = visible ]; then CTX=$c; break; fi +done +[ -n "$CTX" ] || fail "no visible page at Squoosh's start URL: $($M contexts | head -4)" +[ "$($M eval 'matchMedia("(display-mode: standalone)").matches')" = true ] || fail "the page is not running standalone" +# A service worker registered for the app and active: what makes it run +# offline. Whether it controls this very page is the browser's business — a +# page Chrome reloads into the web app is not controlled until the next load. +sw=$($M eval '(async () => (await navigator.serviceWorker.getRegistrations()).some(r => r.active))()' 2>/dev/null || true) +[ "$sw" = true ] || fail "no active service worker is registered for the app" +if [ "$PLATFORM" = ios ]; then + [ "$($M eval 'navigator.standalone')" = true ] || fail "navigator.standalone is not true" +fi +row "standalone" "$CTX, found by its start URL; a service worker active" + +$M map | grep -q 'Paste' || fail "the page's own controls are not in the map" +$M eval '(() => { document.querySelectorAll("button").forEach(e => { if (e.textContent === "Mobium probe") e.remove() }); + const b = document.createElement("button"); b.textContent = "Mobium probe"; + b.style.cssText = "position:fixed;left:30%;top:60%;width:40%;height:60px;font-size:20px;z-index:9"; + window.__taps = 0; b.onclick = () => { window.__taps++ }; document.body.append(b); return "ok" })()' >/dev/null +PROBE=$($M map | grep 'Mobium probe' | awk '{print $1}') || true +[ -n "$PROBE" ] || fail "the button put on the page is not in the map" + +if [ "$PLATFORM" = android ]; then + # Long enough after the launch for Chrome to stop reporting the WebView, + # if it is going to: then the tap must be refused and say why, and the + # page must count nothing. A tap that lands is right too, and is reported. + sleep 5 + if out=$($M tap "$PROBE" 2>&1); then + [ "$($M eval 'window.__taps')" = 1 ] || fail "a tap in the web context was reported but did not reach the page" + row "tap" "a tap on $PROBE in the web context landed: the page counted one" + else + echo "$out" | grep -q "stopped reporting its WebView" || fail "a tap in the web context was refused without saying why: $out" + [ "$($M eval 'window.__taps')" = 0 ] || fail "the refused tap reached the page anyway" + row "refused" "Chrome stopped reporting the WebView; the tap says so (CHALLENGES 200)" + fi +else + if $M tap "$PROBE" >/dev/null 2>&1; then fail "a tap in the web context was not refused"; fi + $M map 2>&1 | grep -q "cannot be tapped" || fail "the map did not say why the page cannot be tapped" + [ "$($M eval 'window.__taps')" = 0 ] || fail "the refused tap reached the page anyway" + row "refused" "a tap in the web context, and the map says why" + $M context NATIVE_APP >/dev/null + $M tap 'label=Mobium probe' >/dev/null + $M context "$CTX" >/dev/null + [ "$($M eval 'window.__taps')" = 1 ] || fail "a tap from NATIVE_APP did not reach the page" + row "tap" "the page's button from NATIVE_APP landed: the page counted one" +fi + +$M context NATIVE_APP >/dev/null + +echo PASS diff --git a/docs/checks/test-runner.sh b/docs/checks/test-runner.sh index f1d55d17..11068484 100755 --- a/docs/checks/test-runner.sh +++ b/docs/checks/test-runner.sh @@ -1,5 +1,5 @@ #!/bin/sh -# `mobium test`, held to the controls decision 0006 promised: a runner is +# `mobium test`, held to its controls: a runner is # only believed once each of its verdicts has been seen to come out the other # way. # diff --git a/docs/checks/test-ui.sh b/docs/checks/test-ui.sh index 7d42833a..aaea5e1b 100755 --- a/docs/checks/test-ui.sh +++ b/docs/checks/test-ui.sh @@ -3,7 +3,7 @@ # buttons. The server's own tests cover what it answers; this covers the page # — which tests its buttons ask for, the steps it draws as they come, the # screens it loads under its content security policy, and what it means by -# "re-run failed" — none of which runs without a browser (docs/decisions/0009). +# "re-run failed" — none of which runs without a browser. # # docs/checks/test-ui.sh # Android emulator-5554 # MOBIUM_IOS_DEVICE= docs/checks/test-ui.sh # and iOS diff --git a/docs/checks/third-party-app-ios.sh b/docs/checks/third-party-app-ios.sh index 3ea309e6..a63db512 100755 --- a/docs/checks/third-party-app-ios.sh +++ b/docs/checks/third-party-app-ios.sh @@ -130,7 +130,8 @@ echo " article $links links from the page, natively ok" # The article's WebView cannot be attached, and not because of the phone: # a phone's WebViews are reachable, and MobiumApp's are. The App Store build # of Wikipedia does not set isInspectable, so WebKit publishes no target for -# it, which is decisions/0004 measured on a shipped app. The article is read +# it: a WebView that has not opted in is unreachable, measured on a shipped +# app. The article is read # through the native links above instead. if $M contexts | grep -q "WEBVIEW_$APP"; then echo " webview Wikipedia now opts into inspection — this check can attach and should" diff --git a/docs/checks/third-party-app.sh b/docs/checks/third-party-app.sh index 427509cc..12a843ab 100755 --- a/docs/checks/third-party-app.sh +++ b/docs/checks/third-party-app.sh @@ -24,8 +24,9 @@ # app is device-signed and will not run on a simulator, which is why this gate # could not be met on iOS until there was a phone — and why the iOS half runs # on a phone only. It drives the native app and follows links through the -# accessibility tree WebKit exposes, since a phone's WebViews cannot be -# attached (yet); the Android half attaches to the WebView instead. +# accessibility tree WebKit exposes: a phone's WebViews can be attached, but +# the App Store build does not opt into inspection, so it publishes none. The +# Android half attaches to the WebView instead. set -e DEV="$1"; APK="$2" if [ -z "$DEV" ]; then echo "usage: $0 [apk]" >&2; exit 2; fi diff --git a/docs/decisions/0001-cdp-not-webdriver-bidi.md b/docs/decisions/0001-cdp-not-webdriver-bidi.md deleted file mode 100644 index c5666e92..00000000 --- a/docs/decisions/0001-cdp-not-webdriver-bidi.md +++ /dev/null @@ -1,182 +0,0 @@ -# 0001 — WebViews use CDP, not WebDriver BiDi - -**Status:** accepted, 2026-09-12 -**Supersedes:** the original step-5 design note, which was wrong - -## The claim that turned out to be false - -The plan for hybrid-app support, written at step 1, said: - -> WebView context switching that hands off to Vibium itself — a WebView is -> BiDi/CDP-attachable, so `vibium:element.*` works unchanged inside it. - -This was the most attractive part of the whole design. Mobium would drive the -native shell, hand the WebView to Vibium, and every one of Vibium's ~120 -element commands would work inside the page for free. No new protocol, no new -element model, and a genuine justification for "built on top of Vibium" rather -than merely "inspired by". - -It does not work. The two halves of "BiDi/CDP-attachable" are not -interchangeable, and that sentence quietly assumed they were. - -## What was actually measured - -Two checks, both cheap, neither done until step 5. - -**1. Does Vibium speak CDP?** No. Searching its source for any CDP surface -returns nothing: - -``` -$ grep -rln "CDP\|chrome-devtools\|/json/list\|webSocketDebuggerUrl" internal/ cmd/ -$ -``` - -Vibium connects over WebDriver BiDi and only BiDi — `internal/bidi/connect.go` -opens a WebSocket and issues `session.new`. There is no second transport. - -**2. Does an Android WebView speak BiDi?** No. Forwarding a WebView's devtools -socket and asking it directly: - -``` -$ curl http://localhost:$P/json/version -{ - "Android-Package": "com.google.android.googlequicksearchbox", - "Browser": "Chrome/124.0.6367.219", - "Protocol-Version": "1.3", - ... -} - -$ curl -o /dev/null -w "%{http_code}" http://localhost:$P/session -404 -``` - -`Protocol-Version: 1.3` is CDP. `/session` — the BiDi entry point — returns -404. An Android WebView publishes the Chrome DevTools Protocol and nothing -else. - -So: Vibium speaks only BiDi, the WebView speaks only CDP, and there is nothing -to hand off. - -## Why the assumption was so easy to make - -Chrome *the browser* supports both. Chrome exposes CDP on its debugging port -and, in recent versions, BiDi as well — and chromedriver has implemented a -BiDi-to-CDP mapper for years. "Chromium supports BiDi" is true and was the -basis for the claim. - -Android **WebView** is a different product with a different surface. It embeds -the same rendering engine but ships no BiDi endpoint and no mapper. The -generalisation from "Chromium" to "anything running Chromium" is where the -error entered. - -## Options considered - -**A. Run chromedriver as a BiDi↔CDP mapper between Vibium and the WebView.** -This would have preserved the original design exactly. Rejected: it puts a -chromedriver binary on the user's machine, matched to the WebView's Chromium -version. That breaks the single-binary, zero-runtime-dependency property, which -is the first of Mobium's three design commitments. - -**B. Implement a BiDi mapper inside Mobium.** Rejected on cost. Chromedriver's -mapper is a substantial piece of software, and Mobium needs roughly two calls' -worth of functionality from it. - -**C. Implement the slice of CDP that Mobium actually needs.** Accepted. - -## The decision - -Mobium implements two CDP methods for WebView contexts: - -- `Runtime.evaluate` — run the map script, read the page text -- `Page.getLayoutMetrics` — read the viewport, to convert coordinates - -That is the entire protocol surface. `internal/webview/cdp.go` is 231 lines -including the JSON-RPC framing. - -**Reads go over CDP; taps stay native.** This is what keeps the surface small. -The page is mapped by evaluating a script in it, the resulting CSS coordinates -are converted to device pixels, and the tap is delivered by the ordinary -UiAutomator2 driver, which already knows how to touch a point. None of CDP's -`Input` domain is used. A web element is acted on exactly as a native one is, -which also means gestures, waits and screenshots need no WebView-specific code -at all. - -The coordinate conversion takes the **native WebView element** as ground truth: -its on-screen rectangle gives the origin, and its width against the page's -visual viewport gives the scale. A `devicePixelRatio` read from the page would -be wrong wherever the WebView does not fill the screen, which is most hybrid -apps. (The viewport choice has its own trap; see -[CHALLENGES.md](../CHALLENGES.md#6-the-visual-viewport).) - -## The cost avoided - -Chromedriver's version must match the device's Chrome: a tool that bundles -one version fails on a device whose Chrome has fallen behind, with `Chrome -version must be >= 76`, and the remedies are to download a matching -chromedriver at run time or keep several. That is option A's cost. - -## Consequences - -**Good.** -- The zero-dependency property survives. Still one binary. -- The CDP surface is small enough to reason about and to test against a fake - server. -- Because taps stay native, WebView support inherited gestures and screenshots - for free. - -**Bad.** -- Mobium is not "built on top of Vibium" in the way the plan promised. The - reuse is architectural — the daemon shape, the tool layer, the `@ref` model, - the design instincts — not code. That should be said plainly rather than - implied. -- Mobium now owns a CDP client it has to maintain, small as it is. -- iOS WebViews are not covered *yet*. **This bullet was pessimistic, and is - now simply wrong**: superseded by - [0002](0002-ios-webviews-are-reachable.md), and the work was built on - 2026-09-14. The path is an ordinary Unix socket, not a wall. It is still a - third protocol — CDP for Android, Remote Web Inspector for iOS — but both - sit under one `webview.Page`, so `Map` and `Text` are shared and only the - transport differs. `app_context` works on both platforms. - -## How to re-check this - -Every claim above is a command, which is the point: a decision record whose -premises were never re-examined is indistinguishable from one whose premises -have quietly stopped being true. - -**Re-run 2026-09-14**: the first three, which need no device, all still hold — -two CDP methods, no `Input` domain anywhere, and no CDP surface in Vibium. The -fourth needs a WebView forwarded to a port and was not re-run; it is the one -that would actually change the decision, so it is worth doing the next time a -hybrid app is on a device. - -```sh -# The protocol surface really is two methods. -grep -n 'call(ctx, "' internal/webview/cdp.go - -# Taps stay native: nothing touches CDP's Input domain. -grep -rn "Input\.\|dispatchMouseEvent\|dispatchTouchEvent" internal/webview/ - -# Vibium still speaks only BiDi. -grep -rln "CDP\|chrome-devtools\|/json/list" ~/Projects/vibium/clicker/internal ~/Projects/vibium/clicker/cmd - -# An Android WebView still publishes CDP and not BiDi. With a WebView -# forwarded to $P: -curl -s http://localhost:$P/json/version | grep Protocol-Version # 1.3 = CDP -curl -o /dev/null -w "%{http_code}" http://localhost:$P/session # 404 = no BiDi -``` - -If the last one ever returns 200, this decision is worth revisiting: a WebView -that speaks BiDi would make the original handoff possible after all. - -## What this should change about how the plan is read - -The rest of the original design held up well under measurement — the driver -seam, the neutral locator vocabulary, the ref model, the daemon. This one item -did not, and it failed in a specific way worth remembering: **it was the only -step whose feasibility rested on a protocol claim nobody had checked.** The -two commands that disproved it took under a minute and could have been run on -day one. - -Any remaining plan item that asserts "X is attachable/compatible/supported" -should be treated the same way: verify before building on it. diff --git a/docs/decisions/0002-ios-webviews-are-reachable.md b/docs/decisions/0002-ios-webviews-are-reachable.md deleted file mode 100644 index aa2fab18..00000000 --- a/docs/decisions/0002-ios-webviews-are-reachable.md +++ /dev/null @@ -1,208 +0,0 @@ -# 0002 — iOS WebViews are reachable, and the path is a Unix socket - -**2026-09-13.** Supersedes the pessimism in -[0001](0001-cdp-not-webdriver-bidi.md) about WKWebView, which called it "a -third protocol again" and left it as a wall. Evidence: -[../checks/ios-webview-probe.sh](../checks/ios-webview-probe.sh), which walks -the whole path and passes on an iPhone 17 Pro simulator running iOS 26.5. - -## What was believed - -That WKWebView needed `com.apple.webinspectord_sim`, which was noted without -anyone checking what that actually is. The phrase "a third protocol after BiDi -and CDP" set the expectation of something like the BiDi problem — a protocol -mismatch with no bridge — and the item sat unstarted because of it. - -## What is actually true - -It is a **Unix domain socket**, dialable with `net.Dial("unix", …)`. No XPC, -no Mach ports, no cgo, no `ios_webkit_debug_proxy`, no third-party binary. - -The host's `com.apple.webinspectord` does expose Mach endpoints, which is the -misleading part and probably where the pessimism came from. The *simulator's* -copy of the same service also publishes a socket, and that is the one to use: - -``` -$ xcrun simctl spawn launchctl print system/com.apple.webinspectord - ... - RWI_LISTEN_SOCKET => /private/var/tmp/com.apple.launchd.XXXX/com.apple.webinspectord_sim.socket -``` - -The path changes per boot, so it is discovered, not hardcoded. - -## The whole path, each step verified - -| Step | Message | Answer | -| --- | --- | --- | -| 1 | dial the socket | connects | -| 2 | `_rpc_reportIdentifier:` | `_rpc_reportCurrentState:`, then `_rpc_reportConnectedApplicationList:` | -| 3 | `_rpc_forwardGetListing:` | `_rpc_applicationSentListing:` — page id, title, URL | -| 4 | `_rpc_forwardSocketSetup:` | `Target.targetCreated` | -| 5 | `_rpc_forwardSocketData:` | `_rpc_applicationSentData:` | - -Framing is four bytes of big-endian length followed by a **binary property -list**. Inside `WIRSocketDataKey` the payload is **JSON** — the same shape -Mobium already speaks for Android. - -Step 3 gives exactly what `app_contexts` needs on Android today: an id, a -title and a URL per page. Two notes from running it. The `WebContent` process -appears in the application list alongside Safari and answers with an *empty* -listing, so the caller has to keep waiting rather than conclude there are no -pages. And Web Inspector must be on: - -```sh -xcrun simctl spawn defaults write com.apple.mobilesafari \ - WebKitDeveloperExtrasEnabledPreferenceKey -bool true -``` - -## The one real gotcha - -**WebKit is multi-target.** Sending `Runtime.evaluate` down the data channel -answers: - -```json -{"error":{"code":-32601,"message":"'Runtime' domain was not found"}} -``` - -which reads like the domain is missing and is not. Since iOS 12.2, commands -must be wrapped in `Target.sendMessageToTarget` with a `targetId` from -`Target.targetCreated`, and replies arrive wrapped in -`Target.dispatchMessageFromTarget`. Wrapped, the same call works: - -``` -Runtime.evaluate -> {"result":{"type":"string","value":"Example Domain @ 402x714"}} -``` - -## Built, 2026-09-14 - -Done and verified against an iPhone 17 Pro simulator: -[../checks/ios-webview.sh](../checks/ios-webview.sh) drives discovery, context -switching, reading and the refusal below, and `internal/webview` now carries -`Inspector`, `IOSSession` and the plist framing. `app_contexts`, `app_context`, -`app_text` and `app_map` work on iOS exactly as they do on Android, because -everything above `webview.Page` is shared — only the transport differs. - -Three things the protocol does that CDP does not, each found by running it and -each now recorded as a defect: - -- **Only the first connection is prompt.** Later ones wait 10.2 seconds before - the first byte, so the connection is held for the life of the session - ([CHALLENGES 45](../CHALLENGES.md)). -- **The application list, and each page's target, are announced once per - connection** — and a page's target belongs to one debugger at a time, so a - daemon left switched into a page blocks Safari's own inspector - ([46](../CHALLENGES.md)). -- **Mobile Safari's WebView element is the whole window, chrome included**, - and the page cannot see its own inset, so coordinates inside Safari are - refused rather than guessed ([47](../CHALLENGES.md)). An app's own WKWebView - is unaffected, and that is the case this is for. - -The remaining gap was honest and worth stating: **no genuine hybrid iOS app had -been driven**, only Safari. **Closed on 2026-09-17** by -[0004](0004-an-app-under-test-of-our-own.md), which also found why it could not -have been closed with an app from the store: on iOS 16.4+ the *app* must set -`isInspectable` or its WebView publishes no target at all. - -## How to re-check this - -```sh -# The socket is still where launchd says, and still a socket. -xcrun simctl spawn booted launchctl print system/com.apple.webinspectord | grep RWI_LISTEN - -# The whole path, with no mobium code in it. -docs/checks/ios-webview-probe.sh - -# And with mobium driving it. -docs/checks/ios-webview.sh -``` - -The probe is the important one: it verifies a **platform** assumption rather -than mobium's behavior, so if it fails the plan needs revisiting rather than -the code. - -## What this meant for the work - -The protocol question was settled; what was left was Go. - -- ~~**A binary plist codec.**~~ **Done** — `internal/plist`, covering - dictionaries, strings, integers, reals, booleans, arrays and data in both - directions, and refusing anything else rather than dropping it silently. - - It is checked three ways, in ascending order of how much they prove. - Round-tripping through itself shows only self-consistency, and a codec can - be self-consistently wrong. Agreeing with **Python's `plistlib`** in both - directions shows it matches somebody else's reading of the format — worth - having, and already installed anywhere `simctl` is. But the program that - decides whether these bytes are a property list is Apple's `webinspectord`, - so a test drives it: on a booted simulator it completes the handshake, lists - Safari's pages, opens the data channel and evaluates `1 + 1`, getting `2` - back. It skips with an actionable message when no simulator is there. -- **The RWI handshake and target wrapping**, which the probe script already - spells out step by step. -- **Reuse the rest.** JSON exists. The context model, `@ref` mapping and - native-tap-after-CDP-read design all come straight from Android. - -One pleasant surprise: Android needed `Page.getLayoutMetrics` and produced -[defect 6](../CHALLENGES.md), the visual-versus-layout viewport bug that put -every tap at a quarter of the right offset. Here `innerWidth`/`innerHeight` -through `Runtime.evaluate` gives the visual viewport directly — as the probe's -`402x714` shows — so the iOS side can avoid that trap rather than repeat it. - -## Where it stands now - -The codec is in. What is left is the RWI client itself — the handshake, the -listing, the target wrapping — and wiring it into `app_contexts` and -`app_context` so an iOS WebView becomes a context like an Android one. The -protocol has no unknowns left in it; the probe script and the codec's own -simulator test between them spell out every message. - -## Why this is written down rather than just done - -Because the previous belief was wrong in a way that cost the item months of -sitting still, and the correction is worth more than the implementation would -have been this session. The probe is checked in and runnable so the next -person starts from a working example rather than this document's prose — and -so that if Apple changes any of it, something fails loudly. - -## A real iPhone, 2026-09-23 - -The same protocol, reached a different way. Verified on an iPhone 15 Plus on -iOS 26.6.2: MobiumApp's WebViews, two at once, and Safari's pages, through -[../checks/mobium-app.sh](../checks/mobium-app.sh). - -**The route is usbmuxd and lockdown** — the one ios_webkit_debug_proxy -takes, and the "remote debugger proxy" people mean when -they say Safari on a device needs one: - -| Step | What | Measured | -| --- | --- | --- | -| 1 | `/var/run/usbmuxd`: list devices, read the pairing record | world-writable, no root; the record the Mac made at "Trust" | -| 2 | usbmuxd `Connect` to device port 62078 | lockdown, `QueryType` answers `com.apple.mobile.lockdown` | -| 3 | `StartSession` with the record's HostID and SystemBUID | `EnableSessionSSL: true`; TLS with the record's host certificate | -| 4 | `StartService com.apple.webinspector` | a port, and `EnableServiceSSL: true` | -| 5 | usbmuxd `Connect` to that port, then TLS | `_rpc_reportIdentifier:` answers `_rpc_reportCurrentState:` | - -From step 5 on it is this document's protocol, message for message, so -`internal/webview` is unchanged except that an `inspector` can start from any -stream. usbmuxd and lockdown accept a binary plist and always answer in XML, -which is the one thing `internal/plist` had to learn. - -**The iOS 17+ route was tried first and ruled out.** The CoreDevice tunnel -Mobium already uses for WebDriverAgent carries the same service as -`com.apple.webinspector.shim.remote`, and after a one-message check-in -(`RSDCheckin`, answered by `RSDCheckin` then `StartService`) it speaks the same -protocol — measured. But: - -- its port is assigned per tunnel connection and changed between two readings - a few minutes apart; -- the service directory that would name it answered nothing on any port the - tunnel had open, and `remotectl netcat - com.apple.webinspector.shim.remote` is refused ("Unable to connect") — - macOS's own tool knows the service and will not open it for us; -- so finding the port means checking in with every service on the phone. - That was done once, to measure it, and one service answered - `ShowDialog: true`. Doing that on every connection, on somebody's phone, is - not a transport. - -The lockdown route needs the phone connected by cable, which the tunnel route -would not have. That is the price. diff --git a/docs/decisions/0004-an-app-under-test-of-our-own.md b/docs/decisions/0004-an-app-under-test-of-our-own.md deleted file mode 100644 index 49037762..00000000 --- a/docs/decisions/0004-an-app-under-test-of-our-own.md +++ /dev/null @@ -1,242 +0,0 @@ -# 0004 — The app under test has to be ours, because of one platform rule - -**2026-09-17.** Closes the gap [0002](0002-ios-webviews-are-reachable.md) left -open in its last section: *"no genuine hybrid iOS app has been driven, only -Safari."* Evidence: [../checks/mobium-app.sh](../checks/mobium-app.sh), which -passes on an iPhone 17 Pro simulator running iOS 26.5 **and on a Pixel 7 AVD -running Android 15**. One script drives both; only the backend differs. - -## What was believed - -That driving a real hybrid iOS app was blocked on hardware and signing — the -reasoning in ROADMAP, that `simctl install` takes a simulator-SDK bundle while -App Store apps ship device-signed arm64 IPAs, so the only third-party apps a -simulator can host are ones compiled from source, "which tests a build of mine -rather than something somebody shipped." - -That reasoning is correct and it is not the binding constraint. - -## What is actually true - -**On iOS 16.4 and later a `WKWebView` is invisible to Remote Web Inspector -unless the app sets `isInspectable`.** An app that does not opt in publishes -no target, so mobium cannot attach to it whatever mobium does — and the opt-in -is a property of the app, not of the device or the debugger. - -Measured, not read: TheApp v1.12.0, a third-party demo app, is a React Native -hybrid app whose -prebuilt simulator bundle installs and runs fine on iOS 26.5. mobium drives its -native side — launch, map, tap, type, navigation. Its WebView is unreachable. -Writing the developer-extras default into the app's own domain, in both -spellings, changes nothing: - -```sh -xcrun simctl spawn defaults write \ - WebKitDeveloperExtrasEnabledPreferenceKey -bool true -``` - -That probe was run three times before it counted, because the first two were -invalid in the way this project keeps finding: the first read "no contexts" on -a screen with no WebView, the second on a screen whose WebView had not loaded. -A screenshot settled it — page rendered, content visible, still zero -attachable contexts. **A zero is not a result until the same measurement has -been shown able to come back non-zero**, and here the positive control had to -be built before the negative one meant anything. - -So the App Store question is moot. Even with a signed IPA on real hardware, -**any app that has not opted in is unreachable.** The app under test must opt -in, which means the app under test must be one we control. This is not a -second-best substitute for a shipped app; for this feature it is the only -thing that can work. - -## What was built - -**MobiumApp** — React Native (Expo 57, RN 0.86.3, react-native-webview -13.16.1), bundle `dev.mobium.mobiumapp`, MIT, at -**[mobiumdev/mobium-app](https://github.com/mobiumdev/mobium-app)**. - -Outside this repository on purpose: it needs Node, Gradle and CocoaPods, and -mobium's single-binary, no-runtime-dependencies property is load-bearing -enough that a toolchain like that does not belong in the tree even under -`examples/`. **It is private today, and so is mobium.** They have to move -together — both to `mobiumdev`, and both to public on the same day — or -[../checks/mobium-app.sh](../checks/mobium-app.sh) tells a stranger to build an -app they cannot obtain. -Every `WebView` sets `webviewDebuggingEnabled`, which compiles to -`_webView.inspectable = …` under `@available(iOS 16.4, *)` — checked in the -library's source at both the setup site and the setter, and confirmed to sit -outside any `RCT_NEW_ARCH_ENABLED` branch, since Expo 57 defaults to the New -Architecture. - -Twelve screens, each a control for something specific. It started at four; the -other eight were each added because a claim here rested on a case nobody had -driven, and adding a screen was the cheapest way to test it. - -| Screen | Controls for | -| --- | --- | -| WebView Demo | `NewFrame`'s **success** path — a WebView whose frame equals its content | -| Wide Viewport Demo | defect 6, the visual-versus-layout viewport, at `width=1200` | -| Dual WebView Demo | two live inspectable pages in one application | -| Login Demo | `role=password` and `uitree.Redact` on iOS; and a screen reachable only by logging in | -| Location Demo | a GPX route watched from inside the app, through the page's own `watchPosition` | -| Pager Demo | horizontal scrolling — the axis nothing in the hierarchy carries (defect 21) | -| Interruption Demo | a real system permission dialog, and what the app still holds after it clears. Called Popup Demo until 2026-09-26 | -| Dialog Demo | every kind of dialog the app can raise, on demand: its **own** alerts with one, two and three buttons and a prompt (the App Alert Demo, merged in on 2026-09-26), an action sheet, the share sheet, the system permission prompts, and iOS's Allow Paste — the controls behind CHALLENGES 105–107 | -| Form Demo | checkboxes, radios and switches that report a state — the screen defect 65 came out of | -| Gestures ▸ Tap and Press, Drag | which gesture actually arrived. Tap, long press and a short hold are the same touches differing only in duration, so only the app can tell them apart. Drag is the **native** drag witness — how long the finger rested, how many moves arrived, how far it traveled, and whether it came up over the target, all four of which a swipe gets wrong. One screen, "Gestures Demo", until 2026-09-23, when every gesture witness moved under one Gestures entry in the order of [GESTURES.md](../GESTURES.md) | -| Gestures ▸ Rotate | a two-finger turn, measured: the page computes its own angle from `touchmove`, since neither accessibility hierarchy reports a rotation | -| Gestures ▸ Double Tap | a **double tap the browser agreed was one**. React Native has no `onDoublePress`, so a native screen could only apply a threshold of this project's own choosing and then test mobium against it; a page's `dblclick` is the platform's verdict instead — on Android. An iOS WKWebView fires no `dblclick` for injected touches, though the same double tap does trigger WebKit's zoom. Called Touch Demo until 2026-09-23, when the drag measurement it also held — the one that **did not work**, see below — was removed | -| Gestures ▸ Flick and Pan, Pinch and Spread, Multi-Touch | how far a list coasts after the lift, which is what separates a flick from a pan; a page's own zoom level; and each finger's landing, lifting and travel with every raw touch event — the evidence that found UiAutomator2 dropping a pausing finger and WebDriverAgent touching a late finger's target early (CHALLENGES 84, 85) | -| Motion Demo | what **Reduce Motion** buys a tool that waits for a target to stop moving: one target that honors the setting, one that ignores it as the negative control, each timed by the app. 2.8s against 1.6s on an iPhone 15 Plus, the control unchanged | -| Crash Demo | a crash caused on purpose, for `app_logs` and `app_crashes` on a **real iPhone**, where nothing outside an app can crash it: a numbered line to the device log, and an unhandled JavaScript error a Release build treats as fatal. Its report was listed three seconds after the tap on an iPhone 15 Plus and one second on a Pixel 7 AVD — and the error's text is in the iPhone's log, but in Android's report. Added 2026-09-25 | -| Layout Demo | the positive control for `app_screen`'s inspection: a 24dp target too small at every size, and a bar an eighth of the screen wide that becomes too narrow below 384dp. Without them a device run that found nothing read the same as one that could not see (CHALLENGES 146) | - -The drag witness is the one that had to move, and the reason belongs here -because it is about what an app under test is *for*. Measuring a drag in a -WebView, the way the rotation is measured, worked for holds up to 480ms and -reported nothing at all from 520ms up — Android's 500ms long-press timeout, -past which the WebView claims the press and the page stops receiving -`touchmove` ([CHALLENGES 68](../CHALLENGES.md)). A drag holds past that -timeout deliberately, so a page can only see the gesture when it is set up -wrongly. A screen that agrees with you whenever the question is easy is worth -less than no screen, because it reports a pass. The drag witness is native and -the double-tap witness is a page, and each is the one the platform will -actually answer. The page's drag half stayed behind after the native one -replaced it on 2026-09-22, driven by nothing, and was removed the next day: a -witness no check reads looks like coverage and is none. - -Pages are shipped inline rather than fetched. The app this was modeled on -whitelists a single domain, and that domain has since lapsed to a parked site -serving somebody else's content — a check that depends on a domain is a check -that fails when somebody stops paying for it. - -## What it closed - -Three things had never executed: - -- **`NewFrame` computing a scale on iOS.** Every previous iOS run went through - Safari, whose WebView element covers the whole window, so only the refusal - had ever run ([CHALLENGES 47](../CHALLENGES.md)). A tap inside an embedded - WebView now lands. -- **Two live WebViews in one application.** What ROADMAP recorded as "done by - accident" was two *tabs* in one browser. Two simultaneous hosts in one app - is the harder case and now passes. -- **Anything past a login**, which had been blocked on not having an account. - -And one rule is now checked on the platform it was not found on: CHALLENGES 43 -was found on Android, where the typed value lands in the node's `text`. iOS -marks the field as `XCUIElementTypeSecureTextField` instead, and `Redact` holds -there too. - -## What it does not close - -**It is our build, so it is not the evidence a third-party app produced on Android.** A -fixture encodes its author's assumptions; that is what a fixture is, and this -project has five recorded cases of a careful one producing confident wrong -behavior. Wikipedia broke three labeling rules on its first screen precisely -because nobody designing mobium had imagined them. MobiumApp cannot do that. - -Its value is the opposite one: it produces **positive controls on demand** — -the non-zero case that makes a zero mean something. Both are needed, and -neither substitutes for the other. Real third-party apps stay on the roadmap. - -## Two things React Native does to a hierarchy - -Found while driving TheApp, and true of MobiumApp too, so they are the -platform's and not one app's: - -- **An accessibility identifier is repeated across the nested views RN renders - for one component.** `label=Webview Demo` matched **14** nodes on one screen; - `testid=navigateBtn` matched 5. The locators `map` derives are still unique — - verified — so nothing is broken, but a hand-written `label=` or `testid=` - locator is unreliable against any RN app. Use a ref from `map`. -- **An empty text input is labeled with its placeholder**, so `map` prints - the placeholder, a URL, followed by `(input)` for a field containing - nothing. It reads - exactly like a value and cost two wrong turns here. - -## A correction - -ROADMAP recorded that TheApp "still cannot be installed on this -machine — `TheApp-v1.10.0.apk` ships `armeabi-v7a` and `x86` against an -`arm64-v8a` emulator." **v1.12.0 ships `arm64-v8a` and `x86_64`.** That -blocker was true of v1.10.0 and is stale. - -TheApp was used here only as a reference for what an app under test should -contain, and nothing was taken from it: it carries **no license at all** — no -`LICENSE` file, no `license` field, `"license": null` from GitHub's API — so -there is nothing to take. The screens here are chosen against this project's -defect log rather than against another tool's feature list. - -## What Android found - -The Android half was built and driven the same day, and it did what a new real -app is supposed to do: it broke something on its first pass. - -Text entry failed on the password field with `locate element (id=password): no -such element` — on a ref `map` had just emitted, for an element plainly on -screen, while `tap` on that same ref worked. React Native reports a **bare** -resource-id, and UiAutomator2's `id` strategy qualifies a bare name with the -package under test, so the device searched for `dev.mobium.mobiumapp:id/password` -and found nothing. Five Android apps had never triggered it because all five -report the qualified form, and because it needs a field whose best locator is -a testid rather than its own text — which an empty login box is and a Settings -row is not. [CHALLENGES 55](../CHALLENGES.md), fixed with a regression test. - -That is the argument for this app working, and it is worth being precise about -why: the defect was not found because the app is ours. It was found because -the app is **React Native**, which is a hierarchy shape mobium had never met. -A real third-party RN app would have found it too, and would have been better -evidence. What ours added was that it took an afternoon rather than a search. - -The app also had a real layout bug of its own on Android, which is worth -recording because it is the same trap in the other direction: React Native's -`SafeAreaView` is iOS-only, so on Android it renders as a plain `View` and the -header sat under the status bar. It looked correct on iOS and had been checked -there. - -## Caveats - -- **`mobium doctor` reporting "ANDROID_SDK_ROOT is not set" does not mean - there is no SDK.** It says so — "only needed to start emulators, not to - drive them" — and it was read here as absence, on top of checking only - `~/Library/Android/sdk`. The SDK was installed the whole time, via Homebrew - at `/opt/homebrew/share/android-commandlinetools`, with both AVDs already - created. A caveat was published saying Android could not be built before - anyone ran `sdkmanager --list_installed`. -- **Do not run the check while a build is installing to the same device.** - Doing that produced `FAIL: no WebView context — the app did not opt into - inspection`, which is exactly what the real failure prints. Builds to the - *other* platform's device are fine and overlap safely. -- **`expo run:android --device emulator-5554` exits 0 while doing nothing.** - It printed `CommandError: Could not find device with name: emulator-5554`, - never ran Gradle, produced no APK, and returned success. The serial `adb` - uses is not the name Expo wants. Drop the flag when one device is attached. - This is the project's own first rule arriving from an unexpected direction: - a zero exit is not evidence, and that is as true of the build tool as of - `adb`. -- **`expo run:android` needs `ANDROID_HOME` exported into the shell that runs - it, and says something else when it is missing.** The build fails with - `Failed to apply plugin 'com.facebook.react.rootproject'` — a Gradle error - that sends you looking at Gradle. The real cause is eight lines earlier and - easy to scroll past: `Failed to resolve the Android SDK path. Default - install location not found: ~/Library/Android/sdk`. A Homebrew install keeps the - SDK at `/opt/homebrew/share/android-commandlinetools`, so the - default lookup finds nothing. Export `ANDROID_HOME` **and** - `ANDROID_SDK_ROOT` before the command; a shell that can run `adb` is not - enough, because `adb` may be on `PATH` while the variable is unset. - -## How to re-check this - -```sh -# Build it, then drive it. Same script for both platforms. -cd ~/MobiumApp && npx expo run:ios --configuration Release --device - -export ANDROID_HOME=/opt/homebrew/share/android-commandlinetools # or it -export ANDROID_SDK_ROOT=$ANDROID_HOME # blames Gradle -cd ~/MobiumApp && npx expo run:android --variant release -docs/checks/mobium-app.sh # iOS simulator -docs/checks/mobium-app.sh emulator-5554 # Android -``` - -See the caveats above before believing a failure. diff --git a/docs/decisions/0005-errors.md b/docs/decisions/0005-errors.md deleted file mode 100644 index 19a1c6e3..00000000 --- a/docs/decisions/0005-errors.md +++ /dev/null @@ -1,140 +0,0 @@ -# 0005 — Every failure has a code - -**2026-09-23.** Mobium's failures carry a stable code, on the wire, in the CLI's -exit status, and as an exception in every client. The codes follow W3C -WebDriver's vocabulary where it has a word for the thing. - -## What it was - -A failure was a sentence. 470 `fmt.Errorf` calls and 13 `errors.New`, two error -types in the whole tool layer, and one place where all of it was flattened to -text: `CallTool`, which returned `isError: true` and a string. Each client -raised one exception type holding that string. The CLI exited 1 for everything, -a mistyped flag included. - -The sentences were good — [CONTRIBUTING.md](../../CONTRIBUTING.md) requires each to name a -cause and a remedy that works — but nothing could act on one without reading -it. Three places in Mobium's own code did exactly that, deciding by message -text: a stale session ("invalid session id"), a missing dialog ("no such -alert"), a server that could not type into one ("unknown command"). Meanwhile -WebDriverAgent and UiAutomator2 were sending proper W3C error codes, which -`errorFrom` flattened into the message. - -A caller had the same problem with no way around it. "No element matches" and -"no device" and "the phone cannot do that" all arrived as the same exception, -and a test that wanted to scroll on the first, fail fast on the second and skip -on the third had to parse prose to tell them apart. - -## What it is - -**One type, `internal/mobiumerr.Error`**: a code, the message exactly as it -always read, a remedy, whether a retry can help, machine-readable details, and -the cause it wraps. `mobiumerr.New` formats the way `fmt.Errorf` does, `%w` -included, so converting a site changed its constructor and never its wording. -The outermost code wins: a timeout that wraps a device-server error is a -timeout. - -**Fifteen codes.** Adding one is safe; renaming or removing one breaks every -client that catches it, so they are named once and checked. - -| Code | Means | Exit | -| --- | --- | --- | -| `no_device` | nothing to drive | 3 | -| `device_not_ready` | there, and not drivable yet: locked, not trusted, Developer Mode off | 3 | -| `toolchain_missing` | missing on this machine: adb, Xcode, a signing certificate, a device agent that would not download or build | 3 | -| `no_such_element` | a locator or ref matched nothing on screen; worth scrolling for | 4 | -| `ambiguous_locator` | matched more than one; narrow it, Mobium never guesses | 4 | -| `element_not_reachable` | found, and not touchable where it is | 4 | -| `no_such_context` | a WebView context that is not there | 4 | -| `no_such_alert` | a dialog was expected and none is up | 4 | -| `unsupported` | this backend or platform cannot, and says why | 5 | -| `timeout` | a wait ran out | 6 | -| `not_confirmed` | the command said it worked and reading the state back disagreed | 7 | -| `invalid_argument` | the request itself is wrong, or cobra refused the command line | 2 | -| `device_server` | the device side failed — a device server, or adb, simctl, devicectl, lockdown — in a way no narrower code names; a server's own W3C code is in `details.w3c` | 1 | -| `internal` | a bug in Mobium | 1 | -| `error` | unclassified: a daemon too old to send a code, or a code a client does not know | 1 | - -`not_confirmed` has no W3C counterpart. It is the failure this project exists -to report — `pm grant` succeeding on a permission the app never declared, iOS -dropping a keystroke while the call returned — and it gets a code of its own -rather than being folded into something vaguer. - -`device_server` was first proposed as WebDriverAgent and UiAutomator2 only. It -was widened while classifying, because 95 remaining errors were the same kind -of thing from the device's other tools, and a separate code for "adb said no" -would not change what any caller does. - -**On the wire**, a failed tool call keeps its text in `content`, so an MCP agent -reading it sees nothing new, and adds `structuredContent`: -`{code, message, remedy, retryable, details}`. `mobium pipe` forwards it -unchanged, which is how the clients get it — **since 2026-09-25**. Until then -this sentence was false: `pipe` built its own failure result without the -structured half, so every client received every failure as its base -exception, and none of the per-code exceptions below had ever been raised -against a real daemon. Both front doors now report a failure through one -function, `agent.ErrorResult` (CHALLENGES 91). - -**In the CLI**, the exit status is the group in the table. `--json` prints the -payload beside the `error` key it has always printed. - -**In the clients**, one exception per code, named the same in all five: -`NoSuchElementError` in Python and JavaScript, `NoSuchElementException` in Java -and .NET, a sentinel `ErrNoSuchElement` for `errors.Is` in Go. Each extends the -base type that already existed, so code catching the base keeps working. One -name departs from its code: `timeout` is `TimedOut`, because `TimeoutError` is -a Python builtin and `TimeoutException` a Java one, and shadowing either makes -an `except` clause catch the wrong thing without a word. - -## How it stays true - -- `internal/apisurface` checks that every code has its exception in every - client, mapped to the right wire string, and that no client carries a code - the server does not send. A control test removes, renames and remaps one - exception and watches all three reported. -- It also parses every file under `internal/` and fails on any `fmt.Errorf` or - `errors.New` that creates an error without a code — one that wraps with `%w` - passes, since its cause's code shows through. Its control is a file with two - bare errors and one wrapping one. -- Each client tests its own mapping, with no framework: Go and Java and .NET in - their existing suites, Python and JavaScript in one small script each that - `make clients` runs. -- The three decisions made by matching text now read the code. - `staleSession` keeps its text match as a fallback, for replies that arrive - with only an HTTP status line. - -## Measured - -Every code a caller is likely to meet was produced by a real failure, through -the CLI, on the iPhone 15 Plus (iOS 26.6.2) and an iPhone 17 Pro simulator: -`no_such_element` (exit 4), `ambiguous_locator` (4, with `matches: 24` in its -details), `unsupported` (5, appearance on a phone), `timeout` (6), -`no_such_context` (4), `invalid_argument` (2, a bad lock state, an app that is -not installed, and a command line cobra refused), `no_device` (3, a serial that -does not exist) and `device_not_ready` (3) — the last by accident: the phone -was still locked from an iPhone Mirroring test, and every command said so at -once instead of waiting two minutes (defect 80's fix, reached through its code). - -**Through the clients**, on 2026-09-25 and not before: `no_such_element`, raised -by tapping a locator that matches nothing, arrives as `NoSuchElementError` in -Python and JavaScript, `ErrNoSuchElement` in Go and `NoSuchElementException` in -Java and .NET — `docs/checks/clients.sh`, against a Pixel 7 AVD. Every code -above was measured through the CLI, which is why the missing half of the pipe -went unseen: the CLI reads the daemon's error directly and never goes through -`pipe`. - -Two results needed a second look, and are recorded because the first look was -wrong both times. A tap meant to fail on an off-screen row succeeded, because -Mobium scrolled to it by itself. A wait meant to time out on "Never" succeeded, -because text matching is a substring match and the screen said "whenever". -Both were re-run with targets that could not exist. - -Not produced live: `not_confirmed`, `element_not_reachable`, -`no_such_alert`, `toolchain_missing`, `device_server` and `internal` — each -comes from a failure that is hard to cause on purpose on a healthy device. -Where they are tested instead: `not_confirmed` at its site (a keystroke iOS -drops, `wda_test.go`); `element_not_reachable` in the locator tests -(`errors_test.go`); `no_such_alert`, `device_server` and `unsupported` in the -W3C mapping (`TestServerErrorsKeepTheirW3CCode`). `toolchain_missing` and -`internal` are assigned by the classification but asserted nowhere yet; the -client suites prove only that each would arrive as its own exception. diff --git a/docs/decisions/0006-a-test-runner.md b/docs/decisions/0006-a-test-runner.md deleted file mode 100644 index f2043625..00000000 --- a/docs/decisions/0006-a-test-runner.md +++ /dev/null @@ -1,373 +0,0 @@ -# 0006 — A test runner: `mobium test`, over JSON test files - -**2026-09-28. Iteration 1 built the same day** — see "What iteration 1 -showed", at the end. Mobium gets a test runner — -`mobium test`, projects, workers, retries, reporters, a report to open — whose tests are **JSON files of the steps -`app_batch` already runs**, executed by the Go binary itself. - -## Why now - -Every tool a test needs exists: actions that wait and refuse, `app_wait_for` -for every state an assertion checks (visible, hidden, text, value, enabled, -checked, focused), `app_batch` to run checked sequences, sessions that put the -device back, and four kinds of device verified. What does not exist is the -thing a CI job runs and publishes: an artifact a CI job can publish is the -difference between a tool and a testing framework. Until now the tests of -Mobium itself have been the shell scripts in `docs/checks/`, which is the -same gap from the inside. - -## What was decided - -**Declarative test files, run by Mobium.** Not tests in the client languages -run by pytest, Jest or JUnit: that is five runners, five report formats and -five sets of fixtures to keep in step, and none of them works without a -language toolchain. A declarative file keeps the property this project rests -on — one static binary, nothing else installed — and it is what an agent -writes most reliably. Tests in the client languages stay possible later; they -are programs that call Mobium, as they are today. - -**JSON, not YAML.** It was YAML until asked why, and the answer held up: - -- The standard library reads it; YAML would be a new module. -- Everything else in Mobium is JSON — `app_batch` steps, MCP arguments, the - tool schemas, `--json`, `docs/api/*.json` — so a test step is **exactly** a - batch step, checked by the same code. -- YAML reads an unquoted `off` as a boolean under some parsers, and Mobium - has real values like `bold_text off`. A test that quietly means something - else is the failure this project exists to refuse. -- It is not a one-way door: every JSON file is valid YAML, so YAML can be - added later as a second reader of the same schema without changing a test. - The loss is comments; `description` on a test and on a step covers most of - it. - -**Steps are `app_batch` steps; assertions are `app_wait_for`** (more in -"Assertions" below). Mutatis -mutandis: no second vocabulary. A step is `{"name": "app_tap", "arguments": -{...}}`; an expectation is an `app_wait_for` step, which already retries until -its timeout and fails saying what the screen showed — an auto-retrying -assertion. A dedicated `expect` shorthand is a later nicety, not -a new mechanism. - -**The runner is a client of the tools, not a tool.** It lives in -`internal/testrun` and calls the daemon as the clients do: one `app_batch` -per test, so a test fails with its step's own error code and number. The -rule that the CLI implements no behavior is about tools — what a tool does, -MCP must get too. A runner reads files and writes reports around tools; it -is the clients' kind of code. If agents need to run a test file over MCP, an -`app_test` tool can wrap the same package later. - -## Assertions - -Three layers, each the existing mechanism with what it lacks added, never a -parallel one. - -**On screen: `app_wait_for`.** An assertion on the screen is a check that -retries until its timeout, and a failure that says what it found. That is -what `app_wait_for` already is — "timed out after 10s waiting for -testid=username to hold \"mo\" — its value is \"mob\"". What a test needs -to assert, against what it had: - -| A test asserts | `app_wait_for` | Gap | -| --- | --- | --- | -| shown, hidden | `visible`, `hidden` | — | -| enabled, disabled | `enabled`, `disabled` | — | -| checked, unchecked | `checked`, `unchecked` | — | -| focused | `focused` | its negation | -| a field's value | `value`, exact; a password field is refused | its negation | -| text containing | `text`, a substring | its negation | -| text, exactly | — | missing | -| how many match | — | missing | -| the opposite of any of these | only where a pair exists | missing | - -Iteration 1 adds three arguments to `app_wait_for` — `exact` for `text`, a -`count` condition, and `not`, which inverts any condition — so the CLI, MCP -and all five clients get them with the runner. - -**Off screen: `expect` on a read-only tool.** A test checks more than the -screen: that the app is in front (`app_state`), a dialog is up -(`app_alert`), the device is offline (`app_network`), a notification -arrived. Every tool already answers with structured data, so one step shape -covers them all: - -```json -{"expect": {"tool": "app_state", "arguments": {"app": "dev.mobium.mobiumapp"}, - "field": "state", "equals": "foreground", "timeout_ms": 5000}} -``` - -The runner calls the tool, compares the one field, and retries until the -timeout by the same polling rule as `app_wait_for`; a failure prints the -value last seen. **Only a read-only tool is accepted**, or a retrying -assertion could tap five times. Nothing marks a tool read-only yet, so -iteration 1 adds MCP's own `readOnlyHint` annotation to every tool's schema — -useful to MCP clients in its own right — and `internal/apisurface` fails the -build on a tool that does not declare it either way. - -**For every assertion:** - -- **A password is never compared.** `value` already refuses a password - field; an `expect` on a field carrying device text goes through the same - redaction as `map` (CHALLENGES 43). -- **A failure leaves evidence**: the screenshot and `map` of the moment it - failed, in the report. On a real phone the report says it holds a - screenshot. -- **Each assertion has its own timeout**, ten seconds by default as - `app_wait_for`'s, inside the test's. -- **An assertion is shown it can fail.** The suite has, for each kind, a - test that must fail, and counts as passing only when it does. - -**Later: soft assertions**, which record a failure and carry on. A test is one -`app_batch`, which stops at the first failure; soft steps mean splitting a -test into batches at each one, which iteration 2 can do. - -**Not planned: screenshot comparison.** Pixels differ between devices, OS -versions, dark mode and a clock in the status bar; `toHaveScreenshot` would -be a source of flaky failures, not of evidence. - -## The files - -A test file, `*.test.json`: - -```json -{ - "description": "The login demo's negative paths", - "app": "dev.mobium.mobiumapp", - "beforeEach": [ - {"name": "app_tap", "arguments": {"target": "label=Login Demo"}} - ], - "tests": [ - { - "name": "wrong password", - "steps": [ - {"name": "app_fill", "arguments": {"target": "testid=username", "text": "mobium"}}, - {"name": "app_fill", "arguments": {"target": "testid=password", "text": "wrongpass1"}}, - {"name": "app_tap", "arguments": {"target": "testid=loginBtn"}}, - {"name": "app_wait_for", "arguments": {"target": "testid=loginError", - "condition": "text", "text": "Incorrect username or password."}} - ] - } - ] -} -``` - -Each test starts from a fresh app: `app` is terminated and launched before -`beforeEach`, and the session's end puts back anything a test changed — -network, accessibility, as sessions already do. Tests in a file run in order -on one device; files are what workers share out. - -A config, `mobium.config.json`, found by walking up from the working -directory: - -```json -{ - "testDir": "tests", - "timeout": 60000, - "retries": 0, - "projects": [ - {"name": "android", "device": "emulator-5554"}, - {"name": "ios", "device": "457C7DC2-C706-45D9-8D68-1D26953E28B1", "driver": "wda"} - ] -} -``` - -A project is a device. -A device serial in a checked-in file is fine for an emulator and a simulator; -a real phone's identifier is personal and belongs in a local override or the -command line, never the repository (the UDID scrub of 2026-09-23 is why). - -## The commands - -| `mobium` | What it does | Iteration | -| --- | --- | --- | -| `mobium test` | every `*.test.json` under `testDir` | 1 | -| `mobium test tests/login.test.json` | one file | 1 | -| `mobium test -g login` | tests whose name matches | 1 | -| `mobium test --last-failed` | what failed last time, from `.mobium-test/last-run.json` | 1 | -| `--project=android` | one project | 1 | -| `--workers` | one per device, projects in parallel; a device holds one session, so never two workers on one | 1 | -| `--retries=2` | a pass after a retry is reported **flaky**, not passed | 1 | -| `--timeout=30000` | per test | 1 | -| `--reporter` | `list` (default), `json`, `junit`, `html` | 1 | -| `mobium show-report` | opens the last HTML report | 1 | -| `--version`, `test --help` | exist already | — | -| a window | a simulator is headless until `open -a Simulator`; an emulator decides at launch | later | -| `--debug` | stop before each step, show it and the map; Enter steps, `c` continues, `q` quits | 2 | -| `--ui` | an interactive mode | later | -| `--trace on`, `retain-on-failure` | each step's screenshot and map, a filmstrip in the HTML report | 2 | -| a trace viewer | the trace is in the HTML report, so `show-report` opens it | 2 | -| recording a test | the tool calls a person or agent makes, kept as a test file — the daemon already sees every call | later | -| setting up | `mobium doctor` checks; the device-side agents install themselves, pinned and checksummed | mostly exists | - -What a failure leaves: the step that failed, its error code and message, and -a screenshot and a `map` taken at the moment of failure, in the report. `map` -is already redacted; a **screenshot of a real phone can show a person's -data**, so on a phone the report says it holds one, and a flag turns them -off. - -Exit status: 0 when every test passed (flaky counts as passed, and is -listed), 1 when any failed, and the existing codes when the run could not -start — no device is `no_device`, a bad file `invalid_argument` — so CI can -tell "the app is broken" from "the test run is". - -## How iteration 1 will be shown to work - -By the rule the rest of this project follows — a result is not a result until -it has been seen to come back the other way: - -- A `tests/` directory of MobiumApp flows, ported from `docs/checks/` - (login, OTP, wait states), run on the emulator and the simulator, and then - the phones. -- **A test that must fail**, kept in the suite behind a flag, to show a - failure is reported with its step, code, screenshot and map, and exits 1. -- **A test that fails once and then passes**, to show `--retries` reports it - flaky rather than passed. -- The JUnit file read by a consumer that is not ours (a JUnit schema check - in CI), because a report nobody can parse is not an artifact. -- `--workers 2` across two devices, timed against one, and each report line - naming the device it ran on. -- `--last-failed` after a run with one failure runs exactly that one. - -## What it does not decide - -- A shorthand for steps (`{"tap": "label=Login Demo"}`). Worth doing once the - long form has been written by hand enough to know which steps are common. -- Test-level parameters and data (the same test on several inputs) — done - later; see "Parameters" below. -- Sharding across machines — the grid already lends devices; how a run - splits across it is iteration 2 at the earliest. - -## What iteration 1 showed - -Built as planned: `internal/testrun`, `mobium test` and `mobium show-report`, -the three `app_wait_for` arguments, and `readOnlyHint` on every tool with -`IsReadCall` per call. The suite is `tests/`, and `docs/checks/test-runner.sh` -holds the runner to its controls. On the Pixel 7 AVD and the iPhone 17 Pro -simulator together, the check passed its first run: an unset device refused -before any test; 14 tests passing on both platforms in 95 seconds for 189 -seconds of work; all six must-fail tests failing with their step, code and a -screenshot, and Python's XML parser agreeing with the JUnit counts; -`--last-failed` running those six and no others; and the flaky control failed -without a retry and reported flaky, on attempt 2, with one. - -Three things the first real runs changed: - -- **A project's device is settled before any test runs.** With a phone and a - simulator both attached, "the iOS device" was ambiguous, and the first run - reported that seven times, once per test, as seven failures. It is now one - refusal before anything runs — a run that cannot start is not a test that - failed. -- **A project's device can come from the environment**, as - `"${MOBIUM_IOS_DEVICE}"`, and an unset variable is refused by name. A - simulator's id is one Mac's and a phone's is somebody's; neither belongs in - a checked-in config. -- **A test that presses enter behaves differently by platform.** Enter in - MobiumApp's password field submits the form on iOS and not on Android, so - a test that pressed it and then tapped Log In signed in twice on iOS and - found no Log In to tap. The map kept with the failure showed `Log Out`, - which is what decided it; the test now taps Log In and nothing else. That - is the report's evidence doing its job on its first day. - -Next, for iteration 2: the step shorthand, soft assertions, a trace per test -(the session-recording item), `--debug`, and runs on the real phones, where a -password needs the phone's keyboard to have its letters (CHALLENGES 159). - -**Through a grid, 2026-09-28.** A grid route is one process's — the lease's -holder, the device, the forward to its node — so with `MOBIUM_GRID` set the -runner gives each project a `mobium pipe` of its own, as every client has -one, and its first call, a session start carrying the project's `platform`, -leases the device. Closing it releases the lease. The runner's own process -makes no call, which would lease a device nothing used. Every result names -the device its project got. `docs/checks/test-grid.sh` passed with two -emulators on one node: two projects on two devices at once, both leases gone -after, and a third project refused with exit 3 and nothing left held. - -**A daemon and a session per project, 2026-09-28.** Off a grid too, each -project now gets its own `mobium pipe` and a daemon named for the run. A -test that went offline had left the emulator shaped after the run, because -the run never ended the CLI's shared session (CHALLENGES 163); now closing a -project's connection ends its session, which puts back what the tests -changed, and the daemon is stopped. Projects no longer queue behind one -daemon: the suite on two emulators took 40 seconds for 75 seconds of work. - -## What iteration 2 showed - -Built on 2026-09-29, and held to its controls in `docs/checks/test-runner.sh` -on the Pixel 7 AVD and the iPhone 17 Pro simulator: - -- **The step shorthand.** `{"tap": "label=Login Demo"}` is a tool's name - without `app_`, and a string goes to the argument the tool's schema makes - its main one — `target` where there is one, else its one required string, - else its only argument — or an object of its arguments. It is only a - spelling: read into the long form, and checked as the long form is, so a - misspelled key is still refused, and a tool with no one main argument says - to name them (`{"swipe": "up"}` is refused; `{"swipe": {"direction": - "up"}}` is not). Written by hand enough to know which steps are common, as - "What it does not decide" asked: `tests/mobiumapp/form.test.json` is in it, - and `login.test.json` keeps the long form, so the suite runs both. -- **Soft assertions**, as "Assertions" planned: `"soft": true` on a - `wait_for` or an expect runs that step alone, outside the batch around it, - and records its failure — with its own screenshot and map — and the test - goes on. The test still fails, and every failure is in the result, the - list line, JUnit and the report. Only an assertion can be soft: an action - that failed leaves nothing after it worth checking. The control, - `tests/controls/soft.test.json`, fails with both of its soft failures and - reaches its last step, which is how it shows the test carried on. -- **A trace per test**: `--trace on` or - `retain-on-failure`, or `trace` in the config. Steps run one at a time - rather than as a batch, and after each the runner keeps a screenshot and - the map; the HTML report shows them as a filmstrip. A traced run is slower - by a screenshot and a map a step, which is why it is off by default. - `--no-screenshots` keeps the maps and no picture, for a phone. Since - 2026-10-01 each kept attempt is also a recording in Vibium's record format, `trace.zip` in - its trace directory, as `mobium trace` writes one, linked from the report - and named in `trace_file`: the runner starts a session trace before the - app is launched and stops it after the last step, and sends its own - screenshots and maps with `_meta` `dev.mobium/untraced`, which keeps them - out of it. -- **`--debug`** stops before every step, prints it in the long form and the - map, and waits: Enter runs the step, `m` maps again, `c` runs the rest of - the test, `q` quits — the test is reported `stopped`, and nothing after it - runs. The test's timeout is off while a person reads. One project only, - since two would stop at once. With nobody answering — input closed — it - runs on rather than hang. - -Left for later: the phones, an interactive mode, recording a test from what a -person does (`codegen`), and test parameters. - -## Parameters - -Added 2026-10-01: a parameterized test, a test in a loop over its data, as -a key on the test. `"each"` is a list of cases, objects, and -the test runs once per case as a test of its own: - -```json -{"name": "a bad sign-in is refused, and says why: ${why}", - "each": [ - {"why": "a wrong password", "pass": "wrongpass1", "error": "loginError"}, - {"why": "a short password", "pass": "abc", "error": "passError"} - ], - "steps": [ - {"fill": {"target": "testid=password", "text": "${pass}"}}, - {"tap": "testid=loginBtn"}, - {"wait_for": {"target": "testid=${error}", "condition": "visible"}} - ]} -``` - -- `${key}` in any string of a step — its arguments, an expect, its - description — is the case's value. A string that is nothing but one - `${key}` takes the value's own type, so `"count": "${rows}"` stays a number. - `$${` is a literal `${`, in any test. -- The name is the case's too. With a `${key}` in it, it is filled in; without - one, the cases are numbered — `[1]`, `[2]` — and not named after their - values, which may be a password and would then be in every report. - Expanded names must be unique, as any test's are, so `-g` and - `--last-failed` find one case. -- Cases become tests when the file is read, before anything runs, and are - checked as any test is. A `${key}` a case does not have is refused naming - the case and the keys it has; one in a test with no `"each"` is refused - too, since it would be typed as written. - -In `mobium.config.json` `${NAME}` is an environment variable — the iOS -project's device — and in a test file it is a case's value; the two are read -separately. The suite's Login test runs three cases, and a copy with one -case's message wrong failed that case alone, on an emulator and a simulator, -with the substituted values in the failure. diff --git a/docs/decisions/0007-an-inspector.md b/docs/decisions/0007-an-inspector.md deleted file mode 100644 index e1e4afd4..00000000 --- a/docs/decisions/0007-an-inspector.md +++ /dev/null @@ -1,85 +0,0 @@ -# 0007 — An inspector: `mobium inspect` - -**2026-09-29. Iteration 1 built the same day** — see "What iteration 1 -showed", at the end. A page on this machine -that shows a device's screen with every element `map` finds drawn over it, -says for the one clicked what `map` says of it — its ref, role, state and -the locator to write — acts on it, and keeps what was done as a test file. - -## Why - -Every other surface asks the reader to already know the vocabulary: which -locator names this button, whether `text=` or `label=` finds it on this -platform, whether it is one element or three. The answer is on the device, -and until now reaching it meant running `map` and reading coordinates. An -inspector is how someone learns the locator vocabulary without reading -anything, and often how they first get anything working at all. - -It is also the missing half of `mobium test`: decision 0006 left recording a -test from what a person does as "later", and the daemon already sees every -call. An inspector that acts through the tools can keep those calls. - -## What was decided - -**A client of the tools, not a tool.** Like the test runner and the five -clients, it calls the tools the CLI calls — `app_screenshot`, `app_map`, -`app_find`, and the actions — through the same daemon, so a terminal and -the inspector drive the one session side by side, and an action from the -page is the action a command takes, waiting and refusing the same way. -Nothing here implements behavior: what the page shows is what the tools -answered. It lives in `internal/inspect`; `mobium inspect` serves it. - -**What the page does, in iteration 1:** - -- the screen, from `app_screenshot` with no path (the PNG comes inline), and - over it a box for every entry `app_map` returns, in the same device pixels - the screenshot is in, so no unit is converted; -- the element clicked: its ref, label, role, checked state, bounds, and the - locator `map` derived for it — the durable one, not the ref, which does - not survive the next screen; -- a locator typed in, resolved with `app_find`, every match outlined and the - count said — so "matches 3 elements" is seen, not read about; -- actions on the element clicked — tap, double tap, long press, type, fill, - check, wait for it — and on the screen — back, home, swipe; each refreshes - the picture after; -- **what was done, recorded** as `*.test.json` steps in the step shorthand, - targeted by locator, and downloaded as a test file `mobium test` runs. - -**Local, and only to the page it printed.** It listens on 127.0.0.1 only, as -`mobium grid ui` does. That is not enough for a page that taps a device: any -web page open in the same browser can send a request to 127.0.0.1. So every -request carries a random token that is in the URL the command prints, and -nowhere else, and its `Host` must be the address it listens on, which -refuses a DNS-rebinding page too. A request without both is refused before -any tool is called. - -**Only the tools it needs.** The page can ask for a fixed list of actions; it -cannot call `app_uninstall`, `app_clear_data` or `app_eval` by naming them. - -## What it does not decide - -- **A live stream.** The picture is a screenshot after each action and on - request, not video. Streaming is its own question, platform by platform. -- **The raw hierarchy.** `app_source` is one call away and can be added as a - view; iteration 1 shows what `map` shows, which is what a test acts on. -- **Phones.** A phone's screen is somebody's; the page shows it only to the - person who started the command, on their own machine, and saves nothing - unless they download a test. - -## What iteration 1 showed - -Built as planned, in `internal/inspect` with `mobium inspect` serving it, and -driven on MobiumApp's Login Demo on an Android 15 emulator: the screen with -`map`'s five elements outlined; the username field clicked, in a real -browser, showing `testid=username`; `role=button` tried and both buttons -outlined as two matches; a request without the token refused, an action by -GET refused, and `app_uninstall` refused by name. Recorded from a fresh -app — tap Login Demo, fill both fields, tap Log In, wait for Welcome — the -downloaded file ran under `mobium test` and passed. - -One thing the first recording changed: **it has to start where a test -starts.** Recorded from the Login Demo, the file began with filling a field -on a screen `mobium test`, which launches the app fresh, never reaches; the -tap that opened the demo had happened before recording began. The page now -has **Start from a fresh app**, which stops and launches the app in front as -the runner does, and starts the recording over. diff --git a/docs/decisions/0008-a-hit-test-below-accessibility.md b/docs/decisions/0008-a-hit-test-below-accessibility.md deleted file mode 100644 index 53777238..00000000 --- a/docs/decisions/0008-a-hit-test-below-accessibility.md +++ /dev/null @@ -1,148 +0,0 @@ -# 0008 — A hit test below accessibility: `mobium hit-test` - -**2026-09-29. Built and checked the same day.** On an iOS simulator, mobium -can now ask UIKit itself which view a touch would reach, and so see the one -case of [CHALLENGES 115](../CHALLENGES.md) no tree could: a tap under an -overlay hidden from accessibility. It is opt-in, because the only way in -from outside stops the app for about two seconds. - -## The problem - -MobiumApp's Obstruction Demo puts something over each of seven buttons, every -cover itself pressable, and a line at the top names whatever a tap really -reached. Since CHALLENGES 115, every action refuses a control over all of its -target, aims around one over its center, waits out a toast, and reports a -view over the point that is not a control. That handled every case on -Android and every case but one on iOS: an overlay hidden from accessibility -(`accessibilityElementsHidden`). WebDriverAgent's tree does not contain it, -so the tap was sent and landed on the overlay while every check passed. - -XCTest's own `hittable` was measured as a way out and is not one: it is an -accessibility hit test, so it says true under the hidden overlay, where a tap -lands on the overlay, and false over the pass-through view, where a tap -reaches the target. What decides where a touch goes is UIKit's -`hitTest:withEvent:`, which knows nothing of accessibility — and until this, -nothing outside the app could ask it. - -## What was decided - -**A tool, `app_hit_test` / `mobium hit-test `, not a check inside -every action.** It resolves the target as `tap` does, takes the point `tap` -would touch — the center, or the clear point when a control covers the -center — and asks UIKit, inside the app, which view a touch there goes to. -When that view is not the target or inside it, the tool fails as a refused -tap does, "failed check receivesEvents", naming the view, its class, its -bounds and whether accessibility can see it. A receiver hidden from -accessibility gets no "tap its ref" remedy, since it has none. - -**How it gets in: lldb, attached for one call.** mobium compiles a small -probe ([internal/device/hitprobe/probe.m](../../internal/device/hitprobe/probe.m)) -against the simulator SDK once per source, caches it, attaches lldb to the -app in front, `dlopen`s the probe, calls one C function that takes numbers -and returns a string — lldb's own expression evaluator could not be trusted -with UIKit's struct types — and detaches. The function's name is derived -from the probe's source, so a library an older mobium loaded into the same -app is never the one called. It finds the target's view by accessibility -identifier, falling back to its accessibility frame, since a covered -element's frame is not always the one accessibility reports. - -**Why opt-in.** The attach stops the app for about two seconds (2.7 to 3 -measured per call), which is too slow and too intrusive to put in front of -every tap. The other way in — loading the probe at launch through -`DYLD_INSERT_LIBRARIES` so it answers in milliseconds — was considered and -not chosen: it puts a listener inside the app under test, and covers only -apps mobium launched. So the check is there to ask where it matters: once -per screen in a test, or when a tap reported success and the app disagrees. - -**Where it runs.** An iOS simulator. Android refuses, saying why it needs -none: its hierarchy lists a view accessibility hides — the hidden overlay -is a clickable, unnamed view there — and every action already refuses a -control over its target. A real iPhone refuses too: attaching to an app -there needs it signed for debugging and a debug server on the phone, which -is not built. - -## What it showed - -[checks/hit-test.sh](../checks/hit-test.sh) holds the hit test to the -truth: for each case it takes the verdict, then touches that same point by -coordinates — no check, no aim — and reads what the app received. On the -iPhone 17 Pro simulator all seven agreed: refused where the touch went to -the full cover, to nothing under the plain view, to the hidden overlay -(named as hidden from accessibility) and to the scrim; "reaches" where the -target got it, at a clear point beside the center cover, past the edge -cover, and through the pass-through view. The app's outcome line was -unchanged by the hit tests themselves. - -## On a real iPhone - -Added 2026-09-30. A phone runs only code signed for it, so the probe cannot -be loaded as a library: the same question is asked as one Objective-C -expression, evaluated in the app by lldb, with the recursion over views made -an explicit stack and UIKit touched on the main thread only. The app must -let a debugger in — `get-task-allow`, which a development build from Xcode -has and an App Store app does not. - -lldb reaches an app on a phone through CoreDevice with `device process -attach`. In batch mode that attach never surfaces the stop: the process -read as running and stopped at once, `process interrupt` answered that it -must be launched, and an lldb that quit on that error took the app down -with it. So the probe runs as a command inside `xcrun lldb`'s own Python -(`internal/device/hitprobe/phone.py`), on a debugger of its own in -asynchronous mode, waits for the stop as an event, and always detaches; no -Python of the user's is involved. Measured on an iPhone 15 Plus, iOS -26.6.2: about nine seconds a case, five of them the attach, and all seven -cases of the Obstruction Demo agreed with where a raw touch went -(`docs/checks/hit-test.sh`), as they do on a simulator. - -## Loaded at launch, on a simulator - -Added 2026-09-30, revisiting "Why opt-in". `launch --hit-test` (`hit_test` -on `app_launch`, `LaunchWithHitTest` in the clients) loads the probe as the -app starts, through `DYLD_INSERT_LIBRARIES` in the environment -WebDriverAgent launches it with, and the probe listens on a Unix socket -named in `MOBIUM_HIT_SOCKET`: a file under `MOBIUM_HOME`, created readable -by its user only. Both objections to this were weighed again: - -- *A listener inside the app under test.* The socket is a file on the - Mac's own disk, which a simulator's apps share; nothing listens on a - network, and the library unsets `DYLD_INSERT_LIBRARIES` as it loads. -- *Only apps Mobium launched.* Still true, and stated: an app without the - probe is not asked, and the per-action check is opt-in per launch. A - relaunch Mobium makes itself — for a language or a time zone — loads it - again; a plain `launch` does not. - -With it loaded, every action on an element — tap, long press, check — -asks before touching, in the same place the tree's cover checks run -(`resolveAim`), and is refused as `hit-test` refuses, after waiting out -the implicit wait as a cover is. An answer of unknown, which comes for an -element named only by its label, leaves the action to the tree's checks. -`hit-test` itself answers from the loaded probe when there is one. - -Measured on the iPhone 17 Pro simulator: an answer in 0.2 to 0.5 ms over -the socket, against about two seconds through lldb; `hit-test` from the -CLI in 0.66 s, all of it the CLI and the read of the screen; a tap on -the pass-through target at a median of 621 ms with the probe and 621 ms -without. `checks/hit-test.sh` runs the seven cases as plain taps: the -three that reach were tapped and the target got each, and the four that -do not were refused with the app untouched — while without the probe, -the hidden overlay's and the plain view's taps went through, the positive -control. - -**Not on a real iPhone.** A probe loaded there would have to listen on -the phone's network for Mobium to reach it, and opening a service on -somebody's phone is not something Mobium does; `launch --hit-test` -refuses, saying so, and `hit-test` there stays the nine-second debugger -call. - -## What it does not do - -- It is not consulted by `tap` unless the app was launched with - `--hit-test`, on a simulator. Otherwise a test that needs the guarantee - calls `hit-test` before the tap. -- It answers for the key window. A system alert is another process, and has - checks of its own (CHALLENGES 105). -- It does not say whether a receiver will *handle* the touch. React Native - decides that in JavaScript: the plain view is the receiver and handles - nothing, so the tap goes nowhere — which the check reports as the touch - reaching the plain view, not the target, and that is the answer that - matters. diff --git a/docs/decisions/0009-a-test-ui.md b/docs/decisions/0009-a-test-ui.md deleted file mode 100644 index 284aa287..00000000 --- a/docs/decisions/0009-a-test-ui.md +++ /dev/null @@ -1,78 +0,0 @@ -# 0009 — A test UI: `mobium test --ui` - -**2026-10-01. Iteration 1.** The interactive mode decision 0006 left for -later: a page on this machine that lists the -suite, runs any part of it on any of its projects, shows every step with the -screen after it as it happens, and runs again what failed. - -## Why - -`mobium test` answers once a run is over, in a terminal and a report. The loop -a person is in while writing a test is shorter than that: change a step, run -that one test, look at the screen it stopped on, change it again. `--debug` -serves one test at a time in a terminal; a page that runs any test and -shows its steps as they happen serves the whole loop. - -## What was decided - -**The runner, not a second one.** A run from the page is `testrun.Run` with -the options `mobium test` builds from the same flags and config — the same -projects, a `mobium pipe` and a daemon of its own per project, the same -retries and timeout — so a test that passes on the page passes in CI. What -the page adds is which tests: `Options.Only`, by the same IDs -`--last-failed` keeps, so "run this one" and "re-run what failed" are exact, -never a pattern. - -**A client of the tools, as the inspector is** (decision 0007). Served on -127.0.0.1, every request refused unless its path carries a token only the -printed URL holds and its Host names the server: a page that taps a device -must not be reachable from any other page open in the same browser. - -**Live, step by step.** The page runs every test traced, so after each step -the runner keeps a screenshot and the map; `Options.StepDone` hands each one -over as it is taken, beside `Progress`, which hands over each result. The -page polls for both. A screenshot is served from the run's output directory -and from nowhere else. - -**The files are read again for every list and every run**, so an edited test -is the test that runs, without restarting anything — the part of -a watch mode that matters while writing one. - -**When a run ends** its HTML and JSON reports are written as `--reporter -html,json` writes them, and `--last-failed` sees its failures. - -## What it does not do, in iteration 1 - -- Stop a run midway. A test cannot be abandoned between two steps without - leaving its device somewhere nobody chose; a run is a few tests at a time - on this page, and ends. -- Pick a locator, or record. That is `mobium inspect`. -- Run on a timer when a file changes. Running is a click. -- `--debug` from the page. - -## What iteration 1 showed - -Built the same day: `internal/testui`, `mobium test --ui` (`--ui-port`, -`--open`), and `Options.Only` and `Options.StepDone` in the runner. Driven -in a browser by [checks/test-ui.sh](../checks/test-ui.sh) — the server's -tests cannot reach the page's own script — on an Android emulator and an -iPhone simulator at once: - -- the suite's nine tests listed on both projects, the three cases of the - parameterized Login test among them; -- ▶ on one test ran it on both; its first step was on the page while the - run was still going, and all fourteen step screens loaded, under a policy - that lets the page load images from itself and nothing else; -- with the page still open, the test file was changed on disk to expect a - message the app does not show: the next run was the changed test, failed - on both, and the failure quoted both messages; -- the file put back, "Re-run failed" ran exactly those two, and they passed, - and the run's HTML and JSON reports were written. - -The check's first draft looked for a step six seconds after the click and -found none: each run starts a daemon and a driver per project, as -`mobium test` does, and the first step waits for them. It now waits for a -step while the run is still going, which is the claim. That failed draft -also found a defect: killed mid-run, `mobium test --ui` left the run's -daemons running, and Ctrl-C did the same to plain `mobium test` -(CHALLENGES 199). Both now close whatever a run left open before exiting. diff --git a/docs/guides/README.md b/docs/guides/README.md index eee33a73..6854b379 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -8,7 +8,7 @@ real device, with every command and line of output what mobium printed. | [The command line](cli.md) | how the commands fit together: sessions, the map-act-map loop, locators and refs, `--json`, exit statuses, `batch`, and the daemon | | [MCP](mcp.md) | Mobium as an agent's tools: connecting a client, an agent driving a device, what a client sees, answers, failures and sessions | | [Auto-wait](autowait.md) | what an action waits for before it touches anything, what it refuses and why, and waiting on purpose with `mobium wait` | -| [Test runner](test-runner.md) | `mobium test`: a config, a test file, a failure with its evidence, retries and flaky, several devices, CI — and the step shorthand, soft assertions, a trace and a debugger | +| [Test runner](test-runner.md) | `mobium test`: a config, a test file, a failure with its evidence, retries and flaky, several devices, CI — and the step shorthand, soft assertions, a trace and a debugger, one test over several cases with `each`, and a page to run tests from with `--ui` | | [Inspector](inspector.md) | `mobium inspect`: the screen with every element outlined, the locator for any of them, trying a locator, and recording what you do as a test | | [Network conditions](network.md) | taking an Android device offline or slowing it down, reading back what it really has, and doing it inside a test | | [Another machine's devices, and a grid](grid.md) | `--remote` to one node over SSH, and `MOBIUM_GRID` to share several nodes' devices between runs | diff --git a/docs/guides/autowait.md b/docs/guides/autowait.md index f5d8f8dd..69249f47 100644 --- a/docs/guides/autowait.md +++ b/docs/guides/autowait.md @@ -6,7 +6,7 @@ checks that a finger could really press it. It **waits** for the checks that time can fix, and **refuses** the ones it can't — with a reason, never with a touch somewhere else reported as success. -This guide goes through each check on [MobiumApp](../decisions/0004-an-app-under-test-of-our-own.md), +This guide goes through each check on [MobiumApp](https://github.com/mobiumdev/mobium-app), Mobium's own app under test, whose screens were built to fail each one. Every command and every line of output below is what an Android 15 emulator printed on 2026-09-28; the iOS simulator behaves the same, and where it differs, it @@ -34,7 +34,7 @@ Five checks, each with a name that a refusal carries: | --- | --- | --- | | `visible` | is the target on the screen, not just in the hierarchy? | scrolls to it, then refuses | | `stable` | has it stopped moving? | waits, up to 5 seconds | -| `enabled` | does it take input? | waits, up to the implicit wait, then refuses | +| `enabled` | does it take input? | waits, up to the implicit wait of 2 seconds, then refuses | | `receivesEvents` | would a finger reach it — no dialog, keyboard or other control over it? | waits for a transient cover (a toast), aims around one over its center, refuses one over all of it | | `editable` | for `type` and `fill`: is it a text field? | refuses at once | @@ -221,15 +221,21 @@ person and may change. Every client raises the code as its own exception: `ElementNotReachableError` in Python and JavaScript, `ElementNotReachableException` in Java and .NET, and in Go an error that `errors.Is(err, mobium.ErrElementNotReachable)` -([decisions/0005](../decisions/0005-errors.md)). +([the codes](cli.md#6-when-a-command-fails)). ## What it does not see -- **An overlay hidden from accessibility on iOS.** WebDriverAgent's tree does - not contain it, so a tap under one lands on it, and nothing outside the app - can tell — WebDriverAgent's own `hittable` gets it wrong too, measured. It is - the one known blind spot ([CHALLENGES](../CHALLENGES.md), "Findings that - were not defects"). +- **An overlay hidden from accessibility on iOS**, unless asked for. + WebDriverAgent's tree does not contain it, so a tap under one lands on it — + WebDriverAgent's own `hittable` gets it wrong too, measured. UIKit's own + hit test does see it: `mobium hit-test ` asks it whether a touch + at the point a tap would use reaches the target, and names what it would + reach instead. It attaches a debugger to the app, about two seconds a call + on a simulator and nine on an iPhone, where the app must be a development + build. On a simulator, `mobium launch --hit-test ` loads the check + into the app as it starts, and then every action on an element in it asks + first, in under a millisecond, and is refused when the touch would land + elsewhere. - **Whether a plain view swallows a tap.** A view with no handler over the target looks exactly like one that lets touches through. Mobium taps and says a view is over the point, in the result, rather than refuse something diff --git a/docs/guides/cli.md b/docs/guides/cli.md index a1baed47..3cf50b3f 100644 --- a/docs/guides/cli.md +++ b/docs/guides/cli.md @@ -7,7 +7,7 @@ commands fit together: the loop, sessions and the daemon, the flags every command takes, reading answers from a script, and exit statuses. Every command and every line of output below is what `mobium` printed on -2026-09-28 against [MobiumApp](../decisions/0004-an-app-under-test-of-our-own.md) +2026-09-28 against [MobiumApp](https://github.com/mobiumdev/mobium-app) on an Android 15 emulator. Where output is trimmed or a path shortened, it says so. The full list of commands and what each takes is generated: [API.md](../API.md) and [FLAGS.md](../FLAGS.md). @@ -35,7 +35,9 @@ emulator-5554 device (android emulator, model: sdk_ ``` (Trimmed: the Mac also listed its iOS simulators, shut down, and a phone.) -With one device running, no command needs to be told which; with several, +To start one, `mobium boot ` boots an Android emulator by its +AVD's name, or an iOS simulator by its name or UDID, and answers once it has +booted. With one device running, no command needs to be told which; with several, `--device ` picks one, on every command. ## 2. A session @@ -108,6 +110,10 @@ $ mobium map A password field is labeled by its id and given the role `password`; what is typed into one is never printed, by `map` or any other command. +`map --diff` answers only what changed since the last map of the device — +`+` appeared, `-` went away, `~` changed label, checked state or place — which +is what the action just did; its refs are the new map's either way. + ## 4. Locators and refs Anything that takes a target takes either: @@ -198,10 +204,32 @@ $ mobium --json tap testid=noSuchButton; echo $? | 5 | `unsupported` | this device or driver cannot do it — retrying will not help | | 6 | `timeout` | it did not happen in time — the one kind worth retrying as it is | | 7 | `not_confirmed` | the command ran and reading back says it did not take — a finding | -| 1 | anything else | read the message | +| 1 | `device_server`, `internal`, `error` | the device side failed, Mobium has a bug, or the failure is unclassified — read the message | + +The codes are the same in every client, as exceptions, and on the MCP wire, +in `structuredContent`. Adding a code is safe; renaming or removing one would +break every client that catches it, so they do not change: -The codes are the same in every client, as exceptions, and on the MCP wire -([decisions/0005](../decisions/0005-errors.md)). +| Code | Means | +| --- | --- | +| `no_device` | nothing to drive | +| `device_not_ready` | there, and not drivable yet: locked, not trusted, Developer Mode off | +| `toolchain_missing` | missing on this machine: adb, Xcode, a signing certificate, a device agent that would not download or build | +| `no_such_element` | a locator or ref matched nothing on screen; worth scrolling for | +| `ambiguous_locator` | matched more than one; narrow it — Mobium never guesses | +| `element_not_reachable` | found, and not touchable where it is | +| `no_such_context` | a WebView context that is not there | +| `no_such_alert` | a dialog was expected and none is up | +| `unsupported` | this backend or platform cannot, and says why | +| `timeout` | a wait ran out | +| `not_confirmed` | the command said it worked and reading the state back disagreed | +| `invalid_argument` | the request itself is wrong, or the command line was refused | +| `device_server` | the device side failed — a device server, or adb, simctl, devicectl, lockdown — in a way no narrower code names; a server's own W3C code is in `details.w3c` | +| `internal` | a bug in Mobium | +| `error` | unclassified: a daemon too old to send a code, or a code a client does not know | + +Decide by the code, never by matching the message's words: the wording can +improve, the code is the contract. ## 7. Several steps in one call: `batch` @@ -262,7 +290,11 @@ Daemon running (pid 4384, up 1s) ``` `mobium daemon stop` stops one, ending its sessions — stop it before shutting -a device down, never after ([SHUTDOWN.md](../SHUTDOWN.md)). Keep session names +a device down, never after ([SHUTDOWN.md](../SHUTDOWN.md)). `mobium shutdown +` ends this daemon's session on the device, +then shuts the emulator or simulator down and returns once it is gone; a +real phone is refused. It ends only its own daemon's session, so stop any +other daemon on the device first. Keep session names short: a name is part of a socket path, which the OS caps at about 104 bytes. diff --git a/docs/guides/grid.md b/docs/guides/grid.md index c56651cf..3393d725 100644 --- a/docs/guides/grid.md +++ b/docs/guides/grid.md @@ -4,8 +4,8 @@ Two steps, the second built on the first: - **`--remote`** drives the devices plugged into one other machine, over SSH — with any command and any client, and no change to its code. -- **`MOBIUM_GRID`** spreads runs over several machines' devices, as - Selenium Grid does, without a hub: each run gets the first free device that +- **`MOBIUM_GRID`** spreads runs over several machines' devices, with no + hub: each run gets the first free device that matches, and waits when there is none. Every command and every line of output below is what mobium printed on diff --git a/docs/guides/inspector.md b/docs/guides/inspector.md index 823df7cd..b269683c 100644 --- a/docs/guides/inspector.md +++ b/docs/guides/inspector.md @@ -3,11 +3,10 @@ A page on your own machine with the device's screen in it: every element `map` finds outlined, the locator for the one you click, actions on it, a place to try a locator and see what it matches — and what you do, kept as a -test `mobium test` runs. The design and its reasons are -[decisions/0007](../decisions/0007-an-inspector.md). +test `mobium test` runs. Everything below was run on 2026-09-29 against -[MobiumApp](../decisions/0004-an-app-under-test-of-our-own.md) on an +[MobiumApp](https://github.com/mobiumdev/mobium-app) on an Android 15 emulator; the output is what mobium printed. Before this guide: [the quick start](../quickstart/README.md), and MobiumApp installed. diff --git a/docs/guides/mcp.md b/docs/guides/mcp.md index f00c1134..347805f0 100644 --- a/docs/guides/mcp.md +++ b/docs/guides/mcp.md @@ -6,7 +6,7 @@ devices with the same tools the command line and the language clients use. This guide is how to connect one, what it sees, and how its answers and failures come back. -Everything below is what ran on 2026-09-28 against [MobiumApp](../decisions/0004-an-app-under-test-of-our-own.md) +Everything below is what ran on 2026-09-28 against [MobiumApp](https://github.com/mobiumdev/mobium-app) on an Android 15 emulator: a Claude Code agent given only Mobium's tools, and the raw protocol, as an MCP client speaks it. Long answers are trimmed where they say so. The full tool list is generated: [API.md](../API.md). @@ -123,7 +123,7 @@ The handshake, as sent and answered: } ``` -`tools/list` answered with 63 tools, 16 of them marked read-only. One, with +`tools/list` answered with 71 tools, 19 of them marked read-only. One, with its description cut short and its schema reduced to the argument names: ```json @@ -150,7 +150,7 @@ its description cut short and its schema reduced to the argument names: Every schema says `additionalProperties: false`, so an argument a tool does not take is refused rather than ignored. `readOnlyHint` is on every tool: -true for the 16 that only read — `app_map`, `app_text`, `app_state` and the +true for the 19 that only read — `app_map`, `app_text`, `app_state` and the like — and false for everything that can change the device. ## 4. Answers: text, data and images @@ -266,8 +266,8 @@ message, a remedy that works, and whether retrying could help: ``` The codes are the same everywhere — the command line's exit status, every -client's exceptions — and listed in [the CLI guide](cli.md#6-when-a-command-fails) -and [decisions/0005](../decisions/0005-errors.md). An agent should decide by +client's exceptions — and listed in [the CLI guide](cli.md#6-when-a-command-fails). +An agent should decide by `code`, and read `remedy` for what to do next. ## 6. Sessions, and sharing a device @@ -293,5 +293,5 @@ disconnects. | Runs as | `mobium ` through the daemon | `mobium mcp`, in its own process | `mobium pipe`, spawned by the client, through the daemon | | Answers | text, or `--json` | `content` and `structuredContent` | typed results and exceptions | -The same 63 tools behind each, checked by the build: a tool reachable from one +The same 71 tools behind each, checked by the build: a tool reachable from one front door and not another fails it ([API.md](../API.md)). diff --git a/docs/guides/network.md b/docs/guides/network.md index 406afdd2..81dbdd1b 100644 --- a/docs/guides/network.md +++ b/docs/guides/network.md @@ -106,7 +106,8 @@ still coming, the bar under the address still moving: ![GitHub's header loaded and the page below it still blank, the progress bar under the address bar part-way](images/network-3-slow.jpg) -`--json` gives the same answer to code: +`--json` gives the same answer to code — read here while the shaping above +was on, before `--reset`: ``` $ mobium --json network diff --git a/docs/guides/test-runner.md b/docs/guides/test-runner.md index b70853ce..068f71d0 100644 --- a/docs/guides/test-runner.md +++ b/docs/guides/test-runner.md @@ -1,13 +1,11 @@ # Quick start: `mobium test` From an empty directory to a test suite that runs on a device, fails with -evidence, retries, and leaves a report CI can publish. The design and its -reasons are -[decisions/0006](../decisions/0006-a-test-runner.md). +evidence, retries, and leaves a report CI can publish. Every command and every line of output below is what `mobium test` printed on 2026-09-28 — section 8 on 2026-09-29 — against -[MobiumApp](../decisions/0004-an-app-under-test-of-our-own.md) on an Android 15 +[MobiumApp](https://github.com/mobiumdev/mobium-app) on an Android 15 emulator. Where a path to a report is shortened, it says so. Before this guide: [the quick start](../quickstart/README.md) — mobium @@ -24,6 +22,8 @@ installed and a device running — and MobiumApp installed - [6. More than one device](#6-more-than-one-device) - [7. In CI](#7-in-ci) - [8. Shorter steps, soft checks, a trace and a debugger](#8-shorter-steps-soft-checks-a-trace-and-a-debugger) +- [9. One test over several cases: `each`](#9-one-test-over-several-cases-each) +- [10. A page to run tests from: `--ui`](#10-a-page-to-run-tests-from---ui) - [Writing tests](#writing-tests) - [The commands](#the-commands) @@ -170,11 +170,13 @@ The previous run's failures are kept in `mobium-report/.last-run.json`. test that fails and then passes is **flaky**: the run passes, and it says so. This is the runner's own control for it, from `tests/controls/` in this repository — it saves a counter, and passes only once the counter reaches 2, -which the retry's relaunch makes it do: +which the retry's relaunch makes it do. It counts from cleared app data, so +clear it first — a counter left at 2 by an earlier run fails every attempt: ``` -$ mobium test tests/storage.test.json --retries 1 - flaky [android] storage.test.json › passes on its second attempt (7.3s) — passed on attempt 2 +$ mobium clear-data dev.mobium.mobiumapp +$ mobium test tests/controls/flaky.test.json --retries 1 + flaky [android] flaky.test.json › passes on its second attempt (7.3s) — passed on attempt 2 0 passed, 1 flaky (7.3s) ``` @@ -207,8 +209,9 @@ so two workers never share one — and `--workers 1` runs them one after another. On a grid, a project can name a `platform` instead of a device, and each project leases its own for the run: [the grid guide](grid.md#4-tests-on-a-grid). Each project also gets a daemon of its own for the run, so projects on different devices never queue behind one another: this repository's suite, -[tests/](../../tests/README.md), ran its 14 tests on two Android emulators in -40 seconds, for 75 seconds of work. When the run ends, each project's +[tests/](../../tests/README.md), ran its tests — 14 of them then; 9 now, +18 over its two projects — on two Android emulators in 40 seconds, for 75 +seconds of work. When the run ends, each project's session ends, and the end of a session puts back whatever the tests changed — the network, accessibility settings. @@ -295,7 +298,11 @@ soft `tap` is refused before anything runs. `--trace on` keeps a screenshot and the map after every step, and the HTML report shows them as a filmstrip; `--trace retain-on-failure` keeps them only -for a test that failed. `trace` in the config sets the default. +for a test that failed. `trace` in the config sets the default. Each +attempt is also kept as a recording in Vibium's record format, at +`mobium-report/artifacts/trace/---/trace.zip`, +which the report links and [player.vibium.dev](https://player.vibium.dev) +opens. ``` $ mobium test mobiumapp/form.test.json -g checkbox --project android --trace on --reporter list,html @@ -344,6 +351,46 @@ Enter runs it · m maps again · c runs the rest of the test · q quits > used. Quitting reports the test `stopped` and runs nothing after it; closed input — a pipe that ends — runs the rest rather than wait. +## 9. One test over several cases: `each` + +A test with `"each"` runs once per case, as a test of its own. `${key}` in +any string of its steps is that case's value; a string that is nothing but +`${key}` keeps the value's type, so a count stays a number. `$${` writes a +literal `${`. + +```json +{"name": "a bad sign-in is refused, and says why: ${why}", + "each": [ + {"why": "a wrong password", "pass": "wrongpass1", "error": "loginError"}, + {"why": "a short password", "pass": "abc", "error": "passError"} + ], + "steps": [ + {"fill": {"target": "testid=password", "text": "${pass}"}}, + {"tap": "testid=loginBtn"}, + {"wait_for": {"target": "testid=${error}", "condition": "visible"}} + ]} +``` + +With a `${key}` in its name, each case is named by it; without one, the +cases are numbered — `[1]`, `[2]` — rather than named after values that may +be a password. A key a case does not have is refused when the file is read, +before anything runs. + +## 10. A page to run tests from: `--ui` + +```sh +mobium test --ui --open +``` + +serves a page on this machine that lists the suite by file, with each test's +result on each project. ▶ runs one test, a file, or everything; each step +appears with the screen after it as it runs, and **Re-run failed** runs +exactly what failed. The files are read again for every run, so a test you +just saved is the test that runs. A run from the page is the run `mobium +test` makes from the same config and flags, and writes the same reports. The +page answers only its own address with the token it prints, as `mobium +inspect`'s does. + ## Writing tests - **Find targets with `map` first.** `mobium map` on the screen you are @@ -379,15 +426,17 @@ input — a pipe that ends — runs the rest rather than wait. | `mobium test --last-failed` | runs only the tests that failed last time | | `mobium test --project android` | runs on the named projects only | | `mobium test --workers 4` | how many devices run at once — at most one per device | -| `mobium test --retries 2` | runs a failed test again, up to twice; a pass after a failure is flaky | +| `mobium test --retries 2` | runs a failed test again, up to twice; a pass after a failure is flaky; `retries` in the config sets the default | | `mobium test --timeout 30s` | the time for each test; `timeout` in the config is milliseconds | | `mobium test --reporter list,json,junit,html` | which reports to write, comma-separated | +| `mobium test --config ci.config.json` | uses this config, not `mobium.config.json` here or above | +| `mobium test --output out` | where reports go; `mobium-report` beside the config by default, `outputDir` in the config | | `mobium test --list` | lists the tests a run would cover, and runs nothing | | `mobium test --no-screenshots` | keeps no screenshot of a failure, and a trace keeps only the maps | -| `mobium test --trace on` | a screenshot and the map after every step; `retain-on-failure` keeps a failed test's only | +| `mobium test --trace on` | a screenshot and the map after every step, and the test as a recording in Vibium's record format that player.vibium.dev opens; `retain-on-failure` keeps a failed test's only | | `mobium test --debug` | stops before each step: Enter steps, `m` maps again, `c` continues, `q` quits | +| `mobium test --ui` | serves a page to pick, run and watch tests (`--open`, `--ui-port`) | | `mobium show-report` | opens the last HTML report | -Not yet: a visible device, an interactive mode, and recording a test from -what you do — [decisions/0006](../decisions/0006-a-test-runner.md) says what -each will be. +Recording a test from what you do is [`mobium inspect`](inspector.md): what +you do on its page is kept as steps, and downloads as a `*.test.json`. diff --git a/docs/quickstart/README.md b/docs/quickstart/README.md index d993978b..cadb2329 100644 --- a/docs/quickstart/README.md +++ b/docs/quickstart/README.md @@ -35,7 +35,7 @@ what it printed. | --- | --- | --- | | **macOS** | Yes | Yes — needs Xcode | | **Linux** | Yes — every page below run on Ubuntu 24.04, x86_64, against an Android 15 emulator. [SETUP.md](../SETUP.md#on-linux) has the emulator's Linux steps | No — iOS needs Xcode, which runs only on macOS | -| **Windows** | Not yet: the code is written and compiles, and has not run on Windows. See [WINDOWS.md](../WINDOWS.md) | No — no Xcode | +| **Windows** | Not yet: everything that needs no device, the named-pipe daemon transport included, passes in CI on a Windows runner; no emulator or phone has been driven from Windows. See [WINDOWS.md](../WINDOWS.md) | No — no Xcode | ## 1. Install mobium @@ -64,13 +64,15 @@ One device is enough. Start any of these. Install [Android Studio](https://developer.android.com/studio) or the command-line tools, then create a virtual device (Device Manager in Android -Studio, or `avdmanager`) and boot it: +Studio, or `avdmanager`) and boot it by name: ```sh -emulator -avd & -adb wait-for-device +mobium boot ``` +It finds the SDK's `emulator` itself, through `ANDROID_HOME` or the SDK's +usual place, and answers once Android has finished booting — not merely once +`adb` can see it. Add `--window` to watch it; it runs headless otherwise. `adb` comes with the SDK's platform-tools; put it on your `PATH`. On Linux the emulator needs KVM. [SETUP.md](../SETUP.md#android-emulator) has the exact commands and two traps whose errors name the wrong cause. @@ -85,11 +87,11 @@ covers Wi-Fi and what each failure means. ### iOS simulator (macOS) Install Xcode from the App Store, open it once, and add an iOS simulator -runtime (Settings → Components). Then boot a simulator: +runtime (Settings → Components). Then boot a simulator by name: ```sh xcrun simctl list devices available | grep iPhone # pick one -xcrun simctl boot "iPhone 17 Pro" +mobium boot "iPhone 17 Pro" open -a Simulator # to watch it; optional ``` @@ -150,7 +152,9 @@ machine's devices. ## When something goes wrong -- **"2 devices are available … pick one"** — `start` refuses to guess. Name +- **`2 devices are ready (…) — pick one with --device `** on Android, + or **`2 iOS devices are available (…) — pick one with --device`** on iOS — more than one device of the session's platform is running, and `start` + refuses to guess. An Android device beside an iOS one is not ambiguous. Name one: `MOBIUM_DEVICE=` for the examples, `--device` on the command line, or the client's `device` option. `mobium devices` lists them. - **The first start seems stuck** — it installs the UiAutomator2 server on diff --git a/docs/quickstart/build.py b/docs/quickstart/build.py index 310c40a7..88e8d1f5 100644 --- a/docs/quickstart/build.py +++ b/docs/quickstart/build.py @@ -40,7 +40,8 @@ "install_note": "Not on PyPI yet — pip installs it straight from GitHub, which needs " "`git` on your `PATH`. After the first release this becomes " "`pip install mobium`.", - "install": ('python3 -m venv .venv\n' + "install": ('mkdir quickstart && cd quickstart\n' + 'python3 -m venv .venv\n' '.venv/bin/pip install "mobium @ git+https://github.com/mobiumdev/mobium.git#subdirectory=clients/python"'), "run": {"dir": "", "cmd": "MOBIUM_PLATFORM={p} .venv/bin/python quickstart.py"}, "start": "`start(platform=..., app=...)`", @@ -59,7 +60,7 @@ "install_note": "Not on npm yet, and npm cannot install a package from a folder " "inside a repository, so until the first release it installs from " "a clone. After the release this becomes `npm install mobium`.", - "install": CLONE + "\nnpm init -y\nnpm install ~/mobium/clients/javascript", + "install": CLONE + "\nmkdir quickstart && cd quickstart\nnpm init -y\nnpm install ~/mobium/clients/javascript", "run": {"dir": "", "cmd": "MOBIUM_PLATFORM={p} node quickstart.mjs"}, "start": "`await start({ platform, app })`", "quit": "`await device.quit()`", @@ -74,7 +75,7 @@ "slug": "go", "name": "Go", "file": "go/main.go", "lang": "go", "needs": "Go 1.24 or later.", "install_note": "The Go module is published through GitHub, so this works today.", - "install": "go mod init quickstart\ngo get github.com/mobiumdev/mobium/clients/go", + "install": "mkdir quickstart && cd quickstart\ngo mod init quickstart\ngo get github.com/mobiumdev/mobium/clients/go", "run": {"dir": "", "cmd": "MOBIUM_PLATFORM={p} go run ."}, "start": "`mobium.Start(ctx, mobium.WithPlatform(...), mobium.WithApp(...))`", "quit": "`device.Quit(ctx)`", @@ -93,7 +94,8 @@ "install_note": "Not on Maven Central yet. Until the first release it installs " "into your local Maven repository from a clone; after it, the " "dependency below resolves from Central with no clone at all.", - "install": CLONE + "\ncd ~/mobium/clients/java && ./mvnw install -DskipTests && cd -", + "install": CLONE + "\ncd ~/mobium/clients/java && ./mvnw install -DskipTests && cd -\n" + "mkdir quickstart && cd quickstart", "run": { "dir": "", "cmd": "MOBIUM_PLATFORM={p} java -cp ~/.m2/repository/dev/mobium/mobium/0.1.0-SNAPSHOT/mobium-0.1.0-SNAPSHOT.jar Quickstart.java", @@ -130,11 +132,11 @@ "install": (CLONE + "\ndotnet pack ~/mobium/clients/dotnet/Mobium -o ~/mobium-packages\n" "dotnet new console -o quickstart && cd quickstart\n" "dotnet add package Mobium --version 0.1.0 --source ~/mobium-packages"), + "save_note": ", replacing the one `dotnet new` generated", "run": {"dir": "", "cmd": "MOBIUM_PLATFORM={p} dotnet run"}, "start": "`Device.Builder().Platform(...).App(...).Start()`", "quit": "`device.Quit()`, or leaving the `using` block", "notes": [ - "Replace the generated `Program.cs` with the example below.", "A `using` block around `Start()` quits when it ends, even on an " "exception. Around `Connect()` it only closes the connection.", ], @@ -183,6 +185,7 @@ def page(c): "", f"Save this as `{fname}`" + (" in the project folder" if c["install"] else "") + + c.get("save_note", "") + f" — it is [examples/{c['file']}](examples/{c['file']}).", "", f"```{c['lang']}", @@ -226,9 +229,11 @@ def page(c): ] lines += [f"- {n_}" for n_ in c["notes"]] lines += [ - "- With more than one device attached, `start` refuses to guess and lists " - "them. Name one with `MOBIUM_DEVICE=`, which the example " - "passes on as the device.", + "- With more than one device of the session's platform running — two " + "Android devices, or two among the booted simulators and attached iPhones " + "— `start` refuses to guess and lists them. One Android device and one iOS " + "device are not ambiguous: the platform picks. Name one with " + "`MOBIUM_DEVICE=`, which the example passes on as the device.", "", "Next: [the rest of the tool surface](../API.md), and " "[setting up phones and simulators](../SETUP.md). Changing the client " diff --git a/docs/quickstart/cli.md b/docs/quickstart/cli.md index 1c5fe6f2..371fe062 100644 --- a/docs/quickstart/cli.md +++ b/docs/quickstart/cli.md @@ -41,8 +41,8 @@ mobium map "$@" | head -5 # 3. Tap a row by its ref, then wait for the screen it opens. A row's label can # carry its summary too ("Network & internet Mobile, Wi-Fi, ..."), so match -# its start. -REF=$(mobium map "$@" | awk -v row="$ROW" 'index($0, " " row) {print $1; exit}') +# its start: the label begins right after the ref. +REF=$(mobium map "$@" | awk -v row="$ROW" 'index($0, $1 " " row) == 1 {print $1; exit}') mobium tap "$REF" "$@" mobium wait "$NEXT" "$@" @@ -112,6 +112,6 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr - Every command after `session start` uses that session, so none of them needs `--driver`. With more than one device attached, pass the same `--device` to each. - `mobium session status` lists the sessions open. `mobium daemon stop` ends all of them at once. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/quickstart/dotnet.md b/docs/quickstart/dotnet.md index df417da8..38c332a8 100644 --- a/docs/quickstart/dotnet.md +++ b/docs/quickstart/dotnet.md @@ -23,7 +23,7 @@ dotnet add package Mobium --version 0.1.0 --source ~/mobium-packages ## 3. The code -Save this as `Program.cs` in the project folder — it is [examples/dotnet/Program.cs](examples/dotnet/Program.cs). +Save this as `Program.cs` in the project folder, replacing the one `dotnet new` generated — it is [examples/dotnet/Program.cs](examples/dotnet/Program.cs). ```csharp // Mobium quick start: start a session, drive Settings, quit. @@ -132,8 +132,7 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr ## Notes -- Replace the generated `Program.cs` with the example below. - A `using` block around `Start()` quits when it ends, even on an exception. Around `Connect()` it only closes the connection. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/quickstart/examples/cli/quickstart.sh b/docs/quickstart/examples/cli/quickstart.sh index d7f79677..8b2e8096 100755 --- a/docs/quickstart/examples/cli/quickstart.sh +++ b/docs/quickstart/examples/cli/quickstart.sh @@ -24,8 +24,8 @@ mobium map "$@" | head -5 # 3. Tap a row by its ref, then wait for the screen it opens. A row's label can # carry its summary too ("Network & internet Mobile, Wi-Fi, ..."), so match -# its start. -REF=$(mobium map "$@" | awk -v row="$ROW" 'index($0, " " row) {print $1; exit}') +# its start: the label begins right after the ref. +REF=$(mobium map "$@" | awk -v row="$ROW" 'index($0, $1 " " row) == 1 {print $1; exit}') mobium tap "$REF" "$@" mobium wait "$NEXT" "$@" diff --git a/docs/quickstart/examples/go/main.go b/docs/quickstart/examples/go/main.go index e88766bf..623aaa82 100644 --- a/docs/quickstart/examples/go/main.go +++ b/docs/quickstart/examples/go/main.go @@ -61,14 +61,19 @@ func main() { // 3. Tap a row by its ref, then wait for the screen it opens. A row's // label can carry its summary too ("Network & internet Mobile, Wi-Fi, // ..."), so match its start. + ref := "" for _, e := range elements { if strings.HasPrefix(e.Label, p.row) { - if err := device.Tap(ctx, e.Ref); err != nil { - log.Fatal(err) - } + ref = e.Ref break } } + if ref == "" { + log.Fatalf("no row starting with %q", p.row) + } + if err := device.Tap(ctx, ref); err != nil { + log.Fatal(err) + } if _, err := device.WaitFor(ctx, p.next, nil); err != nil { log.Fatal(err) } diff --git a/docs/quickstart/go.md b/docs/quickstart/go.md index 7d1c1947..04f6edfe 100644 --- a/docs/quickstart/go.md +++ b/docs/quickstart/go.md @@ -15,6 +15,7 @@ Go 1.24 or later. The Go module is published through GitHub, so this works today. ```sh +mkdir quickstart && cd quickstart go mod init quickstart go get github.com/mobiumdev/mobium/clients/go ``` @@ -87,14 +88,19 @@ func main() { // 3. Tap a row by its ref, then wait for the screen it opens. A row's // label can carry its summary too ("Network & internet Mobile, Wi-Fi, // ..."), so match its start. + ref := "" for _, e := range elements { if strings.HasPrefix(e.Label, p.row) { - if err := device.Tap(ctx, e.Ref); err != nil { - log.Fatal(err) - } + ref = e.Ref break } } + if ref == "" { + log.Fatalf("no row starting with %q", p.row) + } + if err := device.Tap(ctx, ref); err != nil { + log.Fatal(err) + } if _, err := device.WaitFor(ctx, p.next, nil); err != nil { log.Fatal(err) } @@ -170,6 +176,6 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr - `defer device.Quit(ctx)` right after a successful `Start` is the usual shape. A second `Quit` does nothing, so an explicit one before it is fine. - The example in the repository builds against the client beside it, through a `replace` line in its `go.mod`; your own module uses `go get`. - `mobium.Connect` still exists: it opens a connection without touching the device, and `Close` leaves the session open. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/quickstart/java.md b/docs/quickstart/java.md index 5f9259fa..37d816f6 100644 --- a/docs/quickstart/java.md +++ b/docs/quickstart/java.md @@ -17,6 +17,7 @@ Not on Maven Central yet. Until the first release it installs into your local Ma ```sh git clone https://github.com/mobiumdev/mobium.git ~/mobium cd ~/mobium/clients/java && ./mvnw install -DskipTests && cd - +mkdir quickstart && cd quickstart ``` ## 3. The code @@ -169,6 +170,6 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr with `Quickstart.java` in `src/main/java/`, then `gradle run`. - try-with-resources around `start()` quits when it ends. Around `connect()` it only closes the connection, leaving the session open. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/quickstart/javascript.md b/docs/quickstart/javascript.md index fdd0304d..5e949b03 100644 --- a/docs/quickstart/javascript.md +++ b/docs/quickstart/javascript.md @@ -16,6 +16,7 @@ Not on npm yet, and npm cannot install a package from a folder inside a reposito ```sh git clone https://github.com/mobiumdev/mobium.git ~/mobium +mkdir quickstart && cd quickstart npm init -y npm install ~/mobium/clients/javascript ``` @@ -127,6 +128,6 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr - Put `quit()` in a `finally`, as the example does, so an error part-way still ends the session. A second `quit()` does nothing. - `connect()` still exists: it opens a connection without touching the device, and its `close()` leaves the session open. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/quickstart/python.md b/docs/quickstart/python.md index 146527c0..5867ecbb 100644 --- a/docs/quickstart/python.md +++ b/docs/quickstart/python.md @@ -15,6 +15,7 @@ Python 3.9 or later. Not on PyPI yet — pip installs it straight from GitHub, which needs `git` on your `PATH`. After the first release this becomes `pip install mobium`. ```sh +mkdir quickstart && cd quickstart python3 -m venv .venv .venv/bin/pip install "mobium @ git+https://github.com/mobiumdev/mobium.git#subdirectory=clients/python" ``` @@ -127,6 +128,6 @@ The first start on a device is slow: it installs the UiAutomator2 server on Andr - `with start(...) as device:` quits when the block ends, even on an exception. Without `with`, call `device.quit()` yourself — in a `finally`. - `connect()` still exists: it opens a connection without touching the device, and its `close()` leaves the session open for whoever started it. -- With more than one device attached, `start` refuses to guess and lists them. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. +- With more than one device of the session's platform running — two Android devices, or two among the booted simulators and attached iPhones — `start` refuses to guess and lists them. One Android device and one iOS device are not ambiguous: the platform picks. Name one with `MOBIUM_DEVICE=`, which the example passes on as the device. Next: [the rest of the tool surface](../API.md), and [setting up phones and simulators](../SETUP.md). Changing the client itself? [DEVELOPMENT.md](../DEVELOPMENT.md) is the contributor's guide. diff --git a/docs/decisions/0003-drivers-are-processes-not-plugins.md b/examples/drivers/PROTOCOL.md similarity index 58% rename from docs/decisions/0003-drivers-are-processes-not-plugins.md rename to examples/drivers/PROTOCOL.md index 43b40597..036d85d8 100644 --- a/docs/decisions/0003-drivers-are-processes-not-plugins.md +++ b/examples/drivers/PROTOCOL.md @@ -1,59 +1,11 @@ -# 0003 — A third-party driver is a process, not a Go plugin - -**2026-09-13.** Answers a structural criticism of the design: the extension -point is private. `mobiumdriver.Driver` lives under `internal/`, so nobody outside -this repository can add a Roku, Flutter, tvOS or Tizen backend without forking. -An ecosystem of drivers exists only where strangers can add one. - -## The three ways to open it - -**Export the Go interface.** Move `internal/mobiumdriver` to `mobiumdriver/`, and a -third party imports it. Cheapest to do and the worst of the three. It makes -every type reachable from `Driver` public API — `uitree.Node`, `uitree.Tree`, -`device.InstalledApp`, `Rect` — and each of them is still moving. The eleven -capability interfaces would freeze on the day they were exported, which is -about ten interfaces too early. Worse, a driver written this way is a Go -library: the author has to import Mobium and build their own binary, so -"install a driver" means "build a different Mobium". That is a fork with extra -steps. - -**`plugin.Open`.** Rejected outright. Go plugins require the exact same -compiler version, the exact same versions of every shared dependency, and the -exact same build flags; they do not work on Windows at all, and they are -effectively unsupported on macOS with CGO disabled — which is how Mobium is -built. A distribution mechanism that fails on two of three platforms is not -one. - -**A subprocess speaking a protocol.** Chosen. The driver is an executable that -reads JSON-RPC 2.0 on stdin and writes it on stdout, exactly the framing -`mobium pipe` already speaks to its clients. Mobium spawns it, negotiates, and -talks to it for the life of the session. - -## Why the subprocess wins - -- **The single-binary property survives.** Mobium still ships as one static - executable with no runtime dependencies. A driver is a separate file the user - chooses to install; nothing is required to run Mobium without one. -- **No language lock-in.** A plugin must be written in its host's language. - A Mobium driver can be a shell script, a Python file, a Go binary or a - Rust one. For a device whose SDK is Python — which is most of the odd ones — - that is the difference between possible and not. -- **Nothing internal is frozen.** The contract is the wire format, which is - small and deliberately conservative, rather than every Go type reachable from - an interface. `uitree.Node` can keep changing; `WireNode` cannot, and it is - the only thing a driver author sees. -- **A crashing driver is not a crashing Mobium.** In-process plugins share the - address space. A third-party driver segfaulting takes the daemon with it, and - the daemon holds the device session. -- **It matches how this already works.** Mobium speaks JSON-RPC over a pipe to - five clients today. Speaking it downward as well as upward adds a direction, - not a mechanism. - -The cost is a process boundary per call. Measured against the dump backend's -1.96s snapshot, and even against UiAutomator2's 0.04s, the pipe is not where -the time goes: the round trip through a spawned process is tens of -microseconds, and every driver worth writing is talking to a device over USB or -TCP behind it. +# The driver protocol + +What a third-party driver speaks: a process Mobium starts, talking JSON-RPC +over its stdin and stdout, which gives Mobium a platform it does not know — +a Roku, a Flutter engine, a TV. Everything above the driver — refs, waiting, +scrolling, `map` — comes from Mobium unchanged. [README.md](README.md) is the +guide to writing one; this is the protocol it follows, and +[`mobium-driver-adb`](mobium-driver-adb) is a working one in Python. ## The protocol @@ -165,7 +117,7 @@ would notice. printed by `map` — it is labeled by its `testid`, given `role=password`, and read back masked with only its length kept. Android puts a password field's typed value straight into `text`, and mobium leaked one until it honored this -([CHALLENGES 43](../CHALLENGES.md)). Nothing downstream can infer it. +([CHALLENGES 43](../../docs/CHALLENGES.md)). Nothing downstream can infer it. `displayed` and `enabled` are the two fields that default to **true** when absent, so a driver that does not track them does not accidentally report an @@ -179,11 +131,11 @@ use to say "this exists but the user cannot see it": a node marked `displayed:false` is kept in the hierarchy, so a locator still resolves to it and the error explains why it cannot be touched, but it is left out of `map`. -That last part was not true when this was first written — `Actionable` did not -consult the field at all, so a driver could say `false` and be mapped anyway. +That last part was not always true — `Actionable` once did not consult the +field at all, so a driver could say `false` and be mapped anyway. Fixed, with a test that fails if it regresses; recorded as -[CHALLENGES 39](../CHALLENGES.md). A protocol that documents a field which -changes nothing is worse than one that omits it. +[CHALLENGES 39](../../docs/CHALLENGES.md). A protocol that documents a field +which changes nothing is worse than one that omits it. ### Errors @@ -206,40 +158,3 @@ develop a driver without installing it. `mobium doctor` lists the drivers it can find, so "is it installed" has an answer that is not "run it and see". - -## What this does not do - -It does not make Mobium's *tool* layer extensible — there is no way to add an -`app_foo` tool from outside, and there should not be one. The value of a single -shared tool layer is that every client gets every tool with the same semantics; -a third-party tool would break that for whoever did not install it. The -platform seam is the right place to open, and the only one. - -## How to re-check this - -The claims here are about a protocol somebody outside this repository has to -implement, so they are worth re-running rather than trusting: - -```sh -# The reference driver still needs nothing from this module. Every line should -# name a Python standard-library module and nothing else — anything from -# mobium here means the extension point is not as open as this document says. -grep -n "^import\|^from" examples/drivers/mobium-driver-adb - -# Every capability has an As* helper, and nothing above internal/mobiumdriver -# asserts a capability interface directly. -grep -rn 's\.driver\.(mobiumdriver\.' internal/ - -# The whole thing, against a device. -docs/checks/external-driver.sh -``` - -**Re-run 2026-09-14**: the two that need no device both hold — the reference -driver imports `base64`, `json`, `os`, `subprocess`, `sys` and `xml` and -nothing more, and no capability interface is asserted directly anywhere above -`internal/mobiumdriver`. - -It also does not promise version-1 stability yet. `protocolVersion` is `"1"` -and Mobium refuses anything else, so the negotiation is in place from the -start; whether `"1"` is frozen is a decision for the public release, not for -today. diff --git a/examples/drivers/README.md b/examples/drivers/README.md index d2d80380..ae096e71 100644 --- a/examples/drivers/README.md +++ b/examples/drivers/README.md @@ -6,12 +6,11 @@ desktop app, a set-top box, a car head unit, a platform that does not exist yet You do not need to fork Mobium, build Mobium, or write Go. A driver is an executable in any language that reads JSON-RPC on stdin and writes it on -stdout. The protocol is specified in -[../../docs/decisions/0003-drivers-are-processes-not-plugins.md](../../docs/decisions/0003-drivers-are-processes-not-plugins.md); -this page is the short version. +stdout. The protocol is specified in [PROTOCOL.md](PROTOCOL.md); this page is the +short version. Everything above the driver — locators, `@ref`s, waiting, scrolling, the CLI, -MCP and all four client libraries — you get for free. The driver's whole job is +MCP and all five client libraries — you get for free. The driver's whole job is to answer *what is on screen* and *touch this point*. ## The smallest driver that works diff --git a/examples/drivers/mobium-driver-adb b/examples/drivers/mobium-driver-adb index 3ef3be87..65699477 100755 --- a/examples/drivers/mobium-driver-adb +++ b/examples/drivers/mobium-driver-adb @@ -7,8 +7,8 @@ compared against a known-good implementation on the same screen. That comparison is the point: it is how you tell a driver that works from a driver that returns something plausible. -It exists to prove three claims in -docs/decisions/0003-drivers-are-processes-not-plugins.md: +It exists to prove three claims about the protocol in +examples/drivers/PROTOCOL.md: 1. A driver needs nothing from the Mobium codebase. This file imports the Python standard library and nothing else, and was written from the diff --git a/internal/agent/alert.go b/internal/agent/alert.go index 16584592..3243389d 100644 --- a/internal/agent/alert.go +++ b/internal/agent/alert.go @@ -100,8 +100,7 @@ func (h *Handlers) alertOn(ctx context.Context, s *session, args map[string]inte // Native has no prompt there either, `Alert.prompt` being // iOS-only. On iOS the endpoint exists and a plain alert simply // has no field to type into. - // The server's W3C code, kept since docs/decisions/0005, rather - // than its wording. + // The server's W3C code, kept in details, rather than its wording. if e, ok := mobiumerr.As(err); ok && e.Details["w3c"] == "unknown command" { return nil, mobiumerr.New(mobiumerr.Unsupported, "this backend cannot type into a dialog: the "+ "device-side server does not implement it. Android has no prompt "+ diff --git a/internal/agent/alert_test.go b/internal/agent/alert_test.go index 7f4a80d1..4c0487a0 100644 --- a/internal/agent/alert_test.go +++ b/internal/agent/alert_test.go @@ -183,8 +183,8 @@ func TestAlertTypesWithoutAnswering(t *testing.T) { // React Native has no prompt there either, while on iOS a plain alert simply // has no field. func TestAlertSaysWhyItCannotType(t *testing.T) { - // Shaped as the W3C client returns it since docs/decisions/0005: the - // server's own code kept in Details, not only in the wording. + // Shaped as the W3C client returns it: the server's own code kept in + // Details, not only in the wording. unsupported := &alertDriver{text: "Delete?", typeErr: mobiumerr.New(mobiumerr.Unsupported, "unknown command: no such route").WithDetail("w3c", "unknown command")} _, err := alertOn(t, unsupported, map[string]interface{}{"text": "x"}) diff --git a/internal/agent/backend.go b/internal/agent/backend.go index 25140b5b..99662d3c 100644 --- a/internal/agent/backend.go +++ b/internal/agent/backend.go @@ -36,8 +36,7 @@ const DefaultBackend = BackendUIA2 // builtinBackends is every backend compiled into this binary. Any other name // is looked for on PATH as `mobium-driver-`, which is how a third party -// adds a platform without forking — see -// docs/decisions/0003-drivers-are-processes-not-plugins.md. +// adds a platform without forking — see examples/drivers/PROTOCOL.md. var builtinBackends = []Backend{BackendUIA2, BackendDump, BackendWDA} // ParseBackend validates a backend name. diff --git a/internal/agent/backend_test.go b/internal/agent/backend_test.go index 9780bce2..efcbdaef 100644 --- a/internal/agent/backend_test.go +++ b/internal/agent/backend_test.go @@ -25,7 +25,7 @@ func TestParseBackend(t *testing.T) { // An unrecognized name is not an error here any more: it is the name // of a possible third-party driver, and whether one exists is decided // when a session opens, where PATH can be named in the failure. See - // docs/decisions/0003. + // examples/drivers/PROTOCOL.md. {"tizen", "tizen", false}, {"roku", "roku", false}, // A near miss on a built-in name is still an error, because it is a @@ -217,7 +217,7 @@ func TestMissingDriverExplainsItself(t *testing.T) { if err == nil { t.Fatal("expected an error for a driver that is not installed") } - for _, want := range []string{"mobium-driver-roku", "MOBIUM_DRIVER_ROKU", "0003"} { + for _, want := range []string{"mobium-driver-roku", "MOBIUM_DRIVER_ROKU", "PROTOCOL.md"} { if !strings.Contains(err.Error(), want) { t.Errorf("the message does not mention %q:\n%v", want, err) } diff --git a/internal/agent/cover.go b/internal/agent/cover.go index 075a0e0a..458b8b25 100644 --- a/internal/agent/cover.go +++ b/internal/agent/cover.go @@ -87,7 +87,7 @@ func (h *Handlers) resolveAim(ctx context.Context, s *session, target string) (* // loadedHitRefusal asks the hit probe loaded into the app at launch, when // there is one, whether a touch at aim reaches the target, and is the -// refusal when it would not (docs/decisions/0008). With no probe loaded the +// refusal when it would not. With no probe loaded the // question is not asked: through lldb it takes seconds, which is app_hit_test's // to spend. An answer of unknown — no view in the app is the element, which // happens to one named by label — leaves the action to the checks the tree diff --git a/internal/agent/enabled_test.go b/internal/agent/enabled_test.go index 3cf1ef21..1b0d263b 100644 --- a/internal/agent/enabled_test.go +++ b/internal/agent/enabled_test.go @@ -227,3 +227,32 @@ func TestPageBehindOnAndroid(t *testing.T) { t.Error("a page of no known app was called behind") } } + +// Chrome's installed web app keeps its page in front and stops reporting +// its WebView a few seconds after launch. The refusal must not send the +// caller to NATIVE_APP as if the app had moved on: there, Chrome's tree is +// empty too. CHALLENGES 200. +func TestNoWebViewInFrontIsNotCalledNavigatedAway(t *testing.T) { + xml := `` + + `` + + `` + tree, err := uitree.ParseAndroid([]byte(xml)) + if err != nil { + t.Fatal(err) + } + h := NewHandlers() + s := &session{dev: fakeDevice(), driver: &fakeDriver{screens: []*uitree.Tree{tree}}, backend: BackendUIA2, + webApp: "com.android.chrome", webCtx: "WEBVIEW_com.android.chrome_3"} + _, err = h.webFrame(context.Background(), s) + e, ok := mobiumerr.As(err) + if !ok || e.Code != mobiumerr.NoSuchContext { + t.Fatalf("no WebView with the page's app in front: %v, want no_such_context", err) + } + if !strings.Contains(e.Message, "com.android.chrome is in front") || !strings.Contains(e.Message, "app_text") { + t.Errorf("the refusal does not say the app is in front and reading still works: %s", e.Message) + } + if !strings.Contains(e.Remedy, "if the app moved on") { + t.Errorf("the remedy is not conditional on the app having moved on: %q", e.Remedy) + } +} diff --git a/internal/agent/errors_test.go b/internal/agent/errors_test.go index f40f2ee2..b5e57fb3 100644 --- a/internal/agent/errors_test.go +++ b/internal/agent/errors_test.go @@ -14,7 +14,7 @@ import ( ) // A failed tool call carries its code on the wire, beside the unchanged text: -// the structured half is what the clients and --json read. docs/decisions/0005. +// the structured half is what the clients and --json read. func TestAFailedCallCarriesItsCode(t *testing.T) { h := NewHandlers() params, _ := json.Marshal(ToolsCallParams{Name: "app_no_such_tool"}) diff --git a/internal/agent/handlers.go b/internal/agent/handlers.go index 0f3ac58a..968c6950 100644 --- a/internal/agent/handlers.go +++ b/internal/agent/handlers.go @@ -937,7 +937,7 @@ func knownRefs(t *refTable) string { // one that matched too much. The two want opposite responses: nothing found // may be below the fold and worth scrolling for, while an ambiguous match is // already on screen twice and scrolling can only make it worse. It was a type -// of its own here until docs/decisions/0005; it is the code now, so a client +// of its own here before Mobium had error codes; it is the code now, so a client // can tell the two apart as well. func matchedNothing(err error) bool { if isNearMiss(err) { diff --git a/internal/agent/hittest.go b/internal/agent/hittest.go index fdb2b031..9b459ce2 100644 --- a/internal/agent/hittest.go +++ b/internal/agent/hittest.go @@ -27,8 +27,7 @@ type HitTestView struct { // hitTest asks the platform's own hit test, below accessibility, whether a // touch at the point app_tap would use reaches the target. Opt-in, because -// on a simulator it attaches a debugger to the app for about two seconds -// (docs/decisions/0008). +// on a simulator it attaches a debugger to the app for about two seconds. func (h *Handlers) hitTest(ctx context.Context, args map[string]interface{}) (*ToolsCallResult, error) { target := stringArg(args, "target") if target == "" { diff --git a/internal/agent/server.go b/internal/agent/server.go index 93d3b8b0..86c0b969 100644 --- a/internal/agent/server.go +++ b/internal/agent/server.go @@ -174,7 +174,7 @@ func CallTool(params json.RawMessage, h *Handlers) (interface{}, *Error) { // door reports it. The text is unchanged, so an agent reading Content sees // what it always has; the structured half carries the code, the remedy and // whether a retry can help, for callers that act on the failure rather than -// read it. docs/decisions/0005. +// read it. The codes are in docs/guides/cli.md. // // Shared because a second copy of it went wrong: `mobium pipe`, which every // client spawns, built its own result without the structured half, so every diff --git a/internal/agent/webview.go b/internal/agent/webview.go index c7518a34..5c5bc30c 100644 --- a/internal/agent/webview.go +++ b/internal/agent/webview.go @@ -210,8 +210,20 @@ func (h *Handlers) webFrame(ctx context.Context, s *session) (*webview.Frame, er } } if host == nil { - return nil, mobiumerr.New(mobiumerr.NoSuchContext, "no WebView is visible in the native hierarchy — "+ - "the app may have navigated away; switch back with `app_context NATIVE_APP`") + // Two causes look the same from here, and only the first has a + // remedy: the app moved to a screen with no WebView, or it still + // shows the page and has stopped putting the WebView in the + // accessibility tree — Chrome's installed web apps do when opened + // while Chrome is already running, within about five seconds, and + // then nothing native can say where the page is. + // CHALLENGES 200. + return nil, mobiumerr.New(mobiumerr.NoSuchContext, "%s is in front but its accessibility tree has no "+ + "WebView, so where %s sits on screen cannot be measured and a tap is refused. Either the app "+ + "moved to a screen without one, or it still shows the page and stopped reporting its WebView — "+ + "Chrome's installed web apps do this when opened while Chrome is running. Reading the page still works: "+ + "app_text, app_map and app_eval", tree.Package(), s.webCtx). + WithRemedy("if the app moved on, `app_context NATIVE_APP`; if it still shows the page, no tap " + + "inside it can be placed until the app reports its WebView again") } metrics, err := s.web.LayoutMetrics(ctx) @@ -484,7 +496,7 @@ func (h *Handlers) inspectorFor(ctx context.Context, s *session) (*webview.Inspe if err != nil { return nil, err } - // A simulator's inspector is a Unix socket on this Mac (decisions/0002); + // A simulator's inspector is a Unix socket on this Mac; // a phone's is a service on the device, reached through usbmuxd and // lockdown. The protocol after that is the same. var insp *webview.Inspector diff --git a/internal/apisurface/errors.go b/internal/apisurface/errors.go index 5f03d040..0d7c4b2d 100644 --- a/internal/apisurface/errors.go +++ b/internal/apisurface/errors.go @@ -15,7 +15,7 @@ import ( "github.com/mobiumdev/mobium/internal/mobiumerr" ) -// Error codes are public API exactly as tools are (docs/decisions/0005), so +// Error codes are public API exactly as tools are (docs/guides/cli.md), so // they get the same guard: every code the server can send has an exception in // every client, under the name mobiumerr.Name gives it, mapped to the right // wire string — and no client carries a code the server no longer sends. @@ -132,8 +132,8 @@ func errorProblems(sources map[string]string) map[string][]string { // with no code: a fmt.Errorf or errors.New that does not wrap another error // with %w. A wrapping one is fine — mobiumerr.CodeOf sees through it to its // cause's code — but a new error made without mobiumerr.New reaches a client -// as the bare code "error", which is exactly what docs/decisions/0005 exists -// to end. mobiumerr itself and this package are exempt; tests are not scanned. +// as the bare code "error", which is exactly what error codes exist to +// end. mobiumerr itself and this package are exempt; tests are not scanned. func UnclassifiedErrors(root string) ([]string, error) { var out []string fset := token.NewFileSet() diff --git a/internal/apisurface/surface_test.go b/internal/apisurface/surface_test.go index 39ae2a56..7628c94b 100644 --- a/internal/apisurface/surface_test.go +++ b/internal/apisurface/surface_test.go @@ -181,7 +181,7 @@ func TestTheFlagSweepCanFail(t *testing.T) { } // Every error code has its exception in every client, with the right wire -// string. docs/decisions/0005. +// string. func TestEveryClientCoversTheErrorCodes(t *testing.T) { problems, err := ErrorCoverage(filepath.Join("..", "..")) if err != nil { @@ -235,7 +235,7 @@ func TestNoErrorLeavesWithoutACode(t *testing.T) { } for _, at := range found { t.Errorf("%s creates an error with no code — use mobiumerr.New(mobiumerr., ...), "+ - "or wrap a classified error with %%w; see docs/decisions/0005", at) + "or wrap a classified error with %%w; the codes are in docs/guides/cli.md", at) } } diff --git a/internal/device/hitprobe.go b/internal/device/hitprobe.go index e3ec23f8..88e17a7f 100644 --- a/internal/device/hitprobe.go +++ b/internal/device/hitprobe.go @@ -22,8 +22,8 @@ import ( ) // The hit test below accessibility, on an iOS simulator. See -// hitprobe/probe.m for what it asks and why accessibility cannot answer it, -// and docs/decisions/0008 for why it is opt-in: it attaches a debugger to +// hitprobe/probe.m for what it asks and why accessibility cannot answer it. +// It is opt-in: it attaches a debugger to // the app, which stops it for about two seconds — unless the probe was // loaded when the app launched, when it answers on a Unix socket in well // under a millisecond. diff --git a/internal/device/lockdown.go b/internal/device/lockdown.go index 237f1635..80c99d5f 100644 --- a/internal/device/lockdown.go +++ b/internal/device/lockdown.go @@ -14,8 +14,8 @@ import ( "github.com/mobiumdev/mobium/internal/plist" ) -// A real iPhone's Web Inspector is reached the way ios_webkit_debug_proxy -// reaches it, and every step was measured on an iPhone +// A real iPhone's Web Inspector is reached through usbmuxd and lockdown, and +// every step was measured on an iPhone // 15 Plus on iOS 26.6.2 before any of this was written: // // 1. usbmuxd, a Unix socket on every Mac, lists the phone and hands over the diff --git a/internal/inspect/inspect.go b/internal/inspect/inspect.go index 74d80160..a9d22ac2 100644 --- a/internal/inspect/inspect.go +++ b/internal/inspect/inspect.go @@ -1,7 +1,7 @@ // Package inspect is `mobium inspect`: a page on this machine showing a // device's screen with every element map finds drawn over it, the locator // for the one clicked, actions on it, and what was done kept as a test file. -// See docs/decisions/0007. +// See docs/guides/inspector.md. // // It is a client of the tools, as the test runner is: every answer on the // page is a tool's answer, and every action is the tool a command would call, diff --git a/internal/mobiumdriver/capability.go b/internal/mobiumdriver/capability.go index 0b8277ff..21243594 100644 --- a/internal/mobiumdriver/capability.go +++ b/internal/mobiumdriver/capability.go @@ -18,7 +18,7 @@ package mobiumdriver // which on a real iPhone declines what only simctl could do on a simulator. type Capable interface { // HasCapability reports whether this driver supports a named capability. - // The names are the ones in docs/decisions/0003. + // The names are the ones in examples/drivers/PROTOCOL.md. HasCapability(name string) bool } diff --git a/internal/mobiumdriver/external.go b/internal/mobiumdriver/external.go index ee2c9d93..81f65785 100644 --- a/internal/mobiumdriver/external.go +++ b/internal/mobiumdriver/external.go @@ -20,8 +20,7 @@ import ( ) // External drives a device through a driver program that Mobium did not -// compile in — the extension point described in -// docs/decisions/0003-drivers-are-processes-not-plugins.md. +// compile in — the extension point described in examples/drivers/PROTOCOL.md. // // The program reads JSON-RPC 2.0 on stdin and writes it on stdout, one object // per line. Its stderr is its log, not part of the protocol, which is what @@ -116,7 +115,7 @@ func FindDriver(name string) (string, error) { if err != nil { return "", mobiumerr.New(mobiumerr.InvalidArgument, "there is no built-in backend named %q, and no %s on your PATH.\n"+ "A third-party backend is an executable of that name — see "+ - "docs/decisions/0003-drivers-are-processes-not-plugins.md.\n"+ + "examples/drivers/PROTOCOL.md in the mobium repository.\n"+ "To point at one without installing it, set %s=/path/to/driver and then run "+ "`mobium daemon stop`: the daemon is what looks for drivers, and a running one keeps "+ "the environment it started with.", name, exe, env) diff --git a/internal/mobiumdriver/external_test.go b/internal/mobiumdriver/external_test.go index 4a3a0985..1ab8df82 100644 --- a/internal/mobiumdriver/external_test.go +++ b/internal/mobiumdriver/external_test.go @@ -16,7 +16,7 @@ import ( // The driver protocol is the one part of Mobium meant to be implemented by // somebody who cannot read this code, so these tests drive it from the far // side: testdata/fakedriver imports nothing from this module and knows only -// what docs/decisions/0003 says. If it ever needs something from here, the +// what examples/drivers/PROTOCOL.md says. If it ever needs something from here, the // extension point is not open after all. var driverBinary string diff --git a/internal/mobiumdriver/testdata/fakedriver/main.go b/internal/mobiumdriver/testdata/fakedriver/main.go index 759b92a3..b6e69afb 100644 --- a/internal/mobiumdriver/testdata/fakedriver/main.go +++ b/internal/mobiumdriver/testdata/fakedriver/main.go @@ -1,7 +1,7 @@ // A driver used to test the external driver protocol from the other side. // // It is deliberately written the way a third party would write one: it imports -// nothing from mobium, it knows only what docs/decisions/0003 says, and it +// nothing from mobium, it knows only what examples/drivers/PROTOCOL.md says, and it // speaks the protocol by hand. If this file needs anything from the mobium // module to work, the extension point is not actually open. // @@ -152,7 +152,7 @@ func orDefault(s, d string) string { return s } -// screen is a small hierarchy in the shape docs/decisions/0003 specifies: +// screen is a small hierarchy in the shape examples/drivers/PROTOCOL.md specifies: // what the platform said, and nothing derived. func screen() map[string]interface{} { return map[string]interface{}{ diff --git a/internal/mobiumdriver/w3c.go b/internal/mobiumdriver/w3c.go index b7b96390..eff2625d 100644 --- a/internal/mobiumdriver/w3c.go +++ b/internal/mobiumdriver/w3c.go @@ -671,9 +671,9 @@ func (c *w3cClient) doOnce(ctx context.Context, method, path string, body, out i // errorFrom extracts a W3C error payload, returning "" for a success. // errorFrom reads the error a W3C server put in a response body, keeping the // server's own code: WebDriverAgent and UiAutomator2 both answer in W3C -// WebDriver's vocabulary ("no such element", "invalid session id"), and until -// docs/decisions/0005 that code was flattened into the message and later -// recovered by string matching. It is now kept, in Details["w3c"], and mapped +// WebDriver's vocabulary ("no such element", "invalid session id"). That code +// was once flattened into the message and recovered by string matching; it is +// now kept, in Details["w3c"], and mapped // onto Mobium's codes where one names the same thing. nil means no error. func errorFrom(raw json.RawMessage) *mobiumerr.Error { w3c, msg := w3cError(raw) diff --git a/internal/mobiumdriver/wda_test.go b/internal/mobiumdriver/wda_test.go index 5a7e7b21..d1d4e26d 100644 --- a/internal/mobiumdriver/wda_test.go +++ b/internal/mobiumdriver/wda_test.go @@ -326,7 +326,7 @@ func TestWDARefusesWhenTheTextNeverLands(t *testing.T) { t.Errorf("the error does not say what it found (%q): %v", want, err) } } // Reported as the failure this project exists to catch: it said it - // typed, and reading back said otherwise. docs/decisions/0005. + // typed, and reading back said otherwise. if mobiumerr.CodeOf(err) != mobiumerr.NotConfirmed { t.Errorf("code = %s, want not_confirmed", mobiumerr.CodeOf(err)) } diff --git a/internal/mobiumerr/mobiumerr.go b/internal/mobiumerr/mobiumerr.go index 16e0b0f1..73d6d4b9 100644 --- a/internal/mobiumerr/mobiumerr.go +++ b/internal/mobiumerr/mobiumerr.go @@ -6,7 +6,7 @@ // 1 for everything. The sentences were good — the project requires each to // name a cause and a remedy that works — but nothing could act on them without // parsing prose, and the code itself decided "stale session" and "no alert" by -// matching message text. See docs/decisions/0005-errors.md. +// matching message text. The codes are listed in docs/guides/cli.md. // // The codes are modeled on W3C WebDriver's — lowercase, underscore-separated, // stable — because that is the vocabulary WebDriver users already diff --git a/internal/plist/plist.go b/internal/plist/plist.go index bf0f3419..a9311424 100644 --- a/internal/plist/plist.go +++ b/internal/plist/plist.go @@ -5,7 +5,7 @@ // big-endian length followed by a binary plist; the JSON that Mobium actually // cares about travels inside that envelope. The standard library has no // plist, and this is the only reason iOS WebView support is not simply a -// matter of reusing the Android code — see docs/decisions/0002. +// matter of reusing the Android code. // // It is deliberately not a general-purpose plist library. Dictionaries, // strings, integers, booleans, arrays and data are what the protocol uses, diff --git a/internal/plist/webinspector_test.go b/internal/plist/webinspector_test.go index de851d45..b0d59b05 100644 --- a/internal/plist/webinspector_test.go +++ b/internal/plist/webinspector_test.go @@ -125,7 +125,7 @@ func TestDrivesWebinspectord(t *testing.T) { case "Target.targetCreated": // WebKit is multi-target: an unwrapped Runtime.evaluate // answers "'Runtime' domain was not found", which reads like - // something quite different. See docs/decisions/0002. + // something quite different. params, _ := m["params"].(map[string]any) info, _ := params["targetInfo"].(map[string]any) inner, _ := json.Marshal(map[string]any{"id": 10, "method": "Runtime.evaluate", diff --git a/internal/testrun/file.go b/internal/testrun/file.go index 6b4fdc27..25076b0d 100644 --- a/internal/testrun/file.go +++ b/internal/testrun/file.go @@ -2,7 +2,7 @@ // steps and whose assertions are app_wait_for and expect, run against the // devices a config names, with retries, workers and reports. It is a client // of the tools, as the language clients are, and calls them through whatever -// the caller hands it. See docs/decisions/0006. +// the caller hands it. See docs/guides/test-runner.md. package testrun import ( diff --git a/internal/testui/testui.go b/internal/testui/testui.go index 61ab2e24..9f54a170 100644 --- a/internal/testui/testui.go +++ b/internal/testui/testui.go @@ -1,6 +1,6 @@ // Package testui is `mobium test --ui`: a page on this machine that lists a // suite, runs any part of it on its projects, and shows each step with the -// screen after it as it happens. See docs/decisions/0009. +// screen after it as it happens. See docs/guides/test-runner.md, "--ui". // // It runs nothing of its own: a run is the one `mobium test` makes from the // same flags and config, with the tests to run picked by ID. diff --git a/internal/uitree/cover.go b/internal/uitree/cover.go index 6941fc16..b4af2371 100644 --- a/internal/uitree/cover.go +++ b/internal/uitree/cover.go @@ -134,8 +134,8 @@ const aimCols, aimRows = 9, 5 // AimAt decides where to touch target so a tap reaches it. The center, when // no control is drawn over it; otherwise the clear point nearest the center, -// as EarlGrey does for a partly covered element — a button pressed off -// center is the same button. When a control covers every point, Blocker says +// for a partly covered element — a button pressed off center is the same +// button. When a control covers every point, Blocker says // which. func (t *Tree) AimAt(target *Node) Aim { cx, cy := target.Bounds.Center() diff --git a/internal/webview/rwi.go b/internal/webview/rwi.go index 91f30d18..b691f776 100644 --- a/internal/webview/rwi.go +++ b/internal/webview/rwi.go @@ -14,9 +14,9 @@ import ( // Remote Web Inspector: how WebKit exposes a page for debugging, and the only // way into a WKWebView. It is a third protocol after WebDriver BiDi and CDP, -// which is why iOS WebViews sat unstarted for so long — see -// docs/decisions/0002-ios-webviews-are-reachable.md for what turned out to be -// wrong about that. +// which is why iOS WebViews sat unstarted for so long: it was believed to be +// a Mach-port wall, and turned out to be a plain Unix socket carrying binary +// property lists. // // Three things make it unlike CDP, and each one is a trap: // diff --git a/internal/webview/rwi_session.go b/internal/webview/rwi_session.go index 24594f9c..64ea54be 100644 --- a/internal/webview/rwi_session.go +++ b/internal/webview/rwi_session.go @@ -225,43 +225,85 @@ func (s *IOSSession) call(ctx context.Context, method string, params map[string] } } +// evalGroup is the object group Evaluate's handles are kept in, released +// once each call has its value. +const evalGroup = "mobium-eval" + +// iosEvalResult is a Runtime.evaluate, awaitPromise or callFunctionOn reply. +type iosEvalResult struct { + Result struct { + Type string `json:"type"` + Value json.RawMessage `json:"value"` + Description string `json:"description"` + ObjectID string `json:"objectId"` + ClassName string `json:"className"` + } `json:"result"` + // WebKit reports a thrown exception inside a successful reply, the + // same trap CDP has: no error field is not evidence the script ran. + WasThrown bool `json:"wasThrown"` +} + // Evaluate runs an expression in the page and returns its value as a string. +// +// A promise is awaited, as it is over CDP. WebKit's Runtime.evaluate has no +// awaitPromise parameter and ignores one sent, so it answered `{}` for every +// async expression — the promise itself, by value. So the expression is +// evaluated to a handle, a promise is awaited with Runtime.awaitPromise, and +// any other object is read back by value. CHALLENGES 201. func (s *IOSSession) Evaluate(ctx context.Context, expression string) (string, error) { - raw, err := s.call(ctx, "Runtime.evaluate", map[string]any{ - "expression": expression, - "returnByValue": true, - "awaitPromise": true, + r, err := s.evalCall(ctx, "Runtime.evaluate", map[string]any{ + "expression": expression, + "objectGroup": evalGroup, }) if err != nil { return "", err } - var result struct { - Result struct { - Type string `json:"type"` - Value json.RawMessage `json:"value"` - Description string `json:"description"` - } `json:"result"` - // WebKit reports a thrown exception inside a successful reply, the - // same trap CDP has: no error field is not evidence the script ran. - WasThrown bool `json:"wasThrown"` - } - if err := json.Unmarshal(raw, &result); err != nil { - return "", err + if id := r.Result.ObjectID; id != "" && !r.WasThrown { + defer func() { + _, _ = s.call(ctx, "Runtime.releaseObjectGroup", map[string]any{"objectGroup": evalGroup}) + }() + if r.Result.ClassName == "Promise" { + r, err = s.evalCall(ctx, "Runtime.awaitPromise", map[string]any{ + "promiseObjectId": id, + "returnByValue": true, + }) + } else { + r, err = s.evalCall(ctx, "Runtime.callFunctionOn", map[string]any{ + "objectId": id, + "functionDeclaration": "function() { return this; }", + "returnByValue": true, + }) + } + if err != nil { + return "", err + } } - if result.WasThrown { - msg := result.Result.Description + if r.WasThrown { + msg := r.Result.Description if msg == "" { - msg = string(result.Result.Value) + msg = string(r.Result.Value) } return "", mobiumerr.New(mobiumerr.DeviceServer, "page script failed: %s", firstLine(msg)) } - if result.Result.Type == "string" { + if r.Result.Type == "string" { var str string - if json.Unmarshal(result.Result.Value, &str) == nil { + if json.Unmarshal(r.Result.Value, &str) == nil { return str, nil } } - return string(result.Result.Value), nil + return string(r.Result.Value), nil +} + +func (s *IOSSession) evalCall(ctx context.Context, method string, params map[string]any) (*iosEvalResult, error) { + raw, err := s.call(ctx, method, params) + if err != nil { + return nil, err + } + var r iosEvalResult + if err := json.Unmarshal(raw, &r); err != nil { + return nil, err + } + return &r, nil } // LayoutMetrics reads the page's visual viewport. diff --git a/internal/webview/rwi_test.go b/internal/webview/rwi_test.go index c6a0a394..fc9472bc 100644 --- a/internal/webview/rwi_test.go +++ b/internal/webview/rwi_test.go @@ -88,6 +88,23 @@ func TestInspectorListsAndDrivesAPage(t *testing.T) { t.Errorf("1 + 1 came back as %q", got) } + // An async expression is awaited, as over CDP; WebKit's evaluate alone + // answered `{}`, the promise by value. CHALLENGES 201. + for expr, want := range map[string]string{ + "Promise.resolve(42)": "42", + "(async () => 'done')()": "done", + "({a: 1})": `{"a":1}`, + "new Promise(r => setTimeout(() => r([1, 2]), 50))": "[1,2]", + "document.title === document.title": "true", + } { + if got, err := page.Evaluate(ctx, expr); err != nil || got != want { + t.Errorf("%s = %q (err %v), want %q", expr, got, err, want) + } + } + if _, err := page.Evaluate(ctx, "Promise.reject(new Error('nope'))"); err == nil || !strings.Contains(err.Error(), "nope") { + t.Errorf("a rejected promise came back as %v, want the page's error", err) + } + m, err := page.LayoutMetrics(ctx) if err != nil { t.Fatalf("LayoutMetrics: %v", err) diff --git a/packaging/mcp/manifest.json b/packaging/mcp/manifest.json index 45f38750..4635508b 100644 --- a/packaging/mcp/manifest.json +++ b/packaging/mcp/manifest.json @@ -3,8 +3,8 @@ "name": "mobium", "display_name": "Mobium", "version": "@VERSION@", - "description": "Drive native Android and iOS apps on emulators, phones, simulators and iPhones, WebViews included.", - "long_description": "Mobium automates native mobile apps for AI agents: map the screen, tap, type and swipe by locator, wait for states, answer system dialogs, reach into WebViews, take screenshots and recordings, and set network conditions, location and accessibility settings. It drives Android through UiAutomator2 and iOS through WebDriverAgent, directly, with no Node. It needs the platform's own tools on the machine: Android's platform-tools for Android, Xcode for iOS.", + "description": "Drive Android and iOS apps — native, hybrid and cross-platform, WebViews and mobile browser pages included — on emulators, phones, simulators and iPhones.", + "long_description": "Mobium automates mobile apps for AI agents — native, hybrid, cross-platform and pages in a mobile browser: map the screen, tap, type and swipe by locator, wait for states, answer system dialogs, reach into WebViews, take screenshots and recordings, and set network conditions, location and accessibility settings. It drives Android through UiAutomator2 and iOS through WebDriverAgent, directly, with no Node. It needs the platform's own tools on the machine: Android's platform-tools for Android, Xcode for iOS.", "author": { "name": "Lana Begunova", "url": "https://github.com/mobiumdev" diff --git a/packaging/mcp/server.json b/packaging/mcp/server.json index f51ba085..7630d5c1 100644 --- a/packaging/mcp/server.json +++ b/packaging/mcp/server.json @@ -2,7 +2,7 @@ "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", "name": "dev.mobium/mobium", "title": "Mobium", - "description": "Drive native Android and iOS apps: emulators, phones, simulators, iPhones, WebViews.", + "description": "Drive Android and iOS apps, native to hybrid: emulators, phones, simulators, iPhones, WebViews.", "version": "@VERSION@", "websiteUrl": "https://github.com/mobiumdev/mobium", "repository": { diff --git a/skills/mobile-check/SKILL.md b/skills/mobile-check/SKILL.md index 78ec6d43..5d62ede3 100644 --- a/skills/mobile-check/SKILL.md +++ b/skills/mobile-check/SKILL.md @@ -1,11 +1,11 @@ --- name: mobile-check -description: Automate native mobile apps with the Mobium CLI, on Android emulators and phones and iOS simulators and iPhones. Use to launch and drive an app, fill forms, read the screen, answer system dialogs, reach into WebViews, capture screenshots and recordings, or confirm that a mobile change actually works. +description: Automate mobile apps — native, hybrid, cross-platform, and pages in a mobile browser or installed as a PWA — with the Mobium CLI, on Android emulators and phones and iOS simulators and iPhones. Use to launch and drive an app, fill forms, read the screen, answer system dialogs, reach into WebViews, capture screenshots and recordings, or confirm that a mobile change actually works. --- # Mobium Mobile Automation — CLI Reference -The `mobium` CLI automates **native apps** on Android emulators and phones, and +The `mobium` CLI automates **mobile apps** on Android emulators and phones, and iOS simulators and iPhones. A background daemon keeps the device session alive between commands, so each command is fast after the first. @@ -14,8 +14,19 @@ command is fast after the first. mobium launch com.example.shop && mobium map && mobium tap @e5 && mobium map ``` -If you need a *browser* rather than an app, use vibium instead. Mobium drives -apps; a WebView inside an app is covered here, a browser is not. +What the app is built with decides how to reach it: + +| App type | How | +| --- | --- | +| Native (Android SDK, Jetpack Compose, UIKit, SwiftUI) | `map` and act — the default | +| Cross-platform that renders native views (React Native) | the same as native: it renders real native views. Use the `@ref` from `map`, not a hand-written `label=` or `testid=`: one testID repeats across several views | +| Hybrid (a WebView inside the app) | switch context into the WebView — [Hybrid apps](#hybrid-apps-webviews) | +| A page in Chrome or Safari, or a PWA | the same contexts: the page is `WEBVIEW_…` — pick it by the URL `contexts` prints, not by its name. On iOS act from `NATIVE_APP` — taps in Safari's or a home-screen web app's page are refused. On Android, a refusal saying Chrome stopped reporting its WebView means no tap can be placed in that page: read it, and say so | +| Cross-platform that paints (Flutter) | not reachable through `map`; it paints its own widgets. Say so rather than retrying | + +If you need to test a *website* — tabs, browser sessions, a desktop browser — +use vibium instead. Mobium reaches a page already open on the device; it does +not manage the browser. ## Core Workflow diff --git a/tests/README.md b/tests/README.md index af247953..fc2ce7b6 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,8 +1,8 @@ # Tests Mobium's own tests of MobiumApp, run by `mobium test` — and the suite that -holds the runner to its word. The format and the reasons are in -[decisions/0006](../docs/decisions/0006-a-test-runner.md). +holds the runner to its word. The format is in the +[test runner guide](../docs/guides/test-runner.md). ```sh cd tests