Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,8 @@ docs-check:
|| { python3 docs/checks/doc-links.py $$(git ls-files '*.md'); exit 1; }
@python3 docs/quickstart/build.py --check >/dev/null \
|| { python3 docs/quickstart/build.py --check; exit 1; }
@echo "docs: spelling, anchors and quick-start pages clean"
@sh docs/checks/lib-selftest.sh >/dev/null || { sh docs/checks/lib-selftest.sh; exit 1; }
@echo "docs: spelling, anchors, quick-start pages and the checks' prelude clean"

ci: fmt-check vet lint test clients crosscompile java api-check flags-check license-check docs-check dotnet

Expand Down
32 changes: 32 additions & 0 deletions docs/checks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,38 @@ simulator UDID, `ios-webview.sh` defaults to the booted simulator,
`test-runner.sh` and `test-ui.sh` run on `emulator-5554` and read
`MOBIUM_IOS_DEVICE` for iOS, and `test-grid.sh` reads `MOBIUM_GRID`.

## Running several at once

Every check sources [lib.sh](lib.sh), which keeps it from reaching into
another:

- **A daemon of its own** (`MOBIUM_SESSION`), stopped when the check ends.
Checks once shared one: a check that ended on the `uiautomator` backend
handed it to every check after, and one run's `daemon stop` ended another
device's trace. A session the caller set, as a grid run's or
`mobium test`'s, is kept.
- **One check per device.** `check_lock` takes the device, and a second
check on it is refused, naming the first; a lock whose holder has gone is
taken over. A check that runs another shares its lock.
- **A folder of its own**, `CHECK_TMP`, removed at the end, and `at_exit` for
cleanup in place of `trap ... EXIT`, which would replace the prelude's.

So checks on different devices can run side by side:

```sh
./docs/checks/obstruction.sh emulator-5554 &
./docs/checks/obstruction.sh <simulator-udid> &
wait
```

What they share is the machine: an emulator under load stopped answering
`adb` once, mid-sweep, so start with as many devices as the Mac drives
comfortably. `make ci` runs [lib-selftest.sh](lib-selftest.sh), which tests
the prelude without a device and fails when a check stops using it. Four
checks do not: `clean-stop.sh`, which stops the caller's daemon, and
`test-runner.sh`, `test-grid.sh` and `test-ui.sh`, which test the sessions
`mobium test` and the grid give themselves.

| Script | What it proves |
| --- | --- |
| [calculator.sh](calculator.sh) | A full type-and-read loop: 1 + 1 = 2 in Google Calculator, every step asserted against a named element |
Expand Down
6 changes: 4 additions & 2 deletions docs/checks/accessibility.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-28s %-40s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand All @@ -33,7 +35,7 @@ echo "--- $DEV ($PLATFORM)"
if [ "$PLATFORM" = phone ]; then
# Somebody's phone: every change is put back when the daemon stops, and
# the trap stops it on failure too.
trap '$ROOT/bin/mobium daemon stop >/dev/null 2>&1 || true' EXIT
at_exit '$ROOT/bin/mobium daemon stop >/dev/null 2>&1 || true'
SWITCHES="reduce_motion bold_text increase_contrast reduce_transparency button_shapes differentiate_without_color"
before=$($M accessibility) || fail "reading the settings: $before"
$M terminate $APP >/dev/null 2>&1 || true; $M launch $APP >/dev/null
Expand Down Expand Up @@ -103,7 +105,7 @@ live() {
$M daemon stop >/dev/null 2>&1 || true
BEFORE=$(raw)
LIVE_BEFORE=$(live)
trap '$M daemon stop >/dev/null 2>&1 || true' EXIT
at_exit '$M daemon stop >/dev/null 2>&1 || true'

# What each platform has, and the value each is changed to. The last column
# is the app's word for it in a11yState, or - where React Native tells the
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/audit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-10s %-58s ok\n' "$1" "$2"; }
json() { python3 -c "import json,sys; d=json.load(sys.stdin); print($1)"; }
Expand Down
4 changes: 3 additions & 1 deletion docs/checks/autowait.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-16s %-52s ok\n' "$1" "$2"; }
ms() { python3 -c 'import time; print(int(time.time()*1000))'; }
Expand Down Expand Up @@ -63,7 +65,7 @@ reduce() { # reduce on|off
fi
}
cleanup() { reduce off >/dev/null 2>&1 || true; $M terminate "$APP" >/dev/null 2>&1 || true; }
trap cleanup EXIT
at_exit cleanup
# A session first, so the setting's undo belongs to it.
$M current >/dev/null

Expand Down
2 changes: 2 additions & 0 deletions docs/checks/back.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <android-serial | simulator-udid | iphone-udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
APP=dev.mobium.mobiumapp
case "$DEV" in
????????-????????????????) PLATFORM=ios; PHONE=1 ;;
Expand Down
4 changes: 3 additions & 1 deletion docs/checks/battery.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@
# battery is put back — full, on AC — at the end.
set -e
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
check_lock "$DEV"
case "$DEV" in
*-*-*-*-*) PLATFORM=ios; M="$ROOT/bin/mobium --driver wda --device $DEV" ;;
emulator-*) PLATFORM=android; M="$ROOT/bin/mobium --device $DEV" ;;
Expand Down Expand Up @@ -56,7 +58,7 @@ fi
ADB="adb -s $DEV"
restore() { $ADB emu power ac on >/dev/null 2>&1; $ADB emu power status charging >/dev/null 2>&1
$ADB emu power capacity 100 >/dev/null 2>&1; }
trap restore EXIT
at_exit restore

for want in 42 73; do
$ADB emu power ac off >/dev/null
Expand Down
6 changes: 4 additions & 2 deletions docs/checks/biometric.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-14s %-60s ok\n' "$1" "$2"; }

Expand Down Expand Up @@ -76,7 +78,7 @@ restore() {
*) $M biometric enroll >/dev/null 2>&1 || true ;;
esac
}
trap restore EXIT
at_exit restore

$M terminate $APP >/dev/null 2>&1 || true
$M launch $APP >/dev/null
Expand Down Expand Up @@ -178,7 +180,7 @@ else
row "after" "the enrolled finger accepted once the lockout ended"
fi

trap - EXIT
at_exit_clear
restore
case "$was" in
"no "*) now=$($M biometric); case "$now" in "no "*) ;; *) fail "not put back: $now" ;; esac
Expand Down
1 change: 1 addition & 0 deletions docs/checks/boot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ set -e
AVD="$1"; SIM="$2"
if [ -z "$AVD" ]; then echo "usage: $0 <avd> [simulator-udid]" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
M="$ROOT/bin/mobium"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-12s %-60s ok\n' "$1" "$2"; }
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/chrome.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ case "$DEV" in
*) PHONE=1 ;;
esac
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
M="$ROOT/bin/mobium --device $DEV"
# closeTabs closes the pages in the phone's Chrome whose URL matches $1, a
# Python regular expression, through Chrome's own DevTools endpoint: Mobium has
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/clear-data.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
json() { python3 -c "import json,sys; d=json.load(sys.stdin); print($1)"; }
APP=dev.mobium.mobiumapp
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/clients.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <android-serial>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
cd "$ROOT" || exit 1
MOBIUM_E2E_DEVICE="$DEV"; MOBIUM_BIN_PATH="$ROOT/bin/mobium"
export MOBIUM_E2E_DEVICE MOBIUM_BIN_PATH
Expand Down
4 changes: 3 additions & 1 deletion docs/checks/compose-app.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@
# printed.
set -e
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
DEV="${1:?usage: compose-app.sh <serial> [path-to-apk]}"
check_lock "$DEV"
APK="$2"
APP=com.junkfood.seal
M="$ROOT/bin/mobium --device $DEV"
Expand All @@ -45,7 +47,7 @@ cleanup() {
$M uninstall $APP >/dev/null 2>&1 || true
fi
}
trap cleanup EXIT
at_exit cleanup

"$ROOT/bin/mobium" daemon stop >/dev/null 2>&1 || true
echo "--- $DEV"
Expand Down
6 changes: 4 additions & 2 deletions docs/checks/crashes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi

ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
json() { python3 -c "import json,sys; d=json.load(sys.stdin); print($1)"; }

Expand Down Expand Up @@ -233,7 +235,7 @@ if [ "$PLATFORM" = android ] && [ "$APP" = dev.mobium.mobiumapp ]; then
adb -s "$DEV" unroot >/dev/null 2>&1
as_user != || echo "WARNING: adb is still root on $DEV" >&2
set -e; }
trap thaw EXIT
at_exit thaw
adb -s "$DEV" shell am force-stop "$APP"
$M launch "$APP" >/dev/null
# Frozen before its first screen is drawn, an app has no window to be
Expand All @@ -254,7 +256,7 @@ if [ "$PLATFORM" = android ] && [ "$APP" = dev.mobium.mobiumapp ]; then
$M crashes "$anr" | head -1 | grep -q "^ANR: Input dispatching timed out" || fail "ANR $anr does not lead with its reason"
ref=$($M map | grep "Close app" | grep -oE '@e[0-9]+')
$M tap "$ref" >/dev/null
thaw; trap - EXIT
thaw; at_exit_clear
echo " anr a frozen app: its dialog, its log line, its report ok"
else
adb -s "$DEV" unroot >/dev/null 2>&1 || true
Expand Down
4 changes: 3 additions & 1 deletion docs/checks/device-paths.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <android-serial | simulator-udid | iphone-udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
case "$DEV" in
????????-????????????????) PLATFORM=ios; PHONE=1 ;;
*-*-*-*-*) PLATFORM=ios; PHONE="" ;;
Expand All @@ -35,7 +37,7 @@ hash() { shasum -a 256 "$1" | cut -c1-16; }
echo "--- $DEV ($PLATFORM)"

W=$(mktemp -d)
trap 'rm -rf "$W"' EXIT
at_exit 'rm -rf "$W"'
head -c 70000 /dev/urandom > "$W/bin.dat"
mkdir -p "$W/tree/sub"; echo one > "$W/tree/a.txt"; echo two22 > "$W/tree/sub/b.txt"

Expand Down
4 changes: 3 additions & 1 deletion docs/checks/device-state.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@ DEV="$1"; APP="${2:-org.wikipedia}"
if [ -z "$DEV" ]; then echo "usage: $0 <serial> [package]" >&2; exit 2; fi

ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
M="$ROOT/bin/mobium --device $DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
restore() { $M orientation auto >/dev/null 2>&1 || true
$M locale "$APP" "" >/dev/null 2>&1 || true; }
trap restore EXIT
at_exit restore

echo "--- $DEV"

Expand Down
2 changes: 2 additions & 0 deletions docs/checks/dialogs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-18s %-50s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
4 changes: 3 additions & 1 deletion docs/checks/external-driver.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial> (see: mobium devices)" >&2; exit 2; fi

ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
M="$ROOT/bin/mobium --device $DEV"
MOBIUM_DRIVER_ADB="$ROOT/examples/drivers/mobium-driver-adb"
export MOBIUM_DRIVER_ADB
Expand All @@ -26,7 +28,7 @@ export MOBIUM_DRIVER_ADB
"$ROOT/bin/mobium" daemon stop >/dev/null 2>&1 || true

TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
at_exit 'rm -rf "$TMP"'
fail() { echo "FAIL: $*" >&2; exit 1; }

echo "--- $DEV"
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/feed.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-12s %-58s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
6 changes: 4 additions & 2 deletions docs/checks/files.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@
# mobium-report.txt is there already, and removes only the two files it made.
set -e
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
check_lock "$DEV"
# A simulator is a UUID of five groups; a real iPhone's UDID is two,
# 00008120-0001234567890ABC. Anything else is an Android serial.
PHONE=
Expand Down Expand Up @@ -74,7 +76,7 @@ cleanup() {
rm -f "$docs/$UPLOAD" "$docs/mobium-report.txt"
fi
}
trap cleanup EXIT
at_exit cleanup

"$ROOT/bin/mobium" daemon stop >/dev/null 2>&1 || true
echo "--- $DEV ($PLATFORM)"
Expand All @@ -91,7 +93,7 @@ echo "$apps" | grep -q "$APP" || fail "$APP is not installed"
if [ "$PLATFORM" = android ]; then
there=$(adb -s "$DEV" shell ls /sdcard/Download/ 2>/dev/null | tr -d '\r' | grep '^mobium-report' | tr '\n' ' ' | sed 's/ *$//')
if [ -n "$there" ]; then
trap - EXIT
at_exit_clear
fail "the Download folder holds $there already — the app would save under another number and this check would read the wrong one; remove them if they are an earlier run's"
fi
fi
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/flutter.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <android-serial | simulator-udid | iphone-udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
case "$DEV" in
????????-????????????????|*-*-*-*-*) PLATFORM=ios; APP=dev.mobium.mobiumFlutter
M="$ROOT/bin/mobium --driver wda --device $DEV" ;;
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/frames.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-12s %-58s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/gestures.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
set -e
DEV="${1:?usage: gestures.sh <udid|serial>}"
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
case "$DEV" in
[0-9A-Fa-f]*-[0-9A-Fa-f]*-[0-9A-Fa-f]*-[0-9A-Fa-f]*-[0-9A-Fa-f]*)
BACKEND="--driver wda"; PLATFORM=ios ;;
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/graybox-edges.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-8s %-62s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/graybox-hooks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ set -e
DEV="$1"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid>" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-11s %-58s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
2 changes: 2 additions & 0 deletions docs/checks/graybox.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ DEV="$1"
ROUNDS="${2:-10}"
if [ -z "$DEV" ]; then echo "usage: $0 <serial|udid> [rounds]" >&2; exit 2; fi
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
. "$ROOT/docs/checks/lib.sh"
check_lock "$DEV"
fail() { echo "FAIL: $*" >&2; exit 1; }
row() { printf ' %-12s %-58s ok\n' "$1" "$2"; }
APP=dev.mobium.mobiumapp
Expand Down
Loading
Loading