Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,63 @@ test('the first capture after an open does not take a busy indicator for the fir
assert.equal(result?.snapshot.postGestureOutcome?.kind, 'loading');
});

test('the first capture after an open does not take a lone status line for the first screen', async () => {
vi.useFakeTimers();
const session = makeSession('ios');
markPostOpenStabilization(session, 'observable');
// A cold Expo bundle: one progress line, mirrored by React Native's accessibility element.
const statusLine = (label: string) => ({
type: 'StaticText',
label,
rect: { x: 10, y: 70, width: 382, height: 14 },
});
const banner = makeSnapshotState([
APP_ROOT,
{ index: 1, parentIndex: 0, ...statusLine('Bundling 91%…') },
{ index: 2, parentIndex: 1, ...statusLine('Bundling 91%…') },
]);
const home = homeScreenSnapshot();
const { capture } = scriptedCapture([banner, banner, banner, home, home]);

const pendingResult = resolveDeferredInteractionOutcome(resolveParams(session, capture));
await vi.advanceTimersByTimeAsync(2_000);
const result = await pendingResult;

assert.equal(result?.snapshot, home);
assert.equal(result?.snapshot.postGestureOutcome, undefined);
});

test('the first capture after an open settles a screen of two texts on its first quiet pair', async () => {
vi.useFakeTimers();
const session = makeSession('ios');
markPostOpenStabilization(session, 'observable');
const twoTexts = makeSnapshotState([
APP_ROOT,
{
index: 1,
parentIndex: 0,
type: 'StaticText',
label: 'Welcome',
rect: { x: 20, y: 100, width: 350, height: 30 },
},
{
index: 2,
parentIndex: 0,
type: 'StaticText',
label: 'Sign in to continue',
rect: { x: 20, y: 140, width: 350, height: 20 },
},
]);
const { capture, calls } = scriptedCapture([twoTexts, twoTexts]);

const pendingResult = resolveDeferredInteractionOutcome(resolveParams(session, capture));
await vi.advanceTimersByTimeAsync(1_000);
const result = await pendingResult;

assert.equal(calls(), 2);
assert.equal(result?.snapshot.postGestureOutcome, undefined);
});

test('the first capture after an open settles a one-button first screen on its first quiet pair', async () => {
vi.useFakeTimers();
const session = makeSession('ios');
Expand Down
32 changes: 20 additions & 12 deletions packages/daemon/src/core/deferred-interaction-outcome.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,20 +255,28 @@ export function markPostOpenStabilization(
}

/**
* Whether a launched app's tree shows its first screen: some node besides the Application and Window
* roots carries a label or value that is not a busy indicator's. Launch surfaces carry none. Measured
* on Bluesky, the storyboard splash is one unlabeled full-screen Image and the empty React Native
* root is one unlabeled view, and each can hold unchanged for longer than a second. A busy indicator
* is labeled ("In progress") but says only that the screen is not ready. No node count or quiet
* window is needed: a one-button screen is ready at once, and a two-image splash never is.
* Whether a launched app's tree shows its first screen: besides the Application and Window roots, it
* reads something other than a busy indicator or a lone status line. Measured on Bluesky, the
* storyboard splash is one unlabeled full-screen Image and the empty React Native root is one
* unlabeled view, and each can hold unchanged for longer than a second. A busy indicator is labeled
* ("In progress") but says only that the screen is not ready. A lone text is a status line: a cold
* Expo bundle shows only "Bundling 91%…", and two reads 200 ms apart agree on it. So the screen is
* ready once a readable element is not text, such as a button, or two different texts show. No node
* count or quiet window is needed: a one-button screen is ready at once, and a splash never is.
*/
function showsFirstScreen(nodes: readonly SnapshotNode[]): boolean {
return nodes.some(
(node) =>
!isViewportRootNode(node) &&
Boolean(node.label?.trim() || node.value?.trim()) &&
!BUSY_INDICATOR_ROLES.has(formatRole(node.type ?? '')),
);
const texts = new Set<string>();
for (const node of nodes) {
if (isViewportRootNode(node)) continue;
const reading = node.label?.trim() || node.value?.trim();
if (!reading) continue;
const role = formatRole(node.type ?? '');
if (BUSY_INDICATOR_ROLES.has(role)) continue;
if (role !== 'text') return true;
Comment thread
thymikee marked this conversation as resolved.
Resolved
texts.add(reading);
if (texts.size > 1) return true;
}
return false;
}

const BUSY_INDICATOR_ROLES: ReadonlySet<string> = new Set([
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
- `open [app|url] [url]` boots the selected device when needed and opens the target in a session. `open <url>` opens deep links on Android and iOS, and `open <app> <url>` opens a deep link in that app on iOS. See [Desktop targets](#desktop-targets) for macOS activation behavior.
- Remote daemons refuse `file:` URLs in `open` targets and launch URLs with reason `REMOTE_HOST_INPUT_DENIED`. Serve local web content over HTTP(S) instead.
- On iOS devices, `http(s)://` URLs open in Safari when no app is active. Custom-scheme URLs need an active app in the session.
- Local iOS simulator: after `open` launches an app and the backend reports that launch (local iOS simulators do; other devices and backends skip the wait, so their first capture can still show a launch screen), the first capture in the session polls for the app's first screen for up to 10 seconds. A capture that starts before the 10 seconds end still completes and is judged. This includes the initial snapshot of `open <app> --foreground`. A tree counts as the first screen when an element other than the app root has a label or value, and two reads in a row agree. A splash image, an empty view, or only a busy indicator does not count, however long it stays unchanged. An app whose first screen has no labeled element, such as a game canvas, waits the full 10 seconds. If the wait runs out, the snapshot returns what it read with a warning: `postGestureOutcome.kind: "loading"` when it still showed no first screen, or `"unsettled"` when a first screen was there but kept changing. Run `snapshot -i` again, or `wait text <expected>`, before you act on it.
- Local iOS simulator: after `open` launches an app and the backend reports that launch (local iOS simulators do; other devices and backends skip the wait, so their first capture can still show a launch screen), the first capture in the session polls for the app's first screen for up to 10 seconds. A capture that starts before the 10 seconds end still completes and is judged. This includes the initial snapshot of `open <app> --foreground`. A tree counts as the first screen when, besides the app root, it shows a labeled element that is not text, such as a button, or two different texts, and two reads in a row agree. A splash image, an empty view, a busy indicator, or a single status line such as Expo's "Bundling 91%…" does not count, however long it stays unchanged. An app whose first screen has no labeled element, or a single line of text, such as a game canvas, waits the full 10 seconds. If the wait runs out, the snapshot returns what it read with a warning: `postGestureOutcome.kind: "loading"` when it still showed no first screen, or `"unsettled"` when a first screen was there but kept changing. Run `snapshot -i` again, or `wait text <expected>`, before you act on it.

Check warning on line 102 in website/docs/docs/commands.md

View check run for this annotation

apex-sentinel-staging / Apex Sentinel

minor: Docs sentence makes 'game canvas' the example of a single line of text

The sentence "An app whose first screen has no labeled element, or a single line of text, such as a game canvas, waits the full 10 seconds" attaches the example to the newly inserted clause, so a game canvas is described as a single line of text instead of an app with no labeled element. Reorder to "An app whose first screen is a single line of text, or has no labeled element at all, such as a game canvas, waits the full 10 seconds."
Comment thread
thymikee marked this conversation as resolved.
Resolved
Comment thread
thymikee marked this conversation as resolved.
Resolved
- `open <app> --wait <ms>` waits up to that budget for a device another session holds, then fails with `DEVICE_IN_USE`. The wait extends the command's timeout, so you don't need a longer `--timeout`. A claim held by another workspace's daemon is not waited for; `open` returns its recovery command at once. Device-mutating commands like `install` and `reinstall` never wait either: a claim held by another session fails them at once, and when the caller sends a workspace, another workspace's claim under the same session name fails too. See [Sessions](/docs/sessions#wait-for-a-slow-or-busy-device).
- `open <app> --launch-console <path>` captures launch-time stdout/stderr for iOS simulator app launches, including Limrun. It is not valid for URL opens or other targets.
- `open --platform macos --surface app|frontmost-app|desktop|menubar` selects the macOS session surface; `app` is the default when you name an app. See [Desktop targets](#desktop-targets). `--save-script` records the surface so `replay` reopens it; with no surface given, replay opens an app session.
Expand Down
Loading