Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions packages/apple-runner/src/__tests__/index-busy-resend.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -264,3 +264,36 @@ test('a read-only alert get answered ALERT_UNRESOLVED is sent once and reported

assert.deepEqual(sentCommands(), ['alert']);
});

test('an alert get cut off by the watchdog reports its typed unknown outcome, and the next get waits out RUNNER_BUSY', async () => {
vi.useFakeTimers();
mockExecuteRunnerCommandWithSession
.mockRejectedValueOnce(
buildRunnerResponseError({
ok: false,
error: { code: 'MAIN_THREAD_TIMEOUT', message: 'main thread work was abandoned' },
}),
)
.mockRejectedValueOnce(busyRefusal())
.mockRejectedValueOnce(busyRefusal())
.mockResolvedValueOnce({ title: 'Camera Access' });

const first = runAppleRunnerCommand(IOS_SIMULATOR, { command: 'alert', action: 'get' });
const firstError = first.then(
() => undefined,
(error: unknown) => error,
);
await vi.advanceTimersByTimeAsync(0);
const watchdog = await firstError;
assert.ok(watchdog instanceof AppError);
assert.equal(watchdog.details?.runnerErrorCode, 'MAIN_THREAD_TIMEOUT');
assert.equal(watchdog.details?.reason, 'runner_main_thread_timeout');
assert.equal(watchdog.details?.dispatched, 'unknown');
assert.deepEqual(sentCommands(), ['alert']);

const second = runAppleRunnerCommand(IOS_SIMULATOR, { command: 'alert', action: 'get' });
await vi.advanceTimersByTimeAsync(6_000);

assert.deepEqual(await second, { title: 'Camera Access' });
assert.deepEqual(sentCommands(), ['alert', 'alert', 'alert', 'alert']);
});
2 changes: 1 addition & 1 deletion packages/command-surface/src/capture/alert.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { AppError } from '@agent-device/kernel/errors';
const ALERT_COMMAND_NAME = 'alert';

const alertCommandDescription =
"Inspect, wait for, accept, or dismiss a platform alert. The get action reads alert content; accept and dismiss change the active alert state. Each iOS XCTest execution activates once, then only observes. Apple XCTest and Limrun iOS choose recognized English labels by preference (Allow While Using App before Allow Once); on iOS simulators, including Limrun, the runner also recognizes the device's preferred languages' wording of those labels from the OS permission-prompt strings, on any alert; equal-ranked choices and unknown labels refuse, including a lone unknown button. Open, Done, and Confirm… accept; Keep Browsing, Deny, and Skip dismiss. On the Apple XCUITest runner, dismiss taps only a recognized decline label (Cancel, Don't Allow, Ask App Not to Track, and similar) or the close element of a popover whose accessibility marker reads exactly \"dismiss popup\"; when no button fits the action, accept and dismiss answer ALERT_BUTTON_NOT_FOUND having tapped nothing. The get action exposes labels for explicit button targeting. A get result with status none reports no alert. An alert-not-found refusal means no alert was found and no button was pressed. An alert-wait-timeout means no actionable alert resolved within the wait window. A refusal with dispatched: no means no button was activated. After a press, an unconfirmed outcome leaves the resulting alert state unknown.";
"Inspect, wait for, accept, or dismiss a platform alert. The get action reads alert content; accept and dismiss change the active alert state. Each iOS XCTest execution activates once, then only observes. Apple XCTest and Limrun iOS choose recognized English labels by preference (Allow While Using App before Allow Once); on iOS simulators, including Limrun, the runner also recognizes the device's preferred languages' wording of those labels from the OS permission-prompt strings, on any alert; equal-ranked choices and unknown labels refuse, including a lone unknown button. Open, Done, and Confirm… accept; Keep Browsing, Deny, and Skip dismiss. On the Apple XCUITest runner, dismiss taps only a recognized decline label (Cancel, Don't Allow, Ask App Not to Track, and similar) or the close element of a popover whose accessibility marker reads exactly \"dismiss popup\"; when no button fits the action, accept and dismiss answer ALERT_BUTTON_NOT_FOUND having tapped nothing. The get action exposes labels for explicit button targeting. On the Apple XCUITest runner, get takes the timeout argument as the runner's budget (10 s by default, at least 1.5 s, at most 25 s). A get the runner's watchdog (the same budget) cuts off answers runner_main_thread_timeout with dispatched: unknown, which does not mean no alert; the next get waits out RUNNER_BUSY while abandoned work drains, and still answers RUNNER_BUSY (dispatched: no) if that outlasts the bounded resends. A get result with status none reports no alert. An alert-not-found refusal means no alert was found and no button was pressed. An alert-wait-timeout means no actionable alert resolved within the wait window. A refusal with dispatched: no means no button was activated. After a press, an unconfirmed outcome leaves the resulting alert state unknown.";

const alertCommandMetadata = defineFieldCommandMetadata(
ALERT_COMMAND_NAME,
Expand Down
13 changes: 12 additions & 1 deletion packages/contracts/src/alert-contract.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,21 @@ export const ALERT_POLL_INTERVAL_MS = 300;
* The minimum budget for a retry of `alert accept` or `alert dismiss` through the XCTest runner.
* A read cut off by the runner's main-thread watchdog is reported as `MAIN_THREAD_TIMEOUT`, not
* absence. Live late-prompt reads were accepted up to 1492 ms; three were cut off at 1502–1610 ms.
* `alert wait` and the macOS helper do not use it.
* It is also the floor of the budget `alert get` sends the runner. `alert wait` and the macOS
* helper do not use it.
*/
export const ALERT_MIN_READ_BUDGET_MS = 1_500;
export const DEFAULT_ALERT_TIMEOUT_MS = 10_000;

/**
* The ceiling of the budget one XCTest runner alert attempt receives. For `alert`, the runner's
* main-thread watchdog and its inner alert deadline are both this budget, so it also bounds how
* long one attempt can occupy the runner before `MAIN_THREAD_TIMEOUT`. It stays below the 45 s
* runner transport deadline and the 90 s daemon request envelope, so a stalled attempt ends as the
* runner's typed answer, never as a transport or envelope timeout. `alert wait` polls in attempts
* of at most this budget.
*/
export const ALERT_RUNNER_MAX_BUDGET_MS = 25_000;
export const ALERT_ACTION_RETRY_MS = 2_000;

/**
Expand Down
44 changes: 41 additions & 3 deletions packages/platform-apple/src/__tests__/alert.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ vi.mock('../os/macos/helper.ts', () => ({ runMacOsAlertAction: vi.fn() }));
import {
ALERT_MIN_READ_BUDGET_MS,
ALERT_NOT_FOUND_REASON,
ALERT_RUNNER_MAX_BUDGET_MS,
DEFAULT_ALERT_TIMEOUT_MS,
ALERT_NOT_FOUND_RUNNER_CODE,
ALERT_WAIT_TIMEOUT_REASON,
ALERT_POLL_INTERVAL_MS,
Expand Down Expand Up @@ -53,18 +55,54 @@ afterEach(() => {

// R59 moved these windows out of the daemon: how long a transient sheet takes to appear, and how
// many times to re-ask a runner that says it is not there yet, are Apple family mechanics.
test('a read spends the family default, not the caller window', async () => {
test.each([
['the caller window', 3_000, 3_000],
['no window', undefined, DEFAULT_ALERT_TIMEOUT_MS],
['a window below the floor', 37, ALERT_MIN_READ_BUDGET_MS],
['a window above the ceiling', 60_000, ALERT_RUNNER_MAX_BUDGET_MS],
])('a read spends %s as the runner budget', async (_label, timeoutMs, budgetMs) => {
mockRunner.mockResolvedValue({ title: 'Camera Access' });

await readAppleAlert(IOS_SIMULATOR, runnerOptions, { timeoutMs: 37 }, process.env);
await readAppleAlert(
IOS_SIMULATOR,
runnerOptions,
timeoutMs === undefined ? {} : { timeoutMs },
process.env,
);

assert.equal(mockRunner.mock.calls.length, 1);
assert.deepEqual(mockRunner.mock.calls[0]?.[1], {
command: 'alert',
action: 'get',
appBundleId: undefined,
timeoutMs: 10_000,
timeoutMs: budgetMs,
});
});

test('a read cut off by the runner watchdog is reported as itself, never as absence', async () => {
const watchdog = new AppError('COMMAND_FAILED', 'runner main thread timed out', {
runnerErrorCode: 'MAIN_THREAD_TIMEOUT',
reason: 'runner_main_thread_timeout',
dispatched: 'unknown',
});
mockRunner.mockRejectedValue(watchdog);

await assert.rejects(
alertIfPresent(readAppleAlert(IOS_SIMULATOR, runnerOptions, { timeoutMs: 3_000 }, process.env)),
(error: unknown) => error === watchdog,
);
assert.equal(mockRunner.mock.calls.length, 1);
});

test('a wait window above the ceiling is polled in runner attempts within the ceiling', async () => {
mockRunner.mockResolvedValue({ title: 'Camera Access' });

await awaitAppleAlert(IOS_SIMULATOR, runnerOptions, { timeoutMs: 120_000 }, process.env);

assert.equal(
(mockRunner.mock.calls[0]?.[1] as { timeoutMs: number } | undefined)?.timeoutMs,
ALERT_RUNNER_MAX_BUDGET_MS,
);
});

test('a wait polls until one attempt answers, and the first attempt gets the whole window', async () => {
Expand Down
28 changes: 24 additions & 4 deletions packages/platform-apple/src/alert.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import {
ALERT_WAIT_TIMEOUT_REASON,
ALERT_NOT_FOUND_RUNNER_CODE,
ALERT_POLL_INTERVAL_MS,
ALERT_RUNNER_MAX_BUDGET_MS,
ALERT_UNRESOLVED_RUNNER_CODE,
DEFAULT_ALERT_TIMEOUT_MS,
} from '@agent-device/contracts/alert-contract';
Expand Down Expand Up @@ -60,7 +61,12 @@ function runAppleAlert(
return async (action, timeoutMs) =>
(await runAppleRunnerCommand(
device,
{ command: 'alert', action, appBundleId: options?.appBundleId, timeoutMs },
{
command: 'alert',
action,
appBundleId: options?.appBundleId,
timeoutMs: Math.min(timeoutMs, ALERT_RUNNER_MAX_BUDGET_MS),
Comment thread
thymikee marked this conversation as resolved.
Resolved
},
runnerOptions,
)) as Record<string, unknown>;
}
Expand All @@ -75,18 +81,32 @@ export async function alertIfPresent<T>(read: Promise<T>): Promise<T | undefined
}
}

/**
* One read spends the caller's window as the runner's budget, within
* `ALERT_MIN_READ_BUDGET_MS` and `ALERT_RUNNER_MAX_BUDGET_MS`, and `DEFAULT_ALERT_TIMEOUT_MS` when
* the caller gave none. A read the runner's main-thread watchdog cuts off is not absence: its
* `MAIN_THREAD_TIMEOUT` is reported as itself (`reason: runner_main_thread_timeout`, `dispatched:
* unknown`). The next read runs once the abandoned work drains, or reports `RUNNER_BUSY` when the
* bounded resends end first.
*/
export async function readAppleAlert(
device: DeviceInfo,
runnerOptions: RunnerCallOptions,
options: AlertInteractorOptions | undefined,
env: Readonly<EnvMap>,
): Promise<Record<string, unknown>> {
return await runAppleAlert(device, runnerOptions, options, env)('get', DEFAULT_ALERT_TIMEOUT_MS);
const requestedMs = options?.timeoutMs;
const budgetMs =
requestedMs === undefined || !Number.isFinite(requestedMs)
? DEFAULT_ALERT_TIMEOUT_MS
: Math.max(requestedMs, ALERT_MIN_READ_BUDGET_MS);
return await runAppleAlert(device, runnerOptions, options, env)('get', budgetMs);
}

/**
* Poll `get` until one answers or the caller's window runs out. The first attempt gets the whole
* window and later ones only what is left, so a runner that blocks cannot outlive the budget.
* Poll `get` until one answers or the caller's window runs out. The first attempt gets the window,
* capped at `ALERT_RUNNER_MAX_BUDGET_MS`, and later ones only what is left (same cap), so a runner
* that blocks cannot outlive the budget.
* After a completed observation, another read needs more than one polling interval of budget.
* A dispatched read still owns its result, including a watchdog timeout.
*/
Expand Down
6 changes: 6 additions & 0 deletions src/__tests__/command-descriptor-timeout-policy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
resolveCommandPostActionObservationSupport,
resolveCommandTimeoutPolicy,
} from '@agent-device/command-registry/registry';
import { ALERT_RUNNER_MAX_BUDGET_MS } from '@agent-device/contracts/alert-contract';
import { OVERLAY_BUDGET_MS } from '@agent-device/capture-kit/recording-overlay';
import {
DEFAULT_TIMEOUT_POLICY,
Expand Down Expand Up @@ -410,3 +411,8 @@ test('open and prepare startup budgets keep a client-envelope margin over the da
90_000,
);
});

test('the fixed alert request envelope stays above the largest runner alert budget', () => {
const envelopeMs = resolveCommandRequestTimeoutMs(resolveCommandTimeoutPolicy('alert'), {});
assert.ok(envelopeMs !== undefined && envelopeMs > ALERT_RUNNER_MAX_BUDGET_MS);
Comment thread
thymikee marked this conversation as resolved.
Resolved
});
2 changes: 1 addition & 1 deletion website/docs/docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,7 @@ agent-device alert dismiss
- `wait_stable_timeout`: the UI did not settle.
- Branch on `readableCaptures`, `waitedMs`, `matches`, and `firstMatch` instead of parsing error text. `firstMatch` carries identity and text only; absence failures make no visibility or rect claim.
- Polling wait timeouts, and a replayed wait refused with `wait_landmark_identity_mismatch`, also carry `captures`, `readableCaptures`, and `polls` so you can see where the budget went. Each poll has `startedMs`, `durationMs`, and an `outcome`: `readable`, `unreadable`, `retriable`, `deadline`, `runner-restart`, or `readiness`. Long waits keep the first 5 and last 25 polls. A wait that never saw a readable capture reports the error its polls hit, with that error's code and retry details, instead of a generic timeout; `wait stable` and a never-readable strict absence keep their own diagnostics. `logPath` links the full request log.
- `alert` (same as `alert get`) inspects or handles system alerts on iOS simulator, macOS desktop, and Android permission dialogs. On iOS it also covers app-owned modal popups with native blocking markers. On the Apple XCUITest runner it also covers Safari's own page prompts, such as "Open this page in …?". The `[timeout]` argument applies to `alert wait` only, on every backend: `alert wait <ms>` takes a budget up to 2147483647 ms (~24.8 days), larger is refused as `INVALID_ARGS` before anything runs, while `get`/`accept`/`dismiss` run their own fixed per-backend budgets and ignore it.
- `alert` (same as `alert get`) inspects or handles system alerts on iOS simulator, macOS desktop, and Android permission dialogs. On iOS it also covers app-owned modal popups with native blocking markers. On the Apple XCUITest runner it also covers Safari's own page prompts, such as "Open this page in …?". The `[timeout]` argument applies to `alert wait` on every backend: `alert wait <ms>` takes a budget up to 2147483647 ms (~24.8 days), larger is refused as `INVALID_ARGS` before anything runs. On the Apple XCUITest runner, `alert get <ms>` also takes it as the runner's budget for the read, clamped to 1500–25000 ms and 10000 ms without an argument; a longer `alert wait` polls in runner attempts of at most 25 s. Other backends' `get`, and every backend's `accept` and `dismiss`, run their own fixed budgets and ignore it. For `alert`, the runner's main-thread watchdog is that same budget. An `alert get` it cuts off reports `reason: runner_main_thread_timeout` with `dispatched: unknown`: it does not mean there is no alert, which is `alert-not-found`. The next `alert get` waits out the runner's `RUNNER_BUSY` refusal while the abandoned work drains, then runs; the resends are bounded, so if the work has not drained by then it still returns `RUNNER_BUSY` (`dispatched: no`) and you retry.
- On the snapshot-derived Android backends `alert get --json` answers with a typed `status`: `visible`, `none`, or `undecided` (an alert-shaped capture that resolves to no single alert). Branch on `status`, not on the message.
- On those backends, `alert accept` presses the lasting grant when a permission sheet offers several: Android orders its grants by the platform's own ids (full-access "Allow all", then foreground-only, then always, then the picker-limited "Select photos and videos", then one-time) whatever the button language — this is what makes the Android 14+ partial-media sheet answer "Allow all" — and elsewhere the English label table decides ("while using the app" over "only this time"). Android acts only on the grant ids it declares with a role and a rank; a sheet carrying an allow- or deny-shaped grant id it does not declare (a vendor fork's own button, or a newer Android grant) is refused for both `accept` and `dismiss` — `dismiss` answers with the Back key, the cancel that grants nothing. A sheet whose choices no rank orders — or that names the same choice at two points — is refused with `alert_button_undecided` and `dispatched: no`, never pressed by position; read the labels with `alert get` and press your choice by label/ref.
- On those backends, when `alert accept`/`dismiss` fails before pressing anything, `error.details.reason` names why (a capture or transport failure that is not a driver refusal may instead surface as its own error without a `reason`). A budget never cancels a running capture or press: it only ends the polling loop, so every check that started inside the budget finishes and counts. Before any press, `dispatched: no` and the reason is `alert-not-found` (the action budget ended with no alert on screen), `alert_capture_undecided`, or `alert_button_undecided` (the sheet is too ambiguous to act on). After a press, `dispatched` is absent and the reason is `alert-window-expired`; “did not dismiss the visible alert” means the latest post-press capture that answered named the same dialog again, “could not be confirmed” means the press went out but the latest read was unresolvable or never answered, so the dialog may already be gone. If you cancel the request at any point from the press onward — the tap in flight, or the post-press checks — the cancellation reports `dispatched: unknown`: the device may already have received the tap. The one exception is a cancel whose post-press check nevertheless finished and proved the alert gone; then the command reports the handled alert, because the outcome is known. Run `alert get` before acting again.
Expand Down
Loading