Repository navigation
Description
What happened
After text entry, a deterministic tap can use bounds captured before the keyboard moves the form. In an iOS simulator recording, a password-visibility control was resolved at y=298; the keyboard then moved it to about y=211, but the tap landed at y=298. The subsequent assertion failed.
Expected: after text entry, honor the remaining configured transition budget and resolve the next control again before dispatch.
The mobile engine records the pre-entry index after type/fill/clear. settled() skips its transition wait when the first post-entry snapshot has the same bounds. That snapshot can precede keyboard movement, so an unchanged first frame is not sufficient.
Separately, agent-device can refuse a moving target with reason=target_still_moving and dispatched=no. The engine currently classifies that as nonretryable ENGINE_FAILURE, even though re-observing is safe when no input was dispatched. Unknown dispatch status must remain nonretryable.
Minimal reproduction
// Deterministic regression on current main, with no device or model needed:
// https://github.com/shamilovtim/e2e/blob/codex/mobile-keyboard-transitions/packages/mobile/tests/unit/engine.test.ts
// Test: "reobserves after %s when the keyboard moves a control after an unchanged snapshot"
//
// 1. Observe a textbox and an existing button at position A.
// 2. Type, fill, or clear the textbox.
// 3. Return a first post-entry snapshot with the button still at A.
// 4. Return the next snapshot with the same button at B and a new ref.
// 5. Tap the button: main dispatches the old ref; the patch dispatches the new ref.
//
// Run from repo root:
// pnpm --filter e2e run build
// pnpm --filter @e2e-dev/mobile exec vitest run tests/unit/engine.test.ts tests/unit/errors.test.ts -t "reobserves after|retries a moving target"
//
// The real-device symptom is intermittent; a standalone public app reproduction
// has not been prepared. The regression uses the existing scripted client.Versions
e2e 0.21.2; @e2e-dev/mobile 0.12.3; agent-device 0.21.28; Node 24.22.0; iOS 27.0 simulator, macOS 27.0.1. Regression also reproduced against upstream main 93f77d8.
Engine
@e2e-dev/mobile
Anything else
Candidate fix: main...shamilovtim:e2e:codex/mobile-keyboard-transitions
The three delayed-layout cases and the no-dispatch motion refusal fail on unmodified main and pass with the patch. The full mobile package suite passes 349 tests. pnpm check passes.
Equivalent package patch was exercised in focused downstream iOS login and billing-eligibility journeys: each passed three times with zero test retries. One additional cold-start stress run encountered an unrelated native crash before credential entry, so this is not a claim that every run was green or that Android was verified.
Tradeoff: after type/fill/clear, the next deterministic action may wait the remainder of transition even when no keyboard animation actually occurs. No new timeout, retry loop, coordinate workaround, or Return-key action is introduced.
Activity
tadeumaia commentedon Oct 22, 2026
We hit the same no-dispatch refusal on iOS in two situations, so here is another real-world case for the second part of this issue.
After moving
@e2e-dev/mobilefrom 0.12.2 to 0.12.3 (agent-device 0.21.25 to 0.21.28), four of our iOS tests failed with:In agent-device 0.21.28 that is
reason: target_still_movingwithdispatched: no, from the branch where the first re-measure comes back unmeasured: for a ref target the measure needsadoptedFrom === 'observation', and the mobile engine always presses by ref. Each refusal took about 280 ms.screen.getByTestId("set-1-reps-field").first().tap()right after a digit typed on the iOS number pad (device.locator('role=Key label="0"').tap()). Three tests.tap button "Completar" (1 of 2), right aftertap button "Fechar"closed an inline sheet. One test. The same replay passed three times in another test of the same run.Both targets' names appear more than once on screen (once per exercise card), which may be a coincidence.
On 0.12.2, with the same app build, simulator and recordings, all four pass (6 of 6 and 3 of 3 in the two affected files), so we have pinned 0.12.2 for now. We have not tried #1297 against these.
Versions:
e2e0.21.2,@e2e-dev/mobile0.12.3 (agent-device 0.21.28), iPhone 17 simulator on iOS 26.5, Expo dev client (React Native, new architecture), macOS.