Repository navigation
fix(mobile): revalidate input bindings before fill and clear - #1410
Conversation
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
|
@cubic-dev-ai review this PR |
@bheemreddy-samsara I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Re-trigger cubic
|
@cubic-dev-ai review this PR |
@bheemreddy-samsara I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Re-trigger cubic
|
@cubic-dev-ai review this PR |
@bheemreddy-samsara I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 7 files
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Re-trigger cubic
There was a problem hiding this comment.
No issues found across 7 files
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Re-trigger cubic
|
@okwasniewski Can you give a quick look ? |
okwasniewski
left a comment
There was a problem hiding this comment.
Thanks for the issue and the clear scripted repro. I confirmed the gap on main 1a7a331. If the observation's capture outlasts the transition budget, settled() returns early (surface.ts:1199-1200) and fill goes out with no fresh snapshot. Your freshness tests catch it: with surface.ts reverted to main, all 26 in input freshness fail. pnpm check and the 377 mobile unit tests pass on 8d56213.
The bug is real, but the fix is broader than it needs to be, and it breaks cases that work today. I checked each one below with a scratch test against the scripted client, running this head and then main's surface.ts.
Must fix
-
Identical fields can no longer be filled.
uniqueInput(surface.ts:1151) plus!entry.uniqueInput(:1233) means any input that shares role/testId/name/text/placeholder with another one always getsNODE_STALE. Re-observing does not help, so the retry loops until timeout. Repro: two text fields labelledDigitwith no identifier (an OTP row). This head givesNODE_STALEon 3 of 3 attempts. Main fills@e20on the first try. e2e has to work on any user app, and "use distinct test ids" is not an option for apps users don't control.refind()already handles duplicates by picking the nearest one. -
The rect equality check breaks the transition path for inputs.
freshInputcompares the fresh rect with the originalentry(:1234), not with the nodesettled()just relocated. A field that arrived with the last action and was first seen mid-slide (the Android casesettled()exists for) now always fails once. Repro, adapted from the existing "acts on a control that came with the last action..." test with a text field: main fills the landed@e42with 1 capture, this head throwsNODE_STALEafter 2 captures. A moved field with the same identity and the same security facts is still the same sink, so geometry change should not be an error. Following the fresh binding is the fix, and that is how the reported bug actually gets fixed. Rect comparison is a heuristic we would rather not add. -
Every input action in the budget window now takes two captures.
settled()->relocated()takes one, thenfreshInputtakes another and throws away the first result (:1289-1293). I counted 2 captures per fill on this head and 1 on main.
Suggested shape, which deletes most of the diff. For fill/clear, skip the elapsed-budget shortcut and always relocate: one fresh snapshot, refind(), act on the fresh ref. After #1300 lands, relocated() already throws a retryable NODE_STALE when the target is gone. On top of that, keep only the check your secret-authorization argument needs: NODE_STALE when secure / inputPurpose / disabled / hidden changed, or when the fresh tree is truncated. That removes uniqueInput from NodeBinding, sameInputIdentity (it duplicates refind's predicate), freshInput, and the rect check.
isInputRole(surface.ts:240) duplicatesTEXT_INPUT_ROLESinnodes.ts:97. Export and reuse that one.
Nits
packages/e2e/tests/integration/agent-input-stale.test.tsexercises the existing coreNODE_STALEretry through a fake engine that throws on its own. This PR doesn't changepackages/e2e/src, so the test passes on main and can't fail for this change. It's fine to keep as a guard, but it isn't evidence for the fix.- The
docs/reference/mobile.mdx:240-252paragraph and its skill copy list internal comparison fields. Once 1-3 are done, one sentence is enough: "fillandclearre-read the field before typing; if it disappeared or its secure state changed, the action retries after a fresh observation." The changeset can be shortened the same way. - The non-input
UNSUPPORTED_CAPABILITYguard (:1277) is fine and matches Playwright's fill. Consider sharing it with thefocusrole check at:1300instead of adding a second one.
Overlap: #1300 (strict relocated(), Ready) should merge first, and this PR should then build on it. #1297 changes the settled() skip rule as well. Rebase after both.
This was an automated review.
|
I'll wait for #1300 and #1297 to land, then rebase this PR as requested. The requested fixes are still pending: use one fresh relocation for fill/clear, remove the uniqueness and rect-equality guards so identical and sliding inputs keep working, and retain the security-fact/truncated-tree checks. I'll also reuse |
|
Implemented the independent code fixes in c6f49aa. Fill/clear now use one fresh relocation, including after the elapsed-budget shortcut, and follow
#1300 and #1297 are still open. Their changes have not been copied into this branch; the final rebase remains pending until both land. |
There was a problem hiding this comment.
All reported issues were addressed across 7 files (changes from recent commits).
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
View guided diff | Re-trigger cubic
|
Thanks, this is a lot smaller now and it fixes #1409 the right way. I re-checked c6f49aa (2dbda1c only changes docs). Earlier asks
Cubic on c6f49aa
Nits
Rebase: as you said, this conflicts in This was an automated review. |
Asks addressed at c6f49aa, see re-check comment. Automated review.
|
Addressed both cleanup nits in 7d71094. Removed the implementation-level coordinate-race wording from the docs and consumer skill, including the generated plugin copy. Added fill/clear cancellation guards to keep a cancelled transition wait from capturing or dispatching input. The final rebase remains pending on #1300, #1297, and #1413. |
|
@okwasniewski Can you give a another look ? |
|
Re-check at This was an automated review. |
# Conflicts: # packages/mobile/src/surface.ts
An empty Compose Multiplatform field on iOS reports no placeholder; it appends the text it draws to its label, so "Name, Ada Lovelace" read as a field holding a name and the examples nightly's agent tried to clear it. Read that trailing text as the field's placeholder when it is the field's only child. It applies once the field reads editable: agent-device's XCTest tree reports it so today, its simulator AX tree once callstack/agent-device types text-entry-trait elements (follow-up bump). #1410 refused fill and clear on anything but a text input role, which broke the Compose Multiplatform and Flutter iOS examples: agent-device 0.21.33's simulator AX tree reports their fields as an unclassified Other. Type into an Other that holds nothing but text; a container of controls stays refused.
What and why
fillandclearalways dispatch through one fresh mobile snapshot andrefind(), even when capture latency consumed the transition budget. Existing transition waits and nearest-node matching remain in use.NODE_STALEbefore mutation so existing retries re-observe and rerun secret authorization.TEXT_INPUT_ROLES; keep the non-input refusal and shorten the docs, skill, and changeset. No dependency or agent-device changes. Replacement/empty-clear behavior is separate.Issue
Fixes #1409
Workflow
flowchart LR A[Resolved target] --> R{Input role?} R -- no --> U[UNSUPPORTED_CAPABILITY] R -- yes --> B[Existing transition wait when needed] B --> C[One fresh snapshot and refind] C --> D{Target present and security facts unchanged?} D -- yes --> E[Fill or clear with fresh binding] D -- no --> F[NODE_STALE before mutation] F --> G[Existing re-observation and authorization retry] G --> CVerified
Ran it locally: yes, scripted mobile command stream and built packages, not a device.
Validation head:
7d710946a1d7175e5742b4167a97dec59aa97f6c.1a7a331bsettled()signature and separates waiting from the single input relocation.Active blockers and limits
11cffff2d) after fix(mobile): reobserve targets after keyboard transitions #1297 landed: fill/clear keep one input relocation,waitForTransition()keeps fix(mobile): reobserve targets after keyboard transitions #1297's keyboard-visibility check, andbind()takes the projected screen. Mobile typecheck, 408 mobile unit tests, the stale-input integration test, lint, and dead-code pass locally; CI is green on that head. fix(mobile): reject stale refs after transition refresh #1300 and fix(mobile): pin each action's ref to the snapshot it came from #1413 are still open and do not block this merge.