Repository navigation
Closed
Labels
task:readySpec checked. An agent may implement it once its Depends on allow.Spec checked. An agent may implement it once its Depends on allow.
Description
Part of #1024.
Scope
A person on a leased iOS simulator's input socket types with hardware keys on every runtime, opens the app switcher and locks or wakes the screen. The website says so.
If it goes wrong, a key or button the person sends does nothing on the simulator, and nothing says so.
Technical spec
Commit c9d82e3.
Rows
- Behaviours: B1, B2, B3
- Interfaces: I1
- Existing code: X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15
Facts
| # | Fact | Source |
|---|---|---|
| F1 | The Swift test target depends on the simlock-screen-capture executable target, so a test can reach its internal functions |
vendor/screen-capture/Package.swift (SimlockCaptureTests dependencies) |
| F2 | IndigoHIDMessageForButton(source, direction, target) with direction 1 down and 2 up, target 0x33 |
vendor/screen-capture/Sources/simlock-screen-capture/HIDInjector.swift:238-263 |
| F3 | Measured 2026-11-17: Lock (source 1) locked iOS 26.4, 27.0 and 27.2 simulators; a second press woke 26.4 and 27.2; two Home presses (source 0) 0.05–0.2 s apart opened the app switcher; Indigo keys typed on all three runtimes | ADR 0085, Context table |
| F4 | The slow lane already opens a lease's input socket, taps through it and reads the result with an agent-device snapshot | e2e/slow-agent-device.test.ts:250-280 |
| F5 | At agent-device 0.21.36 a snapshot brings the session's app to the front when it is in the background; appstate and screenshot bring nothing forward |
agent-device RunnerTests+Models.swift:153-158, 174-180, 293-302 (spec check of #1659) |
Modules touched
Searches: keyboardUnavailable, app_switcher, keysAvailable, buttonSource, tagOrientation over src, vendor/screen-capture, website, docs/internal.
src/drivers/ios/input/(device input, its test, its manual)vendor/screen-capture/(InputMode.swift,HIDInjector.swift, a new press plan file and its Swift test,NOTICE.md,Package.swiftheader comment)e2e/slow-ios-input.test.ts(new slow-lane test) and its row indocs/internal/agent-rules/toolchain.mdwebsite/src/content/docs/docs/using-devices/watching-the-screen.mdxdocs/internal/adr/0049-…mdstatus line,docs/internal/adr/README.mdrow for 0049
Rules in play
- ADR 0085 §1–4: the decision this task builds.
- ADR 0049 §4: narrowed by 0085; its status line says so (X12), as it does for 0052.
vendor/screen-capture/NOTICE.md: every change to aMODIFIED FILEkeeps its header and the deviation text true (X9, X15).src/drivers/ios/input/AGENTS.md: Home has one owner,simctlin the driver; the helper never sends Home on its own, only as part of the app switcher plan.- Documentation rule 9: the input socket is public surface, so the website page changes in this PR (X10).
Retired words
keyboardUnavailableon an iOS simulator: no device sets it; the field stays in core's0x82shape only.
Tests
The seam for B1 and B2 is createIosDeviceInput with the helper faked, modelled on src/drivers/ios/input/device-input.test.ts; B3 is a Swift test modelled on vendor/screen-capture/Tests/SimlockCaptureTests/PanelTouchTests.swift.
- B1: a socket client sends a key on an iOS simulator: runtime iOS 26.4; iOS 27.0; iOS 28.1; unreadable id
garbage; the config after a foldable's size change has nokeyboardUnavailable - B2: a socket client presses a button on an iOS simulator:
app_switcher;lock;home;back - B3: the helper's press plan for a button name:
lock;app_switcher;back;siri; empty name
Done when
pnpm testpasses, with B1 and B2 insrc/drivers/ios/input/device-input.test.ts.swift test --package-path vendor/screen-capturepasses, with B3.- On a real simulator, the new slow-lane test
e2e/slow-ios-input.test.tspasses: on a leased iOS 27 simulator withcapabilities.input.enabled, throughWS /v1/leases/{id}/input(helpere2e/helpers/input-socket.ts): with Settings (com.apple.Preferences) open in an agent-device session and its search field focused, keys sent on the socket appear in an agent-devicesnapshot;app_switcherchanges the screen; Settings is reopened with agent-deviceopen; thenlockchanges the screen again. Each is read as more than 30% of the pixels changed betweenxcrun simctl io <udid> screenshotbefore and after the press, because agent-deviceappstatestill reports Settings in front behind the app switcher and itsscreenshothangs there (decided in delivery, pending the maintainer). This proves each press changed the screen, not which screen came up; the screenshots are kept for a reader. Model: the input socket and snapshot ine2e/slow-agent-device.test.ts:250-280. docs/internal/agent-rules/toolchain.mdhas a slow-lane machine row fore2e/slow-ios-input.test.ts(an iOS 27 runtime andSIMLOCK_E2E_AGENT_DEVICE), modelled on the iOS foldable row.- The website page
watching-the-screen.mdx, section "On an iOS simulator", says keys work on every runtime,app_switcheropens the app switcher,locklocks or wakes the screen, andbackis dropped (X10).
Out of scope
- Home through the helper; Back on iOS; physical iPhones; CLI or MCP button commands.
Depends on
Written by an agent.
Metadata
Metadata
Assignees
Labels
task:readySpec checked. An agent may implement it once its Depends on allow.Spec checked. An agent may implement it once its Depends on allow.
Activity
test: a socket client sends a key on an iOS simulator of any runtime (#…
feat(ios): keys reach the input helper on every runtime (#1659)
test: the helper's press plan for a button name (#1659)
feat(ios): the helper presses Lock and the app switcher (#1659)
test(e2e): the input socket types, opens the app switcher and locks o…
docs: NOTICE row and input manual say what the helper keeps (#1659)
fix(ios): the helper logs a button it has no plan for; manual stops r…
test(e2e): the iOS input test names its agent-device session, taps th…
test(e2e): the iOS input test no longer waits for a warm replacement …
docs(adr): 0085 to Accepted (#1659)
docs(adr): 0085, an iOS simulator takes keys on every runtime and the…
feat(ios): iOS simulator input takes keys on every runtime, the app s…