Skip to content

Read the screen without silencing a screen reader (dump backend) - #101

Merged
lana-20 merged 1 commit into
mainfrom
dump-reader
Oct 2, 2026
Merged

lana-20 merged 1 commit into
mainfrom
dump-reader

Conversation

@lana-20

@lana-20 lana-20 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

The defect (CHALLENGES 208). Android's test-automation connection suppresses every other accessibility service unless it is opened with FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES. Neither uiautomator dump nor the UiAutomator2 server opens it so — the server accepted disableSuppressAccessibilityService, as a setting and as a capability, and went on unbinding VoiceView; the setting did not read back. So every read silenced TalkBack or VoiceView for as long as it ran, and an app that publishes its contents only to a screen reader stopped publishing them before the read could see them.

The fix, for the dump backend. internal/mobiumdriver/reader is a reader of Mobium's own: Main.java, built to a ~8 KB reader.dex by make reader (d8 9.3.16 reproduces the pinned bytes), embedded and checked against its SHA-256 before every use. Whenever an accessibility service is enabled, a read pushes it into /data/local/tmp/mobium-reader, runs it with app_process, and deletes the folder — with the runtime's compiled copy inside — after every read, failed reads included. It writes what uiautomator dump writes, between marker lines, so a vendor library printing to stdout first (a Fire TV's does) cannot corrupt it. With no service enabled, nothing changes. MOBIUM_DUMP_READER=mobium|uiautomator forces one; the daemon reads it, so it applies from the daemon's start (SETUP, "With a screen reader on").

Measured, with the screen reader's binding checked during each read, not only before and after:

uiautomator dump Mobium's reader
Fire TV, Android 9, VoiceView unbound bound, every read
Prime Video on it 12 nodes, 0 labeled 14 nodes, 3 labeled
Pixel 8 Pro, Android 17, TalkBack unbound, both reads bound, both reads; 13 nodes either way

The first build died on Android 17 within 150 ms — newer accessibility code builds a Handler on the main looper, which a bare app_process lacks — and now has one. Reads take ~1.9 s on the TV and ~1.3 s on the Pixel, about what the dump takes.

Not in this PR: UiAutomator2, the default driver, still silences a screen reader (ROADMAP). Tests: reader_test.go (checksum, markers, errors) and dumpreader_test.go (chosen with a service on and not otherwise, cleanup after a failed read, the override checked). make ci passes.

Base automatically changed from gpm-not-now to main October 2, 2026 06:53
Android's test-automation connection suppresses every other accessibility
service unless it is opened with FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES,
and neither uiautomator dump nor the UiAutomator2 server opens it so — the
server accepted disableSuppressAccessibilityService, as a setting and as a
capability, and went on unbinding the screen reader. So every read silenced
TalkBack or VoiceView, and an app that publishes its contents only to a
screen reader stopped publishing them before the read saw them.

The dump backend now reads through a reader of Mobium's own whenever an
accessibility service is enabled: internal/mobiumdriver/reader, a small dex
pinned by checksum and rebuilt from Main.java by `make reader`, pushed into
a folder of its own, run with app_process and deleted, with the runtime's
compiled copy, after every read. It writes what uiautomator dump writes.
MOBIUM_DUMP_READER=mobium or =uiautomator chooses one regardless; the daemon
reads it, so it applies from the daemon's start.

Measured: on a Fire TV (Android 9) VoiceView stayed bound through every
read and Prime Video gave 14 nodes, 3 labeled, where the dump gave 12 and
none; on a Pixel 8 Pro (Android 17) uiautomator dump unbound TalkBack in
both reads and the reader in neither, 13 nodes either way. The first build
died on Android 17, which builds a Handler on the main looper during
connection; the reader now has one. With no service enabled nothing
changes. UiAutomator2 still silences a screen reader (ROADMAP).

CHALLENGES 208; SETUP, "With a screen reader on".
@lana-20
lana-20 merged commit 8d23948 into main Oct 2, 2026
5 checks passed
@lana-20
lana-20 deleted the dump-reader branch October 2, 2026 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant