Skip to content

fix(android): read supplemental description so WebView 150+ inputs stay matchable - #3498

Open
Timbological wants to merge 2 commits into
mobile-dev-inc:mainfrom
Timbological:fix/webview-150-supplemental-description
Open

Timbological wants to merge 2 commits into
mobile-dev-inc:mainfrom
Timbological:fix/webview-150-supplemental-description

Conversation

@Timbological

Copy link
Copy Markdown

Proposed changes

Fixes the WebView 150+ regression reported in #3497: every text input inside a WebView is
unmatchable by name
, so tapOn/assertVisible on a field's placeholder, aria-label or
<label for> fails with Element not found for an element that is plainly visible and correctly
labelled.

Cause is upstream and intentional. Chromium M150 flipped one flag's default in
content/public/common/content_features.cc:

BrowserAccessibilityAndroid::GetAndroidHint() is byte-identical between the two builds and is
gated on that flag (browser_accessibility_android.cc#L1139):

// TODO(accessibility): Remove this path once we roll out supplemental descriptions.
if (!base::FeatureList::IsEnabled(
        features::kAccessibilityPopulateSupplementalDescriptionApi)) {
  // If we're returning the value as the main text, the name needs to be part of the hint.
  if (std::u16string name = GetNameAsString16(); !name.empty()) {
    strings.push_back(name);
  }
}

With the flag on, the name is no longer folded into hintText; it goes to
AccessibilityNodeInfoCompat.setSupplementalDescription() instead — natively on API 36, in the
node's extras below that. A text input's name lives only there, which is why inputs break and
buttons don't (their names ride on text/contentDescription). Screen readers are unaffected, so
this is specific to consumers reading raw AccessibilityNodeInfo fields.

Four edits, because a device-side change alone is silently dropped:

  1. AccessibilityNodeInfoExt.kt — read the value, native getter on API 36+ with the AndroidX
    extras key as the pre-36 fallback.
  2. ViewHierarchy.kt — serialise it as a supplementalDescription attribute, beside hintText.
  3. AndroidDriver.kt — carry it through mapHierarchy, whose attribute allowlist otherwise drops
    anything new even though the device emitted it.
  4. Filters.kt — add it to the textMatches/hintTextMatches/accessibilityTextMatches union,
    guarded to fire only when text, hintText and accessibilityText are all empty. AndroidX
    documents a supplemental description as "purely supplemental", so it must never outrank a real
    name; the guard keeps that honest while making nameless-everywhere-else inputs matchable.

I've left the tracked maestro-app.apk/maestro-server.apk alone, since update-drivers.yaml
rebuilds them on main with the canonical toolchain after merge — happy to commit rebuilt
artifacts instead if you'd rather.

Testing

Bisected with a single unchanged APK on one device, swapping only the WebView provider, so the
WebView version is the only variable:

WebView tapOn by label
124.0.6367.219 pass
133.0.6943.137 pass
145.0.7632.45 pass
147.0.7727.55 pass
149.0.7827.5 pass
150.0.7871.46 fail
152.0.7977.30 (beta) fail

Upstream checks: ./gradlew test detekt and ./gradlew detektMain are BUILD SUCCESSFUL on
this branch (all module unit tests plus :maestro-test integration tests).

A/B/A control at the identical commit. Control CLI built from a git worktree at the same
commit with a clean git status, so the only difference is maestro-client.jar (which carries both
the client change and the bundled driver APK). Verified unpatched: 0 of its dex files contain
supplementalDescription, against exactly 1 (classes5.dex) in this build. Both report 2.8.0.
Same emulator (API 35, WebView 150.0.7871.46), same app, same flow:

A: stock   -> Tap on "Search"... FAILED      (Element not found: Text matching regex: Search)
B: patched -> Tap on "Search"... COMPLETED
A: stock   -> Tap on "Search"... FAILED

Because the run was on API 35, the value came from the AndroidX extras Bundle rather than the
native API-36 getter — the compatibility fallback is exercised.

Probes (bare HTML, no framework) — every labelling mechanism, nameless on 150 before this change:

{'supplementalDescription': 'Alpha',   'resource-id': 'probe-alpha'}     # placeholder
{'supplementalDescription': 'Bravo',   'resource-id': 'probe-bravo'}     # aria-label
{'supplementalDescription': 'Charlie', 'resource-id': 'probe-charlie'}   # <label for>
{'supplementalDescription': 'Delta',   'resource-id': 'probe-delta'}     # framework text field
{'accessibilityText':       'Foxtrot', 'resource-id': 'probe-foxtrot'}   # button, unchanged
{'text':                    'Golf',    'resource-id': 'probe-golf'}      # button, unchanged

Suite-level regression check. Ten flows from a real suite, each run under both CLIs back to back
on the same WebView-150 emulator. Six touch the search field by name; four never do (controls).
Recording the first failing step rather than just pass/fail, because some flows in that suite are
independently red on stale assertions — under stock they die at the search field before reaching
them.

Flow Stock This branch
search-filters FAIL Tap on "Search" PASS fixed
back-navigation FAIL Assert "Search" visible PASS fixed
home FAIL Assert "Search" visible FAIL Assert "Select a Region" visible advanced past the field
navigation FAIL Assert "Search" visible FAIL Assert "Select a Region" visible advanced past the field
dark-theme FAIL Assert "Search" visible FAIL Assert "Select a Region" visible advanced past the field
orientation FAIL Assert "Search" visible FAIL Assert "Select a Region" visible advanced past the field
about (control) PASS PASS identical
deep-link-offline (control) PASS PASS identical
deep-link-qr (control) FAIL Assert "Spanish" visible FAIL Assert "Spanish" visible identical
deep-link-legacy-resume (control) FAIL Assert "Spanish" visible FAIL Assert "Spanish" visible identical

No flow regressed: two went red → green, four advanced from the search field to a later
pre-existing failure of that suite's own making (Select a Region is stale text the app no longer
renders), and all four controls are identical — including the two already failing for their own
reasons.

Blast radius is small, and the change restores rather than invents matches. On a real screen of
279 nodes only 2 carry a supplemental description and just 1 becomes newly matchable — the
android.webkit.WebView root, whose name is the document title. Stock Maestro on WebView 145
already reported it as {'text': '5fish', 'class': 'android.webkit.WebView'}; on 150 it is
nameless; with this change it is matchable again. (Worth noting the regression isn't limited to
inputs: the WebView root's title moved too, so assertVisible: "<page title>" also broke silently
on 150.) Pre-150 the change is inert — across 3 screens and 221 nodes on WebView 145, zero nodes
carry a supplemental description. No measurable cost: 3130 ms per hierarchy dump on this branch
vs 3211 ms stock (mean of 3, same 279-node screen).

iOS and web are unaffected despite Filters.kt being shared — IOSDriver populates hintText
from placeholderValue and nothing sets supplementalDescription, so the new branch never fires.

On e2e tests: I haven't added one, because reproducing this needs a device with WebView 150+
and the Android CI image pins its WebView (a google_apis image has no Play Store, so it stays at
whatever the image bundles — 133 for android-36). That pinning is also why this regression is
invisible to CI while every developer device auto-updates past 150 and fails. Glad to add an
e2e/demo_app screen and flow if you can point me at a runner where the WebView version can be
raised.

Issues fixed

Refs #3497. Same underlying cause as #3435, which was closed as not_planned on the premise that
the name isn't in the tree — it is, it just moved to a field the driver doesn't read. (That
reporter's own data had the answer: WebView 150.0.7871.46 failing vs 149.0.7827.160 working;
they discarded the version hypothesis because the OS versions ran the other way.)

@Fishbowler

Copy link
Copy Markdown
Contributor

Can you reduce kdoc? It's a little much.

@Timbological

Copy link
Copy Markdown
Author

Trimmed — thanks. The Chromium flag/call-site detail is all in #3497, so I've cut it from the code and left one line at each site plus the reason for the empty-name guard in Filters.kt (that one's load-bearing: without it the precedence rule looks arbitrary and invites someone to simplify it away). 20 comment lines down to 4. ./gradlew test detekt still green.

@Timbological

This comment was marked as outdated.

@Fishbowler
Fishbowler force-pushed the fix/webview-150-supplemental-description branch 2 times, most recently from 3e80085 to f89516f Compare August 21, 2026 14:59
Comment on lines +94 to +99
// Only when the node has no other name: a supplemental description is documented
// as purely supplemental, but on WebView 150+ an input's name arrives only here.
if (!it.attributes["text"].isNullOrEmpty()
|| !it.attributes["hintText"].isNullOrEmpty()
|| !it.attributes["accessibilityText"].isNullOrEmpty()
) return@filter false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? This check should be inert, since we expect the others to be empty. But if it weren't, wouldn't a user expect matching to be simpler, rather than special cased?

@Fishbowler
Fishbowler force-pushed the fix/webview-150-supplemental-description branch from 121e4d1 to 84f49c8 Compare August 25, 2026 08:17
@Fishbowler
Fishbowler force-pushed the fix/webview-150-supplemental-description branch 2 times, most recently from 92c4006 to 6998f5c Compare September 15, 2026 16:25
…ay matchable

From WebView 150 (Chromium M150) the accessible name of a web text input is no
longer folded into hintText: kAccessibilityPopulateSupplementalDescriptionApi
became enabled by default, which skips the branch of
BrowserAccessibilityAndroid::GetAndroidHint() that used to push the name into
the hint. Chromium delivers it through
AccessibilityNodeInfoCompat.setSupplementalDescription() instead — natively on
API 36, and in the node's extras below that.

The Android driver reads only text, hintText and contentDescription, so every
WebView <input> now arrives nameless and tapOn/assertVisible on its placeholder,
aria-label or <label for> fails with "Element not found" for a field that is
plainly visible and correctly labelled. Buttons are unaffected, since their
names ride on text/contentDescription. Bisected across six WebView builds with a
single unchanged APK: 124, 133, 145, 147 and 149 all pass; 150.0.7871.46 and
152.0.7977.30 fail.

Serialise the value as a supplementalDescription attribute, carry it through
mapHierarchy's attribute allowlist, and consult it in textMatches only when
text, hintText and accessibilityText are all empty — a supplemental description
is documented as purely supplemental, so it must not outrank a real name, but on
WebView 150+ an input's name exists nowhere else.

Refs mobile-dev-inc#3497
Cut the Chromium flag and call-site detail from the KDoc and inline comments —
that archaeology lives in the issue. What stays is what a reader needs in place:
WebView 150+ puts a web text input's name here instead of in hintText, and why
the match consults it only when no other name is present.

Refs mobile-dev-inc#3497
@Fishbowler
Fishbowler force-pushed the fix/webview-150-supplemental-description branch from 6998f5c to 1311b11 Compare October 16, 2026 16:23
@Fishbowler Fishbowler added the awaiting maestro 3.0 Driver work: on hold until after Maestro 3.0 label Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting maestro 3.0 Driver work: on hold until after Maestro 3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants