Skip to content

Enable pinch-to-zoom in Hermes WebUI - #111

Merged
nesquena-hermes merged 10 commits into
hermes-webui:mainfrom
sacgsxr:feature/pinch-to-zoom
Sep 17, 2026
Merged

nesquena-hermes merged 10 commits into
hermes-webui:mainfrom
sacgsxr:feature/pinch-to-zoom

Conversation

@sacgsxr

@sacgsxr sacgsxr commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • enable native WebView zoom gestures while keeping the deprecated on-screen zoom controls hidden
  • override Hermes WebUI’s restrictive maximum-scale=1, user-scalable=no viewport directives on the trusted WebUI origin
  • apply the viewport shim both at document start and through the existing runtime fallback path
  • document the behavior and add a unit contract for the injected script

Why

Hermes WebUI currently disables browser scaling in its viewport metadata. On Android this prevents users from pinch-zooming chat content, which is useful for accessibility and for inspecting dense content on a phone.

The script remains scoped to the configured Hermes WebUI origin; OAuth/provider pages are not modified.

Verification

  • ./gradlew --no-daemon testDebugUnitTest lintDebug assembleDebug
  • python3 -m unittest discover -s tools/tests -p "test_*.py" -v
  • python3 tools/check_markdown.py
  • python3 tools/extract_webui_scripts.py
  • npx --yes eslint@^10 --no-config-lookup -c eslint.runtime-guard.config.mjs "build/webui-scripts/**/*.js"

@nesquena-hermes

Copy link
Copy Markdown
Contributor

Review at da7751e43 — clean, correctly scoped, one runtime-path hardening before merge

Thanks @sacgsxr — this is a well-built accessibility PR. The WebSettings zoom enablement (setSupportZoom(true) / builtInZoomControls=true / displayZoomControls=false, so no deprecated on-screen +/- buttons) plus the document-start viewport-override shim is the right approach, the test is meaningful, and CI is green across lint + both instrumentation matrices.

I reviewed it end-to-end and ran it through our senior-advisor gate (Codex), which executed the shim in a sandbox rather than just reading it. Two things verified positively:

  • The directive rewrite is correct. Running pinchZoomScript against width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no produces width=device-width, initial-scale=1, maximum-scale=5, user-scalable=yeswidth=device-width and initial-scale are preserved, only the scale-locking directives are replaced. The MutationObserver fires and disconnects exactly once.
  • The document-start injection path is properly origin-scoped. installHermesWebUiDocumentStartFixes wires pinchZoomScript through addDocumentStartScript(view, originRule, …), so on the primary path it only ever runs on the configured Hermes WebUI origin — provider/OAuth pages are untouched.

One finding to close before merge — execution-time origin guard on the runtime fallback

MainActivity.kt:1631 (applyHermesWebUiRuntimeScripts) is the runtime fallback path. It's gated by a synchronous matchesConfiguredWebUiRoute(effectiveUrl) check (MainActivity.kt:1611), but evaluateJavascript runs asynchronously. During a Hermes→OAuth navigation, the route check can pass against a stale Hermes URL while the page that actually executes the script has already become the provider page — so pinchZoomScript (and the other runtime scripts) could rewrite a provider page's viewport, which violates the stated "scoped to the Hermes WebUI origin" contract.

To be fair: this async-gap property predates this PR and is shared by every script in applyHermesWebUiRuntimeScripts (viewport-fix, mic-fallback, clarify-autofocus, notification-bridge, route-recovery). The incremental risk from pinch-zoom specifically is low (viewport zoom is cosmetic, not a data/security boundary). But since this PR adds another script to that path, the clean fix is a real execution-time origin check rather than trusting the synchronous route check:

  • Have the runtime scripts self-guard at execution time, e.g. only apply when window.location.origin === <configured Hermes origin>. There's already a builder-pattern precedent in this file for templating a value into an injected script (buildHermesWebUiNotificationBridgeScript() / buildHermesWebUiRouteRecoveryScript()), so pinchZoomScript could become a small builder that bakes the trusted origin into a leading if (window.location.origin !== '<origin>') return; guard.
  • Add an instrumentation test for the stale-Hermes-callback → current-provider-page scenario, confirming the runtime path does not mutate the provider page's viewport.

I'm handing this back rather than pushing the fix myself because it's a navigation-timing path that needs on-device/instrumentation verification (the async race), which I can't exercise from here — you're better positioned to validate it with the WebView instrumentation gate. Everything else is merge-ready; once the runtime guard lands and the gate is green, this is good to go.

Gate: Codex (GPT-5.6-sol) — SHIP WITH THE HIGH-VALUE FIXES (1). Reviewed at exact head da7751e437d77d1df01bfd25ea128cbb870aab50.

@sacgsxr

sacgsxr commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in d1e4704.

I added an execution-time origin guard around every script in applyHermesWebUiRuntimeScripts, using the configured Hermes origin serialized into the injected JavaScript. This closes the async gap for pinch zoom as well as the pre-existing viewport, mic fallback, clarify autofocus, notification bridge, and route-recovery scripts.

I also added coverage for:

  • origin serialization and allow/block behavior at execution time; and
  • the stale-Hermes-callback → current-provider-page instrumentation scenario, verifying the provider viewport remains unchanged.

The updated CI run is fully green, including unit tests, lint, debug APK build, injected-JS checks, and Android instrumentation on API 35 and API 36:
https://github.com/hermes-webui/hermes-android/actions/runs/34511679934

nesquena-hermes and others added 8 commits September 15, 2026 22:23
…tructor

The runtime-script guard resolved its trusted origin with `new URL(...)`, but
`URL` is a page-controlled global. A foreign origin can replace it before the
asynchronously-evaluated script runs, return its own origin from the
constructor, and satisfy the check — so the guarded payload executes off-origin,
which is the exact hole the guard was added to close. Reproduced in a sandbox:
with `window.URL` shadowed, a hostile page ran the guarded payload.

Compare `window.location.origin` against a natively-canonicalized string literal
instead, so nothing the page controls participates in the decision.

The literal must match what a browser reports, which is NOT the WebViewCompat
allow-rule: `documentStartOriginRule` keeps an explicitly-specified default port
(`http://host:80`) while the browser drops it. Add `UrlOrigins.pageOrigin` for
the browser-shaped origin and use it at the guard's call site.

Tests: an instrumentation case that replaces `window.URL` on a provider page and
asserts the viewport is untouched; unit cases covering default-port dropping,
non-default ports, IPv6 bracketing, host lowercasing, and non-web scheme
rejection. Mutation-verified — reverting the guard to the `new URL(...)` shape
makes the bypass case regress.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
The literal the origin guard compares against must be byte-identical to what the
page reports in `window.location.origin`, or the guard never matches and every
runtime shim is silently suppressed for that server.

`java.net.URI` does not apply WHATWG host parsing, so two configurable forms
diverged: a numeric host (`http://2130706433` and `http://0x7f000001`, which
browsers resolve to `http://127.0.0.1`) and an expanded IPv6 literal
(`http://[0:0:0:0:0:0:0:1]`, which browsers compress to `http://[::1]`).

Canonicalize the host natively before building the literal: WHATWG numeric-IPv4
parsing (decimal/octal/hex, with a short last part filling the remaining bytes)
and RFC 5952 IPv6 compression. A host we cannot canonicalize confidently returns
null, so the caller skips injection rather than emitting an unmatchable literal.

IPv6 compression is deliberately pure string handling rather than InetAddress:
this runs on the main thread during script injection and must never risk a
name-resolution call.

Verified by porting the implementation to JS and differential-testing it against
the platform's own WHATWG URL parser: 21 hand-picked cases plus 2600 fuzzed
inputs (random IPv6 in exploded and compressed spellings, random IPv4 in
dotted/integer/hex forms, DNS names, default and non-default ports) — zero
divergences. Rejected hosts also match: every input we return null for is one
the browser's URL parser throws on.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
The assertion dropped the `:9000` port when transcribing from the differential
oracle, so it expected `http://[2001:db8::1428:57ab]` where both a real browser
and the implementation produce `http://[2001:db8::1428:57ab]:9000`. The test was
wrong, not the code — CI caught it (169/170 passing).

Re-checked EVERY pageOrigin assertion in this file against the platform's own
WHATWG URL parser; all 13 expected values now agree with the browser, and each
null-case is one the browser's parser also rejects.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
Differential-testing the canonicalizer against a real headless Chromium (not
Node's URL) surfaced four remaining divergences. Each would emit a literal the
guard can never match, silently suppressing every runtime shim:

- `http://0x` — an empty hex payload is zero, so Chromium yields `http://0.0.0.0`.
  A bare `0` prefix is likewise octal-with-empty-payload.
- `http://2130706433.` / `http://127.0.0.1.` — a NUMERIC host drops one trailing
  dot. A DNS name does NOT: Chromium keeps `http://hermes.example.com.` verbatim,
  so the two cases are handled separately.
- `http://[::ffff:127.0.0.010]` — the embedded dotted-quad uses the same
  radix-aware part parsing as a bare IPv4 host, so octal `010` is 8
  (`…:7f00:8`), not decimal 10 (`…:7f00:a`). Chromium also accepts components
  longer than three characters when they carry leading zeroes.
- `http://999.1.1.1`, `http://1.2.3.4.5` — these are numeric candidates the
  browser REJECTS, not DNS names. Previously they fell through and returned the
  raw host. They now fail closed via an INVALID_NUMERIC_HOST sentinel so the
  caller skips injection.

Also adds the mutation-coverage case the reviewer asked for: `[1:2:3:4:5:6:7:8:9]`
is admitted by java.net.URI, so it actually reaches canonicalBrowserHost and the
assertion fails if canonicalization is reduced to `return host`.

Verified against real Chromium over CDP: 2627 probes (hand-picked edge cases plus
fuzzed IPv6 in exploded/compressed spellings and IPv4 in dotted/integer/hex forms)
with zero divergences, and all 23 value assertions in UrlPolicyTest re-checked
against Chromium's own output.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
Running the compiled canonicalizer against a real headless Chromium surfaced a
class java.net.URI silently drops: it is RFC 2396-strict and returns a null host
(and port -1) for spellings a browser accepts — a trailing-dot IPv4
(`127.0.0.1.`, `2130706433.:8080`) and `0x.0x.0x.0x`. Those hit `?: return null`
and disabled every runtime shim for such a configured server.

Add a raw-authority fallback used only when URI yields no host: read the host
(and port) straight from the authority substring, stripping userinfo to match a
browser's origin (`user:pass@host` → `host`), then run the same WHATWG
canonicalization. Anything unreadable still fails closed.

Verification tightened to remove the CI round-trips that caught my earlier test
typos: the production Kotlin is now compiled locally (kotlinc + JDK 21) and run
against real Chromium over CDP across 2750 probes with ZERO divergences and zero
wrong-value results (every remaining difference is a fail-closed null on a host
Chromium also rejects). All 37 assertions in UrlPolicyTest were generated from
the compiled Kotlin's own output, so the suite matches the implementation which
matches the browser.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
…fallback

Two ways the raw-authority fallback could synthesize a literal that disagrees
with a browser's location.origin (both found by attacking the compiled Kotlin
against real Chromium):

- Ports were parsed with toIntOrNull, which accepts signs and overflows.
  `:-1`, `:+80`, `:65536`, `:8_7` all produced a valid-looking origin where
  Chromium rejects the URL. Parse ports as ASCII-digits-only in 0..65535 on the
  fallback path AND range-check uri.port on the normal branch; fail closed
  otherwise.
- The fallback passed a host through verbatim, but it only exists to recover the
  numeric/ASCII hosts java.net.URI wrongly rejects — it does not implement WHATWG
  percent-decoding or IDNA. `foo%2ebar` and `münchen.de` would emit an
  un-decoded/un-punycoded literal that never matches the browser's origin. The
  fallback now returns null on any host carrying `%` or a non-ASCII character.
  (A real self-hosted server URL is an IP or an ASCII hostname, both of which
  java.net.URI already accepts on the normal path — so this loses no legitimate
  case; it only refuses to guess where a browser would decode.)

Every residual difference from a browser is now a fail-closed null (skip
injection), never a wrong literal. Verified against real Chromium via the
compiled production Kotlin across 2770 probes (base + fuzz + adversarial
authorities + IDNA/percent/port edge cases): zero wrong-value results. All 45
UrlPolicyTest assertions were regenerated from the compiled Kotlin's own output.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
…losed

The canonicalizer treated a host as a DNS name whenever IPv4 parsing failed, so
`http://foo.1`, `http://example.99`, `http://09`, `http://1..2.3`, and
`http://1.2.3.09` were passed through verbatim — but a browser REJECTS all of
them, so the emitted literal never matched location.origin (a wrong-value, not a
fail-closed null).

WHATWG's rule: if a host's last label ends in a number, the whole host MUST parse
as a valid IPv4 address or the host is invalid. Implement that directly — detect
"ends in a number" first (last label all-digits or a parseable IPv4 part), and if
so, any parse/range/empty-label failure returns INVALID_NUMERIC_HOST (caller
fails closed) instead of a DNS pass-through. A host that does not end in a number
stays an ordinary DNS name. Also reject an empty IPv4 part so interior empty
labels (`1..2.3`) fail rather than parse.

Verified against real Chromium via the compiled production Kotlin across 3379
probes (base + fuzz + numeric-ending edge cases + DNS names ending in digits):
ZERO wrong-value AND zero conservative-null — the implementation now matches
Chromium exactly on every input. All 54 UrlPolicyTest assertions regenerated from
the compiled Kotlin's output.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
Two more wrong-value cases the compiled-Kotlin-vs-Chromium diff surfaced:

- Overflowing hex like `http://0x8000000000000000` was mistaken for a DNS name:
  the "ends in a number" test went through parseIpv4Part, which returned null on
  Long overflow, so the host fell through to a pass-through. Split the concern:
  `ipv4PartLooksNumeric` is a pure SYNTAX test (a digit run, or `0x`+valid-hex)
  that decides "ends in a number" independent of magnitude and octal validity;
  parseIpv4Part still enforces value validity. An overflowing-but-numeric host
  now fails closed. This also fixes `09` (looks numeric, is an invalid octal →
  the browser rejects it → we must too), which the previous round regressed.

- The raw-authority fallback emitted browser-divergent literals for hosts with a
  character a browser percent-encodes (`foo*bar` → `foo%2Abar`). The fallback
  does not encode, so it now accepts a host verbatim ONLY when every character is
  one a browser also keeps verbatim: the LDH set plus `_` and `~` (derived by
  probing all 94 printable-ASCII chars against Chromium). This RECOVERS real
  hostnames java.net.URI wrongly rejects (`my_host.local`, `foo_bar` — common on
  Docker/internal networks) while failing closed on the exotic-char hosts no
  self-hosted server uses.

Verified against real Chromium via the compiled production Kotlin across 4074
probes (base + fuzz + numeric-ending + overflow + all-ASCII-char host probes +
underscore/tilde recovery): ZERO wrong-value results. All 62 UrlPolicyTest
assertions regenerated from the compiled Kotlin's own output.

Co-authored-by: sacgsxr <sacgsxr@users.noreply.github.com>
@nesquena-hermes

Copy link
Copy Markdown
Contributor

Re-review at 5c8271d05 — the origin guard is sound, and the canonicalizer holds up. Ready to merge.

Thanks @sacgsxr. You didn't just add the guard I asked for — you closed the async gap for every script in applyHermesWebUiRuntimeScripts (viewport-fix, mic-fallback, clarify-autofocus, notification-bridge, route-recovery), including the pre-existing ones this PR didn't introduce. That's the right instinct.

The eight follow-up commits on UrlPolicy.kt were the part I wanted a hard look at: a hand-rolled URL canonicalizer feeding a security comparison, corrected that many times in quick succession, is usually still wrong somewhere. I gated it and verified it independently. It holds up.

What I verified

The guard construction is safe. HermesWebUiScripts.kt:16-24 wraps each payload in a real IIFE with 'use strict', so the early return is valid rather than the top-level-return syntax error that would silently no-op the whole script. All eight runtime payloads (MainActivity.kt:1631-1647) go through the wrapper — including the hardware-keyboard flag, which you inlined to replace the previously-unguarded syncHardwareKeyboardState(view) call. Good catch; that one would have slipped straight through.

The literal comparison is the right design. Not calling new URL(...) matters more than it looks: URL is a page-controlled global that a hostile origin can replace before an asynchronously-evaluated script runs, which would make the constructor return the attacker's own origin and defeat the check entirely. Comparing window.location.origin (unforgeable in Chromium) against a natively-canonicalized string literal is correct. pageOrigin's output alphabet is scheme plus [a-z0-9.:\[\]_~-], so JSONObject.quote has nothing dangerous to escape — no string-literal breakout is reachable.

The canonicalizer agrees with Chromium. I ran every test-vector expectation plus a batch of adversarial probes through a WHATWG parser. Everything matched except two IPv6-embedded-IPv4 octal cases where WHATWG rejects — so I went to Chromium's url/url_canon_ip.h: DoIPv6AddressToNumber feeds the embedded quad through the same radix-aware DoIPv4AddressToNumber used for bare hosts, and Chromium's own unit table accepts [::ffff:0xC0.0Xa8.0x0.0x1]. Your Chromium-divergent expectations are the correct ones for a WebView, which is what actually matters here. The bare-IPv4 path (trailing-dot drop, right-to-left "ends in a number", broken-octal rejection, >4 parts, overflow) mirrors Chromium step for step, the IPv6 contraction matches ChooseIPv6ContractionRange, and _/~ do pass through verbatim in Chromium's host table as your fallback assumes.

Fail-closed is real. Percent-escapes, non-ASCII, *, space, bracket-less colons, bad ports, scoped IPv6, and invalid numeric hosts all return null, which skips injection rather than emitting a literal that could never match — the right direction, since a stray literal would silently disable every runtime shim instead of loosening the boundary.

Strict mode didn't break the legacy scripts. Five of them had no 'use strict' before and now run under one. I audited for implicit globals, arguments, with, octal literals/escapes, reserved-word bindings, block-level function declarations, and writes to read-only targets. None present, and CI's eslint no-undef pass over the extracted strings covers regressions going forward.

Tests are non-vacuous where it counts. Both instrumentation tests fail with the guard removed — one asserts the provider viewport is untouched, the other asserts the Hermes rewrite still happens. (The unit-test doesNotContain("new URL(") assertion is vacuous alone, but it's paired with a positive assertion on the guard line, so the pair is fine.) CI is green across all 11 checks at this head, including API 35 and API 36 instrumentation.

Two follow-ups — not blockers, worth a look later

  1. Nothing exercises pageOrigin against a real WebView. Every instrumentation test hardcodes https://hermes.test (HermesWebUiCompatibilityTest.kt:45), so those 300 lines of canonicalization were validated only by unit tests whose expectations I had to check against Chromium source by hand. One parameterized test that loads a fixture at a battery of base URLs (default-port, uppercase, trailing-dot name, [0:0:0:0:0:0:0:1], 0x7f000001, 010.0.0.1) and asserts window.location.origin equals UrlOrigins.pageOrigin(baseUrl) would give you an oracle that catches future Chromium drift automatically. That's the test I'd most want to exist a year from now.

  2. Most of the canonicalizer is unreachable in production. ServerUrlValidator.isValid requires a non-null java.net.URI host on every write path, so hosts Java rejects can never become a configured server URL — the raw-authority fallback (UrlPolicy.kt:179-217) can't be reached from a real config. Separately, WebTrustPolicy.isConfiguredWebUiRoute string-compares Java hosts against the WebView's canonical URL, so non-canonical spellings like 2130706433 already fail before pageOrigin runs. In practice only lowercasing and default-port elision are load-bearing. That's fine — defense in depth — but the KDoc at UrlPolicy.kt:121 reads as though the numeric/IPv6/fallback paths protect live deployments, and a future maintainer will treat them as such. Worth a sentence naming the two upstream gates, or trimming the fallback to match reality.

Minor, take or leave: the comment at UrlPolicy.kt:209 says a browser percent-encodes (, but Chromium's host table keeps it verbatim and escapes only space and * — the fail-closed decision is still right, just the rationale is off. And HermesWebUiScriptsTest.kt:13 asserts the JVM org.json rendering while Android's JSONObject.quote emits "https:\/\/..." on device; both are identical JS strings, so it passes, but a tolerant regex would be less brittle.

Merging. Thanks for the thorough work on this one — the eight-commit correction arc landed somewhere genuinely solid.

Gate: Fable (claude-fable-5-1) — SHIP AS-IS, verified against Chromium's url_canon_ip.h and a WHATWG parser, plus my own read of every changed file. Reviewed at exact head 5c8271d05b329a3a27e512cd0349a809142d73cb. Codex was unavailable this round (expired auth token).

@nesquena-hermes
nesquena-hermes merged commit 32618e3 into hermes-webui:main Sep 17, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants