W-572: trigger reliably in Slack and other Electron apps - #190
Conversation
The Slack trigger drops arrive as engine.secureFieldBlocked, but the log can't tell "field not classified yet" (fail-closed default) from "classified as a genuine secure/opaque role". Thread the cache's haveFieldInfo and the raw AXRole through ActiveContext to both block sites and into the debug report's Now section, so a controlled repro can disambiguate the two mechanisms before any fix. Diagnostics only. No behavior change to the capture/secure-field path. Refs: W-572
…used field Chromium/Electron apps (Slack, VS Code, Discord…) don't build their accessibility tree until a client sets AXManualAccessibility on the app element. Until then AXUIElementCopyAttributeValue(app, focusedUIElement) returns nothing, so the focused-element cache seeds nil, and every : trigger hits the "unknown focus → fail closed as secure" guard and is dropped. That's why Mojito fired in Slack only intermittently (roughly when a relaunch happened to catch the tree already built) while working everywhere else. Set the attribute on the app element at seed time. Native apps don't implement it and ignore the set; Chromium flips into full AX mode and exposes its focused AXTextArea, which then classifies as editable / non-secure and drives the picker normally. Verified end to end in Slack: capture opens, picker shows, emoji inserts. Also carries diagnostics that pinned the cause: haveInfo / role / elem / editable on the secureFieldBlocked log line, and field-classification state in the debug report's Now section. Refs: W-572
There was a problem hiding this comment.
The root-cause diagnosis and fix are solid. AXManualAccessibility is the right lever, placed at the right time (before the observer fires in seed), and the existing seedTimeout on axApp bounds how long that set can block. Native apps silently ignore the attribute, Chromium builds its tree, and the reseed fallback handles the lazy-build race. The secure-field guard is unchanged — AXSecureTextField still classifies and blocks regardless. Diagnostic fields are well-scoped (diagnostics-only, clearly labeled) and the tuple-type extension through classify → publishFieldInfo → ActiveContext is clean.
Nit: DebugReport.swift now emits both focusedElementCached: \(element != nil) (pre-existing) and focusedElementNil: \(cache.element == nil) (new) — they're boolean inverses of each other. Not harmful, but one of them is redundant.
Follow-ups from review of the initial fix: - isSecure now matches the AXSecureTextField *subrole*, not just the role. "AXSecureTextField" is a subrole only (there is no secure role constant); native NSSecureTextField and Chromium/WebKit <input type=password> all report role AXTextField with that subrole, so the prior role-only check never matched a real password field. Now that Electron/web AX trees are enabled their password fields are visible, so the subrole check is what keeps them out of the picker. Verified against Chromium/WebKit/Apple sources. - AXManualAccessibility is set only when the focused-element read comes back empty, instead of on every app. Native apps return a focused element and are never touched; only tree-gating apps (Electron, Chromium) get the flag, and only when focused. Also moved past the seed generation guard so a superseded seed no longer does the write. Refs: W-572
Follow-up hardening (from review of the first commit)Review flagged that the initial fix widened exposure without tightening the secure-field guard. Two follow-ups, pushed: 1. Secure-field detection now catches web/Electron password fields. 2. Verification
|
There was a problem hiding this comment.
The fix is correct and well-scoped. A few notes:
What looks good
AXManualAccessibilityis gated onseeded == nil— native apps that already return a focused element are never touched. Electron apps that return nil get the flag set once; Chromium remembers it for the lifetime of the process, so subsequent visits won't re-set it unnecessarily but are also harmless if they do.- The subrole check for
AXSecureTextFieldis the right fix now that Electron AX trees are exposed. NativeNSSecureTextFieldreports roleAXTextFieldwith subroleAXSecureTextField, so the old role-only check silently missed it. The role OR is kept as a zero-cost defensive fallback and the comment explains the distinction clearly. - Both
secureFieldBlockedcall sites get the same diagnostic fields, so the log is consistent regardless of which path trips it. focusedRoleis correctly nil-ed inelement'sdidSetso a stale role from the previous focus can't bleed into the next one.
Nit
DebugReport.now() now emits both focusedElementCached (existing) and focusedElementNil (new). These are the exact inverse of each other and read from the same cache instance, so one is redundant. Could drop focusedElementNil and rely on focusedElementCached, or rename the existing line to focusedHaveElement for symmetry with the new focusedHaveFieldInfo — either way it's a minor polish item, not a blocker.
|
Gap closed: live-tested a password field in a Chromium browser — the picker stays hidden, is not captured. Secure-field block confirmed. Full matrix now verified: Slack ✓, Helium/Chromium ✓, native (Messages) ✓, password field blocked ✓. |
Problem
Mojito's
:trigger fired in Slack only ~10% of the time, while working everywhere else. The reported theory was event-tap startup sequencing.Actual cause
The event tap was fine. Every dropped Slack trigger was hitting the secure-field guard. Instrumentation confirmed the mechanism:
haveInfo=false role=nil elem=falseon every attempt — the focused-element cache was nil for Slack.Chromium/Electron apps don't build their accessibility tree until a client sets
AXManualAccessibilityon the app element. Until thenAXUIElementCopyAttributeValue(app, focusedUIElement)returns nothing, the seed caches nil, andAppContextDetector.current()fails closed (unknown focus → secure) and drops the trigger. Native apps (Mail, Messages) expose focus by default, so only Electron apps broke. The rare successes lined up with a relaunch catching the tree already built.Fix
Set
AXManualAccessibilityon the app element at seed time (FocusedElementCache.seed). Native apps don't implement it and ignore the set; Chromium flips into full AX mode and exposes its focusedAXTextArea, which classifies as editable / non-secure and drives the picker normally.The fix does not weaken the secure-field guard — a real password field still classifies as
AXSecureTextFieldand stays blocked.Diagnostics (also in this PR)
haveInfo/role/elem/editableon thesecureFieldBlockedlog line.focusedHaveFieldInfo,focusedIsSecure,focusedClassifiedRole,focusedElementNil) in the debug report's Now section.These flow to both the in-app report and the
MOJITO_E2E_LOGstream.Test plan
Verified end-to-end on a live Debug build in Slack (macOS 27):
Last-picker-context confirmed Slack's compose box now exposes the full Chromium AX tree (
AXTextArea,ChromeAXNodeId,AXDOMClassList). Unit suite (scripts/run-tests.sh) passes.Note
Enabling Chromium accessibility adds a small ongoing AX-tree cost inside each Electron app (the same tradeoff VoiceOver makes). Necessary for Mojito to function there at all.
Refs W-572