Skip to content

W-572: trigger reliably in Slack and other Electron apps - #190

Merged
wr merged 3 commits into
mainfrom
wells/w-572-unreliably-triggering-in-slack
Aug 22, 2026
Merged

W-572: trigger reliably in Slack and other Electron apps#190
wr merged 3 commits into
mainfrom
wells/w-572-unreliably-triggering-in-slack

Conversation

@wr

@wr wr commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Problem

Mojito's : trigger fired in Slack only ~10% of the time, while working everywhere else. The reported theory was event-tap startup sequencing.

Actual cause

The event tap was fine. Every dropped Slack trigger was hitting the secure-field guard. Instrumentation confirmed the mechanism: haveInfo=false role=nil elem=false on every attempt — the focused-element cache was nil for Slack.

Chromium/Electron apps don't build their accessibility tree until a client sets AXManualAccessibility on the app element. Until then AXUIElementCopyAttributeValue(app, focusedUIElement) returns nothing, the seed caches nil, and AppContextDetector.current() fails closed (unknown focus → secure) and drops the trigger. Native apps (Mail, Messages) expose focus by default, so only Electron apps broke. The rare successes lined up with a relaunch catching the tree already built.

Fix

Set AXManualAccessibility on the app element at seed time (FocusedElementCache.seed). Native apps don't implement it and ignore the set; Chromium flips into full AX mode and exposes its focused AXTextArea, which classifies as editable / non-secure and drives the picker normally.

The fix does not weaken the secure-field guard — a real password field still classifies as AXSecureTextField and stays blocked.

Diagnostics (also in this PR)

  • haveInfo / role / elem / editable on the secureFieldBlocked log line.
  • Field-classification state (focusedHaveFieldInfo, focusedIsSecure, focusedClassifiedRole, focusedElementNil) in the debug report's Now section.

These flow to both the in-app report and the MOJITO_E2E_LOG stream.

Test plan

Verified end-to-end on a live Debug build in Slack (macOS 27):

engine.colon excluded=false          → capture opened
picker.open  outcome=elementTopLeft  → picker showed
insert.exactMatch                    → emoji inserted

Last-picker-context confirmed Slack's compose box now exposes the full Chromium AX tree (AXTextArea, ChromeAXNodeId, AXDOMClassList). Unit suite (scripts/run-tests.sh) passes.

Note

Enabling Chromium accessibility adds a small ongoing AX-tree cost inside each Electron app (the same tradeoff VoiceOver makes). Necessary for Mojito to function there at all.

Refs W-572

wr added 2 commits August 22, 2026 13:12
The Slack trigger drops arrive as engine.secureFieldBlocked, but the log
can't tell "field not classified yet" (fail-closed default) from
"classified as a genuine secure/opaque role". Thread the cache's
haveFieldInfo and the raw AXRole through ActiveContext to both block
sites and into the debug report's Now section, so a controlled repro
can disambiguate the two mechanisms before any fix.

Diagnostics only. No behavior change to the capture/secure-field path.

Refs: W-572
…used field

Chromium/Electron apps (Slack, VS Code, Discord…) don't build their
accessibility tree until a client sets AXManualAccessibility on the app
element. Until then AXUIElementCopyAttributeValue(app, focusedUIElement)
returns nothing, so the focused-element cache seeds nil, and every :
trigger hits the "unknown focus → fail closed as secure" guard and is
dropped. That's why Mojito fired in Slack only intermittently (roughly
when a relaunch happened to catch the tree already built) while working
everywhere else.

Set the attribute on the app element at seed time. Native apps don't
implement it and ignore the set; Chromium flips into full AX mode and
exposes its focused AXTextArea, which then classifies as editable /
non-secure and drives the picker normally. Verified end to end in Slack:
capture opens, picker shows, emoji inserts.

Also carries diagnostics that pinned the cause: haveInfo / role / elem /
editable on the secureFieldBlocked log line, and field-classification
state in the debug report's Now section.

Refs: W-572
@linear-code

linear-code Bot commented Aug 22, 2026

Copy link
Copy Markdown

W-572

@wr-claude-reviewer wr-claude-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root-cause diagnosis and fix are solid. AXManualAccessibility is the right lever, placed at the right time (before the observer fires in seed), and the existing seedTimeout on axApp bounds how long that set can block. Native apps silently ignore the attribute, Chromium builds its tree, and the reseed fallback handles the lazy-build race. The secure-field guard is unchanged — AXSecureTextField still classifies and blocks regardless. Diagnostic fields are well-scoped (diagnostics-only, clearly labeled) and the tuple-type extension through classifypublishFieldInfoActiveContext is clean.

Nit: DebugReport.swift now emits both focusedElementCached: \(element != nil) (pre-existing) and focusedElementNil: \(cache.element == nil) (new) — they're boolean inverses of each other. Not harmful, but one of them is redundant.

Follow-ups from review of the initial fix:

- isSecure now matches the AXSecureTextField *subrole*, not just the
  role. "AXSecureTextField" is a subrole only (there is no secure role
  constant); native NSSecureTextField and Chromium/WebKit
  <input type=password> all report role AXTextField with that subrole,
  so the prior role-only check never matched a real password field. Now
  that Electron/web AX trees are enabled their password fields are
  visible, so the subrole check is what keeps them out of the picker.
  Verified against Chromium/WebKit/Apple sources.

- AXManualAccessibility is set only when the focused-element read comes
  back empty, instead of on every app. Native apps return a focused
  element and are never touched; only tree-gating apps (Electron,
  Chromium) get the flag, and only when focused. Also moved past the
  seed generation guard so a superseded seed no longer does the write.

Refs: W-572
@wr

wr commented Aug 22, 2026

Copy link
Copy Markdown
Owner Author

Follow-up hardening (from review of the first commit)

Review flagged that the initial fix widened exposure without tightening the secure-field guard. Two follow-ups, pushed:

1. Secure-field detection now catches web/Electron password fields.
isSecure matched role == "AXSecureTextField", but that string is a subrole, not a role — there is no secure role constant. Native NSSecureTextField and Chromium/WebKit <input type=password> all report role AXTextField with subrole AXSecureTextField, so the old role-only check never matched a real password field. It now matches the subrole (role kept as a zero-cost defensive OR). Confirmed against Chromium (browser_accessibility_cocoa.mm), WebKit (WebAccessibilityObjectWrapperMac.mm), and Apple's AXRoleConstants.h. This is what protects Electron/web password fields now that their AX trees are enabled — they no longer lean on the exclusion list.

2. AXManualAccessibility is now scoped to apps that need it.
Previously set on every app on every activation. Now set only when the focused-element read comes back empty — native apps always return an element and are never touched; only tree-gating apps (Electron/Chromium) get the flag, and only when focused. Also moved past the seed generation guard.

Verification

  • Slack: still triggers (regression check passed).
  • Helium (Chromium browser): triggers — confirms the fix generalizes beyond Slack.
  • Native apps (Messages): unaffected.
  • Gap: I did not get a clean live keystroke test in an actual password field. The subrole block rests on authoritative source confirmation + the code, not a live repro. Worth one manual check before/after merge.

@wr-claude-reviewer wr-claude-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix is correct and well-scoped. A few notes:

What looks good

  • AXManualAccessibility is gated on seeded == nil — native apps that already return a focused element are never touched. Electron apps that return nil get the flag set once; Chromium remembers it for the lifetime of the process, so subsequent visits won't re-set it unnecessarily but are also harmless if they do.
  • The subrole check for AXSecureTextField is the right fix now that Electron AX trees are exposed. Native NSSecureTextField reports role AXTextField with subrole AXSecureTextField, so the old role-only check silently missed it. The role OR is kept as a zero-cost defensive fallback and the comment explains the distinction clearly.
  • Both secureFieldBlocked call sites get the same diagnostic fields, so the log is consistent regardless of which path trips it.
  • focusedRole is correctly nil-ed in element's didSet so a stale role from the previous focus can't bleed into the next one.

Nit

DebugReport.now() now emits both focusedElementCached (existing) and focusedElementNil (new). These are the exact inverse of each other and read from the same cache instance, so one is redundant. Could drop focusedElementNil and rely on focusedElementCached, or rename the existing line to focusedHaveElement for symmetry with the new focusedHaveFieldInfo — either way it's a minor polish item, not a blocker.

@wr

wr commented Aug 22, 2026

Copy link
Copy Markdown
Owner Author

Gap closed: live-tested a password field in a Chromium browser — the picker stays hidden, is not captured. Secure-field block confirmed. Full matrix now verified: Slack ✓, Helium/Chromium ✓, native (Messages) ✓, password field blocked ✓.

@wr
wr merged commit 0acf344 into main Aug 22, 2026
10 checks passed
@wr
wr deleted the wells/w-572-unreliably-triggering-in-slack branch August 22, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant