Skip to content

Add allowlisted macOS and Windows computer use - #648

Open
naliazheli wants to merge 2 commits into
andrewyng:mainfrom
naliazheli:restore/pr-543-computer-use
Open

Add allowlisted macOS and Windows computer use#648
naliazheli wants to merge 2 commits into
andrewyng:mainfrom
naliazheli:restore/pr-543-computer-use

Conversation

@naliazheli

Copy link
Copy Markdown

Restores #543. GitHub automatically closed the original PR when its source fork was temporarily made private. This replacement preserves the original change.

Summary

  • add opt-in macOS and Windows Computer use settings with an explicit application allowlist
  • accept validated .app bundles on macOS and exact .exe paths on Windows
  • bundle pinned Cua Driver 0.22.0 builds with SHA-256 verification, vendor signature verification on macOS, and the MIT license
  • guide macOS users through Accessibility and Screen Recording permission setup
  • run Cua Driver with exact executable, PID, and window manifests and expose accessibility discovery, screenshots, app launch, click, text input, and key press tools
  • keep the signed macOS application bundle immutable by writing the short-lived runtime manifest to the user's state directory

Security model

  • disabled by default, with an empty allowlist
  • exact application paths only; command interpreters and automation applications are rejected
  • macOS bundles must contain a valid Info.plist, an executable Mach-O, and no escaping symlinks
  • no coordinate-based click API
  • interaction tools require fresh one-time element tokens whose labels are verified before action
  • app launch, screenshot writes, clicks, typing, and key presses remain approval-gated
  • no shell, arbitrary process, filesystem, or browser-profile capabilities are admitted
  • Cua Driver telemetry is disabled; manifests are deny-by-default, expire after 8 hours, and use a 30-minute idle timeout
  • runtime manifests are stored outside the signed app bundle with 0600 permissions on macOS

The bundled archives and checksums come from the official Cua Driver 0.22.0 release.

Screenshot

Computer use settings

Verification

  • 19 passed: focused backend connector/settings tests, including real macOS PID-to-executable resolution
  • 136 passed: complete frontend unit suite
  • npm run build
  • cargo check
  • full backend suite: 1877 passed, 1 skipped; 3 existing URL-guard tests fail only because this machine's proxy DNS maps public test domains to 198.18.x.x
  • built a complete macOS DMG and verified the packaged Cua Driver 0.22.0 binaries are Cua AI-signed universal arm64 + x86_64 Mach-O files
  • exercised the real driver against TextEdit: only the allowlisted window reached OpenWorker; a foreign visible window was filtered out before the model-facing result
  • exercised the packaged server settings API: .app validation, driver reload, external 0600 runtime manifest, and unchanged in-bundle policy hash all passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant