feat(tools): add computer-use tool (§6.13) - #331
Merged
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
gnanirahulnutakki
pushed a commit
that referenced
this pull request
Jul 14, 2026
PR #331's CI failed two checks: cargo-deny (RUSTSEC-2026-0194/-0195, DoS advisories in quick-xml 0.39.4) and ubuntu-latest/stable (wayland-sys build script couldn't find wayland-client.pc via pkg-config). Root cause for both: enigo/xcap/image were plain dependencies, so Cargo resolved xcap's Linux Wayland backend (libwayshot-xcap -> wayland-client -> wayland-scanner -> quick-xml) into the graph even though this crate only targets macOS. Move enigo/xcap/image to a `[target.'cfg(target_os = "macos")'.dependencies]` section and cfg-gate driver.rs's real implementation behind `#[cfg(target_os = "macos")]`, with a `#[cfg(not(target_os = "macos"))]` fallback that reports the platform as unsupported rather than failing to compile. Verified via cargo tree --target x86_64-unknown-linux-gnu and cargo metadata's dep_kinds that the wayland chain is now unreachable for any single real build target. Also add the two RUSTSEC ids to deny.toml's advisories.ignore (matching the file's existing 4-entry precedent) as defense-in-depth: wayland-scanner 0.31.10 (latest) hard-pins quick-xml = "^0.39" with no upgrade path available, and the parser only runs at build time over vendored, trusted protocol XML, not attacker-reachable input. Checkpoint: architect/sessions/2026-07-14-computer-use-6.13/journal.md Signed-off-by: Gnani Nutakki <gnani.nutakki@gmail.com>
gnanirahulnutakki
force-pushed
the
feat/computer-use-6.13-2026-07-14
branch
from
July 14, 2026 07:08
805d4ca to
30dae2c
Compare
Signed-off-by: Ardur <team@ardur.ai>
gnanirahulnutakki
force-pushed
the
feat/computer-use-6.13-2026-07-14
branch
from
July 17, 2026 19:25
bb10f6e to
2480bb0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crates/computer-use(ardur-computer-use), Phase 1 of the §6.13 computer-use blueprint: a closed 10-action vocabulary (capture,click,double_click,right_click,middle_click,drag,scroll,type,key,wait), an in-substrateBlockedActionPolicydenylist (5 hard-blocked key combos + 6 dangerous-text patterns, adapted from Hermes'scomputer_use/tool.py:80-107) enforced regardless of cap-token or consent state, a cap-token-shapedComputerUseCaveat(per-action-kind permitted set), a session-scopedConsentBroker(grant/revoke/TTL), thecomputer.{session.started,consent.granted,consent.revoked,action.invoked,action.denied,screenshot.taken}.v1receipt family with aDenialSourcethat distinguishes blocked-action/cap-token/consent refusals, and aComputerUseTool(computer.use) gated by a customcomputer_usecapability. Gate order: denylist → caveat → consent.LocalInputDriver) usesenigo(mouse/keyboard synthesis) +xcap(screen capture) over public macOS APIs. The blueprint's own MVP decision explicitly rejects the private SkyLight-SPI path Hermes uses (SLEventPostToPid, etc.), gating it behind a separatecomputer_use_skylight_spicap-token caveat reserved for a future phase — so this PR isn't under-delivering relative to the blueprint's own stated MVP scope.lib.rs:set_value/list_apps/focus_app(needAXUIElementaccessibility-tree access, which neitherenigonorxcapexpose), thesom/axcapture modes,AXIsProcessTrustedWithOptions/CGPreflightScreenCaptureAccessOS-permission preflight, a real interactive consent dialog UI (ConsentBrokerships the state machine only), and cap-token-to-caveat projection from a verified Biscuit block./goaldirective's tool-surface domain (code-execution + browser/computer-use). Browser automation (§6.9/§6.10) already ships ascrates/browserand is not touched here.Test plan
cargo build -p ardur-computer-usecargo test -p ardur-computer-use(33/33 passing — denylist/caveat/consent gate-order tests, action-parsing tests, and awaitdispatch test; deliberately no test exercises real click/type/screenshot execution, since that would require Accessibility/ScreenRecording permission granted to the CI runner)cargo fmt --checkcargo clippy -p ardur-computer-use --all-targets -- -D warningscargo check --workspace --all-features(no cross-crate breakage from the newenigo/xcap/image/hexdependencies)