Skip to content

feat(browser): expand agent interaction operations #13

Closed

Description

@trmosala

Problem

Only click, fill and keypress operations are exposed.

Implementation order

Browser backlog item 09 of 29. Phase: Agent browsing. This is ordered planning, not a claim that Codex implements every item. Items 24–29 are optional additions.

Acceptance criteria

  • Add select-option, hover, double/right-click and drag using Chromium input facilities where appropriate.
  • Maintain snapshot/tab identity and stale-reference rejection.
  • Validate menus, native selects, drag targets and revoked access.

Dependencies

Engineering and validation

Reuse existing Chromium/Electron/OS facilities where available. Preserve main-process ownership and permission enforcement, add no placeholder controls, use existing i18n patterns, and test behavior with synthetic fixtures. Document platform limitations. Baseline: browser-tabs at 47fef3e; see packages/cm-browser/README.md and password-security.md.

Activity

  1. trmosala commented on Sep 16, 2026

    @trmosala
    OwnerAuthor

    Published and remote-confirmed: af5842e. One commit and normal push, exactly 18 reviewed files; pre-push typechecks 31/31 passed (28 cached). Only the two excluded unrelated files remain dirty and were not inspected or changed.

    Added hover, double/right-click, identity-bound native single-select choice and bounded same-tab pointer drag. Existing source/access approvals, stale-node rejection, native-settlement ownership and interrupted-input quarantine remain intact. User-approved right-click behavior suppresses native menus on that tab until actual settlement, including concurrent manual menus; DOM handlers, other tabs and later manual menus remain unaffected.

    Final cumulative correctness/spec and standards reviews found no actionable findings. Parent directly verified 125 browser tests / 1,981 expectations and the full isolated Windows native smoke after the final shared-setup correction. Separate focused --interactions passed after that correction; 60 cm-browser tests, 9 native-i18n tests and desktop/app/cm-browser typechecks passed. Scoped lint had warnings and no errors; the previously recorded unrelated root lint failure was not repaired or replayed.

    Limits: single-select events are untrusted and changed-only; queued dispatched selection cannot be recalled. Selection-specific queued cancellation, main-world tampering and synchronous event-handler mutation are not independently exercised. Drag uses existing snapshot-addressable controls and four fixed moves, not arbitrary coordinates, plain drop div discovery, universal HTML5 or file/native dragging; success is not proof of drop acceptance. Validation and native dispatch are not atomic. No cleanup release or automatic replay on interrupted input.

    Issue remains open for physical-input/native accessibility, packaged and macOS/Linux evidence, verified translations and separate #34 assurance. One new typed stale-ref phrase uses the existing English fallback; known unrelated localization failures remain. No installed app/server/bridge restart, model configuration change, release or deployment. Next ordered slice: #15 screenshots, then #16 tab lifecycle.

  2. trmosala commented on Sep 21, 2026

    @trmosala
    OwnerAuthor

    Implemented and published in af5842e (feat(browser): add identity-bound interaction operations). Later hardening in f617913 and 1f50d96 added final delivery-authority checks; current code also binds owner context and host-policy revision across operations.\n\nVerified again on 2026-09-21:\n- packages/desktop: bun scripts/browser-smoke.ts --interactions passed on macOS with Electron 44.3.0 / Chromium 152.0.7977.78\n- Native coverage passed for hover, double/right-click, native select, drag targets, stale identity, revocation, quarantine, and menu suppression\n- Commit diff check passed\n- Current app browser tests: 43 passed\n- Current cm-browser tests: 91 passed\n- Current desktop browser tests: 257 passed\n- App, cm-browser, and desktop typechecks passed\n- Push hook repository typecheck: 31/31 tasks passed\n- Spec review found no acceptance gaps; historical review findings against the original commit are covered by later hardening and current tests\n\nPackaged-app, physical accessibility/input, broader platform, verified translation, and external #34 assurance remain separate release-validation work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions