Skip to content

Wire pane/browser control dispatcher to GTK state#25

Draft
mvbmir wants to merge 3 commits into
mainfrom
mvbmir/wire-browser-dispatcher
Draft

Wire pane/browser control dispatcher to GTK state#25
mvbmir wants to merge 3 commits into
mainfrom
mvbmir/wire-browser-dispatcher

Conversation

@mvbmir
Copy link
Copy Markdown
Owner

@mvbmir mvbmir commented Apr 17, 2026

Summary

Wires the pane, surface, and browser control-socket methods to the real GTK state. Until now limux-cli exposed a rich set of commands but the GTK host's control_bridge.rs only registered workspace.* + surface.send_text, so anything else returned -32601: unknown method. Closes #22 (fork tracking) and fills the gap flagged in #21.

Two commits, split by layer:

1. Pane/surface introspection + browser navigation (no JS eval)

  • pane.list, pane.surfaces, surface.list, surface.current
  • browser.open_split, browser.navigate, browser.url.get, browser.back, browser.forward, browser.reload, browser.screenshot, browser.eval

IDs: pane u32 as string, surface UUID string, refs pane:N / surface:UUID. Inputs accept raw ids or prefixed refs; outputs always return both forms.

browser.open_split defaults to hosting the new browser in a pane other than the focused (caller) pane. If only one pane exists it splits the focused pane horizontally and places the browser in the new split, so a caller never clobbers its own terminal. The source pane can be overridden via source_surface.

browser.screenshot uses webkit6::WebView::snapshot with SnapshotRegion::Visiblegdk_texture_save_to_png. Rejects empty textures rather than writing zero-byte files. The CLI forwards --out as the path parameter so the server writes directly to the caller's target.

2. Browser JS-eval tier (snapshot, actions, console, errors)

An init script (browser_init.js) is installed via UserContentManager at InjectionTime::Start on every top-frame navigation. It exposes window.__limux with:

  • Ref tagger + MutationObserver that keep data-limux-ref=\"eN\" on every interactive node so refs survive DOM mutations within a page.
  • Ring buffers for console.* and window.onerror / unhandledrejection, capped at 5000, with a monotonic seq counter the caller filters by (webkit6 clamps Date.now() / performance.timeOrigin to i32::MINperformance.now() is the only working time source, so wall-clock ts_ms is informational and seq is authoritative).
  • history.pushState / replaceState hooks for SPA navigation tracking.
  • isReady() probe (readyState complete + DOM quiet ≥ 500ms).
  • isEditable() probe for focused-element state.

A snapshot walker (browser_snapshot.js) returns a token-efficient AX tree keyed on the init-script's refs:

page https://example.com/login  title "Sign in"
- banner
  - link "Home" [ref=e1]
  - navigation "Main"
    - link "Docs" [ref=e2]
- main
  - heading "Sign in" [level=1]
  - form
    - textbox "Email" [ref=e4, required]
    - button "Sign in" [ref=e7]

Each snapshot also returns a djb2 hash (future-facing --since diff flag already plumbed through). Scope flags (--selector, --max-depth, --full-tree, --raw-html) supported.

Methods wired via a generic BrowserEval dispatcher:

  • browser.snapshot
  • browser.click, dblclick, hover, focus
  • browser.fill, type, press
  • browser.check, uncheck, select
  • browser.scroll, scroll_into_view
  • browser.wait, wait_ready
  • browser.get.{text, title, html, value, attr, count, box}
  • browser.find.{role, text, label, placeholder, testid}
  • browser.console.{list, clear}
  • browser.errors.{list, clear}
  • browser.is_ready, is_editable

Every action goes through a resolve-target helper that distinguishes refs (via window.__limux.refInfo) from selectors, returning a structured REF_NOT_FOUND error when a ref is no longer attached rather than silently acting on the wrong element. find.text prefers interactive descendants over structural ancestors so a paragraph wrapping a single link doesn't shadow the link itself.

Not in this PR (follow-ups)

Planned in a stacked PR on top of this one:

  • Cookies / storage / state save+load (login persistence)
  • Tabs / frames / addinitscript / highlight
  • High-level primitives (navigate_and_wait, click_and_await_nav, fill_and_submit)
  • browser.screenshot --annotate ref overlay
  • browser.snapshot --since <hash> diff mode (already plumbed through, needs JS-side diff computation)

Test plan

  • limux-cli --json list-panes returns real panes (not -32601)
  • limux-cli --json browser open 'about:blank' returns a surface id
  • limux-cli browser --surface <S> navigate https://example.com + url round-trip
  • limux-cli browser --surface <S> screenshot --out /tmp/shot.png produces a valid PNG (non-empty, correct dimensions)
  • limux-cli browser --surface <S> snapshot returns AX tree + refs keyed on interactive nodes
  • limux-cli browser --surface <S> click --selector @e1 fires and navigates where expected
  • browser.console.list + browser.errors.list via raw socket return ring-buffer contents with monotonic seq
  • browser.open_split without --source-surface targets a non-caller pane (splits if none exists)

mvbmir added 3 commits April 18, 2026 01:31
Previously the control-socket METHODS array only exposed workspace.* and
surface.send_text. Every pane.*, surface.*, and browser.* call returned
-32601 unknown method. Same gap on upstream am-will/main.

This commit wires the first tier of methods end-to-end against live GTK
state, no JS eval:

  pane.list, pane.surfaces, surface.list, surface.current
  browser.open_split, browser.navigate, browser.url.get
  browser.back, browser.forward, browser.reload
  browser.screenshot, browser.eval

IDs and refs: pane ids are u32 as string, surface ids are UUID strings,
refs are "pane:N" / "surface:UUID". Inputs accept raw ids or prefixed
refs via normalize_handle. Outputs always include both id and ref so
callers never have to reconstruct either form.

browser.open_split defaults to hosting the new browser in a pane other
than the focused (caller) pane. If only one pane exists it splits the
focused pane horizontally and places the browser in the new split, so a
caller never clobbers its own terminal. The source pane can be
overridden explicitly via source_surface.

browser.screenshot uses webkit6::WebView::snapshot with SnapshotRegion::
Visible → gdk_texture_save_to_png, rejecting empty textures instead of
writing a zero-byte file. The CLI now forwards --out as the `path`
parameter so the server writes directly to the caller's target rather
than leaking a random /tmp path.

Async ops (navigate/open_split/screenshot/eval) use mpsc reply channels
with a 30s timeout; sync ops default to 5s.
…rors)

Adds the JS-eval tier of the browser dispatcher on top of the pane/nav
wiring from 1697f3c. No CLI changes — all methods reachable via
limux-cli browser subcommands or raw control socket RPC.

Init script (browser_init.js) is installed via UserContentManager at
InjectionTime::Start on every top-frame navigation. It exposes
window.__limux with:

  - Ref tagger + MutationObserver that keep `data-limux-ref="eN"` on
    every interactive node so refs survive DOM mutations within a page.
  - Ring buffers for console.{log,warn,error,info,debug} and for
    window.onerror + unhandledrejection, capped at 5000 entries each,
    with a monotonic seq counter the caller filters by (webkit6 clamps
    Date.now() / performance.timeOrigin to i32::MIN so wall-clock is
    unusable — performance.now() is the only working time source).
  - history.pushState / replaceState hook that fires a
    limux:navigation event + bumps a navCount for SPA handling.
  - isReady() probe (readyState complete AND DOM quiet ≥ 500ms).
  - isEditable() probe for active-element focus state.

Snapshot walker (browser_snapshot.js) returns a token-efficient AX tree
keyed on the init-script's refs. Output format:

  page <url>  title "<title>"
  - banner
    - link "Home" [ref=e1]
    - navigation "Main"
      - link "Docs" [ref=e2]
  - main
    - heading "Sign in" [level=1]
    - form
      - textbox "Email" [ref=e4, required]
      - button "Sign in" [ref=e7]

Each snapshot returns {url, title, hash, snapshot_text, refs, ...}
with djb2 hash for future --since diff support. Scope flags
(--selector, --max-depth, --full-tree, --raw-html) included.

New methods wired via generic BrowserEval dispatcher:

  browser.snapshot
  browser.click, dblclick, hover, focus
  browser.fill, type, press
  browser.check, uncheck, select
  browser.scroll, scroll_into_view
  browser.wait, wait_ready
  browser.get.{text, title, html, value, attr, count, box}
  browser.find.{role, text, label, placeholder, testid}
  browser.console.{list, clear}
  browser.errors.{list, clear}
  browser.is_ready, is_editable

Every action goes through a resolve-target helper that distinguishes
refs (via window.__limux.refInfo) from selectors, returning a
structured REF_NOT_FOUND error when a ref is no longer attached rather
than silently acting on the wrong element. find.text prefers
interactive descendants over structural ancestors so a paragraph
wrapping a single link doesn't shadow the link itself.
Rust 1.95 stable (released between main's last green CI run and this
PR's) promotes collapsible_match to a warning, which the workspace's
check.sh treats as an error via -D warnings.

The flagged patterns in limux-core are `match combo_norm { pattern
=> { if palette_visible { … true } else { false } } … }` — they
collapse cleanly into match guards: `pattern if palette_visible =>
{ … true }`. Behavior is unchanged; any non-match combo hits the
catch-all `_ => false` arm.

Unrelated to the redesign; included here to keep CI green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wire pane/browser control dispatcher to GTK state

1 participant