Repository navigation
Agent integration: pointer/axis IPC + contrib/driftwm-agent + docs - #341
youssefvdel wants to merge 6 commits into
Conversation
- msg pointer <x> <y> [--button left|right|middle]: move the seat pointer in the output layout and optionally click there. Works in nested sessions (no uinput) and gives remote viewers/agents a coordinate input path. - docs/agents.md: how to drive driftwm with AT-SPI agents; docs/ipc.md: pointer command. - contrib/driftwm-agent: reference adapter (state/shot/focus/center/snap/click/type) and a WebSocket browser viewer with keyboard input. - cli docs regenerated.
- msg axis <dx> <dy>: scroll the focused surface by wheel notches (verified: wev sees axis_source wheel, value120, value) - contrib viewer: mouse-wheel forwarding, optional --jpeg (ffmpeg transcode, ~39 KB/frame at 1600x900) and --fps. - docs/ipc.md scrolling section, cli docs regenerated, hyphen values accepted for pointer/axis args.
Without it the udev backend prefers the system primary GPU. On a headless box that card has no active scanout; an explicit device (e.g. /dev/dri/card0 from vkms) lets driftwm run a real 1920x1080 session with no monitor and no X server. Verified locally: driftwm --backend udev --drm-device /dev/dri/card0 on vkms, output 'Virtual-1' at 1920x1080@60, IPC state + 1920x1080 canvas screenshot.
Maybe this should be in pixels to match |
FocusedWindow gains pid (same socket-credential source as WindowInfo) so agents map focus to AT-SPI without a second state round-trip; stand-ins report None. OutputInfo gains layout_origin (the output top-left in global layout coords, same space msg cursor/pointer use). Reload and cold boot now refresh/seed active_layout display name (was stale English (US) / empty until first keypress).
The agent integration shipped in the compositor itself: inventory, pid, focus+reveal, pointer/axis injection and canvas capture are all 'driftwm msg' subcommands, in-process. The contrib/driftwm-agent CLI was pure indirection over those commands plus a wtype/ydotool shim and a viewer that duplicates RustDesk, so it is removed rather than maintained as a second interface. agents.md now documents the built-in surface and how a standard AT-SPI client pairs with it: map windows by pid, focus to reveal, capture via the compositor. Notes the two upstream limits that are not ours to fix: agent-desktop lists displays only through Mutter DisplayConfig, and xdg-desktop-portal-wlr writes its Screenshot result to a hardcoded /tmp/out.png (ScreenCast through the same portal works against our screencopy).
`msg pointer <x> <y>` takes output-layout logical pixels and `cursor` reports the same space, but `msg axis <dx> <dy>` took wheel notches, so a caller had to know two units to drive one pointer. Reported by Granitosaurus on malbiruk#341. `axis` now takes the same logical pixels. The frame still carries both `axis_value120` and `axis`, so clients that scroll discretely keep getting whole notches: a notch is 15 px, which is the ratio the old notch-based call already used internally, so one notch in is still one notch out.
|
Good catch — changed in 4494e50.
The frame still carries both fields, so clients that scroll discretely keep getting whole notches: a notch is 15 px, which is the ratio the old notch-based call already used internally — one notch in is still one notch out.
|
|
While re-verifying the pixel change I could not reproduce scroll delivery to a client, and I want to check my method before trusting the change. On a live driftwm session (udev backend), with The real wheel case is explained by the scroll binding: What I cannot explain is the IPC path: Question: in the run where |
|
Follow-up with a controlled run, because the first one was inconclusive (my pointer was not actually over the client). Setup: So focus was real and established before the injection. Real input reaches the client; That suggests the IPC path updates seat state without a client-facing frame, rather than my pixel change being wrong. Before I open anything as a bug: is |
|
Reproduced with a real client that actually commits a buffer, so
Nothing moved: the terminal content is identical, so the wheel never reached the app, and the camera did not move, so no scroll action consumed it. Both Happy to instrument and dig further, but this is outside my change: |
|
Correction — I was wrong in my two previous comments. Scroll delivery works. The controlled run, nested ( The after-shot shows the terminal had moved from the tail of the output to lines 104-141, and the scrollbar thumb moved. So I instrumented What went wrong in my earlier attempts: on the live udev session The pixel change itself stands: |
Adds the missing input path for AI agents and remote viewers, plus a reference adapter.
What
msg pointer <x> <y> [--button left|right|middle]— move the seat pointer in the output layout and optionally click. No uinput, so it works in nested sessions where host input devices are invisible to driftwm.msg axis <dx> <dy>— scroll the focused surface by output-layout logical pixels, the same spacepointerandcursoruse; a notch is 15 px (source wheel, v120 + value), verified inwev.contrib/driftwm-agent— thin adapter (Bun/TS):state,shot --scale,center,focus,snap(AT-SPI),click,type(wtype),msg; plus a WebSocket browser viewer (PNG or--jpeg, wheel/mouse/keyboard, token/view-only options).docs/agents.md— how to drive driftwm with AT-SPI agents;docs/ipc.mdpointer/scroll sections;docs/cli.mdregenerated.Verification
cargo clippy --all-targets -- -D warningsclean.cargo test --bin driftwm ipc— 113 passed.640 400→pointer 400 300→cursor 400 300;pointer 640 400 --button leftlands;axis 0 -15producesaxis_source wheel, value120 -120, value -15inwev; viewer streams 1600×900 at ~6 fps, JPEG ~39 KB/frame.Notes
cursor), matching the frame a viewer receives.