Skip to content

Game controllers, mouse look and held keys for Linux games (refs #163) - #314

Open
szymonkucko wants to merge 1 commit into
SimonSchubert:mainfrom
szymonkucko:input-devices
Open

szymonkucko wants to merge 1 commit into
SimonSchubert:mainfrom
szymonkucko:input-devices

Conversation

@szymonkucko

Copy link
Copy Markdown

Linux games can only be played by touch today. This PR lets them use what's connected to the phone: a game controller becomes a real evdev pad in the guest, a game that locks the pointer gets the mouse captured with relative motion, and a physical keyboard's held keys stay held. Programs that use none of this see no change, and a pad still moves around the launcher whenever no Linux program has it open.

Refs #163 (mouse and pointer capture for desktop use). Here the game asks for capture through pointer-constraints and lets go the same way; there's no shortcut to toggle it.

How it works

  • Game controllers, without root (linux/Gamepads.kt, linux/PadLayout.kt, patches/proot/gamepads/gamepads.c): the app can't create devices in /dev/input, so each pad Android sees is a unix socket files/input/eventN, bound over the guest's /dev/input. libgamepads.so is a freestanding library like the fast lane, built by fetch-proot.sh and preloaded ahead of it. It turns open("/dev/input/eventN") into a connect and reads a description of an Xbox 360 pad (xpad's layout, 045e:028e). It answers evdev's ioctls (EVIOCGBIT, EVIOCGABS, EVIOCGNAME, EVIOCGID, …) from that description, and stat/fstat report a character device 13:(64+N). The program reads real input_events from the socket. SDL and gilrs find pads by watching /dev/input (SDL_JOYSTICK_DISABLE_UDEV=1, GILRS_DISABLE_UDEV=1), including hotplug.
  • Programs that only ask libudev (RetroArch's default joypad driver, Godot): UdevRecords writes udev's record of each pad into the /sys stand-in and into /run/udev/data, a new bind of files/udev. systemd 262's libudev refuses entries that aren't on a real sysfs, so the guest gets SYSTEMD_DEVICE_VERIFY_SYSFS=0. No udev daemon runs, so these programs see the pads that are there when they start.
  • Routing: a pad's input goes to Linux only while a program holds its socket and a Linux window is in front. Otherwise it moves around the launcher as before. When the window goes to the back, the program sees every button released and every stick at rest. A keyboard that also claims a joystick stays a keyboard (Gamepads.isPad, unit-tested; a Razer Ornata V3 reports KEYBOARD | DPAD | JOYSTICK).
  • Mouse look (wayland/PointerConstraints.kt, wayland/Input.kt, ui/WindowManager.kt, LinuxWindowActivity): zwp_pointer_constraints_v1 and zwp_relative_pointer_manager_v1. A lock captures the mouse for the window's SurfaceView (requestPointerCapture) and sends relative motion. The pointer stays where it was locked and doesn't leave the surface, because winit only unlocks a pointer that is over its window. Android lets go of the capture when the window loses focus.
  • Held keys (ui/Keyboard.kt, KeyboardBridge.preIme): while the soft keyboard is wanted, Samsung Keyboard turned a held physical key into an instant press and release, so W in a game was a tap. Keys from a physical alphabetic keyboard are now taken in dispatchKeyEventPreIme, before the keyboard app. AltGr, Super and characters the US layout types differently still go the old way. In a window that locked the pointer, Android's key repeat is no longer replayed as release and press.

Limitations

  • Holding Guide for a second should leave the window as Back does, but the test controller's Guide button never reached the app, so that path is untested.
  • No rumble: EVIOCSFF fails and the pad reports 0 effects.
  • Statically linked and non-glibc programs don't see pads, because the library comes in through LD_PRELOAD.
  • statx isn't hooked, so coreutils stat still shows a socket. SDL and RetroArch use fstat and stat.
  • Programs that use libudev get no hotplug. RetroArch's udev driver also needs its controller profiles (Online Updater › Update Controller Profiles), which Arch doesn't package; its sdl2 driver works without them.
  • Not covered: looking around by touch in games.

Relation to open PRs

None of the open PRs adds controllers or pointer locking. I applied every open PR that touches these files on top of this branch. The conflicts this branch adds are all neighbouring lines: imports (#153, #222), the locked-pointer check at the top of Input.pointerMotion next to the drag handling (#220, #221), new fields side by side in WindowManager (#40, #128), and paragraphs in ARCHITECTURE.md (#313). I'm happy to rebase on whichever lands first.

  • input: an on-screen gamepad for Linux windows #217 (on-screen gamepad) sends keys straight to the compositor, so it doesn't meet the controller or keyboard code here. It could later feed a pad socket, so games see it as a controller.
  • Touchpad mode for Linux windows #153 (touchpad mode) moves the cursor with absolute motion, which a locked pointer ignores. In a game that locks the pointer, the touchpad can't look around until it sends relative motion while a lock is active. That would be a small follow-up once both are in.

Tested

  • ./gradlew :owl:test :app:testFossDebugUnitTest :owl:owlJar :app:assembleFossDebug :app:lintFossDebug passes, and :app:lintPlayDebug (locally: 342 app tests, 238 owl tests, 0 lint errors). New tests: PadLayoutTest, GamepadsTest, UdevRecordsTest.
  • Tried on a device: Snapdragon 8 Gen 3 (Adreno 750). Details below.
  • New UI text: none.
  • OWL changes: none.
  • Launcher-written files changed: libgamepads.so is a new managed file in /usr/local/lib (Policy.Owned). files/input and files/udev are new directories bound into the guest and created on every start; sockets and records of an earlier run are cleared at start. Nothing to migrate.

On the phone (Snapdragon 8 Gen 3):

  • Xbox Wireless Controller over Bluetooth:
    • A raw evdev reader.
    • A Rust/winit game using gilrs.
    • SuperTuxKart: SDL2 through sdl2-compat on SDL 3.4, including unplugging and plugging the pad back in.
    • RetroArch 1.22.2: the sdl2 driver finds the pad. The udev driver finds it and configures it with the stock "Microsoft X-Box 360 pad" profile; input through the udev driver wasn't exercised.
  • Logitech MX Master over Bluetooth: mouse look in the gilrs game, and menus in SuperTuxKart.
  • Razer Ornata V3 over USB: held WASD in the game, with the pad connected at the same time.
  • Godot is untested. Its Linux joypad code lists pads through libudev the same way RetroArch does.
  • Rerun on this commit: the guest sees the launcher's folders at /dev/input and /run/udev, the pad appears as /dev/input/event0, libudev lists it, and stat gives char 13:64.

Trying it

Pair a controller, open a Linux terminal and ls /dev/input: there's an eventN per pad. Then e.g. sudo pacman -S supertuxkart and play with the pad, or with mouse and keyboard in any game that locks the pointer.

🤖 Generated with Claude Code

…becomes an Xbox 360 pad in the guest, found through /dev/input and udev's records (SDL, gilrs, RetroArch), a game that locks the pointer gets the mouse captured to look around with, and a physical keyboard's held keys stay held instead of turning into taps

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant