Skip to content

Add keyboard and mouse input - #1

Merged
mlopezsegura merged 1 commit into
masterfrom
feature/keyboard-mouse
Nov 8, 2026
Merged

mlopezsegura merged 1 commit into
masterfrom
feature/keyboard-mouse

Conversation

@mlopezsegura

Copy link
Copy Markdown
Collaborator

Keyboard and mouse now work natively, as in a Moonlight client.

  • Capture: click the picture. The page takes pointer lock (raw, unaccelerated motion; the PC applies its own acceleration) and, in fullscreen, keyboard lock, so Esc, Alt+Tab and the Windows key reach the PC. Ctrl+Alt+Shift+Z (as in Moonlight) or holding Esc gives them back. A hint shows once the picture is playing.
  • Packets: web/input.mjs turns events into Moonlight input packets: keys by the Windows virtual-key code of their US-layout position (the PC's layout gives them meaning; Ñ and < > included), relative motion with sub-pixel carry-over, all five mouse buttons, vertical and horizontal wheel at 120 units per notch. Held keys repeat on the PC, so browser auto-repeat is not sent.
  • Release: everything held is released when the capture ends, the window loses focus, the tab is hidden or the stream stops.
  • Compatibility: uses Sunshine-Web-RTC's new input DataChannel (feat(webrtc): accept keyboard and mouse on an input DataChannel Sunshine-Web-RTC#5). With older servers the channel is absent and the client works as before, gamepads only.
  • Service worker cache bumped to version 3; README updated.

Testing

  • npm test: 11 passed, including new packet-layout and KeyboardMouse tests.
  • End to end in headless Edge against a Sunshine-Web-RTC build with #5: pair, start, pointer lock, F15 down/up, exact motion deltas, X1 button, scroll ±120, Ctrl+Alt+Shift+Z releasing held modifiers; all seen in the server's input log, no console errors.
  • pointermove is used rather than pointerrawupdate: in testing the first raw update of a burst came without its motion.
  • Not tested: a manual session with a physical keyboard and mouse.

🤖 Generated with Claude Code

Clicking the picture captures the pointer and, in fullscreen, the
keyboard. Keys, raw relative mouse motion, all five buttons and both
wheels go to the PC as Moonlight input packets on Sunshine-Web-RTC's
new "input" DataChannel; Ctrl+Alt+Shift+Z or holding Esc gives them
back, and everything held is released when the capture ends, the
window loses focus or the stream stops. Servers without the channel
keep working with gamepads only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mlopezsegura
mlopezsegura merged commit ad9a0e1 into master Nov 8, 2026
3 checks passed
github-actions Bot pushed a commit that referenced this pull request Nov 8, 2026
Clicking the picture captures the pointer and, in fullscreen, the
keyboard. Keys, raw relative mouse motion, all five buttons and both
wheels go to the PC as Moonlight input packets on Sunshine-Web-RTC's
new "input" DataChannel; Ctrl+Alt+Shift+Z or holding Esc gives them
back, and everything held is released when the capture ends, the
window loses focus or the stream stops. Servers without the channel
keep working with gamepads only.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mlopezsegura added a commit that referenced this pull request Nov 10, 2026
Clicking the picture captures the pointer and, in fullscreen, the
keyboard. Keys, raw relative mouse motion, all five buttons and both
wheels go to the PC as Moonlight input packets on Sunshine-Web-RTC's
new "input" DataChannel; Ctrl+Alt+Shift+Z or holding Esc gives them
back, and everything held is released when the capture ends, the
window loses focus or the stream stops. Servers without the channel
keep working with gamepads only.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mlopezsegura added a commit that referenced this pull request Nov 10, 2026
Clicking the picture captures the pointer and, in fullscreen, the
keyboard. Keys, raw relative mouse motion, all five buttons and both
wheels go to the PC as Moonlight input packets on Sunshine-Web-RTC's
new "input" DataChannel; Ctrl+Alt+Shift+Z or holding Esc gives them
back, and everything held is released when the capture ends, the
window loses focus or the stream stops. Servers without the channel
keep working with gamepads only.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant