Skip to content

Add MatrixRTC calls over LiveKit - #31

Open
TroyHernandez wants to merge 3 commits into
mainfrom
matrixrtc
Open

TroyHernandez wants to merge 3 commits into
mainfrom
matrixrtc

Conversation

@TroyHernandez

@TroyHernandez TroyHernandez commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

The Matrix side of MatrixRTC calls over LiveKit, following what Element Call and FluffyChat do on the wire.

Call layer (R/call.R)

  • mx_call_join(): finds the LiveKit JWT service (call members' foci_preferred, then MSC4143 rtc/transports, then well-known rtc_foci), trades an OpenID token for a media token, publishes this device's org.matrix.msc3401.call.member state event (state key _<user>_<device>_m.call, 4 h expiry), makes a 16-byte media key and sends it Olm-encrypted to every device in the call as io.element.call.encryption_keys. With connect = TRUE it joins the LiveKit room through livekitr with per-participant HKDF keys and a 256-slot key ring.
  • mx_call_handle(): applies a sync response: peers' keys, membership changes with the rotation policy (leaver → new key for everyone; joiner within 10 s → current key to the joiner; later joiner → new key), hourly membership refresh.
  • mx_call_poll() for programs without their own sync loop, mx_call_leave() clears the membership. mx_call_members(), mx_call_service_url(), mx_call_key_parse(), mx_call_key_plan() expose the pieces.

Underneath

  • mx_crypto_process_sync() returns other decrypted Olm to-device events in a new to_device element (envelope sender must match the plaintext sender).
  • mx_send_to_device_encrypted() / mx_crypto_encrypt_to_device() send an Olm-encrypted to-device event of any type.

Tests: 126 new expectations, two Olm accounts exchanging real payloads, HTTP mocked at the mx.api boundary. 918 total, all pass. R CMD check: 1 NOTE (livekitr not in a mainstream repository).

Dependencies, for review

  • Imports floor raised to mx.api 0.3.1.1 (Add the endpoints a MatrixRTC client needs mx.api#22) for the new endpoints.
  • livekitr added to Suggests; only the media functions need it.
  • CI will fail at the mx.api floor until 0.3.1.1 is on the drat, or the workflow pins an mx.api commit the way it pins mx.crypto.

End to end, locally: two mx.client devices on a Tuwunel 1.9.3 (open registration, server name with port, TLS front for the federation OpenID endpoint), lk-jwt-service 0.7.0 and livekit-server 1.13.7 with room.auto_create: false. Alice joined (the JWT service created the LiveKit room, identity @alice:…:DEVICE as expected), Bob joined and sent his key to Alice over Olm, Alice's next sync saw Bob's membership and sent hers; Alice's 1 s tone arrived at Bob decrypted (43 606 of 48 000 tone samples above the loudness threshold, attributed to Alice's identity). Leaving cleared the memberships. Two findings went into the last commit: a room created without a call client's power levels refuses org.matrix.msc3401.call.member at the default level 50 (now a clear error), and Tuwunel holds an empty incremental sync for 5 s whatever the timeout, so mx_call_poll() polls media first and documents the hold. Not yet tested with FluffyChat or Element Call as the other party.

🤖 Generated with Claude Code

mx_call_join() finds the LiveKit JWT service, trades an OpenID token
for a media token, publishes this device's
org.matrix.msc3401.call.member state event, and makes a media key that
goes Olm-encrypted to every device in the call as
io.element.call.encryption_keys. mx_call_handle() applies each sync:
peers' keys, membership changes with the Element/FluffyChat rotation
policy (leaver rotates, joiner within 10 s gets the current key), and
the hourly membership refresh. mx_call_connect() joins the LiveKit room
through livekitr with per-participant HKDF keys. mx_call_leave() clears
the membership.

Underneath: mx_crypto_process_sync() now returns other decrypted Olm
to-device events in to_device, and mx_send_to_device_encrypted() sends
an Olm-encrypted to-device event of any type.
Found running two devices through a call on a local Tuwunel 1.9.3 with
lk-jwt-service 0.7.0 and livekit-server 1.13.7: a room made without a
call client's power levels refuses org.matrix.msc3401.call.member at
the default level 50, and Tuwunel holds an empty incremental sync for
5 s whatever the timeout, so mx_call_poll() runs the media poll first
and documents the hold.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant