Add MatrixRTC calls over LiveKit - #31
Open
TroyHernandez wants to merge 3 commits into
Open
TroyHernandez wants to merge 3 commits into
TroyHernandez wants to merge 3 commits into
Conversation
mx_call_join() finds the LiveKit JWT service, trades an OpenID token for a media token, publishes this device's org.matrix.msc3401.call.member state event, and makes a media key that goes Olm-encrypted to every device in the call as io.element.call.encryption_keys. mx_call_handle() applies each sync: peers' keys, membership changes with the Element/FluffyChat rotation policy (leaver rotates, joiner within 10 s gets the current key), and the hourly membership refresh. mx_call_connect() joins the LiveKit room through livekitr with per-participant HKDF keys. mx_call_leave() clears the membership. Underneath: mx_crypto_process_sync() now returns other decrypted Olm to-device events in to_device, and mx_send_to_device_encrypted() sends an Olm-encrypted to-device event of any type.
Found running two devices through a call on a local Tuwunel 1.9.3 with lk-jwt-service 0.7.0 and livekit-server 1.13.7: a room made without a call client's power levels refuses org.matrix.msc3401.call.member at the default level 50, and Tuwunel holds an empty incremental sync for 5 s whatever the timeout, so mx_call_poll() runs the media poll first and documents the hold.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Matrix side of MatrixRTC calls over LiveKit, following what Element Call and FluffyChat do on the wire.
Call layer (
R/call.R)mx_call_join(): finds the LiveKit JWT service (call members'foci_preferred, then MSC4143rtc/transports, then well-knownrtc_foci), trades an OpenID token for a media token, publishes this device'sorg.matrix.msc3401.call.memberstate event (state key_<user>_<device>_m.call, 4 h expiry), makes a 16-byte media key and sends it Olm-encrypted to every device in the call asio.element.call.encryption_keys. Withconnect = TRUEit joins the LiveKit room through livekitr with per-participant HKDF keys and a 256-slot key ring.mx_call_handle(): applies a sync response: peers' keys, membership changes with the rotation policy (leaver → new key for everyone; joiner within 10 s → current key to the joiner; later joiner → new key), hourly membership refresh.mx_call_poll()for programs without their own sync loop,mx_call_leave()clears the membership.mx_call_members(),mx_call_service_url(),mx_call_key_parse(),mx_call_key_plan()expose the pieces.Underneath
mx_crypto_process_sync()returns other decrypted Olm to-device events in a newto_deviceelement (envelope sender must match the plaintext sender).mx_send_to_device_encrypted()/mx_crypto_encrypt_to_device()send an Olm-encrypted to-device event of any type.Tests: 126 new expectations, two Olm accounts exchanging real payloads, HTTP mocked at the mx.api boundary. 918 total, all pass.
R CMD check: 1 NOTE (livekitr not in a mainstream repository).Dependencies, for review
End to end, locally: two mx.client devices on a Tuwunel 1.9.3 (open registration, server name with port, TLS front for the federation OpenID endpoint), lk-jwt-service 0.7.0 and livekit-server 1.13.7 with
room.auto_create: false. Alice joined (the JWT service created the LiveKit room, identity@alice:…:DEVICEas expected), Bob joined and sent his key to Alice over Olm, Alice's next sync saw Bob's membership and sent hers; Alice's 1 s tone arrived at Bob decrypted (43 606 of 48 000 tone samples above the loudness threshold, attributed to Alice's identity). Leaving cleared the memberships. Two findings went into the last commit: a room created without a call client's power levels refusesorg.matrix.msc3401.call.memberat the default level 50 (now a clear error), and Tuwunel holds an empty incremental sync for 5 s whatever the timeout, somx_call_poll()polls media first and documents the hold. Not yet tested with FluffyChat or Element Call as the other party.🤖 Generated with Claude Code