Repository navigation
Let a Matrix client take part in a room's call - #25
Merged
Merged
Conversation
chat_call_join() announces the client as a MatrixRTC member, obtains the media server's address and token, and exchanges per-member media keys over Olm for as long as the client polls; chat_call_media() hands a media side what it needs to connect; chat_call_updates() reports the keys and membership that arrived since the last look; chat_call_leave() withdraws. The media itself is the caller's, typically in another process. mx.client does the protocol. The adapter owns what the protocol needs and would otherwise be copied out of it: each operation is handed the live credentials (a call outlives relogins) and the adapter's Olm sessions, and the sessions a key send opens come back into the adapter's crypto state, or its next save would write an older set over them. The call reads the to-device events the adapter already decrypted for the poll rather than decrypting the sync again. A call that cannot be advanced is a warning, not an error: the cursor has moved, and the next poll retries. Needs an e2ee client and mx.client 0.2.1.1 (the floor in Suggests). The capability flag is calls, FALSE on every other adapter.
A consumer that builds a chat client per poll (corteza does) lost its calls with each client. The crypto context is interned per identity and store and holds the Olm sessions the call's keys live in, so the calls live there too: any client for the identity advances them.
The call surface's mx.client floor is 0.2.1.1, released on the drat and not yet on CRAN (0.2.1); mx.client 0.2.1.1 in turn imports mx.api 0.3.1.1, also drat-only for now. With the drat listed first, available.packages() takes its builds for both. The drat entry can go once CRAN carries these versions.
chat_poll() on an adapter with the calls capability returns `calls`: one notice per channel where someone other than the client announced or withdrew a call membership in this poll, with the identities now in the call (mx.client's reading of the events) and those who left. A bot that joins calls on demand acts on these; the call button in a Matrix client posts a membership event and nothing else, so this is the only signal a bot gets that a call started.
FluffyChat sends its call key as a room event rather than to-device, so the call needs the poll's decrypted room events beside its to-device events; they go to mx_call_handle() as processed$events. While a call is on, each poll logs what arrived by type (raw and decrypted to-device events, call keys in a room timeline), which is what told the first live call's missing key apart from a parse failure.
mx.client now hands back every decrypted room event, and a call's key
or a ring notification has no body; one of those became a chat_message
with an empty body, which a consumer's message loop tripped over
("argument is of length zero" in corteza's bot, once per call). Also
chat_call_media() reports the identities the client's key reached at
join, since the first send happens inside the join, before any poll.
A bot's top level never returns, so a deferred warning about an Olm message that could not be decrypted is never printed; a peer's media key dying that way left no trace. During a call those warnings are messages.
A call trims an interrupted reply by editing the message it posted to the room, and calls run in encrypted rooms, where chat_edit refused: an edit carries its replacement text in m.new_content, and the plain path would put that on the homeserver in the clear. But mx_send_encrypted takes the whole event content, so the relation and m.new_content can ride inside the Megolm ciphertext like any other field. The crypto send path gains an extra-content argument, chat_edit routes through it when the room is encrypted, and chat_capabilities()$edits is now TRUE on an e2ee client.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the signaling side of a MatrixRTC call to the Matrix adapter, so a bot can join the call a room's members started (Element Call, FluffyChat 2.10) while something else handles the media.
Surface
chat_call_join(client, channel, intent = "voice")announces the client as a call member, obtains the media server's address and token, and starts exchanging per-member media keys over Olm. Returns achat_call.chat_call_media(call): url, jwt, identity, own key, peers' keys. What a media side needs to connect.chat_call_updates(call): peer keys, own rotated key, members, ended, since the last look. Cleared on read.chat_call_leave(client, call).calls: TRUE on ane2eeMatrix client with mx.client >= 0.2.1.1, FALSE on loopback, IRC, Slack and Telegram, whose generics stop with the usual "not supported ... Check chat_capabilities()$calls".What the adapter owns (
R/matrix-call.R)mx.client does the protocol (
mx_call_join/mx_call_handle/mx_call_leave). The adapter supplies what those need and would otherwise be copied out of it:client\$env\$mxat the point of use, since a call outlives relogins;matrix_crypto_decrypt()keeps this poll's on the context, so the call reads peers' keys without decrypting the sync a second time (which would advance the ratchets twice).The hook runs in
chat_poll()after the cursor commit; a call that cannot be advanced warns and the next poll retries, rather than costing the poll its messages. Calls live on the identity's interned crypto context rather than the client object, because a consumer that builds a client per poll (corteza) would otherwise lose them.Also
.callconstructor seam for tests;inst/tinytest/test_matrix_call.R(65 assertions, no mx.client needed). Full suite 1473/0 locally.CI note: mx.client 0.2.1.1 is not on CRAN yet (CRAN has 0.2.1), and the workflow installs mx.client from CRAN, so the "Verify adapter dependencies" step will fail on the floor until it is released or the workflow takes mx.client from the drat (which has 0.2.1.1) for the interim.
Not yet exercised against a real homeserver and SFU; the consumer is cornball-ai/corteza's
livekit-callbranch.