diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 318f621..c5f7da9 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -24,21 +24,34 @@ jobs: with: backend: RAPT + # The cross-signing APIs are development releases published to drat. + # Install them before install_deps checks the hard mx.api floor. rapt + # otherwise selects an older r2u binary by name, ignoring that floor. + - name: Install Matrix development dependencies + run: | + Rscript -e 'install.packages(c("curl", "jsonlite"))' + Rscript -e ' + if (isTRUE(getOption("rapt.enabled"))) rapt::disable() + floors <- c("mx.api" = "0.3.0.2", "mx.crypto" = "0.2.1.1") + utils::install.packages(names(floors), + repos = "https://cornball-ai.github.io/drat", + type = "source", dependencies = FALSE) + for (p in names(floors)) { + if (!requireNamespace(p, quietly = TRUE) || + utils::packageVersion(p) < floors[[p]]) + stop("CI needs ", p, " >= ", floors[[p]], + "; publish the upstream development release to drat first") + message(p, " ", utils::packageVersion(p)) + }' + - name: Dependencies run: ./run.sh install_deps - - name: Install suggested packages - # install_deps covers Imports only. Without mx.crypto every E2EE - # test calls exit_file() and CI goes green having exercised none of - # the encryption path -- which is how unverified homeserver keys - # shipped in the first place. simplermarkdown is needed for R CMD - # check to treat vignettes/e2ee.md as a vignette rather than a - # stray file. - # - # Both arrive as binaries (r2u on Linux, CRAN on macOS), so no Rust - # toolchain is needed despite mx.crypto being a Rust package. The - # runners ship rustc anyway if a source build is ever forced. - run: Rscript -e 'install.packages(c("mx.crypto", "simplermarkdown"))' + # mx.crypto above builds from its vendored Rust sources using the + # hosted runner's Rust toolchain. simplermarkdown is still needed for + # R CMD check to recognize vignettes/e2ee.md as a vignette. + - name: Install vignette builder + run: Rscript -e 'install.packages("simplermarkdown")' - name: Test run: ./run.sh run_tests diff --git a/DESCRIPTION b/DESCRIPTION index db3df7d..7e8a9b4 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,8 +1,8 @@ Package: mx.client Type: Package Title: Stateful Matrix Client Helpers -Version: 0.2.0.7 -Date: 2026-08-20 +Version: 0.2.0.8 +Date: 2026-09-04 Authors@R: c( person("Troy", "Hernandez", role = c("aut", "cre"), email = "troy@cornball.ai", @@ -22,12 +22,12 @@ Depends: R (>= 4.0) Imports: jsonlite, - mx.api (>= 0.3.0), + mx.api (>= 0.3.0.2), stats, tools, utils Suggests: - mx.crypto (>= 0.2.0), + mx.crypto (>= 0.2.1.1), simplermarkdown, tinytest VignetteBuilder: simplermarkdown diff --git a/NAMESPACE b/NAMESPACE index 8a6e956..4ca1533 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -12,6 +12,8 @@ export(mx_client_session) export(mx_crypto_account) export(mx_crypto_account_save) export(mx_crypto_claim_otks) +export(mx_crypto_cross_signing_bootstrap) +export(mx_crypto_cross_signing_load) export(mx_crypto_decrypt_event) export(mx_crypto_device_keys) export(mx_crypto_encrypt_event) @@ -19,9 +21,11 @@ export(mx_crypto_encrypt_for_devices) export(mx_crypto_handle_to_device) export(mx_crypto_inbound_session) export(mx_crypto_known_devices) +export(mx_crypto_mark_key_requests_sent) export(mx_crypto_process_sync) export(mx_crypto_publish_keys) export(mx_crypto_room_key_payload) +export(mx_crypto_send_key_requests) export(mx_crypto_sessions_load) export(mx_crypto_sessions_new) export(mx_crypto_sessions_save) diff --git a/NEWS.md b/NEWS.md index 44dbded..4d6f769 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,39 @@ +# mx.client 0.2.0.8 + +## New + +* Missing Megolm sessions now create persistent `m.room_key_request` + messages, deduplicate repeated requests, accept requested + `m.forwarded_room_key` only from this user's cross-signed devices, and + emit request cancellations after recovery. + +* `mx_crypto_cross_signing_bootstrap()` creates and durably stores Matrix + master, self-signing, and user-signing keys; publishes them through UIA; + signs the master with the current device; and signs that device with the + self-signing key. Existing server identities are never reset implicitly. +* `mx_crypto_known_devices()` now reports `cross_signed` and `master_key` + after verifying the server-provided master -> self-signing -> device chain. + The client's own devices additionally require a matching local + `self_master_key` pin to be marked cross-signed. +* `mx_crypto_process_sync()` now returns decryptable encrypted echoes of the + client's own messages with `is_self = TRUE`, as it already does for + cleartext echoes. Consumers can use this flag to ignore their own messages. +* Sent but unanswered room-key requests remain stored until recovery; + automatic expiry/pruning is deferred to a follow-up. + +## Fixes + +* Unsatisfied room-key requests now persist an explicit transport state and + retry with the same request id until successfully sent. +* Outbound Megolm sessions retain a local inbound copy so the client's own + echoed messages decrypt without requesting their keys from itself. +* Requests originating from this same device are not surfaced to key-sharing + policy, and forwarded keys never mark the original room sender verified. +* Malformed peer cross-signatures are treated as invalid instead of aborting + verification of every device returned by `/keys/query`. +* Rerunning cross-signing bootstrap skips valid signatures already on the + server. New request ids use integer milliseconds and one random suffix. + # mx.client 0.2.0.7 ## Fixes diff --git a/R/cross-signing.R b/R/cross-signing.R new file mode 100644 index 0000000..bcb03eb --- /dev/null +++ b/R/cross-signing.R @@ -0,0 +1,305 @@ +# Matrix cross-signing for long-lived bot devices. +# +# The three private signing keys stay in the existing crypto store and use +# the same encrypted vodozemac pickle plus mode-0600 file discipline as the +# Olm account. The server only receives the public CrossSigningKey objects. + +mx_crypto_cross_signing_path <- function(store_dir) { + file.path(store_dir, "cross-signing.json") +} + +mx_crypto_cross_signing_new <- function() { + mx_require_crypto() + list(master = mx.crypto::mxc_signing_key_new(), + self_signing = mx.crypto::mxc_signing_key_new(), + user_signing = mx.crypto::mxc_signing_key_new()) +} + +mx_crypto_cross_signing_save <- function(keys, store_dir) { + mx_require_crypto() + dir.create(store_dir, showWarnings = FALSE, recursive = TRUE) + pickle_key <- mx_crypto_key(store_dir) + blob <- list( + version = 1L, + master = mx.crypto::mxc_signing_key_pickle(keys$master, pickle_key), + self_signing = mx.crypto::mxc_signing_key_pickle( + keys$self_signing, pickle_key), + user_signing = mx.crypto::mxc_signing_key_pickle( + keys$user_signing, pickle_key) + ) + path <- mx_crypto_cross_signing_path(store_dir) + writeLines(jsonlite::toJSON(blob, auto_unbox = TRUE), path) + Sys.chmod(path, mode = "0600") + invisible(path) +} + +#' Load locally persisted Matrix cross-signing keys +#' +#' Returns NULL when this crypto store has never bootstrapped a cross-signing +#' identity. A present but malformed store is an error: generating over it +#' would silently reset the user's Matrix identity. +#' +#' @param store_dir Character. Crypto store directory. +#' @return A list containing master, self-signing and user-signing key handles, +#' or NULL. +#' @export +mx_crypto_cross_signing_load <- function(store_dir) { + mx_require_crypto() + path <- mx_crypto_cross_signing_path(store_dir) + if (!file.exists(path)) { + return(NULL) + } + blob <- jsonlite::fromJSON(paste(readLines(path, warn = FALSE), + collapse = "\n"), + simplifyVector = FALSE) + needed <- c("master", "self_signing", "user_signing") + if (!identical(as.integer(blob$version), 1L) || + any(vapply(needed, function(nm) is.null(blob[[nm]]), logical(1)))) { + stop("mx.client: invalid cross-signing store at ", path, + "; refusing to replace an identity whose private keys may be lost", + call. = FALSE) + } + pickle_key <- mx_crypto_key(store_dir) + list(master = mx.crypto::mxc_signing_key_unpickle(blob$master, pickle_key), + self_signing = mx.crypto::mxc_signing_key_unpickle( + blob$self_signing, pickle_key), + user_signing = mx.crypto::mxc_signing_key_unpickle( + blob$user_signing, pickle_key)) +} + +mx_crypto_signable_json <- function(object) { + object$signatures <- NULL + object$unsigned <- NULL + mx.api::mx_canonical_json(object) +} + +mx_crypto_add_signature <- function(object, signer, user_id, key_id) { + sig <- mx.crypto::mxc_signing_key_sign( + signer, mx_crypto_signable_json(object)) + own <- object$signatures[[user_id]] %||% list() + own[[key_id]] <- sig + object$signatures[[user_id]] <- own + object +} + +mx_crypto_cross_signing_key <- function(key, user_id, usage) { + public <- mx.crypto::mxc_signing_key_public(key) + list(user_id = user_id, usage = list(usage), + keys = stats::setNames(list(public), paste0("ed25519:", public))) +} + +mx_crypto_cross_signing_objects <- function(keys, user_id, device_account, + device_id) { + master_public <- mx.crypto::mxc_signing_key_public(keys$master) + master <- mx_crypto_cross_signing_key(keys$master, user_id, "master") + device_sig <- mx.crypto::mxc_account_sign( + device_account, mx_crypto_signable_json(master)) + master$signatures <- stats::setNames( + list(stats::setNames(list(device_sig), paste0("ed25519:", device_id))), + user_id) + + self <- mx_crypto_cross_signing_key( + keys$self_signing, user_id, "self_signing") + self <- mx_crypto_add_signature( + self, keys$master, user_id, paste0("ed25519:", master_public)) + user <- mx_crypto_cross_signing_key( + keys$user_signing, user_id, "user_signing") + user <- mx_crypto_add_signature( + user, keys$master, user_id, paste0("ed25519:", master_public)) + list(master = master, self_signing = self, user_signing = user) +} + +mx_crypto_cross_signing_public <- function(object, user_id, usage) { + if (!is.list(object) || !identical(object$user_id, user_id) || + !usage %in% unlist(object$usage, use.names = FALSE) || + length(object$keys) != 1L) { + stop("mx.client: invalid ", usage, " cross-signing key for ", user_id, + call. = FALSE) + } + key_id <- names(object$keys)[[1L]] + public <- as.character(object$keys[[1L]]) + if (!identical(key_id, paste0("ed25519:", public))) { + stop("mx.client: ", usage, + " cross-signing key id does not match its public key", + call. = FALSE) + } + public +} + +mx_crypto_signature_valid <- function(object, user_id, public, + key_id = paste0("ed25519:", public)) { + tryCatch({ + sig <- object$signatures[[user_id]][[key_id]] + !is.null(sig) && isTRUE(mx.crypto::mxc_ed25519_verify( + public, charToRaw(mx_crypto_signable_json(object)), sig)) + }, error = function(e) FALSE) +} + +# Verify that the server's master -> self-signing -> device chain is +# internally sound. Trusting the master remains a separate user decision. +mx_crypto_cross_signed_devices <- function(device_keys_map, master_keys, + self_signing_keys) { + out <- list() + for (uid in names(device_keys_map %||% list())) { + master <- master_keys[[uid]] + self <- self_signing_keys[[uid]] + if (is.null(master) || is.null(self)) { + next + } + chain <- tryCatch({ + master_public <- mx_crypto_cross_signing_public( + master, uid, "master") + self_public <- mx_crypto_cross_signing_public( + self, uid, "self_signing") + if (!mx_crypto_signature_valid(self, uid, master_public)) { + stop("self-signing key is not signed by the master key") + } + list(master = master_public, self = self_public) + }, error = function(e) { + warning("mx.client: invalid cross-signing chain for ", uid, + ": ", conditionMessage(e), call. = FALSE) + NULL + }) + if (is.null(chain)) { + next + } + for (device_id in names(device_keys_map[[uid]] %||% list())) { + device <- device_keys_map[[uid]][[device_id]] + if (mx_crypto_signature_valid(device, uid, chain$self)) { + out[[paste(uid, device_id, sep = "|")]] <- chain$master + } + } + } + out +} + +mx_crypto_upload_cross_signing <- function(client, objects, password = NULL, + auth = NULL) { + session <- mx_client_session(client) + upload <- function(completed_auth = NULL) { + mx.api::mx_keys_device_signing_upload( + session, master_key = objects$master, + self_signing_key = objects$self_signing, + user_signing_key = objects$user_signing, + auth = completed_auth) + } + if (!is.null(auth)) { + return(upload(auth)) + } + tryCatch(upload(), mx_error = function(e) { + if (!identical(e$status, 401L) || is.null(password)) { + stop(e) + } + uia_session <- e$body$session + if (is.null(uia_session)) { + stop("mx.client: cross-signing upload requested UIA but returned ", + "no UIA session id", call. = FALSE) + } + completed <- list( + type = "m.login.password", + identifier = list(type = "m.id.user", user = client$user_id), + password = password, session = uia_session) + upload(completed) + }) +} + +#' Bootstrap and sign this Matrix device's cross-signing identity +#' +#' Creates master, self-signing and user-signing keys only when neither the +#' local crypto store nor homeserver already has an identity. Existing local +#' keys are reused; a server/local mismatch aborts instead of resetting the +#' identity. The current device signs the master key and the self-signing key +#' signs the current device. +#' Valid signatures already returned by the homeserver are not re-uploaded. +#' +#' @param client Matrix client config. +#' @param device_account This device's persisted Olm account. +#' @param store_dir Character. Crypto store directory. +#' @param password Character or NULL. Account password used only for UIA. +#' @param auth Completed Matrix UIA object or NULL. +#' @return Public master, self-signing and user-signing key ids, invisibly. +#' @export +mx_crypto_cross_signing_bootstrap <- function(client, device_account, + store_dir, password = NULL, + auth = NULL) { + mx_require_crypto() + if (is.null(client$user_id) || is.null(client$device_id)) { + stop("mx.client: cross-signing requires user_id and device_id", + call. = FALSE) + } + query <- stats::setNames(list(list()), client$user_id) + current <- mx.api::mx_keys_query(mx_client_session(client), query) + server_master <- current$master_keys[[client$user_id]] + keys <- mx_crypto_cross_signing_load(store_dir) + + if (is.null(keys)) { + if (!is.null(server_master)) { + stop("mx.client: the homeserver already has a cross-signing ", + "identity for ", client$user_id, " but this crypto store ", + "does not hold its private keys; refusing an implicit reset", + call. = FALSE) + } + keys <- mx_crypto_cross_signing_new() + mx_crypto_cross_signing_save(keys, store_dir) + } + + objects <- mx_crypto_cross_signing_objects( + keys, client$user_id, device_account, client$device_id) + local_master <- mx_crypto_cross_signing_public( + objects$master, client$user_id, "master") + if (!is.null(server_master)) { + published <- mx_crypto_cross_signing_public( + server_master, client$user_id, "master") + if (!identical(local_master, published)) { + stop("mx.client: local and homeserver cross-signing master keys ", + "differ; refusing an implicit identity reset", call. = FALSE) + } + } else { + mx_crypto_upload_cross_signing(client, objects, password, auth) + } + + raw_device <- current$device_keys[[client$user_id]][[client$device_id]] + if (is.null(raw_device)) { + stop("mx.client: homeserver did not return this device's published keys", + call. = FALSE) + } + verified <- mx.crypto::mxc_verify_device_keys( + raw_device, client$user_id, client$device_id) + actual_ed <- mx.crypto::mxc_account_identity_keys(device_account)$ed25519 + if (!identical(verified$ed25519, actual_ed)) { + stop("mx.client: published device Ed25519 key does not match this ", + "crypto store", call. = FALSE) + } + self_public <- mx.crypto::mxc_signing_key_public(keys$self_signing) + signatures <- list() + if (!mx_crypto_signature_valid(raw_device, client$user_id, self_public)) { + # This endpoint adds signatures; include only the new one. + raw_device$signatures <- NULL + signatures[[client$device_id]] <- mx_crypto_add_signature( + raw_device, keys$self_signing, client$user_id, + paste0("ed25519:", self_public)) + } + if (!mx_crypto_signature_valid(server_master, client$user_id, actual_ed, + paste0("ed25519:", client$device_id))) { + # Existing signatures stay on the server and need not be re-uploaded. + master <- server_master %||% objects$master + master$signatures <- NULL + device_key_id <- paste0("ed25519:", client$device_id) + master$signatures[[client$user_id]][[device_key_id]] <- + mx.crypto::mxc_account_sign( + device_account, mx_crypto_signable_json(master)) + signatures[[local_master]] <- master + } + if (length(signatures)) { + response <- mx.api::mx_keys_signatures_upload( + mx_client_session(client), + stats::setNames(list(signatures), client$user_id)) + if (length(response$failures)) { + stop("mx.client: homeserver rejected cross-signing signatures for ", + paste(names(response$failures), collapse = ", "), call. = FALSE) + } + } + invisible(list( + master = local_master, self_signing = self_public, + user_signing = mx.crypto::mxc_signing_key_public(keys$user_signing))) +} diff --git a/R/e2ee.R b/R/e2ee.R index 4ae47c0..8be9146 100644 --- a/R/e2ee.R +++ b/R/e2ee.R @@ -5,12 +5,14 @@ # sending to-device, sending the event) stays with the caller / mx.api; # this layer is pure crypto state, so it is testable without a server. # -# A session set is a list with four named maps: +# A session set is a list with five named maps: # olm peer Curve25519 -> outbound Olm session (we encrypt to them) # olm_in peer Curve25519 -> inbound Olm session (they encrypt to us) # megolm_out room id -> list(session, shared = peer curves) # megolm_in "room|session_id" -> list(session, sender, sender_ed25519, # sender_bound) +# key_requests "room|session_id" -> outstanding m.room_key_request metadata, +# including whether transport succeeded # # megolm_in carries the sender identity the Olm payload claimed when the # key was shared, plus whether that claim was tied to a device whose @@ -22,13 +24,14 @@ #' Create an empty E2EE session set #' #' @return A session set: named lists \code{olm}, \code{olm_in}, -#' \code{megolm_out}, \code{megolm_in}. +#' \code{megolm_out}, \code{megolm_in}, and \code{key_requests}. #' @examples #' s <- mx_crypto_sessions_new() #' names(s) #' @export mx_crypto_sessions_new <- function() { - list(olm = list(), olm_in = list(), megolm_out = list(), megolm_in = list()) + list(olm = list(), olm_in = list(), megolm_out = list(), + megolm_in = list(), key_requests = list()) } #' Persist a session set to the crypto store @@ -62,12 +65,13 @@ mx_crypto_sessions_save <- function(sessions, store_dir) { list(session = mx.crypto::mxc_megolm_outbound_pickle(m$session, key), shared = as.list(m$shared)) }), - megolm_in = lapply(sessions$megolm_in, function(e) { + megolm_in = lapply(sessions$megolm_in, function(e) { list(session = mx.crypto::mxc_megolm_inbound_pickle(e$session, key), sender = e$sender %||% NA_character_, sender_ed25519 = e$sender_ed25519 %||% NA_character_, sender_bound = isTRUE(e$sender_bound)) - }) + }), + key_requests = sessions$key_requests %||% list() ) path <- file.path(store_dir, "sessions.json") writeLines(jsonlite::toJSON(blob, auto_unbox = TRUE), path) @@ -132,6 +136,7 @@ mx_crypto_sessions_load <- function(store_dir) { sender_ed25519 = e$sender_ed25519 %||% NA_character_, sender_bound = isTRUE(e$sender_bound)) } + out$key_requests <- blob$key_requests %||% list() out } @@ -188,6 +193,23 @@ mx_crypto_encrypt_for_devices <- function(account, sessions, room_id, mo <- list(session = mx.crypto::mxc_megolm_outbound_new(), shared = character()) } + # Keep an inbound copy of our outbound session. Homeservers echo our room + # events through /sync; without this, own echoes generate key requests to + # this same device. This also repairs older stores on their next send. + outbound_info <- mx.crypto::mxc_megolm_outbound_info(mo$session) + own_key <- paste(room_id, outbound_info$session_id, sep = "|") + if (is.null(sessions$megolm_in[[own_key]])) { + bound <- is.character(sender_user_id) && + length(sender_user_id) == 1L && + !is.na(sender_user_id) && nzchar(sender_user_id) + sessions$megolm_in[[own_key]] <- list( + session = mx.crypto::mxc_megolm_inbound_new( + outbound_info$session_key), + sender = sender_user_id %||% NA_character_, + sender_ed25519 = sender_ed25519, + sender_bound = bound) + } + to_device <- list() for (r in recipients) { @@ -229,10 +251,11 @@ mx_crypto_encrypt_for_devices <- function(account, sessions, room_id, #' Process a sync response: store room keys, decrypt room events #' #' Handles inbound to-device \code{m.room.encrypted} (Olm) messages, -#' storing any \code{m.room_key} as an inbound Megolm session, then -#' decrypts \code{m.room.encrypted} timeline events whose session is -#' known. Returns normalized text events in the same shape as -#' \code{mx_extract_text_events()}, plus the updated session set. +#' storing direct \code{m.room_key} events and requested +#' \code{m.forwarded_room_key} events as inbound Megolm sessions. It then +#' decrypts timeline events whose session is known and queues stable, +#' retryable \code{m.room_key_request} messages for missing sessions. The +#' caller sends those requests to this user's other devices. #' #' @param account An mx.crypto account handle. #' @param sessions A session set. @@ -243,12 +266,19 @@ mx_crypto_encrypt_for_devices <- function(account, sessions, room_id, #' carrying a claimed sender, and anyone who can reach this device can #' send one, so the claim is only worth something once it is matched #' against a device whose \code{device_keys} verified. Without this -#' list decrypted events always report \code{sender_verified = FALSE}: -#' they still decrypt, but nothing attests to who sent them. +#' list peers' decrypted events report \code{sender_verified = FALSE}: +#' they still decrypt, but nothing attests to who sent them. Own echoes +#' can be verified against the locally retained outbound session. Forwarded +#' keys require this user's cross-signed devices, queried with a trusted +#' local \code{self_master_key}; they never verify the original sender. #' @param self_id Character or NULL. This user's Matrix id, for -#' \code{is_self} tagging. -#' @return List with \code{events} (decrypted, normalized) and the updated -#' \code{sessions}. +#' \code{is_self} tagging and as the recipient of key requests. +#' @param self_device_id Character or NULL. This device id. Both this and +#' \code{self_id} are required to create room-key requests. +#' @return List with \code{events} (decrypted, normalized), updated +#' \code{sessions}, unsent \code{key_requests}, matching +#' \code{key_request_cancellations}, and \code{incoming_key_requests} +#' for a policy-aware sharing layer to inspect. #' @examples #' \donttest{ #' if (requireNamespace("mx.crypto", quietly = TRUE)) { @@ -262,12 +292,30 @@ mx_crypto_encrypt_for_devices <- function(account, sessions, room_id, #' @export mx_crypto_process_sync <- function(account, sessions, sync_resp, self_curve25519, self_id = NULL, - devices = NULL) { + devices = NULL, self_device_id = NULL) { mx_require_crypto() self_ed25519 <- mx.crypto::mxc_account_identity_keys(account)$ed25519 + sessions$key_requests <- sessions$key_requests %||% list() + cancellations <- list() + incoming_requests <- list() # 1. To-device: recover shared room keys. for (ev in sync_resp$to_device$events %||% list()) { + if (isTRUE(ev$type == "m.room_key_request")) { + c <- ev$content + # Wildcard delivery includes this device; do not surface our own + # request to a future key-sharing layer. + if (!is.null(self_device_id) && + identical(c$requesting_device_id, self_device_id)) next + if (isTRUE(c$action %in% c("request", "request_cancellation")) && + is.character(c$request_id) && nzchar(c$request_id) && + is.character(c$requesting_device_id) && + nzchar(c$requesting_device_id)) { + incoming_requests[[length(incoming_requests) + 1L]] <- list( + user_id = ev$sender, content = c) + } + next + } if (!isTRUE(ev$type == "m.room.encrypted") || !isTRUE(ev$content$algorithm == MX_OLM)) { next @@ -297,6 +345,9 @@ mx_crypto_process_sync <- function(account, sessions, sync_resp, } if (identical(decoded$type, "m.room_key")) { c <- decoded$content + if (!identical(c$algorithm, MX_MEGOLM)) { + next + } key <- paste(c$room_id, c$session_id, sep = "|") # Record who the payload claimed to be from, and separately # whether that claim was tied to a verified device. Keeping the @@ -309,6 +360,57 @@ mx_crypto_process_sync <- function(account, sessions, sync_resp, sender = decoded$sender, sender_ed25519 = decoded$keys$ed25519, sender_bound = chk$bound) + pending <- sessions$key_requests[[key]] + if (!is.null(pending)) { + cancellations[[length(cancellations) + 1L]] <- + mx_crypto_key_request_cancellation(pending) + sessions$key_requests[[key]] <- NULL + } + } else if (identical(decoded$type, "m.forwarded_room_key")) { + c <- decoded$content + key <- paste(c$room_id, c$session_id, sep = "|") + pending <- sessions$key_requests[[key]] + # Forwarded room keys are deliberately much stricter than an + # ordinary m.room_key. The Matrix spec limits this mechanism to + # verified devices owned by the requesting user. An unsolicited + # key, a key from another user, or one from a merely self-signed + # (not cross-signed) device is ignored. + forwarder <- mx_crypto_matching_device(decoded, sender, devices) + trusted_forwarder <- isTRUE(chk$bound) && + identical(decoded$sender, self_id) && + isTRUE(forwarder$cross_signed) + valid <- !is.null(pending) && trusted_forwarder && + identical(c$algorithm, MX_MEGOLM) && + identical(c$room_id, pending$room_id) && + identical(c$session_id, pending$session_id) && + is.character(c$session_key) && nzchar(c$session_key) + if (!valid) { + warning("mx.client: ignoring unsolicited or untrusted ", + "m.forwarded_room_key", call. = FALSE) + next + } + inbound <- tryCatch( + mx.crypto::mxc_megolm_inbound_import(c$session_key), + error = function(e) NULL) + if (is.null(inbound) || !identical( + mx.crypto::mxc_megolm_inbound_info(inbound)$session_id, + pending$session_id)) { + warning("mx.client: ignoring forwarded room key whose ", + "exported session does not match its session_id", + call. = FALSE) + next + } + # Olm authenticates the forwarding device, not the original room + # sender. Imported history must never be labelled sender-verified. + sessions$megolm_in[[key]] <- list( + session = inbound, + sender = pending$event_sender, + sender_ed25519 = c$sender_claimed_ed25519_key %||% + NA_character_, + sender_bound = FALSE) + cancellations[[length(cancellations) + 1L]] <- + mx_crypto_key_request_cancellation(pending) + sessions$key_requests[[key]] <- NULL } } @@ -324,6 +426,25 @@ mx_crypto_process_sync <- function(account, sessions, sync_resp, key <- paste(rid, ev$content$session_id, sep = "|") entry <- sessions$megolm_in[[key]] if (is.null(entry)) { + # Older stores can have an outbound session without its local + # inbound mirror. Do not request a key this device created. + own_out <- sessions$megolm_out[[rid]] + own_session_id <- if (is.null(own_out)) NULL else tryCatch( + mx.crypto::mxc_megolm_outbound_info( + own_out$session)$session_id, + error = function(e) NULL) + if (identical(ev$content$session_id, own_session_id)) { + next + } + if (!is.null(self_id) && !is.null(self_device_id) && + is.null(sessions$key_requests[[key]])) { + request <- mx_crypto_key_request( + self_id, self_device_id, rid, + ev$content$session_id, + ev$content$sender_key %||% NULL, + ev$sender %||% NA_character_) + sessions$key_requests[[key]] <- request + } next } dec <- tryCatch(mx_crypto_decrypt_event(entry$session, ev$content), @@ -365,5 +486,13 @@ mx_crypto_process_sync <- function(account, sessions, sync_resp, } } - list(events = events, sessions = sessions) + # A request is durable work until transport records a successful send. + # Requeue unsent entries even after the original timeline event is gone. + key_requests <- unname(Filter( + function(request) !isTRUE(request$sent), sessions$key_requests)) + + list(events = events, sessions = sessions, + key_requests = key_requests, + key_request_cancellations = cancellations, + incoming_key_requests = incoming_requests) } diff --git a/R/key-requests.R b/R/key-requests.R new file mode 100644 index 0000000..49d1388 --- /dev/null +++ b/R/key-requests.R @@ -0,0 +1,115 @@ +# Matrix room-key request bookkeeping and transport. + +mx_crypto_request_id <- function() { + millis <- sprintf("%.0f", as.numeric(Sys.time()) * 1000) + paste0(millis, ".", sprintf("%09d", sample.int(999999999L, 1L))) +} + +mx_crypto_key_request <- function(user_id, device_id, room_id, session_id, + sender_key = NULL, + event_sender = NA_character_) { + body <- list(algorithm = MX_MEGOLM, room_id = room_id, + session_id = session_id) + # Deprecated since Matrix v1.3, but clients are still asked to copy it + # when present. It is never used here to locate or trust a session. + if (!is.null(sender_key) && nzchar(sender_key)) { + body$sender_key <- sender_key + } + request_id <- mx_crypto_request_id() + list( + user_id = user_id, + room_id = room_id, + session_id = session_id, + event_sender = event_sender, + request_id = request_id, + sent = FALSE, + requesting_device_id = device_id, + content = list(action = "request", body = body, + request_id = request_id, requesting_device_id = device_id) + ) +} + +mx_crypto_key_request_cancellation <- function(request) { + list( + user_id = request$user_id, + room_id = request$room_id, + session_id = request$session_id, + request_id = request$request_id, + requesting_device_id = request$requesting_device_id, + content = list(action = "request_cancellation", + request_id = request$request_id, + requesting_device_id = request$requesting_device_id) + ) +} + +#' Mark successfully transmitted room-key requests +#' +#' Updates durable request records after transport succeeds. Requests remain +#' queued until marked, allowing a failed send to retry on a later poll even +#' after the original encrypted event has left the sync timeline. +#' +#' @param sessions An E2EE session set. +#' @param requests Request descriptors sent successfully. +#' @return The updated session set. +#' @export +mx_crypto_mark_key_requests_sent <- function(sessions, requests) { + if (!length(requests)) return(sessions) + ids <- vapply(requests, function(request) { + id <- request$request_id %||% request$content$request_id + if (!is.character(id) || length(id) != 1L || is.na(id) || + !nzchar(id)) { + return(NA_character_) + } + id + }, character(1)) + ids <- unique(ids[!is.na(ids)]) + for (key in names(sessions$key_requests %||% list())) { + if (isTRUE(sessions$key_requests[[key]]$request_id %in% ids)) { + sessions$key_requests[[key]]$sent <- TRUE + } + } + sessions +} + +# Return the verified device matching an authenticated Olm payload. +mx_crypto_matching_device <- function(decoded, sender_curve25519, devices) { + for (d in devices %||% list()) { + if (identical(d$user_id, decoded$sender) && + identical(d$ed25519, decoded$keys$ed25519) && + identical(d$curve25519, sender_curve25519)) { + return(d) + } + } + NULL +} + +#' Send queued Matrix room-key requests or cancellations +#' +#' Sends each descriptor returned by [mx_crypto_process_sync()] as an +#' unencrypted \code{m.room_key_request} to every device of this user. The +#' receiving clients apply their own verified-device sharing policy. +#' +#' @param client Matrix client config. +#' @param requests A list of request/cancellation descriptors returned by +#' [mx_crypto_process_sync()]. +#' @return The endpoint responses, invisibly. +#' @examples +#' \dontrun{ +#' mx_crypto_send_key_requests(client, result$key_requests) +#' } +#' @export +mx_crypto_send_key_requests <- function(client, requests) { + if (!length(requests)) { + return(invisible(list())) + } + s <- mx_client_session(client) + responses <- lapply(requests, function(request) { + if (is.null(request$user_id) || is.null(request$content)) { + stop("invalid room-key request descriptor", call. = FALSE) + } + messages <- stats::setNames( + list(list("*" = request$content)), request$user_id) + mx.api::mx_send_to_device(s, "m.room_key_request", messages) + }) + invisible(responses) +} diff --git a/R/transport.R b/R/transport.R index 5bc0351..da62216 100644 --- a/R/transport.R +++ b/R/transport.R @@ -71,20 +71,30 @@ mx_crypto_publish_keys <- function(client, account, store_dir, n_otks = 50L) { #' @param user_ids Character vector of Matrix user ids. #' @param strict Logical. Treat a \code{failures} map as an error rather #' than a warning. +#' @param self_master_key Character or NULL. Trusted local cross-signing +#' master public key for \code{client$user_id}. This user's devices are +#' cross-signed only when their verified chain matches this key. NULL +#' leaves this user's devices not cross-signed; other users' chains are +#' checked for internal consistency only. #' @return List of verified devices, each \code{list(user_id, device_id, -#' curve25519, ed25519)}. +#' curve25519, ed25519, cross_signed, master_key)}. \code{master_key} is +#' the server-reported key from a valid chain, even if it fails the pin. #' @examples #' \dontrun{ #' mx_crypto_known_devices(client, "@bob:example.org") #' } #' @export -mx_crypto_known_devices <- function(client, user_ids, strict = FALSE) { +mx_crypto_known_devices <- function(client, user_ids, strict = FALSE, + self_master_key = NULL) { mx_require_crypto() s <- mx_client_session(client) query <- stats::setNames(rep(list(list()), length(user_ids)), user_ids) resp <- mx.api::mx_keys_query(s, device_keys = query) mx_crypto_report_failures(resp$failures, "/keys/query", strict) - mx_crypto_verify_device_map(resp$device_keys) + mx_crypto_verify_device_map(resp$device_keys, resp$master_keys, + resp$self_signing_keys, + self_id = client$user_id, + self_master_key = self_master_key) } # A server we could not reach is not a user with no devices. @@ -122,9 +132,15 @@ mx_crypto_report_failures <- function(failures, what, strict = FALSE) { # user with no devices from a user whose every device was dropped here, # and those need different answers: the first is nobody to encrypt to, # the second is everybody unreachable. -mx_crypto_verify_device_map <- function(device_keys_map) { +mx_crypto_verify_device_map <- function(device_keys_map, master_keys = NULL, + self_signing_keys = NULL, + self_id = NULL, + self_master_key = NULL) { out <- list() seen <- list() + cross_signed <- mx_crypto_cross_signed_devices( + device_keys_map, master_keys %||% list(), + self_signing_keys %||% list()) for (uid in names(device_keys_map %||% list())) { devs <- device_keys_map[[uid]] for (dev in names(devs %||% list())) { @@ -139,8 +155,18 @@ mx_crypto_verify_device_map <- function(device_keys_map) { if (is.null(keys)) { next } - out[[length(out) + 1L]] <- list(user_id = uid, device_id = dev, - curve25519 = keys$curve25519, ed25519 = keys$ed25519) + chain_key <- paste(uid, dev, sep = "|") + chain_master <- cross_signed[[chain_key]] + trusted_chain <- !is.null(chain_master) + if (identical(uid, self_id)) { + trusted_chain <- trusted_chain && + identical(chain_master, self_master_key) + } + out[[length(out) + 1L]] <- list( + user_id = uid, device_id = dev, + curve25519 = keys$curve25519, ed25519 = keys$ed25519, + cross_signed = trusted_chain, + master_key = chain_master %||% NA_character_) } } attr(out, "seen") <- seen diff --git a/README.md b/README.md index 8401dcd..f98b35e 100644 --- a/README.md +++ b/README.md @@ -95,14 +95,30 @@ sessions <- res$sessions my_curve <- mx.crypto::mxc_account_identity_keys(acct)$curve25519 sync <- mx_sync_update(client)$sync out <- mx_crypto_process_sync(acct, sessions, sync, - my_curve, self_id = client$user_id) + my_curve, self_id = client$user_id, + self_device_id = client$device_id) +# Save ratchet state before network I/O. Mark only requests that were sent. +sessions <- out$sessions +mx_crypto_sessions_save(sessions, store) +sent <- list() +for (request in out$key_requests) { + ok <- tryCatch({ + mx_crypto_send_key_requests(client, list(request)); TRUE + }, error = function(e) FALSE) + if (ok) sent[[length(sent) + 1L]] <- request +} +sessions <- mx_crypto_mark_key_requests_sent(sessions, sent) +mx_crypto_sessions_save(sessions, store) +for (cancel in out$key_request_cancellations) { + try(mx_crypto_send_key_requests(client, list(cancel)), silent = TRUE) +} out$events # same shape as mx_extract_text_events() ``` -Security model, in brief: device keys are trusted on first use (no -cross-signing trust store yet), and there is no key-request or -forwarded-key flow. See `vignette("e2ee", package = "mx.client")` for -the full flow, what happens on the wire, and the current limitations. +Security model, in brief: self-signed device keys are checked before use; +cross-signing bootstrap verifies the master -> self-signing -> device chain; +and missing-session requests accept forwarded keys only from this user's +cross-signed devices. See `vignette("e2ee", package = "mx.client")`. ## The package family diff --git a/inst/skills/mx.client/matrix-messaging/SKILL.md b/inst/skills/mx.client/matrix-messaging/SKILL.md index 92d5970..f8ad261 100644 --- a/inst/skills/mx.client/matrix-messaging/SKILL.md +++ b/inst/skills/mx.client/matrix-messaging/SKILL.md @@ -134,9 +134,11 @@ mx.client::mx_with_relogin(client, function(cl) { Olm/Megolm send/receive orchestrated over `mx.crypto`, aimed at bots and controlled deployments. `mx.crypto` is a Suggests and is only touched from the E2EE entry points, so plaintext clients install and run without a Rust -toolchain. Security model is trust-on-first-use (no cross-signing trust store -yet, no key-request flow). Check a room's state with `mx_room_encrypted()` -before choosing the encrypted or plaintext path. +toolchain. Cross-signing bootstrap is fail-closed, and missing Megolm sessions +produce durable key requests that accept forwarded keys only from the same +user's cross-signed devices. Check a room's state with `mx_room_encrypted()` +before choosing the encrypted or plaintext path. This does not include SAS +verification or cross-user history recovery. The full flow (store, account, key publish, `mx_send_encrypted()`, `mx_crypto_process_sync()`) and its current limitations are in diff --git a/inst/tinytest/test_cross_signing.R b/inst/tinytest/test_cross_signing.R new file mode 100644 index 0000000..abcc258 --- /dev/null +++ b/inst/tinytest/test_cross_signing.R @@ -0,0 +1,225 @@ +library(tinytest) + +if (!requireNamespace("mx.crypto", quietly = TRUE) || + utils::packageVersion("mx.crypto") < "0.2.1.1" || + utils::packageVersion("mx.api") < "0.3.0.2") { + exit_file("cross-signing dependencies are not available") +} +library(mx.client) + +local({ +UID <- "@tiny:example.org" +DEV <- "TINYDEV" +device <- mx.crypto::mxc_account_new() +store <- tempfile("cross-signing-") +dir.create(store, recursive = TRUE) +on.exit(unlink(store, recursive = TRUE), add = TRUE) + +keys <- mx.client:::mx_crypto_cross_signing_new() +mx.client:::mx_crypto_cross_signing_save(keys, store) +loaded <- mx_crypto_cross_signing_load(store) +expect_identical(mx.crypto::mxc_signing_key_public(loaded$master), + mx.crypto::mxc_signing_key_public(keys$master)) +expect_identical(as.octmode(file.info(file.path( + store, "cross-signing.json"))$mode), as.octmode("600")) + +objects <- mx.client:::mx_crypto_cross_signing_objects( + loaded, UID, device, DEV) +master_public <- mx.client:::mx_crypto_cross_signing_public( + objects$master, UID, "master") +self_public <- mx.client:::mx_crypto_cross_signing_public( + objects$self_signing, UID, "self_signing") +expect_true(mx.client:::mx_crypto_signature_valid( + objects$self_signing, UID, master_public)) +expect_true(mx.client:::mx_crypto_signature_valid( + objects$user_signing, UID, master_public)) + +raw_device <- mx_crypto_device_keys(device, UID, DEV) +signed_device <- mx.client:::mx_crypto_add_signature( + raw_device, loaded$self_signing, UID, paste0("ed25519:", self_public)) +chain <- mx.client:::mx_crypto_cross_signed_devices( + setNames(list(setNames(list(signed_device), DEV)), UID), + setNames(list(objects$master), UID), + setNames(list(objects$self_signing), UID)) +expect_identical(chain[[paste(UID, DEV, sep = "|")]], master_public) + +# Our own chains must match the local master; another user's valid chain +# remains available even when the homeserver substitutes our identity. +local({ + other_uid <- "@peer:example.org" + other_objects <- mx.client:::mx_crypto_cross_signing_objects( + loaded, other_uid, device, DEV) + other_device <- mx.client:::mx_crypto_add_signature( + mx_crypto_device_keys(device, other_uid, DEV), loaded$self_signing, + other_uid, paste0("ed25519:", self_public)) + response <- list( + device_keys = setNames(list(setNames(list(signed_device), DEV), + setNames(list(other_device), DEV)), + c(UID, other_uid)), + master_keys = setNames(list(objects$master, other_objects$master), + c(UID, other_uid)), + self_signing_keys = setNames( + list(objects$self_signing, other_objects$self_signing), + c(UID, other_uid))) + original_query <- mx.api::mx_keys_query + assignInNamespace("mx_keys_query", function(...) response, ns = "mx.api") + on.exit(assignInNamespace("mx_keys_query", original_query, ns = "mx.api")) + client <- list(server = "https://example.invalid", token = "tok", + user_id = UID, device_id = DEV) + query <- function(pin = NULL) mx_crypto_known_devices( + client, c(UID, other_uid), self_master_key = pin) + trusted <- query(master_public) + expect_true(trusted[[1]]$cross_signed) + expect_true(trusted[[2]]$cross_signed) + unpinned <- query() + expect_false(unpinned[[1]]$cross_signed) + expect_true(unpinned[[2]]$cross_signed) + expect_identical(unpinned[[1]]$ed25519, trusted[[1]]$ed25519) + + # The replacement chain is cryptographically valid, but not our identity. + forged_keys <- mx.client:::mx_crypto_cross_signing_new() + forged <- mx.client:::mx_crypto_cross_signing_objects( + forged_keys, UID, device, DEV) + response$master_keys[[UID]] <- forged$master + response$self_signing_keys[[UID]] <- forged$self_signing + response$device_keys[[UID]][[DEV]] <- mx.client:::mx_crypto_add_signature( + raw_device, forged_keys$self_signing, UID, + paste0("ed25519:", mx.crypto::mxc_signing_key_public( + forged_keys$self_signing))) + replaced <- query(master_public) + expect_false(replaced[[1]]$cross_signed) + expect_true(replaced[[2]]$cross_signed) + expect_identical(replaced[[1]]$master_key, + mx.crypto::mxc_signing_key_public(forged_keys$master)) + expect_identical(replaced[[1]]$curve25519, trusted[[1]]$curve25519) + expect_true(query(replaced[[1]]$master_key)[[1]]$cross_signed) +}) + +# Tampering either link invalidates the chain. +bad_self <- objects$self_signing +bad_self$usage <- list("user_signing") +expect_warning(bad <- mx.client:::mx_crypto_cross_signed_devices( + setNames(list(setNames(list(signed_device), DEV)), UID), + setNames(list(objects$master), UID), + setNames(list(bad_self), UID))) +expect_equal(length(bad), 0L) +bad_device <- signed_device +bad_device$algorithms <- list("m.megolm.v1.aes-sha2") +expect_equal(length(mx.client:::mx_crypto_cross_signed_devices( + setNames(list(setNames(list(bad_device), DEV)), UID), + setNames(list(objects$master), UID), + setNames(list(objects$self_signing), UID))), 0L) + +# Malformed signature bytes from one peer are invalid, not fatal. +malformed_device <- signed_device +malformed_device$signatures[[UID]][[paste0("ed25519:", self_public)]] <- + "*** not base64 ***" +expect_false(mx.client:::mx_crypto_signature_valid( + malformed_device, UID, self_public)) +expect_equal(length(mx.client:::mx_crypto_cross_signed_devices( + setNames(list(setNames(list(malformed_device), DEV)), UID), + setNames(list(objects$master), UID), + setNames(list(objects$self_signing), UID))), 0L) + +# Bootstrap performs a UIA retry, then uploads master and device signatures. +local({ + client <- list(server = "https://example.invalid", token = "tok", + user_id = UID, device_id = DEV) + fresh_store <- tempfile("bootstrap-") + dir.create(fresh_store, recursive = TRUE) + on.exit(unlink(fresh_store, recursive = TRUE), add = TRUE) + uploaded <- list() + signature_body <- NULL + attempts <- 0L + original_query <- mx.api::mx_keys_query + original_cross <- mx.api::mx_keys_device_signing_upload + original_signatures <- mx.api::mx_keys_signatures_upload + signature_calls <- 0L + current <- list( + device_keys = setNames(list(setNames(list(raw_device), DEV)), UID), + master_keys = list(), self_signing_keys = list()) + assignInNamespace("mx_keys_query", function(...) current, ns = "mx.api") + assignInNamespace("mx_keys_device_signing_upload", function( + session, master_key = NULL, self_signing_key = NULL, + user_signing_key = NULL, auth = NULL) { + attempts <<- attempts + 1L + uploaded[[attempts]] <<- list(master = master_key, self = self_signing_key, + user = user_signing_key, auth = auth) + if (attempts == 1L) { + cond <- structure(list(message = "UIA required", call = NULL, + status = 401L, body = list(session = "uia-1")), + class = c("mx_error", "error", "condition")) + stop(cond) + } + list() + }, ns = "mx.api") + assignInNamespace("mx_keys_signatures_upload", function(session, signatures) { + signature_calls <<- signature_calls + 1L + signature_body <<- signatures + list(failures = list()) + }, ns = "mx.api") + on.exit({ + assignInNamespace("mx_keys_query", original_query, ns = "mx.api") + assignInNamespace("mx_keys_device_signing_upload", original_cross, + ns = "mx.api") + assignInNamespace("mx_keys_signatures_upload", original_signatures, + ns = "mx.api") + }, add = TRUE) + + result <- mx_crypto_cross_signing_bootstrap( + client, device, fresh_store, password = "not-persisted") + expect_equal(attempts, 2L) + expect_null(uploaded[[1]]$auth) + expect_identical(uploaded[[2]]$auth$session, "uia-1") + expect_identical(uploaded[[2]]$auth$password, "not-persisted") + expect_true(!is.null(signature_body[[UID]][[DEV]])) + expect_true(!is.null(signature_body[[UID]][[result$master]])) + persisted <- paste(readLines(file.path(fresh_store, "cross-signing.json"), + warn = FALSE), collapse = "") + expect_false(grepl("not-persisted", persisted, fixed = TRUE)) + + # Reruns skip both signatures already reported by the server. + current$master_keys[[UID]] <- signature_body[[UID]][[result$master]] + current$self_signing_keys[[UID]] <- uploaded[[2]]$self + # The upload endpoint merges signatures into the existing object. + current$device_keys[[UID]][[DEV]] <- utils::modifyList( + raw_device, signature_body[[UID]][[DEV]]) + complete <- current + expect_identical(mx_crypto_cross_signing_bootstrap( + client, device, fresh_store), result) + expect_equal(signature_calls, 1L) + expect_equal(attempts, 2L) + + # Upload only the missing link, preserving an existing valid signature. + current$master_keys[[UID]]$signatures <- NULL + current$master_keys[[UID]]$signatures[["@peer:example.org"]] <- + list("ed25519:PEER" = "already-stored") + mx_crypto_cross_signing_bootstrap(client, device, fresh_store) + expect_equal(signature_calls, 2L) + expect_identical(names(signature_body[[UID]]), result$master) + expect_identical(names(signature_body[[UID]][[result$master]]$signatures), UID) + expect_identical(names(signature_body[[UID]][[result$master]]$signatures[[UID]]), + paste0("ed25519:", DEV)) + expect_true(mx.client:::mx_crypto_signature_valid( + signature_body[[UID]][[result$master]], UID, + mx.crypto::mxc_account_identity_keys(device)$ed25519, + paste0("ed25519:", DEV))) + current <- complete + current$device_keys[[UID]][[DEV]]$signatures[[UID]][[ + paste0("ed25519:", result$self_signing)]] <- "*** invalid ***" + mx_crypto_cross_signing_bootstrap(client, device, fresh_store) + expect_equal(signature_calls, 3L) + expect_identical(names(signature_body[[UID]]), DEV) + expect_identical(names(signature_body[[UID]][[DEV]]$signatures[[UID]]), + paste0("ed25519:", result$self_signing)) + expect_true(mx.client:::mx_crypto_signature_valid( + signature_body[[UID]][[DEV]], UID, result$self_signing)) + + # A published identity change is still refused before signature upload. + current <- complete + current$master_keys[[UID]] <- objects$master + expect_error(mx_crypto_cross_signing_bootstrap( + client, device, fresh_store), "master keys differ") + expect_equal(signature_calls, 3L) +}) +}) diff --git a/inst/tinytest/test_key_requests.R b/inst/tinytest/test_key_requests.R new file mode 100644 index 0000000..d6d9efb --- /dev/null +++ b/inst/tinytest/test_key_requests.R @@ -0,0 +1,202 @@ +library(tinytest) + +if (!requireNamespace("mx.crypto", quietly = TRUE) || + utils::packageVersion("mx.crypto") < "0.2.1.1") { + exit_file("mx.crypto >= 0.2.1.1 not available (needs a Rust toolchain)") +} +library(mx.client) + +local({ +uid <- "@bob:example.org" +device_id <- "BOB-NEW" +room_id <- "!history:example.org" + +# Alice has an existing outbound session, but Bob's new device missed the +# original m.room_key share. +alice <- mx.crypto::mxc_account_new() +alice_keys <- mx.crypto::mxc_account_identity_keys(alice) +outbound <- mx.crypto::mxc_megolm_outbound_new() +outbound_info <- mx.crypto::mxc_megolm_outbound_info(outbound) +encrypted <- mx_crypto_encrypt_event( + outbound, list(msgtype = "m.text", body = "recover me"), room_id, + alice_keys$curve25519, "ALICE") + +bob <- mx.crypto::mxc_account_new() +bob_keys <- mx.crypto::mxc_account_identity_keys(bob) +timeline <- list( + to_device = list(events = list()), + rooms = list(join = stats::setNames(list(list(timeline = list(events = + list(list(type = "m.room.encrypted", event_id = "$missing", + sender = "@alice:example.org", content = encrypted))))), + room_id)) +) + +missing <- mx_crypto_process_sync( + bob, mx_crypto_sessions_new(), timeline, bob_keys$curve25519, + self_id = uid, self_device_id = device_id) +expect_equal(length(missing$events), 0L) +expect_equal(length(missing$key_requests), 1L) +request <- missing$key_requests[[1]] +expect_equal(request$user_id, uid) # requests go to our own other devices +expect_equal(request$content$action, "request") +expect_equal(request$content$requesting_device_id, device_id) +expect_equal(request$content$body$room_id, room_id) +expect_equal(request$content$body$session_id, outbound_info$session_id) +expect_equal(request$content$body$sender_key, alice_keys$curve25519) +expect_true(nzchar(request$content$request_id)) +expect_true(grepl("^[0-9]+[.][0-9]{9}$", request$content$request_id)) +expect_identical(request$content$request_id, request$request_id) +expect_false(request$sent) + +# Outstanding unsent requests survive restart and retry with the same id. +store <- tempfile("key-request-store-") +on.exit(unlink(store, recursive = TRUE), add = TRUE) +mx_crypto_sessions_save(missing$sessions, store) +persisted <- mx_crypto_sessions_load(store) +expect_equal(length(persisted$key_requests), 1L) +retry <- mx_crypto_process_sync( + bob, persisted, timeline, bob_keys$curve25519, + self_id = uid, self_device_id = device_id) +expect_equal(length(retry$key_requests), 1L) +expect_equal(retry$key_requests[[1]]$request_id, request$request_id) +sent_sessions <- mx_crypto_mark_key_requests_sent( + retry$sessions, retry$key_requests) +expect_true(sent_sessions$key_requests[[1]]$sent) +mx_crypto_sessions_save(sent_sessions, store) +persisted_sent <- mx_crypto_sessions_load(store) +again <- mx_crypto_process_sync( + bob, persisted_sent, timeline, bob_keys$curve25519, + self_id = uid, self_device_id = device_id) +expect_equal(length(again$key_requests), 0L) +expect_equal(again$sessions$key_requests[[1]]$request_id, + request$request_id) + +# A cross-signed device belonging to Bob forwards the exported session over +# Olm. The pending-request match and imported session-id check happen before +# the key is installed. +other <- mx.crypto::mxc_account_new() +other_keys <- mx.crypto::mxc_account_identity_keys(other) +mx.crypto::mxc_account_generate_one_time_keys(bob, 1L) +bob_otk <- mx.crypto::mxc_account_one_time_keys(bob)[[1]] +olm <- mx.crypto::mxc_olm_create_outbound( + other, bob_keys$curve25519, bob_otk) +original_inbound <- mx.crypto::mxc_megolm_inbound_new( + outbound_info$session_key) +forwarded_key <- mx.crypto::mxc_megolm_inbound_export(original_inbound) +forwarded_plain <- list( + type = "m.forwarded_room_key", + content = list( + algorithm = "m.megolm.v1.aes-sha2", + room_id = room_id, + session_id = outbound_info$session_id, + session_key = forwarded_key, + sender_key = alice_keys$curve25519, + sender_claimed_ed25519_key = alice_keys$ed25519, + forwarding_curve25519_key_chain = list()), + sender = uid, + recipient = uid, + recipient_keys = list(ed25519 = bob_keys$ed25519), + keys = list(ed25519 = other_keys$ed25519)) +olm_ciphertext <- mx.crypto::mxc_olm_encrypt( + olm, charToRaw(mx.api::mx_canonical_json(forwarded_plain))) +to_device <- list( + type = "m.room.encrypted", sender = uid, + content = list( + algorithm = "m.olm.v1.curve25519-aes-sha2", + sender_key = other_keys$curve25519, + ciphertext = stats::setNames( + list(list(type = olm_ciphertext$type, body = olm_ciphertext$body)), + bob_keys$curve25519))) +recovery_sync <- timeline +recovery_sync$to_device$events <- list(to_device) +devices <- list( + list(user_id = uid, device_id = "BOB-OLD", + curve25519 = other_keys$curve25519, + ed25519 = other_keys$ed25519, cross_signed = TRUE), + list(user_id = "@alice:example.org", device_id = "ALICE", + curve25519 = alice_keys$curve25519, + ed25519 = alice_keys$ed25519, cross_signed = TRUE)) +recovered <- mx_crypto_process_sync( + bob, again$sessions, recovery_sync, bob_keys$curve25519, + self_id = uid, self_device_id = device_id, devices = devices) +expect_equal(length(recovered$events), 1L) +expect_equal(recovered$events[[1]]$body, "recover me") +expect_false(recovered$events[[1]]$sender_verified) +expect_equal(length(recovered$key_request_cancellations), 1L) +expect_equal(recovered$key_request_cancellations[[1]]$content$action, + "request_cancellation") +expect_equal(recovered$key_request_cancellations[[1]]$content$request_id, + request$request_id) +expect_equal(length(recovered$sessions$key_requests), 0L) + +# Plaintext request events are surfaced for a separate, policy-aware sharing +# layer; processing sync never auto-forwards history. +incoming_sync <- list( + to_device = list(events = list( + list( + type = "m.room_key_request", sender = uid, + content = list(action = "request", request_id = "req-1", + requesting_device_id = "BOB-OLD", + body = request$content$body)), + list(type = "m.room_key_request", sender = uid, + content = list(action = "request", request_id = "req-self", + requesting_device_id = device_id, + body = request$content$body)))), + rooms = list(join = list())) +incoming <- mx_crypto_process_sync( + bob, recovered$sessions, incoming_sync, bob_keys$curve25519, + self_id = uid, self_device_id = device_id, devices = devices) + +# New outbound Megolm sessions retain a local inbound copy, so own echoes +# decrypt without requesting their key from this same device. +self_out <- mx_crypto_encrypt_for_devices( + bob, mx_crypto_sessions_new(), room_id, + list(msgtype = "m.text", body = "own echo"), + bob_keys$curve25519, device_id, sender_user_id = uid) +self_timeline <- list( + to_device = list(events = list()), + rooms = list(join = stats::setNames(list(list(timeline = list(events = + list(list(type = "m.room.encrypted", event_id = "$self", + sender = uid, content = self_out$event))))), room_id))) +self_result <- mx_crypto_process_sync( + bob, self_out$sessions, self_timeline, bob_keys$curve25519, + self_id = uid, self_device_id = device_id) +expect_equal(length(self_result$events), 1L) +expect_equal(self_result$events[[1]]$body, "own echo") +expect_true(self_result$events[[1]]$sender_verified) +expect_equal(length(self_result$key_requests), 0L) + +# A legacy store may have the outbound session but no local inbound mirror; +# its echo is skipped without generating a request to ourselves. +legacy_self <- self_out$sessions +legacy_self$megolm_in <- list() +legacy_result <- mx_crypto_process_sync( + bob, legacy_self, self_timeline, bob_keys$curve25519, + self_id = uid, self_device_id = device_id) +expect_equal(length(legacy_result$events), 0L) +expect_equal(length(legacy_result$key_requests), 0L) +expect_equal(length(incoming$incoming_key_requests), 1L) +expect_equal(incoming$incoming_key_requests[[1]]$content$request_id, "req-1") + +# Transport uses an unencrypted wildcard to-device event for this user. +local({ + sent <- NULL + old_session <- mx.client::mx_client_session + old_send <- mx.api::mx_send_to_device + assignInNamespace("mx_client_session", function(client, ...) list(ok = TRUE), + ns = "mx.client") + assignInNamespace("mx_send_to_device", function(session, event_type, + messages, txn_id = NULL) { + sent <<- list(type = event_type, messages = messages) + list() + }, ns = "mx.api") + on.exit({ + assignInNamespace("mx_client_session", old_session, ns = "mx.client") + assignInNamespace("mx_send_to_device", old_send, ns = "mx.api") + }) + mx_crypto_send_key_requests(list(), list(request)) + expect_equal(sent$type, "m.room_key_request") + expect_equal(sent$messages[[uid]][["*"]]$request_id, + request$request_id) +}) +}) diff --git a/man/mx_crypto_cross_signing_bootstrap.Rd b/man/mx_crypto_cross_signing_bootstrap.Rd new file mode 100644 index 0000000..062acfc --- /dev/null +++ b/man/mx_crypto_cross_signing_bootstrap.Rd @@ -0,0 +1,35 @@ +% tinyrox says don't edit this manually, but it can't stop you! +\name{mx_crypto_cross_signing_bootstrap} +\alias{mx_crypto_cross_signing_bootstrap} +\title{Bootstrap and sign this Matrix device's cross-signing identity} +\usage{ +mx_crypto_cross_signing_bootstrap( + client, + device_account, + store_dir, + password = NULL, + auth = NULL +) +} +\arguments{ +\item{client}{Matrix client config.} + +\item{device_account}{This device's persisted Olm account.} + +\item{store_dir}{Character. Crypto store directory.} + +\item{password}{Character or NULL. Account password used only for UIA.} + +\item{auth}{Completed Matrix UIA object or NULL.} +} +\value{ +Public master, self-signing and user-signing key ids, invisibly. +} +\description{ +Creates master, self-signing and user-signing keys only when neither the +local crypto store nor homeserver already has an identity. Existing local +keys are reused; a server/local mismatch aborts instead of resetting the +identity. The current device signs the master key and the self-signing key +signs the current device. +Valid signatures already returned by the homeserver are not re-uploaded. +} diff --git a/man/mx_crypto_cross_signing_load.Rd b/man/mx_crypto_cross_signing_load.Rd new file mode 100644 index 0000000..9805569 --- /dev/null +++ b/man/mx_crypto_cross_signing_load.Rd @@ -0,0 +1,19 @@ +% tinyrox says don't edit this manually, but it can't stop you! +\name{mx_crypto_cross_signing_load} +\alias{mx_crypto_cross_signing_load} +\title{Load locally persisted Matrix cross-signing keys} +\usage{ +mx_crypto_cross_signing_load(store_dir) +} +\arguments{ +\item{store_dir}{Character. Crypto store directory.} +} +\value{ +A list containing master, self-signing and user-signing key handles, + or NULL. +} +\description{ +Returns NULL when this crypto store has never bootstrapped a cross-signing +identity. A present but malformed store is an error: generating over it +would silently reset the user's Matrix identity. +} diff --git a/man/mx_crypto_known_devices.Rd b/man/mx_crypto_known_devices.Rd index bf9ebff..b0b0806 100644 --- a/man/mx_crypto_known_devices.Rd +++ b/man/mx_crypto_known_devices.Rd @@ -3,7 +3,12 @@ \alias{mx_crypto_known_devices} \title{List the devices (and identity keys) of some users} \usage{ -mx_crypto_known_devices(client, user_ids, strict = FALSE) +mx_crypto_known_devices( + client, + user_ids, + strict = FALSE, + self_master_key = NULL +) } \arguments{ \item{client}{Matrix client config.} @@ -12,10 +17,17 @@ mx_crypto_known_devices(client, user_ids, strict = FALSE) \item{strict}{Logical. Treat a \code{failures} map as an error rather than a warning.} + +\item{self_master_key}{Character or NULL. Trusted local cross-signing +master public key for \code{client$user_id}. This user's devices are +cross-signed only when their verified chain matches this key. NULL +leaves this user's devices not cross-signed; other users' chains are +checked for internal consistency only.} } \value{ List of verified devices, each \code{list(user_id, device_id, - curve25519, ed25519)}. + curve25519, ed25519, cross_signed, master_key)}. \code{master_key} is + the server-reported key from a valid chain, even if it fails the pin. } \description{ Queries \code{/keys/query} and flattens the result to a list of devices. diff --git a/man/mx_crypto_mark_key_requests_sent.Rd b/man/mx_crypto_mark_key_requests_sent.Rd new file mode 100644 index 0000000..1628fad --- /dev/null +++ b/man/mx_crypto_mark_key_requests_sent.Rd @@ -0,0 +1,20 @@ +% tinyrox says don't edit this manually, but it can't stop you! +\name{mx_crypto_mark_key_requests_sent} +\alias{mx_crypto_mark_key_requests_sent} +\title{Mark successfully transmitted room-key requests} +\usage{ +mx_crypto_mark_key_requests_sent(sessions, requests) +} +\arguments{ +\item{sessions}{An E2EE session set.} + +\item{requests}{Request descriptors sent successfully.} +} +\value{ +The updated session set. +} +\description{ +Updates durable request records after transport succeeds. Requests remain +queued until marked, allowing a failed send to retry on a later poll even +after the original encrypted event has left the sync timeline. +} diff --git a/man/mx_crypto_process_sync.Rd b/man/mx_crypto_process_sync.Rd index 9cfa2b0..5ecbf2d 100644 --- a/man/mx_crypto_process_sync.Rd +++ b/man/mx_crypto_process_sync.Rd @@ -9,7 +9,8 @@ mx_crypto_process_sync( sync_resp, self_curve25519, self_id = NULL, - devices = NULL + devices = NULL, + self_device_id = NULL ) } \arguments{ @@ -22,26 +23,35 @@ mx_crypto_process_sync( \item{self_curve25519}{Character. This device's Curve25519 key.} \item{self_id}{Character or NULL. This user's Matrix id, for -\code{is_self} tagging.} +\code{is_self} tagging and as the recipient of key requests.} \item{devices}{List of verified devices from \code{mx_crypto_known_devices()}, or NULL. Room keys arrive over Olm carrying a claimed sender, and anyone who can reach this device can send one, so the claim is only worth something once it is matched against a device whose \code{device_keys} verified. Without this -list decrypted events always report \code{sender_verified = FALSE}: -they still decrypt, but nothing attests to who sent them.} +list peers' decrypted events report \code{sender_verified = FALSE}: +they still decrypt, but nothing attests to who sent them. Own echoes +can be verified against the locally retained outbound session. Forwarded +keys require this user's cross-signed devices, queried with a trusted +local \code{self_master_key}; they never verify the original sender.} + +\item{self_device_id}{Character or NULL. This device id. Both this and +\code{self_id} are required to create room-key requests.} } \value{ -List with \code{events} (decrypted, normalized) and the updated - \code{sessions}. +List with \code{events} (decrypted, normalized), updated + \code{sessions}, unsent \code{key_requests}, matching + \code{key_request_cancellations}, and \code{incoming_key_requests} + for a policy-aware sharing layer to inspect. } \description{ Handles inbound to-device \code{m.room.encrypted} (Olm) messages, -storing any \code{m.room_key} as an inbound Megolm session, then -decrypts \code{m.room.encrypted} timeline events whose session is -known. Returns normalized text events in the same shape as -\code{mx_extract_text_events()}, plus the updated session set. +storing direct \code{m.room_key} events and requested +\code{m.forwarded_room_key} events as inbound Megolm sessions. It then +decrypts timeline events whose session is known and queues stable, +retryable \code{m.room_key_request} messages for missing sessions. The +caller sends those requests to this user's other devices. } \examples{ \donttest{ diff --git a/man/mx_crypto_send_key_requests.Rd b/man/mx_crypto_send_key_requests.Rd new file mode 100644 index 0000000..d8b3fd3 --- /dev/null +++ b/man/mx_crypto_send_key_requests.Rd @@ -0,0 +1,26 @@ +% tinyrox says don't edit this manually, but it can't stop you! +\name{mx_crypto_send_key_requests} +\alias{mx_crypto_send_key_requests} +\title{Send queued Matrix room-key requests or cancellations} +\usage{ +mx_crypto_send_key_requests(client, requests) +} +\arguments{ +\item{client}{Matrix client config.} + +\item{requests}{A list of request/cancellation descriptors returned by +[mx_crypto_process_sync()].} +} +\value{ +The endpoint responses, invisibly. +} +\description{ +Sends each descriptor returned by [mx_crypto_process_sync()] as an +unencrypted \code{m.room_key_request} to every device of this user. The +receiving clients apply their own verified-device sharing policy. +} +\examples{ +\dontrun{ +mx_crypto_send_key_requests(client, result$key_requests) +} +} diff --git a/man/mx_crypto_sessions_new.Rd b/man/mx_crypto_sessions_new.Rd index b47429f..0c133f0 100644 --- a/man/mx_crypto_sessions_new.Rd +++ b/man/mx_crypto_sessions_new.Rd @@ -7,7 +7,7 @@ mx_crypto_sessions_new() } \value{ A session set: named lists \code{olm}, \code{olm_in}, - \code{megolm_out}, \code{megolm_in}. + \code{megolm_out}, \code{megolm_in}, and \code{key_requests}. } \description{ Create an empty E2EE session set diff --git a/vignettes/e2ee.md b/vignettes/e2ee.md index 9f9f033..8d97155 100644 --- a/vignettes/e2ee.md +++ b/vignettes/e2ee.md @@ -21,13 +21,22 @@ plaintext Matrix clients install and run without Rust. Read this first; it frames what the rest of the vignette delivers. -- **Trust on first use.** Device keys come from `/keys/query` and are - used as-is. There is no cross-signing trust store yet, so a malicious - homeserver could substitute device keys on first contact. - (`mx.crypto` ships the verification primitives, - `mxc_verify_device_keys()`; wiring a trust store is future work.) -- **No key requests or forwarded keys.** A device that missed the - original `m.room_key` cannot ask peers to re-share it. +- **Verified device objects.** Every `/keys/query` device object must carry a + valid Ed25519 self-signature. Cross-signed devices additionally require a + valid master -> self-signing -> device chain. Trusting the master remains a + user decision; an internally valid chain alone does not establish that the + master belongs to the expected person. +- **Fail-closed bootstrap.** Cross-signing private keys are encrypted in the + local crypto store. Bootstrap never resets an existing server identity when + its private keys are absent or disagree. +- **Same-user key recovery.** Missing Megolm sessions generate durable, + deduplicated `m.room_key_request` events. A forwarded key is accepted only + for an outstanding request and only over Olm from this user's cross-signed + device. Pass the locally trusted master to `mx_crypto_known_devices()` as + `self_master_key`; without it, this user's devices are not cross-signed. + This cannot recover another user's historical outbound session, and + decrypted history from a forwarded key never reports its original sender as + verified. - **No SAS (emoji) verification.** - **Local-key storage.** Ratchet state is pickled with a locally stored 32-byte key (file mode 0600). That guards against casual inspection; @@ -76,6 +85,37 @@ signs and uploads one-time keys (`/keys/upload`), marks them published, and saves the account. Run it again whenever the server's `one_time_key_counts` runs low. +## Cross-signing bootstrap + +Stop any other process using this device's crypto store, then load the same +persisted account and bootstrap its cross-signing identity: + +```r +keys <- mx_crypto_cross_signing_bootstrap( + client, acct, store, + password = Sys.getenv("MATRIX_PASSWORD")) + +devices <- mx_crypto_known_devices(client, client$user_id, strict = TRUE, + self_master_key = keys$master) +mine <- Filter(function(device) { + identical(device$device_id, client$device_id) +}, devices) +stopifnot(length(mine) == 1L, isTRUE(mine[[1L]]$cross_signed)) +``` + +The password is used only when the homeserver requires password-based UIA and +must not be printed or embedded in source. A completed `auth` object can be +passed instead. Bootstrap saves the private master, self-signing, and +user-signing keys before uploading public objects. It is idempotent when the +local and server master keys agree and fails closed if the server identity has +no matching local private keys. +Valid device and master signatures already returned by `/keys/query` are +skipped on rerun. + +Cross-signed is not the same as trusted. Another client must independently +verify the master identity before treating its signature chain as belonging to +the expected person. + ## Sending ```r @@ -112,27 +152,60 @@ a single Megolm encrypt + send. ```r res <- mx_sync_update(client, timeout = 30000L) +signing <- mx_crypto_cross_signing_load(store) +master_pin <- if (is.null(signing)) NULL else + mx.crypto::mxc_signing_key_public(signing$master) +# Include our own devices for history recovery, plus every encrypted sender +# in this sync for attribution. This example has one other participant. +devices <- mx_crypto_known_devices( + client, c(client$user_id, "@friend:example.org"), + self_master_key = master_pin) + my_curve <- mx.crypto::mxc_account_identity_keys(acct)$curve25519 out <- mx_crypto_process_sync(acct, sessions, res$sync, my_curve, - self_id = client$user_id) + self_id = client$user_id, + self_device_id = client$device_id, + devices = devices) sessions <- out$sessions mx_crypto_sessions_save(sessions, store) +sent <- list() +for (request in out$key_requests) { + ok <- tryCatch({ + mx_crypto_send_key_requests(client, list(request)); TRUE + }, error = function(e) FALSE) + if (ok) sent[[length(sent) + 1L]] <- request +} +sessions <- mx_crypto_mark_key_requests_sent(sessions, sent) +mx_crypto_sessions_save(sessions, store) +for (cancel in out$key_request_cancellations) { + try(mx_crypto_send_key_requests(client, list(cancel)), silent = TRUE) +} + for (ev in out$events) cat(ev$sender, ":", ev$body, "\n") ``` `mx_crypto_process_sync()` makes two passes over the sync response: 1. **To-device events**: Olm-decrypts anything addressed to this - device's Curve25519 key. Each recovered `m.room_key` becomes an - inbound Megolm session, stored under `room_id|session_id`. + device's Curve25519 key. Direct `m.room_key` events become inbound + Megolm sessions. Requested `m.forwarded_room_key` events are imported + only after their request, sender, cross-signing chain, and computed + session ID validate. 2. **Room timelines**: decrypts every `m.room.encrypted` event whose - session is known, returning records in the same shape as - `mx_extract_text_events()` — `room_id`, `event_id`, `sender`, - `is_self`, `body`, `msgtype`, `mentions`. - -Events whose keys haven't arrived yet are skipped, not errored; they -decrypt on a later pass once the to-device message lands. + session is known. An unknown session creates one persistent request to + the current user's other devices. Decrypted records match + `mx_extract_text_events()` and add `sender_verified`. + +The caller saves returned crypto state before making any request transport +call. Successfully sent requests are then marked and saved again. Failed +requests remain queued with the same stable id and are returned again on later +polls; cancellation failures are safe to drop. This ordering prevents a +network error from replaying a sync batch against already-advanced Olm and +Megolm ratchets. The `chat.api` Matrix adapter performs this sequence +automatically and loads its own master pin from the local crypto store for +device queries. Sent requests with no answer currently remain stored until +their key arrives; automatic expiry/pruning is deferred. ## Persistence @@ -142,7 +215,8 @@ Everything stateful lives in the crypto store directory: |---|---| | `pickle.key` | the locally stored 32-byte key the pickles are encrypted with | | `account.pickle` | device identity (Curve25519 + Ed25519 keys, OTK state) | -| `sessions.json` | pickled Olm sessions and Megolm in/outbound sessions | +| `sessions.json` | pickled Olm/Megolm sessions and outstanding key requests | +| `cross-signing.json` | encrypted master, self-signing, and user-signing private keys | `mx_crypto_sessions_save()` / `mx_crypto_sessions_load()` round-trip the session set, so an established room key keeps decrypting across process