Protocol v4: rebuild X3DH and the Double Ratchet (BREAKING) - #14
Merged
Conversation
added 3 commits
July 20, 2026 23:22
…rchive
The vendored libsodium.a could not be linked at all on Apple Silicon. Both of
its slices were built for the iOS *device* platform, so every simulator build
died at link time:
ld: building for 'iOS-simulator', but linking in object file
(libsodium.a[arm64][12](libsodium_la-curve25519_ref10.o)) built for 'iOS'
A fat archive cannot express the fix. Device-arm64 and simulator-arm64 are the
same architecture, and a `.a` is indexed by architecture alone, so the two
cannot coexist in one. An XCFramework is the only container that distinguishes
them by platform, which is why this is a repackaging and not a version bump
that happens to also change the container.
libsodium 1.0.22 is therefore vendored as
nuntius/libsodium/Clibsodium.xcframework, carrying ios-arm64_arm64e and
ios-arm64_arm64e_x86_64-simulator. The all-platform build is 23 MB; the other
platforms are unreachable from an iOS-only framework, so they are trimmed and
what ships is 4.2 MB.
Consequences:
- The 63 file references under the `libsodium` group are gone, and with them
HEADER_SEARCH_PATHS, LIBRARY_SEARCH_PATHS and USER_HEADER_SEARCH_PATHS. The
XCFramework carries its own headers and module map, so the include becomes
<Clibsodium/sodium.h>.
- IPHONEOS_DEPLOYMENT_TARGET moves 10.3 -> 13.0. 10.3 is below the 12.0 floor
that current Xcode will build at all, so this is not a policy choice.
No protocol, wire-format or API change. The existing suite passes unchanged —
which is worth stating precisely, because it is also true of the far larger
change that follows: those tests assert round-trip success and key agreement,
both of which hold when the underlying key agreement is broken.
v3 was not dated, it was broken. The full analysis is in SPEC.md §14: thirteen
confirmed defects, plus nine more found while writing the specification. The
two that matter most:
* X3DH silently collapsed to a single Diffie-Hellman. The key derivation
called crypto_kdf_derive_from_key, which reads exactly 32 bytes of key
material whatever length you hand it. The 96-128 byte X3DH input therefore
contributed only DH1 — and because BOTH sides collapsed identically, every
test passed and every message round-tripped. Authentication and forward
secrecy came entirely from a construction that was not running.
* The root key never chained. Each ratchet step re-derived from the same
material instead of feeding the previous root key forward, so the DH
ratchet provided no post-compromise security.
Neither is observable from a round-trip test, which is why the old suite was
green throughout. That is the lesson this commit is built around: positive
round-trip tests certify nothing about this protocol.
The fix is not repairable in place. Correcting the key agreement changes every
derived key, so v4 is a hard break: a v4 receiver rejects a first byte that is
not 0x04 with ERR_UNSUPPORTED_VERSION, there is no migration path and no
dual-stack mode, and existing identities must be re-registered because the
identity key type itself changed.
What v4 is:
- SPEC.md — the normative contract, and the actual deliverable here. All four
implementations (this one and the Java, Kotlin and Swift ports) are written
against it rather than against each other, and it is what makes "these
interoperate" a checkable claim instead of an aspiration.
- HKDF-SHA256 (RFC 5869) everywhere a key is derived, so the input length is
the input length.
- ChaCha20-Poly1305 IETF (RFC 8439) replacing AES-256-CBC + HMAC-SHA256 via
CommonCrypto — one AEAD, no encrypt-then-MAC assembly, and no Apple-only
dependency standing between this and the ports.
- A two-key identity: Ed25519 for signing (RFC 8032 pure, never prehashed) and
X25519 for ECDH (RFC 7748), as separate key pairs. The Ed25519->X25519
conversion is banned outright; neither the JDK nor CryptoKit can perform it.
- Domain-separated labels on every derivation, and a transcript hash binding
the handshake to every key and identifier it used.
- Nominal key types, so an Ed25519 public key is not assignable to an X25519
parameter and the compiler says so.
- Ordered parsing gates in which no offset is ever derived from a received
byte. v3 read a wire-supplied length and used it as a subdata bound.
- Atomic decryption: the ratchet step, chain advance, counter increment and
skipped-key insert all happen on a snapshot committed only after the AEAD
authenticates. Otherwise one unauthenticated message permanently
desynchronises a live session.
- Every libsodium and Sec* return value checked. NSMutableData dataWithLength:
zero-fills, so an unchecked RNG failure yields an all-zero key that both
parties agree on, with nothing to observe.
- An explicit zeroization schedule covering failure exits, not just success.
- A single injectable clock and CSPRNG, because a frozen conformance suite that
reads the wall clock has a shelf life of seven days.
Layering is enforced rather than described: IRSodiumCryptoProvider is the only
file permitted to call libsodium, and every layer above it holds an
id<IRCryptoProvider>. That seam is where a platform crypto backend gets
swapped, which is precisely what the three ports do.
tools/lint_banned_apis.py runs as a pre-Sources build phase, so a banned API,
a second crypto_sign_detached call site, or a raw *error assignment stops the
build rather than shipping.
526 tests, 0 failures. The suite deliberately includes property-based tests for
the invariants a round-trip cannot see — that altering any single DH input
changes SK, that every message key moves when SK moves, that every bit flip in
a message fails to decrypt, and that a failed authentication leaves the session
state byte-identical.
The podspec goes. CocoaPods entered maintenance mode and the trunk is winding down; shipping a v4 that can only be installed through it would be shipping a dependency on something being retired. Package.swift replaces it, with no version field — SwiftPM takes that from the git tag. The manifest carries a long comment that is not decoration. Sources here are flat in nuntius/ and import each other framework-style as <nuntius/Foo.h>, which Xcode resolves through a header map and SwiftPM does not have. SwiftPM passes dependents exactly one include directory, so that directory must *contain* a directory named nuntius. include/nuntius is a relative symlink to ../nuntius that produces exactly that shape without moving a single file. Three simpler layouts were tried first; each fails at graph load, and the manifest records the verbatim error for each so nobody has to rediscover them. The symlink looks redundant and is not — deleting it breaks the package. The README described software that no longer exists: the v3 API surface, all of it deleted; AES-256-CBC + HMAC via CommonCrypto, replaced; and `pod "nuntius"`, gone. It is rewritten against the headers that are actually in the tree, and it states the security posture honestly, including that the transcript-hash binding has not been externally reviewed and that the conformance vector corpus specified in SPEC §15 has not been generated yet. The CHANGELOG's 1.0.0 entry leads with the break rather than burying it. The Security section names the concrete v3 defects, because "upgrade for security fixes" is not actionable and anyone still running 0.0.9 needs to know that the key agreement was not doing what it claimed.
This was referenced Jul 21, 2026
Closed
added 3 commits
July 21, 2026 01:30
SPEC §15 has specified this corpus since the rewrite began; it did not exist, which
made "all four implementations interoperate byte-for-byte" a claim with nothing
behind it. It exists now: 88 vectors across six files under spec/vectors/,
covering every id §15.3 and §15.4 require.
This is the artifact the Java, Kotlin and Swift ports are graded against. They are
written against SPEC.md rather than against this code, so the vectors are the only
thing that can actually catch a divergence — and §15.1 exists because the v3 suite
was green for years while X3DH silently ran on a single Diffie-Hellman.
Three properties make it worth trusting:
* The RFC-anchored vectors are TRANSCRIBED from RFC 5869, 7748, 8032 and 8439,
never produced here. A corpus generated entirely by the implementation under
test proves self-consistency and nothing else; these are the only external
check in it. Every value was re-fetched from rfc-editor.org and compared
mechanically, and the nuntius derivations were independently recomputed in
Python from the SPEC prose — the X3DH handshakes end to end, and
RATCHET-LINEAR reimplemented from scratch through §6, §7, §8 and §9.
* The suite rebuilds the corpus in memory on every run and compares it
byte-for-byte against the frozen files. Implementation drift fails a test
rather than quietly rewriting the contract. Rewriting takes the explicit
spec/vectors/.regenerate sentinel, which then fails the run on purpose so the
diff gets reviewed. §15.6 step 4 makes a changed vector a spec version bump,
and a failing port is never fixed by regenerating a vector.
* No vector reads the host clock. Every clock read routes through one injectable
source, and the suite runs again with the clock ten years forward. Without
that, a corpus green on the day it is frozen goes red seven days later on the
skipped-key TTL and ninety days later on the prekey validity window — with
§15.6 forbidding the obvious workaround.
Building it found two defects in the specification, both of which would have been
frozen into the contract:
* §3.4 claimed Ed25519 signing produces byte-identical output across libsodium,
the JDK and CryptoKit. It does not. RFC 8032 §8.2 permits additional
randomness and CryptoKit takes that option — signing one message three times
under one seed yields three different valid signatures, none equal to RFC
8032's published one. As written, §15.3's mandatory RFC8032-ED25519 vector was
unpassable by a conformant Swift port. Signatures are now asserted verify-side
only: a signature is always an input, never an expected output (§15.5 rule 8).
* §4.4 claimed CryptoKit performs no small-order check. It does — key agreement
throws. The MUST is unchanged, and now says why: an implementation may not
skip its own check because it trusts the library underneath, and that trust is
exactly what was wrong here, in the direction that sounds safe.
Also corrected: X25519-ZERO listed six small-order u-coordinates with bit 255
clear when there are seven, confirmed from the curve arithmetic rather than from a
copied blacklist. And the §10.3 bundle rejections are kind "wire", not "state" —
a prekey bundle is a §5.4 wire structure, and a port switching on kind would
otherwise hand 251 bytes of bundle to its state-blob parser.
529 tests, 0 failures.
Each is a place where SPEC.md left a port to guess, and all three were found the
same way: three people implementing the same document from scratch reached the
same three gaps.
* Error code 7106 named three conditions and §4.4 defined two. "Reflected own
key" had no home — the reflection checks existed, but scattered across §10.1
check 8, §10.2 check 11, §10.7 step 6 and §11.2, so a port reading §4.4 as the
definition of public-key validation implemented two thirds of the code. Now
§4.4 check 2b, with an exhaustive table of the four sites and the value each
compares against, and a MUST NOT on adding a fifth: an extra rejection is an
interop divergence, not extra safety.
Numbered 2b rather than 3 on purpose. Six sections cite "checks 1-2" and
"check 3" by number, so renumbering would have silently redirected all of
them.
* Error code 7118 was specified as a sodium_init() failure. Neither BouncyCastle
nor CryptoKit has an initialization step, so on half the target platforms the
code was unreachable with undefined meaning — which is an invitation to invent
a use for it. Restated backend-neutrally, and it now says explicitly that
having no reachable condition is conformant rather than a gap, and names the
three plausible misuses it MUST NOT be stretched to cover.
* §13.1 claimed the platform CSPRNG "throws rather than returning a status".
False for SecureRandom.nextBytes, which returns void and declares no checked
exception — and false in the direction that sounds safe. The section now leads
with the invariant rather than a claim about any library: a fill that does not
succeed MUST NOT leave usable bytes behind and the caller MUST NOT proceed,
because NSMutableData dataWithLength: / new byte[n] / Data(count:) all
zero-fill, so a silent failure yields an all-zero key both parties agree on.
That is v3's defect 4 verbatim.
One requirement was strengthened rather than restated: the all-zero tripwire
is now REQUIRED on the JVM. With neither a status nor a guaranteed exception
it is the only mechanism by which the invariant can be discharged there, so
without it the JVM had no conformant way to satisfy a MUST.
Also swept the document for passages still asserting the two claims retracted
while building the corpus — that Ed25519 signing is byte-reproducible across
platforms, and that CryptoKit performs no small-order check. Three survived, in
§6.2, §11.3 and §17.8; all corrected, and §17.8 additionally no longer tells each
port to extend primitives.json unilaterally, which §15.6 step 4 forbids.
No frozen vector changed. 529 tests, 0 failures.
Adds a "Read this first" notice to the README, a banner to the CHANGELOG, and the same posture to CLAUDE.md so a contributor does not have to read the README to learn it. The notice is deliberate about one distinction: v4 is complete AND unaudited, and those are two different claims. Saying "experimental" must not be read as "unfinished, check back later" — it is specified to the byte, implemented, and graded against a frozen corpus. What it has never had is review by anyone who did not write it, and SPEC §17.7 singles out the transcript-hash binding in particular. The strongest argument in the notice is this repo's own history. v3 shipped for years with a green test suite while its X3DH silently collapsed to a single Diffie-Hellman, because the suite asserted that both parties derived the same 32 bytes — which is exactly what the collapse also produces. "The tests pass, therefore the cryptography is sound" is the inference that kept it in production, and no amount of care in this release makes it safe to repeat. That is worth saying plainly next to a corpus of 88 passing vectors. It also says outright that carrying a specification, a reference implementation, a frozen corpus and three ports through to this point is a scope LLM-assisted development made reachable for one person. That is part of what the project is, and leaving it to be inferred would be coy. Renames "Production wiring" to "Wiring for a real deployment", with both referring anchors updated. The section body was always honest about the in-memory stores not being production stores, but the heading is what a skimmer sees, and it sat four TOC entries below a notice saying not for production. Replaces .travis.yml with .github/workflows/ci.yml. The Travis file pinned xcode8.3 and an iPhone 7 / iOS 10.3.1 simulator that no longer exists, so it could not run — and the README asserts in two places that CI exists, which deleting without replacing would have made false at exactly the moment the repo goes public. The new workflow resolves its simulator at run time rather than pinning a name, because pinning is what rotted the old one. Its last step is `git diff --exit-code -- spec/vectors`: the suite regenerates the corpus in memory and compares, so a run that MODIFIED a frozen file means the compare path never executed, and §15.6 step 4 makes that a four-repo spec version bump. CI is where that should surface, not a release. Also untracks two per-user Xcode state files that predate the .gitignore rule already covering them, and ignores IMPLEMENTATION_PLAN.md — internal scaffolding written in the future tense that asserts a v3 test suite which no longer exists. 529 tests, 0 failures. No frozen vector changed.
ivRodriguezCA
marked this pull request as ready for review
July 21, 2026 12:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds X3DH and the Double Ratchet as protocol v4, and deletes v3 rather than deprecating it.
This is a hard, deliberate break. Correcting the key agreement changes every key derived downstream
of it, so no version of this fix stays wire-compatible. A v4 receiver rejects a first byte that is
not
0x04withERR_UNSUPPORTED_VERSION, there is no migration path and no dual-stack mode, andexisting identities must be re-registered because the identity key type itself changed.
Why
v3 was cryptographically broken, not merely dated. Two of the defects were reported by users and
went unfixed for years:
crypto_kdf_derive_from_keytakesconst unsigned char k[crypto_kdf_KEYBYTES]and reads exactly32 bytes;
IRTripleDHServicehanded it a 96- or 128-byteDH1 ‖ DH2 ‖ DH3 [‖ DH4]. DH2, DH3 andDH4 were computed, concatenated, and discarded. DH1 is
X25519(IK_A, SPK_B)— both long-lived —so the handshake had no forward secrecy and the one-time prekey contributed nothing.
implementations, for two independent reasons: the API took Ed25519 public keys and ran
crypto_sign_ed25519_pk_to_curve25519over whatever it was given (discarding the return value, soa rejected input left an uninitialized stack buffer in use as a private scalar), and the result
was
BLAKE2b(q ‖ pk_s ‖ pk_r)rather thanq.SPEC.md§14 has the full table: thirteen confirmed defects, plus nine more found while writing thespecification.
Every one of these passed the v3 test suite. In each case both parties agreed — and agreement
was never the property in question.
IRTripleDHServiceSpecasserted that Alice's and Bob's sharedkeys were equal and 32 bytes long, which is exactly as true of a single DH as of a triple one. That
is the organising constraint of this PR: positive round-trip tests certify nothing about this
protocol.
What this PR contains
Four commits, split by concern:
8d13a6d— libsodium 1.0.13 → 1.0.22, repackaged as an XCFramework. Not a routine bump: thevendored
libsodium.ahad x86_64 and arm64 slices both built for the iOS device platform, sono simulator build linked on Apple Silicon. A
.ais indexed by architecture alone anddevice-arm64 and simulator-arm64 are the same architecture, so no fat archive can carry both.
Deployment target 10.3 → 13.0. Supersedes libsodium v1.0.14 #11.
4552ac7— the protocol rewrite. BREAKING.9e62be4— SwiftPM replaces the podspec, README rewritten, CHANGELOG added.7f41a0c— the frozen conformance vector corpus. See below.The specification is the actual deliverable
SPEC.mdis normative, and the Java, Kotlin and Swift ports now in progress are written against itrather than against this code. It is what makes "these four interoperate" a checkable claim instead
of an aspiration.
Protocol changes
(ikm, ikm_len)signature makes theIncorrect triple-DH implementation #13 truncation inexpressible rather than merely fixed.
AEAD call: no encrypt-then-MAC assembly, no separate IV, no MAC comparator, and no Apple-only
dependency standing between this and the ports.
(RFC 7748), as separate key pairs. The Ed25519→X25519 conversion is banned outright.
every key and identifier it used.
compiler enforces it.
wire-supplied length and used it as a
subdataWithRange:bound.happen on a snapshot committed only after the AEAD authenticates. Otherwise one unauthenticated
message permanently desynchronises a live session.
NSMutableData dataWithLength:zero-fills, so an unchecked RNGfailure yields an all-zero key that both parties agree on, with nothing to observe.
has a shelf life of seven days.
Structure
IRSodiumCryptoProvideris the only file permitted to call libsodium; every layer above holds anid<IRCryptoProvider>. That seam is where a platform crypto backend gets swapped, which is exactlywhat the three ports do.
tools/lint_banned_apis.pyruns as a pre-Sources build phase, so a bannedAPI, a second
crypto_sign_detachedcall site, or a raw*error =assignment stops the build.Testing
529 tests, 0 failures, zero source warnings, on a clean build.
The suite deliberately includes property-based tests for the invariants a round-trip cannot see:
that altering any single DH input changes
SK, that every message key moves whenSKmoves, thatevery bit flip in a message fails to decrypt, and that a failed authentication leaves session state
byte-identical.
The conformance corpus (
7f41a0c)spec/vectors/*.json— 88 vectors across six files, covering every idSPEC.md§15.3 and §15.4require. This is the artifact the Java, Kotlin and Swift ports are graded against; they are written
against
SPEC.mdrather than against this code, so the vectors are the only thing that can catch adivergence.
here — a corpus generated entirely by the implementation under test proves self-consistency and
nothing else. Every value was re-fetched from rfc-editor.org and compared mechanically, and the
nuntius derivations were independently recomputed in Python from the spec prose: both X3DH
handshakes end to end, and
RATCHET-LINEARreimplemented from scratch through §6–§9.instead of rewriting the contract. Rewriting needs the explicit
.regeneratesentinel, which thenfails the run on purpose so the diff gets reviewed.
Building it found two defects in this specification, both of which would otherwise have been
frozen into the contract by §15.6 step 4:
is not — RFC 8032 §8.2 permits added randomness and CryptoKit takes it. Signing one message three
times under one seed gives three different valid signatures, none equal to RFC 8032's published
one. The mandatory
RFC8032-ED25519vector was therefore unpassable by a conformant Swift port.Signatures are now verify-side only (§15.5 rule 8): always an input, never an expected output.
is unchanged and now says why: an implementation may not skip its own check because it trusts the
library underneath, and that trust is exactly what was wrong here, in the direction that sounds
safe.
Also corrected:
X25519-ZEROlisted six small-order u-coordinates with bit 255 clear when there areseven (confirmed from the curve arithmetic, not a copied blacklist), and the §10.3 bundle rejections
are
kind: "wire"rather than"state"— a prekey bundle is a §5.4 wire structure, and a portswitching on
kindwould otherwise hand 251 bytes of bundle to its state-blob parser.Review guidance
SPEC.md§6 (X3DH) and §7 (Double Ratchet) are the sections worth human attention before this istagged 1.0.0; everything else follows from them. §17 records the open risks and deliberate
non-goals, including that the transcript-hash binding has not been externally reviewed.
Addresses #13 and #12.