Skip to content

Protocol v4: rebuild X3DH and the Double Ratchet (BREAKING) - #14

Merged
ivRodriguezCA merged 6 commits into
mainfrom
v4-protocol
Jul 21, 2026
Merged

Protocol v4: rebuild X3DH and the Double Ratchet (BREAKING)#14
ivRodriguezCA merged 6 commits into
mainfrom
v4-protocol

Conversation

@ivRodriguezCA

@ivRodriguezCA ivRodriguezCA commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Rebuilds X3DH and the Double Ratchet as protocol v4, and deletes v3 rather than deprecating it.

This is a hard, deliberate break. Correcting the key agreement changes every key derived downstream
of it, so no version of this fix stays wire-compatible. A v4 receiver rejects a first byte that is
not 0x04 with ERR_UNSUPPORTED_VERSION, there is no migration path and no dual-stack mode, and
existing identities must be re-registered because the identity key type itself changed.

Why

v3 was cryptographically broken, not merely dated. Two of the defects were reported by users and
went unfixed for years:

  • Incorrect triple-DH implementation #13 (@digal, May 2019) — X3DH collapsed to a single Diffie-Hellman.
    crypto_kdf_derive_from_key takes const unsigned char k[crypto_kdf_KEYBYTES] and reads exactly
    32 bytes; IRTripleDHService handed it a 96- or 128-byte DH1 ‖ DH2 ‖ DH3 [‖ DH4]. DH2, DH3 and
    DH4 were computed, concatenated, and discarded. DH1 is X25519(IK_A, SPK_B) — both long-lived —
    so the handshake had no forward secrecy and the one-time prekey contributed nothing.
  • This is not standart curve25519 algoritm. #12 (@NoVoLuMe, @raojunbo) — the library could not agree a key with standard X25519
    implementations, for two independent reasons: the API took Ed25519 public keys and ran
    crypto_sign_ed25519_pk_to_curve25519 over whatever it was given (discarding the return value, so
    a rejected input left an uninitialized stack buffer in use as a private scalar), and the result
    was BLAKE2b(q ‖ pk_s ‖ pk_r) rather than q.

SPEC.md §14 has the full table: thirteen confirmed defects, plus nine more found while writing the
specification.

Every one of these passed the v3 test suite. In each case both parties agreed — and agreement
was never the property in question. IRTripleDHServiceSpec asserted that Alice's and Bob's shared
keys were equal and 32 bytes long, which is exactly as true of a single DH as of a triple one. That
is the organising constraint of this PR: positive round-trip tests certify nothing about this
protocol.

What this PR contains

Four commits, split by concern:

  1. 8d13a6d — libsodium 1.0.13 → 1.0.22, repackaged as an XCFramework. Not a routine bump: the
    vendored libsodium.a had x86_64 and arm64 slices both built for the iOS device platform, so
    no simulator build linked on Apple Silicon. A .a is indexed by architecture alone and
    device-arm64 and simulator-arm64 are the same architecture, so no fat archive can carry both.
    Deployment target 10.3 → 13.0. Supersedes libsodium v1.0.14 #11.
  2. 4552ac7 — the protocol rewrite. BREAKING.
  3. 9e62be4 — SwiftPM replaces the podspec, README rewritten, CHANGELOG added.
  4. 7f41a0c — the frozen conformance vector corpus. See below.

The specification is the actual deliverable

SPEC.md is normative, and the Java, Kotlin and Swift ports now in progress are written against it
rather than against this code. It is what makes "these four interoperate" a checkable claim instead
of an aspiration.

Protocol changes

  • HKDF-SHA256 (RFC 5869) everywhere a key is derived — its (ikm, ikm_len) signature makes the
    Incorrect triple-DH implementation #13 truncation inexpressible rather than merely fixed.
  • ChaCha20-Poly1305 IETF (RFC 8439) replaces AES-256-CBC + HMAC-SHA256 via CommonCrypto. One
    AEAD call: no encrypt-then-MAC assembly, no separate IV, no MAC comparator, and no Apple-only
    dependency standing between this and the ports.
  • Two-key identity: Ed25519 for signing (RFC 8032 pure, never prehashed) and X25519 for ECDH
    (RFC 7748), as separate key pairs. The Ed25519→X25519 conversion is banned outright.
  • Domain-separated labels on every derivation, and a transcript hash binding the handshake to
    every key and identifier it used.
  • Nominal key types — an Ed25519 public key is not assignable to an X25519 parameter and the
    compiler enforces it.
  • Ordered parsing gates in which no offset is ever derived from a received byte. v3 read a
    wire-supplied length and used it as a subdataWithRange: bound.
  • Atomic decryption — ratchet step, chain advance, counter increment and skipped-key insert all
    happen on a snapshot committed only after the AEAD authenticates. Otherwise one unauthenticated
    message permanently desynchronises a live session.
  • Every return value checked. NSMutableData dataWithLength: zero-fills, so an unchecked RNG
    failure yields an all-zero key that both parties agree on, with nothing to observe.
  • An explicit zeroization schedule covering failure exits, not just success paths.
  • One injectable clock and CSPRNG, because a frozen conformance suite that reads the wall clock
    has a shelf life of seven days.

Structure

IRSodiumCryptoProvider is the only file permitted to call libsodium; every layer above holds an
id<IRCryptoProvider>. That seam is where a platform crypto backend gets swapped, which is exactly
what the three ports do. tools/lint_banned_apis.py runs as a pre-Sources build phase, so a banned
API, a second crypto_sign_detached call site, or a raw *error = assignment stops the build.

Testing

529 tests, 0 failures, zero source warnings, on a clean build.

The suite deliberately includes property-based tests for the invariants a round-trip cannot see:
that altering any single DH input changes SK, that every message key moves when SK moves, that
every bit flip in a message fails to decrypt, and that a failed authentication leaves session state
byte-identical.

The conformance corpus (7f41a0c)

spec/vectors/*.json88 vectors across six files, covering every id SPEC.md §15.3 and §15.4
require. This is the artifact the Java, Kotlin and Swift ports are graded against; they are written
against SPEC.md rather than against this code, so the vectors are the only thing that can catch a
divergence.

  • The RFC-anchored vectors are transcribed from RFC 5869, 7748, 8032 and 8439, never produced
    here — a corpus generated entirely by the implementation under test proves self-consistency and
    nothing else. Every value was re-fetched from rfc-editor.org and compared mechanically, and the
    nuntius derivations were independently recomputed in Python from the spec prose: both X3DH
    handshakes end to end, and RATCHET-LINEAR reimplemented from scratch through §6–§9.
  • The suite rebuilds the corpus in memory each run and compares byte-for-byte, so drift fails a test
    instead of rewriting the contract. Rewriting needs the explicit .regenerate sentinel, which then
    fails the run on purpose so the diff gets reviewed.
  • No vector reads the host clock; the suite runs again with the clock ten years forward.

Building it found two defects in this specification, both of which would otherwise have been
frozen into the contract by §15.6 step 4:

  • §3.4 claimed Ed25519 signing is byte-reproducible across libsodium, the JDK and CryptoKit. It
    is not — RFC 8032 §8.2 permits added randomness and CryptoKit takes it. Signing one message three
    times under one seed gives three different valid signatures, none equal to RFC 8032's published
    one. The mandatory RFC8032-ED25519 vector was therefore unpassable by a conformant Swift port.
    Signatures are now verify-side only (§15.5 rule 8): always an input, never an expected output.
  • §4.4 claimed CryptoKit performs no small-order check. It does — key agreement throws. The MUST
    is unchanged and now says why: an implementation may not skip its own check because it trusts the
    library underneath, and that trust is exactly what was wrong here, in the direction that sounds
    safe.

Also corrected: X25519-ZERO listed six small-order u-coordinates with bit 255 clear when there are
seven (confirmed from the curve arithmetic, not a copied blacklist), and the §10.3 bundle rejections
are kind: "wire" rather than "state" — a prekey bundle is a §5.4 wire structure, and a port
switching on kind would otherwise hand 251 bytes of bundle to its state-blob parser.

Review guidance

SPEC.md §6 (X3DH) and §7 (Double Ratchet) are the sections worth human attention before this is
tagged 1.0.0; everything else follows from them. §17 records the open risks and deliberate
non-goals, including that the transcript-hash binding has not been externally reviewed.

Addresses #13 and #12.

Ivan added 3 commits July 20, 2026 23:22
…rchive

The vendored libsodium.a could not be linked at all on Apple Silicon. Both of
its slices were built for the iOS *device* platform, so every simulator build
died at link time:

    ld: building for 'iOS-simulator', but linking in object file
    (libsodium.a[arm64][12](libsodium_la-curve25519_ref10.o)) built for 'iOS'

A fat archive cannot express the fix. Device-arm64 and simulator-arm64 are the
same architecture, and a `.a` is indexed by architecture alone, so the two
cannot coexist in one. An XCFramework is the only container that distinguishes
them by platform, which is why this is a repackaging and not a version bump
that happens to also change the container.

libsodium 1.0.22 is therefore vendored as
nuntius/libsodium/Clibsodium.xcframework, carrying ios-arm64_arm64e and
ios-arm64_arm64e_x86_64-simulator. The all-platform build is 23 MB; the other
platforms are unreachable from an iOS-only framework, so they are trimmed and
what ships is 4.2 MB.

Consequences:

- The 63 file references under the `libsodium` group are gone, and with them
  HEADER_SEARCH_PATHS, LIBRARY_SEARCH_PATHS and USER_HEADER_SEARCH_PATHS. The
  XCFramework carries its own headers and module map, so the include becomes
  <Clibsodium/sodium.h>.
- IPHONEOS_DEPLOYMENT_TARGET moves 10.3 -> 13.0. 10.3 is below the 12.0 floor
  that current Xcode will build at all, so this is not a policy choice.

No protocol, wire-format or API change. The existing suite passes unchanged —
which is worth stating precisely, because it is also true of the far larger
change that follows: those tests assert round-trip success and key agreement,
both of which hold when the underlying key agreement is broken.
v3 was not dated, it was broken. The full analysis is in SPEC.md §14: thirteen
confirmed defects, plus nine more found while writing the specification. The
two that matter most:

  * X3DH silently collapsed to a single Diffie-Hellman. The key derivation
    called crypto_kdf_derive_from_key, which reads exactly 32 bytes of key
    material whatever length you hand it. The 96-128 byte X3DH input therefore
    contributed only DH1 — and because BOTH sides collapsed identically, every
    test passed and every message round-tripped. Authentication and forward
    secrecy came entirely from a construction that was not running.

  * The root key never chained. Each ratchet step re-derived from the same
    material instead of feeding the previous root key forward, so the DH
    ratchet provided no post-compromise security.

Neither is observable from a round-trip test, which is why the old suite was
green throughout. That is the lesson this commit is built around: positive
round-trip tests certify nothing about this protocol.

The fix is not repairable in place. Correcting the key agreement changes every
derived key, so v4 is a hard break: a v4 receiver rejects a first byte that is
not 0x04 with ERR_UNSUPPORTED_VERSION, there is no migration path and no
dual-stack mode, and existing identities must be re-registered because the
identity key type itself changed.

What v4 is:

- SPEC.md — the normative contract, and the actual deliverable here. All four
  implementations (this one and the Java, Kotlin and Swift ports) are written
  against it rather than against each other, and it is what makes "these
  interoperate" a checkable claim instead of an aspiration.
- HKDF-SHA256 (RFC 5869) everywhere a key is derived, so the input length is
  the input length.
- ChaCha20-Poly1305 IETF (RFC 8439) replacing AES-256-CBC + HMAC-SHA256 via
  CommonCrypto — one AEAD, no encrypt-then-MAC assembly, and no Apple-only
  dependency standing between this and the ports.
- A two-key identity: Ed25519 for signing (RFC 8032 pure, never prehashed) and
  X25519 for ECDH (RFC 7748), as separate key pairs. The Ed25519->X25519
  conversion is banned outright; neither the JDK nor CryptoKit can perform it.
- Domain-separated labels on every derivation, and a transcript hash binding
  the handshake to every key and identifier it used.
- Nominal key types, so an Ed25519 public key is not assignable to an X25519
  parameter and the compiler says so.
- Ordered parsing gates in which no offset is ever derived from a received
  byte. v3 read a wire-supplied length and used it as a subdata bound.
- Atomic decryption: the ratchet step, chain advance, counter increment and
  skipped-key insert all happen on a snapshot committed only after the AEAD
  authenticates. Otherwise one unauthenticated message permanently
  desynchronises a live session.
- Every libsodium and Sec* return value checked. NSMutableData dataWithLength:
  zero-fills, so an unchecked RNG failure yields an all-zero key that both
  parties agree on, with nothing to observe.
- An explicit zeroization schedule covering failure exits, not just success.
- A single injectable clock and CSPRNG, because a frozen conformance suite that
  reads the wall clock has a shelf life of seven days.

Layering is enforced rather than described: IRSodiumCryptoProvider is the only
file permitted to call libsodium, and every layer above it holds an
id<IRCryptoProvider>. That seam is where a platform crypto backend gets
swapped, which is precisely what the three ports do.

tools/lint_banned_apis.py runs as a pre-Sources build phase, so a banned API,
a second crypto_sign_detached call site, or a raw *error assignment stops the
build rather than shipping.

526 tests, 0 failures. The suite deliberately includes property-based tests for
the invariants a round-trip cannot see — that altering any single DH input
changes SK, that every message key moves when SK moves, that every bit flip in
a message fails to decrypt, and that a failed authentication leaves the session
state byte-identical.
The podspec goes. CocoaPods entered maintenance mode and the trunk is winding
down; shipping a v4 that can only be installed through it would be shipping a
dependency on something being retired. Package.swift replaces it, with no
version field — SwiftPM takes that from the git tag.

The manifest carries a long comment that is not decoration. Sources here are
flat in nuntius/ and import each other framework-style as <nuntius/Foo.h>,
which Xcode resolves through a header map and SwiftPM does not have. SwiftPM
passes dependents exactly one include directory, so that directory must
*contain* a directory named nuntius. include/nuntius is a relative symlink to
../nuntius that produces exactly that shape without moving a single file. Three
simpler layouts were tried first; each fails at graph load, and the manifest
records the verbatim error for each so nobody has to rediscover them. The
symlink looks redundant and is not — deleting it breaks the package.

The README described software that no longer exists: the v3 API surface, all of
it deleted; AES-256-CBC + HMAC via CommonCrypto, replaced; and `pod "nuntius"`,
gone. It is rewritten against the headers that are actually in the tree, and it
states the security posture honestly, including that the transcript-hash
binding has not been externally reviewed and that the conformance vector corpus
specified in SPEC §15 has not been generated yet.

The CHANGELOG's 1.0.0 entry leads with the break rather than burying it. The
Security section names the concrete v3 defects, because "upgrade for security
fixes" is not actionable and anyone still running 0.0.9 needs to know that the
key agreement was not doing what it claimed.
Ivan added 3 commits July 21, 2026 01:30
SPEC §15 has specified this corpus since the rewrite began; it did not exist, which
made "all four implementations interoperate byte-for-byte" a claim with nothing
behind it. It exists now: 88 vectors across six files under spec/vectors/,
covering every id §15.3 and §15.4 require.

This is the artifact the Java, Kotlin and Swift ports are graded against. They are
written against SPEC.md rather than against this code, so the vectors are the only
thing that can actually catch a divergence — and §15.1 exists because the v3 suite
was green for years while X3DH silently ran on a single Diffie-Hellman.

Three properties make it worth trusting:

  * The RFC-anchored vectors are TRANSCRIBED from RFC 5869, 7748, 8032 and 8439,
    never produced here. A corpus generated entirely by the implementation under
    test proves self-consistency and nothing else; these are the only external
    check in it. Every value was re-fetched from rfc-editor.org and compared
    mechanically, and the nuntius derivations were independently recomputed in
    Python from the SPEC prose — the X3DH handshakes end to end, and
    RATCHET-LINEAR reimplemented from scratch through §6, §7, §8 and §9.

  * The suite rebuilds the corpus in memory on every run and compares it
    byte-for-byte against the frozen files. Implementation drift fails a test
    rather than quietly rewriting the contract. Rewriting takes the explicit
    spec/vectors/.regenerate sentinel, which then fails the run on purpose so the
    diff gets reviewed. §15.6 step 4 makes a changed vector a spec version bump,
    and a failing port is never fixed by regenerating a vector.

  * No vector reads the host clock. Every clock read routes through one injectable
    source, and the suite runs again with the clock ten years forward. Without
    that, a corpus green on the day it is frozen goes red seven days later on the
    skipped-key TTL and ninety days later on the prekey validity window — with
    §15.6 forbidding the obvious workaround.

Building it found two defects in the specification, both of which would have been
frozen into the contract:

  * §3.4 claimed Ed25519 signing produces byte-identical output across libsodium,
    the JDK and CryptoKit. It does not. RFC 8032 §8.2 permits additional
    randomness and CryptoKit takes that option — signing one message three times
    under one seed yields three different valid signatures, none equal to RFC
    8032's published one. As written, §15.3's mandatory RFC8032-ED25519 vector was
    unpassable by a conformant Swift port. Signatures are now asserted verify-side
    only: a signature is always an input, never an expected output (§15.5 rule 8).

  * §4.4 claimed CryptoKit performs no small-order check. It does — key agreement
    throws. The MUST is unchanged, and now says why: an implementation may not
    skip its own check because it trusts the library underneath, and that trust is
    exactly what was wrong here, in the direction that sounds safe.

Also corrected: X25519-ZERO listed six small-order u-coordinates with bit 255
clear when there are seven, confirmed from the curve arithmetic rather than from a
copied blacklist. And the §10.3 bundle rejections are kind "wire", not "state" —
a prekey bundle is a §5.4 wire structure, and a port switching on kind would
otherwise hand 251 bytes of bundle to its state-blob parser.

529 tests, 0 failures.
Each is a place where SPEC.md left a port to guess, and all three were found the
same way: three people implementing the same document from scratch reached the
same three gaps.

  * Error code 7106 named three conditions and §4.4 defined two. "Reflected own
    key" had no home — the reflection checks existed, but scattered across §10.1
    check 8, §10.2 check 11, §10.7 step 6 and §11.2, so a port reading §4.4 as the
    definition of public-key validation implemented two thirds of the code. Now
    §4.4 check 2b, with an exhaustive table of the four sites and the value each
    compares against, and a MUST NOT on adding a fifth: an extra rejection is an
    interop divergence, not extra safety.

    Numbered 2b rather than 3 on purpose. Six sections cite "checks 1-2" and
    "check 3" by number, so renumbering would have silently redirected all of
    them.

  * Error code 7118 was specified as a sodium_init() failure. Neither BouncyCastle
    nor CryptoKit has an initialization step, so on half the target platforms the
    code was unreachable with undefined meaning — which is an invitation to invent
    a use for it. Restated backend-neutrally, and it now says explicitly that
    having no reachable condition is conformant rather than a gap, and names the
    three plausible misuses it MUST NOT be stretched to cover.

  * §13.1 claimed the platform CSPRNG "throws rather than returning a status".
    False for SecureRandom.nextBytes, which returns void and declares no checked
    exception — and false in the direction that sounds safe. The section now leads
    with the invariant rather than a claim about any library: a fill that does not
    succeed MUST NOT leave usable bytes behind and the caller MUST NOT proceed,
    because NSMutableData dataWithLength: / new byte[n] / Data(count:) all
    zero-fill, so a silent failure yields an all-zero key both parties agree on.
    That is v3's defect 4 verbatim.

    One requirement was strengthened rather than restated: the all-zero tripwire
    is now REQUIRED on the JVM. With neither a status nor a guaranteed exception
    it is the only mechanism by which the invariant can be discharged there, so
    without it the JVM had no conformant way to satisfy a MUST.

Also swept the document for passages still asserting the two claims retracted
while building the corpus — that Ed25519 signing is byte-reproducible across
platforms, and that CryptoKit performs no small-order check. Three survived, in
§6.2, §11.3 and §17.8; all corrected, and §17.8 additionally no longer tells each
port to extend primitives.json unilaterally, which §15.6 step 4 forbids.

No frozen vector changed. 529 tests, 0 failures.
Adds a "Read this first" notice to the README, a banner to the CHANGELOG, and
the same posture to CLAUDE.md so a contributor does not have to read the README
to learn it.

The notice is deliberate about one distinction: v4 is complete AND unaudited,
and those are two different claims. Saying "experimental" must not be read as
"unfinished, check back later" — it is specified to the byte, implemented, and
graded against a frozen corpus. What it has never had is review by anyone who
did not write it, and SPEC §17.7 singles out the transcript-hash binding in
particular.

The strongest argument in the notice is this repo's own history. v3 shipped for
years with a green test suite while its X3DH silently collapsed to a single
Diffie-Hellman, because the suite asserted that both parties derived the same 32
bytes — which is exactly what the collapse also produces. "The tests pass,
therefore the cryptography is sound" is the inference that kept it in
production, and no amount of care in this release makes it safe to repeat. That
is worth saying plainly next to a corpus of 88 passing vectors.

It also says outright that carrying a specification, a reference implementation,
a frozen corpus and three ports through to this point is a scope LLM-assisted
development made reachable for one person. That is part of what the project is,
and leaving it to be inferred would be coy.

Renames "Production wiring" to "Wiring for a real deployment", with both
referring anchors updated. The section body was always honest about the
in-memory stores not being production stores, but the heading is what a skimmer
sees, and it sat four TOC entries below a notice saying not for production.

Replaces .travis.yml with .github/workflows/ci.yml. The Travis file pinned
xcode8.3 and an iPhone 7 / iOS 10.3.1 simulator that no longer exists, so it
could not run — and the README asserts in two places that CI exists, which
deleting without replacing would have made false at exactly the moment the repo
goes public. The new workflow resolves its simulator at run time rather than
pinning a name, because pinning is what rotted the old one. Its last step is
`git diff --exit-code -- spec/vectors`: the suite regenerates the corpus in
memory and compares, so a run that MODIFIED a frozen file means the compare path
never executed, and §15.6 step 4 makes that a four-repo spec version bump. CI is
where that should surface, not a release.

Also untracks two per-user Xcode state files that predate the .gitignore rule
already covering them, and ignores IMPLEMENTATION_PLAN.md — internal scaffolding
written in the future tense that asserts a v3 test suite which no longer exists.

529 tests, 0 failures. No frozen vector changed.
@ivRodriguezCA
ivRodriguezCA marked this pull request as ready for review July 21, 2026 12:10
@ivRodriguezCA
ivRodriguezCA merged commit 0fa225c into main Jul 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant