From 6c4a1f89166675f774af7a6a5233395195196cbb Mon Sep 17 00:00:00 2001 From: Patrick Scheid Date: Thu, 23 Jul 2026 15:51:21 +0200 Subject: [PATCH] Refresh benchmarks against current main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Regenerate both the CLI comparison and the BenchmarkDotNet microbenchmarks on the current tree (Apple M2, .NET 10). Numbers now reflect the IAeadCipher backend seam and the pre-auth parser bounds. CLI wall-clock is essentially unchanged (within noise; every relative ranking holds). Microbenchmark times moved a few percent; the notable change is the per-operation ChaCha-path allocation (Encrypt 1 KB ~12->20 KB, X25519 wrap ~3.6->6.6 KB) from the cipher seam. The armored decrypt path — the only hot path the parser bounds touch — stayed within noise, confirming the bound's one-scan-per-read cost is negligible. --- docs/BENCHMARKS.md | 100 ++++++++++++++++++++++++--------------------- 1 file changed, 53 insertions(+), 47 deletions(-) diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 887a8ff..6319fac 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -1,10 +1,14 @@ # Benchmarks +Measured on Apple M2 (8 cores, 16 GB), macOS 26, .NET 10, on 2026-07-23. +Numbers are refreshed against `main` at the current release (includes the +`IAeadCipher` backend seam and the pre-authentication parser bounds). + ## CLI Comparison: age (Go) vs rage (Rust) vs age-sharp (C#/.NET) Wall-clock time for encrypt and decrypt at various file sizes, averaged -over 5 iterations. Measured on Apple M2, .NET 10, AOT-compiled binary. -Includes process startup, key parsing, header processing, and I/O. +over 5 iterations. AOT-compiled binary. Includes process startup, key +parsing, header processing, and I/O. All times in milliseconds (lower is better). @@ -12,41 +16,41 @@ All times in milliseconds (lower is better). | Size | Op | age (Go) | rage (Rust) | age-sharp (C#) | |---|---|---:|---:|---:| -| 1 KB | enc | 21 ms | 20 ms | 22 ms | -| 1 KB | dec | 21 ms | 20 ms | 21 ms | +| 1 KB | enc | 21 ms | 21 ms | 22 ms | +| 1 KB | dec | 20 ms | 22 ms | 22 ms | | 64 KB | enc | 21 ms | 21 ms | 22 ms | -| 64 KB | dec | 21 ms | 21 ms | 22 ms | -| 1 MB | enc | 22 ms | 23 ms | 24 ms | -| 1 MB | dec | 22 ms | 23 ms | 25 ms | -| 10 MB | enc | 34 ms | 47 ms | 41 ms | -| 10 MB | dec | 34 ms | 51 ms | 46 ms | -| 100 MB | enc | 155 ms | 285 ms | 211 ms | -| 100 MB | dec | 144 ms | 324 ms | 242 ms | +| 64 KB | dec | 20 ms | 21 ms | 22 ms | +| 1 MB | enc | 23 ms | 24 ms | 25 ms | +| 1 MB | dec | 22 ms | 24 ms | 25 ms | +| 10 MB | enc | 37 ms | 49 ms | 44 ms | +| 10 MB | dec | 35 ms | 52 ms | 47 ms | +| 100 MB | enc | 173 ms | 298 ms | 222 ms | +| 100 MB | dec | 147 ms | 323 ms | 244 ms | ### ASCII Armor (-a) | Size | Op | age (Go) | rage (Rust) | age-sharp (C#) | |---|---|---:|---:|---:| | 1 KB | enc | 21 ms | 20 ms | 22 ms | -| 1 KB | dec | 21 ms | 20 ms | 21 ms | +| 1 KB | dec | 20 ms | 20 ms | 22 ms | | 64 KB | enc | 21 ms | 21 ms | 22 ms | -| 64 KB | dec | 21 ms | 22 ms | 22 ms | -| 1 MB | enc | 25 ms | 24 ms | 24 ms | +| 64 KB | dec | 21 ms | 21 ms | 22 ms | +| 1 MB | enc | 27 ms | 24 ms | 25 ms | | 1 MB | dec | 24 ms | 25 ms | 26 ms | -| 10 MB | enc | 67 ms | 55 ms | 45 ms | -| 10 MB | dec | 51 ms | 64 ms | 59 ms | -| 100 MB | enc | 436 ms | 359 ms | 251 ms | -| 100 MB | dec | 319 ms | 450 ms | 383 ms | +| 10 MB | enc | 72 ms | 56 ms | 46 ms | +| 10 MB | dec | 53 ms | 64 ms | 60 ms | +| 100 MB | enc | 456 ms | 366 ms | 275 ms | +| 100 MB | dec | 325 ms | 449 ms | 409 ms | ### Key Takeaways - **Up to 1 MB**: All three implementations are within noise of each other - (~20-26 ms), dominated by process startup overhead. -- **Binary 100 MB**: Go still leads at 144-155 ms thanks to assembly-optimized - ChaCha20-Poly1305. AgeSharp (211-242 ms) beats rage (285-324 ms) after - switching to .NET's hardware-accelerated `ChaCha20Poly1305`. -- **Armored 100 MB encrypt**: AgeSharp (251 ms) now beats both rage (359 ms) - and Go (436 ms) after routing the push-encrypt path through `EncryptStream` + (~20-27 ms), dominated by process startup overhead. +- **Binary 100 MB**: Go still leads at 147-173 ms thanks to assembly-optimized + ChaCha20-Poly1305. AgeSharp (222-244 ms) beats rage (298-323 ms) using + .NET's hardware-accelerated `ChaCha20Poly1305`. +- **Armored 100 MB encrypt**: AgeSharp (275 ms) beats both rage (366 ms) + and Go (456 ms) — the push-encrypt path runs through `EncryptStream` + `ArmorStream` without any intermediate buffer. - **Bounded memory**: all public push APIs (`Encrypt`, `Decrypt`, `EncryptDetached`, `DecryptDetached`) stream chunk-by-chunk with pooled, @@ -75,47 +79,49 @@ process startup overhead. | Operation | 1 KB | 64 KB | 1 MB | |---|---:|---:|---:| -| Encrypt | 98 us | 195 us | 2,038 us | -| Decrypt | 95 us | 206 us | 2,088 us | -| Encrypt (armored) | 99 us | 223 us | 2,349 us | -| Decrypt (armored) | 99 us | 281 us | 3,309 us | +| Encrypt | 102 us | 204 us | 2,092 us | +| Decrypt | 101 us | 215 us | 2,118 us | +| Encrypt (armored) | 102 us | 235 us | 2,485 us | +| Decrypt (armored) | 108 us | 294 us | 3,382 us | -Throughput at 1 MB: ~500 MB/s encrypt, ~490 MB/s decrypt. -Armored adds ~15-60% overhead due to Base64 encoding/decoding. +Throughput at 1 MB: ~500 MB/s encrypt, ~495 MB/s decrypt. +Armored adds ~15-60% overhead due to Base64 encoding/decoding. The +pre-auth parser bounds add one vectorized newline scan per buffered read +on the armored-decrypt path; the cost is within measurement noise. -Allocations at 1 KB are ~12 KB (Encrypt) / ~9 KB (Decrypt) — chunk -scratch buffers are rented from `ArrayPool.Shared` and reused -across operations. At 1 MB, most allocation is the output -`MemoryStream`'s growth, not the crypto path. +Allocations at 1 KB are ~20 KB (Encrypt) / ~17 KB (Decrypt) — a small, +constant per-operation cost (cipher setup plus chunk scratch buffers +rented from `ArrayPool.Shared` and reused across chunks). At 1 MB, +most allocation is the output `MemoryStream`'s growth, not the crypto path. ### Key Generation | Operation | Time | Allocated | |---|---:|---:| -| X25519 | 1,919 ns | 880 B | -| ML-KEM-768-X25519 | 246 ns | 88 B | +| X25519 | 1,982 ns | 880 B | +| ML-KEM-768-X25519 | 251 ns | 88 B | ### Recipient Wrap / Unwrap | Operation | Time | Allocated | |---|---:|---:| -| X25519 Wrap | 87 us | 3.6 KB | -| X25519 Unwrap | 84 us | 2.9 KB | -| ML-KEM-768-X25519 Wrap | 132 us | 46.7 KB | -| ML-KEM-768-X25519 Unwrap | 174 us | 69.9 KB | -| Scrypt Wrap | 1,785 us | 1,035 KB | -| Scrypt Unwrap | 1,778 us | 1,035 KB | - -X25519 is the fastest at ~85 us. ML-KEM hybrid adds ~60-100% overhead -(still sub-200 us). Scrypt is intentionally slow (~1.8 ms) due to the +| X25519 Wrap | 89 us | 6.6 KB | +| X25519 Unwrap | 88 us | 5.8 KB | +| ML-KEM-768-X25519 Wrap | 137 us | 46.6 KB | +| ML-KEM-768-X25519 Unwrap | 182 us | 70.0 KB | +| Scrypt Wrap | 1,858 us | 1,035 KB | +| Scrypt Unwrap | 1,857 us | 1,035 KB | + +X25519 is the fastest at ~88 us. ML-KEM hybrid adds ~55-105% overhead +(still sub-200 us). Scrypt is intentionally slow (~1.9 ms) due to the password-hashing work factor. ### Random Access | Operation | Time | Allocated | |---|---:|---:| -| Sequential Read | 29.7 ms | 32.0 MB | -| Random Read | 31.5 ms | 34.2 MB | +| Sequential Read | 30.2 ms | 32.0 MB | +| Random Read | 32.1 ms | 34.2 MB | Random reads are only ~6% slower than sequential thanks to the chunk-based design.