diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index e7f5d42..6cb999c 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -105,6 +105,34 @@ jobs: cp scripts/check.sh "$RUNNER_TEMP/check.sh" git rebase --exec "bash $RUNNER_TEMP/check.sh" "origin/$BASE_REF" + # WP-8 / issue #16: disassemble the release build for the targets we ship and + # look for division and for conditional branches nobody has reviewed. The + # toolchain is pinned on purpose: the reviewed branch counts in + # scripts/asm_check_allowlist.json belong to one compiler, and bumping it is + # a decision that comes with a new run of the script (see SECURITY.md). It is + # deliberately not part of scripts/check.sh, so it does not slow down the + # per-commit gate. The statistical timing test (scripts/timing_decaps.py) is + # a manual step: shared runners are too noisy for it. + asm-check: + name: release assembly check + runs-on: ubuntu-latest + env: + ASM_CHECK_TOOLCHAIN: '1.95.0' + steps: + - uses: actions/checkout@v6 + - uses: actions/setup-python@v6 + with: + python-version: '3.x' + - name: Install Rust toolchain + run: > + rustup toolchain install "$ASM_CHECK_TOOLCHAIN" --profile minimal --no-self-update + --component llvm-tools + --target aarch64-unknown-linux-gnu + --target i686-unknown-linux-gnu + --target armv7-unknown-linux-gnueabihf + - name: Check the release assembly + run: python scripts/asm_check.py --toolchain "$ASM_CHECK_TOOLCHAIN" + linux: runs-on: ${{ matrix.platform.runner }} strategy: diff --git a/CLAUDE.md b/CLAUDE.md index 27948c8..9d945d0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,7 +29,7 @@ pytest python/tests/test_kyber.py::test_kyber768_new_api `make clean` deletes `target/`, the built `.so`, and the `.pyi`; rebuild afterwards. -Known baseline (2026-10-30, after WP-4 / #12 and WP-5 / #13 merged): `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test` and `pytest python/tests` all pass on `dev-v2`, and the legacy golden vectors (#6) are in. CI (`.github/workflows/CI.yml`) has a `test` job (Linux/macOS/Windows, Python 3.8 and newest) and an `every commit passes` job that runs `scripts/check.sh` over each commit of a PR via `git rebase --exec`; making them required checks on `dev-v2` is a branch-protection setting only the maintainer can change (#7). The clippy lints `needless_range_loop`, `identity_op` and `explicit_counter_loop` are allowed per module in `src/kyber/` because the legacy loops mirror the reference C code; don't add new code that needs them. CI installs the latest stable toolchain, which can be newer than the one on your machine and bring new clippy lints (rustc 1.99 added `chunks_exact_to_as_chunks`, which failed PR #28 although 1.95 was clean), so run clippy with a recent stable before pushing: `rustup toolchain install stable` and `cargo +stable clippy --all-targets -- -D warnings`. +Known baseline (2026-10-30, after WP-4 / #12, WP-5 / #13, WP-6 / #14 and WP-7 / #15 merged): `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test` and `pytest python/tests` all pass on `dev-v2`, and the legacy golden vectors (#6) are in. The `Kyber*` classes warn once per process on first use (`KyberDeprecationWarning`, counted in subprocess tests), `docs/migration.md` is the Kyber to ML-KEM guide, and the version in `Cargo.toml` and `pyproject.toml` is already 2.0.0 with its changelog entry. `scripts/asm_check.py` (release-assembly check: division and unreviewed conditional branches in the decaps closure for x86_64, aarch64, i686, armv7; reviewed branches live in `scripts/asm_check_allowlist.json`) runs in its own CI job `release assembly check` on a pinned rustc, not in `scripts/check.sh`; rerun it on every toolchain bump and release. `scripts/timing_decaps.py` is the manual dudect-style timing test; results are in `SECURITY.md`. CI (`.github/workflows/CI.yml`) has a `test` job (Linux/macOS/Windows, Python 3.8 and newest) and an `every commit passes` job that runs `scripts/check.sh` over each commit of a PR via `git rebase --exec`; making them required checks on `dev-v2` is a branch-protection setting only the maintainer can change (#7). The clippy lints `needless_range_loop`, `identity_op` and `explicit_counter_loop` are allowed per module in `src/kyber/` because the legacy loops mirror the reference C code; don't add new code that needs them. CI installs the latest stable toolchain, which can be newer than the one on your machine and bring new clippy lints (rustc 1.99 added `chunks_exact_to_as_chunks`, which failed PR #28 although 1.95 was clean), so run clippy with a recent stable before pushing: `rustup toolchain install stable` and `cargo +stable clippy --all-targets -- -D warnings`. ## Architecture diff --git a/SECURITY.md b/SECURITY.md index 5b82b44..493b280 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -109,3 +109,194 @@ The `keccak_squeeze` function used `store_u64` blindly, which could write past t The 7-layer Number Theoretic Transform (NTT) used `i16` intermediate values. While mathematically bounded, it was close to the limit. - **Resolution:** Hardened NTT butterfly logic using explicit `i32` arithmetic and updated `barrett_reduce` to handle a wider input range. - **Status:** FIXED. + +## Release gate: compiler-level results (WP-8, issue #16) + +Run on 2026-10-30. This section records what the two tools found, what they +cannot show, and how to rerun them. Both tools look at one compiler, one flag +set and the targets named here. None of it is a constant-time guarantee. + +### Tools and versions + +| item | value | +| --- | --- | +| rustc / cargo | 1.95.0 (59807616e 2026-04-14) / 1.95.0 (f2d3ce0bd 2026-03-21) | +| disassembler | `llvm-objdump` from the toolchain's `llvm-tools` component, LLVM 22.1.2-rust-1.95.0-stable | +| build | `cargo rustc --locked --release --lib --crate-type rlib --target `, no `RUSTFLAGS`, release profile (opt-level 3, codegen-units 16, no LTO) | +| targets | `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu` (full), `i686-unknown-linux-gnu`, `armv7-unknown-linux-gnueabihf` (spot check; the same checks run) | +| timing machine | Intel Core i7-7700, Linux 7.0, CPython 3.14.4, `powersave` governor, process pinned to CPU 2 | + +### Assembly check (`scripts/asm_check.py`) + +The script builds an rlib per target, disassembles every function, and starts +at the decapsulation entry points: ML-KEM `mlkem::kem::decaps` (three +parameter sets) and the legacy `kyber::kem::crypto_kem_dec`, `_512`, `_1024`. +It follows the call relocations and checks every function reachable from them +(39 to 47 functions per family). That is how the leaf functions are found +without touching `src/`: release builds inline `poly_from_msg`, the compress +routines, `verify` and `cmov` into their callers, and the closure covers them +wherever they end up. The script also lists which of the leaf functions named +in the issue exist as symbols of their own. + +Per function it checks: + +- division: `div`/`idiv`/`udiv`/`sdiv`, calls to `__udivdi3`, `__aeabi_idiv` + and the like, and the panic for a division by a non-constant; +- calls to `memcmp`/`bcmp`; +- computed jumps (jump tables, `br`, `tbb`, `jmp *reg`); +- conditional branches, classified by shape because a tool cannot tell a + secret from a public condition: a backward branch is a loop, a branch to a + block that ends in a panic call is a bounds/length guard, and everything + else is "other". Every "other" branch must be covered by + `scripts/asm_check_allowlist.json`: a reviewed ceiling per function and + target, with a reason. More branches than the ceiling is a finding, so a + compiler or source change that adds a branch forces a new review. + +Result for the five checks above, on all four targets, both families: + +| area | ML-KEM | legacy Kyber | +| --- | --- | --- | +| division instruction or helper call in the decaps closure | no findings | no findings | +| `poly_tomsg`, `poly_compress`, `poly_decompress`, `polyvec_*compress` | n/a (ML-KEM uses `compress::poly_compress`/`poly_decompress`: no findings) | no findings | +| `poly_frommsg` / `poly_from_msg` | no findings | no findings (inlined into `indcpa_enc`; no branch on the message) | +| compare and select (`reject_on_mismatch`; `verify`, `cmov`) | no findings | no findings (inlined into `crypto_kem_dec`: only the three length checks branch) | +| computed jumps | no findings | no findings | +| unreviewed conditional branches | none | none | + +What the reviewed "other" branches are: input length checks at entry, +`Result` tag tests, remainder and loop-exit tests on public lengths and +counters (vector-loop prologues, zeroize loops), rejection sampling of the +public matrix A, and sponge-buffer position tests. In every listing read the +compared operand was an immediate, a length, a position counter, or a tag, and +none came from `m'`, `K`, `c'` or the reject flag. The one exception to "no +data compare" is the `dk` hash check `H(ek) == h` (FIPS 203 section 7.3), +which compares two values derived from the public `ek`; on armv7 that 32-byte +compare is a `bcmp` call and is allowlisted for that reason. The 144 +branches of `shake256` on aarch64 are unrolled byte-copy tails; on i686 the +length they are compared with lives in a stack slot, which I inferred and did +not trace. + +Mutation check of the tool itself (on a scratch copy, not committed): +replacing a multiply-shift in `compress.rs` by a division by a non-constant was +reported as `div`/`udiv` plus the divide-by-zero panic on x86_64 and aarch64, +and replacing the mask select in `reject_on_mismatch` by `if black_box(diff) != +0` raised the branch count of `decaps` above its ceiling on both. Rewriting it +with a plain `if diff != 0` did not produce a finding because LLVM turned it +back into a select; the tool reports what the compiler emitted, which is the +point, and also why a source-level `if` is not proof of a branch. + +What this check does not show: + +- Backward branches are assumed to be loops on public counters and panic + guards to depend on public lengths. A secret-indexed table lookup would look + like a bounds-check panic guard and would not be caught; that rule rests on + source review. +- It reads the rlib, not the final extension module, and the rlib is not linked + with LTO. A different profile, `-C target-cpu`, LTO, PGO or another LLVM + can change the result. Cross-compiled targets were not executed. +- Cryptographic state in registers and stack is not looked at (zeroization is + checked only by the existing tests and review). +- Not covered: `s390x`, `ppc64le`, the Android targets, `musl`, Windows and + macOS builds, all of which are in the wheel matrix. No floating-point check. +- The `armv7` and `i686` entries were checked by listing the compare operand of + each branch, as for the others, but they are spot checks of the same + closure, not a separate review of the source. + +### Timing test (`scripts/timing_decaps.py`) + +For one fixed key it times decapsulation of valid ciphertexts against +ciphertexts with one flipped bit and against random bytes. Run: +`--sets 512,768,1024 -n 100000 --cpu 2` (seed 20261030, 100 000 timings per +class, 300 000 per parameter set and family, interleaved in a seeded random +order, Welch t on raw timings and after cropping at the 99th, 95th, 90th and +80th percentile; a pair is flagged only if |t| > 5.0 overall and in both halves +with the same sign). + +Result: no flag for either family at 512, 768 and 1024. The largest |t| over +all tests was 2.59 (ML-KEM-1024). The smallest difference the run could have +detected at |t| = 5, from the observed variance, was 300 to 920 ns on the +uncropped timings, 160 to 450 ns at the 99th percentile crop, and 16 to 410 ns +at the 80th percentile crop (the smallest values are for the quiet runs), +against decapsulation times of 40 to 100 us. Positive control (`--inject-ns`): 100 ns and 200 ns +added to the non-valid classes were flagged for ML-KEM-768 and Kyber-768 at +the same count (on the 95th-percentile and 80th-percentile crops; the uncropped +test did not reach threshold at 100 ns). So a leak of a few hundred ns that shows on every call would be +found; one below roughly 100 ns in a call of 60 us (about 0.15 percent) is not +excluded, and the noisier runs (Kyber-768 here) detect less. + +Limits: the valid ciphertexts come from `encapsulate()` and use the OS random +generator, so a rerun is not bit-identical (key pair, flip positions, random +ciphertexts, order are seeded). Timing goes through the Python call, so +interpreter jitter adds to the noise. Only x86_64 was measured. A different CPU +or compiler needs its own run. + +The test is a measurement tool, not a correctness test, so it is not part of +`cargo test`, `pytest` or `scripts/check.sh`. + +The valgrind/ctgrind variant (mark the secret key undefined and let memcheck +report branches and indexes that depend on it) was **not run**: valgrind is not +installed on the review machine and could not be installed. It also needs a +test inside the crate (the ML-KEM internals are `pub(crate)`) that issues +`VALGRIND_MAKE_MEM_UNDEFINED` on `dk`, which would be a change under `src/` or a +test module that this task did not make. To try it later: +`valgrind --error-exitcode=1 --track-origins=yes target/release/deps/ ` +on a release build of such a test. + +### Findings + +Format as in AGENTS.md. Nothing under `src/` or `python/pykyber/` was changed. + +``` +ID: SEC-001 +Title: Legacy verify/cmov/poly_frommsg have no optimization barrier +Severity: Low (not demonstrated: the release assembly is branch-free on all four targets) +Location: src/kyber/verify.rs (verify, cmov); src/kyber/poly.rs (poly_frommsg) +Class: timing +Evidence: source reads `(r.wrapping_neg() >> 63)`, `b.wrapping_neg()` masks and `((msg >> j) & 1).wrapping_neg()` with no black_box. rustc 1.95.0 keeps them branch-free + (crypto_kem_dec has three length-check branches only, indcpa_enc/indcpa_dec only counters). The pattern is the one of CVE-2024-37880; nothing guarantees a + different compiler or flag does the same. Hypothesis for other toolchains. +Related: CVE-2024-37880 https://nvd.nist.gov/vuln/detail/CVE-2024-37880 ; https://pqshield.com/?p=3736 +Legacy impact: no (a barrier changes no output; golden vectors still decide) +Suggested fix + test: route the masks through core::hint::black_box as src/mlkem/ct.rs and compress.rs do; keep scripts/asm_check.py in CI as the regression test. +Proposed for #18, not filed. + +ID: SEC-002 +Title: variable-time 32-byte compare in the dk hash check on armv7 +Severity: Info (both operands derive from the public ek) +Location: src/mlkem/check.rs check_dk, inlined into kem::decaps +Class: timing +Evidence: `bcmp` call at decaps+0x88 in the armv7 listing; aarch64 and x86_64 use a word-compare chain and a branch. +Related: FIPS 203 section 7.3 +Legacy impact: no +Suggested fix + test: none needed; if a stricter policy is wanted, use the constant-time compare of src/mlkem/ct.rs. The allowlist records the single permitted call. +Not a finding against "ML-KEM passes": it is on public data. +``` + +No finding was made for the ML-KEM code. "ML-KEM passes" is met in the +sense that the tools above found nothing; the limits listed above apply. + +### Sources checked (2026-10-30) + +- KyberSlash: could not be fetched here (parse error). RUSTSEC-2023-0079 (`pqc_kyber`, division of a secret by a public value) read via search results at . No change to what is assumed; the script has no division in either decaps closure. No RustSec advisory for the `ml-kem` crate turned up in one search (not confirmed against the RustSec index). +- CVE-2024-37880 (reference implementation, Clang, `poly_frommsg`): ; the NVD page body did not load, the description was confirmed from search snippets and . Treated as confirmed for the pattern; see SEC-001. +- CVE-2026-6330 (wolfSSL, ARM64 NEON ML-KEM ciphertext comparison covers half of the input): confirmed on NVD (published 2026-06-25) via search results, . Different library and code; relevant here as the class "comparison that skips part of the ciphertext". `reject_on_mismatch` is exercised by a test that flips every bit of a full-length ciphertext (src/mlkem/ct.rs) and the timing test flips random positions over the whole ciphertext; the release assembly shows a full-length loop. Not affected as far as these show. +- CVE-2026-115408 (crypto11): no result in one search; still unconfirmed. The earlier note in PLAN-2.0.md stands. +- 2026 side-channel news item (SEALSQ) and the AFIT framework: not rechecked; no change. + +### Rerunning + +```bash +rustup component add llvm-tools +rustup target add aarch64-unknown-linux-gnu i686-unknown-linux-gnu armv7-unknown-linux-gnueabihf +python scripts/asm_check.py # exit 1 on a finding; --show-other lists each reviewed branch with its compare +python scripts/asm_check.py --toolchain 1.95.0 # pin a toolchain (CI does) + +python -m venv .venv && .venv/bin/pip install -r requirements-dev.txt +.venv/bin/maturin develop --release +.venv/bin/python scripts/timing_decaps.py --sets 512,768,1024 -n 100000 --cpu 2 +.venv/bin/python scripts/timing_decaps.py --sets 768 -n 100000 --cpu 2 --inject-ns 100 # positive control +``` + +Rerun the assembly check for every release and whenever the Rust toolchain +changes. When the ceilings in `scripts/asm_check_allowlist.json` have to move, +read the new listing with `--show-other` first and update the entry's reason. diff --git a/scripts/asm_check.py b/scripts/asm_check.py new file mode 100755 index 0000000..9208d5c --- /dev/null +++ b/scripts/asm_check.py @@ -0,0 +1,517 @@ +#!/usr/bin/env python3 +"""Compiler-level check of the release build (PLAN-2.0 WP-8, issue #16). + +KyberSlash (division by q) and CVE-2024-37880 (a mask turned into a branch) +came from the compiler, not from the source. This script builds the library +for several targets in release mode, disassembles it and looks at what the +compiler actually emitted for the decapsulation paths of ML-KEM and of the +legacy Kyber code. + +How the functions are found. Nothing under src/ is changed and no probe crate +is needed: `cargo rustc --release --lib --crate-type rlib` builds the same +objects as the shipped build (same profile, same codegen units) and needs no +linker, so it also works for targets we cannot link on this machine. rustc puts +every function in its own section, so llvm-objdump gives one body per symbol. +The roots are the decapsulation entry points (ML-KEM `mlkem::kem::decaps`, +legacy `kyber::kem::crypto_kem_dec*`). The script follows the call relocations +and checks every function reachable from a root, because inlining moves +`poly_from_msg`, `compress` and friends between symbols from one compiler +release to the next. The named leaf functions are listed when they exist as +symbols of their own, or reported as inlined. + +Checks, per function in the closure: + division a div/idiv/udiv/sdiv instruction, or a call to a division + helper (__udivdi3, __aeabi_idiv, ...), or the panic for a + division by a non-constant (KyberSlash class). + variable-time memory compare a call to memcmp/bcmp/strcmp. + indirect branch jump tables and computed jumps (a secret index). + conditional branches a tool cannot tell a secret branch from a public + one, so every conditional branch is classified by shape: + loop backward branch (a loop back edge), + panic one side is a block that ends in a panic call + (bounds and length checks), + other everything else. + "other" branches must be listed per function in + scripts/asm_check_allowlist.json with a reason and a ceiling. + A function that is not listed, or that has more than its + ceiling, is a finding. Each allowlist entry was reviewed by + reading the listing printed by --show-other. + +What this does not prove: loop and panic branches are assumed to depend on +public lengths and counters (the loop bounds are visible in the source); a +passing run is evidence for one compiler, one flag set and the targets named +in the report, not a constant-time guarantee. +""" + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +import tempfile +from collections import OrderedDict, defaultdict + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +ALLOWLIST = os.path.join(REPO, "scripts", "asm_check_allowlist.json") + +# name, rust triple, family, depth. "full" targets also print the branch +# detail; "spot" targets (32-bit, shipped in the wheels) get the same checks. +TARGETS = OrderedDict( + [ + ("x86_64", ("x86_64-unknown-linux-gnu", "x86", "full")), + ("aarch64", ("aarch64-unknown-linux-gnu", "arm64", "full")), + ("i686", ("i686-unknown-linux-gnu", "x86", "spot")), + ("armv7", ("armv7-unknown-linux-gnueabihf", "arm", "spot")), + ] +) + +# Decapsulation entry points. The ML-KEM symbol is generic over K, so the +# three parameter sets are three symbols with the same demangled name. +ROOTS = [ + ("ML-KEM", r"^pykyber::mlkem::kem::decaps$"), + ("legacy Kyber", r"^pykyber::kyber::kem::crypto_kem_dec(_512|_1024)?$"), +] + +# Leaf functions named in issue #16. Matched against the name without hash. +LEAVES = [ + ("poly_tomsg (legacy)", r"^pykyber::kyber::poly::poly_tomsg$"), + ("poly_frommsg (legacy)", r"^pykyber::kyber::poly::poly_frommsg$"), + ("poly_compress (legacy)", r"^pykyber::kyber::poly::poly_compress(_1024)?$"), + ("polyvec_compress (legacy)", r"^pykyber::kyber::polyvec::polyvec_compress(_1024|_512)?$"), + ("poly_decompress (legacy)", r"^pykyber::kyber::poly::poly_decompress(_1024)?$"), + ("verify (legacy)", r"^pykyber::kyber::verify::verify$"), + ("cmov (legacy)", r"^pykyber::kyber::verify::cmov$"), + ("poly_compress (ML-KEM)", r"^pykyber::mlkem::compress::poly_compress$"), + ("poly_decompress (ML-KEM)", r"^pykyber::mlkem::compress::poly_decompress$"), + ("poly_from_msg (ML-KEM)", r"^pykyber::mlkem::compress::poly_from_msg$"), + ("reject_on_mismatch (ML-KEM)", r"^pykyber::mlkem::ct::reject_on_mismatch$"), +] + +DIV_INSN = re.compile(r"^(i?div[bwlq]?|udiv|sdiv)$") +DIV_CALL = re.compile( + r"(^__(u|s)?(div|mod)(si|di|ti)3$)|(^__u?divmod(si|di|ti)4$)|(^__aeabi_(u|i)?l?div)|" + r"(^__aeabi_.*divmod)|(div_by_zero)|(rem_by_zero)|(^__(u|s)?divmoddi4$)" +) +MEMCMP_CALL = re.compile(r"^(memcmp|bcmp|strcmp|strncmp)$|core::slice::cmp|compiler_builtins::mem::(memcmp|bcmp)") +PANIC_CALL = re.compile( + r"core::panicking::|core::slice::index::|core::result::unwrap_failed|" + r"core::option::(expect|unwrap)_failed|core::slice::::copy_from_slice::len_mismatch_fail|" + r"slice_(start|end)_index|slice_index_order_fail|panic_bounds_check|len_mismatch_fail|" + r"alloc::raw_vec::|alloc::alloc::handle_alloc_error|capacity_overflow|core::unreachable|" + r"rust_panic|std::panicking::|core::str::|core::cell::panic|core::array::" +) + +COND_X86 = re.compile(r"^j(?!mp$)[a-z]+$") +ARM_CC = r"(eq|ne|hs|cs|lo|cc|mi|pl|vs|vc|hi|ls|ge|lt|gt|le)" +COND_ARM_BRANCH = re.compile(r"^(b|bl|bx|blx)" + ARM_CC + r"(\.[nw])?$") +COND_ARM_PC = re.compile(r"^(pop|ldr|ldm[a-z]*|mov)" + ARM_CC + r"(\.[nw])?$") +COND_A64 = re.compile(r"^(b\.[a-z]{2}|cbz|cbnz|tbz|tbnz)$") + +HASH = re.compile(r"::h[0-9a-f]{16}\b") +LLVM_SUFFIX = re.compile(r"\s*\(\.llvm\.\d+\)|\.llvm\.\d+") +FUNC_RE = re.compile(r"^[0-9a-f]+ <(.+)>:$") +SECTION_RE = re.compile(r"^Disassembly of section (\S+):$") +INSN_RE = re.compile(r"^\s+([0-9a-f]+):\s+(.*)$") +RELOC_RE = re.compile(r"^\s+[0-9a-f]+:\s+(R_\S+)\s+(.*)$") +TARGET_RE = re.compile(r"0x([0-9a-f]+)") + + +def norm(name): + return LLVM_SUFFIX.sub("", name).strip() + + +def key(name): + """Name without the hash, for the allowlist and the root patterns.""" + return HASH.sub("", norm(name)) + + +def strip_addend(sym): + return re.sub(r"[+-]0x[0-9a-f]+$", "", sym.strip()) + + +class Insn(object): + __slots__ = ("addr", "mnem", "ops", "reloc", "reloc_type") + + def __init__(self, addr, mnem, ops): + self.addr = addr + self.mnem = mnem + self.ops = ops + self.reloc = None + self.reloc_type = None + + +class Func(object): + def __init__(self, name, section): + self.name = name + self.section = section + self.insns = [] + + @property + def key(self): + return key(self.name) + + +def run(cmd, **kw): + return subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True, **kw) + + +def find_objdump(explicit, toolchain): + if explicit: + return explicit + tc = ["+" + toolchain] if toolchain else [] + sysroot = run(["rustc"] + tc + ["--print", "sysroot"]).stdout.strip() + host = [l.split(": ")[1] for l in run(["rustc"] + tc + ["-vV"]).stdout.splitlines() if l.startswith("host:")][0] + cand = os.path.join(sysroot, "lib", "rustlib", host, "bin", "llvm-objdump") + if os.path.exists(cand): + return cand + found = shutil.which("llvm-objdump") + if found: + return found + sys.exit("llvm-objdump not found: run `rustup component add llvm-tools` or pass --objdump") + + +def build(triple, target_dir, toolchain): + cmd = ["cargo"] + (["+" + toolchain] if toolchain else []) + [ + "rustc", "--locked", "--release", "--lib", "--crate-type", "rlib", + "--target", triple, "--target-dir", target_dir, + ] + p = run(cmd, cwd=REPO) + if p.returncode != 0: + sys.exit("build failed for %s:\n%s" % (triple, p.stderr[-3000:])) + rlib = os.path.join(target_dir, triple, "release", "libpykyber.rlib") + if not os.path.exists(rlib): + sys.exit("no rlib at " + rlib) + return rlib + + +def disassemble(objdump, rlib): + p = run([objdump, "-C", "-d", "-r", "--no-show-raw-insn", rlib]) + if p.returncode != 0: + sys.exit("objdump failed: " + p.stderr[-2000:]) + return p.stdout + + +def parse(text): + """Returns (funcs by normalized name, section name -> normalized name).""" + funcs = OrderedDict() + sections = {} + cur = None + section = None + last = None + for line in text.splitlines(): + m = SECTION_RE.match(line) + if m: + section = m.group(1) + cur = None + continue + m = FUNC_RE.match(line) + if m: + name = norm(m.group(1)) + cur = Func(name, section) + funcs[name] = cur + sections[section] = name + last = None + continue + if cur is None: + continue + m = RELOC_RE.match(line) + if m: + if last is not None: + last.reloc = strip_addend(m.group(2)) + last.reloc_type = m.group(1) + continue + m = INSN_RE.match(line) + if m: + body = m.group(2).split("//")[0].split(";")[0].strip() + if not body: + continue + parts = body.split(None, 1) + mnem = parts[0] + # x86 prefixes + while mnem in ("rep", "repz", "repnz", "lock", "notrack", "bnd") and len(parts) > 1: + parts = parts[1].split(None, 1) + mnem = parts[0] + ops = parts[1] if len(parts) > 1 else "" + last = Insn(int(m.group(1), 16), mnem, ops) + cur.insns.append(last) + return funcs, sections + + +def is_indirect(insn, family): + m = insn.mnem + if family == "x86": + return m.startswith("jmp") and "*" in insn.ops + if family == "arm64": + return m in ("br", "braa", "brab", "braaz", "brabz") + return m in ("tbb", "tbh", "bx", "bx.n") and not re.search(r"\blr\b", insn.ops) or ( + m in ("ldr", "mov", "add") and re.match(r"^pc\b", insn.ops) is not None) + + +def callee_name(insn, sections): + sym = insn.reloc + if sym is None: + return None + if sym.startswith(".text."): + return sections.get(sym) + return norm(sym) + + +def is_transfer(insn, family): + m = insn.mnem + if family == "x86": + return m == "ret" or m.startswith("jmp") or m.startswith("call") or COND_X86.match(m) is not None + if family == "arm64": + return m in ("ret", "b", "bl", "br", "blr") or COND_A64.match(m) is not None + return m in ("b", "bl", "bx", "blx", "b.w", "b.n") or COND_ARM_BRANCH.match(m) is not None + + +def is_call(insn, family): + m = insn.mnem + if family == "x86": + return m.startswith("call") + if family == "arm64": + return m in ("bl", "blr") + return m in ("bl", "blx") + + +def cond_kind(insn, family): + m = insn.mnem + if family == "x86": + return COND_X86.match(m) is not None + if family == "arm64": + return COND_A64.match(m) is not None + if COND_ARM_BRANCH.match(m) or m in ("cbz", "cbnz"): + return True + return COND_ARM_PC.match(m) is not None and re.search(r"\bpc\b", insn.ops) is not None + + +def local_target(insn): + """Target address of an intra-function jump, or None.""" + if insn.reloc is not None: + return None + m = TARGET_RE.search(insn.ops) + return int(m.group(1), 16) if m else None + + +def block_panics(func, start_addr, index, sections, family, depth=0): + """True if the block at start_addr ends in a call to a panic function.""" + i = index.get(start_addr) + if i is None: + return False + for insn in func.insns[i:]: + if is_transfer(insn, family): + callee = callee_name(insn, sections) + if is_call(insn, family) or insn.reloc is not None: + return callee is not None and PANIC_CALL.search(callee) is not None + t = local_target(insn) + if insn.mnem.startswith("jmp") or insn.mnem in ("b", "b.w", "b.n"): + if t is not None and depth < 3: + return block_panics(func, t, index, sections, family, depth + 1) + return False + return False + + +def analyse_func(func, sections, family): + """Returns dict with divisions, memcmps, indirect, callees, branches.""" + index = {insn.addr: n for n, insn in enumerate(func.insns)} + res = {"div": [], "memcmp": [], "indirect": [], "callees": set(), "ext": set(), + "loop": [], "panic": [], "other": []} + for n, insn in enumerate(func.insns): + if DIV_INSN.match(insn.mnem): + res["div"].append((insn.addr, insn.mnem + " " + insn.ops)) + callee = callee_name(insn, sections) + # Any call or jump that carries a relocation names its target: a + # direct call (PLT32, CALL26), or a call through the GOT on x86 + # (`callq *(%rip)` with GOTPCREL), which is how the shipped code is linked. + if callee is not None and (is_call(insn, family) or is_transfer(insn, family)): + res["callees"].add(callee) + if DIV_CALL.search(callee): + res["div"].append((insn.addr, "call " + callee)) + if MEMCMP_CALL.search(callee): + res["memcmp"].append((insn.addr, "call " + callee)) + # A computed jump has no relocation; `jmp *GOT(%rip)` is a tail call. + if insn.reloc is None and is_indirect(insn, family): + res["indirect"].append((insn.addr, insn.mnem + " " + insn.ops)) + if not cond_kind(insn, family): + continue + # An externally targeted conditional (tail call) has a relocation. + t = local_target(insn) + nxt = func.insns[n + 1].addr if n + 1 < len(func.insns) else None + item = (insn.addr, insn.mnem + " " + insn.ops, n, flag_setter(func, n, family)) + if t is not None and t <= insn.addr: + res["loop"].append(item) + elif block_panics(func, t, index, sections, family) if t is not None else False: + res["panic"].append(item) + elif nxt is not None and block_panics(func, nxt, index, sections, family): + res["panic"].append(item) + else: + res["other"].append(item) + return res + + +X86_FLAGS = re.compile(r"^(cmp|test|sub|add|and|or|xor|neg|inc|dec|shl|shr|sar|sal|adc|sbb|bt[src]?|bsf|bsr|tzcnt|lzcnt|popcnt)[bwlq]?$") +A64_FLAGS = re.compile(r"^(cmp|cmn|tst|ccmp|ccmn|fcmp|[a-z]+s)$") +ARM_FLAGS = re.compile(r"^(cmp|cmn|tst|teq|[a-z]{2,5}s)(\.[nw])?$") + + +def flag_setter(func, n, family): + """Nearest earlier instruction that writes the condition flags (display only).""" + pat = {"x86": X86_FLAGS, "arm64": A64_FLAGS, "arm": ARM_FLAGS}[family] + insn = func.insns[n] + if insn.mnem in ("cbz", "cbnz", "cbz.n", "cbnz.n", "tbz", "tbnz"): + return "(tests register itself)" + for k in range(n - 1, max(-1, n - 40), -1): + prev = func.insns[k] + if pat.match(prev.mnem): + return "%s %s" % (prev.mnem, prev.ops) + return "?" + + +def closure(funcs, roots, results): + seen = OrderedDict() + work = list(roots) + while work: + name = work.pop() + if name in seen or name not in funcs: + continue + seen[name] = funcs[name] + for c in results[name]["callees"]: + if c in funcs and c not in seen: + work.append(c) + return seen + + +def load_allowlist(): + with open(ALLOWLIST) as f: + data = json.load(f) + return data + + +def show_context(func, item): + return " +0x%-5x %-34s <- %s" % (item[0], item[1].split(" <")[0][:34], item[3]) + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + ap.add_argument("--targets", default=",".join(TARGETS), help="comma list of: " + ",".join(TARGETS)) + ap.add_argument("--toolchain", default=None, help="rustup toolchain name, e.g. stable or 1.95.0") + ap.add_argument("--objdump", default=None, help="path to llvm-objdump (default: from the rust sysroot)") + ap.add_argument("--target-dir", default=None, help="build directory (default: a temp dir, removed on exit)") + ap.add_argument("--show-other", action="store_true", help="print the code around every 'other' branch") + ap.add_argument("--dump", default=None, help="write the disassembly of each target into this directory") + args = ap.parse_args() + + objdump = find_objdump(args.objdump, args.toolchain) + tc = ["+" + args.toolchain] if args.toolchain else [] + print("== environment") + print("rustc : " + run(["rustc"] + tc + ["-V"]).stdout.strip()) + print("cargo : " + run(["cargo"] + tc + ["-V"]).stdout.strip()) + print("objdump : " + objdump + " (" + run([objdump, "--version"]).stdout.splitlines()[1].strip() + ")") + print("RUSTFLAGS: " + repr(os.environ.get("RUSTFLAGS", "")) + " profile: release (opt-level 3, codegen-units 16, no LTO)") + allow = load_allowlist() + allow_funcs = dict((e["function"], e) for e in allow["other_branches"]) + ok_calls = allow.get("memcmp_calls", {}) + + tmp = None + target_dir = args.target_dir + if target_dir is None: + tmp = tempfile.mkdtemp(prefix="pykyber-asm-") + target_dir = tmp + findings = [] + try: + for tname in [t.strip() for t in args.targets.split(",") if t.strip()]: + if tname not in TARGETS: + sys.exit("unknown target " + tname) + triple, family, depth = TARGETS[tname] + print("\n==== %s (%s, %s check)" % (tname, triple, depth)) + rlib = build(triple, target_dir, args.toolchain) + text = disassemble(objdump, rlib) + if args.dump: + if not os.path.isdir(args.dump): + os.makedirs(args.dump) + with open(os.path.join(args.dump, tname + ".txt"), "w") as f: + f.write(text) + funcs, sections = parse(text) + results = dict((n, analyse_func(f, sections, family)) for n, f in funcs.items()) + shown = set() + print("%d function symbols, %d instructions" % (len(funcs), sum(len(f.insns) for f in funcs.values()))) + + print("\nleaf functions of interest (standalone symbols):") + for label, pat in LEAVES: + hits = [f for f in funcs.values() if re.match(pat, f.key)] + print(" %-30s %s" % (label, "%d symbol(s)" % len(hits) if hits else "inlined (checked inside its callers)")) + + for rlabel, rpat in ROOTS: + roots = [n for n, f in funcs.items() if re.match(rpat, f.key)] + if not roots: + findings.append("%s: no root symbol for %s (pattern %s)" % (tname, rlabel, rpat)) + print("\n[%s] NO ROOT FOUND" % rlabel) + continue + cl = closure(funcs, roots, results) + print("\n[%s] %d root symbol(s), %d functions in the call closure, %d instructions" + % (rlabel, len(roots), len(cl), sum(len(f.insns) for f in cl.values()))) + agg = defaultdict(lambda: {"loop": 0, "panic": 0, "other": 0, "n": 0, "max_other": 0}) + for name, f in cl.items(): + r = results[name] + a = agg[f.key] + a["n"] += 1 + a["loop"] += len(r["loop"]) + a["panic"] += len(r["panic"]) + a["other"] += len(r["other"]) + a["max_other"] = max(a["max_other"], len(r["other"])) + where = "%s [%s]" % (f.key, rlabel) + for addr, txt in r["div"]: + findings.append("%s: DIVISION in %s at +0x%x: %s" % (tname, where, addr, txt)) + for addr, txt in r["indirect"]: + findings.append("%s: INDIRECT BRANCH in %s at +0x%x: %s" % (tname, where, addr, txt)) + cmp_ok = ok_calls.get(f.key, {}).get("max", {}).get(tname, 0) + if len(r["memcmp"]) > cmp_ok: + for addr, txt in r["memcmp"]: + findings.append("%s: VARIABLE-TIME COMPARE in %s at +0x%x: %s (allowed: %d)" + % (tname, where, addr, txt, cmp_ok)) + ent = allow_funcs.get(f.key) + n_other = len(r["other"]) + ceiling = ent["max"].get(tname, 0) if ent is not None else 0 + if n_other > ceiling: + findings.append("%s: %s has %d unreviewed 'other' conditional branch(es), allowlist ceiling %d" + " (review with --show-other, then update scripts/asm_check_allowlist.json)" + % (tname, where, n_other, ceiling)) + if args.show_other and n_other and name not in shown: + shown.add(name) + print(" -- %s: %d other branch(es)" % (f.key, n_other)) + for it in r["other"]: + print(show_context(f, it)) + print(" %-58s %3s %5s %5s %5s" % ("function (hash removed)", "n", "loop", "panic", "other")) + print(" (loop, panic: summed over the n symbols of that name; other: the largest of them)") + for k in sorted(agg): + a = agg[k] + flag = "" + if a["max_other"]: + e = allow_funcs.get(k) + flag = " allowlist: " + (e["group"] if e else "NOT IN ALLOWLIST") + print(" %-58s %3d %5d %5d %5d%s" % (k[:58], a["n"], a["loop"], a["panic"], a["max_other"], flag)) + divs = sum(len(results[n]["div"]) for n in cl) + inds = sum(len(results[n]["indirect"]) for n in cl) + cmps = sum(len(results[n]["memcmp"]) for n in cl) + print(" division instructions/calls: %d, indirect branches: %d, memcmp-like calls: %d" % (divs, inds, cmps)) + finally: + if tmp: + shutil.rmtree(tmp, ignore_errors=True) + + print("\n== result") + if findings: + seen = set() + for f in findings: + if f not in seen: + print("FINDING: " + f) + seen.add(f) + print("%d finding(s)" % len(seen)) + return 1 + print("no findings (this is evidence for the toolchain and targets above, not a guarantee)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/asm_check_allowlist.json b/scripts/asm_check_allowlist.json new file mode 100644 index 0000000..2d153fd --- /dev/null +++ b/scripts/asm_check_allowlist.json @@ -0,0 +1,404 @@ +{ + "comment": "Reviewed conditional branches that are neither loop back edges nor panic guards, per function (name without hash) and per target; see scripts/asm_check.py. 'max' is the ceiling per target; a target that is absent allows none. Entries were reviewed by reading the compare that feeds each branch (--show-other). A change in the count means the code generation changed and the entry needs a new review.", + "other_branches": [ + { + "function": "core::ptr::drop_in_place$LT$zeroize..Zeroizing$LT$alloc..vec..Vec$LT$u8$GT$$GT$$GT$", + "group": "zeroize", + "max": { + "x86_64": 8, + "aarch64": 3, + "i686": 3, + "armv7": 3 + }, + "reason": "Zeroizing> drop: loop over the (public) vector length with a head/tail split; the bytes being wiped are never compared." + }, + { + "function": "pykyber::kyber::fips202::keccak_absorb_once", + "group": "keccak", + "max": { + "x86_64": 5, + "aarch64": 5, + "i686": 4, + "armv7": 4 + }, + "reason": "Sponge buffering: tests of the input and output lengths and of the byte position inside the rate (0x88 for SHAKE256, 0x68/0x48 for the SHA3 rates). Lengths are public in every call here (fixed per parameter set, or the ciphertext length). The permutation itself has no data-dependent branch. Unrolled byte-copy tails give the large counts; on i686 the length is spilled to the stack, so the compare reads memory." + }, + { + "function": "pykyber::kyber::fips202::sha3_256", + "group": "keccak", + "max": { + "x86_64": 21, + "aarch64": 29, + "i686": 3, + "armv7": 17 + }, + "reason": "Sponge buffering: tests of the input and output lengths and of the byte position inside the rate (0x88 for SHAKE256, 0x68/0x48 for the SHA3 rates). Lengths are public in every call here (fixed per parameter set, or the ciphertext length). The permutation itself has no data-dependent branch. Unrolled byte-copy tails give the large counts; on i686 the length is spilled to the stack, so the compare reads memory." + }, + { + "function": "pykyber::kyber::fips202::sha3_512", + "group": "keccak", + "max": { + "x86_64": 49, + "aarch64": 57, + "i686": 7, + "armv7": 33 + }, + "reason": "Sponge buffering: tests of the input and output lengths and of the byte position inside the rate (0x88 for SHAKE256, 0x68/0x48 for the SHA3 rates). Lengths are public in every call here (fixed per parameter set, or the ciphertext length). The permutation itself has no data-dependent branch. Unrolled byte-copy tails give the large counts; on i686 the length is spilled to the stack, so the compare reads memory." + }, + { + "function": "pykyber::kyber::fips202::shake256", + "group": "keccak", + "max": { + "x86_64": 144, + "aarch64": 144, + "i686": 37, + "armv7": 15 + }, + "reason": "Sponge buffering: tests of the input and output lengths and of the byte position inside the rate (0x88 for SHAKE256, 0x68/0x48 for the SHA3 rates). Lengths are public in every call here (fixed per parameter set, or the ciphertext length). The permutation itself has no data-dependent branch. Unrolled byte-copy tails give the large counts; on i686 the length is spilled to the stack, so the compare reads memory." + }, + { + "function": "pykyber::kyber::indcpa::gen_matrix", + "group": "sampling", + "max": { + "x86_64": 9, + "aarch64": 9, + "i686": 8, + "armv7": 6 + }, + "reason": "Rejection sampling of the matrix A from the XOF output keyed by the public rho (FIPS 203 Algorithm 7). Branches test a sampled 12-bit value against q, the output count against 256, and the buffer position against the SHAKE128 rate; all of it is public data." + }, + { + "function": "pykyber::kyber::indcpa::gen_matrix_1024", + "group": "sampling", + "max": { + "x86_64": 9, + "aarch64": 9, + "i686": 8, + "armv7": 6 + }, + "reason": "Rejection sampling of the matrix A from the XOF output keyed by the public rho (FIPS 203 Algorithm 7). Branches test a sampled 12-bit value against q, the output count against 256, and the buffer position against the SHAKE128 rate; all of it is public data." + }, + { + "function": "pykyber::kyber::indcpa::gen_matrix_512", + "group": "sampling", + "max": { + "x86_64": 15, + "aarch64": 15, + "i686": 14, + "armv7": 7 + }, + "reason": "Rejection sampling of the matrix A from the XOF output keyed by the public rho (FIPS 203 Algorithm 7). Branches test a sampled 12-bit value against q, the output count against 256, and the buffer position against the SHAKE128 rate; all of it is public data." + }, + { + "function": "pykyber::kyber::indcpa::indcpa_dec", + "group": "indcpa", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Loop-exit test on the output buffer position (cmp against 0x800/0xa00/2560) and a vector-loop prologue test; counters only." + }, + { + "function": "pykyber::kyber::indcpa::indcpa_dec_512", + "group": "indcpa", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Loop-exit test on the output buffer position (cmp against 0x800/0xa00/2560) and a vector-loop prologue test; counters only." + }, + { + "function": "pykyber::kyber::indcpa::indcpa_enc", + "group": "indcpa", + "max": { + "x86_64": 2, + "aarch64": 2, + "i686": 2, + "armv7": 1 + }, + "reason": "Loop-exit test on the output buffer position (cmp against 0x800/0xa00/2560) and a vector-loop prologue test; counters only." + }, + { + "function": "pykyber::kyber::indcpa::indcpa_enc_1024", + "group": "indcpa", + "max": { + "x86_64": 2, + "aarch64": 2, + "i686": 2, + "armv7": 1 + }, + "reason": "Loop-exit test on the output buffer position (cmp against 0x800/0xa00/2560) and a vector-loop prologue test; counters only." + }, + { + "function": "pykyber::kyber::indcpa::indcpa_enc_512", + "group": "indcpa", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Loop-exit test on the output buffer position (cmp against 0x800/0xa00/2560) and a vector-loop prologue test; counters only." + }, + { + "function": "pykyber::kyber::kem::crypto_kem_dec", + "group": "decaps-entry", + "max": { + "x86_64": 3, + "aarch64": 3, + "i686": 3, + "armv7": 3 + }, + "reason": "Length checks on dk, c and the shared secret buffer (public sizes, validated before any secret-dependent work, as AGENTS.md requires), tests of the Result tag, the dk hash check H(ek) == h (both sides are derived from the public ek, so it may branch or call bcmp), and remainder loops of zeroize, copy and the hash check on public lengths. In every listing the compared operand is an immediate, a length or a loop counter; none is derived from m', K, c' or the reject flag." + }, + { + "function": "pykyber::kyber::kem::crypto_kem_dec_1024", + "group": "decaps-entry", + "max": { + "x86_64": 3, + "aarch64": 3, + "i686": 3, + "armv7": 3 + }, + "reason": "Length checks on dk, c and the shared secret buffer (public sizes, validated before any secret-dependent work, as AGENTS.md requires), tests of the Result tag, the dk hash check H(ek) == h (both sides are derived from the public ek, so it may branch or call bcmp), and remainder loops of zeroize, copy and the hash check on public lengths. In every listing the compared operand is an immediate, a length or a loop counter; none is derived from m', K, c' or the reject flag." + }, + { + "function": "pykyber::kyber::kem::crypto_kem_dec_512", + "group": "decaps-entry", + "max": { + "x86_64": 3, + "aarch64": 3, + "i686": 3, + "armv7": 3 + }, + "reason": "Length checks on dk, c and the shared secret buffer (public sizes, validated before any secret-dependent work, as AGENTS.md requires), tests of the Result tag, the dk hash check H(ek) == h (both sides are derived from the public ek, so it may branch or call bcmp), and remainder loops of zeroize, copy and the hash check on public lengths. In every listing the compared operand is an immediate, a length or a loop counter; none is derived from m', K, c' or the reject flag." + }, + { + "function": "pykyber::kyber::ntt::ntt_inverse", + "group": "ntt", + "max": { + "x86_64": 18, + "aarch64": 4, + "i686": 4 + }, + "reason": "Loop-structure tests (cmp against a counter or the stride, vector-loop prologue) in the layer loops of the inverse NTT; no coefficient is compared." + }, + { + "function": "pykyber::kyber::poly::poly_compress", + "group": "compress", + "max": { + "x86_64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::poly::poly_compress_1024", + "group": "compress", + "max": { + "x86_64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::poly::poly_decompress_1024", + "group": "compress", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::poly::poly_tomsg", + "group": "compress", + "max": { + "x86_64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_compress", + "group": "compress", + "max": { + "aarch64": 3, + "i686": 3 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_compress_1024", + "group": "compress", + "max": { + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_compress_512", + "group": "compress", + "max": { + "aarch64": 2, + "i686": 2 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_decompress", + "group": "compress", + "max": { + "x86_64": 3, + "aarch64": 3, + "i686": 3 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_decompress_1024", + "group": "compress", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_decompress_512", + "group": "compress", + "max": { + "x86_64": 2, + "aarch64": 2, + "i686": 2 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_frombytes", + "group": "compress", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_frombytes_1024", + "group": "compress", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::polyvec::polyvec_frombytes_512", + "group": "compress", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1 + }, + "reason": "Vector-loop prologue and epilogue tests (cmp against 3, 7, 8 or 15 left-over elements) that choose between the unrolled and the scalar loop; counts of coefficients, never coefficient values." + }, + { + "function": "pykyber::kyber::symmetric::xof_squeeze_blocks", + "group": "keccak", + "max": { + "aarch64": 2 + }, + "reason": "Sponge buffering: tests of the input and output lengths and of the byte position inside the rate (0x88 for SHAKE256, 0x68/0x48 for the SHA3 rates). Lengths are public in every call here (fixed per parameter set, or the ciphertext length). The permutation itself has no data-dependent branch. Unrolled byte-copy tails give the large counts; on i686 the length is spilled to the stack, so the compare reads memory." + }, + { + "function": "pykyber::mlkem::ct::reject_on_mismatch", + "group": "reject", + "max": { + "x86_64": 5, + "aarch64": 6, + "i686": 5, + "armv7": 1 + }, + "reason": "Tests of the slice length (zero length, 4/8/32-byte vector-loop thresholds) and of the remainder after the vector loop (cmp len, done). The accumulated difference `diff` is only used by the final cset/neg/bif select: it never reaches a flag-setting instruction that feeds a branch. Read on aarch64 in full; the x86_64, i686 and armv7 listings show the same compare forms (counter or length against counter, length against immediate)." + }, + { + "function": "pykyber::mlkem::kem::decaps", + "group": "decaps-entry", + "max": { + "x86_64": 26, + "aarch64": 14, + "i686": 14, + "armv7": 14 + }, + "reason": "Length checks on dk, c and the shared secret buffer (public sizes, validated before any secret-dependent work, as AGENTS.md requires), tests of the Result tag, the dk hash check H(ek) == h (both sides are derived from the public ek, so it may branch or call bcmp), and remainder loops of zeroize, copy and the hash check on public lengths. In every listing the compared operand is an immediate, a length or a loop counter; none is derived from m', K, c' or the reject flag." + }, + { + "function": "pykyber::mlkem::kpke::decrypt", + "group": "k-pke", + "max": { + "x86_64": 6, + "aarch64": 8, + "i686": 6, + "armv7": 12 + }, + "reason": "Length checks at entry, zero-length and loop-exit tests (cmp against 0x200 and similar: the coefficient count of a polynomial or the byte length of a vector), Result tags, and the rejection-sampling guard in matrix_row. The compared operands are immediates, lengths or counters in every listing." + }, + { + "function": "pykyber::mlkem::kpke::encrypt", + "group": "k-pke", + "max": { + "x86_64": 1, + "aarch64": 2, + "i686": 1, + "armv7": 1 + }, + "reason": "Length checks at entry, zero-length and loop-exit tests (cmp against 0x200 and similar: the coefficient count of a polynomial or the byte length of a vector), Result tags, and the rejection-sampling guard in matrix_row. The compared operands are immediates, lengths or counters in every listing." + }, + { + "function": "pykyber::mlkem::kpke::matrix_row", + "group": "k-pke", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1, + "armv7": 1 + }, + "reason": "Length checks at entry, zero-length and loop-exit tests (cmp against 0x200 and similar: the coefficient count of a polynomial or the byte length of a vector), Result tags, and the rejection-sampling guard in matrix_row. The compared operands are immediates, lengths or counters in every listing." + }, + { + "function": "pykyber::mlkem::kpke::sample_noise", + "group": "k-pke", + "max": { + "x86_64": 1, + "aarch64": 1, + "i686": 1, + "armv7": 1 + }, + "reason": "Length checks at entry, zero-length and loop-exit tests (cmp against 0x200 and similar: the coefficient count of a polynomial or the byte length of a vector), Result tags, and the rejection-sampling guard in matrix_row. The compared operands are immediates, lengths or counters in every listing." + }, + { + "function": "pykyber::mlkem::sample::sample_ntt", + "group": "sampling", + "max": { + "x86_64": 6, + "aarch64": 6, + "i686": 6, + "armv7": 2 + }, + "reason": "Rejection sampling of the matrix A from the XOF output keyed by the public rho (FIPS 203 Algorithm 7). Branches test a sampled 12-bit value against q, the output count against 256, and the buffer position against the SHAKE128 rate; all of it is public data." + } + ], + "memcmp_calls": { + "pykyber::mlkem::kem::decaps": { + "max": { + "armv7": 1 + }, + "reason": "check_dk (FIPS 203 section 7.3) compares sha3_256(ek) with the h stored in dk. Both are derived from the public encapsulation key, so a variable-time compare reveals nothing about the secret part of dk. On armv7 the 32-byte compare of this one site is a bcmp call; the ciphertext comparison does not use it (it goes through reject_on_mismatch)." + } + } +} diff --git a/scripts/timing_decaps.py b/scripts/timing_decaps.py new file mode 100755 index 0000000..158b489 --- /dev/null +++ b/scripts/timing_decaps.py @@ -0,0 +1,249 @@ +#!/usr/bin/env python3 +"""Statistical timing test of decapsulation (PLAN-2.0 WP-8, issue #16). + +A dudect-style measurement tool, not a correctness test: it is not part of +`cargo test`, `pytest` or scripts/check.sh because it is slow and its result +depends on the machine. Run it by hand before a release (see SECURITY.md). + +What it measures. For one fixed key pair it times decapsulation of three +classes of ciphertext through the private extension functions (no Python +wrapper, so the only Python in the timed window is the call itself): + + valid ciphertexts from encapsulate(); + flip a valid ciphertext with one bit flipped (a near miss, the case in + which an early-exit comparison would leak the position); + random uniformly random bytes of the right length (rejected as well). + +Implicit rejection means all three return a 32-byte secret and none raises, so +a difference in running time between "valid" and the others is exactly the +leak of the comparison/select code that this test looks for. + +Method. The classes are interleaved in a random order (fixed seed) to spread +drift and cache effects over all classes. Each class draws from a pool of +distinct ciphertexts. For each pair (valid vs flip, valid vs random) Welch's +t statistic is computed on the raw timings and after cropping the slowest +values at several percentiles of the pooled data (dudect's way of dealing +with the heavy right tail of timing noise). A pair is flagged only if |t| +exceeds the threshold on the whole run and on both halves of the run (by time) +with the same sign. + +Reproducibility. The key pair (from a fixed 64-byte seed), the bit positions, +the random ciphertexts, the pool indices and the measurement order are all +derived from --seed. The valid ciphertexts are not reproducible byte for byte: +encapsulate() draws its message from the OS generator and the extension offers +no seeded variant. They are identically distributed, so this does not matter +for the statistic, but two runs are not bit-identical. + +Power. The tool prints the smallest mean difference it could have detected at +the threshold (from the observed variances). A run on a noisy machine has a +large detectable difference and then proves little. `--inject-ns N` is a +positive control: it adds N ns of busy-waiting to the non-valid classes +inside the timed window; run it to see what the test does detect on this +machine. Nothing here proves constant time; it fails to find a leak of some +size, with one compiler, one CPU and one run. + +Build the extension in release mode first: `maturin develop --release`. +""" + +import argparse +import gc +import math +import os +import platform +import random +import sys +import time + +DEFAULT_COUNT = 100000 # measurements per class +DEFAULT_SEED = 20261030 # the date of the WP-8 work, as an integer +DEFAULT_THRESHOLD = 5.0 # |t|; one test p ~ 6e-7, ten tests per set < 1e-5 +POOL_SIZE = 256 # distinct ciphertexts per class +WARMUP = 2000 # untimed calls before measuring +CROP_PERCENTILES = (100, 99, 95, 90, 80) # 100 = no cropping +SEED_BYTES = 64 # d || z for ML-KEM, the same layout for legacy keygen +SETS = ("512", "768", "1024") +FAMILIES = ("mlkem", "kyber") + + +def load_functions(family, level): + from pykyber import _pykyber as ext + if family == "mlkem": + keygen = getattr(ext, "_mlkem_%s_keygen" % level) + encaps = getattr(ext, "_mlkem_%s_encaps" % level) + decaps = getattr(ext, "_mlkem_%s_decaps" % level) + # ML-KEM: keygen -> (ek, dk), encaps -> (c, K), decaps(c, dk) + return keygen, encaps, decaps + keygen = getattr(ext, "_keypair_%s" % level) + encaps = getattr(ext, "_encapsulate_%s" % level) + decaps = getattr(ext, "_decapsulate_%s" % level) + # legacy: keygen -> (pk, sk), encaps -> (ct, ss), decaps(ct, sk) + return keygen, encaps, decaps + + +def welch_t(a, b): + na, nb = len(a), len(b) + if na < 2 or nb < 2: + return 0.0, 0.0 + ma, mb = sum(a) / na, sum(b) / nb + va = sum((x - ma) ** 2 for x in a) / (na - 1) + vb = sum((x - mb) ** 2 for x in b) / (nb - 1) + se = math.sqrt(va / na + vb / nb) + if se == 0: + return 0.0, 0.0 + return (ma - mb) / se, se + + +def percentile(sorted_values, pct): + idx = min(len(sorted_values) - 1, int(len(sorted_values) * pct / 100.0)) + return sorted_values[idx] + + +def analyse(label, valid, other, threshold): + """valid, other: lists of (index_in_run, ns). Returns (flagged, lines).""" + pooled = sorted(v for _, v in valid) + sorted(v for _, v in other) + pooled.sort() + lines = [] + flagged = False + for pct in CROP_PERCENTILES: + cut = percentile(pooled, pct) if pct < 100 else float("inf") + a = [v for _, v in valid if v <= cut] + b = [v for _, v in other if v <= cut] + t_all, se = welch_t(a, b) + # halves by time: the run index is shared by all classes + mid = max(i for i, _ in valid + other) // 2 + t_first, _ = welch_t([v for i, v in valid if i <= mid and v <= cut], [v for i, v in other if i <= mid and v <= cut]) + t_second, _ = welch_t([v for i, v in valid if i > mid and v <= cut], [v for i, v in other if i > mid and v <= cut]) + mean_diff = (sum(a) / len(a) - sum(b) / len(b)) if a and b else 0.0 + mde = threshold * se + hit = (abs(t_all) > threshold and abs(t_first) > threshold and abs(t_second) > threshold + and (t_all > 0) == (t_first > 0) == (t_second > 0)) + flagged = flagged or hit + lines.append(" %-14s crop<=p%-3d t=%+8.2f (halves %+7.2f %+7.2f) mean(valid)-mean(other)=%+9.1f ns " + "detectable at |t|=%.1f: %7.1f ns n=%d/%d %s" + % (label, pct, t_all, t_first, t_second, mean_diff, threshold, mde, len(a), len(b), + "LEAK?" if hit else "ok")) + return flagged, lines + + +def run_one(family, level, count, seed, threshold, inject_ns): + keygen, encaps, decaps = load_functions(family, level) + rng = random.Random("%s-%s-%d" % (family, level, seed)) + key_seed = bytes(rng.getrandbits(8) for _ in range(SEED_BYTES)) + pub, sec = keygen(key_seed) + pub, sec = bytes(pub), bytes(sec) + + valid_pool = [bytes(encaps(pub)[0]) for _ in range(POOL_SIZE)] + ct_len = len(valid_pool[0]) + flip_pool = [] + for i in range(POOL_SIZE): + ct = bytearray(valid_pool[rng.randrange(POOL_SIZE)]) + pos = rng.randrange(ct_len * 8) + ct[pos // 8] ^= 1 << (pos % 8) + flip_pool.append(bytes(ct)) + random_pool = [bytes(rng.getrandbits(8) for _ in range(ct_len)) for _ in range(POOL_SIZE)] + pools = {"valid": valid_pool, "flip": flip_pool, "random": random_pool} + + # sanity: decapsulation of a valid ciphertext must agree with encaps + c0, k0 = encaps(pub) + assert bytes(decaps(bytes(c0), sec)) == bytes(k0), "decapsulation does not round-trip" + assert bytes(decaps(flip_pool[0], sec)) != bytes(k0) + + schedule = ["valid", "flip", "random"] * count + rng.shuffle(schedule) + picks = [rng.randrange(POOL_SIZE) for _ in schedule] + + clock = time.perf_counter_ns + for k in range(WARMUP): + decaps(valid_pool[k % POOL_SIZE], sec) + + results = {"valid": [], "flip": [], "random": []} + gc.collect() + gc.disable() + try: + for i, (cls, pick) in enumerate(zip(schedule, picks)): + ct = pools[cls][pick] + t0 = clock() + decaps(ct, sec) + if inject_ns and cls != "valid": + end = t0 + inject_ns + while clock() < end: + pass + t1 = clock() + results[cls].append((i, t1 - t0)) + finally: + gc.enable() + + print("\n== %s %s: %d measurements per class, ciphertext %d bytes, threshold |t| > %.1f%s" + % (family, level, count, ct_len, threshold, + ", INJECTED %d ns into the non-valid classes" % inject_ns if inject_ns else "")) + med = sorted(v for _, v in results["valid"]) + print(" valid-class timing: median %d ns, p5 %d ns, p95 %d ns" % (percentile(med, 50), percentile(med, 5), percentile(med, 95))) + flagged = False + for other in ("flip", "random"): + hit, lines = analyse("valid/" + other, results["valid"], results[other], threshold) + flagged = flagged or hit + print("\n".join(lines)) + return flagged + + +def describe_machine(): + model = "?" + try: + with open("/proc/cpuinfo") as f: + for line in f: + if line.startswith("model name"): + model = line.split(":", 1)[1].strip() + break + except OSError: + pass + governor = "?" + try: + with open("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor") as f: + governor = f.read().strip() + except OSError: + pass + import pykyber + print("python : %s %s" % (platform.python_implementation(), platform.python_version())) + print("machine : %s, %s, governor %s" % (platform.machine(), model, governor)) + print("pykyber : %s from %s" % (getattr(pykyber, "__version__", "?"), os.path.dirname(pykyber.__file__))) + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + ap.add_argument("--family", default=",".join(FAMILIES), help="comma list of: " + ",".join(FAMILIES)) + ap.add_argument("--sets", default="768", help="comma list of: " + ",".join(SETS)) + ap.add_argument("-n", "--count", type=int, default=DEFAULT_COUNT, help="measurements per class (default %(default)d)") + ap.add_argument("--seed", type=int, default=DEFAULT_SEED) + ap.add_argument("--threshold", type=float, default=DEFAULT_THRESHOLD) + ap.add_argument("--cpu", type=int, default=None, help="pin the process to this CPU (Linux)") + ap.add_argument("--inject-ns", type=int, default=0, help="positive control: add this many ns to non-valid classes") + args = ap.parse_args() + + if args.cpu is not None and hasattr(os, "sched_setaffinity"): + os.sched_setaffinity(0, {args.cpu}) + print("== environment") + describe_machine() + print("count : %d per class, seed %d, threshold |t| > %.1f, pool %d, warmup %d" + % (args.count, args.seed, args.threshold, POOL_SIZE, WARMUP)) + + any_flag = False + for family in args.family.split(","): + for level in args.sets.split(","): + any_flag = run_one(family.strip(), level.strip(), args.count, args.seed, args.threshold, args.inject_ns) or any_flag + + print("\n== result") + if args.inject_ns: + if any_flag: + print("positive control: the injected %d ns difference WAS flagged" % args.inject_ns) + return 0 + print("positive control: the injected %d ns difference was NOT flagged; the test is too weak at this count/noise" % args.inject_ns) + return 1 + if any_flag: + print("LEAK? flagged above: repeat on a quiet machine (--cpu N) before drawing conclusions") + return 1 + print("no timing difference found at the stated power (not a proof of constant time)") + return 0 + + +if __name__ == "__main__": + sys.exit(main())