Skip to content

Dependency cleanup - #638

Open
illuzen wants to merge 15 commits into
mainfrom
illuzen/dep-cleanup
Open

Dependency cleanup#638
illuzen wants to merge 15 commits into
mainfrom
illuzen/dep-cleanup

Conversation

@illuzen

@illuzen illuzen commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Clears all cargo audit vulnerabilities on this tree (22 → 0) and cuts warnings (25 → 11) without a full Substrate client bump.

  • Easy vulns: bump rustls-webpki to 0.103.13; drop unused substrate-test-utils from pallet-scheduler (removes ancient tokio 0.2).
  • Wasmtime (main vuln mass): vendor sc-executor-wasmtime / sp-wasm-interface unchanged, then patch wasmtime 35.0.0 → 36.0.13 via [patch.crates-io] (clears the Apr 2026 RustSec set, including sandbox-escape advisories). Split into two commits so the patch is auditable as a git diff.
  • Yanked / easy warnings: bump cid, keccak, spin, event-listener, rand, memmap2 (0.9 line), cxx.
  • litep2p: migrate multiaddr / multihash (like upstream litep2p 0.14) so yanked core2 is gone; Protocol::P2p now carries PeerId.
  • PQ Noise: bump clatter 1.1 → 2.2 with rust-crypto ML-KEM only (default-features = false); drop all pqcrypto-* / pqclean Kyber from the graph. Simplify ClatterSession now that PqHandshake owns its RNG.

Remaining cargo audit warnings (11)

Kind Crate Notes
unsound anyhow@1.0.100 Exact pin in Quantus qp-* crates; needs their release / path patch to ≥1.0.103
unsound lru@0.12.5 Via smoldot-lightsubxt-lightclient (SDK light-client path, not block production)
unsound memmap2@0.5.10 Via parity-db 0.4.13sc-client-db (still crates.io; Substrate still pins parity-db = "0.4.12", so 0.5.x with fixed memmap2 0.9 is not a drop-in)
unmaintained derivative, fxhash, instant, libsecp256k1, parity-wasm, paste, proc-macro-error, proc-macro-error2 Mostly FRAME / Substrate / transitive; leave for upstream

sc-client-db is not inlined; clearing memmap2 needs either upstream Substrate to take parity-db ≥0.5.1, or vendoring/patching sc-client-db / parity-db ourselves.

Commits

  1. 95ac1d18 — easy cargo-audit findings (webpki + tokio 0.2)
  2. 5c95b6ef — vendor executor wasmtime crates unchanged
  3. 45061511 — bump vendored wasmtime to 36.0.13
  4. 16421463 — easy yanked/unsound warning bumps
  5. 6619d055 — litep2p multihash/multiaddr migration (drop core2)
  6. 21221090 — clatter 2.2 + drop pqcrypto-kyber

Test plan

  • cargo audit0 vulnerabilities (warnings may remain as above)
  • cargo check -p quantus-node
  • cargo test -p litep2p --lib (Noise / multihash changes)
  • Smoke: start a local node, peer connect, sync a few blocks
  • Confirm PQ Noise handshakes still succeed between two nodes (clatter 2.2 path)
  • Review git diff 5c95b6ef..45061511 -- client/executor-wasmtime client/wasm-interface for the wasmtime patch surface

Note

High Risk
Touches WASM JIT/sandbox execution (wasmtime major bump), P2P crypto handshakes (clatter/multihash), and a large lockfile—regressions could affect runtime correctness, peer connectivity, or security posture despite audit clearance.

Overview
Clears cargo audit vulnerabilities (22 → 0) by vendoring and patching the WASM executor stack, refreshing networking/crypto deps, and trimming obsolete test crates—without a full Substrate client upgrade.

Wasmtime / runtime execution: Adds in-tree sc-executor-wasmtime and sp-wasm-interface (patched via [patch.crates-io]) and bumps wasmtime 35 → 36.0.13 to address Apr 2026 RustSec issues (including sandbox-escape class advisories). sc-executor-wasmtime gains extra build/test deps (cargo_metadata, parity-scale-codec, wat, etc.) aligned with the newer wasmtime line.

litep2p / multihash: Moves to multiaddr 0.18 and multihash 0.19 with multihash-codetable, dropping yanked core2. Peer addressing uses Protocol::P2p(PeerId) instead of raw multihash bytes.

PQ Noise: clatter 1.1 → 2.2 with rust-crypto ML-KEM only (default-features = false); removes pqcrypto-kyber and related pqclean graph. ClatterSession no longer boxes an external RNG—PqHandshake owns RNG internally.

Other dependency hygiene: rustls-webpki bump, removal of substrate-test-utils (and old tokio 0.2), plus assorted yanked/warning fixes (cid, keccak, rand, event-listener, memmap2 0.9 line, cxx, etc.).

Reviewed by Cursor Bugbot for commit 2122109. Configure here.

illuzen and others added 8 commits August 3, 2026 15:57
Bump rustls-webpki to 0.103.13 and drop the unused substrate-test-utils
dev-dep (and its ancient tokio 0.2) from pallet-scheduler.

Co-authored-by: Cursor <cursoragent@cursor.com>
Vendored from crates.io (sc-executor-wasmtime 0.43.0, sp-wasm-interface
24.0.0) for subsequent wasmtime 35 -> 36.0.13 bump. This commit contains
the original sources to make the patch auditable via git diff.

Co-authored-by: Cursor <cursoragent@cursor.com>
Wire the vendored crates through [patch.crates-io], bump wasmtime
35.0.0 -> 36.0.13 (clearing Apr 2026 RustSec advisories), and drop
upstream unit tests that need unpublished sc-runtime-test.

Co-authored-by: Cursor <cursoragent@cursor.com>
Bump cid (drops core2 via that path), keccak, spin, event-listener, rand,
memmap2 0.9, and cxx. Remaining yanked core2 is pinned by litep2p's
multihash 0.17 and needs a multiaddr/multihash upgrade.

Co-authored-by: Cursor <cursoragent@cursor.com>
Upgrade multiaddr 0.17 -> 0.18.2 and multihash 0.17 -> 0.19.5 (plus
multihash-codetable), matching upstream litep2p 0.14's migration so
Protocol::P2p carries PeerId and identity hashes use Multihash::wrap.

Co-authored-by: Cursor <cursoragent@cursor.com>
Upgrade litep2p's Noise stack to clatter 2.2 with rust-crypto ML-KEM
only (no pqclean/kyber features), simplifying ClatterSession now that
PqHandshake owns its RNG.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Cursor <cursoragent@cursor.com>

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPT-5.6 review

Verdict: Request changes.

The dependency cleanup is directionally good: the current RustSec database reports 0 vulnerabilities and 11 allowed warnings, the resolved graph contains Wasmtime 36.0.13 and clatter 2.2.0 with the pqcrypto-kyber dependency removed, and all GitHub checks pass.

There is one blocking regression in the multiaddr migration. sc_network_types::Multiaddr now parses protocols supported by multiaddr 0.18, but conversion into the local Protocol enum falls through to a wildcard panic. I reproduced this with the valid address /ip4/127.0.0.1/udp/443/quic-v1/webtransport: parsing succeeds, while address.iter().count() panics at protocol.rs:132. Because network addresses can originate outside the process, this creates a reachable node panic. Please either represent the newly supported protocol variants or reject them at a fallible parsing/conversion boundary, and add a regression test covering this case.

One additional coverage concern: this PR removes the entire direct sc-executor-wasmtime test module during a major Wasmtime upgrade. The production adapter is otherwise unchanged and the workspace CI is green, but preserving/adapting the WAT-only executor tests (or equivalent focused coverage) would materially reduce risk around instantiation strategies, stack limits, and precompiled artifacts.

Validation performed: reviewed the full base-to-head diff; verified the dependency graph; ran cargo audit; checked git diff --check; and compiled/ran a focused regression test that demonstrates the multiaddr panic.

— Reviewed by GPT-5.6

LiteP2pProtocol::Utp => Protocol::Utp,
LiteP2pProtocol::Ws(str) => Protocol::Ws(str),
LiteP2pProtocol::Wss(str) => Protocol::Wss(str),
_ => panic!("unsupported multiaddr protocol variant"),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: multiaddr 0.18 accepts additional valid protocol variants (including webtransport), but this infallible conversion panics for them. Reproducer: parse /ip4/127.0.0.1/udp/443/quic-v1/webtransport, then call iter(); parsing succeeds and iteration panics here. Since peer/listen addresses may be externally supplied, this is a reachable node panic. Please model the supported variants or make validation/conversion fallible and reject unsupported variants before storing the address, with a regression test for this input. — GPT-5.6

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by Fable (Claude Fable 5).

Verdict: Approve — with one follow-up that should land before or immediately after merge (see issue 1)

This is a well-executed dependency sweep. The commit structure (vendor-unchanged, then patch) makes the risky part auditable, and I verified the key claims independently rather than taking the PR description's word for them.

What I verified

  • Vendoring is genuinely unchanged. I diffed client/executor-wasmtime and client/wasm-interface at the vendor commit (5c95b6ef) against the actual crates.io tarballs (sc-executor-wasmtime 0.43.0, sp-wasm-interface 24.0.0): all source files are byte-identical; the only deltas are a reformatted Cargo.toml, the omitted Cargo.lock, and the added .cargo-checksum.json. The subsequent patch commit touches exactly what the README documents (wasmtime 35.0.0 → 36.0.13 + runtime/std features, test removal), and later commits touch the vendored crates only for manifest key reordering (fmt).
  • Lockfile matches every claim: wasmtime 36.0.13, clatter 2.2.0, rustls-webpki 0.103.13, multiaddr 0.18.2 / multihash 0.19.5, single tokio 1.47.1; core2, substrate-test-utils, and all pqcrypto-* crates are gone. The remaining-warnings table is honest: memmap2 0.5.10 (via parity-db) and the old event-listener 2.5.3 line are still present, as documented.
  • PQ Noise handshake stays wire-compatible across the upgrade. The old code already used clatter's rust_crypto_ml_kem::MlKem768 (the pqcrypto-kyber crates were dead weight from clatter 1.1's default features — never on our code path). I diffed clatter 1.1.0 vs 2.2.0 sources: the RustCrypto ML-KEM implementation differs only in API surface (both resolve to ml-kem 0.2.1), the KEM name strings (MLKEM768) and the Noise protocol-name composition are identical, so mixed-version nodes handshake fine during a rolling deploy. Removing the unsafe self-referential RNG borrow in ClatterSession is a real soundness win on its own.
  • PeerId/multihash migration is careful: identity code handled via Multihash::wrap(0x00, …), Deserialize goes through the validating from_bytes, and litep2p's accepted multihash forms (sha2-256, identity ≤ 42 bytes) match multiaddr::PeerId's, so the panicking From<PeerId> for multiaddr::PeerId is unreachable for validated peer IDs. New roundtrip tests cover both hash paths.
  • TI test fix (fb49d791) is sound: 100 genesis dev accounts instead of 1000, derivations cached in a OnceLock, and the skip-set in ensure_ti_valid matches the genesis count exactly.
  • CI is fully green on the head commit, including both build & test matrices.

Wasmtime 35→36 executes the same deterministic WASM semantics (upstream did the identical bump in polkadot-sdk#11793), so mixed-version block execution is consensus-safe. The two-node handshake smoke test in the plan is still worth running before release.

Issue 1 — panic! on unhandled multiaddr protocol variants (should fix)

client/network-types/src/multiaddr/protocol.rs adds _ => panic!("unsupported multiaddr protocol variant") to From<LiteP2pProtocol> for Protocol. litep2p::types::multiaddr is now a straight re-export of the multiaddr 0.18 crate, which parses eight variants the local enum lacks: WebRTCDirect, WebTransport, Ip6zone, Ipcidr, Garlic64, Garlic32, Sni, P2pStardust. Under multiaddr 0.17 these failed at parse time and were rejected with error logging; now they parse successfully and crash the node the moment the address is iterated as an sc-network-types Multiaddr.

Reachable today from: operator-supplied listen/bootnode/reserved addresses (Multiaddr::from_str accepts them, then the iter() in Litep2pNetworkBackend panics instead of hitting the existing log-and-skip arms), reserved-peer addresses via unsafe RPC, and DHT FIND_NODE results converted for DhtEvent subscribers (none in-tree currently, which is why I'm not treating this as a remote DoS today — the remote hot paths stay in litep2p's own types). But it's a latent landmine: any future code that iterates a converted remote address becomes remotely crashable by a peer advertising /webrtc-direct.

Fix is small: either add the missing variants to the local Protocol enum, or make the conversion fallible / map unknown variants to a logged skip. A bad address should be rejected, not take down the node.

Issue 2 — panicking expect in the reverse P2p conversion (minor)

The same file converts Protocol::P2p(multihash)LiteP2pProtocol::P2p with .expect("valid peer id multihash"). Since the local Protocol::P2p still carries a raw Multihash, any programmatically constructed non-peer-ID multihash (e.g. sha2-512) panics here. All in-tree callers use valid peer IDs, so this is currently unreachable — but a TryFrom would express the actual contract. Fine as a follow-up alongside issue 1.

Notes

  • The vendored executor's unit tests were deleted (they need the unpublished sc-runtime-test). Documented in the README and the node exercises the crate through sc-executor, so acceptable — but it is a small coverage loss on the sandbox-critical path; worth restoring if the crates stay vendored long-term.
  • client/executor-wasmtime/.cargo-checksum.json is now stale relative to the patched files. Harmless for a path dependency, but deleting it would avoid implying the tree still matches the registry checksums.
  • Housekeeping: commit 45061511 carries a Co-authored-by: Cursor trailer — repo convention is no AI attribution on anything published; squash-merging without the trailer would keep history consistent.

illuzen and others added 2 commits August 3, 2026 20:49
Model WebTransport and other 0.18 Protocol variants, reject unknown
protocols at parse time, and add a webtransport parse/iter regression test.

Co-authored-by: Cursor <cursoragent@cursor.com>
Bring back focused sc-executor-wasmtime tests covering instantiation
strategies, stack limits, NaN canonicalization, memory growth, and
precompiled artifacts without depending on unpublished sc-runtime-test.

Co-authored-by: Cursor <cursoragent@cursor.com>

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPT issue sounds bad

illuzen and others added 2 commits August 3, 2026 20:59
Replace the panicking expect on non-peer-id multihashes with TryFrom,
add try_push/try_with, and keep legacy push/with non-panicking via a
lossy placeholder fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPT-5.6 re-review

Verdict: Request changes.

Reviewed the four-commit delta from fb49d791 to be4d8726. The original blocker is resolved: all multiaddr 0.18 variants are modeled, the WebTransport parse/iteration regression passes, and unsupported future variants are rejected by the fallible parse paths. The Wasmtime coverage concern is also resolved; the restored focused suite passes locally (33 passed, 1 intentionally ignored). sc-network-types passes all 10 committed tests.

The follow-up P2P conversion commit introduces a new blocking correctness issue. Legacy push, with, From<Protocol>, and FromIterator paths now replace an invalid /p2p multihash with a fixed, valid placeholder peer ID. This silently changes which peer the address names; logging does not give the caller a chance to reject or recover from the corrupted address. I reproduced the round-trip with a SHA-512 multihash: the result contains identity multihash code 0 with a 32-byte zero digest instead of the supplied code 19 multihash.

Please do not fabricate a peer identity. Make invalid states unrepresentable, propagate ProtocolConversionError through construction paths, or preserve an explicit fail-fast behavior for legacy infallible APIs while directing arbitrary input through try_push/try_with.

The PR head was unchanged when this review was submitted. Format and config checks are green; the full Linux, macOS, and Clippy/doc jobs are still running.

— Reviewed by GPT-5.6

Comment thread client/network-types/src/multiaddr/protocol.rs Outdated
illuzen and others added 3 commits August 3, 2026 21:53
Makes an invalid /p2p component unrepresentable (multiaddr 0.18 semantics), so
conversion to litep2p is infallible again. Drops the lossy placeholder peer id
along with the ProtocolConversionError / try_push / try_with error surface.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
sc-authority-discovery 0.55.0 (pulled in via polkadot-node-network-protocol
under runtime-benchmarks/try-runtime) re-validates the Protocol::P2p payload
with from_multihash; keep it source-compatible now that the payload is PeerId.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants