Skip to content

About

BFT consensus engine throughput upgrade.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

malachite-usdc

Measured: consensus finality that stays flat as validators are added, and 1.7-2x more transactions per second - by referencing transactions instead of re-sending them during consensus. Validated against circlefin/malachite (the consensus engine behind Circle's Arc blockchain) with real USDC traffic from Ethereum mainnet. Every number below is reproducible from this repository.

The benchmarks

Finality vs validator count (1 MiB blocks, emulated 5-region global network, real malachite binaries, medians with repeats):

What consensus transmits per block 4 validators 10 16 20 30*
Full transactions (1 MiB) - status quo 292 ms 400 461 526 768
32-byte references (168 KB) 267 ms 325 326 331 394
One ordering commitment (136 B) 171 ms 253 272 284 336

Cost of each added validator: 11.8 ms (full transactions) vs 0.8 ms (references). That difference is what makes growing a validator set expensive today - and nearly free with references. (*30-validator cells were CPU-limited on the test host; shown for shape.)

Throughput (20 validators, unthrottled, same logical transactions in every arm):

Block time Transactions/s Finality
Full transactions, 1 MiB blocks 572 ms ~8,800 480 ms
References, same content 332 ms ~14,800 306 ms
Full transactions, 4 MiB blocks 1,187 ms ~16,800 1,035 ms
References, same content 585 ms ~33,600 507 ms

Note the second pattern: with references, quadrupling block content barely moves finality (306 -> 507 ms) while the status quo doubles it (480 -> 1,035 ms). Block size stops being a consensus problem.

Execution side, on real money traffic (251,050 actual USDC transfers pulled from Ethereum mainnet, replayed through a real EVM running the actual USDC contract bytecode):

  • Sequential execution: 170,855 tx/s - roughly 10x more than reference-based consensus can feed it, so the executor needs no changes at all.
  • Verifying a result by its content address instead of re-executing it: 44x cheaper (146k vs 6.46M verifications/s), with a 20,000/20,000 correctness gate.
  • Honest negatives, measured twice: parallel execution loses on real USDC traffic (hub accounts dominate; only ~35% of transactions are conflict-free), and only 11.7% of real transfers are exact repeats - so we recommend against those two optimizations. The data says keep execution simple.

The idea

Today, a proposer re-broadcasts every transaction inside the consensus round, even though validators already received them via the mempool. The bytes are paid for twice, and the second payment sits on the latency-critical path - growing with both block size and validator count. Circle's own engineering notes describe the same pressure: transaction data must arrive "by the decision deadline."

The fix: transactions spread continuously in the background as they arrive. Consensus then transmits only references - or ultimately one ordering commitment - and each validator rebuilds the block from what it already holds. Anything missing is fetched through malachite's existing sync path, so nodes that don't participate keep working unchanged.

References are content addresses: an identifier derived from the bytes themselves, so any node can verify that what it holds is what was proposed - no trust in the sender required. The addressing model follows the UOR Foundation specification (uor-foundation on crates.io): canonical byte serialization, BLAKE3 as the primary digest, and verification-on-receipt. One practical lesson from this project: our first witness implementation hashed data in nondeterministic order and failed its own correctness gate - the canonical-serialization discipline the UOR spec mandates is exactly what fixed it.

What is in this repository

Path Contents
implementation/ The malachite patch (+87/-5, default-off) that makes propagation cost measurable, and pointers to the live upstream engagement
benchmarks/consensus/ 5-region network emulation harness, cell definitions, and unmodified raw results for every table above
benchmarks/execution/ The real-EVM/real-trace benchmark (source + results) and the 251k-transfer USDC trace with provenance
docs/ The full lab notebook: validation report (including every dead end), target architecture, and the pre-registered test plan for a cloud-scale run

Upstream: filed as circlefin/malachite#1576 with branches feat/test-app-payload-streaming and bench/value-propagation-harness, following the repo's issue-first contribution policy.

Reproduce (30 minutes on any Linux box)

# 1. build malachite with the measurement patch
git clone -b feat/test-app-payload-streaming https://github.com/humuhumu33/malachite
cd malachite/code && cargo build --release -p arc-malachitebft-test-app

# 2. run the consensus benchmark (root needed for network emulation)
cd ../../malachite-usdc/benchmarks/consensus
export MALACHITE_BIN=<path-to>/malachitebft-test-app
sudo -E node geo.mjs cells/sweep1.json results.json

# 3. run the execution benchmark (no root, no network)
cd ../execution/uorbench2
gunzip -k trace/usdc-trace-250k.jsonl.gz
cargo run --release -- trace/usdc-trace-250k.jsonl

Honest limitations

Consensus numbers come from a single-host network emulation (real latency matrix and bandwidth caps, but shared CPU and no real-internet weather); ratios were stable across repeats, absolute numbers need a multi-region cloud confirmation - the pre-registered plan for that run is in docs/TEST-PLAN.md. Reference-based arms assume background transaction dissemination has already happened; building and stress-testing that dissemination protocol (cache misses, cold nodes, partitions) is the main remaining engineering work, and the test plan covers it.

References

License: Apache-2.0 (matching upstream malachite).

About

BFT consensus engine throughput upgrade.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages