Measured: consensus finality that stays flat as validators are added, and 1.7-2x more transactions per second - by referencing transactions instead of re-sending them during consensus. Validated against circlefin/malachite (the consensus engine behind Circle's Arc blockchain) with real USDC traffic from Ethereum mainnet. Every number below is reproducible from this repository.
Finality vs validator count (1 MiB blocks, emulated 5-region global network, real malachite binaries, medians with repeats):
| What consensus transmits per block | 4 validators | 10 | 16 | 20 | 30* |
|---|---|---|---|---|---|
| Full transactions (1 MiB) - status quo | 292 ms | 400 | 461 | 526 | 768 |
| 32-byte references (168 KB) | 267 ms | 325 | 326 | 331 | 394 |
| One ordering commitment (136 B) | 171 ms | 253 | 272 | 284 | 336 |
Cost of each added validator: 11.8 ms (full transactions) vs 0.8 ms (references). That difference is what makes growing a validator set expensive today - and nearly free with references. (*30-validator cells were CPU-limited on the test host; shown for shape.)
Throughput (20 validators, unthrottled, same logical transactions in every arm):
| Block time | Transactions/s | Finality | |
|---|---|---|---|
| Full transactions, 1 MiB blocks | 572 ms | ~8,800 | 480 ms |
| References, same content | 332 ms | ~14,800 | 306 ms |
| Full transactions, 4 MiB blocks | 1,187 ms | ~16,800 | 1,035 ms |
| References, same content | 585 ms | ~33,600 | 507 ms |
Note the second pattern: with references, quadrupling block content barely moves finality (306 -> 507 ms) while the status quo doubles it (480 -> 1,035 ms). Block size stops being a consensus problem.
Execution side, on real money traffic (251,050 actual USDC transfers pulled from Ethereum mainnet, replayed through a real EVM running the actual USDC contract bytecode):
- Sequential execution: 170,855 tx/s - roughly 10x more than reference-based consensus can feed it, so the executor needs no changes at all.
- Verifying a result by its content address instead of re-executing it: 44x cheaper (146k vs 6.46M verifications/s), with a 20,000/20,000 correctness gate.
- Honest negatives, measured twice: parallel execution loses on real USDC traffic (hub accounts dominate; only ~35% of transactions are conflict-free), and only 11.7% of real transfers are exact repeats - so we recommend against those two optimizations. The data says keep execution simple.
Today, a proposer re-broadcasts every transaction inside the consensus round, even though validators already received them via the mempool. The bytes are paid for twice, and the second payment sits on the latency-critical path - growing with both block size and validator count. Circle's own engineering notes describe the same pressure: transaction data must arrive "by the decision deadline."
The fix: transactions spread continuously in the background as they arrive. Consensus then transmits only references - or ultimately one ordering commitment - and each validator rebuilds the block from what it already holds. Anything missing is fetched through malachite's existing sync path, so nodes that don't participate keep working unchanged.
References are content addresses: an identifier derived from the bytes themselves, so any
node can verify that what it holds is what was proposed - no trust in the sender required.
The addressing model follows the UOR Foundation specification
(uor-foundation on crates.io): canonical byte
serialization, BLAKE3 as the primary digest, and verification-on-receipt. One practical lesson
from this project: our first witness implementation hashed data in nondeterministic order and
failed its own correctness gate - the canonical-serialization discipline the UOR spec mandates
is exactly what fixed it.
| Path | Contents |
|---|---|
implementation/ |
The malachite patch (+87/-5, default-off) that makes propagation cost measurable, and pointers to the live upstream engagement |
benchmarks/consensus/ |
5-region network emulation harness, cell definitions, and unmodified raw results for every table above |
benchmarks/execution/ |
The real-EVM/real-trace benchmark (source + results) and the 251k-transfer USDC trace with provenance |
docs/ |
The full lab notebook: validation report (including every dead end), target architecture, and the pre-registered test plan for a cloud-scale run |
Upstream: filed as circlefin/malachite#1576
with branches feat/test-app-payload-streaming
and bench/value-propagation-harness,
following the repo's issue-first contribution policy.
# 1. build malachite with the measurement patch
git clone -b feat/test-app-payload-streaming https://github.com/humuhumu33/malachite
cd malachite/code && cargo build --release -p arc-malachitebft-test-app
# 2. run the consensus benchmark (root needed for network emulation)
cd ../../malachite-usdc/benchmarks/consensus
export MALACHITE_BIN=<path-to>/malachitebft-test-app
sudo -E node geo.mjs cells/sweep1.json results.json
# 3. run the execution benchmark (no root, no network)
cd ../execution/uorbench2
gunzip -k trace/usdc-trace-250k.jsonl.gz
cargo run --release -- trace/usdc-trace-250k.jsonlConsensus numbers come from a single-host network emulation (real latency matrix and bandwidth
caps, but shared CPU and no real-internet weather); ratios were stable across repeats, absolute
numbers need a multi-region cloud confirmation - the pre-registered plan for that run is in
docs/TEST-PLAN.md. Reference-based arms assume background transaction
dissemination has already happened; building and stress-testing that dissemination protocol
(cache misses, cold nodes, partitions) is the main remaining engineering work, and the test
plan covers it.
- UOR Foundation - the content-addressing specification
uor-foundation- typed Rust traits for the ontology- circlefin/malachite - the consensus engine (Apache-2.0)
- Malachite ADR-003 - value propagation modes ("Consensus by Reference")
- Arc: deterministic finality - Circle's published consensus architecture and numbers
License: Apache-2.0 (matching upstream malachite).