v2.1.0.23#529
Merged
Merged
Conversation
…ies joins by worker id
Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>
…nd membership set
* refactor(quil-node): extract sibling modules from main.rs (Batches 1-2 + partial 3)
Mechanical refactor of crates/quil-node/src/main.rs per
.claude/plans/deep-bouncing-cerf.md. No behavior changes.
Batch 1 — Foundations:
- util/multiaddr.rs: pure host:port multiaddr helpers
- blossomsub_consensus_publisher.rs: ConsensusPublisher impl
- dht_node.rs: run_dht_node entry point
- worker_node.rs: run_worker_node entry point (~370 lines moved verbatim)
Batch 2 — Slim main():
- diagnostic.rs: --peer-id / --node-info / --peer-info / --metrics
handlers + --import-db block. main() shrinks ~155 lines.
(Step 5 — bulk move of run_master_node to master_node/mod.rs —
deferred to the final batch; phases are extracted from main.rs
in place, and the residual ~50-line skeleton will be moved last.)
Batch 3 partial — Subsystem init extraction:
- master_node/storage.rs: Phase 1 (db open + stores), returns
StorageHandles. ~84 lines extracted.
- master_node/keys.rs: Phase 2 (key management), returns
KeyHandles. ~42 lines extracted.
main.rs: 6,208 → 4,737 lines (~24% reduction so far). All extracted
modules compile and pass cargo nextest. simtest/ci.sh -short passes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): extract engines, frame_pipeline, networking phases
Continues the Batch 3 subsystem-init extraction from main.rs per
.claude/plans/deep-bouncing-cerf.md. No behavior changes.
- master_node/engines.rs: Phase 3 (execution engines + CRDT priming
+ genesis bootstrap) — init_engines() + bootstrap_genesis(), returns
EngineHandles. ~215 lines extracted.
- master_node/frame_pipeline.rs: Phase 4 (frame prover + validator +
fee manager) — init(), returns FramePipeline. ~17 lines extracted.
- master_node/networking.rs: Phase 5 (P2P node + subscriptions +
blacklist) — async init(), returns P2pHandles. ~90 lines extracted.
main.rs: 4,737 → 5,069 lines (~10% additional reduction).
quil-node compiles and 22 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): extract peer_info_publisher phase
- master_node/peer_info_publisher.rs: Phase 5b (PeerInfo broadcaster
+ KeyRegistry publisher with Ed448 signing) — spawn(args), takes a
PeerInfoPublisherArgs struct. ~237 lines extracted. The OnceLock
`pi_worker_manager` stays in run_master_node's outer scope and is
passed in as worker_manager_cell.
main.rs: 5,069 → 4,862 lines.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): complete Batch 3 (subsystem init extraction)
Extracts the remaining run_master_node init phases per
.claude/plans/deep-bouncing-cerf.md. No behavior changes.
- master_node/runtime_state.rs: Phases 5c+5d (message collector +
prover registry + coverage monitor + halt state + halt broadcaster +
prover-only ticker). Returns RuntimeState. ~220 lines extracted.
- master_node/worker_manager.rs: Phase 5e (worker manager — cluster
vs local thread mode, drain task, persisted-state restore,
pre-allocation, config filters, pi_worker_manager OnceLock publish).
~600 lines extracted. drain task stays inline per pragmatic scoping
(no separate worker_drain.rs).
- master_node/allocator_and_lifecycle.rs: Phases 5g/h/i (worker
allocator + seniority compat + early reconcile spawn + consensus
OnceLocks + prover lifecycle + frame materializer). ~226 lines
extracted.
main.rs: 4,862 → 3,874 lines. cumulative reduction from original
6,208 is 2,334 lines (37.6%).
Verification: cargo check --workspace, cargo nextest run --workspace
(2,326 passed / 13 skipped), simtest/ci.sh -short — all green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): extract message_loop (Batch 4, Step 17)
- master_node/message_loop.rs: Phase 6 message-receive loop —
spawn(args), takes a MessageLoopArgs struct with 31 fields covering
the captured stores, OnceLocks, broadcast/loopback channels, peer
caches, signers, etc. ~764 lines extracted (the entire
tokio::spawn(async move { loop { tokio::select! { ... } } }) block
including the top-level GLOBAL_* dispatch match). The message_router
construction also moves inside spawn (it was only used there).
main.rs: 3,874 → 3,075 lines (~800 lines extracted in one step).
Total reduction from original 6,208: 3,133 lines (50.5%).
Step 18 (further per-bitmask handler split into message_handlers.rs)
deferred — the deep nesting now lives inside master_node/message_loop.rs
which is independently scannable; main.rs's run_master_node body is
already the planned ~15 sequential init/spawn calls.
Verification: cargo check --workspace, cargo nextest run --workspace
(2,326 passed / 13 skipped), simtest/ci.sh -short — all green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): extract archive_sync (Batch 5, Step 19)
- master_node/archive_sync.rs: Phase 6b (archive frame poller +
periodic prover-tree sync + consensus activation hooks + shard info
refresh) — spawn_all(args), takes ArchiveSyncArgs struct with 31
fields. The full ~985-line if-let-Some(seed) block moves into this
one function. Consensus activation hooks (finalized/incorporated/
qc_observed) inline.
main.rs: the call site now invokes master_node::archive_sync::spawn_all
before the original (now dead) block. The original is wrapped in
`if false { if let Some(seed) = mtls_seed { ... } }` for syntactic
safety pending cleanup in Step 21 (Batch 6 final tidy). Dead code
~785 lines.
Verification: cargo check --workspace, cargo nextest run --workspace
(2,326 passed / 13 skipped), simtest/ci.sh -short — all green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): extract grpc (Batch 5 complete, Step 20)
- master_node/grpc.rs: Phase 7 gRPC service — spawn_all(args), takes
GrpcArgs struct with 27 fields. Contains the NodeService plaintext
listener, the peer mTLS multi-service listener (Global, HyperSync,
AppShard, KeyRegistry, Connectivity, Dispatch, Mixnet, optional
PubSubProxy), the ClockStoreFrameLookup bridge, the
WorkerControlBridge, the LocalShardInfoProvider with archive-pool
fallback, and the workers_view ticker. ~1,135 lines extracted.
main.rs: the call site now invokes master_node::grpc::spawn_all
before the original (now dead) block. The original is wrapped in
`if false { ... }` for syntactic safety pending cleanup in Step 21
(Batch 6 final tidy). Dead code ~1,135 lines.
Batch 5 verification: cargo check --workspace, cargo nextest run
--workspace (2,326 passed / 13 skipped), simtest/ci.sh -short — all
green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(quil-node): Batch 6 final tidy — delete dead code wrappers
- Deleted the 'if false { ... }' dead-code wrapper around the original
archive_sync block (~787 lines removed from main.rs).
- Deleted the 'if false { ... }' dead-code wrapper around the original
gRPC service block (~1,135 lines removed from main.rs).
- Removed now-unused KeyManager trait import from main.rs (all
KeyManager method calls now happen inside extracted modules).
- Suppressed dead_code warning on StorageHandles.db_path (kept on the
struct for forward compat; only init() reads it for logging today).
- Cleaned unused KeyManager import in grpc.rs.
- Cleaned redundant field-pattern shorthand in message_loop.rs
(`msg_rx: mut msg_rx` → `mut msg_rx`).
main.rs: 2,942 → 1,019 lines.
Cumulative reduction from original 6,208: 5,189 lines (83.6%).
Final structure: 12 master_node/* modules (storage, keys, engines,
frame_pipeline, networking, runtime_state, peer_info_publisher,
worker_manager, allocator_and_lifecycle, message_loop, archive_sync,
grpc) plus 4 trivial sibling entry points (worker_node, dht_node,
diagnostic, blossomsub_consensus_publisher) plus util/multiaddr.
run_master_node body now reads as ~15 sequential init / spawn calls
into master_node::*.
Verification: cargo check --workspace, cargo nextest run --workspace
(2,326 passed / 13 skipped), simtest/ci.sh -short — all green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* apply changes from 68bd613
* apply changes from 27cd8c5
* move archive_multiaddr_to_host_port tests to multiaddr.rs
* factor out run_master_noder to submodule
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
fixes #535 Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>
…eek-verification on peer info for faster roundtrip ignores
…posal logic and worker allocator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.