Add beacon chain update plugin and cranker executable#280
Add beacon chain update plugin and cranker executable#280brianjohnson5972 wants to merge 12 commits intomasterfrom
Conversation
…nsaction from running indefinitely and using wait_for before calling get to allow for timeout.
There was a problem hiding this comment.
Pull request overview
This PR adds new infrastructure for periodically fetching Ethereum beacon-chain metrics and pushing them on-chain, plus a lightweight runner executable to operate those updates without a full node.
Changes:
- Introduces
beacon_chain_update_pluginto fetch queue/APY data from beaconcha.in and submit contract updates via the outpost Ethereum client. - Adds
crankeras a standalone executable that initializes only the plugin subset required for beacon-chain updates. - Adds a cron expression parser to
cron_pluginand updates libfc Ethereum/JSON utilities (tx block identification + big-integer JSON parsing fixes).
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| programs/cranker/src/main.cpp | New minimal entrypoint that boots only the plugins required for beacon-chain updates. |
| programs/cranker/README.md | Documents cranker usage and CLI configuration. |
| programs/cranker/CMakeLists.txt | Adds build target for the cranker executable. |
| programs/CMakeLists.txt | Adds cranker subdirectory to the programs build. |
| plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp | Updates client parsing and makes getters const. |
| plugins/outpost_ethereum_client_plugin/include/sysio/outpost_ethereum_client_plugin.hpp | Makes client/ABI getters const. |
| plugins/cron_plugin/test/test_cron_parser.cpp | Adds unit tests for cron expression parsing behavior. |
| plugins/cron_plugin/src/services/cron_parser.cpp | Implements cron string parsing into cron_service::job_schedule. |
| plugins/cron_plugin/include/sysio/services/cron_parser.hpp | Exposes cron parser API. |
| plugins/cron_plugin/CRON_PARSER_USAGE.md | Usage documentation for the new cron parser. |
| plugins/CMakeLists.txt | Adds beacon chain update plugin to the plugins build. |
| plugins/beacon_chain_update_plugin/test/test_beacon_chain_update_plugin.cpp | Adds unit tests for beacon-chain update parsing utilities. |
| plugins/beacon_chain_update_plugin/test/main.cpp | Boost.Test entrypoint for beacon-chain update plugin tests. |
| plugins/beacon_chain_update_plugin/test/CMakeLists.txt | Builds and registers the beacon-chain update plugin test binary. |
| plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp | Implements the plugin: scheduling, HTTP fetch, contract tx submission, and parsing helpers. |
| plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_plugin.hpp | Declares the beacon chain update plugin. |
| plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_detail.hpp | Declares testable parsing/utility helpers (queue length, APY conversion). |
| plugins/beacon_chain_update_plugin/CMakeLists.txt | Builds the new plugin and links required dependencies (cron/outpost/curl). |
| libraries/libfc/src/network/ethereum/ethereum_client.cpp | Fixes call/tx hex prefixing and adds identify_block_for_transaction. |
| libraries/libfc/include/fc/network/ethereum/ethereum_client.hpp | Declares the new identify_block_for_transaction API. |
| libraries/libfc/src/network/ethereum/ethereum_abi.cpp | Adjusts ABI decoding to tolerate missing name (e.g., receive/constructor). |
| libraries/libfc/include/fc/network/ethereum/ethereum_abi.hpp | Adds receive to ABI invoke target type enum/reflection. |
| libraries/libfc/src/io/json.cpp | Fixes numeric routing boundaries for big integers when parsing JSON. |
| CRON_PARSER_SUMMARY.md | High-level documentation summary of the cron parser addition. |
| cmake/chain-tools.cmake | Adds beacon_chain_update_plugin to the default chain_target() link set. |
Comments suppressed due to low confidence (2)
plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp:94
chain_idis now always passed as a value (default-constructed to 0) even when the spec omits the optional chain id. This changes behavior vs the previousstd::optionalapproach and can cause transactions to be signed with chainId=0. Consider keepingstd::optional<fc::uint256> chain_idand only setting it whenparts.size() == 4(otherwise passstd::nulloptintoethereum_client).
fc::uint256 chain_id;
fc::ostring chain_id_str;
if (parts.size() == 4) {
chain_id_str = parts[3];
if (chain_id_str.has_value())
chain_id = fc::to_uint256(chain_id_str.value());
} else {
ilog("chainId: none");
}
auto sig_provider = plug_sig->get_provider(sig_id);
my->add_client(id,
std::make_shared<ethereum_client_entry_t>(
id,
url,
sig_provider,
std::make_shared<ethereum_client>(sig_provider, url,
chain_id)));
cmake/chain-tools.cmake:33
- Adding
beacon_chain_update_pluginto the globalchain_target()link list will pull this plugin (and its dependencies like libcurl/OpenSSL) into every executable that useschain_target(including unrelated programs/tests). If onlycrankerneeds this plugin, prefer linking it only to that target instead of the global list.
prometheus_plugin
resource_monitor_plugin
state_history_plugin
signature_provider_manager_plugin
outpost_client_plugin
outpost_ethereum_client_plugin
outpost_solana_client_plugin
beacon_chain_update_plugin
test_control_api_plugin
test_control_plugin
trace_api_plugin
chain_plugin
appbase
${PLUGIN_DEFAULT_DEPENDENCIES}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| asio::io_context ioc; | ||
| asio::ssl::context ssl_ctx{asio::ssl::context::tlsv12_client}; | ||
| tcp::resolver resolver{ioc}; | ||
| auto dest = resolver.resolve(host, port); | ||
|
|
||
| http::request<http::string_body> req{http::verb::post, path, 11}; | ||
| req.set(http::field::host, host); | ||
| req.set(http::field::content_type, "application/json"); | ||
| req.set(http::field::authorization, "Bearer " + api_key); | ||
| req.body() = R"({"chain":"mainnet"})"; | ||
| req.prepare_payload(); | ||
|
|
||
| beast::ssl_stream<beast::tcp_stream> stream(ioc, ssl_ctx); | ||
| if (!SSL_set_tlsext_host_name(stream.native_handle(), host.c_str())) | ||
| throw beast::system_error(beast::error_code(static_cast<int>(::ERR_get_error()), | ||
| asio::error::get_ssl_category())); | ||
|
|
||
| beast::get_lowest_layer(stream).connect(dest); | ||
| stream.handshake(asio::ssl::stream_base::client); | ||
| http::write(stream, req); |
There was a problem hiding this comment.
TLS is used here but certificate verification is never enabled (default verify mode is verify_none), which makes the HTTPS fetch vulnerable to MITM. Configure the SSL context/stream to verify peers (set default verify paths, set verify_peer, and use hostname verification) before performing the handshake.
There was a problem hiding this comment.
Waiting on direction for this one
| file(GLOB_RECURSE SRC_FILES src/*.cpp src/*.hpp) | ||
|
|
||
| chain_target(${TARGET_NAME} SOURCE_FILES ${SRC_FILES}) |
There was a problem hiding this comment.
cranker is intended to be lightweight, but using chain_target() will link in the full default plugin set from cmake/chain-tools.cmake (net, producer, state_history, etc.), increasing binary size and dependencies. Consider creating a minimal executable target (add_executable + target_link_libraries) that links only the required libraries/plugins for cranker.
| return parser_type == json::parse_type::legacy_parser_with_string_doubles ? variant(s) : variant(to_double(s)); | ||
| if( neg ) { | ||
| if( str.size() < check_int128.min_len || | ||
| (str.size() == check_int128.min_len && str < check_int128.min_str) ) |
There was a problem hiding this comment.
@heifner can you double check that I got theses correct:
correct minimums, correct comparator, etc.
Claude wanted to turn the "<"s to "<="s and I checked that I was correct, but would like another pair of eyes.
There was a problem hiding this comment.
Best thing would be to add complete test coverage.
| parse_components(vo.inputs, "inputs"); | ||
| parse_components(vo.outputs, "outputs"); | ||
| bool missed = true; | ||
| if(deferred_name) { |
There was a problem hiding this comment.
I had initially put this in for reporting what we don't expect to see, and determine if something else needs to be reported. Not sure if we should drop or at least keep it in here while we are still working on ethereum client scripts
| const std::string& block_tag, | ||
| const contract_invoke_data_items& params) { | ||
| auto abi_call_encoded = contract_encode_data(abi, params); | ||
| auto abi_call_encoded = "0x" + contract_encode_data(abi, params); |
There was a problem hiding this comment.
Here and 1 or 2 other places where contract_encode_data is called, the prefix is added. Wondering if we should add a default parameter to the method so that the underlying to_hex call can add it. Of course there will still be other places with "0x" manually added, so not sure it is worth it, unless we resolve not to do that.
There was a problem hiding this comment.
Seems to me contract_encode_data call to fc::to_hex should pass add_prefix=true.
| std::promise<uint64_t> promise; | ||
| std::future<uint64_t> future = promise.get_future(); | ||
|
|
||
| constexpr int max_retries = 600; // 10 minutes at 1s/poll |
There was a problem hiding this comment.
@heifner what is your suggestion for a better design for this? Both the creation/deletion of threads and sleep and retries.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| outpost_client_plugin | ||
| outpost_ethereum_client_plugin | ||
| outpost_solana_client_plugin | ||
| beacon_chain_update_plugin |
There was a problem hiding this comment.
I don't think this goes in chain_target as it is not needed for nodeop.
| return parser_type == json::parse_type::legacy_parser_with_string_doubles ? variant(s) : variant(to_double(s)); | ||
| if( neg ) { | ||
| if( str.size() < check_int128.min_len || | ||
| (str.size() == check_int128.min_len && str < check_int128.min_str) ) |
There was a problem hiding this comment.
Best thing would be to add complete test coverage.
| const std::string& block_tag, | ||
| const contract_invoke_data_items& params) { | ||
| auto abi_call_encoded = contract_encode_data(abi, params); | ||
| auto abi_call_encoded = "0x" + contract_encode_data(abi, params); |
There was a problem hiding this comment.
Seems to me contract_encode_data call to fc::to_hex should pass add_prefix=true.
heifner
left a comment
There was a problem hiding this comment.
Code review of cranker/beacon_chain_update_plugin changes. 16 findings across 5 files.
| std::optional<fc::uint256> chain_id; | ||
| if (chain_id_str.has_value()) | ||
| chain_id = std::make_optional<fc::uint256>(fc::to_uint256(chain_id_str.value())); | ||
| fc::uint256 chain_id; |
There was a problem hiding this comment.
Bug: chain_id always passed as 0 instead of nullopt when omitted
The old code used std::optional<fc::uint256> chain_id and only set it when the string was present, passing nullopt to the ethereum_client constructor otherwise — which triggers a chain ID query to the node.
The new code declares a non-optional fc::uint256 chain_id (default-constructed to 0). When parts.size() != 4, chain_id stays at 0 and gets implicitly converted to std::optional<fc::uint256>(0) when passed to the constructor. This tells the client "use chain_id 0" rather than "query the node for the chain ID."
Fix: keep chain_id as std::optional<fc::uint256> and only set it inside the parts.size() == 4 branch.
| auto path = url.path().value_or(std::filesystem::path("/")).string(); | ||
|
|
||
| asio::io_context ioc; | ||
| asio::ssl::context ssl_ctx{asio::ssl::context::tlsv12_client}; |
There was a problem hiding this comment.
No TLS certificate verification on beacon chain API calls
Both get_queues_mainnet (here) and get_ethstore_latest (line 180) create an SSL context but never call ssl_ctx.set_default_verify_paths() or set stream.set_verify_mode(ssl::verify_peer). The server certificate is not validated, making the connection vulnerable to MITM attacks. This matters because the Bearer API key is sent over this connection and responses directly drive on-chain contract calls (setWithdrawDelay, updateApyBPS).
| throw beast::system_error(beast::error_code(static_cast<int>(::ERR_get_error()), | ||
| asio::error::get_ssl_category())); | ||
|
|
||
| beast::get_lowest_layer(stream).connect(dest); |
There was a problem hiding this comment.
Missing connection timeout
get_queues_mainnet correctly sets beast::get_lowest_layer(stream).expires_after(std::chrono::seconds(120)) (line 128), but get_ethstore_latest does not set any timeout before calling connect(). If DNS or TCP connect hangs, the cron thread blocks indefinitely.
| ? default_days | ||
| : *deposit_queue_len_sec / seconds_per_day; // convert sec to min, min to hours, hours to days | ||
| if(!deposit_queue_len_sec) | ||
| wlog("defaulting the {} withdrawal delay to {} day since {} was not a finite number", |
There was a problem hiding this comment.
wlog has mismatched format args
3 {} placeholders but 4 arguments — beacon_chain_detail::epa_field is silently dropped. Likely should be "since {}::{} was not a finite number" (4 placeholders) to match the pattern used at lines 467-469 for the exit queue warning.
Note: fmt silently ignores extra arguments (only too few args is an error), so this compiles and runs without complaint.
| */ | ||
| std::future<uint64_t> ethereum_client::identify_block_for_transaction(const std::string& tx_hash) { | ||
| std::promise<uint64_t> promise; | ||
| std::future<uint64_t> future = promise.get_future(); |
There was a problem hiding this comment.
Detached thread — can't be cancelled or joined on shutdown
std::thread(...).detach() spawns an unjoinable thread that polls for up to 10 minutes. On shutdown, these threads can't be cancelled or awaited. Multiple concurrent calls also accumulate threads.
Suggested approach: named_thread_pool + boost::asio::steady_timer
chain/include/sysio/chain/thread_utils.hpp has named_thread_pool<Tag> which provides an io_context, named threads, clean start()/stop() lifecycle with thread joining, and proper exception handling. ethereum_client could own a 1-thread pool:
struct eth_poll{};
named_thread_pool<eth_poll> _poll_pool; // started in ctor or lazily
std::future<uint64_t> identify_block_for_transaction(const std::string& tx_hash) {
auto promise = std::make_shared<std::promise<uint64_t>>();
auto future = promise->get_future();
auto timer = std::make_shared<boost::asio::steady_timer>(_poll_pool.get_executor());
auto poll = [weak=weak_from_this(), tx_hash, promise, timer,
attempt=std::make_shared<int>(0)]
(auto& self, boost::system::error_code ec) mutable {
if (ec == boost::asio::error::operation_aborted) {
promise->set_exception(std::make_exception_ptr(
std::runtime_error("operation cancelled")));
return;
}
auto client = weak.lock();
if (!client) { promise->set_exception(...); return; }
auto receipt = client->get_transaction_receipt(tx_hash);
if (/* has blockNumber */) { promise->set_value(bn); return; }
if (++(*attempt) >= 600) { promise->set_exception(...); return; }
timer->expires_after(std::chrono::seconds(1));
timer->async_wait([&self](auto ec) { self(self, ec); });
};
boost::asio::post(_poll_pool.get_executor(),
[poll]() mutable { poll(poll, {}); });
return future;
}Key benefits:
- Cancellable —
_poll_pool.stop()cancels timers and joins the thread - No detached threads — clean shutdown in destructor
- Named thread — visible in htop/debugger as
eth_poll-0
Since named_thread_pool currently lives in sysio::chain but ethereum_client is in libfc, it would need to be moved to fc (its only dependencies are fc::set_thread_name, fc::exception, and boost::asio — the move is clean).
If moving named_thread_pool is out of scope for this PR, the same pattern works with a raw boost::asio::io_context + work guard + single std::thread owned by ethereum_client, joined in the destructor. The key improvement is replacing the detached thread + sleep loop with a cancellable timer on a joinable thread.
There was a problem hiding this comment.
Another option would be to use the cron scheduler on a next schedule to pick up the work from there.
|
|
||
| const std::regex regex(R"(^(.+?)(?:V\d+)?$)"); | ||
|
|
||
| [[maybe_unused]] inline fc::logger& logger() { |
There was a problem hiding this comment.
Unused logger() function — remove
This is [[maybe_unused]] and indeed never called. All logging uses ilog/wlog/elog with the default logger. Should be removed since it's dead code.
| const auto deposit_queue_len_sec = beacon_chain_detail::get_queue_length(queues, deposit_queue); | ||
| const auto default_days = 1; | ||
| const auto seconds_per_day = 60 * 60 * 24; | ||
| uint64_t depositQDaysFl = !deposit_queue_len_sec |
There was a problem hiding this comment.
Mixed naming conventions
depositQDaysFl and aprFraction (line 521) use camelCase while the rest of the file uses snake_case consistently (exit_queue_len_sec, exit_queue_delay_len_sec, deposit_queue_len_sec, etc.).
|
|
||
| constexpr auto deposit_queue = "deposit_queue"; | ||
| const auto deposit_queue_len_sec = beacon_chain_detail::get_queue_length(queues, deposit_queue); | ||
| const auto default_days = 1; |
There was a problem hiding this comment.
Inconsistent const vs constexpr
These are compile-time constants but use const while similar values at lines 464-465 use constexpr. Should all be constexpr for consistency.
| const auto default_days = 1; | |
| constexpr auto default_days = 1; | |
| constexpr auto seconds_per_day = 60 * 60 * 24; |
| : ethereum_contract_client(client, contract_address_compat, contracts) | ||
| , finalizeEpoch(create_tx<fc::variant>(get_abi("finalizeEpoch"))) { | ||
|
|
||
| }; |
There was a problem hiding this comment.
Unnecessary semicolons after constructor bodies
All three contract structs (OPP, deposit_manager, withdrawal_queue) have }; after the constructor body. Valid C++ but unusual style — the semicolons are unnecessary here (they're only needed after the class/struct definition, not after member function bodies).
| } | ||
| }, | ||
| cron_service::job_metadata_t{ | ||
| .one_at_a_time = true, .tags = {"ethereum", "gas"}, .label = "cron_1min_heartbeat" |
There was a problem hiding this comment.
Job metadata labels don't match job purpose
Both the "just once" startup job (here) and the interval jobs (line 608) use label "cron_1min_heartbeat". This doesn't describe what these jobs do. Use descriptive labels (e.g. "beacon_chain_startup", "beacon_chain_update") — especially useful for debugging when multiple cron jobs are active.
Summary
contracts.
What the plugin does
beacon_chain_update_plugin runs on a configurable cron schedule and:
with the estimated wait time in seconds.
All on-chain calls are submitted via the existing outpost_ethereum_client_plugin. Block confirmation is awaited using a new identify_block_for_transaction future.
Configuration options (all via nodeop config):
Changes by area
plugins/beacon_chain_update_plugin/ (new)
plugins/beacon_chain_update_plugin/test/ (new)
epsilon robustness).
programs/cranker/ (new)
plugins/cron_plugin/ (new files)
libraries/libfc/src/network/ethereum/ethereum_client.cpp
(10 min); captures weak_from_this() to prevent use-after-free if the client is destroyed before the transaction mines.
libraries/libfc/src/io/json.cpp
numbers ≥ 922... to fall through to int128, 20-digit unsigned values to be routed to uint256 instead of uint128, and the int256/uint256 overflow checks to use inverted direction
comparisons. All four boundaries and their comparison operators are corrected.