Skip to content

Add beacon chain update plugin and cranker executable#280

Open
brianjohnson5972 wants to merge 12 commits intomasterfrom
feature/crank_queues_apy
Open

Add beacon chain update plugin and cranker executable#280
brianjohnson5972 wants to merge 12 commits intomasterfrom
feature/crank_queues_apy

Conversation

@brianjohnson5972
Copy link
Copy Markdown
Contributor

Summary

  • Introduces beacon_chain_update_plugin, a new nodeop plugin that periodically fetches Ethereum beacon chain state from beaconcha.in and pushes updates into on-chain smart
    contracts.
  • Adds cranker, a lightweight standalone executable that runs only the plugins needed for beacon chain updates (no full node required).
  • Adds a cron expression parser to cron_plugin for human-readable schedule configuration.
  • Fixes big-integer type routing in libfc's JSON parser.
  • Adds a bounded-retry identify_block_for_transaction method to ethereum_client.

What the plugin does

beacon_chain_update_plugin runs on a configurable cron schedule and:

  1. Fetches queue lengths from beaconcha.in/api/v2/ethereum/queues — both entry_queue and exit_queue — and calls DepositManager.setEntryQueue and WithdrawalQueue.setWithdrawDelay
    with the estimated wait time in seconds.
  2. Fetches current APY from beaconcha.in/api/v1/ethstore/latest and calls DepositManager.updateApyBPS with the value expressed in basis points.
  3. Finalizes epochs on a separate configurable interval by calling OPP.finalizeEpoch.

All on-chain calls are submitted via the existing outpost_ethereum_client_plugin. Block confirmation is awaited using a new identify_block_for_transaction future.

Configuration options (all via nodeop config):

  • beacon-chain-queue-url — queue endpoint (default: beaconcha.in)
  • beacon-chain-apy-url — APY endpoint (default: beaconcha.in)
  • beacon-chain-api-key — Bearer token for beaconcha.in API
  • beacon-chain-contracts-addrs — JSON map of contract name → address
  • beacon-chain-update-interval — cron schedule for queue/APY updates
  • beacon-chain-finalize-epoch-interval — cron schedule for epoch finalization

Changes by area

plugins/beacon_chain_update_plugin/ (new)

  • Plugin implementation with pimpl pattern; pure utility functions extracted to beacon_chain_update_detail.hpp for testability.
  • Contract client structs (OPP, deposit_manager, withdrawal_queue) wrapping ethereum_contract_client.
  • HTTP fetchers using Boost.Beast over TLS for both the queue and APY endpoints.
  • Loop capture bug fixed: cron job lambdas capture shared_ptr + string by value rather than dangling loop-local references.
  • Unsigned wrap guard in get_queue_length: returns nullopt if the API returns a past timestamp rather than wrapping to a huge value.

plugins/beacon_chain_update_plugin/test/ (new)

  • 15 Boost.Test cases covering get_field_from_object, get_queue_length (including missing fields, wrong types, past timestamps), and apy_fraction_to_bps (including floating-point
    epsilon robustness).

programs/cranker/ (new)

  • Minimal main.cpp that boots signature_provider_manager_plugin, outpost_ethereum_client_plugin, cron_plugin, and beacon_chain_update_plugin — no chain, no p2p.

plugins/cron_plugin/ (new files)

  • cron_parser.hpp/cpp: parses standard 5-field and extended 6-field (milliseconds) cron expressions into cron_service::job_schedule objects, with full range/step/list syntax.

libraries/libfc/src/network/ethereum/ethereum_client.cpp

  • identify_block_for_transaction(tx_hash): polls eth_getTransactionReceipt on a detached thread, returning a std::future<uint64_t> for the block number. Bounded to 600 attempts
    (10 min); captures weak_from_this() to prevent use-after-free if the client is destroyed before the transaction mines.
  • Fixed ABI call encoding: 0x prefix added to encoded call data; to_hex called with with_prefix=true.

libraries/libfc/src/io/json.cpp

  • Fixed big-integer type routing in number_from_stream. The min_len boundary values were all off by one (min_str.size() - 1 instead of min_str.size()), causing 18-digit negative
    numbers ≥ 922... to fall through to int128, 20-digit unsigned values to be routed to uint256 instead of uint128, and the int256/uint256 overflow checks to use inverted direction
    comparisons. All four boundaries and their comparison operators are corrected.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds new infrastructure for periodically fetching Ethereum beacon-chain metrics and pushing them on-chain, plus a lightweight runner executable to operate those updates without a full node.

Changes:

  • Introduces beacon_chain_update_plugin to fetch queue/APY data from beaconcha.in and submit contract updates via the outpost Ethereum client.
  • Adds cranker as a standalone executable that initializes only the plugin subset required for beacon-chain updates.
  • Adds a cron expression parser to cron_plugin and updates libfc Ethereum/JSON utilities (tx block identification + big-integer JSON parsing fixes).

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
programs/cranker/src/main.cpp New minimal entrypoint that boots only the plugins required for beacon-chain updates.
programs/cranker/README.md Documents cranker usage and CLI configuration.
programs/cranker/CMakeLists.txt Adds build target for the cranker executable.
programs/CMakeLists.txt Adds cranker subdirectory to the programs build.
plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp Updates client parsing and makes getters const.
plugins/outpost_ethereum_client_plugin/include/sysio/outpost_ethereum_client_plugin.hpp Makes client/ABI getters const.
plugins/cron_plugin/test/test_cron_parser.cpp Adds unit tests for cron expression parsing behavior.
plugins/cron_plugin/src/services/cron_parser.cpp Implements cron string parsing into cron_service::job_schedule.
plugins/cron_plugin/include/sysio/services/cron_parser.hpp Exposes cron parser API.
plugins/cron_plugin/CRON_PARSER_USAGE.md Usage documentation for the new cron parser.
plugins/CMakeLists.txt Adds beacon chain update plugin to the plugins build.
plugins/beacon_chain_update_plugin/test/test_beacon_chain_update_plugin.cpp Adds unit tests for beacon-chain update parsing utilities.
plugins/beacon_chain_update_plugin/test/main.cpp Boost.Test entrypoint for beacon-chain update plugin tests.
plugins/beacon_chain_update_plugin/test/CMakeLists.txt Builds and registers the beacon-chain update plugin test binary.
plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Implements the plugin: scheduling, HTTP fetch, contract tx submission, and parsing helpers.
plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_plugin.hpp Declares the beacon chain update plugin.
plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_detail.hpp Declares testable parsing/utility helpers (queue length, APY conversion).
plugins/beacon_chain_update_plugin/CMakeLists.txt Builds the new plugin and links required dependencies (cron/outpost/curl).
libraries/libfc/src/network/ethereum/ethereum_client.cpp Fixes call/tx hex prefixing and adds identify_block_for_transaction.
libraries/libfc/include/fc/network/ethereum/ethereum_client.hpp Declares the new identify_block_for_transaction API.
libraries/libfc/src/network/ethereum/ethereum_abi.cpp Adjusts ABI decoding to tolerate missing name (e.g., receive/constructor).
libraries/libfc/include/fc/network/ethereum/ethereum_abi.hpp Adds receive to ABI invoke target type enum/reflection.
libraries/libfc/src/io/json.cpp Fixes numeric routing boundaries for big integers when parsing JSON.
CRON_PARSER_SUMMARY.md High-level documentation summary of the cron parser addition.
cmake/chain-tools.cmake Adds beacon_chain_update_plugin to the default chain_target() link set.
Comments suppressed due to low confidence (2)

plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp:94

  • chain_id is now always passed as a value (default-constructed to 0) even when the spec omits the optional chain id. This changes behavior vs the previous std::optional approach and can cause transactions to be signed with chainId=0. Consider keeping std::optional<fc::uint256> chain_id and only setting it when parts.size() == 4 (otherwise pass std::nullopt into ethereum_client).
      fc::uint256 chain_id;
      fc::ostring chain_id_str;
      if (parts.size() == 4) {
         chain_id_str = parts[3];
         if (chain_id_str.has_value())
            chain_id = fc::to_uint256(chain_id_str.value());
      } else {
         ilog("chainId: none");
      }

      auto  sig_provider = plug_sig->get_provider(sig_id);
      my->add_client(id,
                     std::make_shared<ethereum_client_entry_t>(
                        id,
                        url,
                        sig_provider,
                        std::make_shared<ethereum_client>(sig_provider, url,
                           chain_id)));

cmake/chain-tools.cmake:33

  • Adding beacon_chain_update_plugin to the global chain_target() link list will pull this plugin (and its dependencies like libcurl/OpenSSL) into every executable that uses chain_target (including unrelated programs/tests). If only cranker needs this plugin, prefer linking it only to that target instead of the global list.
            prometheus_plugin
            resource_monitor_plugin
            state_history_plugin
            signature_provider_manager_plugin
            outpost_client_plugin
            outpost_ethereum_client_plugin
            outpost_solana_client_plugin
            beacon_chain_update_plugin
            test_control_api_plugin
            test_control_plugin
            trace_api_plugin
            chain_plugin
            appbase
            ${PLUGIN_DEFAULT_DEPENDENCIES}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +111 to +130
asio::io_context ioc;
asio::ssl::context ssl_ctx{asio::ssl::context::tlsv12_client};
tcp::resolver resolver{ioc};
auto dest = resolver.resolve(host, port);

http::request<http::string_body> req{http::verb::post, path, 11};
req.set(http::field::host, host);
req.set(http::field::content_type, "application/json");
req.set(http::field::authorization, "Bearer " + api_key);
req.body() = R"({"chain":"mainnet"})";
req.prepare_payload();

beast::ssl_stream<beast::tcp_stream> stream(ioc, ssl_ctx);
if (!SSL_set_tlsext_host_name(stream.native_handle(), host.c_str()))
throw beast::system_error(beast::error_code(static_cast<int>(::ERR_get_error()),
asio::error::get_ssl_category()));

beast::get_lowest_layer(stream).connect(dest);
stream.handshake(asio::ssl::stream_base::client);
http::write(stream, req);
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLS is used here but certificate verification is never enabled (default verify mode is verify_none), which makes the HTTPS fetch vulnerable to MITM. Configure the SSL context/stream to verify peers (set default verify paths, set verify_peer, and use hostname verification) before performing the handshake.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting on direction for this one

Comment on lines +4 to +6
file(GLOB_RECURSE SRC_FILES src/*.cpp src/*.hpp)

chain_target(${TARGET_NAME} SOURCE_FILES ${SRC_FILES})
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cranker is intended to be lightweight, but using chain_target() will link in the full default plugin set from cmake/chain-tools.cmake (net, producer, state_history, etc.), increasing binary size and dependencies. Consider creating a minimal executable target (add_executable + target_link_libraries) that links only the required libraries/plugins for cranker.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@brianjohnson5972 brianjohnson5972 Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing

return parser_type == json::parse_type::legacy_parser_with_string_doubles ? variant(s) : variant(to_double(s));
if( neg ) {
if( str.size() < check_int128.min_len ||
(str.size() == check_int128.min_len && str < check_int128.min_str) )
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heifner can you double check that I got theses correct:
correct minimums, correct comparator, etc.
Claude wanted to turn the "<"s to "<="s and I checked that I was correct, but would like another pair of eyes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best thing would be to add complete test coverage.

parse_components(vo.inputs, "inputs");
parse_components(vo.outputs, "outputs");
bool missed = true;
if(deferred_name) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had initially put this in for reporting what we don't expect to see, and determine if something else needs to be reported. Not sure if we should drop or at least keep it in here while we are still working on ethereum client scripts

const std::string& block_tag,
const contract_invoke_data_items& params) {
auto abi_call_encoded = contract_encode_data(abi, params);
auto abi_call_encoded = "0x" + contract_encode_data(abi, params);
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and 1 or 2 other places where contract_encode_data is called, the prefix is added. Wondering if we should add a default parameter to the method so that the underlying to_hex call can add it. Of course there will still be other places with "0x" manually added, so not sure it is worth it, unless we resolve not to do that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to me contract_encode_data call to fc::to_hex should pass add_prefix=true.

std::promise<uint64_t> promise;
std::future<uint64_t> future = promise.get_future();

constexpr int max_retries = 600; // 10 minutes at 1s/poll
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heifner what is your suggestion for a better design for this? Both the creation/deletion of threads and sleep and retries.

@brianjohnson5972 brianjohnson5972 requested a review from qhool April 3, 2026 13:39
outpost_client_plugin
outpost_ethereum_client_plugin
outpost_solana_client_plugin
beacon_chain_update_plugin
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this goes in chain_target as it is not needed for nodeop.

return parser_type == json::parse_type::legacy_parser_with_string_doubles ? variant(s) : variant(to_double(s));
if( neg ) {
if( str.size() < check_int128.min_len ||
(str.size() == check_int128.min_len && str < check_int128.min_str) )
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best thing would be to add complete test coverage.

const std::string& block_tag,
const contract_invoke_data_items& params) {
auto abi_call_encoded = contract_encode_data(abi, params);
auto abi_call_encoded = "0x" + contract_encode_data(abi, params);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to me contract_encode_data call to fc::to_hex should pass add_prefix=true.

Copy link
Copy Markdown
Contributor

@heifner heifner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review of cranker/beacon_chain_update_plugin changes. 16 findings across 5 files.

std::optional<fc::uint256> chain_id;
if (chain_id_str.has_value())
chain_id = std::make_optional<fc::uint256>(fc::to_uint256(chain_id_str.value()));
fc::uint256 chain_id;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: chain_id always passed as 0 instead of nullopt when omitted

The old code used std::optional<fc::uint256> chain_id and only set it when the string was present, passing nullopt to the ethereum_client constructor otherwise — which triggers a chain ID query to the node.

The new code declares a non-optional fc::uint256 chain_id (default-constructed to 0). When parts.size() != 4, chain_id stays at 0 and gets implicitly converted to std::optional<fc::uint256>(0) when passed to the constructor. This tells the client "use chain_id 0" rather than "query the node for the chain ID."

Fix: keep chain_id as std::optional<fc::uint256> and only set it inside the parts.size() == 4 branch.

auto path = url.path().value_or(std::filesystem::path("/")).string();

asio::io_context ioc;
asio::ssl::context ssl_ctx{asio::ssl::context::tlsv12_client};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No TLS certificate verification on beacon chain API calls

Both get_queues_mainnet (here) and get_ethstore_latest (line 180) create an SSL context but never call ssl_ctx.set_default_verify_paths() or set stream.set_verify_mode(ssl::verify_peer). The server certificate is not validated, making the connection vulnerable to MITM attacks. This matters because the Bearer API key is sent over this connection and responses directly drive on-chain contract calls (setWithdrawDelay, updateApyBPS).

throw beast::system_error(beast::error_code(static_cast<int>(::ERR_get_error()),
asio::error::get_ssl_category()));

beast::get_lowest_layer(stream).connect(dest);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing connection timeout

get_queues_mainnet correctly sets beast::get_lowest_layer(stream).expires_after(std::chrono::seconds(120)) (line 128), but get_ethstore_latest does not set any timeout before calling connect(). If DNS or TCP connect hangs, the cron thread blocks indefinitely.

? default_days
: *deposit_queue_len_sec / seconds_per_day; // convert sec to min, min to hours, hours to days
if(!deposit_queue_len_sec)
wlog("defaulting the {} withdrawal delay to {} day since {} was not a finite number",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wlog has mismatched format args

3 {} placeholders but 4 arguments — beacon_chain_detail::epa_field is silently dropped. Likely should be "since {}::{} was not a finite number" (4 placeholders) to match the pattern used at lines 467-469 for the exit queue warning.

Note: fmt silently ignores extra arguments (only too few args is an error), so this compiles and runs without complaint.

*/
std::future<uint64_t> ethereum_client::identify_block_for_transaction(const std::string& tx_hash) {
std::promise<uint64_t> promise;
std::future<uint64_t> future = promise.get_future();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detached thread — can't be cancelled or joined on shutdown

std::thread(...).detach() spawns an unjoinable thread that polls for up to 10 minutes. On shutdown, these threads can't be cancelled or awaited. Multiple concurrent calls also accumulate threads.

Suggested approach: named_thread_pool + boost::asio::steady_timer

chain/include/sysio/chain/thread_utils.hpp has named_thread_pool<Tag> which provides an io_context, named threads, clean start()/stop() lifecycle with thread joining, and proper exception handling. ethereum_client could own a 1-thread pool:

struct eth_poll{};
named_thread_pool<eth_poll> _poll_pool; // started in ctor or lazily

std::future<uint64_t> identify_block_for_transaction(const std::string& tx_hash) {
   auto promise = std::make_shared<std::promise<uint64_t>>();
   auto future = promise->get_future();
   auto timer = std::make_shared<boost::asio::steady_timer>(_poll_pool.get_executor());

   auto poll = [weak=weak_from_this(), tx_hash, promise, timer,
                attempt=std::make_shared<int>(0)]
               (auto& self, boost::system::error_code ec) mutable {
      if (ec == boost::asio::error::operation_aborted) {
         promise->set_exception(std::make_exception_ptr(
            std::runtime_error("operation cancelled")));
         return;
      }
      auto client = weak.lock();
      if (!client) { promise->set_exception(...); return; }

      auto receipt = client->get_transaction_receipt(tx_hash);
      if (/* has blockNumber */) { promise->set_value(bn); return; }
      if (++(*attempt) >= 600) { promise->set_exception(...); return; }

      timer->expires_after(std::chrono::seconds(1));
      timer->async_wait([&self](auto ec) { self(self, ec); });
   };

   boost::asio::post(_poll_pool.get_executor(),
                     [poll]() mutable { poll(poll, {}); });
   return future;
}

Key benefits:

  • Cancellable_poll_pool.stop() cancels timers and joins the thread
  • No detached threads — clean shutdown in destructor
  • Named thread — visible in htop/debugger as eth_poll-0

Since named_thread_pool currently lives in sysio::chain but ethereum_client is in libfc, it would need to be moved to fc (its only dependencies are fc::set_thread_name, fc::exception, and boost::asio — the move is clean).

If moving named_thread_pool is out of scope for this PR, the same pattern works with a raw boost::asio::io_context + work guard + single std::thread owned by ethereum_client, joined in the destructor. The key improvement is replacing the detached thread + sleep loop with a cancellable timer on a joinable thread.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to use the cron scheduler on a next schedule to pick up the work from there.


const std::regex regex(R"(^(.+?)(?:V\d+)?$)");

[[maybe_unused]] inline fc::logger& logger() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused logger() function — remove

This is [[maybe_unused]] and indeed never called. All logging uses ilog/wlog/elog with the default logger. Should be removed since it's dead code.

const auto deposit_queue_len_sec = beacon_chain_detail::get_queue_length(queues, deposit_queue);
const auto default_days = 1;
const auto seconds_per_day = 60 * 60 * 24;
uint64_t depositQDaysFl = !deposit_queue_len_sec
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mixed naming conventions

depositQDaysFl and aprFraction (line 521) use camelCase while the rest of the file uses snake_case consistently (exit_queue_len_sec, exit_queue_delay_len_sec, deposit_queue_len_sec, etc.).


constexpr auto deposit_queue = "deposit_queue";
const auto deposit_queue_len_sec = beacon_chain_detail::get_queue_length(queues, deposit_queue);
const auto default_days = 1;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent const vs constexpr

These are compile-time constants but use const while similar values at lines 464-465 use constexpr. Should all be constexpr for consistency.

Suggested change
const auto default_days = 1;
constexpr auto default_days = 1;
constexpr auto seconds_per_day = 60 * 60 * 24;

: ethereum_contract_client(client, contract_address_compat, contracts)
, finalizeEpoch(create_tx<fc::variant>(get_abi("finalizeEpoch"))) {

};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary semicolons after constructor bodies

All three contract structs (OPP, deposit_manager, withdrawal_queue) have }; after the constructor body. Valid C++ but unusual style — the semicolons are unnecessary here (they're only needed after the class/struct definition, not after member function bodies).

}
},
cron_service::job_metadata_t{
.one_at_a_time = true, .tags = {"ethereum", "gas"}, .label = "cron_1min_heartbeat"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Job metadata labels don't match job purpose

Both the "just once" startup job (here) and the interval jobs (line 608) use label "cron_1min_heartbeat". This doesn't describe what these jobs do. Use descriptive labels (e.g. "beacon_chain_startup", "beacon_chain_update") — especially useful for debugging when multiple cron jobs are active.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants