diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 9a636f2b094..b85ccb2a13e 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -24,6 +24,16 @@ Test fixtures for use by clients are available for each release on the [Github r - 🐞 Fix a bug with `consume sync` tests where some clients don't have JSON-RPC immediately available after syncing and can't yet serve the synced block ([#1670](https://github.com/ethereum/execution-specs/pull/1670)). +#### `execute` + +- ✨ `execute hive` and `execute remote` now defer funding of accounts until the minimum amount required to send the test transactions is calculated, in order to optimize the amount of Eth used to execute the tests ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- ✨ Dynamically fetch gas prices from the network and update all transactions to use 1.5x the current values ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- ✨ New `--dry-run` flag to calculate the amount of Eth that will be spent executing a test given the current network gas prices ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- 🔀 Load balancing mode of `execute` for xdist was updated from `loadscope` to `load` ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- 💥 `--eoa-fund-amount-default` has been deprecated since the command now automatically calculates the funding amount ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- 💥 `--sender-key-initial-balance` flag of `execute hive` has been renamed to `--seed-key-initial-balance` ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). +- 🔀 Flags --default-gas-price, --default-max-fee-per-gas and --default-max-priority-fee-per-gas now default to None and ideally should be omitted because, when unset, the command now defaults to fetch the value from the network, which is a more reliable behavior ([#1822](https://github.com/ethereum/execution-specs/pull/1822)). + ### 📋 Misc - 🐞 WELDed the EEST tox environments relevant to producing documentation into EELS, and added a tool to cleanly add codespell whitelist entries. ([#1695](https://github.com/ethereum/execution-specs/pull/1659)). diff --git a/docs/running_tests/execute/hive.md b/docs/running_tests/execute/hive.md index 128938a29ce..6af210f9090 100644 --- a/docs/running_tests/execute/hive.md +++ b/docs/running_tests/execute/hive.md @@ -4,12 +4,20 @@ Tests can be executed on a local hive-controlled single-client network by runnin ## The `eest/execute-blobs` Simulator -The `blob_transaction_test` execute test spec sends blob transactions to a running client in order to verify its `engine_getBlobsVX` endpoint behavior. These tests can be run using: +The `blob_transaction_test` execute test spec sends blob transactions to a running client. Blob transactions are fully supported in execute mode: + +- Blob transactions can be sent via `eth_sendRawTransaction` +- Blob validation via `engine_getBlobsVX` endpoints (when Engine RPC available) +- Automatic gas pricing is used for the blob gas fees + +Tests can be run using: ```bash -./hive --client besu --client-file ./configs/osaka.yaml --sim ethereum/eest/execute-blobs +./hive --client besu --client-file ./configs/osaka.yaml --sim ethereum/eels/execute-blobs ``` +**Note**: If the Engine RPC is unavailable, blob transactions will be sent and `getBlobsV*` validation is skipped. + See [Hive](../hive/index.md) for help installing and configuring Hive. ## Running `execute` tests with Hive in Dev Mode diff --git a/docs/running_tests/execute/index.md b/docs/running_tests/execute/index.md index 596e6c67cc2..8f576abb259 100644 --- a/docs/running_tests/execute/index.md +++ b/docs/running_tests/execute/index.md @@ -39,4 +39,12 @@ Reasoning behind the random generation of the sender keys is that one can execut The `fill` plugin will use a fixed and minimum gas price for all the transactions it uses for testing, but this is not possible with the `execute` plugin, as the gas price is determined by the current state of the network. -At the moment, the `execute` plugin does not query the client for the current gas price, but instead uses a fixed increment to the gas price in order to avoid the transactions to be stuck in the mempool. +The `execute` plugin queries the network for current gas prices and defaults to 1.5x the network price to ensure transaction inclusion. Gas prices can be overridden via command-line flags (`--default-gas-price`, `--default-max-fee-per-gas`, `--default-max-priority-fee-per-gas`). + +### Deferred EOA Funding + +EOAs are funded after gas prices are determined, enabling accurate balance calculations based on actual network conditions. This ensures sufficient funds are allocated for all test transactions. + +### Blob Transaction Support + +Blob transactions are fully supported in execute mode, including automatic gas pricing for blob gas fees and validation via `engine_getBlobsVX` endpoints when the Engine RPC is available. diff --git a/docs/running_tests/execute/remote.md b/docs/running_tests/execute/remote.md index 77c3231316c..e382de0b90a 100644 --- a/docs/running_tests/execute/remote.md +++ b/docs/running_tests/execute/remote.md @@ -24,6 +24,42 @@ One last requirement is that the `--chain-id` flag is set to the chain id of the uv run execute remote --fork=Prague --rpc-endpoint=https://rpc.endpoint.io --rpc-seed-key 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f --chain-id 12345 ``` +## Test Accounts and Contracts + +Since `execute remote` does not control the blockchain where it runs the tests, and therefore cannot modify the genesis pre-allocation like `fill` does, all accounts and contracts need to be deployed via transactions sent to the network. + +These transactions are created from the seed or worker accounts provided via the command flags. + +When the test is executed, all `pre.fund_eoa` and `pre.deploy_contract` calls generate transactions that create the accounts on chain, instead of placing them directly in the `pre` object like `fill` does. + +The transactions are collected and only sent after the test function finishes execution. This is done in order to perform optimizations based on the transactions that the test requires to perform its verifications. + +One optimization is the deferred calculation of the funding amount for the EOA, which is calculated on the fly depending the test transactions that use the account as sender, and this amount is the minimum balance that the account would need in order for the transactions to be included given the current network gas prices. + +### Dry Run Mode + +Dry run mode calculates the minimum balance required without executing any transactions on chain: + +```bash +uv run execute remote --fork=Prague --rpc-endpoint=https://rpc.endpoint.io --dry-run ./tests/prague/eip7702_set_code_tx/ +``` + +This outputs the minimum balance needed and total gas consumption per test, useful for: + +- Estimating gas costs before execution +- Verifying the seed account has sufficient funds +- Planning parallel execution funding requirements + +### Limit Gas Used By Tests + +A limit of the total gas consumption per test can be specified with the `--max-gas-per-test` flag: + +```bash +uv run execute remote --fork=Prague --rpc-endpoint=https://rpc.endpoint.io --max-gas-per-test 30000000 --rpc-seed-key 0x... --chain-id 12345 +``` + +Tests exceeding this limit will fail with an assertion error and will not send any transactions to the chain. + ## Engine RPC Endpoint (Optional) By default, the `execute remote` command assumes that the execution client is connected to a beacon node and the chain progresses automatically. However, you can optionally specify an Engine RPC endpoint to drive the chain manually when new transactions are submitted. diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py index 452938291b7..93b84b80653 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py @@ -12,11 +12,11 @@ from execution_testing.execution import BaseExecute from execution_testing.forks import Fork +from execution_testing.logging import get_logger from execution_testing.rpc import EngineRPC, EthRPC from execution_testing.specs import BaseTest from execution_testing.test_types import ( EnvironmentDefaults, - TransactionDefaults, ) from ..shared.execute_fill import ALL_FIXTURE_PARAMETERS @@ -28,6 +28,8 @@ from ..spec_version_checker.spec_version_checker import EIPSpecTestItem from .pre_alloc import Alloc +logger = get_logger(__name__) + def default_html_report_file_path() -> str: """ @@ -47,9 +49,10 @@ def pytest_addoption(parser: pytest.Parser) -> None: action="store", dest="default_gas_price", type=int, - default=10**9, + default=None, help=( - "Default gas price used for transactions, unless overridden by the test." + "Default gas price used for transactions, unless overridden by the test. " + "Default=None (1.5x current network gas price)" ), ) execute_group.addoption( @@ -57,9 +60,10 @@ def pytest_addoption(parser: pytest.Parser) -> None: action="store", dest="default_max_fee_per_gas", type=int, - default=10**9, + default=None, help=( - "Default max fee per gas used for transactions, unless overridden by the test." + "Default max fee per gas used for transactions, unless overridden by the test. " + "Default=None (1.5x current network max fee per gas)" ), ) execute_group.addoption( @@ -67,10 +71,22 @@ def pytest_addoption(parser: pytest.Parser) -> None: action="store", dest="default_max_priority_fee_per_gas", type=int, - default=10**9, + default=None, help=( "Default max priority fee per gas used for transactions, " "unless overridden by the test." + "Default=None (1.5x current network max priority fee per gas)" + ), + ) + execute_group.addoption( + "--default-max-fee-per-blob-gas", + action="store", + dest="default_max_fee_per_blob_gas", + type=int, + default=None, + help=( + "Default max fee per blob gas used for transactions, unless overridden by the test." + "Default=None (1.5x current network max fee per blob gas)" ), ) execute_group.addoption( @@ -105,6 +121,23 @@ def pytest_addoption(parser: pytest.Parser) -> None: "Time to wait after sending a forkchoice_updated before getting the payload." ), ) + execute_group.addoption( + "--max-gas-per-test", + action="store", + dest="max_gas_per_test", + default=None, + type=int, + help=( + "Maximum gas limit for all transactions in a test. Default=None (No limit)" + ), + ) + execute_group.addoption( + "--dry-run", + action="store_true", + dest="dry_run", + default=False, + help="Don't send transactions, just print the minimum balance required per test.", + ) report_group = parser.getgroup( "tests", "Arguments defining html report behavior" @@ -142,10 +175,12 @@ def pytest_configure(config: pytest.Config) -> None: EnvironmentDefaults.gas_limit = config.getoption( "transaction_gas_limit" ) + + config.engine_rpc_supported = False # type: ignore[attr-defined] + if is_help_or_collectonly_mode(config): return - config.engine_rpc_supported = False # type: ignore[attr-defined] if ( config.getoption("disable_html") and config.getoption("htmlpath") is None @@ -255,43 +290,178 @@ def transactions_per_block( @pytest.fixture(scope="session") -def default_gas_price(request: pytest.FixtureRequest) -> int: +def default_gas_price(request: pytest.FixtureRequest) -> int | None: """Return default gas price used for transactions.""" gas_price = request.config.getoption("default_gas_price") - assert gas_price > 0, "Gas price must be greater than 0" + if gas_price is not None: + assert gas_price > 0, "Gas price must be greater than 0" + logger.debug( + f"Using configured default gas price: {gas_price / 10**9:.2f} Gwei" + ) + else: + logger.debug( + "No default gas price configured, will use network gas price * 1.5" + ) return gas_price +@pytest.fixture(scope="session") +def dry_run(request: pytest.FixtureRequest) -> bool: + """Return True if the test is a dry run.""" + return request.config.getoption("dry_run") + + @pytest.fixture(scope="session") def default_max_fee_per_gas( request: pytest.FixtureRequest, -) -> int: +) -> int | None: """Return default max fee per gas used for transactions.""" - return request.config.getoption("default_max_fee_per_gas") + max_fee_per_gas = request.config.getoption("default_max_fee_per_gas") + if max_fee_per_gas is not None: + logger.debug( + f"Using configured default max fee per gas: {max_fee_per_gas / 10**9:.2f} Gwei" + ) + else: + logger.debug( + "No default max fee per gas configured, will use network gas price * 1.5" + ) + return max_fee_per_gas @pytest.fixture(scope="session") def default_max_priority_fee_per_gas( request: pytest.FixtureRequest, -) -> int: +) -> int | None: """Return default max priority fee per gas used for transactions.""" - return request.config.getoption("default_max_priority_fee_per_gas") + max_priority_fee_per_gas = request.config.getoption( + "default_max_priority_fee_per_gas" + ) + if max_priority_fee_per_gas is not None: + logger.debug( + f"Using configured default max priority fee per gas: {max_priority_fee_per_gas / 10**9:.2f} Gwei" + ) + else: + logger.debug( + "No default max priority fee per gas configured, will use network max priority fee * 1.5" + ) + return max_priority_fee_per_gas -@pytest.fixture(autouse=True, scope="session") -def modify_transaction_defaults( - default_gas_price: int, - default_max_fee_per_gas: int, - default_max_priority_fee_per_gas: int, -) -> None: - """ - Modify transaction defaults to values better suited for live networks. - """ - TransactionDefaults.gas_price = default_gas_price - TransactionDefaults.max_fee_per_gas = default_max_fee_per_gas - TransactionDefaults.max_priority_fee_per_gas = ( - default_max_priority_fee_per_gas +@pytest.fixture(scope="session") +def default_max_fee_per_blob_gas( + request: pytest.FixtureRequest, +) -> int | None: + """Return default max fee per blob gas used for transactions.""" + max_fee_per_blob_gas = request.config.getoption( + "default_max_fee_per_blob_gas" + ) + if max_fee_per_blob_gas is not None: + logger.debug( + f"Using configured default max fee per blob gas: {max_fee_per_blob_gas / 10**9:.2f} Gwei" + ) + else: + logger.debug( + "No default max fee per blob gas configured, will use network blob base fee * 1.5" + ) + return max_fee_per_blob_gas + + +@pytest.fixture(scope="function") +def max_priority_fee_per_gas( + eth_rpc: EthRPC, + default_max_priority_fee_per_gas: int | None, +) -> int: + """Return max priority fee per gas used for transactions in a given test.""" + max_priority_fee_per_gas = default_max_priority_fee_per_gas + if max_priority_fee_per_gas is None: + network_max_priority_fee = eth_rpc.max_priority_fee_per_gas() + max_priority_fee_per_gas = int(network_max_priority_fee * 1.5) + logger.info( + f"Calculated max priority fee per gas from network: {network_max_priority_fee / 10**9:.2f} Gwei * 1.5 = {max_priority_fee_per_gas / 10**9:.2f} Gwei" + ) + else: + logger.info( + f"Using default max priority fee per gas: {max_priority_fee_per_gas / 10**9:.2f} Gwei" + ) + return max_priority_fee_per_gas + + +@pytest.fixture(scope="function") +def max_fee_per_gas( + eth_rpc: EthRPC, + default_max_fee_per_gas: int | None, + max_priority_fee_per_gas: int, +) -> int: + """Return max fee per gas used for transactions in a given test.""" + max_fee_per_gas = default_max_fee_per_gas + if max_fee_per_gas is None: + network_gas_price = eth_rpc.gas_price() + max_fee_per_gas = int(network_gas_price * 1.5) + logger.info( + f"Calculated max fee per gas from network: {network_gas_price / 10**9:.2f} Gwei * 1.5 = {max_fee_per_gas / 10**9:.2f} Gwei" + ) + else: + logger.info( + f"Using default max fee per gas: {max_fee_per_gas / 10**9:.2f} Gwei" + ) + if max_priority_fee_per_gas > max_fee_per_gas: + # Depending on the timing of the request, the priority fee may be + # greater than the max fee. This is a workaround to ensure that the + # transaction is valid. + logger.warning( + f"Max priority fee per gas ({max_priority_fee_per_gas / 10**9:.2f} Gwei) is greater than max fee per gas ({max_fee_per_gas / 10**9:.2f} Gwei), " + f"adjusting max fee per gas to {(max_priority_fee_per_gas + 1) / 10**9:.2f} Gwei" + ) + max_fee_per_gas = max_priority_fee_per_gas + 1 + logger.debug( + f"Final max fee per gas: {max_fee_per_gas / 10**9:.2f} Gwei, max priority fee per gas: {max_priority_fee_per_gas / 10**9:.2f} Gwei" + ) + return max_fee_per_gas + + +@pytest.fixture(scope="function") +def max_fee_per_blob_gas( + eth_rpc: EthRPC, + default_max_fee_per_blob_gas: int | None, +) -> int: + """Return max fee per blob gas used for transactions in a given test.""" + max_fee_per_blob_gas = default_max_fee_per_blob_gas + if max_fee_per_blob_gas is None: + network_blob_base_fee = eth_rpc.blob_base_fee() + max_fee_per_blob_gas = int(network_blob_base_fee * 1.5) + logger.info( + f"Calculated max fee per blob gas from network: {network_blob_base_fee / 10**9:.2f} Gwei * 1.5 = {max_fee_per_blob_gas / 10**9:.2f} Gwei" + ) + else: + logger.info( + f"Using default max fee per blob gas: {max_fee_per_blob_gas / 10**9:.2f} Gwei" + ) + return max_fee_per_blob_gas + + +@pytest.fixture(scope="function") +def gas_price(max_fee_per_gas: int, max_priority_fee_per_gas: int) -> int: + """Return gas price used for transactions in a given test.""" + calculated_gas_price = max_fee_per_gas + max_priority_fee_per_gas + logger.debug( + f"Calculated gas price: {max_fee_per_gas / 10**9:.2f} Gwei (max fee) + {max_priority_fee_per_gas / 10**9:.2f} Gwei (max priority fee) = {calculated_gas_price / 10**9:.2f} Gwei" ) + return calculated_gas_price + + +@pytest.fixture() +def max_gas_limit_per_test(request: pytest.FixtureRequest) -> int | None: + """Return the total gas limit for all transactions in a given test.""" + max_gas_limit = request.config.getoption("max_gas_per_test") + if max_gas_limit is not None: + logger.info( + f"Using configured max gas limit per test: {max_gas_limit}" + ) + else: + logger.debug( + "No max gas limit per test configured, no limit will be enforced" + ) + return max_gas_limit @dataclass(kw_only=True) @@ -324,6 +494,53 @@ def collector( yield collector +@dataclass(kw_only=True) +class GasInfo: + """A class that contains gas limit and minimum balance for a test.""" + + gas_limit: int + minimum_balance: int + + +@dataclass(kw_only=True) +class GasInfoAccumulator: + """A class that accumulates gas limit for all tests.""" + + test_gas_info: Dict[str, GasInfo] = field(default_factory=dict) + + def add( + self, test_name: str, gas_limit: int, minimum_balance: int + ) -> None: + """Add gas limit and minimum balance for a test.""" + self.test_gas_info[test_name] = GasInfo( + gas_limit=gas_limit, minimum_balance=minimum_balance + ) + + def total_gas_limit(self) -> int: + """Return the total gas limit for all tests.""" + return sum( + gas_info.gas_limit for gas_info in self.test_gas_info.values() + ) + + def total_minimum_balance(self) -> int: + """Return the total minimum balance for all tests.""" + return sum( + gas_info.minimum_balance + for gas_info in self.test_gas_info.values() + ) + + +@pytest.fixture(scope="session") +def gas_limit_accumulator() -> Generator[GasInfoAccumulator, None, None]: + """Return the gas limit accumulator for all tests.""" + gas_limit_accumulator = GasInfoAccumulator() + yield gas_limit_accumulator + logger.info(f"Total gas limit: {gas_limit_accumulator.total_gas_limit()}") + logger.info( + f"Total minimum balance: {gas_limit_accumulator.total_minimum_balance() / 10**18:.18f}" + ) + + def base_test_parametrizer(cls: Type[BaseTest]) -> Any: """ Generate pytest.fixture for a given BaseTest subclass. @@ -345,9 +562,16 @@ def base_test_parametrizer_func( pre: Alloc, eth_rpc: EthRPC, engine_rpc: EngineRPC | None, + dry_run: bool, collector: Collector, gas_benchmark_value: int, fixed_opcode_count: int | None, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + max_gas_limit_per_test: int | None, + gas_limit_accumulator: GasInfoAccumulator, ) -> Type[BaseTest]: """ Fixture used to instantiate an auto-fillable BaseTest object from @@ -392,6 +616,47 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: super(BaseTestWrapper, self).__init__(*args, **kwargs) self._request = request + execute = self.execute(execute_format=execute_format) + + # get balances of required sender accounts + required_balances = execute.get_required_sender_balances( + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + fork=fork, + ) + + minimum_balance, gas_consumption = ( + pre.minimum_balance_for_pending_transactions( + required_balances, + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + ) + ) + if max_gas_limit_per_test is not None: + assert gas_consumption <= max_gas_limit_per_test, ( + f"Test gas consumption ({gas_consumption}) exceeds the gas limit allowed " + f"per test({max_gas_limit_per_test})." + ) + + gas_limit_accumulator.add( + test_name=request.node.nodeid, + gas_limit=gas_consumption, + minimum_balance=minimum_balance, + ) + + if dry_run: + logger.info( + f"Minimum balance required: {minimum_balance / 10**18:.18f}" + ) + logger.info(f"Gas consumption: {gas_consumption}") + return + + # send the funds to the required sender accounts + pre.send_pending_transactions() # wait for pre-requisite transactions to be included in blocks pre.wait_for_transactions() @@ -411,7 +676,6 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: [str(eoa) for eoa in pre._funded_eoa] ) - execute = self.execute(execute_format=execute_format) execute.execute( fork=fork, eth_rpc=eth_rpc, diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/pre_alloc.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/pre_alloc.py index 3a3c64e378a..516ca1472b6 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/pre_alloc.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/pre_alloc.py @@ -14,6 +14,8 @@ Address, Bytes, EthereumTestRootModel, + Hash, + HexNumber, Number, Storage, StorageRootType, @@ -25,6 +27,7 @@ NumberConvertible, ) from execution_testing.forks import Fork +from execution_testing.logging import get_logger from execution_testing.rpc import EthRPC from execution_testing.rpc.rpc_types import TransactionByHashResponse from execution_testing.test_types import ( @@ -42,6 +45,8 @@ MAX_BYTECODE_SIZE = 24576 MAX_INITCODE_SIZE = MAX_BYTECODE_SIZE * 2 +logger = get_logger(__name__) + class AddressStubs(EthereumTestRootModel[Dict[str, Address]]): """ @@ -112,14 +117,6 @@ def pytest_addoption(parser: pytest.Parser) -> None: choices=list(EVMCodeType), help="Type of EVM code to deploy in each test by default.", ) - pre_alloc_group.addoption( - "--eoa-fund-amount-default", - action="store", - dest="eoa_fund_amount_default", - default=10**17, - type=int, - help="The default amount of wei to fund each EOA in each test with.", - ) pre_alloc_group.addoption( "--skip-cleanup", action="store_true", @@ -150,13 +147,25 @@ def address_stubs( If the address stubs are not supported by the subcommand, return None. """ - return request.config.getoption("address_stubs", None) + address_stubs = request.config.getoption("address_stubs", None) + if address_stubs is not None: + logger.info( + f"Using address stubs with {len(address_stubs.root)} entries" + ) + else: + logger.debug("No address stubs configured") + return address_stubs @pytest.fixture(scope="session") def skip_cleanup(request: pytest.FixtureRequest) -> bool: """Return whether to skip cleanup phase after each test.""" - return request.config.getoption("skip_cleanup") + skip = request.config.getoption("skip_cleanup") + if skip: + logger.info("Cleanup phase will be skipped after each test") + else: + logger.debug("Cleanup phase enabled after each test") + return skip @pytest.fixture(scope="session") @@ -164,16 +173,29 @@ def eoa_iterator(request: pytest.FixtureRequest) -> Iterator[EOA]: """Return an iterator that generates EOAs.""" eoa_start = request.config.getoption("eoa_iterator_start") print(f"Starting EOA index: {hex(eoa_start)}") + logger.info( + f"Initializing EOA iterator with start index: {hex(eoa_start)}" + ) return iter(EOA(key=i, nonce=0) for i in count(start=eoa_start)) +class PendingTransaction(Transaction): + """ + Custom transaction class that defines a transaction that is yet to be sent. + The value is allowed to be `None` to allow for the value to be set until the + transaction is sent. + """ + + value: HexNumber | None = None # type: ignore + + class Alloc(BaseAlloc): """A custom class that inherits from the original Alloc class.""" _fork: Fork = PrivateAttr() _sender: EOA = PrivateAttr() _eth_rpc: EthRPC = PrivateAttr() - _txs: List[Transaction] = PrivateAttr(default_factory=list) + _pending_txs: List[PendingTransaction] = PrivateAttr(default_factory=list) _deployed_contracts: List[Tuple[Address, Bytes]] = PrivateAttr( default_factory=list ) @@ -191,7 +213,6 @@ def __init__( eth_rpc: EthRPC, eoa_iterator: Iterator[EOA], chain_id: int, - eoa_fund_amount_default: int, evm_code_type: EVMCodeType | None = None, node_id: str = "", address_stubs: AddressStubs | None = None, @@ -205,25 +226,9 @@ def __init__( self._eoa_iterator = eoa_iterator self._evm_code_type = evm_code_type self._chain_id = chain_id - self._eoa_fund_amount_default = eoa_fund_amount_default self._node_id = node_id self._address_stubs = address_stubs or AddressStubs(root={}) - # always refresh _sender nonce from RPC ("pending") before building tx - def _refresh_sender_nonce(self) -> None: - """ - Synchronize self._sender.nonce with the node's view. - Prefer 'pending' to account for in-flight transactions. - """ - try: - rpc_nonce = self._eth_rpc.get_transaction_count( - self._sender, block_number="pending" - ) - except TypeError: - # If EthRPC.get_transaction_count has no 'block' kwarg - rpc_nonce = self._eth_rpc.get_transaction_count(self._sender) - self._sender.nonce = Number(rpc_nonce) - def __setitem__( self, address: Address | FixedSizeBytesConvertible, @@ -276,6 +281,10 @@ def deploy_contract( f"Stub name {stub} not found in address stubs" ) contract_address = self._address_stubs[stub] + logger.info( + f"Using address stub '{stub}' at {contract_address} " + f"(label={label})" + ) code = self._eth_rpc.get_code(contract_address) if code == b"": raise ValueError( @@ -283,6 +292,10 @@ def deploy_contract( ) balance = self._eth_rpc.get_balance(contract_address) nonce = self._eth_rpc.get_transaction_count(contract_address) + logger.debug( + f"Stub contract {contract_address}: balance={balance / 10**18:.18f} ETH, " + f"nonce={nonce}, code_size={len(code)} bytes" + ) super().__setitem__( contract_address, Account( @@ -344,28 +357,34 @@ def deploy_contract( # Limit the gas limit deploy_gas_limit = min(deploy_gas_limit * 2, 30_000_000) - print(f"Deploying contract with gas limit: {deploy_gas_limit}") - - self._refresh_sender_nonce() - deploy_tx = Transaction( + deploy_tx = PendingTransaction( sender=self._sender, to=None, data=initcode, value=balance, gas_limit=deploy_gas_limit, - ).with_signature_and_sender() + ) deploy_tx.metadata = TransactionTestMetadata( test_id=self._node_id, phase="setup", action="deploy_contract", target=label, - tx_index=len(self._txs), + tx_index=len(self._pending_txs), + ) + self._pending_txs.append(deploy_tx) + logger.info( + f"Contract deployment tx created (label={label}): " + f"tx_nonce={deploy_tx.nonce}, gas_limit={deploy_gas_limit}, " + f"code_size={len(code)} bytes, initcode_size={len(initcode)} bytes, " + f"balance={Number(balance) / 10**18:.18f} ETH, storage_slots={len(storage.root)}" ) - self._eth_rpc.send_transaction(deploy_tx) - self._txs.append(deploy_tx) contract_address = deploy_tx.created_contract + logger.debug( + f"Contract will be deployed at {contract_address} " + f"(label={label}, tx_index={len(self._pending_txs) - 1})" + ) self._deployed_contracts.append((contract_address, Bytes(code))) assert Number(nonce) >= 1, ( @@ -400,13 +419,22 @@ def fund_eoa( assert nonce is None, "nonce parameter is not supported for execute" eoa = next(self._eoa_iterator) eoa.label = label + amount_str = ( + f"{Number(amount) / 10**18:.18f} ETH" + if amount is not None + else "Deferred" + ) + logger.debug( + f"Funding EOA {eoa} (label={label}): amount={amount_str}, " + f"delegation={delegation}, storage={storage is not None}" + ) # Send a transaction to fund the EOA - if amount is None: - amount = self._eoa_fund_amount_default - - fund_tx: Transaction | None = None + fund_tx: PendingTransaction | None = None if delegation is not None or storage is not None: if storage is not None: + logger.debug( + f"Deploying storage contract for EOA {eoa} with {len(storage.root)} storage slots" + ) sstore_address = self.deploy_contract( code=( sum( @@ -416,10 +444,11 @@ def fund_eoa( + Op.STOP ) ) + logger.debug( + f"Storage contract deployed at {sstore_address} for EOA {eoa}" + ) - self._refresh_sender_nonce() - - set_storage_tx = Transaction( + set_storage_tx = PendingTransaction( sender=self._sender, to=eoa, authorization_list=[ @@ -431,19 +460,16 @@ def fund_eoa( ), ], gas_limit=100_000, - ).with_signature_and_sender() + ) eoa.nonce = Number(eoa.nonce + 1) set_storage_tx.metadata = TransactionTestMetadata( test_id=self._node_id, phase="setup", action="eoa_storage_set", target=label, - tx_index=len(self._txs), + tx_index=len(self._pending_txs), ) - self._eth_rpc.send_transaction(set_storage_tx) - self._txs.append(set_storage_tx) - - self._refresh_sender_nonce() + self._pending_txs.append(set_storage_tx) if delegation is not None: if ( @@ -453,7 +479,7 @@ def fund_eoa( delegation = eoa # TODO: This tx has side-effects on the EOA state because of # the delegation - fund_tx = Transaction( + fund_tx = PendingTransaction( sender=self._sender, to=eoa, value=amount, @@ -466,10 +492,10 @@ def fund_eoa( ), ], gas_limit=100_000, - ).with_signature_and_sender() + ) eoa.nonce = Number(eoa.nonce + 1) else: - fund_tx = Transaction( + fund_tx = PendingTransaction( sender=self._sender, to=eoa, value=amount, @@ -483,18 +509,16 @@ def fund_eoa( ), ], gas_limit=100_000, - ).with_signature_and_sender() + ) eoa.nonce = Number(eoa.nonce + 1) else: - if Number(amount) > 0: - self._refresh_sender_nonce() - - fund_tx = Transaction( + if amount is None or Number(amount) > 0: + fund_tx = PendingTransaction( sender=self._sender, to=eoa, value=amount, - ).with_signature_and_sender() + ) if fund_tx is not None: fund_tx.metadata = TransactionTestMetadata( @@ -502,18 +526,31 @@ def fund_eoa( phase="setup", action="fund_eoa", target=label, - tx_index=len(self._txs), + tx_index=len(self._pending_txs), ) - self._eth_rpc.send_transaction(fund_tx) - self._txs.append(fund_tx) - super().__setitem__( - eoa, - Account( - nonce=eoa.nonce, - balance=amount, - ), - ) + self._pending_txs.append(fund_tx) + logger.info( + f"Added funding transaction for EOA {eoa} (label={label}): " + f"tx_nonce={fund_tx.nonce}, " + f"tx_index={len(self._pending_txs) - 1}" + ) + account_kwargs: Dict[str, Any] = { + "nonce": eoa.nonce, + } + if amount is not None: + account_kwargs["balance"] = amount + account = Account(**account_kwargs) + super().__setitem__(eoa, account) self._funded_eoa.append(eoa) + balance_str = ( + f"{Number(amount) / 10**18:.18f} ETH" + if amount is not None + else "Deferred" + ) + logger.info( + f"EOA {eoa} funding tx created (label={label}):" + f"tx_nonce={eoa.nonce}, balance={balance_str}" + ) return eoa def fund_address( @@ -525,32 +562,40 @@ def fund_address( If the address is already present in the pre-alloc the amount will be added to its existing balance. """ - self._refresh_sender_nonce() - - fund_tx = Transaction( + logger.debug( + f"Funding address {address} (label={address.label}): " + f"{Number(amount) / 10**18:.18f} ETH" + ) + fund_tx = PendingTransaction( sender=self._sender, to=address, value=amount, - ).with_signature_and_sender() + ) fund_tx.metadata = TransactionTestMetadata( test_id=self._node_id, phase="setup", action="fund_address", target=address.label, - tx_index=len(self._txs), + tx_index=len(self._pending_txs), ) - self._eth_rpc.send_transaction(fund_tx) - self._txs.append(fund_tx) + self._pending_txs.append(fund_tx) if address in self: account = self[address] if account is not None: current_balance = account.balance or 0 - account.balance = ZeroPaddedHexNumber( - current_balance + Number(amount) + new_balance = current_balance + Number(amount) + account.balance = ZeroPaddedHexNumber(new_balance) + logger.debug( + f"Updated balance for existing address {address}: " + f"{current_balance / 10**18:.18f} ETH -> {new_balance / 10**18:.18f} ETH" ) return super().__setitem__(address, Account(balance=amount)) + logger.info( + f"Address {address} funding tx created (label={address.label}): " + f"{Number(amount) / 10**18:.18f} ETH" + ) def empty_account(self) -> Address: """ @@ -572,6 +617,7 @@ def empty_account(self) -> Address: """ eoa = next(self._eoa_iterator) + logger.debug(f"Creating empty account at {eoa}") super().__setitem__( eoa, @@ -582,9 +628,69 @@ def empty_account(self) -> Address: ) return Address(eoa) + def minimum_balance_for_pending_transactions( + self, + sender_balances: Dict[Address, int], + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + ) -> Tuple[int, int]: + """ + Calculate the minimum balance required by the sender to send all pending + transactions. + """ + minimum_balance = 0 + gas_consumption = 0 + for tx in self._pending_txs: + if tx.value is None: + # WARN: This currently fails if there's an account with `pre.fund_eoa()` that + # never sends a transaction during the test. + assert tx.to in sender_balances, ( + "Sender balance must be set before sending" + ) + sender_balance = sender_balances[tx.to] + logger.info( + f"Deferred EOA balance for {tx.to} set to {sender_balance / 10**18:.18f} ETH" + ) + tx.value = HexNumber(sender_balance) + tx.set_gas_price( + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + ) + gas_consumption += tx.gas_limit + minimum_balance += tx.signer_minimum_balance(fork=self._fork) + return minimum_balance + gas_consumption * gas_price, gas_consumption + + def send_pending_transactions(self) -> List[Hash]: + """Send all pending transactions.""" + logger.info( + f"Sending {len(self._pending_txs)} pending transactions " + f"(deployed_contracts={len(self._deployed_contracts)}, " + f"funded_eoas={len(self._funded_eoa)})" + ) + txs = [tx.with_signature_and_sender() for tx in self._pending_txs] + tx_hashes = self._eth_rpc.send_transactions(txs) + logger.info( + f"Sent {len(tx_hashes)} transactions: {[str(h) for h in tx_hashes[:5]]}" + + (f" and {len(tx_hashes) - 5} more" if len(tx_hashes) > 5 else "") + ) + return tx_hashes + def wait_for_transactions(self) -> List[TransactionByHashResponse]: """Wait for all transactions to be included in blocks.""" - return self._eth_rpc.wait_for_transactions(self._txs) + logger.info( + f"Waiting for {len(self._pending_txs)} transactions to be included in blocks" + ) + for tx in self._pending_txs: + assert tx.value is not None, ( + "Transaction value must be set before waiting for it to be included in a block" + ) + responses = self._eth_rpc.wait_for_transactions(self._pending_txs) + logger.info(f"All {len(responses)} transactions confirmed in blocks") + return responses @pytest.fixture(autouse=True) @@ -595,28 +701,25 @@ def evm_code_type(request: pytest.FixtureRequest) -> EVMCodeType: assert type(parameter_evm_code_type) is EVMCodeType, ( "Invalid EVM code type" ) + logger.info(f"Using EVM code type: {parameter_evm_code_type}") return parameter_evm_code_type + logger.debug(f"Using default EVM code type: {EVMCodeType.LEGACY}") return EVMCodeType.LEGACY -@pytest.fixture(scope="session") -def eoa_fund_amount_default(request: pytest.FixtureRequest) -> int: - """Get the gas price for the funding transactions.""" - return request.config.option.eoa_fund_amount_default - - @pytest.fixture(autouse=True, scope="function") def pre( fork: Fork, - sender_key: EOA, + worker_key: EOA, eoa_iterator: Iterator[EOA], eth_rpc: EthRPC, evm_code_type: EVMCodeType, chain_config: ChainConfig, - eoa_fund_amount_default: int, - default_gas_price: int, address_stubs: AddressStubs | None, skip_cleanup: bool, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + dry_run: bool, request: pytest.FixtureRequest, ) -> Generator[Alloc, None, None]: """Return default pre allocation for all tests (Empty alloc).""" @@ -626,18 +729,19 @@ def pre( assert hasattr(request.node, "fork") actual_fork = request.node.fork - # Record the starting balance of the sender - sender_test_starting_balance = eth_rpc.get_balance(sender_key) - # Prepare the pre-alloc + logger.debug( + f"Initializing pre-alloc for test {request.node.nodeid} " + f"(fork={actual_fork}, chain_id={chain_config.chain_id}, " + f"evm_code_type={evm_code_type})" + ) pre = Alloc( - fork=fork, - sender=sender_key, + fork=actual_fork, + sender=worker_key, eth_rpc=eth_rpc, eoa_iterator=eoa_iterator, evm_code_type=evm_code_type, chain_id=chain_config.chain_id, - eoa_fund_amount_default=eoa_fund_amount_default, node_id=request.node.nodeid, address_stubs=address_stubs, ) @@ -645,34 +749,83 @@ def pre( # Yield the pre-alloc for usage during the test yield pre - if not skip_cleanup: - # Refund all EOAs (regardless of whether the test passed or failed) - refund_txs = [] - for idx, eoa in enumerate(pre._funded_eoa): - remaining_balance = eth_rpc.get_balance(eoa) - eoa.nonce = Number(eth_rpc.get_transaction_count(eoa)) - refund_gas_limit = 21_000 - tx_cost = refund_gas_limit * default_gas_price - if remaining_balance < tx_cost: - continue - refund_tx = Transaction( - sender=eoa, - to=sender_key, - gas_limit=21_000, - gas_price=default_gas_price, - value=remaining_balance - tx_cost, - ).with_signature_and_sender() - refund_tx.metadata = TransactionTestMetadata( - test_id=request.node.nodeid, - phase="cleanup", - action="refund_from_eoa", - target=eoa.label, - tx_index=idx, + if dry_run: + logger.debug("Dry run: skipping cleanup phase") + return + if skip_cleanup: + logger.info("Skipping cleanup phase as requested") + return + + # Refund all EOAs (regardless of whether the test passed or failed) + logger.info( + f"Starting cleanup phase: refunding {len(pre._funded_eoa)} funded EOAs" + ) + refund_txs = [] + skipped_refunds = 0 + error_refunds = 0 + for idx, eoa in enumerate(pre._funded_eoa): + remaining_balance = eth_rpc.get_balance(eoa) + eoa.nonce = Number(eth_rpc.get_transaction_count(eoa)) + refund_gas_limit = 21_000 + tx_cost = refund_gas_limit * max_fee_per_gas + if remaining_balance < tx_cost: + logger.debug( + f"Skipping refund for EOA {eoa} (label={eoa.label}): " + f"insufficient balance {remaining_balance / 10**18:.18f} ETH < " + f"transaction cost {tx_cost / 10**18:.18f} ETH" ) + skipped_refunds += 1 + continue + refund_value = remaining_balance - tx_cost + logger.debug( + f"Preparing refund transaction for EOA {eoa} (label={eoa.label}): " + f"{refund_value / 10**18:.18f} ETH (remaining: {remaining_balance / 10**18:.18f} ETH, " + f"cost: {tx_cost / 10**18:.18f} ETH)" + ) + refund_tx = Transaction( + sender=eoa, + to=worker_key, + gas_limit=21_000, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + value=refund_value, + ).with_signature_and_sender() + refund_tx.metadata = TransactionTestMetadata( + test_id=request.node.nodeid, + phase="cleanup", + action="refund_from_eoa", + target=eoa.label, + tx_index=idx, + ) + try: + logger.info( + f"Sending refund transaction for EOA {eoa}: {refund_tx.hash}" + ) + refund_tx_hash = eth_rpc.send_transaction(refund_tx) + logger.info(f"Refund transaction sent: {refund_tx_hash}") refund_txs.append(refund_tx) - eth_rpc.send_wait_transactions(refund_txs) - - # Record the ending balance of the sender - sender_test_ending_balance = eth_rpc.get_balance(sender_key) - used_balance = sender_test_starting_balance - sender_test_ending_balance - print(f"Used balance={used_balance / 10**18:.18f}") + except Exception as e: + eoa_key = eoa.key + logger.error( + f"Error sending refund transaction for EOA {eoa}: {e}." + ) + if eoa_key is not None: + logger.info( + f"Retrieve funds manually from EOA {eoa} " + f"using private key {eoa_key.hex()}." + ) + error_refunds += 1 + continue + if refund_txs: + logger.info( + f"Waiting for {len(refund_txs)} refund transactions " + f"({skipped_refunds} skipped due to insufficient balance, " + f"{error_refunds} errored)" + ) + eth_rpc.wait_for_transactions(refund_txs) + logger.info(f"All {len(refund_txs)} refund transactions confirmed") + else: + logger.info( + f"No refund transactions to send ({skipped_refunds} EOAs skipped " + f"due to insufficient balance, {error_refunds} errored)" + ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py index 73fd80daeca..0b2bf7a228e 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py @@ -5,7 +5,7 @@ import time from pathlib import Path -from typing import Any, Dict, Iterator, List +from typing import Any, Dict, Iterator, List, Sequence from filelock import FileLock from pydantic import RootModel @@ -13,7 +13,7 @@ from execution_testing.base_types import Address, Hash, HexNumber from execution_testing.forks import Fork -from execution_testing.rpc import EngineRPC +from execution_testing.rpc import EngineRPC, TransactionProtocol from execution_testing.rpc import EthRPC as BaseEthRPC from execution_testing.rpc.rpc_types import ( ForkchoiceState, @@ -21,7 +21,6 @@ PayloadStatusEnum, TransactionByHashResponse, ) -from execution_testing.test_types import Transaction from execution_testing.test_types.trie import keccak256 @@ -362,7 +361,7 @@ def generate_block(self: "ChainBuilderEthRPC") -> None: if tx_hash in self.pending_tx_hashes: self.pending_tx_hashes.remove(tx_hash) - def send_transaction(self, transaction: Transaction) -> Hash: + def send_transaction(self, transaction: TransactionProtocol) -> Hash: """`eth_sendRawTransaction`: Send a transaction to the client.""" returned_hash = super().send_transaction(transaction) with self.pending_tx_hashes: @@ -372,7 +371,7 @@ def send_transaction(self, transaction: Transaction) -> Hash: return returned_hash def wait_for_transaction( - self, transaction: Transaction + self, transaction: TransactionProtocol ) -> TransactionByHashResponse: """ Wait for a specific transaction to be included in a block. @@ -390,7 +389,7 @@ def wait_for_transaction( return self.wait_for_transactions([transaction])[0] def wait_for_transactions( - self, transactions: List[Transaction] + self, transactions: Sequence[TransactionProtocol] ) -> List[TransactionByHashResponse]: """ Wait for all transactions in the provided list to be included in a diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py index 969a1cc554f..9272138ca63 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py @@ -43,14 +43,14 @@ def pytest_addoption(parser: pytest.Parser) -> None: "Arguments defining the hive RPC client properties for the test.", ) hive_rpc_group.addoption( - "--sender-key-initial-balance", + "--seed-key-initial-balance", action="store", - dest="sender_key_initial_balance", + dest="seed_key_initial_balance", type=int, default=10**26, help=( - "Initial balance of each sender key. There is one sender key per worker process " - "(`-n` option)." + "Initial balance of the seed key. This balance will be " + "distributed equally between each pytest worker (`-n` option)." ), ) hive_rpc_group.addoption( @@ -70,9 +70,12 @@ def pytest_configure(config: pytest.Config) -> None: # noqa: D103 @pytest.fixture(scope="session") -def seed_sender(session_temp_folder: Path) -> EOA: - """Determine the seed sender account for the client's genesis.""" - base_name = "seed_sender" +def seed_key(session_temp_folder: Path) -> EOA: + """ + Determine the seed account, which is used to fund all worker accounts, + and place it in the client's genesis. + """ + base_name = "seed_key" base_file = session_temp_folder / base_name base_lock_file = session_temp_folder / f"{base_name}.lock" @@ -80,31 +83,24 @@ def seed_sender(session_temp_folder: Path) -> EOA: if base_file.exists(): with base_file.open("r") as f: seed_sender_key = Hash(f.read()) - seed_sender = EOA(key=seed_sender_key) + seed_key = EOA(key=seed_sender_key) else: - seed_sender = EOA(key=randint(0, 2**256)) + seed_key = EOA(key=randint(0, 2**256)) with base_file.open("w") as f: - f.write(str(seed_sender.key)) - return seed_sender + f.write(str(seed_key.key)) + return seed_key @pytest.fixture(scope="session") def base_pre( request: pytest.FixtureRequest, - seed_sender: EOA, - worker_count: int, + seed_key: EOA, ) -> Alloc: """Pre-allocation for the client's genesis.""" - sender_key_initial_balance = request.config.getoption( - "sender_key_initial_balance" - ) - return Alloc( - { - seed_sender: Account( - balance=(worker_count * sender_key_initial_balance) + 10**18 - ) - } + seed_key_initial_balance = request.config.getoption( + "seed_key_initial_balance" ) + return Alloc({seed_key: Account(balance=seed_key_initial_balance)}) @pytest.fixture(scope="session") diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote_seed_sender.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote_seed_sender.py index 79db2e3481d..9f26bcfa4cb 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote_seed_sender.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote_seed_sender.py @@ -1,11 +1,16 @@ """Seed sender on a remote execution client.""" +from typing import Generator + import pytest from execution_testing.base_types import Hash, Number +from execution_testing.logging import get_logger from execution_testing.rpc import EthRPC from execution_testing.test_types import EOA +logger = get_logger(__name__) + def pytest_addoption(parser: pytest.Parser) -> None: """Add command-line options to pytest.""" @@ -29,10 +34,23 @@ def pytest_addoption(parser: pytest.Parser) -> None: @pytest.fixture(scope="session") -def seed_sender(request: pytest.FixtureRequest, eth_rpc: EthRPC) -> EOA: - """Create seed sender account by checking its balance and nonce.""" +def seed_key( + request: pytest.FixtureRequest, eth_rpc: EthRPC +) -> Generator[EOA, None, None]: + """ + Get the seed key from the command flags and create the EOA account object + with the updated nonce value from the network. + """ rpc_seed_key = Hash(request.config.getoption("rpc_seed_key")) # check the nonce through the rpc client - seed_sender = EOA(key=rpc_seed_key) - seed_sender.nonce = Number(eth_rpc.get_transaction_count(seed_sender)) - return seed_sender + seed_key = EOA(key=rpc_seed_key) + seed_key.nonce = Number(eth_rpc.get_transaction_count(seed_key)) + + # Record the start balance of the worker key + start_balance = eth_rpc.get_balance(seed_key) + + yield seed_key + + final_balance = eth_rpc.get_balance(seed_key) + used_balance = start_balance - final_balance + logger.info(f"Seed used balance={used_balance / 10**18:.18f}") diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/sender.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/sender.py index d2f59e1f9bf..e1305047b37 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/sender.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/sender.py @@ -8,9 +8,12 @@ from pytest_metadata.plugin import metadata_key from execution_testing.base_types import Number, Wei +from execution_testing.logging import get_logger from execution_testing.rpc import EthRPC from execution_testing.test_types import EOA, Transaction +logger = get_logger(__name__) + def pytest_addoption(parser: pytest.Parser) -> None: """Add command-line options to pytest.""" @@ -36,7 +39,8 @@ def pytest_addoption(parser: pytest.Parser) -> None: type=Wei, default=None, help=( - "Gas price set for the funding transactions of each worker's sender key." # noqa: E501 + "Gas price set for the funding transactions of each worker's sender key. " + "Default=None (1.5x current network gas price)" ), ) @@ -54,14 +58,22 @@ def pytest_addoption(parser: pytest.Parser) -> None: @pytest.fixture(scope="session") def sender_funding_transactions_gas_price( - request: pytest.FixtureRequest, default_gas_price: int + request: pytest.FixtureRequest, + eth_rpc: EthRPC, ) -> int: """Get the gas price for the funding transactions.""" gas_price: int | None = ( request.config.option.sender_funding_transactions_gas_price ) if gas_price is None: - gas_price = default_gas_price + network_gas_price = eth_rpc.gas_price() + gas_price = int(network_gas_price * 1.5) + logger.info( + f"Using calculated gas price: {gas_price / 10**9:.9f} Gwei " + f"(1.5x network gas price: {network_gas_price / 10**9:.9f} Gwei)" + ) + else: + logger.info(f"Using specified gas price: {gas_price / 10**9:.9f} Gwei") assert gas_price > 0, "Gas price must be greater than 0" return gas_price @@ -69,31 +81,41 @@ def sender_funding_transactions_gas_price( @pytest.fixture(scope="session") def sender_fund_refund_gas_limit(request: pytest.FixtureRequest) -> int: """Get the gas limit of the funding transactions.""" - return request.config.option.sender_fund_refund_gas_limit + gas_limit = request.config.option.sender_fund_refund_gas_limit + logger.info(f"Using gas limit for funding transactions: {gas_limit}") + return gas_limit @pytest.fixture(scope="session") def seed_account_sweep_amount(request: pytest.FixtureRequest) -> int | None: """Get the seed account sweep amount.""" - return request.config.option.seed_account_sweep_amount + sweep_amount = request.config.option.seed_account_sweep_amount + if sweep_amount is not None: + logger.info( + f"Using specified seed account sweep amount: {sweep_amount / 10**18:.18f} ETH" + ) + else: + logger.info( + "No seed account sweep amount specified, will sweep entire balance" + ) + return sweep_amount @pytest.fixture(scope="session") -def sender_key_initial_balance( - request: pytest.FixtureRequest, - seed_sender: EOA, +def worker_key_funding_amount( + seed_key: EOA, eth_rpc: EthRPC, session_temp_folder: Path, worker_count: int, sender_funding_transactions_gas_price: int, sender_fund_refund_gas_limit: int, seed_account_sweep_amount: int | None, -) -> int: +) -> int | None: """ - Calculate the initial balance of each sender key. + Calculate the initial balance of each worker key. - The way to do this is to fetch the seed sender balance and divide it by the - number of workers. This way we can ensure that each sender key has the same + The way to do this is to fetch the seed key balance and divide it by the + number of workers. This way we can ensure that each worker key has the same initial balance. We also only do this once per session, because if we try to fetch the @@ -103,78 +125,127 @@ def sender_key_initial_balance( It's not really possible to calculate the transaction costs of each test that each worker is going to run, so we can't really calculate the initial balance of each sender key based on that. + + If we are not running tests in parallel, this method is skipped since + all tests will run from the seed account. """ - base_name = "sender_key_initial_balance" + if worker_count <= 1: + return None + + base_name = "worker_key_funding_amount" base_file = session_temp_folder / base_name base_lock_file = session_temp_folder / f"{base_name}.lock" with FileLock(base_lock_file): if base_file.exists(): - with base_file.open("r") as f: - sender_key_initial_balance = int(f.read()) - else: - if seed_account_sweep_amount is None: - seed_account_sweep_amount = eth_rpc.get_balance(seed_sender) - seed_sender_balance_per_worker = ( - seed_account_sweep_amount // worker_count - ) - assert seed_sender_balance_per_worker > 100, ( - "Seed sender balance too low" + # Some other worker already did this for us, use that value. + cached_amount = int(base_file.read_text()) + logger.info( + f"Using cached worker key funding amount: {cached_amount / 10**18:.18f} ETH" ) - # Subtract the cost of the transaction that is going to be sent to - # the seed sender - sender_key_initial_balance = seed_sender_balance_per_worker - ( - sender_fund_refund_gas_limit - * sender_funding_transactions_gas_price + return cached_amount + + logger.info("Calculating worker key funding amount") + available_amount = ( + seed_account_sweep_amount + if seed_account_sweep_amount is not None + # No sweep amount specified, sweep the entire balance. + else eth_rpc.get_balance(seed_key) + ) + amount_source = ( + "Specified sweep amount" + if seed_account_sweep_amount is not None + else "Seed account balance" + ) + logger.info( + f"{amount_source}: {available_amount / 10**18:.18f} ETH, " + f"distributing across {worker_count} workers" + ) + seed_sender_balance_per_worker = available_amount // worker_count + # Calculate the cost of the transaction to send the amount. + funding_tx_cost = ( + sender_fund_refund_gas_limit + * sender_funding_transactions_gas_price + ) + logger.info( + f"Funding transaction cost: {funding_tx_cost / 10**18:.18f} ETH " + f"(gas_limit={sender_fund_refund_gas_limit}, " + f"gas_price={sender_funding_transactions_gas_price / 10**9:.9f} Gwei)" + ) + # Subtract the cost of the transaction that is going to be sent to + # the seed sender + worker_key_funding_amount = ( + seed_sender_balance_per_worker - funding_tx_cost + ) + if worker_key_funding_amount <= 0: + logger.error( + f"{amount_source} is too low to distribute to {worker_count} workers. " + f"Available: {available_amount / 10**18:.6f} ETH, " + f"Funding cost: {funding_tx_cost / 10**18:.6f} ETH" ) - - # ensure sender has enough balance for eoa_fund_amount_default - eoa_fund_amount_default: int = ( - request.config.option.eoa_fund_amount_default - ) - # Reserve gas for funding tx (21000 gas * gas_price) - funding_tx_cost = ( - sender_fund_refund_gas_limit - * sender_funding_transactions_gas_price + raise AssertionError( + f""" + {amount_source} is too low to distribute to the + specified number of workers ({worker_count}). + {available_amount / 10**18:.6f} ETH, + when distributed across all workers, and subtracting the + funding transaction cost + ({funding_tx_cost / 10**18:.6f} ETH), results in a zero or + negative value. + """ ) - if ( - sender_key_initial_balance - < eoa_fund_amount_default + funding_tx_cost - ): - raise ValueError( - f"Seed sender balance too low for --eoa-fund-amount-default=" # noqa: E501 - f"{eoa_fund_amount_default}. " - f"Available balance per worker: {sender_key_initial_balance} wei " # noqa: E501 - f"({sender_key_initial_balance / 10**18:.6f} ETH). " - f"Required: {eoa_fund_amount_default + funding_tx_cost} wei " # noqa: E501 - f"({(eoa_fund_amount_default + funding_tx_cost) / 10**18:.6f} ETH). " # noqa: E501 - f"Either fund the seed account with more ETH or use " - f"--eoa-fund-amount-default={sender_key_initial_balance - funding_tx_cost} " # noqa: E501 - f"or lower." - ) - - with base_file.open("w") as f: - f.write(str(sender_key_initial_balance)) - return sender_key_initial_balance + logger.info( + f"Calculated worker key funding amount: {worker_key_funding_amount / 10**18:.18f} ETH " + f"({seed_sender_balance_per_worker / 10**18:.18f} ETH per worker - " + f"{funding_tx_cost / 10**18:.18f} ETH transaction cost)" + ) + # Write the value to the file for the rest of the workers to use. + base_file.write_text(str(worker_key_funding_amount)) + return worker_key_funding_amount @pytest.fixture(scope="session") -def sender_key( +def session_worker_key( request: pytest.FixtureRequest, - seed_sender: EOA, - sender_key_initial_balance: int, + seed_key: EOA, + worker_count: int, + worker_key_funding_amount: int | None, eoa_iterator: Iterator[EOA], eth_rpc: EthRPC, session_temp_folder: Path, sender_funding_transactions_gas_price: int, sender_fund_refund_gas_limit: int, + dry_run: bool, ) -> Generator[EOA, None, None]: """ - Get the sender keys for all tests. + Get the key for this worker in this session that will be the account + that funds all EOAs and contracts in the tests that this worker executes. - The seed sender is going to be shared among different processes, so we need - to lock it before we produce each funding transaction. + Each worker will have a different key, but coordination is required + because all worker keys come from the same seed key. + + If we are not running tests in parallel, this method simply returns the + seed account directly. """ + if worker_count <= 1: + logger.info("Not running tests in parallel, using seed key directly") + starting_balance = eth_rpc.get_balance(seed_key) + yield seed_key + + remaining_balance = eth_rpc.get_balance(seed_key) + used_balance = starting_balance - remaining_balance + logger.info( + f"Seed {seed_key} used balance: {used_balance / 10**18:.18f} ETH " + f"(remaining: {remaining_balance / 10**18:.18f} ETH)" + ) + request.config.stash[metadata_key]["Senders"][str(seed_key)] = ( + f"Used balance={used_balance / 10**18:.18f}" + ) + return None + + assert worker_key_funding_amount is not None, ( + "`worker_key_funding_amount` is None" + ) # For the seed sender we do need to keep track of the nonce because it is # shared among different processes, and there might not be a new block # produced between the transactions. @@ -183,32 +254,62 @@ def sender_key( seed_sender_nonce_file = session_temp_folder / seed_sender_nonce_file_name seed_sender_lock_file = session_temp_folder / seed_sender_lock_file_name - sender = next(eoa_iterator) + worker_key = next(eoa_iterator) + logger.info(f"Allocated worker key: {worker_key}") - # prepare funding transaction + # Prepare funding transaction for this specific worker. + # Each worker locks the next nonce by using a file lock to coordinate. with FileLock(seed_sender_lock_file): if seed_sender_nonce_file.exists(): with seed_sender_nonce_file.open("r") as f: - seed_sender.nonce = Number(f.read()) + seed_key.nonce = Number(f.read()) + logger.debug( + f"Loaded seed key nonce from file: {seed_key.nonce}" + ) + else: + logger.debug( + "No existing seed key nonce file, using current nonce" + ) fund_tx = Transaction( - sender=seed_sender, - to=sender, + sender=seed_key, + to=worker_key, gas_limit=sender_fund_refund_gas_limit, gas_price=sender_funding_transactions_gas_price, - value=sender_key_initial_balance, + value=worker_key_funding_amount, ).with_signature_and_sender() - eth_rpc.send_transaction(fund_tx) + logger.info( + f"Preparing funding transaction: {worker_key_funding_amount / 10**18:.18f} ETH " + f"from {seed_key} to {worker_key} (nonce={seed_key.nonce})" + ) + if not dry_run: + eth_rpc.send_transaction(fund_tx) + logger.info(f"Sent funding transaction: {fund_tx.hash}") + else: + logger.info("Dry run: skipping funding transaction send") with seed_sender_nonce_file.open("w") as f: - f.write(str(seed_sender.nonce)) - eth_rpc.wait_for_transaction(fund_tx) - - yield sender - - # refund seed sender - remaining_balance = eth_rpc.get_balance(sender) - sender.nonce = Number(eth_rpc.get_transaction_count(sender)) - used_balance = sender_key_initial_balance - remaining_balance - request.config.stash[metadata_key]["Senders"][str(sender)] = ( + f.write(str(seed_key.nonce)) + if not dry_run: + logger.info( + f"Waiting for funding transaction to be mined: {fund_tx.hash}" + ) + eth_rpc.wait_for_transaction(fund_tx) + logger.info(f"Funding transaction confirmed: {fund_tx.hash}") + + # Run all tests for this worker. + yield worker_key + + # All tests for this worker have completed, refund seed key. + logger.info( + f"All tests completed for worker {worker_key}, preparing refund" + ) + remaining_balance = eth_rpc.get_balance(worker_key) + worker_key.nonce = Number(eth_rpc.get_transaction_count(worker_key)) + used_balance = worker_key_funding_amount - remaining_balance + logger.info( + f"Worker {worker_key} used balance: {used_balance / 10**18:.18f} ETH " + f"(remaining: {remaining_balance / 10**18:.18f} ETH)" + ) + request.config.stash[metadata_key]["Senders"][str(worker_key)] = ( f"Used balance={used_balance / 10**18:.18f}" ) @@ -217,23 +318,78 @@ def sender_key( # any other transaction that might have been sent by the sender. refund_gas_price = sender_funding_transactions_gas_price * 2 tx_cost = refund_gas_limit * refund_gas_price + logger.debug( + f"Refund transaction cost: {tx_cost / 10**18:.18f} ETH " + f"(gas_limit={refund_gas_limit}, gas_price={refund_gas_price / 10**9:.9f} Gwei)" + ) if (remaining_balance - 1) < tx_cost: + logger.warning( + f"Insufficient balance for refund: {remaining_balance / 10**18:.18f} ETH < " + f"{tx_cost / 10**18:.18f} ETH (transaction cost). Skipping refund." + ) return # Update the nonce of the sender in case one of the pre-alloc transactions # failed - sender.nonce = Number(eth_rpc.get_transaction_count(sender)) + worker_key.nonce = Number(eth_rpc.get_transaction_count(worker_key)) + refund_value = remaining_balance - tx_cost - 1 + logger.info( + f"Preparing refund transaction: {refund_value / 10**18:.18f} ETH " + f"from {worker_key} to {seed_key} (nonce={worker_key.nonce})" + ) refund_tx = Transaction( - sender=sender, - to=seed_sender, + sender=worker_key, + to=seed_key, gas_limit=refund_gas_limit, gas_price=refund_gas_price, - value=remaining_balance - tx_cost - 1, + value=refund_value, ).with_signature_and_sender() + logger.info( + f"Sending and waiting for refund transaction: {refund_tx.hash}" + ) eth_rpc.send_wait_transaction(refund_tx) + logger.info(f"Refund transaction confirmed: {refund_tx.hash}") + + +@pytest.fixture(scope="function") +def worker_key( + eth_rpc: EthRPC, session_worker_key: EOA +) -> Generator[EOA, None, None]: + """Prepare the worker key for the current test.""" + logger.debug(f"Preparing worker key {session_worker_key} for test") + rpc_nonce = Number( + eth_rpc.get_transaction_count( + session_worker_key, block_number="pending" + ) + ) + if rpc_nonce != session_worker_key.nonce: + logger.info( + f"Worker key nonce mismatch: {session_worker_key.nonce} != {rpc_nonce}" + ) + logger.info(f"Updating worker key nonce to {rpc_nonce}") + session_worker_key.nonce = rpc_nonce + + # Record the start balance of the worker key + worker_key_start_balance = eth_rpc.get_balance(session_worker_key) + logger.debug( + f"Worker key start balance: {worker_key_start_balance / 10**18:.18f} ETH" + ) + + yield session_worker_key + + logger.debug( + f"Test completed, checking worker key {session_worker_key} balance" + ) + final_balance = eth_rpc.get_balance(session_worker_key) + used_balance = worker_key_start_balance - final_balance + logger.info( + f"Worker key {session_worker_key} used balance: {used_balance / 10**18:.18f} ETH " + f"(start: {worker_key_start_balance / 10**18:.18f} ETH, " + f"final: {final_balance / 10**18:.18f} ETH)" + ) def pytest_sessionstart(session: pytest.Session) -> None: diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute-hive.ini b/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute-hive.ini index 6871ce9538f..e019e4a7a18 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute-hive.ini +++ b/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute-hive.ini @@ -17,7 +17,7 @@ addopts = -p execution_testing.cli.pytest_commands.plugins.pytest_hive.pytest_hive -p execution_testing.cli.pytest_commands.plugins.help.help --tb short - --dist loadscope + --dist load --ignore tests/cancun/eip4844_blobs/point_evaluation_vectors/ --ignore tests/json_infra --ignore tests/evm_tools \ No newline at end of file diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute.ini b/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute.ini index 51ba2052d48..3b1408206bc 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute.ini +++ b/packages/testing/src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-execute.ini @@ -19,7 +19,7 @@ addopts = -p execution_testing.cli.pytest_commands.plugins.help.help -p execution_testing.cli.pytest_commands.plugins.custom_logging.plugin_logging --tb short - --dist loadscope + --dist load --ignore tests/cancun/eip4844_blobs/point_evaluation_vectors/ --ignore tests/json_infra --ignore tests/evm_tools \ No newline at end of file diff --git a/packages/testing/src/execution_testing/execution/base.py b/packages/testing/src/execution_testing/execution/base.py index 720e9e679a8..71aed551664 100644 --- a/packages/testing/src/execution_testing/execution/base.py +++ b/packages/testing/src/execution_testing/execution/base.py @@ -6,7 +6,7 @@ from pydantic import PlainSerializer, PlainValidator from pytest import FixtureRequest -from execution_testing.base_types import CamelModel +from execution_testing.base_types import Address, CamelModel from execution_testing.forks import Fork from execution_testing.rpc import EngineRPC, EthRPC @@ -32,6 +32,20 @@ def __pydantic_init_subclass__(cls, **kwargs: Any) -> None: # Register the new execute format BaseExecute.formats[cls.format_name] = cls + def get_required_sender_balances( + self, + *, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + fork: Fork, + ) -> Dict[Address, int]: + """Get the required sender balances.""" + raise Exception( + f"Method `get_required_sender_balances` not implemented for {self.format_name}" + ) + @abstractmethod def execute( self, diff --git a/packages/testing/src/execution_testing/execution/blob_transaction.py b/packages/testing/src/execution_testing/execution/blob_transaction.py index d0d1c9fcd6b..b3f2473b506 100644 --- a/packages/testing/src/execution_testing/execution/blob_transaction.py +++ b/packages/testing/src/execution_testing/execution/blob_transaction.py @@ -7,10 +7,10 @@ from execution_testing.base_types import Address, Hash from execution_testing.base_types.base_types import Bytes +from execution_testing.forks import Fork from execution_testing.logging import ( get_logger, ) -from execution_testing.forks import Fork from execution_testing.rpc import ( BlobAndProofV1, BlobAndProofV2, @@ -69,11 +69,35 @@ class BlobTransaction(BaseExecute): "Send blob transactions to the execution client and validate their availability via " "`engine_getBlobsV*`" ) - requires_engine_rpc: ClassVar[bool] = True txs: List[NetworkWrappedTransaction | Transaction] nonexisting_blob_hashes: List[Hash] | None = None + def get_required_sender_balances( + self, + *, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + fork: Fork, + ) -> Dict[Address, int]: + """Get the required sender balances.""" + balances: Dict[Address, int] = {} + for tx in self.txs: + sender = tx.sender + assert sender is not None, "Sender is None" + tx.set_gas_price( + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + ) + if sender not in balances: + balances[sender] = 0 + balances[sender] += tx.signer_minimum_balance(fork=fork) + return balances + def execute( self, fork: Fork, @@ -82,25 +106,19 @@ def execute( request: FixtureRequest, ) -> None: """Execute the format.""" - assert engine_rpc is not None, ( - "Engine RPC is required for this format." - ) versioned_hashes: Dict[Hash, BlobAndProofV1 | BlobAndProofV2] = {} sent_txs: List[Transaction] = [] for tx_index, tx in enumerate(self.txs): + tx = tx.with_signature_and_sender() + expected_hash = tx.hash + to_address = tx.to if isinstance(tx, NetworkWrappedTransaction): - tx.tx = tx.tx.with_signature_and_sender() sent_txs.append(tx.tx) - expected_hash = tx.tx.hash versioned_hashes.update( versioned_hashes_with_blobs_and_proofs(tx) ) - to_address = tx.tx.to else: - tx = tx.with_signature_and_sender() sent_txs.append(tx) - expected_hash = tx.hash - to_address = tx.to label = ( to_address.label if isinstance(to_address, Address) else None ) @@ -116,6 +134,13 @@ def execute( assert expected_hash == received_hash, ( f"Expected hash {expected_hash} does not match received hash {received_hash}." ) + + if engine_rpc is None: + logger.info( + "Engine RPC is not available, skipping getBlobsV* validation." + ) + return + version = fork.engine_get_blobs_version() assert version is not None, ( "Engine get blobs version is not supported by the fork." diff --git a/packages/testing/src/execution_testing/execution/transaction_post.py b/packages/testing/src/execution_testing/execution/transaction_post.py index 59c99249067..248d93928a2 100644 --- a/packages/testing/src/execution_testing/execution/transaction_post.py +++ b/packages/testing/src/execution_testing/execution/transaction_post.py @@ -1,6 +1,6 @@ """Simple transaction-send then post-check execution format.""" -from typing import ClassVar, List +from typing import ClassVar, Dict, List import pytest from pytest import FixtureRequest @@ -14,6 +14,7 @@ SendTransactionExceptionError, ) from execution_testing.test_types import ( + NetworkWrappedTransaction, Transaction, TransactionTestMetadata, ) @@ -43,6 +44,32 @@ class TransactionPost(BaseExecute): "Simple transaction sending, then post-check after all transactions are included" ) + def get_required_sender_balances( + self, + *, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + fork: Fork, + ) -> Dict[Address, int]: + """Get the required sender balances.""" + balances: Dict[Address, int] = {} + for block in self.blocks: + for tx in block: + sender = tx.sender + assert sender is not None, "Sender is None" + tx.set_gas_price( + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + ) + if sender not in balances: + balances[sender] = 0 + balances[sender] += tx.signer_minimum_balance(fork=fork) + return balances + def execute( self, fork: Fork, @@ -53,15 +80,18 @@ def execute( """Execute the format.""" del fork del engine_rpc - assert not any(tx.ty == 3 for block in self.blocks for tx in block), ( - "Transaction type 3 is not supported in execute mode." - ) + for block in self.blocks: + for tx in block: + if not isinstance(tx, NetworkWrappedTransaction): + assert tx.ty != 3, ( + "Unwrapped transaction type 3 is not supported in execute mode." + ) # Track transaction hashes for gas validation (benchmarking) all_tx_hashes = [] for block in self.blocks: - signed_txs = [] + signed_txs: List[Transaction] = [] for tx_index, tx in enumerate(block): # Add metadata tx = tx.with_signature_and_sender() diff --git a/packages/testing/src/execution_testing/rpc/__init__.py b/packages/testing/src/execution_testing/rpc/__init__.py index 888cdf3c633..0f9ac05fe26 100644 --- a/packages/testing/src/execution_testing/rpc/__init__.py +++ b/packages/testing/src/execution_testing/rpc/__init__.py @@ -17,6 +17,7 @@ EthConfigResponse, ForkConfig, ForkConfigBlobSchedule, + TransactionProtocol, ) __all__ = [ @@ -32,4 +33,5 @@ "ForkConfigBlobSchedule", "NetRPC", "SendTransactionExceptionError", + "TransactionProtocol", ] diff --git a/packages/testing/src/execution_testing/rpc/rpc.py b/packages/testing/src/execution_testing/rpc/rpc.py index 15a91df8f66..3838d64ad5c 100644 --- a/packages/testing/src/execution_testing/rpc/rpc.py +++ b/packages/testing/src/execution_testing/rpc/rpc.py @@ -7,7 +7,7 @@ import time from itertools import count from pprint import pprint -from typing import Any, ClassVar, Dict, List, Literal +from typing import Any, ClassVar, Dict, List, Literal, Sequence import requests from jwt import encode @@ -24,7 +24,6 @@ from execution_testing.logging import ( get_logger, ) -from execution_testing.test_types import Transaction from .rpc_types import ( EthConfigResponse, @@ -36,6 +35,7 @@ PayloadAttributes, PayloadStatus, TransactionByHashResponse, + TransactionProtocol, ) logger = get_logger(__name__) @@ -47,13 +47,13 @@ class SendTransactionExceptionError(Exception): Represent an exception that is raised when a transaction fails to be sent. """ - tx: Transaction | None = None + tx: TransactionProtocol | None = None tx_rlp: Bytes | None = None def __init__( self, *args: Any, - tx: Transaction | None = None, + tx: TransactionProtocol | None = None, tx_rlp: Bytes | None = None, ) -> None: """ @@ -201,6 +201,11 @@ class EthRPC(BaseRPC): transaction_wait_timeout: int = 60 poll_interval: float = 1.0 # how often to poll for tx inclusion + gas_information_stale_seconds: int + + _gas_information_cache: Dict[str, int] + _gas_information_cache_timestamp: Dict[str, float] + BlockNumberType = int | Literal["latest", "earliest", "pending"] def __init__( @@ -208,6 +213,7 @@ def __init__( *args: Any, transaction_wait_timeout: int = 60, poll_interval: float | None = None, + gas_information_stale_seconds: int = 12, **kwargs: Any, ) -> None: """ @@ -233,6 +239,17 @@ def __init__( self.poll_interval = 1.0 else: self.poll_interval = 1.0 + self.gas_information_stale_seconds = gas_information_stale_seconds + self._gas_information_cache = { + "gasPrice": 0, + "maxPriorityFeePerGas": 0, + "blobBaseFee": 0, + } + self._gas_information_cache_timestamp = { + "gasPrice": 0.0, + "maxPriorityFeePerGas": 0.0, + "blobBaseFee": 0.0, + } def config(self, timeout: int | None = None) -> EthConfigResponse | None: """ @@ -394,13 +411,40 @@ def get_storage_at( response = self.post_request(method="getStorageAt", params=params) return Hash(response) + def _get_gas_information( + self, + *, + method: Literal["gasPrice", "maxPriorityFeePerGas", "blobBaseFee"], + ) -> int: + """Get gas information from the cache or the RPC server.""" + if ( + time.time() - self._gas_information_cache_timestamp[method] + > self.gas_information_stale_seconds + ): + response = self.post_request(method=method) + logger.info(f"Requesting stale {method}") + self._gas_information_cache[method] = int(response, 16) + self._gas_information_cache_timestamp[method] = time.time() + return self._gas_information_cache[method] + def gas_price(self) -> int: """ `eth_gasPrice`: Returns the gas price. """ - logger.info("Requesting gas price") - response = self.post_request(method="gasPrice") - return int(response, 16) + return self._get_gas_information(method="gasPrice") + + def max_priority_fee_per_gas(self) -> int: + """ + `eth_maxPriorityFeePerGas`: Return the current max priority fee per + gas of the network. + """ + return self._get_gas_information(method="maxPriorityFeePerGas") + + def blob_base_fee(self) -> int: + """ + `eth_blobBaseFee`: Return the current blob base fee per gas of the network. + """ + return self._get_gas_information(method="blobBaseFee") def send_raw_transaction( self, transaction_rlp: Bytes, request_id: int | str | None = None @@ -422,9 +466,11 @@ def send_raw_transaction( str(e), tx_rlp=transaction_rlp ) from e - def send_transaction(self, transaction: Transaction) -> Hash: - """`eth_sendRawTransaction`: Send a transaction to the client.""" - # TODO: is this a copypaste error from above? + def send_transaction(self, transaction: TransactionProtocol) -> Hash: + """ + Convenience method to send a single transaction to the client via + `eth_sendRawTransaction`. + """ try: logger.info("Sending tx..") response = self.post_request( @@ -439,7 +485,9 @@ def send_transaction(self, transaction: Transaction) -> Hash: except Exception as e: raise SendTransactionExceptionError(str(e), tx=transaction) from e - def send_transactions(self, transactions: List[Transaction]) -> List[Hash]: + def send_transactions( + self, transactions: Sequence[TransactionProtocol] + ) -> List[Hash]: """ Use `eth_sendRawTransaction` to send a list of transactions to the client. @@ -463,7 +511,7 @@ def storage_at_keys( return results def wait_for_transaction( - self, transaction: Transaction + self, transaction: TransactionProtocol ) -> TransactionByHashResponse: """ Use `eth_getTransactionByHash` to wait until a transaction is included @@ -485,7 +533,7 @@ def wait_for_transaction( ) def wait_for_transactions( - self, transactions: List[Transaction] + self, transactions: Sequence[TransactionProtocol] ) -> List[TransactionByHashResponse]: """ Use `eth_getTransactionByHash` to wait until all transactions in list @@ -523,13 +571,13 @@ def wait_for_transactions( f"after {self.transaction_wait_timeout} seconds" ) - def send_wait_transaction(self, transaction: Transaction) -> Any: + def send_wait_transaction(self, transaction: TransactionProtocol) -> Any: """Send transaction and waits until it is included in a block.""" self.send_transaction(transaction) return self.wait_for_transaction(transaction) def send_wait_transactions( - self, transactions: List[Transaction] + self, transactions: Sequence[TransactionProtocol] ) -> List[Any]: """ Send list of transactions and waits until all of them are included in a diff --git a/packages/testing/src/execution_testing/rpc/rpc_types.py b/packages/testing/src/execution_testing/rpc/rpc_types.py index b0e2c50ef61..19ce78ef1da 100644 --- a/packages/testing/src/execution_testing/rpc/rpc_types.py +++ b/packages/testing/src/execution_testing/rpc/rpc_types.py @@ -4,7 +4,7 @@ from binascii import crc32 from enum import Enum from hashlib import sha256 -from typing import Annotated, Any, Dict, List, Self +from typing import Annotated, Any, Dict, List, Protocol, Self from pydantic import AliasChoices, Field, model_validator @@ -257,3 +257,24 @@ class EthConfigResponse(CamelModel): current: ForkConfig next: ForkConfig | None = None last: ForkConfig | None = None + + +class TransactionProtocol(Protocol): + """Protocol for a transaction that can be sent to the client.""" + + def rlp(self) -> Bytes: + """Return the RLP of the transaction.""" + ... + + @property + def hash(self) -> Hash: + """Return the hash of the transaction.""" + ... + + def metadata_string(self) -> str | None: + """Return the metadata field as a formatted json string or None.""" + ... + + def model_dump_json(self) -> str: + """Return the transaction as a json string.""" + ... diff --git a/packages/testing/src/execution_testing/test_types/transaction_types.py b/packages/testing/src/execution_testing/test_types/transaction_types.py index 77a0ba03bd1..ffa49e46e43 100644 --- a/packages/testing/src/execution_testing/test_types/transaction_types.py +++ b/packages/testing/src/execution_testing/test_types/transaction_types.py @@ -3,7 +3,7 @@ from dataclasses import dataclass from enum import IntEnum from functools import cached_property -from typing import Any, ClassVar, Dict, Generic, List, Literal, Sequence +from typing import Any, ClassVar, Dict, Generic, List, Literal, Self, Sequence import ethereum_rlp as eth_rlp from coincurve.keys import PrivateKey, PublicKey @@ -32,10 +32,11 @@ TestAddress, TestPrivateKey, ) +from execution_testing.exceptions import TransactionException +from execution_testing.forks import Fork from execution_testing.logging import ( get_logger, ) -from execution_testing.exceptions import TransactionException from .account_types import EOA from .blob_types import Blob @@ -61,8 +62,8 @@ class TransactionType(IntEnum): class TransactionDefaults: """Default values for transactions.""" - gas_price = 10 - max_fee_per_gas = 7 + gas_price: int = 10 + max_fee_per_gas: int = 7 max_priority_fee_per_gas: int = 0 @@ -391,6 +392,7 @@ def model_post_init(self, __context: Any) -> None: # Set default values for fields that are required for certain tx types if self.ty <= 1 and self.gas_price is None: self.gas_price = HexNumber(TransactionDefaults.gas_price) + self.model_fields_set.remove("gas_price") if self.ty >= 1 and self.access_list is None: self.access_list = [] if self.ty < 1: @@ -400,10 +402,12 @@ def model_post_init(self, __context: Any) -> None: self.max_fee_per_gas = HexNumber( TransactionDefaults.max_fee_per_gas ) + self.model_fields_set.remove("max_fee_per_gas") if self.ty >= 2 and self.max_priority_fee_per_gas is None: self.max_priority_fee_per_gas = HexNumber( TransactionDefaults.max_priority_fee_per_gas ) + self.model_fields_set.remove("max_priority_fee_per_gas") if self.ty < 2: assert self.max_fee_per_gas is None, "max_fee_per_gas must be None" assert self.max_priority_fee_per_gas is None, ( @@ -412,6 +416,7 @@ def model_post_init(self, __context: Any) -> None: if self.ty == 3 and self.max_fee_per_blob_gas is None: self.max_fee_per_blob_gas = HexNumber(1) + self.model_fields_set.remove("max_fee_per_blob_gas") if self.ty != 3: assert self.blob_versioned_hashes is None, ( "blob_versioned_hashes must be None" @@ -529,7 +534,7 @@ def sign(self: "Transaction") -> None: def with_signature_and_sender( self, *, keep_secret_key: bool = False - ) -> "Transaction": + ) -> Self: """Return signed version of the transaction using the private key.""" updated_values: Dict[str, Any] = {} @@ -586,7 +591,7 @@ def with_signature_and_sender( updated_values["secret_key"] = None - updated_tx: "Transaction" = self.model_copy(update=updated_values) + updated_tx = self.model_copy(update=updated_values) # Remove the secret key if requested if keep_secret_key: @@ -772,6 +777,56 @@ def created_contract(self) -> Address: ).keccak256() return Address(hash_bytes[-20:]) + def set_gas_price( + self, + *, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + ) -> None: + """ + Set the gas price to the appropriate values of the current + execution environment. + + Values are only set if they were not + set during instance creation. + """ + if self.ty <= 1: + if "gas_price" not in self.model_fields_set: + self.gas_price = HexNumber(gas_price) + else: + if "max_fee_per_gas" not in self.model_fields_set: + self.max_fee_per_gas = HexNumber(max_fee_per_gas) + if "max_priority_fee_per_gas" not in self.model_fields_set: + self.max_priority_fee_per_gas = HexNumber( + max_priority_fee_per_gas + ) + if self.ty == 3: + if "max_fee_per_blob_gas" not in self.model_fields_set: + self.max_fee_per_blob_gas = HexNumber(max_fee_per_blob_gas) + + def signer_minimum_balance(self, *, fork: Fork) -> int: + """Return minimum balance of the signer.""" + gas_price = self.gas_price or self.max_fee_per_gas + assert gas_price is not None, ( + "Impossible to calculate minimum balance without gas price" + ) + gas_limit = self.gas_limit + if self.ty == 3 and self.blob_versioned_hashes is not None: + max_fee_per_blob_gas = self.max_fee_per_blob_gas + assert max_fee_per_blob_gas is not None, ( + "Impossible to calculate minimum balance without max_fee_per_blob_gas" + ) + return ( + gas_price * gas_limit + + self.value + + max_fee_per_blob_gas + * (fork.blob_gas_per_blob() * len(self.blob_versioned_hashes)) + ) + else: + return gas_price * gas_limit + self.value + class NetworkWrappedTransaction(CamelModel, RLPSerializable): """ @@ -873,6 +928,41 @@ def get_rlp_fields(self) -> List[str]: return rlp_fields + @property + def hash(self) -> Hash: + """Return the hash of the transaction.""" + return self.tx.hash + + @property + def sender(self) -> EOA | None: + """Return the sender of the transaction.""" + return self.tx.sender + + @property + def to(self) -> Address | None: + """Return the to address of the transaction.""" + return self.tx.to + + def set_gas_price( + self, + *, + gas_price: int, + max_fee_per_gas: int, + max_priority_fee_per_gas: int, + max_fee_per_blob_gas: int, + ) -> None: + """Set the gas price to the appropriate values of the current execution environment.""" + self.tx.set_gas_price( + gas_price=gas_price, + max_fee_per_gas=max_fee_per_gas, + max_priority_fee_per_gas=max_priority_fee_per_gas, + max_fee_per_blob_gas=max_fee_per_blob_gas, + ) + + def signer_minimum_balance(self, *, fork: Fork) -> int: + """Return minimum balance of the signer.""" + return self.tx.signer_minimum_balance(fork=fork) + def get_rlp_prefix(self) -> bytes: """ Return the transaction type as bytes to be appended at the beginning of @@ -881,3 +971,15 @@ def get_rlp_prefix(self) -> bytes: if self.tx.ty > 0: return bytes([self.tx.ty]) return b"" + + def with_signature_and_sender( + self, *, keep_secret_key: bool = False + ) -> Self: + """Return a new transaction with the signature and sender added.""" + return self.model_copy( + update={ + "tx": self.tx.with_signature_and_sender( + keep_secret_key=keep_secret_key + ), + } + )