Description
Execute hive mode works well currrently, but can be improved in the future primarily to aid test fail debugging.
Following the comments on this hive PR: ethereum/hive#1291
About the -m get_blobs...
Here's just one other small bit of relevant info... and that's that we only parametrize a simulator with the format if it's supported by the particular simulator (_supported_fixture_formats is defined in the simulator's local conftest.py):
https://github.com/ethereum/execution-spec-tests/blob/020f728eefedc75a68e6fba6dbc65322e1d9f490/src/pytest_plugins/consume/consume.py#L411-L423
This avoids having to add these command-line args, which is beneficial if we want to allow the user to specify other markers (as in ethereum/hive#1248).
Future improvements summarized below from my understanding:
- Remove the marker usage -m from the dockerfile.
- Replace this directly within EEST where we specify the test types we want to execute hive mode to run.
- Run execute hive mode using pytests session level so we can easily filter tests within the hive ui like we do for consume!
To achieve this we must be able to run multiple tests against a single client instance within hive. This is currently a work in progress by @danceratopz for consume engine to allow us to run all of the engine tests significantly faster. Once the latter is completed we can move forward with the execute hive mode improvements.
Description
Execute hive mode works well currrently, but can be improved in the future primarily to aid test fail debugging.
Following the comments on this hive PR: ethereum/hive#1291
To achieve this we must be able to run multiple tests against a single client instance within hive. This is currently a work in progress by @danceratopz for
consume engineto allow us to run all of the engine tests significantly faster. Once the latter is completed we can move forward with the execute hive mode improvements.