Skip to content

feat(execute): report individual test results in hive UI#2446

Closed
spencer-tb wants to merge 1 commit into
ethereum:forks/amsterdamfrom
spencer-tb:feat/execute-hive-mode-improvements
Closed

feat(execute): report individual test results in hive UI#2446
spencer-tb wants to merge 1 commit into
ethereum:forks/amsterdamfrom
spencer-tb:feat/execute-hive-mode-improvements

Conversation

@spencer-tb

@spencer-tb spencer-tb commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

🗒️ Description

Add per-test Hive test records so each execute test appears individually in hiveview with its own pass/fail status, instead of aggregating all results into a single entry.

Screenshot 2026-03-09 at 11 14 04

Reproduce

  1. Start hive in dev mode:
./hive --dev --client go-ethereum
  1. Run execute hive without this PR (single aggregated result):
export HIVE_SIMULATOR=http://127.0.0.1:3000
uv run execute hive -m transaction_test --fork Osaka -k "test_empty_authorization_list or test_invalid_auth_signature" -s

Hiveview shows 1 entry: "Base Hive Test", all tests aggregated into a single pass/fail.

  1. Run execute hive with this PR (individual results):
export HIVE_SIMULATOR=http://127.0.0.1:3000
uv run execute hive -m transaction_test --fork Osaka -k "test_empty_authorization_list or test_invalid_auth_signature" -s

Hiveview shows 10 entries: 1 "Client Deployment" + 9 individual test cases, each with their own pass/fail status.

  1. View results:
./hiveview --serve --logdir workspace/logs

Open http://localhost:8080

Note: hiveview in dev mode requires ethereum/hive#1400 to display results.

🔗 Related Issues or PRs

Closes #1550

✅ Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx tox -e static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).

Add per-test Hive test records so each execute test appears
individually in hiveview with its own pass/fail status, instead
of aggregating all results into a single entry.

Closes ethereum#1550
@spencer-tb
spencer-tb force-pushed the feat/execute-hive-mode-improvements branch from 731a453 to aa28d01 Compare March 9, 2026 10:36
@codecov

codecov Bot commented Mar 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.97%. Comparing base (68fae57) to head (aa28d01).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2446   +/-   ##
================================================
  Coverage            85.97%   85.97%           
================================================
  Files                  599      599           
  Lines                36916    36916           
  Branches              3771     3771           
================================================
  Hits                 31738    31738           
  Misses                4560     4560           
  Partials               618      618           
Flag Coverage Δ
unittests 85.97% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@danceratopz danceratopz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does a duplication of the troublesome (#1534) and ugly code from pytest_hive:
https://github.com/spencer-tb/execution-specs/blob/aa28d01577c94c1802d2ff7c8a03070485da6e42/packages/testing/src/execution_testing/cli/pytest_commands/plugins/pytest_hive/pytest_hive.py#L250-L373

We should strive to keep this single source, or as much as possible.

I'm not sure whether using a multi-test client is overkill here, cf https://github.com/danceratopz/execution-specs/blob/82a06a33e8a52059064241a14a589bcb7a0a25e8/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py, but I think we should align it for improved log reporting, we're so close! See ethereum/hive#1402

@spencer-tb

Copy link
Copy Markdown
Contributor Author

Closing! :)

@spencer-tb spencer-tb closed this Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(execute): improve execute hive mode markers and session scope

2 participants