feat(execute): report individual test results in hive UI#2446
feat(execute): report individual test results in hive UI#2446spencer-tb wants to merge 1 commit into
Conversation
Add per-test Hive test records so each execute test appears individually in hiveview with its own pass/fail status, instead of aggregating all results into a single entry. Closes ethereum#1550
731a453 to
aa28d01
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #2446 +/- ##
================================================
Coverage 85.97% 85.97%
================================================
Files 599 599
Lines 36916 36916
Branches 3771 3771
================================================
Hits 31738 31738
Misses 4560 4560
Partials 618 618
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
danceratopz
left a comment
There was a problem hiding this comment.
This does a duplication of the troublesome (#1534) and ugly code from pytest_hive:
https://github.com/spencer-tb/execution-specs/blob/aa28d01577c94c1802d2ff7c8a03070485da6e42/packages/testing/src/execution_testing/cli/pytest_commands/plugins/pytest_hive/pytest_hive.py#L250-L373
We should strive to keep this single source, or as much as possible.
I'm not sure whether using a multi-test client is overkill here, cf https://github.com/danceratopz/execution-specs/blob/82a06a33e8a52059064241a14a589bcb7a0a25e8/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py, but I think we should align it for improved log reporting, we're so close! See ethereum/hive#1402
|
Closing! :) |
🗒️ Description
Add per-test Hive test records so each execute test appears individually in hiveview with its own pass/fail status, instead of aggregating all results into a single entry.
Reproduce
Hiveview shows 1 entry: "Base Hive Test", all tests aggregated into a single pass/fail.
Hiveview shows 10 entries: 1 "Client Deployment" + 9 individual test cases, each with their own pass/fail status.
Open http://localhost:8080
🔗 Related Issues or PRs
Closes #1550
✅ Checklist
toxchecks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx tox -e statictype(scope):.