Skip to content

feat: display per-test client log sections (hive TestLogOffsets)#76

Merged
skylenet merged 8 commits into
ethpandaops:masterfrom
danceratopz:enginex-log-offsets
Jul 6, 2026
Merged

feat: display per-test client log sections (hive TestLogOffsets)#76
skylenet merged 8 commits into
ethpandaops:masterfrom
danceratopz:enginex-log-offsets

Conversation

@danceratopz

@danceratopz danceratopz commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Hive can reuse one client container across many tests (ethereum/hive#1402, currently used by eels/consume-enginex); each test's clientInfo[].logOffsets then gives the byte range of the shared client log belonging to that test. This PR surfaces those ranges — simulator-agnostic, keyed only off the presence of logOffsets/multiTestContext in the suite JSON.

Suite page

  • New Shared Client Sessions panel: one row per shared client instance (tests served, log size, lifecycle badge). The multiTestContext pseudo-test is hidden from the table and stats.
  • Expanded test rows gain a Client Log excerpt showing only that test's slice, next to the existing test-details excerpt. Excerpt headers now show absolute line numbers alongside byte offsets.

Log viewer

  • Per-test Logs links anchor the viewer on the test's byte range: only ±64 KB of context is fetched via HTTP Range requests, the section is highlighted and centred — render time is independent of log size (logs can reach hundreds of MB on full fixture runs). "Load earlier/later" expanders, a "Load full file" escape hatch, and absolute line numbers resolved in the background (checkpoint-cached per file).
  • New Tests in this log sidebar: every test section of the current log in log order, with pass/fail dots and a failures filter; clicking re-anchors the viewer. Useful for "what ran on this client before my failing test".

Notes

  • Old result files render as before; empty ranges (begin == end, client produced no output) are handled explicitly.
  • Servers without Range support degrade gracefully (the UI slices client-side / falls back to the full view). Results hosting must not gzip logs on the fly, or Range requests break.
  • Tested against a 5,593-test enginex run (116 shared geth containers, 6.5 MB logs) and a production discv4 suite for backward compatibility.

Local testing: My agent made this tarball (that includes results) so your agent should be able to run it locally, jlmk if you have any trouble!

hive-view_shared-client-logs-demo.mp4

Hive can now reuse client containers across many tests (ethereum/hive#1402)
and reports the byte range of the client log belonging to each test in
clientInfo[].logOffsets. Surface those ranges in the UI:

- types: TestLogOffsets, clientInfo[].logOffsets, multiTestContext,
  optional summaryResult fields
- hide multi-test context pseudo-tests from the results table and stats
- per-test Logs links anchor the viewer on the test's byte range
- expanded rows show a client-log excerpt (Range request, capped at
  128 KB) next to the existing test-details excerpt
- LogViewer: ?begin&end now loads the full file, maps byte offsets to
  lines (byte-accurate), highlights the section and scrolls to it
- byteRange utils defend against servers that ignore Range headers
When anchored on a test's byte range (?begin&end), fetch only the range
plus 64 KB of context via HTTP Range requests instead of the whole file,
so shared client logs of any size render instantly:

- expand-up/expand-down buttons load more context in 64 KB steps,
  preserving the scroll position when prepending
- absolute line numbers resolve in the background by streaming the
  window prefix and counting newlines (window-relative until then)
- 'Load full file' escape hatch switches to the full view
- servers without Range support fall back to the full view for free
Adds the inverse mapping of per-test log offsets — from a shared client
log back to the tests it contains:

- LogViewer sidebar listing every test section of the current log in
  log order, with pass/fail dots, a failures filter, and per-segment
  links to the test details; clicking a segment re-anchors the viewer
- suite page 'Shared Client Sessions' panel summarising the client
  instances owned by multi-test context entries (tests served, log
  size, lifecycle status), replacing the hidden pseudo test case
tanstack-virtual smooth scrolling does not reliably complete over long
distances on a freshly mounted virtualizer, leaving anchored views at
the top of the window instead of on the highlighted section.
Counted directly when the server returned the whole file, otherwise by
streaming the prefix in the background; skipped for excerpts starting
beyond 8 MB so expanding a row never triggers huge downloads.
- share one isRealTestCase predicate (5 inline multiTestContext filters)
  and one logViewerUrl builder (3 inline route constructions)
- move window chunk helpers (EOF detection, partial-line trims) into
  byteRange.ts, shared by initial load and both expand directions
- replace per-request prefix streaming with checkpoint-cached, bounded
  line-number resolution (resolveLineNumber); navigating between tests
  in the same log now only streams the gap since the last count
- drop the HEAD pre-flight (Content-Range already carries the size),
  LogViewer's duplicate byte formatter, derivable lineCount/truncated
  state, dead resets, and the stale-result ref machinery
- VirtualizedLogContent takes prependedLines (its own units) instead of
  a pixel scroll adjustment; line-height knowledge stays internal
- suite JSON query marked immutable (staleTime: Infinity); themed CSS
  variables replace hardcoded border/text colors in the new components
Production results record logOffsets with begin == end when a client
produced no output during a test; requesting that range can even yield
416 at EOF. fetchByteRange now short-circuits empty ranges and expanded
rows say so instead of rendering an empty excerpt.

Scroll-to-index timeouts in the virtualized views are cleared on re-run
so a pending scroll never fires against an outdated index, and sidebar
rows guard against virtual items that momentarily outlive a shrinking
filtered list.
@skylenet
skylenet merged commit 6d085e2 into ethpandaops:master Jul 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants