Skip to content

πŸ› fix(logread): a cursor into a file with a first line over 64 KiB no longer expires - #16901

Merged
hivecommons-hive[bot] merged 1 commit into
hivecommons:v6from
Danathar:fix/logread-cursor-long-first-line
Nov 6, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
hivecommons:v6from
Danathar:fix/logread-cursor-long-first-line

Conversation

@Danathar

@Danathar Danathar commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Summary

log_read pages backwards through a hive's logs with a cursor. For a log file whose first line is longer than 64 KiB, the cursor never worked: the second page always came back cursor_expired, so only the newest page of that file could be read. This PR makes the cursor work for such files. Files with ordinary first lines were never affected.

What went wrong

A cursor names a file by a digest of its first line, so it survives the file being rotated and compressed between two calls.

  • When a page is returned, the digest was taken over the whole first line.
  • When the cursor is looked up, only the first 64 KiB of each file is read. With no newline in that much, the file got no identity at all.

For a first line longer than 64 KiB the two never match:

first line 60 KiB:  page 2 -> ok, 100 lines
first line 70 KiB:  page 2 -> cursor_expired, "the history file this cursor points into is no longer retained"

It happens in hive.log and its rotated history when a file opens with a very long record. It happens more easily in the proxy, litellm and hive-output sources: a line that never ends is stored in 64 KiB pieces, and each piece is longer than 64 KiB once wrapped as a record, so any file that opens with one is affected.

What changes

The identity of a file is the digest of at most the first 64 KiB of its first line, computed the same way on both sides. Cursors issued before this change for files with ordinary first lines stay valid, because their digest is unchanged.

Files

  • src/crux/logread/files.go: headOf.
  • src/crux/logread/cursor_long_first_line_test.go: new.
  • src/docs/design/admin-mcp.md: one sentence under "Paging".
  • changelog.d/fixed-logread-cursor-long-first-line.md

Tests

  • TestCursorIntoAFileWithALongFirstLine: six first-line sizes around the 64 KiB boundary.
  • TestCursorIntoACompressedFileWithALongFirstLine
  • TestCursorIntoAnOutputSourceThatOpensWithARunawayLine
  • TestPagingThroughAFileWithALongFirstLineReturnsEveryLineOnce
  • TestHeadOf
  • All five fail on v6 without the change. The whole ./crux/logread/ package passes with it.

πŸ€– Generated with Claude Code

… longer expires

A log file is identified by a digest of its first line. headOf returned
the whole line when it was cut from a loaded region, and nothing when the
line had no newline inside the 64 KiB that a cursor lookup reads. The two
digests never matched for a first line longer than 64 KiB, so every cursor
into that file answered cursor_expired and only its newest page could be
read.

headOf now returns at most the first 64 KiB of the line either way.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Danathar <Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 6, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v6

Conflicts are checked with every PR merged onto the current tip of v6, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#16884 ✨ feat(adminmcp): read and act on the Activity monitor; writes run as the verified caller same files src/docs/design/admin-mcp.md

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the two paths that disagreed; the fix is correct.

  • firstLine (src/crux/logread/files.go:149-156) passes the whole loaded region to headOf when base == 0, but only a maxHeadBytes ReadAt otherwise; the gzip path (files.go:238) also limits to maxHeadBytes. Old headOf returned nil when no newline was present, so a >64 KiB first line produced a full-line digest on page cut and a nil head on lookup β€” never matching.
  • New headOf caps at maxHeadBytes on every path and still returns nil for a partial line shorter than the cap (file still being written). The exact-64 KiB boundary resolves identically on both sides (i == maxHeadBytes is not > maxHeadBytes; a cap-sized read with no newline also yields maxHeadBytes), and TestHeadOf pins both.

Looks correct to me.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit a8564ed into hivecommons:v6 Nov 6, 2026
72 of 75 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

Evidence bundle `#169017145a8b` sha256:fb762eefc3290b414d1ca7a2f367cbcbdc4e90418c855bc46aca1ff52c471354 β€” download from the dashboard or `hivectl review evidence #16901`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant