Repository navigation
π fix(logread): a cursor into a file with a first line over 64 KiB no longer expires - #16901
Conversation
β¦ longer expires A log file is identified by a digest of its first line. headOf returned the whole line when it was cut from a loaded region, and nothing when the line had no newline inside the 64 KiB that a cursor lookup reads. The two digests never matched for a first line longer than 64 KiB, so every cursor into that file answered cursor_expired and only its newest page could be read. headOf now returns at most the first 64 KiB of the line either way. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Verified the two paths that disagreed; the fix is correct.
firstLine(src/crux/logread/files.go:149-156) passes the whole loaded region toheadOfwhenbase == 0, but only amaxHeadBytesReadAtotherwise; the gzip path (files.go:238) also limits tomaxHeadBytes. OldheadOfreturnednilwhen no newline was present, so a >64 KiB first line produced a full-line digest on page cut and a nil head on lookup β never matching.- New
headOfcaps atmaxHeadByteson every path and still returnsnilfor a partial line shorter than the cap (file still being written). The exact-64 KiB boundary resolves identically on both sides (i == maxHeadBytesis not> maxHeadBytes; a cap-sized read with no newline also yieldsmaxHeadBytes), andTestHeadOfpins both.
Looks correct to me.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Summary
log_readpages backwards through a hive's logs with a cursor. For a log file whose first line is longer than 64 KiB, the cursor never worked: the second page always came backcursor_expired, so only the newest page of that file could be read. This PR makes the cursor work for such files. Files with ordinary first lines were never affected.What went wrong
A cursor names a file by a digest of its first line, so it survives the file being rotated and compressed between two calls.
For a first line longer than 64 KiB the two never match:
It happens in
hive.logand its rotated history when a file opens with a very long record. It happens more easily in theproxy,litellmandhive-outputsources: a line that never ends is stored in 64 KiB pieces, and each piece is longer than 64 KiB once wrapped as a record, so any file that opens with one is affected.What changes
The identity of a file is the digest of at most the first 64 KiB of its first line, computed the same way on both sides. Cursors issued before this change for files with ordinary first lines stay valid, because their digest is unchanged.
Files
src/crux/logread/files.go:headOf.src/crux/logread/cursor_long_first_line_test.go: new.src/docs/design/admin-mcp.md: one sentence under "Paging".changelog.d/fixed-logread-cursor-long-first-line.mdTests
TestCursorIntoAFileWithALongFirstLine: six first-line sizes around the 64 KiB boundary.TestCursorIntoACompressedFileWithALongFirstLineTestCursorIntoAnOutputSourceThatOpensWithARunawayLineTestPagingThroughAFileWithALongFirstLineReturnsEveryLineOnceTestHeadOfv6without the change. The whole./crux/logread/package passes with it.π€ Generated with Claude Code