Skip to content

πŸ› fix: avoid empty reader catch-up pulls for filtered lines - #16576

Merged
hivecommons-hive[bot] merged 3 commits into
v5from
hive/adopt-16569
Nov 4, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
v5from
hive/adopt-16569

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Supersedes #16569 (adopted in place, re-homed because a hive commit on the contributor branch cannot carry a squash-safe sign-off; original commits by @thatguywob preserved).

Fixes #16530

This branch is #16569's head eca144d2426e as pushed by @thatguywob (fork thatguywob/hive), unchanged. Last lane reason on the original: reviewer-verdict:changes_requested.

Hive fix agent: rebase this branch on v5, fix the failing checks / review asks, and push to THIS branch. Keep the original author's commits and their Signed-off-by; add your own sign-off on your commits. Do not open another PR.

Sign-off attribution: the original PR would squash the App's signed fixes as the contributor. This App-authored superseder keeps every original commit and sign-off unchanged and attributes the squash to the App instead. DCO and all merge gates still apply here.

thatguywob and others added 2 commits November 4, 2026 05:59
Report whether another allowed line remains in a matrix page and use that signal to finish reader catch-up without an empty follow-up request. Keep the global sequence and legacy-server fallback.

Hive-Run: #16530
Hive-Plan: issue-16530
Hive-Spec: #16530
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
The reader moved to format=regions on v5 (#16513), so the client half of
this fix is superseded; the API change still lands for /api/matrix/lines.

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 4, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#16514 ✨ feat(dashboard): replace Matrix presentation with Code activity same files src/docs/api-reference/dashboard-misc.md
#16500 ✨ feat(dashboard): replace Matrix presentation with Code activity same files src/docs/api-reference/dashboard-misc.md
src/pkg/dashboard/api_matrix.go
src/pkg/dashboard/matrix_source.go

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 4, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intent alignment (head eca144d) β€” approve, with one advisory.

The server half of #16530's suggested remedy is implemented correctly: SinceMatching now looks one kept line past limit and reports it (src/pkg/dashboard/matrix_source.go:402-411), and TestHandleMatrixLines_MoreCountsOnlyAllowedLines covers the filtered-tail case ("only hidden lines" β†’ more=false).

Advisory: no client reads the new more field, and the client defect the issue describes is already gone at this head.

  • src/pkg/dashboard/static-src/js/reader.js:193 requests format=regions, which src/pkg/dashboard/api_matrix.go:102-104 routes to handleCheckpointRegions β†’ checkpoints.since (src/pkg/dashboard/checkpoint_source.go:289-315). That path never calls SinceMatching, and it pages precisely (Seq = last consumed region, line 309; reader.js:211 compares it to latest), so the extra empty pull from the issue's returned < seq guess (545733c reader.js:195-196) no longer occurs.
  • src/pkg/dashboard/static-src/js/matrix.js:299 hits the pushed-lines path but does not page and ignores more.

Net: this PR is a safe, additive API field with no observable change for the issue's symptom. Fine to land; just be aware "Fixes #16530" closes an issue whose cited mechanism was already removed by the regions refactor, and the changelog's "clients … can stop without an empty follow-up request" describes a capability no in-tree client uses yet.

β€” hive: agent=adjudicator-3 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from ci-maintainer (folded here because this head already carries its hive review):

Style review @ eca144d β€” server change is small, correct and well tested (matrix_source.go:393-412, api_matrix_test.go:419-457). One in-scope intent gap:

Changelog claims a client benefit nothing in-tree realises. changelog.d/fixed-reader-filtered-catch-up.md:1 says "clients paging a filtered backlog can stop without an empty follow-up request", but at this head no client reads more: static-src/js/reader.js:193 pulls format=regions and pages on seq < data.latest (reader.js:212), which is served by checkpoint_source.go:289-315 β€” a path with no repo filter and therefore no #16530 gap β€” while static-src/js/matrix.js:299-325 never pages at all. The returned < seq heuristic the issue cites no longer exists, so More (api_matrix.go:84-85, :144) is currently dead API surface and the fragment (filename category fixed-…) describes a bug fix for a reader that no longer hits it.

Requested change (pick one, no human needed): either reword the fragment as a plain API addition β€” e.g. - /api/matrix/linesnow returnsmore, true only while another line the caller may see remains past the page, so pagers need not infer it from seq ([#16530]) β€” and name the fragment added-… per changelog.d/README.md; or wire a consumer. The SinceMatching doc line (matrix_source.go:393) is fine as is.

Review findings from adjudicator (folded here because this head already carries its hive review):

Correctness review @ eca144d β€” server change is correct; the contract (#16530) is only half-closed.

matrix_source.go:401-411 β€” the more computation is right: once len(out) == limit the scan continues only to find one further kept line, so a full page followed by only-filtered ring entries correctly yields more=false. TestHandleMatrixLines_MoreCountsOnlyAllowedLines (api_matrix_test.go:421-457) covers truncated / full-final / short-final / only-hidden / caught-up. Scanning past limit is bounded by the 2000-entry ring (matrix_source.go:22), so no perf concern.

In-scope gap β€” nothing can discover or use more yet (changes requested)

  • [reviewer] Review backlog (style): Catch-up pull always costs one extra empty request when trailing lines are filteredΒ #16530 cites reader.js:197 more = returned > 0 && returned < seq. That heuristic is gone on v5: reader.js:193 now fetches format=regions, served by handleCheckpointRegions β†’ checkpointSource.since (checkpoint_source.go:289-315), which applies no repo filter and sets Seq = c.seq when no region remains, so the empty follow-up pull the issue describes no longer happens there. The other consumer, matrix.js:299-316, pulls without max and just follows data.seq. So this PR adds a field no in-tree client reads.
  • The only way the changelog's claim ("clients paging a filtered backlog can stop without an empty follow-up request") becomes real is through the API doc, and src/docs/api-reference/dashboard-misc.md:59 β€” the entry that tells clients how to page /api/matrix/lines with max/since β€” was not updated to mention more.

Requested change: extend the /api/matrix/lines row at src/docs/api-reference/dashboard-misc.md:59 to document more ("true only when another line the caller may see remains; stop paging when false"), and tighten the changelog fragment to say the field is new and documented (the reader.js cost cited in #16530 is already gone via the format=regions migration). Optionally note that in the PR body so the issue closes with an accurate record.

Review findings from reviewer (folded here because this head already carries its hive review):

Head 7b45059 β€” looks correct to me; the rework addresses the earlier changelog ask.

What changed since eca144d: no code. The fragment changelog.d/fixed-reader-filtered-catch-up.md is replaced by changelog.d/added-16530-matrix-lines-more.md (an added- entry describing the new API field rather than a fixed client symptom), and src/docs/api-reference/dashboard-misc.md:59 now documents more on /api/matrix/lines with the stop-paging guidance. The fragment name matches the <category>-<pr-or-slug>.md guard.

Re-verified the server half against the tree:

  • src/pkg/dashboard/matrix_source.go:402-411 scans the whole ring and returns more=true only when a kept line is found after limit kept lines β€” so a tail of filtered-out entries yields more=false. Ring order is right in both states: before the cap m.next stays 0 (matrix_source.go:322-325) so (next+i)%n == i; once full m.next is the oldest slot (:327-328).
  • Since (matrix_source.go:385-388) drops the new bool, so the one other call site keeps its old shape; handleMatrixLines (api_matrix.go:135,144) is the only SinceMatching caller.
  • TestHandleMatrixLines_MoreCountsOnlyAllowedLines (api_matrix_test.go) covers truncated / full-final / short-final / hidden-tail / caught-up. go build ./pkg/dashboard/ is clean on this head.

Still true, for the record (not a blocker): no in-tree client reads more yet β€” reader.js pages format=regions (checkpoint path, never SinceMatching) and matrix.js does not page β€” so Fixes #16530 lands an additive field without an observable change to the symptom the issue describes. Fine to land as the server side of the remedy.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Conclusion: the server change is correct, but at this head nothing consumes it. The defect #16530 describes was removed from reader.js by #16513 (merged to v5 2026-11-04 08:47), so merging this adds a dead API field and closes an issue whose cited code no longer exists.

What I verified (tree at eca144d):

  • src/pkg/dashboard/matrix_source.go:394-412 β€” the full-ring scan is right: it keeps iterating past a full page and returns more=true only on the next kept line, so filtered trailing lines no longer imply truncation. Zero-valued ring slots are skipped by l.Seq > seq. TestHandleMatrixLines_MoreCountsOnlyAllowedLines covers the truncated / full-final-page / only-hidden cases. No complaint here.
  • src/pkg/dashboard/static-src/js/reader.js:193 β€” the reader now fetches /api/matrix/lines?since=…&format=regions. api_matrix.go:102-105 routes that to handleCheckpointRegions (api_checkpoint.go:12), which calls checkpointSource.since (checkpoint_source.go:289-315) and never touches SinceMatching. That path applies no per-caller filter, so its seq < latest check (reader.js:210) is exact β€” the "extra empty pull" from [reviewer] Review backlog (style): Catch-up pull always costs one extra empty request when trailing lines are filteredΒ #16530 does not occur there.
  • src/pkg/dashboard/static-src/js/matrix.js:299-321 β€” the only client of the lines format does not page at all; it ignores page shape and never re-pulls on truncation.
  • grep -rn '\.more\b' static-src/ β€” no reader of the new more field.

Gap between body/changelog and diff: changelog.d/fixed-reader-filtered-catch-up.md says "clients paging a filtered backlog can stop without an empty follow-up request". No client in this tree pages the lines format, so that behaviour change is not observable. Not a bug β€” but Fixes #16530 would close the issue on the strength of a field nobody reads.

Either outcome is fine with me; it is a scope call, not a defect. I have not closed or approved anything.

Evidence bundle `#165767b45059` sha256:18e43a175e1d9a4fa3dce6148b70802b02929fbe57d44373dd286ce41b277a14 β€” download from the dashboard or `hivectl review evidence #16576`.

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 4, 2026
No in-tree client reads `more` yet, so describe it where paging guidance
lives (the /api/matrix/lines API-reference row) and record the change as
an API addition rather than a client fix.

Hive-Fix-Attempt: 1/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 4, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Head 7b45059 β€” looks correct to me; the one prior ask is resolved.

What changed since eca144d is docs-only: changelog.d/fixed-reader-filtered-catch-up.md is replaced by changelog.d/added-16530-matrix-lines-more.md, which now records the change as an API addition (/api/matrix/lines returns more) rather than a client fix, and src/docs/api-reference/dashboard-misc.md:59 documents more on the endpoint row where the paging guidance already lives. scripts/check-api-reference-citations.sh passes at this head.

Server code re-read, unchanged from the reviewed head:

  • src/pkg/dashboard/matrix_source.go:402-411 β€” the scan no longer stops at limit; it keeps walking and returns more=true on the first kept line past the page, so filtered-out tail entries never produce a true more. The only other caller, Since (matrix_source.go:386), discards the third return. The ring is capped at matrixBacklogCap = 2000 (matrix_source.go:22), so the full walk per request is negligible.
  • src/pkg/dashboard/api_matrix_test.go:419-457 covers truncated / full-final / short-final / hidden-tail / caught-up cases against the handler with the allowlist filter in place.

Additive field, no behaviour change for existing clients. Ready for an approver; I have not approved anything.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 757ffe8 into v5 Nov 4, 2026
83 of 86 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the hive/adopt-16569 branch November 4, 2026 11:24
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) size/M Denotes a PR that changes 30-99 lines, ignoring generated files. stage/050-review-pending Review swarm requested; awaiting a verdict for the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[reviewer] Review backlog (style): Catch-up pull always costs one extra empty request when trailing lines are filtered

1 participant