diff --git a/changelog.d/added-16530-matrix-lines-more.md b/changelog.d/added-16530-matrix-lines-more.md new file mode 100644 index 0000000000..5a95281519 --- /dev/null +++ b/changelog.d/added-16530-matrix-lines-more.md @@ -0,0 +1 @@ +- `/api/matrix/lines` now returns `more`, true only while another line the caller may see remains past the page, so clients paging a filtered backlog can stop without an empty follow-up request ([#16530](https://github.com/hivecommons/hive/issues/16530)). diff --git a/src/docs/api-reference/dashboard-misc.md b/src/docs/api-reference/dashboard-misc.md index 74debd9a15..cf87b2ca02 100644 --- a/src/docs/api-reference/dashboard-misc.md +++ b/src/docs/api-reference/dashboard-misc.md @@ -56,7 +56,7 @@ The monitor renders CI and Review concurrently on one shared PR row; see | `POST` | `/api/budget-ignore` | Owner only | Budget Ignore Set | `pkg/dashboard/api.go:177` | | `GET` | `/api/summaries` | Dashboard auth/session | Summaries | `pkg/dashboard/api.go:189` | | `GET` | `/api/stage-tracker` | Dashboard auth/session | Stage Tracker: every open issue and pull request of the watched repos for the Hive activity monitor — actionable, held, claimed, in-progress, covered-by-PR, filtered (`needs-*`), draft and exempt-labelled items alike, each with `stage`, `stage_since`, `labels`, `created_at`, `author_kind`, `modifiers`, `claimant: {kind: hive-agent|clanker|person, login, agent, hive, since, source: claim-ledger|claim-comment|assignee|kick-history|author}` naming who holds a claimed / in-progress issue or a PR under review or rework (absent when nobody is named; mirrored onto `linked_prs[]`), (PRs) `sentinel_alert` / `sentinel_findings` / `sentinel_reason` when the sentinel flagged the PR, `decision_needed` on a row parked on a person (085 / 045 / 015, the needs-human pill; mirrored onto the issue's `linked_prs[]` entry): `{question, asked_by, since, blocker, blocker_reason, options: [{action, label, enabled, reason_if_disabled}]}` — the reviewer's own decision question when the verdict record has one, the blocker in plain words, and the applicable actions under the row context menu's action ids (`decide:delegate` first), (PRs) `lineage` on the hive's copy of an adopted PR — `{kind: adopted|superseded|rebased-by-hive, from_number, from_url, from_author, from_author_kind, to_number, to_url, at, reason}`, derived from the stage store's adoption record or the `hive/adopt-` head branch, mirrored onto the issue's `linked_prs[]` entry; absent on every other row — and (issues) `linked_prs`; items with no stage yet report stage `0` "Open"; `needs_human_labels` is the "a person must act" label set behind the monitor's Needs human facet (the same list as `/api/status.labelPresets[needs-human]`); `recently_merged` lists the PRs (`repo`, `number`, `merged_at`) whose `pr_merged` audit entry is under 30 minutes old, so a client diffing two payloads can tell a merge from a close; a PR in a conflict-aware merge train carries `merge_train` `{id, state, front, position, size, attempt, max_attempts, overlap_files, ahead, length, head_number, head_state, head_reason, head_since, eta_s, eta_at, eta_basis, late, stalled, stalled_s}` — `head_state` one of `needs-human` / `ci` / `soaking` / `ready` / `merging` / `fixing` / `unknown` with `head_reason` in plain words; `eta_s` / `eta_at` the estimated wait for this member's own merge attempt (own remaining soak + each PR ahead's remaining soak or one sweep + the next sweep tick; absent while the head waits on a person or CI), `eta_at` anchored to the member's first estimate at its position so a countdown survives polls, `eta_basis` its short reading ("2 ahead · head soaking 12m · next sweep in 40s"), `late` once now has overrun the first estimate twice over, `stalled` / `stalled_s` once the head has sat longer than `governor.merge_train_stall_warn_s` ([#15395](https://github.com/hivecommons/hive/issues/15395)); a PR the queue holds because another open PR declares the same `Fixes #N` carries no `merge_train` and a `merge_gate` of `{state: "duplicate-of:", duplicate_of, duplicate_issue}` plus the `needs-decision` modifier (the queue keeps one PR per issue: green first, then the earlier-opened); a PR GitHub reports conflicting on its own, or one under repair (stage/070, 071, 072, 074, 085, or a red required check), carries no `merge_train` and is not counted in any member's `size` / `length` — a repaired member the view dropped carries `left_train: {why: fixing|needs-author|returned-to-contributor|adopted-by-hive|needs-decision|checks-failing, note}` instead (mirrored onto the issue's `linked_prs[]` entry), and a train left with one member reports no `merge_train` at all; a PR short of the merge queue (below stage/080) the planner ordered rides no train either — it carries `overlap: {behind, with, files, note}` ("overlaps #15637 (sidebar.html, index.html) — will queue behind it") and no `merge_train`, and counts in no member's `size` — the state slugs (`front`, `queued-behind`) are payload values only, the SPA renders ordinal places ("Merge queued · 1st of 2", with the train's tag when more than one train is on the page). Served from a server-side cache rebuilt at most every `dashboard.stage_tracker.cache_ttl_s` (default `governor.eval_interval_s`) from the spoke's existing snapshot — a request never calls GitHub. Carries `generated_at`, `source` (`cache` or `fresh`), `cache_ttl_s`, an `ETag` and `Cache-Control: private, max-age=`; `If-None-Match` yields 304. Each hit is recorded under REST caller `dashboard:stage-tracker` so `/api/gh-rate-limits` shows it charged nothing. | `pkg/dashboard/api.go:190` | -| `GET` | `/api/matrix/lines` | Dashboard auth/session | Matrix Lines: serve bounded backlog of pushed lines behind dashboard.matrix with repo allowlist filtering. `max` counts only lines the repo allowlist admits, so a page is never emptied by filtered-out lines. A client paging with `max` should use the last returned line's `seq` as its next `since`, not the top-level `seq`, which is the source's latest sequence number and would skip lines still past the page. | `pkg/dashboard/api.go:191` | +| `GET` | `/api/matrix/lines` | Dashboard auth/session | Matrix Lines: serve bounded backlog of pushed lines behind dashboard.matrix with repo allowlist filtering. `max` counts only lines the repo allowlist admits, so a page is never emptied by filtered-out lines. A client paging with `max` should use the last returned line's `seq` as its next `since`, not the top-level `seq`, which is the source's latest sequence number and would skip lines still past the page. `more` is true only while another line the caller may see remains past the page; stop paging when it is false. | `pkg/dashboard/api.go:191` | | `POST` | `/api/prs/{owner}/{repo}/{number}/queue-automerge` | Merger/owner role | Queue PRAuto Merge | `pkg/dashboard/api.go:192` | | `GET` | `/api/inference/models/{backend}` | Dashboard auth/session | Inference Models | `pkg/dashboard/api.go:369` | | `GET` | `/api/hive-id` | Dashboard auth/session | Hive IDGet | `pkg/dashboard/api.go:431` | diff --git a/src/pkg/dashboard/api_matrix.go b/src/pkg/dashboard/api_matrix.go index be10478ede..684cee8605 100644 --- a/src/pkg/dashboard/api_matrix.go +++ b/src/pkg/dashboard/api_matrix.go @@ -81,6 +81,8 @@ type matrixLinesResponse struct { Seq uint64 `json:"seq"` Source string `json:"source"` Lines []matrixLine `json:"lines"` + // More is true only when another line this caller may see remains. + More bool `json:"more"` // LastIngestAt is when the pass that feeds this source last finished // (RFC 3339), or null before the first; NextIngestInS is the whole // seconds until the next pass is due, 0 when unknown. Together they let @@ -130,7 +132,7 @@ func (s *Server) handleMatrixLines(w http.ResponseWriter, r *http.Request) { } // Filter inside the scan so max counts only lines this caller may see. - lines, latest := src.SinceMatching(since, limit, func(l matrixLine) bool { + lines, latest, more := src.SinceMatching(since, limit, func(l matrixLine) bool { return cfg.RepoAllowed(l.Repo) }) @@ -139,6 +141,7 @@ func (s *Server) handleMatrixLines(w http.ResponseWriter, r *http.Request) { Seq: latest, Source: "pushed-lines", Lines: lines, + More: more, } if last, nextS := src.ingestStatus(src.now()); !last.IsZero() { resp.LastIngestAt = &last diff --git a/src/pkg/dashboard/api_matrix_test.go b/src/pkg/dashboard/api_matrix_test.go index 836635b6cf..f1ff675abe 100644 --- a/src/pkg/dashboard/api_matrix_test.go +++ b/src/pkg/dashboard/api_matrix_test.go @@ -416,6 +416,46 @@ func getMatrixLines(t *testing.T, s *Server, target string) matrixLinesResponse return resp } +// More describes visible work, even when the page is full and newer ring +// entries are all filtered out (#16530). +func TestHandleMatrixLines_MoreCountsOnlyAllowedLines(t *testing.T) { + s := newTestServer() + s.MatrixObserver(config.MatrixConfig{Enabled: true, Repos: []string{"hivecommons/allowed"}}) + src := s.matrix + t.Cleanup(src.close) + helperAddMatrixLine(src, "hivecommons/denied", "hidden before") + helperAddMatrixLine(src, "hivecommons/allowed", "first") + helperAddMatrixLine(src, "hivecommons/denied", "hidden between") + helperAddMatrixLine(src, "hivecommons/allowed", "last") + helperAddMatrixLine(src, "hivecommons/denied", "hidden after") + + for _, tc := range []struct { + name string + query string + seqs []uint64 + more bool + }{ + {"truncated", "since=0&max=1", []uint64{2}, true}, + {"full final page", "since=2&max=1", []uint64{4}, false}, + {"short final page", "since=2&max=2", []uint64{4}, false}, + {"initial backlog", "since=0", []uint64{2, 4}, false}, + {"only hidden lines", "since=4&max=1", nil, false}, + {"caught up", "since=5&max=1", nil, false}, + } { + t.Run(tc.name, func(t *testing.T) { + resp := getMatrixLines(t, s, "/api/matrix/lines?"+tc.query) + if resp.Seq != 5 || resp.More != tc.more || len(resp.Lines) != len(tc.seqs) { + t.Fatalf("response = %+v, want latest=5 more=%v seqs=%v", resp, tc.more, tc.seqs) + } + for i, seq := range tc.seqs { + if resp.Lines[i].Seq != seq { + t.Fatalf("line %d seq = %d, want %d", i, resp.Lines[i].Seq, seq) + } + } + }) + } +} + // /api/matrix/lines reports when the ingest pass last ran and when the next // is due (#16276), null/0 before the first pass, so the kiosk can tell "no // pushes" from "no ingest". diff --git a/src/pkg/dashboard/matrix_source.go b/src/pkg/dashboard/matrix_source.go index 2a891a285c..6891d3fbd2 100644 --- a/src/pkg/dashboard/matrix_source.go +++ b/src/pkg/dashboard/matrix_source.go @@ -383,13 +383,15 @@ func (m *matrixSource) removeLocked(key string) { // Since returns up to limit backlog lines with Seq > seq, oldest first, and the // latest sequence number. limit <= 0 means a default page. func (m *matrixSource) Since(seq uint64, limit int) ([]matrixLine, uint64) { - return m.SinceMatching(seq, limit, nil) + lines, latest, _ := m.SinceMatching(seq, limit, nil) + return lines, latest } // SinceMatching is Since restricted to lines keep accepts (nil keeps all). // limit counts only kept lines, so a page is never emptied by lines the caller // may not see while kept lines remain past it (hivecommons/hive#15979). -func (m *matrixSource) SinceMatching(seq uint64, limit int, keep func(matrixLine) bool) ([]matrixLine, uint64) { +// The final result reports whether another kept line remains after this page. +func (m *matrixSource) SinceMatching(seq uint64, limit int, keep func(matrixLine) bool) ([]matrixLine, uint64, bool) { if limit <= 0 { limit = matrixDefaultSinceMax } @@ -397,13 +399,16 @@ func (m *matrixSource) SinceMatching(seq uint64, limit int, keep func(matrixLine defer m.mu.Unlock() n := len(m.ring) out := make([]matrixLine, 0, min(limit, n)) - for i := 0; i < n && len(out) < limit; i++ { + for i := 0; i < n; i++ { l := m.ring[(m.next+i)%n] if l.Seq > seq && (keep == nil || keep(l)) { + if len(out) == limit { + return out, m.seq, true + } out = append(out, l) } } - return out, m.seq + return out, m.seq, false } // Latest is the sequence number of the newest accepted line; 0 when none.