Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/added-16530-matrix-lines-more.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- `/api/matrix/lines` now returns `more`, true only while another line the caller may see remains past the page, so clients paging a filtered backlog can stop without an empty follow-up request ([#16530](https://github.com/hivecommons/hive/issues/16530)).
2 changes: 1 addition & 1 deletion src/docs/api-reference/dashboard-misc.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ The monitor renders CI and Review concurrently on one shared PR row; see
| `POST` | `/api/budget-ignore` | Owner only | Budget Ignore Set | `pkg/dashboard/api.go:177` |
| `GET` | `/api/summaries` | Dashboard auth/session | Summaries | `pkg/dashboard/api.go:189` |
| `GET` | `/api/stage-tracker` | Dashboard auth/session | Stage Tracker: every open issue and pull request of the watched repos for the Hive activity monitor — actionable, held, claimed, in-progress, covered-by-PR, filtered (`needs-*`), draft and exempt-labelled items alike, each with `stage`, `stage_since`, `labels`, `created_at`, `author_kind`, `modifiers`, `claimant: {kind: hive-agent|clanker|person, login, agent, hive, since, source: claim-ledger|claim-comment|assignee|kick-history|author}` naming who holds a claimed / in-progress issue or a PR under review or rework (absent when nobody is named; mirrored onto `linked_prs[]`), (PRs) `sentinel_alert` / `sentinel_findings` / `sentinel_reason` when the sentinel flagged the PR, `decision_needed` on a row parked on a person (085 / 045 / 015, the needs-human pill; mirrored onto the issue's `linked_prs[]` entry): `{question, asked_by, since, blocker, blocker_reason, options: [{action, label, enabled, reason_if_disabled}]}` — the reviewer's own decision question when the verdict record has one, the blocker in plain words, and the applicable actions under the row context menu's action ids (`decide:delegate` first), (PRs) `lineage` on the hive's copy of an adopted PR — `{kind: adopted|superseded|rebased-by-hive, from_number, from_url, from_author, from_author_kind, to_number, to_url, at, reason}`, derived from the stage store's adoption record or the `hive/adopt-<n>` head branch, mirrored onto the issue's `linked_prs[]` entry; absent on every other row — and (issues) `linked_prs`; items with no stage yet report stage `0` "Open"; `needs_human_labels` is the "a person must act" label set behind the monitor's Needs human facet (the same list as `/api/status.labelPresets[needs-human]`); `recently_merged` lists the PRs (`repo`, `number`, `merged_at`) whose `pr_merged` audit entry is under 30 minutes old, so a client diffing two payloads can tell a merge from a close; a PR in a conflict-aware merge train carries `merge_train` `{id, state, front, position, size, attempt, max_attempts, overlap_files, ahead, length, head_number, head_state, head_reason, head_since, eta_s, eta_at, eta_basis, late, stalled, stalled_s}` — `head_state` one of `needs-human` / `ci` / `soaking` / `ready` / `merging` / `fixing` / `unknown` with `head_reason` in plain words; `eta_s` / `eta_at` the estimated wait for this member's own merge attempt (own remaining soak + each PR ahead's remaining soak or one sweep + the next sweep tick; absent while the head waits on a person or CI), `eta_at` anchored to the member's first estimate at its position so a countdown survives polls, `eta_basis` its short reading ("2 ahead · head soaking 12m · next sweep in 40s"), `late` once now has overrun the first estimate twice over, `stalled` / `stalled_s` once the head has sat longer than `governor.merge_train_stall_warn_s` ([#15395](https://github.com/hivecommons/hive/issues/15395)); a PR the queue holds because another open PR declares the same `Fixes #N` carries no `merge_train` and a `merge_gate` of `{state: "duplicate-of:<winner>", duplicate_of, duplicate_issue}` plus the `needs-decision` modifier (the queue keeps one PR per issue: green first, then the earlier-opened); a PR GitHub reports conflicting on its own, or one under repair (stage/070, 071, 072, 074, 085, or a red required check), carries no `merge_train` and is not counted in any member's `size` / `length` — a repaired member the view dropped carries `left_train: {why: fixing|needs-author|returned-to-contributor|adopted-by-hive|needs-decision|checks-failing, note}` instead (mirrored onto the issue's `linked_prs[]` entry), and a train left with one member reports no `merge_train` at all; a PR short of the merge queue (below stage/080) the planner ordered rides no train either — it carries `overlap: {behind, with, files, note}` ("overlaps #15637 (sidebar.html, index.html) — will queue behind it") and no `merge_train`, and counts in no member's `size` — the state slugs (`front`, `queued-behind`) are payload values only, the SPA renders ordinal places ("Merge queued · 1st of 2", with the train's tag when more than one train is on the page). Served from a server-side cache rebuilt at most every `dashboard.stage_tracker.cache_ttl_s` (default `governor.eval_interval_s`) from the spoke's existing snapshot — a request never calls GitHub. Carries `generated_at`, `source` (`cache` or `fresh`), `cache_ttl_s`, an `ETag` and `Cache-Control: private, max-age=<remaining>`; `If-None-Match` yields 304. Each hit is recorded under REST caller `dashboard:stage-tracker` so `/api/gh-rate-limits` shows it charged nothing. | `pkg/dashboard/api.go:190` |
| `GET` | `/api/matrix/lines` | Dashboard auth/session | Matrix Lines: serve bounded backlog of pushed lines behind dashboard.matrix with repo allowlist filtering. `max` counts only lines the repo allowlist admits, so a page is never emptied by filtered-out lines. A client paging with `max` should use the last returned line's `seq` as its next `since`, not the top-level `seq`, which is the source's latest sequence number and would skip lines still past the page. | `pkg/dashboard/api.go:191` |
| `GET` | `/api/matrix/lines` | Dashboard auth/session | Matrix Lines: serve bounded backlog of pushed lines behind dashboard.matrix with repo allowlist filtering. `max` counts only lines the repo allowlist admits, so a page is never emptied by filtered-out lines. A client paging with `max` should use the last returned line's `seq` as its next `since`, not the top-level `seq`, which is the source's latest sequence number and would skip lines still past the page. `more` is true only while another line the caller may see remains past the page; stop paging when it is false. | `pkg/dashboard/api.go:191` |
| `POST` | `/api/prs/{owner}/{repo}/{number}/queue-automerge` | Merger/owner role | Queue PRAuto Merge | `pkg/dashboard/api.go:192` |
| `GET` | `/api/inference/models/{backend}` | Dashboard auth/session | Inference Models | `pkg/dashboard/api.go:369` |
| `GET` | `/api/hive-id` | Dashboard auth/session | Hive IDGet | `pkg/dashboard/api.go:431` |
Expand Down
5 changes: 4 additions & 1 deletion src/pkg/dashboard/api_matrix.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ type matrixLinesResponse struct {
Seq uint64 `json:"seq"`
Source string `json:"source"`
Lines []matrixLine `json:"lines"`
// More is true only when another line this caller may see remains.
More bool `json:"more"`
// LastIngestAt is when the pass that feeds this source last finished
// (RFC 3339), or null before the first; NextIngestInS is the whole
// seconds until the next pass is due, 0 when unknown. Together they let
Expand Down Expand Up @@ -130,7 +132,7 @@ func (s *Server) handleMatrixLines(w http.ResponseWriter, r *http.Request) {
}

// Filter inside the scan so max counts only lines this caller may see.
lines, latest := src.SinceMatching(since, limit, func(l matrixLine) bool {
lines, latest, more := src.SinceMatching(since, limit, func(l matrixLine) bool {
return cfg.RepoAllowed(l.Repo)
})

Expand All @@ -139,6 +141,7 @@ func (s *Server) handleMatrixLines(w http.ResponseWriter, r *http.Request) {
Seq: latest,
Source: "pushed-lines",
Lines: lines,
More: more,
}
if last, nextS := src.ingestStatus(src.now()); !last.IsZero() {
resp.LastIngestAt = &last
Expand Down
40 changes: 40 additions & 0 deletions src/pkg/dashboard/api_matrix_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,46 @@ func getMatrixLines(t *testing.T, s *Server, target string) matrixLinesResponse
return resp
}

// More describes visible work, even when the page is full and newer ring
// entries are all filtered out (#16530).
func TestHandleMatrixLines_MoreCountsOnlyAllowedLines(t *testing.T) {
s := newTestServer()
s.MatrixObserver(config.MatrixConfig{Enabled: true, Repos: []string{"hivecommons/allowed"}})
src := s.matrix
t.Cleanup(src.close)
helperAddMatrixLine(src, "hivecommons/denied", "hidden before")
helperAddMatrixLine(src, "hivecommons/allowed", "first")
helperAddMatrixLine(src, "hivecommons/denied", "hidden between")
helperAddMatrixLine(src, "hivecommons/allowed", "last")
helperAddMatrixLine(src, "hivecommons/denied", "hidden after")

for _, tc := range []struct {
name string
query string
seqs []uint64
more bool
}{
{"truncated", "since=0&max=1", []uint64{2}, true},
{"full final page", "since=2&max=1", []uint64{4}, false},
{"short final page", "since=2&max=2", []uint64{4}, false},
{"initial backlog", "since=0", []uint64{2, 4}, false},
{"only hidden lines", "since=4&max=1", nil, false},
{"caught up", "since=5&max=1", nil, false},
} {
t.Run(tc.name, func(t *testing.T) {
resp := getMatrixLines(t, s, "/api/matrix/lines?"+tc.query)
if resp.Seq != 5 || resp.More != tc.more || len(resp.Lines) != len(tc.seqs) {
t.Fatalf("response = %+v, want latest=5 more=%v seqs=%v", resp, tc.more, tc.seqs)
}
for i, seq := range tc.seqs {
if resp.Lines[i].Seq != seq {
t.Fatalf("line %d seq = %d, want %d", i, resp.Lines[i].Seq, seq)
}
}
})
}
}

// /api/matrix/lines reports when the ingest pass last ran and when the next
// is due (#16276), null/0 before the first pass, so the kiosk can tell "no
// pushes" from "no ingest".
Expand Down
13 changes: 9 additions & 4 deletions src/pkg/dashboard/matrix_source.go
Original file line number Diff line number Diff line change
Expand Up @@ -383,27 +383,32 @@ func (m *matrixSource) removeLocked(key string) {
// Since returns up to limit backlog lines with Seq > seq, oldest first, and the
// latest sequence number. limit <= 0 means a default page.
func (m *matrixSource) Since(seq uint64, limit int) ([]matrixLine, uint64) {
return m.SinceMatching(seq, limit, nil)
lines, latest, _ := m.SinceMatching(seq, limit, nil)
return lines, latest
}

// SinceMatching is Since restricted to lines keep accepts (nil keeps all).
// limit counts only kept lines, so a page is never emptied by lines the caller
// may not see while kept lines remain past it (hivecommons/hive#15979).
func (m *matrixSource) SinceMatching(seq uint64, limit int, keep func(matrixLine) bool) ([]matrixLine, uint64) {
// The final result reports whether another kept line remains after this page.
func (m *matrixSource) SinceMatching(seq uint64, limit int, keep func(matrixLine) bool) ([]matrixLine, uint64, bool) {
if limit <= 0 {
limit = matrixDefaultSinceMax
}
m.mu.Lock()
defer m.mu.Unlock()
n := len(m.ring)
out := make([]matrixLine, 0, min(limit, n))
for i := 0; i < n && len(out) < limit; i++ {
for i := 0; i < n; i++ {
l := m.ring[(m.next+i)%n]
if l.Seq > seq && (keep == nil || keep(l)) {
if len(out) == limit {
return out, m.seq, true
}
out = append(out, l)
}
}
return out, m.seq
return out, m.seq, false
}

// Latest is the sequence number of the newest accepted line; 0 when none.
Expand Down
Loading