diff --git a/changelog.d/added-close-stale-agent-findings.md b/changelog.d/added-close-stale-agent-findings.md new file mode 100644 index 0000000000..4dec1e91bb --- /dev/null +++ b/changelog.d/added-close-stale-agent-findings.md @@ -0,0 +1 @@ +- An agent finding that the issue filter keeps off the queues is now `CLOSE` when every permalink it cites is gone from the default branch (`cited_lines_gone`), instead of waiting to be admitted or triaged. The enumeration reads the cited lines at the pinned commit and the same file on the default branch. It checks at most 3 findings per repository per pass, trusts a result for 6 hours, and treats a read failure as unknown. A file that is gone from the default branch counts as unknown too, because a rename keeps the lines. projectbluefin/testsuite#1085, for example, cited `SKILL.md:163` for a fallback the guide no longer described. diff --git a/src/docs/dashboard.md b/src/docs/dashboard.md index 8b9b325c76..77a8c742e0 100644 --- a/src/docs/dashboard.md +++ b/src/docs/dashboard.md @@ -749,6 +749,7 @@ so the reason text can change without breaking them. | `decided_not_admitted`, `agent_finding_not_admitted`, `person_issue_not_admitted` | `ADMIT` | | `reporter_confirmation_stale` | `CLOSE` | | `verified_done` | `CLOSE` | +| `cited_lines_gone` | `CLOSE` | | `agent_filed_unacknowledged` | `TRIAGE` | | `level_gate_hold`, `outreach_hold`, `policy_hold`, `hold_source_unknown` | `REVIEW` | | `hive_pr_needs_human` (L3–L4) | `REVIEW` | @@ -797,6 +798,7 @@ means the situation cannot occur at that level. | PR of any author that `/api/review/pipeline` holds in `human_hold` for a review-swarm reason: a `requires_human` verdict (`review_requires_human`; one on an earlier head counts until a merge-eligible `approve` is recorded after it), a `reject` verdict (`review_rejected`) — either one released once a person's `APPROVED` review or `/hive approve` reply after the verdict has been recorded by the self-merge sweep, the release the sweep itself honours ([review-swarm.md](review-swarm.md)), a dispatch hold on the current head — reviews exhausted their retries, fix cap reached, no fixer agent (`review_dispatch_hold`; a hold on an older head does not count) — or the fix-cycle cap (`review_loop_cap`). `review.human_decision_label` does not change the decision. Once a person has answered the `requires_human` / `reject` verdict on the current head with an instruction (`/hive decision ` or a `CHANGES_REQUESTED` review, recorded after the verdict) the row is Not yours — "@who answered Hive's review on this head …; the hive is fixing it" — until a new head or a newer verdict, except on a PR the hive adopted whose branch it cannot push (fork-rehome-refused on the head, a fork without maintainer edits, an adoption escalated for a workflow-file diff, or a recorded capability blocker), which stays Yours — `DECIDE` (`adopted_push_refused`, naming why); a dispatch hold or the loop cap stays `REVIEW`, and a bare approval is left to the release rules above | Yours — `REVIEW` | same | Yours — `REVIEW`; a Hive PR that also carries `needs-human` waits for the adjudicator first, as in the row above | same | | Sentinel-flagged PR | Yours — `INSPECT` | same | same | same | | Issue Hive verified as done but could not close | Yours — `CLOSE` | same | same | same | +| Agent finding the issue filter refuses whose every cited permalink (`…/blob//#L`, same repository) is gone from the default branch: none of the cited lines of 16 to 400 characters is still a line of the file. A file the default branch no longer has is not enough (a rename or move keeps the lines), nor is a citation with no such line or a path the pinned commit lacks. The enumeration checks it for up to 3 findings per repository per pass and trusts a result for 6 hours; the evidence lines read at a pinned SHA are kept (never the file), and a citation that settles as unknown costs no further reads | Yours — `CLOSE` (`cited_lines_gone`, naming the citation: close it, or admit it if the finding still holds) | same | same | same | | Issue covered by an open PR | Not yours — the PR closes it on merge | same | same | same | | Issue parked on a decision (`needs_decision`) that an open or held PR authored by this hive's App closes (GitHub closing reference) | Not yours — "Waits on hive PR #N, which closes it"; the park label stays, so the ask comes back if the PR closes unmerged. A closing PR by anyone else leaves it `DECIDE` | same | same | same | | `blocked` with an open GitHub blocked-by link | Not yours — waiting on the linked item | same | same | same | @@ -822,6 +824,8 @@ open hive-authored PR closes is Not yours; (6) untrusted reporter awaiting admission → `ADMIT`; (7) awaiting reporter confirmation → Not yours until it has had no activity for the stale period, then `CLOSE` (`reporter_confirmation_stale`); (8) verified done but not closed → `CLOSE`; +(8b) an agent finding whose cited lines are gone from the default branch → +`CLOSE` (`cited_lines_gone`); (9) agent-filed and unacknowledged where acknowledgment is required → `TRIAGE`; (9b) an issue the issue filter refuses that carries `approved-direction` → `ADMIT` (`decided_not_admitted`), that an agent filed diff --git a/src/docs/scale-envelope.md b/src/docs/scale-envelope.md index 99c8580dc2..0e9663f960 100644 --- a/src/docs/scale-envelope.md +++ b/src/docs/scale-envelope.md @@ -34,7 +34,7 @@ ones to: 2. **Truncation must preserve the prioritization signal, not cut an arbitrary slice.** This holds today and is easy to assume backwards: - Issues are sorted by actionable rank tier, then **oldest first** within each tier before any cap is applied - (`RankActionableIssues`, `src/pkg/github/client.go:5350-5360`, descending `AgeMinutes` within each tier). + (`RankActionableIssues`, `src/pkg/github/client.go:5361-5371`, descending `AgeMinutes` within each tier). - PRs are sorted by **review class then oldest-first within class** (`SortPullRequestsForReview`, `src/pkg/github/review_priority.go:334`; fixes → refactors/docs → tests). diff --git a/src/pkg/dashboard/ownership.go b/src/pkg/dashboard/ownership.go index d2474d6566..2a36fece94 100644 --- a/src/pkg/dashboard/ownership.go +++ b/src/pkg/dashboard/ownership.go @@ -81,6 +81,10 @@ const ( OwnershipReasonDecidedNotAdmitted = "decided_not_admitted" OwnershipReasonAgentFindingNotAdmitted = "agent_finding_not_admitted" OwnershipReasonPersonIssueNotAdmitted = "person_issue_not_admitted" + // OwnershipReasonCitedLinesGone: an agent finding the issue filter + // refuses whose cited lines are gone from the default branch + // (RepoWorkIssue.CitedLinesGone). + OwnershipReasonCitedLinesGone = "cited_lines_gone" ) // ownershipReasonVerbs is the fixed reason-code set with the one verb each @@ -113,6 +117,7 @@ var ownershipReasonVerbs = map[string]string{ OwnershipReasonDecidedNotAdmitted: OwnershipVerbAdmit, OwnershipReasonAgentFindingNotAdmitted: OwnershipVerbAdmit, OwnershipReasonPersonIssueNotAdmitted: OwnershipVerbAdmit, + OwnershipReasonCitedLinesGone: OwnershipVerbClose, } // OwnershipActionReleaseHold is the dashboard's existing release-hold action. @@ -529,6 +534,10 @@ type issueOwnershipInput struct { createdAt time.Time updatedAt time.Time + // citedLinesGone is RepoWorkIssue.CitedLinesGone for a non-actionable + // row. + citedLinesGone string + // lane is the lane the classifier routed the issue to. lane string // liveClaim is true while the issue carries an unexpired ClaimedBy claim. @@ -634,6 +643,12 @@ func decideIssueOwnership(in issueOwnershipInput, env ownershipEnv) Ownership { if ownershipVerifiedDone(in, labels, env) { return yourTurn(OwnershipReasonVerifiedDone, "Hive verified this is done but could not close it") } + // (8b) an agent finding whose cited lines are gone from the default + // branch describes code main no longer has: close it rather than + // acknowledge or admit it (projectbluefin/testsuite#1085). + if in.citedLinesGone != "" { + return yourTurn(OwnershipReasonCitedLinesGone, "The cited lines no longer match main ("+in.citedLinesGone+") — close it, or admit it if the finding still holds") + } // (9) agent-filed and unacknowledged while acknowledgment is required. if role := issueAgentRole(in.labels); role != "" && !in.acknowledged && !labels[ownershipLabelApprovedDirection] && env.selfAuthorizationHoldFor(in.repo) { return yourTurn(OwnershipReasonAgentFiledUnacknowledged, "Filed by agent "+role+" — acknowledge it or close it") @@ -967,6 +982,7 @@ func decideRepoOwnership(repos []FrontendRepo, cfg config.DashboardIssueBandsCon authorIsHuman: item.AuthorIsHuman, bucket: item.Bucket, bucketReason: item.Reason, + citedLinesGone: item.CitedLinesGone, decisionBlocker: decisionBlockerFor(nonEmpty(item.Repo, repoName)+"#"+strconv.Itoa(item.Number), item.OpenBlockers, decisionParks, repoName), } in.hiveClosingPR = hiveClosingPRFor(*repo, repoName, item.Number) diff --git a/src/pkg/dashboard/ownership_11565_test.go b/src/pkg/dashboard/ownership_11565_test.go index ff54edd64e..d695ae7be5 100644 --- a/src/pkg/dashboard/ownership_11565_test.go +++ b/src/pkg/dashboard/ownership_11565_test.go @@ -502,8 +502,8 @@ func TestOwnershipSafetyNet11565(t *testing.T) { } func TestOwnershipFixedSets11565(t *testing.T) { - if len(ownershipReasonVerbs) != 35+len(operatorWaitReasonVerbs) { - t.Fatalf("reason codes = %d, want the 20 of #11507, review_requires_human (#12329), blocked_unexplained (#12376), the three review-hold codes of #12377, the three PR gates of #12378, superseded_recommended_close (#13114), superseded_extra_files_review (#14176), adjudicator_handed_off (#13183), adopted_push_refused, the three issue-filter ADMIT codes and the run and approval waits (#12380)", len(ownershipReasonVerbs)) + if len(ownershipReasonVerbs) != 36+len(operatorWaitReasonVerbs) { + t.Fatalf("reason codes = %d, want the 20 of #11507, review_requires_human (#12329), blocked_unexplained (#12376), the three review-hold codes of #12377, the three PR gates of #12378, superseded_recommended_close (#13114), superseded_extra_files_review (#14176), adjudicator_handed_off (#13183), adopted_push_refused, the three issue-filter ADMIT codes, cited_lines_gone and the run and approval waits (#12380)", len(ownershipReasonVerbs)) } verbs := map[string]bool{} for _, verb := range ownershipVerbOrder { diff --git a/src/pkg/dashboard/ownership_cited_lines_gone_test.go b/src/pkg/dashboard/ownership_cited_lines_gone_test.go new file mode 100644 index 0000000000..e750ba7bab --- /dev/null +++ b/src/pkg/dashboard/ownership_cited_lines_gone_test.go @@ -0,0 +1,64 @@ +package dashboard + +import ( + "strings" + "testing" + "time" + + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/github" +) + +// An agent finding the issue filter refuses whose cited lines are gone from +// the default branch (RepoWorkIssue.CitedLinesGone, read by the enumeration) +// is CLOSE, not ADMIT or TRIAGE: projectbluefin/testsuite#1085 cited +// SKILL.md:163 for a fallback main no longer documented. +func TestCitedLinesGoneOffersClose(t *testing.T) { + l6 := ownershipEnv11565(6) + l6.selfAuthorizationHold = func(string) bool { return false } + l5 := ownershipEnv11565(5) + l5.selfAuthorizationHold = func(string) bool { return true } + gone := func(cite string, bucket, reason string, labels ...string) issueOwnershipInput { + in := nonActionable11565(bucket, reason, labels...) + in.citedLinesGone = cite + return in + } + closeIt := want11565{OwnershipVerbClose, OwnershipReasonCitedLinesGone} + for _, tc := range []struct { + name string + env ownershipEnv + in issueOwnershipInput + want want11565 + }{ + {"testsuite#1085 at L6", l6, gone("docs/skills/test-authoring/gnome/SKILL.md:163", repoWorkBucketFiltered, github.IssueFilterReason, "agent/reviewer"), closeIt}, + {"at L5 it is CLOSE rather than TRIAGE", l5, gone("scripts/vm_validation_plan.py:108", repoWorkBucketFiltered, github.IssueFilterReason, "agent/reviewer"), closeIt}, + {"a park keeps its ask", l6, gone("a.md:5", repoWorkBucketFiltered, config.DefaultNeedsDecisionLabel, "agent/reviewer", "stage/045-needs-decision"), want11565{OwnershipVerbDecide, OwnershipReasonParkedNeedsDecision}}, + {"verified done stays verified done", l6, gone("a.md:5", repoWorkBucketFiltered, github.IssueFilterReason, "agent/reviewer", "hive/already-done"), want11565{OwnershipVerbClose, OwnershipReasonVerifiedDone}}, + {"lines still on main: unchanged", l5, gone("", repoWorkBucketFiltered, github.IssueFilterReason, "agent/reviewer"), want11565{OwnershipVerbTriage, OwnershipReasonAgentFiledUnacknowledged}}, + } { + t.Run(tc.name, func(t *testing.T) { + checkOwnership11565(t, decideIssueOwnership(tc.in, tc.env), tc.want) + }) + } + got := decideIssueOwnership(gone("docs/x.md:163", repoWorkBucketFiltered, github.IssueFilterReason, "agent/reviewer"), l6) + if !strings.Contains(got.Reason, "no longer match main (docs/x.md:163)") { + t.Fatalf("reason %q must name the citation", got.Reason) + } +} + +// End to end from the enumeration row the dashboard stores. +func TestRepoOwnershipClosesStaleFindingRow(t *testing.T) { + now := ownershipNow11565 + env := ownershipEnv11565(6) + env.selfAuthorizationHold = func(string) bool { return false } + repos := []FrontendRepo{{Name: "testsuite", Full: "projectbluefin/testsuite", NonActionableIssues: []any{ + github.RepoWorkIssue{Repo: "projectbluefin/testsuite", Number: 1085, Labels: []string{"agent/reviewer"}, + Bucket: repoWorkBucketFiltered, Reason: github.IssueFilterReason, CitedLinesGone: "docs/skills/test-authoring/gnome/SKILL.md:163", + CreatedAt: now.Add(-24 * time.Hour), UpdatedAt: now.Add(-time.Hour)}, + }}} + decideRepoOwnership(repos, config.DashboardIssueBandsConfig{}, env) + d := ownershipFor(repos[0].Ownership, ownershipKindIssue, 1085) + if d == nil || !d.YourTurn || d.Verb != OwnershipVerbClose || d.ReasonCode != OwnershipReasonCitedLinesGone { + t.Fatalf("#1085 = %+v, want CLOSE cited_lines_gone", d) + } +} diff --git a/src/pkg/github/client.go b/src/pkg/github/client.go index 07dd75c3be..5581baa76a 100644 --- a/src/pkg/github/client.go +++ b/src/pkg/github/client.go @@ -140,6 +140,8 @@ type Client struct { // the installation's answer (installationGrantsWorkflows). workflowsSelfPush func(agent string) bool workflowsGrant workflowsGrantCache + // citedLines caches the stale-finding check (issue_cited_lines.go). + citedLines citedCache // reporterConfirmationEnabled restores the legacy close gate for all // human-filed bug-family issues when configured. Nil/false means only // per-issue opt-in markers activate the gate. @@ -1981,6 +1983,8 @@ func (c *Client) splitRepo(repo string) (owner, repoName string) { func (c *Client) fetchIssues(ctx context.Context, repo string, now time.Time, reporterTrustWaitBudget *reporterTrustWaitBudget) (actionable []Issue, held []HoldItem, totalIssues int, breakdown RepoIssueBreakdown, details []RepoWorkIssue, err error) { issueFilter := c.getIssueFilter() owner, repoName := c.splitRepo(repo) + // Stale-finding checks this pass may refresh (issue_cited_lines.go). + citedBudget := citedMaxChecksPerRepo // Taken before the list call: a hold recorded after the listing snapshot // must outlive this pass's prune of the local hold-source record. passStart := time.Now().UTC() @@ -2145,7 +2149,9 @@ func (c *Client) fetchIssues(ctx context.Context, repo string, now time.Time, re if !issueFilter.Admits(labels) { breakdown.Filtered++ - details = append(details, c.workIssue(repo, issue, labels, "filtered", issueFilterReason)) + row := c.workIssue(repo, issue, labels, "filtered", issueFilterReason) + row.CitedLinesGone = c.citedLinesGone(ctx, owner, repoName, issue, labels, &citedBudget) + details = append(details, row) continue } @@ -5106,6 +5112,11 @@ type RepoWorkIssue struct { Reason string `json:"reason,omitempty"` // ReporterConfirmationGate mirrors Issue.ReporterConfirmationGate. ReporterConfirmationGate bool `json:"reporter_confirmation_gate,omitempty"` + // CitedLinesGone names the citation ("path:163") of an agent finding the + // issue filter refuses when every permalink it cites is gone from the + // default branch (issue_cited_lines.go); "" when any is still there or + // it was not checked. + CitedLinesGone string `json:"cited_lines_gone,omitempty"` // ClaimKind, ClaimedBy and ClaimPRStarted are Issue.ClaimKind / // Issue.ClaimedBy / Issue.ClaimPRStarted, copied for the issue stage // reconciler. diff --git a/src/pkg/github/issue_cited_lines.go b/src/pkg/github/issue_cited_lines.go new file mode 100644 index 0000000000..cba41047a0 --- /dev/null +++ b/src/pkg/github/issue_cited_lines.go @@ -0,0 +1,299 @@ +package github + +import ( + "context" + "errors" + "net/http" + "regexp" + "strconv" + "strings" + "sync" + "time" + + gh "github.com/google/go-github/v72/github" +) + +// Stale agent findings (hivecommons/hive#16831 follow-up). An agent's finding +// cites the code it is about with a permalink pinned to a commit: the review +// backlog writes `Evidence: \`path:163\` (https://github.com/o/r/blob//path#L163)`. +// When those lines are gone from the default branch, the finding no longer +// describes main, and the dashboard offers CLOSE for it instead of ADMIT +// (projectbluefin/testsuite#1085 cited SKILL.md:163, a fallback the docs no +// longer described). Ownership reads no GitHub, so the enumeration records +// the result on the row (RepoWorkIssue.CitedLinesGone). + +// citedPermalinkRE is a same-host permalink to a file at a full commit SHA, +// with a line or a line range. +var citedPermalinkRE = regexp.MustCompile(`https://github\.com/([A-Za-z0-9_.-]+)/([A-Za-z0-9_.-]+)/blob/([0-9a-f]{40})/([^\s#)>\]]+)#L(\d+)(?:-L(\d+))?`) + +const ( + // citedMaxRefs bounds the citations read per issue. + citedMaxRefs = 3 + // citedMaxSpan bounds the lines read per citation. + citedMaxSpan = 20 + // citedMinLine is the shortest trimmed line that counts as evidence: + // the park re-check's threshold (parkLandedMinLine), so a brace or + // `return nil` never decides. + citedMinLine = parkLandedMinLine + // citedTTL is how long a result is trusted before main is read again. + citedTTL = 6 * time.Hour + // citedMaxChecksPerRepo bounds the refreshes one enumeration pass makes + // per repository; the rest keep their cached result (or none) until a + // later pass. + citedMaxChecksPerRepo = 3 + // citedMaxLine is the longest line kept as evidence: a minified or + // generated line is not a citation anyone reads, and it bounds what the + // evidence cache holds. + citedMaxLine = 400 + // citedMaxEvidenceEntries bounds the pinned-evidence cache; when full it + // starts over (the entries are only an optimisation: each is one read). + citedMaxEvidenceEntries = 1024 +) + +// citedRef is one permalink citation. +type citedRef struct { + path string + sha string + first, last int +} + +func (r citedRef) display() string { + if r.last > r.first { + return r.path + ":" + strconv.Itoa(r.first) + "-" + strconv.Itoa(r.last) + } + return r.path + ":" + strconv.Itoa(r.first) +} + +// citedRefs returns the permalinks in body that point into owner/repo, first +// citedMaxRefs, deduplicated. +func citedRefs(body, owner, repo string) []citedRef { + var out []citedRef + seen := map[string]bool{} + for _, m := range citedPermalinkRE.FindAllStringSubmatch(body, -1) { + if !strings.EqualFold(m[1], owner) || !strings.EqualFold(m[2], repo) { + continue + } + first, err := strconv.Atoi(m[5]) + if err != nil || first < 1 { + continue + } + last := first + if m[6] != "" { + if n, err := strconv.Atoi(m[6]); err == nil && n >= first { + last = n + } + } + if last-first+1 > citedMaxSpan { + last = first + citedMaxSpan - 1 + } + path := m[4] + // GitHub's line permalinks on rendered files carry a query + // (`SKILL.md?plain=1#L163`); the path ends before it. + if i := strings.IndexByte(path, '?'); i >= 0 { + path = path[:i] + } + if path == "" { + continue + } + r := citedRef{path: path, sha: m[3], first: first, last: last} + if seen[r.sha+r.path+r.display()] { + continue + } + seen[r.sha+r.path+r.display()] = true + out = append(out, r) + if len(out) == citedMaxRefs { + break + } + } + return out +} + +// citedEvidenceLines returns the trimmed lines first..last of content that +// are long enough to be evidence. +func citedEvidenceLines(content string, first, last int) []string { + lines := strings.Split(content, "\n") + var out []string + for i := first; i <= last && i <= len(lines); i++ { + if l := strings.TrimSpace(lines[i-1]); len(l) >= citedMinLine && len(l) <= citedMaxLine { + out = append(out, l) + } + } + return out +} + +// citedLinesStillPresent reports whether any evidence line is still a line of +// main's content. +func citedLinesStillPresent(evidence []string, main string) bool { + have := map[string]bool{} + for _, l := range strings.Split(main, "\n") { + have[strings.TrimSpace(l)] = true + } + for _, l := range evidence { + if have[l] { + return true + } + } + return false +} + +type citedResult struct { + at time.Time + gone string +} + +// citedCache holds per-issue results and the evidence lines read at a +// pinned SHA. Pinned content cannot change, so its evidence (none, for a +// path the pinned commit does not have or lines too short to count) is +// settled once; only the lines are kept, never the file. +type citedCache struct { + mu sync.Mutex + results map[string]citedResult + evidence map[string][]string +} + +func citedEvidenceKey(owner, repo string, ref citedRef) string { + return owner + "/" + repo + "@" + ref.sha + ":" + ref.display() +} + +// cachedEvidence returns the settled evidence for ref, and whether it is +// settled. +func (c *Client) cachedEvidence(owner, repo string, ref citedRef) ([]string, bool) { + c.citedLines.mu.Lock() + defer c.citedLines.mu.Unlock() + ev, ok := c.citedLines.evidence[citedEvidenceKey(owner, repo, ref)] + return ev, ok +} + +func (c *Client) settleEvidence(owner, repo string, ref citedRef, ev []string) { + c.citedLines.mu.Lock() + defer c.citedLines.mu.Unlock() + if c.citedLines.evidence == nil || len(c.citedLines.evidence) >= citedMaxEvidenceEntries { + c.citedLines.evidence = map[string][]string{} + } + c.citedLines.evidence[citedEvidenceKey(owner, repo, ref)] = ev +} + +var errCitedNotFound = errors.New("not found") + +// readCitedFile reads path at ref ("" is the default branch); a 404 is +// errCitedNotFound. +func (c *Client) readCitedFile(ctx context.Context, owner, repo, path, ref string) (string, error) { + var opts *gh.RepositoryContentGetOptions + if ref != "" { + opts = &gh.RepositoryContentGetOptions{Ref: ref} + } + fc, _, resp, err := c.client.Repositories.GetContents(ctx, owner, repo, path, opts) + if err != nil { + if resp != nil && resp.StatusCode == http.StatusNotFound { + return "", errCitedNotFound + } + return "", err + } + if fc == nil { + return "", errCitedNotFound + } + return fc.GetContent() +} + +// citedRefGone reports whether ref's evidence is gone from the default +// branch. settled is false only for a read that failed and may succeed on a +// later pass. Everything else is a settled answer: +// - no evidence (the pinned commit has no such path, or the cited lines are +// too short or too long to count) is "not gone"; +// - a file the default branch no longer has is "not gone" as well: a +// rename or move keeps the cited lines somewhere, and finding where would +// need a search the check does not spend; +// - otherwise the evidence is gone when none of its lines is a line of the +// default branch's file. +func (c *Client) citedRefGone(ctx context.Context, owner, repo string, ref citedRef) (gone, settled bool) { + evidence, ok := c.cachedEvidence(owner, repo, ref) + if !ok { + content, err := c.readCitedFile(ctx, owner, repo, ref.path, ref.sha) + switch { + case errors.Is(err, errCitedNotFound): + evidence = nil + case err != nil: + return false, false + default: + evidence = citedEvidenceLines(content, ref.first, ref.last) + } + c.settleEvidence(owner, repo, ref, evidence) + } + if len(evidence) == 0 { + return false, true + } + main, err := c.readCitedFile(ctx, owner, repo, ref.path, "") + if errors.Is(err, errCitedNotFound) { + return false, true + } + if err != nil { + return false, false + } + return !citedLinesStillPresent(evidence, main), true +} + +// citedLinesGone is the enumeration's read for one agent finding the issue +// filter refuses: the citation it names ("path:163") when every permalink the +// body cites is gone from the default branch, else "". Results are cached for +// citedTTL; budget counts the refreshes left for this repository in this +// pass, and a row past it keeps its cached result. +func (c *Client) citedLinesGone(ctx context.Context, owner, repo string, issue *gh.Issue, labels []string, budget *int) string { + if issue == nil || !hasAgentLaneLabel(labels) { + return "" + } + refs := citedRefs(issue.GetBody(), owner, repo) + if len(refs) == 0 { + return "" + } + key := owner + "/" + repo + "#" + strconv.Itoa(issue.GetNumber()) + c.citedLines.mu.Lock() + cached, ok := c.citedLines.results[key] + c.citedLines.mu.Unlock() + if ok && time.Since(cached.at) < citedTTL { + return cached.gone + } + // A citation whose pinned evidence is already settled empty can never + // be gone, so neither can the issue: settle it without a read. + for _, ref := range refs { + if ev, settled := c.cachedEvidence(owner, repo, ref); settled && len(ev) == 0 { + c.storeCitedResult(key, "") + return "" + } + } + if budget == nil || *budget <= 0 { + if ok { + return cached.gone + } + return "" + } + *budget-- + gone := "" + for _, ref := range refs { + isGone, settled := c.citedRefGone(ctx, owner, repo, ref) + if !settled { + // Unreadable: try again on a later pass, keep what was known. + if ok { + return cached.gone + } + return "" + } + if !isGone { + gone = "" + break + } + if gone == "" { + gone = ref.display() + } + } + c.storeCitedResult(key, gone) + return gone +} + +func (c *Client) storeCitedResult(key, gone string) { + c.citedLines.mu.Lock() + defer c.citedLines.mu.Unlock() + if c.citedLines.results == nil { + c.citedLines.results = map[string]citedResult{} + } + c.citedLines.results[key] = citedResult{at: time.Now(), gone: gone} +} diff --git a/src/pkg/github/issue_cited_lines_test.go b/src/pkg/github/issue_cited_lines_test.go new file mode 100644 index 0000000000..50a693f221 --- /dev/null +++ b/src/pkg/github/issue_cited_lines_test.go @@ -0,0 +1,297 @@ +package github + +import ( + "context" + "encoding/base64" + "encoding/json" + "net/http" + "net/http/httptest" + "strconv" + "strings" + "sync" + "testing" + + gh "github.com/google/go-github/v72/github" + + "github.com/hivecommons/hive/pkg/config" +) + +const ( + citedOldSHA = "20f3aa8c15968c434ad2428280fac614d907df6d" + citedFile = "docs/skills/test-authoring/gnome/SKILL.md" +) + +// citedBody is projectbluefin/testsuite#1085's evidence line. +func citedBody(org, repo, path string, line int) string { + return "Filed from an out-of-scope finding in the Hive review of PR #1081.\n\n" + + "Evidence: `" + path + ":" + strconv.Itoa(line) + "` (https://github.com/" + org + "/" + repo + "/blob/" + citedOldSHA + "/" + path + "#L" + strconv.Itoa(line) + ")\n" + + "Finding: SKILL.md still documents a Firefox per-name tree.root.application() fallback that no longer exists\n" +} + +func citedIssue(n int, body string) *gh.Issue { + return &gh.Issue{Number: gh.Ptr(n), Body: gh.Ptr(body)} +} + +func TestCitedRefsParsesPermalinks(t *testing.T) { + body := citedBody("projectbluefin", "testsuite", citedFile, 163) + + "also https://github.com/projectbluefin/testsuite/blob/" + citedOldSHA + "/scripts/vm_validation_plan.py#L108-L110\n" + + "elsewhere https://github.com/other/repo/blob/" + citedOldSHA + "/x.go#L1\n" + + "a branch link https://github.com/projectbluefin/testsuite/blob/main/README.md#L3\n" + + "a huge range https://github.com/projectbluefin/testsuite/blob/" + citedOldSHA + "/big.txt#L10-L500\n" + + "a UI permalink https://github.com/projectbluefin/testsuite/blob/" + citedOldSHA + "/docs/guide.md?plain=1#L42\n" + refs := citedRefs(body, "projectbluefin", "testsuite") + var got []string + for _, r := range refs { + got = append(got, r.display()) + } + want := []string{citedFile + ":163", "scripts/vm_validation_plan.py:108-110", "big.txt:10-29"} + // The UI permalink is past the cap above; parse it on its own. + ui := citedRefs("https://github.com/projectbluefin/testsuite/blob/"+citedOldSHA+"/docs/guide.md?plain=1#L42", "projectbluefin", "testsuite") + if len(ui) != 1 || ui[0].path != "docs/guide.md" { + t.Fatalf("UI permalink path = %+v, want docs/guide.md without ?plain=1", ui) + } + if strings.Join(got, ",") != strings.Join(want, ",") { + t.Fatalf("refs = %v, want %v (same repo, pinned SHA, span capped, other repos and branch links ignored)", got, want) + } +} + +// citedRepo serves file contents at a pinned SHA and on the default branch, +// counting reads. +type citedRepo struct { + mu sync.Mutex + files map[string]string // ":"; ref "" is the default branch + reads int + fail bool +} + +func (cr *citedRepo) handle(w http.ResponseWriter, r *http.Request) { + cr.mu.Lock() + cr.reads++ + fail := cr.fail + path := r.URL.Path[strings.Index(r.URL.Path, "/contents/")+len("/contents/"):] + content, ok := cr.files[r.URL.Query().Get("ref")+":"+path] + cr.mu.Unlock() + if fail { + w.WriteHeader(http.StatusBadGateway) + return + } + if !ok { + w.WriteHeader(http.StatusNotFound) + _, _ = w.Write([]byte(`{"message":"Not Found"}`)) + return + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]any{"type": "file", "encoding": "base64", "path": path, "content": base64.StdEncoding.EncodeToString([]byte(content))}) +} + +func fileWithLine(n int, line string) string { + lines := make([]string, n+5) + for i := range lines { + lines[i] = "filler line number " + strconv.Itoa(i+1) + " of the skill guide" + } + lines[n-1] = line + return strings.Join(lines, "\n") +} + +// projectbluefin/testsuite#1085's shape end to end through the enumeration: +// the agent finding the issue filter refuses records the citation main no +// longer has; a finding whose line is still there, a person's issue and an +// admitted issue record nothing. +func TestEnumerationRecordsCitedLinesGone(t *testing.T) { + const org, repo = "testorg", "testrepo" + cited := "`_firefox_app()` falls back to per-name tree.root.application() lookups" + issues := []wireIssue{ + {Number: 1085, Title: "[reviewer] stale doc", User: wireUser{"hive[bot]"}, Labels: []wireLabel{{Name: "agent/reviewer"}}, Body: citedBody(org, repo, citedFile, 163), CreatedAt: hoursAgo(5)}, + {Number: 1086, Title: "[reviewer] still true", User: wireUser{"hive[bot]"}, Labels: []wireLabel{{Name: "agent/reviewer"}}, Body: citedBody(org, repo, "scripts/orca_steps.py", 12), CreatedAt: hoursAgo(4)}, + {Number: 1087, Title: "[reviewer] file renamed on main", User: wireUser{"hive[bot]"}, Labels: []wireLabel{{Name: "agent/reviewer"}}, Body: citedBody(org, repo, "scripts/gone.py", 3), CreatedAt: hoursAgo(3)}, + {Number: 1088, Title: "a person's", User: wireUser{"erin"}, Body: citedBody(org, repo, citedFile, 163), CreatedAt: hoursAgo(2)}, + {Number: 1089, Title: "admitted finding", User: wireUser{"hive[bot]"}, Labels: []wireLabel{{Name: "agent/reviewer"}, {Name: "triage/accepted"}}, Body: citedBody(org, repo, citedFile, 163), CreatedAt: hoursAgo(1)}, + } + cr := &citedRepo{files: map[string]string{ + citedOldSHA + ":" + citedFile: fileWithLine(163, cited), + ":" + citedFile: fileWithLine(170, "`_firefox_app()` uses the shared accessible lookup only"), + citedOldSHA + ":scripts/orca_steps.py": fileWithLine(12, "def press_key_combination(context, combo):"), + ":scripts/orca_steps.py": fileWithLine(40, "def press_key_combination(context, combo):"), + citedOldSHA + ":scripts/gone.py": fileWithLine(3, "ASSERT_PASSED_LANE = ('kde-smoke', GUI)"), + }} + mux := buildMux(t, org, repo, issues, nil) + mux.HandleFunc("/repos/"+org+"/"+repo+"/contents/", cr.handle) + server := httptest.NewServer(mux) + t.Cleanup(server.Close) + c := newTestClient(t, server, org, []string{repo}) + c.SetIssueFilter(config.IssueFilterConfig{RequireLabels: []string{"triage/accepted"}}) + + gone := func() map[int]string { + result, err := c.EnumerateActionable(context.Background()) + if err != nil { + t.Fatalf("EnumerateActionable: %v", err) + } + out := map[int]string{} + for _, d := range result.WorkDetailsByRepo[repo].Issues { + out[d.Number] = d.CitedLinesGone + } + return out + } + got := gone() + // #1087's file is gone from main: a rename or move keeps the lines + // somewhere, so it is not offered CLOSE. + want := map[int]string{1085: citedFile + ":163", 1086: "", 1087: "", 1088: ""} + for n, w := range want { + if got[n] != w { + t.Errorf("#%d CitedLinesGone = %q, want %q", n, got[n], w) + } + } + if _, ok := got[1089]; ok { + t.Errorf("#1089 is admitted and has no work row: %v", got) + } + first := cr.reads + if first == 0 || first > 2*citedMaxChecksPerRepo { + t.Fatalf("reads = %d, want at most two per checked finding", first) + } + // A second pass within the TTL reads nothing. + if again := gone(); again[1085] != want[1085] || cr.reads != first { + t.Fatalf("second pass: #1085 = %q, reads %d → %d; want the cached result and no reads", again[1085], first, cr.reads) + } +} + +func TestCitedLinesGoneBudgetAndFailures(t *testing.T) { + const org, repo = "o", "r" + cr := &citedRepo{files: map[string]string{ + citedOldSHA + ":" + citedFile: fileWithLine(163, "a line long enough to count as evidence here"), + ":" + citedFile: "rewritten entirely", + }} + mux := http.NewServeMux() + mux.HandleFunc("/repos/"+org+"/"+repo+"/contents/", cr.handle) + server := httptest.NewServer(mux) + t.Cleanup(server.Close) + c := newTestClient(t, server, org, []string{repo}) + finding := func(n int) *gh.Issue { return citedIssue(n, citedBody(org, repo, citedFile, 163)) } + agent := []string{"agent/reviewer"} + + budget := 1 + if g := c.citedLinesGone(context.Background(), org, repo, finding(1), agent, &budget); g != citedFile+":163" || budget != 0 { + t.Fatalf("first = %q budget %d", g, budget) + } + reads := cr.reads + if g := c.citedLinesGone(context.Background(), org, repo, finding(2), agent, &budget); g != "" || cr.reads != reads { + t.Fatalf("over budget: %q with %d new reads, want nothing read", g, cr.reads-reads) + } + budget = 5 + if g := c.citedLinesGone(context.Background(), org, repo, finding(3), nil, &budget); g != "" || budget != 5 { + t.Fatalf("not an agent finding: %q budget %d, want no check", g, budget) + } + // An unreadable main is not "gone", and is not cached as an answer. + cr.mu.Lock() + cr.fail = true + cr.mu.Unlock() + if g := c.citedLinesGone(context.Background(), org, repo, finding(4), agent, &budget); g != "" { + t.Fatalf("read failure = %q, want unknown", g) + } + cr.mu.Lock() + cr.fail = false + cr.mu.Unlock() + if g := c.citedLinesGone(context.Background(), org, repo, finding(4), agent, &budget); g != citedFile+":163" { + t.Fatalf("after the failure = %q, want it re-checked", g) + } + // Cited lines too short to be evidence decide nothing, and settle: once + // read, later passes (even past the TTL) spend no budget and no read. + cr.mu.Lock() + cr.files[citedOldSHA+":short.go"] = fileWithLine(2, "}") + cr.files[":short.go"] = "rewritten" + cr.mu.Unlock() + short := citedIssue(5, "https://github.com/o/r/blob/"+citedOldSHA+"/short.go#L2") + if g := c.citedLinesGone(context.Background(), org, repo, short, agent, &budget); g != "" { + t.Fatalf("short evidence = %q, want unknown", g) + } + // A path the pinned commit does not have settles the same way. + missing := citedIssue(6, "https://github.com/o/r/blob/"+citedOldSHA+"/never.go#L9") + if g := c.citedLinesGone(context.Background(), org, repo, missing, agent, &budget); g != "" { + t.Fatalf("pinned 404 = %q, want unknown", g) + } + c.citedLines.mu.Lock() + for k, r := range c.citedLines.results { + r.at = r.at.Add(-2 * citedTTL) + c.citedLines.results[k] = r + } + c.citedLines.mu.Unlock() + reads, before := cr.reads, budget + for _, iss := range []*gh.Issue{short, missing} { + if g := c.citedLinesGone(context.Background(), org, repo, iss, agent, &budget); g != "" { + t.Fatalf("settled unknown = %q", g) + } + } + if cr.reads != reads || budget != before { + t.Fatalf("settled unknowns cost %d reads and %d budget, want none", cr.reads-reads, before-budget) + } + // The cache keeps the evidence lines, never a file. + for k, ev := range c.citedLines.evidence { + if len(ev) > citedMaxSpan { + t.Fatalf("evidence cache %s holds %d lines", k, len(ev)) + } + } +} + +// A file present at the pinned SHA and gone from main (a rename or move) is +// unknown, not gone, and settled like one. +func TestCitedLinesRenamedFileIsNotGone(t *testing.T) { + const org, repo = "o", "r" + cr := &citedRepo{files: map[string]string{ + citedOldSHA + ":old/name.md": fileWithLine(4, "the cited sentence that moved with its file"), + ":new/name.md": fileWithLine(4, "the cited sentence that moved with its file"), + }} + mux := http.NewServeMux() + mux.HandleFunc("/repos/"+org+"/"+repo+"/contents/", cr.handle) + server := httptest.NewServer(mux) + t.Cleanup(server.Close) + c := newTestClient(t, server, org, []string{repo}) + budget := 3 + issue := citedIssue(1, "https://github.com/o/r/blob/"+citedOldSHA+"/old/name.md#L4") + if g := c.citedLinesGone(context.Background(), org, repo, issue, []string{"agent/reviewer"}, &budget); g != "" { + t.Fatalf("renamed file = %q, want not gone", g) + } + reads := cr.reads + if g := c.citedLinesGone(context.Background(), org, repo, issue, []string{"agent/reviewer"}, &budget); g != "" || cr.reads != reads { + t.Fatalf("second pass = %q with %d reads, want the cached answer", g, cr.reads-reads) + } +} + +// The evidence cache is bounded. +func TestCitedEvidenceCacheIsBounded(t *testing.T) { + c := &Client{} + for i := 0; i < citedMaxEvidenceEntries+10; i++ { + c.settleEvidence("o", "r", citedRef{path: "f" + strconv.Itoa(i), sha: citedOldSHA, first: 1, last: 1}, []string{"a line long enough to be evidence"}) + } + if n := len(c.citedLines.evidence); n > citedMaxEvidenceEntries { + t.Fatalf("evidence entries = %d, want at most %d", n, citedMaxEvidenceEntries) + } +} + +// Two citations: CLOSE only when every one is gone. +func TestCitedLinesGoneNeedsEveryCitationGone(t *testing.T) { + const org, repo = "o", "r" + cr := &citedRepo{files: map[string]string{ + citedOldSHA + ":a.md": fileWithLine(5, "the first cited sentence that moved away"), + ":a.md": "nothing like it", + citedOldSHA + ":b.md": fileWithLine(7, "the second cited sentence that is still here"), + ":b.md": fileWithLine(9, "the second cited sentence that is still here"), + }} + mux := http.NewServeMux() + mux.HandleFunc("/repos/"+org+"/"+repo+"/contents/", cr.handle) + server := httptest.NewServer(mux) + t.Cleanup(server.Close) + c := newTestClient(t, server, org, []string{repo}) + body := "https://github.com/o/r/blob/" + citedOldSHA + "/a.md#L5 and https://github.com/o/r/blob/" + citedOldSHA + "/b.md#L7" + budget := 3 + if g := c.citedLinesGone(context.Background(), org, repo, citedIssue(1, body), []string{"agent/quality"}, &budget); g != "" { + t.Fatalf("one citation still on main: %q, want nothing", g) + } +} + +// Evidence is a readable line: a brace is too short, a minified line too +// long; both bound what the cache keeps. +func TestCitedEvidenceLinesBounds(t *testing.T) { + content := strings.Join([]string{"}", "a line long enough to be evidence", strings.Repeat("x", citedMaxLine+1)}, "\n") + if got := citedEvidenceLines(content, 1, 3); len(got) != 1 || got[0] != "a line long enough to be evidence" { + t.Fatalf("evidence = %q, want only the readable line", got) + } +} diff --git a/src/pkg/hiveadvisor/human_gate.go b/src/pkg/hiveadvisor/human_gate.go index 5898b843ea..6043232773 100644 --- a/src/pkg/hiveadvisor/human_gate.go +++ b/src/pkg/hiveadvisor/human_gate.go @@ -62,6 +62,7 @@ var gateReasons = map[string]gateReason{ "decided_not_admitted": {"decided by a maintainer but kept off the agent queues by the issue filter", "add the admission label to the decided issues in %s"}, "agent_finding_not_admitted": {"agent findings the issue filter keeps off the agent queues", "admit or close the agent findings in %s"}, "person_issue_not_admitted": {"filed by people and kept off the agent queues by the issue filter", "admit or close the people's issues in %s"}, + "cited_lines_gone": {"agent findings whose cited lines are gone from main", "close the agent findings main no longer matches in %s"}, "pr_gate_label": {"held by a gate label", "act on the gate labels or remove them"}, "hand_merge_required": {"that only a person can merge", "merge the reviewed PRs by hand in %s"}, "review_decision_required": {"waiting for an approving review that branch protection requires", "approve PRs in %s"},