Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/fixed-18608-prow-refusal-repo-memory.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Remember per repository that Prow refuses the App's `/ok-to-test`, so a new fork PR there gets the needs-human note instead of a first command Prow rejects; the memory is re-derived from the repository's comments after a restart and clears when the `hive-ok-to-test` workflow is installed (#18608).
3 changes: 3 additions & 0 deletions src/docs/fork-pr-presubmit-stall-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ ask that owner to comment `/approve`.
admitted non-member fork PR. Making the hive App (or contributor agents) a
Prow trusted user remains a Prow configuration choice; until then Prow
rejects the App's command and step 2 applies.
- Done in #18608: once Prow has refused the App anywhere in a repository, the
lane skips the first `/ok-to-test` on later fork PRs there and leaves the
needs-human note directly; step 2 applies to those PRs too.
- Relay `/lgtm` and `/approve` with the approving human's token, or add the
hive App to `OWNERS`; surface Prow's rejection reply on the board instead of
showing "approved".
Expand Down
12 changes: 11 additions & 1 deletion src/docs/hive-merge.md
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,17 @@ mutation-boundary dedupe as the trusted-lane command. If Prow replies
`skipped=prow-rejects-bot-commands`, leaves one
`<!-- hive-ok-to-test-rejected -->` needs-human note and never posts the
command on that PR again; making the App a Prow trusted user is the
repository's Prow configuration. The merge-request relay's read-only
repository's Prow configuration. Prow's answer is the repository's, so it is
also remembered per repository (hivecommons/hive#18608): once Prow has refused
the App anywhere in the repository, the lane skips the first command on every
later fork PR there and leaves the `<!-- hive-ok-to-test-rejected -->` note
directly. The memory is not stored: after a restart it is re-derived from one
bounded read of the repository's comments of the last 30 days (Prow's refusal
addressed to the App, or the hive's own note quoting it), kept for an hour, and
cleared while the repository carries `.github/workflows/hive-ok-to-test.yml`.
A repository with no refusal on record behaves as before. The note in that case
does not quote Prow's wording, so only a real refusal renews the memory and it
ages out if Prow later trusts the App. The merge-request relay's read-only
evaluation posts nothing.

A PR merges through this tier only when all of the following hold:
Expand Down
20 changes: 20 additions & 0 deletions src/pkg/github/automerge/automerge_contributor_lane_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"github.com/hivecommons/hive/pkg/authorkind"
hgithub "github.com/hivecommons/hive/pkg/github"
"github.com/hivecommons/hive/pkg/intent"
"github.com/hivecommons/hive/pkg/oktotest"
"github.com/hivecommons/hive/pkg/review"
hreview "github.com/hivecommons/hive/pkg/review"
"github.com/hivecommons/hive/pkg/stageengine"
Expand Down Expand Up @@ -117,6 +118,12 @@ type contributorLaneFixture struct {
// headStatuses is what GET commits/sha7/statuses (every commit status,
// with its creator) serves; nil serves an empty list.
headStatuses []map[string]any
// repoComments is what GET issues/comments (the repository-wide comment
// listing the Prow refusal memory reads) serves; repoCommentLists counts
// those reads. workflowInstalled serves the hive-ok-to-test workflow file.
repoComments []map[string]any
repoCommentLists int
workflowInstalled bool
// baseRef, when set, gives the PR base a branch name and serves the
// branch, protection and compare routes the current-base gate
// (automerge_current_base.go) and release hold need; empty leaves the
Expand Down Expand Up @@ -336,6 +343,19 @@ func newContributorLaneAPI(t *testing.T, fx *contributorLaneFixture) *httptest.S
json.NewEncoder(w).Encode(fx.issueComments)
case r.Method == http.MethodGet && r.URL.Path == "/repos/acme/widget/issues/7/comments":
json.NewEncoder(w).Encode(fx.prComments)
case r.Method == http.MethodGet && r.URL.Path == "/repos/acme/widget/issues/comments":
fx.repoCommentLists++
repoComments := fx.repoComments
if repoComments == nil {
repoComments = []map[string]any{}
}
json.NewEncoder(w).Encode(repoComments)
case r.Method == http.MethodGet && r.URL.Path == "/repos/acme/widget/contents/"+oktotest.WorkflowPath:
if !fx.workflowInstalled {
http.NotFound(w, r)
return
}
json.NewEncoder(w).Encode(map[string]any{"type": "file", "name": "hive-ok-to-test.yml", "path": oktotest.WorkflowPath})
case r.Method == http.MethodGet && r.URL.Path == "/repos/acme/widget/pulls/7/files":
if fx.filesStatus != 0 {
http.Error(w, "files error", fx.filesStatus)
Expand Down
17 changes: 16 additions & 1 deletion src/pkg/github/automerge/automerge_contributor_prow_oktotest.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,9 @@ var prowTrustedAssociations = map[string]bool{
// a fork PR by a non-member on a Prow-managed repository (Prow's labels or its
// approve plugin's comment), carries no `ok-to-test` label, and Prow has
// reported no presubmit status on the head. A Prow rejection of the App's
// command is recorded once per PR and stops every later post. The caller
// command is recorded once per PR and stops every later post; it is also
// remembered per repository (automerge_prow_refusal_memory.go), so a later
// fork PR there gets the needs-human note without the command being posted. The caller
// must already have admitted the PR to the contributor lane.
func (c *Engine) requestProwPresubmitsForFork(ctx context.Context, displayRepo, owner, repo string, pr *gh.PullRequest, number int, headSHA string) {
if c == nil || c.gh == nil || pr == nil || strings.TrimSpace(headSHA) == "" || !c.isForkPR(pr) {
Expand All @@ -61,13 +63,15 @@ func (c *Engine) requestProwPresubmitsForFork(ctx context.Context, displayRepo,
if hasLabel(labels, prowOkToTestLabel) || c.workflowApprovalUntrustedKnown(owner, repo, number) {
return
}
knownRefusal := c.prowRefusesApp(ctx, displayRepo, owner, repo)
_, first := c.markWorkflowApprovalCommandSeen(owner, repo, number, headSHA)
state, scanned := c.workflowApprovalCommentState(ctx, displayRepo, owner, repo, number, headSHA, first)
if !scanned {
return
}
if state.rejected {
c.markWorkflowApprovalUntrusted(owner, repo, number)
c.rememberProwRefusal(owner, repo)
args := []any{"repo", displayRepo, "pr", number, "sha", headSHA, "skipped", hgithub.PRApprovalCommandsSkippedProw}
const msg = "contributor-lane ok-to-test not posted again: Prow rejected the App's command on this fork PR; a trusted user must post it"
if state.rejectionNoted {
Expand All @@ -84,6 +88,17 @@ func (c *Engine) requestProwPresubmitsForFork(ctx context.Context, displayRepo,
if c.prowPresubmitStatusPresent(ctx, displayRepo, owner, repo, number, headSHA) {
return
}
if knownRefusal {
// Prow already refused the App in this repository: skip the first
// command and hand the PR to a person directly (hivecommons/hive#18608).
c.markWorkflowApprovalUntrusted(owner, repo, number)
if !state.rejectionNoted {
c.postWorkflowApprovalUntrustedNote(ctx, displayRepo, owner, repo, number, headSHA, prowKnownRefusalNote(strings.TrimSpace(c.transport.AppBotLogin())))
}
c.debug("contributor-lane ok-to-test not posted: Prow already refused the App's command on this repository; a trusted user must post it",
"repo", displayRepo, "pr", number, "sha", headSHA, "skipped", hgithub.PRApprovalCommandsSkippedProw)
return
}
c.postWorkflowApprovalCommand(ctx, displayRepo, owner, repo, number, headSHA)
}

Expand Down
3 changes: 3 additions & 0 deletions src/pkg/github/automerge/automerge_held_runs.go
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,9 @@ func (c *Engine) workflowApprovalAttention(ctx context.Context, displayRepo, own
// users: every further post gets the same rejection, so stop and
// hand the command to a person once.
c.markWorkflowApprovalUntrusted(owner, repo, number)
if state.rejected {
c.rememberProwRefusal(owner, repo)
}
if !state.rejectionNoted {
c.postWorkflowApprovalUntrustedNote(ctx, displayRepo, owner, repo, number, headSHA, workflowApprovalUntrustedNote(strings.TrimSpace(c.transport.AppBotLogin()), state.rejected))
}
Expand Down
151 changes: 151 additions & 0 deletions src/pkg/github/automerge/automerge_prow_refusal_memory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package automerge

// Repo-level memory that Prow refuses the App's /ok-to-test
// (hivecommons/hive#18608). Whether Prow trusts the App is the repository's
// Prow configuration, so one refusal on any PR answers the question for every
// later fork PR in that repository. The per-PR memory
// (workflowApprovalUntrusted) cannot: it starts empty for each new PR and for
// each restarted process, so the first /ok-to-test on every new fork PR drew
// Prow's "Cannot trigger testing" reply.
//
// The memory is a cache over the repository's own comments, so it survives a
// restart without a store: one repository-wide comment listing, bounded by
// prowRefusalLookback and prowRefusalMaxPages, looks for Prow's refusal
// addressed to the App or for the hive's own needs-human note that quotes it.
// The result, positive or negative, is kept for okToTestWorkflowCheckTTL. It
// clears when the repository carries the self-contained hive-ok-to-test
// workflow, which approves held runs whatever Prow thinks of the App.

import (
"context"
"strings"
"time"

gh "github.com/google/go-github/v72/github"
hgithub "github.com/hivecommons/hive/pkg/github"
)

const (
// prowRefusalLookback is how far back the repository's comments are read
// for evidence of a refusal.
prowRefusalLookback = 30 * 24 * time.Hour
// prowRefusalMaxPages bounds the quota one derivation may spend.
prowRefusalMaxPages = 3
// prowRefusalPageSize is the comments per page of that listing.
prowRefusalPageSize = 100
)

// prowRefusalMemo is one repository's remembered answer and when it was read.
type prowRefusalMemo struct {
at time.Time
refused bool
}

func prowRefusalKey(owner, repo string) string {
return strings.ToLower(owner + "/" + repo)
}

// rememberProwRefusal records that Prow refused the App's command in this
// repository, so no later PR here is asked first.
func (c *Engine) rememberProwRefusal(owner, repo string) {
c.storeProwRefusal(owner, repo, true)
}

func (c *Engine) storeProwRefusal(owner, repo string, refused bool) {
c.workflowApprovalCommandsMu.Lock()
defer c.workflowApprovalCommandsMu.Unlock()
if c.prowRefusals == nil {
c.prowRefusals = make(map[string]prowRefusalMemo)
}
c.prowRefusals[prowRefusalKey(owner, repo)] = prowRefusalMemo{at: c.clock(), refused: refused}
}

// prowRefusesApp reports whether Prow is known to refuse the App's
// /ok-to-test in this repository. A repository that carries the
// hive-ok-to-test workflow is never reported as refusing. Unknown, or a
// lookup that failed, reports false so the existing post-once behaviour holds.
func (c *Engine) prowRefusesApp(ctx context.Context, displayRepo, owner, repo string) bool {
key := prowRefusalKey(owner, repo)
c.workflowApprovalCommandsMu.Lock()
memo, ok := c.prowRefusals[key]
c.workflowApprovalCommandsMu.Unlock()
if !ok || c.clock().Sub(memo.at) >= okToTestWorkflowCheckTTL {
memo.refused = c.deriveProwRefusal(ctx, displayRepo, owner, repo)
c.storeProwRefusal(owner, repo, memo.refused)
}
if !memo.refused {
return false
}
if c.okToTestWorkflowInstalled(ctx, displayRepo, owner, repo) {
c.storeProwRefusal(owner, repo, false)
return false
}
return true
}

// deriveProwRefusal reads the repository's recent comments for a Prow refusal
// addressed to the App, or the hive's own note quoting one.
func (c *Engine) deriveProwRefusal(ctx context.Context, displayRepo, owner, repo string) bool {
appLogin := ""
if c.transport != nil {
appLogin = strings.TrimSpace(c.transport.AppBotLogin())
}
since := c.clock().Add(-prowRefusalLookback)
opts := &gh.IssueListCommentsOptions{
Sort: gh.Ptr("created"),
Direction: gh.Ptr("desc"),
Since: &since,
ListOptions: gh.ListOptions{PerPage: prowRefusalPageSize},
}
for page := 0; page < prowRefusalMaxPages; page++ {
comments, resp, err := c.gh.Issues.ListComments(hgithub.WithRESTCaller(ctx, "hive:automerge_sweep"), owner, repo, 0, opts)
if err != nil {
c.warn("automerge could not list repository comments for Prow ok-to-test refusals", "repo", displayRepo, "error", err)
return false
}
for _, comment := range comments {
if comment == nil || !strings.Contains(comment.GetBody(), prowOkToTestRejection) {
continue
}
body := comment.GetBody()
login := hgithub.SafeGetLogin(comment.GetUser())
if appLogin != "" && strings.EqualFold(login, appLogin) {
if strings.Contains(body, workflowApprovalRejectedMarker) {
return true
}
continue
}
if appLogin == "" || strings.Contains(strings.ToLower(body), "@"+strings.ToLower(appLogin)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any commenter can forge this. No author check, so a non-bot comment with the refusal phrase and @app flips the whole repo to skip /ok-to-test for 30 days. If AppBotLogin is empty, any comment with the phrase matches. Require a Bot author (ideally Prow's configured login) and drop the empty-login fallback.

Suggested change
if appLogin == "" || strings.Contains(strings.ToLower(body), "@"+strings.ToLower(appLogin)) {
if appLogin != "" && comment.GetUser().GetType() == "Bot" && strings.Contains(strings.ToLower(body), "@"+strings.ToLower(appLogin)) {

return true
}
}
if resp == nil || resp.NextPage == 0 {
break
}
opts.Page = resp.NextPage
}
return false
}

// prowKnownRefusalNote is the one needs-human comment posted on a fork PR
// when the repository is already known to refuse the App's command, instead
// of posting the command to be refused. It deliberately omits Prow's refusal
// wording: only a real refusal may renew the repository's memory, so the
// memory ages out if Prow later trusts the App.
func prowKnownRefusalNote(appLogin string) string {
if appLogin == "" {
appLogin = "the Hive App"
}
var b strings.Builder
b.WriteString(workflowApprovalRejectedMarker)
b.WriteString("\nneeds-human: Prow has not run this fork PR's presubmits. Its author is not one of Prow's trusted users, so the trigger plugin waits for `")
b.WriteString(workflowApprovalCommand)
b.WriteString("` from a trusted user. Prow already refused the command from `")
b.WriteString(appLogin)
b.WriteString("` on this repository, so Hive did not post it here.\n\nA maintainer can comment `")
b.WriteString(workflowApprovalCommand)
b.WriteString("` (plus `/test all` for the current head), or make `")
b.WriteString(appLogin)
b.WriteString("` a Prow trusted user so Hive's command counts.")
return b.String()
}
Loading
Loading