diff --git a/changelog.d/fixed-blocked-repair-writeback-gate.md b/changelog.d/fixed-blocked-repair-writeback-gate.md new file mode 100644 index 0000000000..a735fa6b4d --- /dev/null +++ b/changelog.d/fixed-blocked-repair-writeback-gate.md @@ -0,0 +1 @@ +- Blocked sweep: a `blocked` label the hive applied itself, with nothing naming what the issue waits on, is now turned into a `needs-human` ask on every issue, including one created before the hive could write to the repository and one without the review opt-in label. The repair went through the write-back gate as `gate_comment`, so the backfill guard (`writeback.unsolicited.backfill` off) and the opt-in label requirement refused it on every sweep, although the hive had already labelled the issue; the issue stayed `blocked`, out of every agent's reach and in no inbox, for days. The repair is now its own write-back kind, `blocked_repair` (`writeback.kinds.blocked_repair`, default minimum level 1, shown in Settings → Features → Write-back containment and settable with the admin MCP `writeback.kind_policy`): when the sweep has established that the hive's own bot applied the label, neither of those two checks applies. A paused repository, `writeback.per_repo` mode `off`, `issues-only` or `labels-only`, a repository outside `project.repos`, `own_only`, the kind's minimum level and the notification burst hold still refuse it, and a person's `blocked` is left alone as before. A held repair is logged once per issue and reason (`blocked sweep: needs-human ask held by writeback gate`) instead of on every sweep; repeats go to the debug log. diff --git a/src/docs/acmm-policy-matrix.md b/src/docs/acmm-policy-matrix.md index 27cef1ea37..e343d26a9f 100644 --- a/src/docs/acmm-policy-matrix.md +++ b/src/docs/acmm-policy-matrix.md @@ -274,6 +274,7 @@ gated. Defaults (override under `writeback.kinds.`): | `claim_comment` | — | opt-in label | opt-in label | opt-in label | | `welcome_comment` | — | — | opt-in label | all items | | `issue_file` (new hive-authored issues) | L3+ | yes | yes | yes | +| `blocked_repair` (a `blocked` label the hive applied itself, with no blocker named) | L1+ | yes | yes | yes | `ci_rerun` also covers the one transient re-run of a red PR from `auto_merge.trusted_bot_authors` (a hive PR is its own item and is not gated): @@ -283,6 +284,14 @@ hands the PR to a person with the reason. `bot_rebase_request` only ticks Renovate's rebase/retry box; a stranded Dependabot PR is handed to a person instead, because Dependabot refuses commands from apps. +`blocked_repair` replaces a marker-less `blocked` the hive applied with a +`needs-human` ask (see +[`blocked`](labels-and-control-signals/reference.md#blocked)). It needs no +opt-in label and ignores the backfill guard, because the hive has already +written to that issue: a contributor relay applies the label with a task +token at any level, so the repair has the lowest floor. On an issue whose +`blocked` the hive did not apply, the kind is never used. + In practice: **L4 means hive files issues and talks only on its own PRs**; **L5 adds comments and labels on PRs a maintainer labelled `hive-review`**; **L6 lets sentinel and welcome comments reach unlabelled PRs**. A repo diff --git a/src/docs/labels-and-control-signals/reference.md b/src/docs/labels-and-control-signals/reference.md index 1a2cb32710..66427af32e 100644 --- a/src/docs/labels-and-control-signals/reference.md +++ b/src/docs/labels-and-control-signals/reference.md @@ -94,7 +94,7 @@ Some hive-authored labels look like provenance but are read as signals, so they | `needs-sha` from SHA-hold | Issues | SHA-hold sweep | Trusted-reporter `kind/bug` without a SHA gets this soft label plus the same `` notice instead of `hold`; the issue stays actionable. The sweep removes it once a SHA appears. | Soft (informational) | Hive | Eval tick SHA sweep | Add SHA evidence, or remove the label | `issues.sha_hold_trusted_label` (default `needs-sha`; `hold` restores the hard hold for everyone) | | `do-not-merge` and `do-not-merge*` | Issues, PRs | Exempt filter, merge sweeps, task-list sweep | Permanently exempt; exact match is case-insensitive but prefix matching follows Go `strings.HasPrefix` on the original label. | Hard suppress | Human | Enumeration/sweeps | Human removes | Built-in `PermanentExemptLabels` (`src/pkg/github/client.go:750-755`, `src/pkg/github/client.go:2035-2053`) | | `governor.labels.exempt` entries | Issues, PRs | Exempt filter | Same exempt behavior as `do-not-merge`; wins over required-label admission. Defaults include `nightly-tests`, `LFX`, `meta-tracker`, `auto-qa-tuning-report`, `adopters`, `changes-requested`, `waiting-on-author`. | Hard suppress | Operator/dashboard | Enumeration/sweeps | Remove label or config entry | `governor.labels.exempt` (`src/pkg/config/governor_config.go:175-184`, `src/pkg/config/defaults.go:260-270`) | -| `blocked` | Issues and PRs | Enumeration, stage derivation, claim gate, review backlog, blocked sweep | The issue waits on an unmerged PR (or another outside blocker): the review backlog files a finding `blocked` when its evidence file exists only on the reviewed PR (absent on the base branch), and `hive-open-issue comment --blocked-on-pr ` parks an existing one; both write ``. A `blocked` issue is filtered (reason `blocked on owner/repo#N`), derives no stage label, is refused by every automated claim producer (`blocked: owner/repo#N`) including the contributor relay, and never carries `help wanted` ([#13093](https://github.com/hivecommons/hive/issues/13093)). When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker, but only on a review-backlog item (its body carries `filed by Hive review backlog`) that GitHub reports no open "blocked by" link on ([#13364](https://github.com/hivecommons/hive/issues/13364)). A human's marker-less `blocked` stays. A contributor relay's `blocked` verdict that named nothing (GitHub records it as the App bot too, because the relay labels with its hub-minted task token) is turned into a `needs-human` ask instead: `needs-human` is added, `blocked` removed and one note says what is missing. Lifting it only sent the issue back to the next agent, which re-applied it, and keeping it hid the issue from every inbox. A blocked verdict whose reason is empty or only the relay footer is booked idle and its label released, like a tooling failure. A marker in a comment counts like one in the body: the row's reason (`blocked on owner/repo#N`) and the band signal name it. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). On a PR, `blocked` holds the merge: the blocked sweep puts it on an open, non-draft PR whose `Depends-on:` line names an open pull request (same repository or another), with one `` comment per open target, and every merge lane refuses a `blocked` PR (`label:blocked`). A person's `blocked` is never lifted; a person who removes the hive's `blocked` is not overridden for the same dependencies. | Hard suppress | Hive (reserved for agents; relay only) | Review backlog filing; blocked-on comment relay | The blocked sweep after each stage-label sync removes it, with one `` note, once the tracked PR is merged or closed, or once a recheck-after deadline passes (`deadline passed; re-triaging`) unless the body also carries ``. On a PR it removes the hive's own `blocked` once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. An ops-tracking issue on the hive's own project keeps `blocked` once its deadline passes and instead gets soak triage: the hub's `/api/hub/hives` (or, with no hub, the spoke's own `/api/version`) is probed for every `stable`-channel hive (fresh heartbeat, no restart loop, running the candidate `sha256:` digest or `v5 ` named in the body), ONE `` comment is posted, only the checklist lines that evidence proves are ticked, and `` is appended to the body so the sweep does not repeat ([#13311](https://github.com/hivecommons/hive/issues/13311)); a human may remove it at any time | Fixed label (`src/pkg/github/issue_blocked.go`, `src/pkg/github/pr_blocked_depends_on.go`, `src/pkg/github/issue_recheck_soak.go`, `src/pkg/github/issue_stage_labels.go`, `src/pkg/github/issue_claim_gate.go`) | +| `blocked` | Issues and PRs | Enumeration, stage derivation, claim gate, review backlog, blocked sweep | The issue waits on an unmerged PR (or another outside blocker): the review backlog files a finding `blocked` when its evidence file exists only on the reviewed PR (absent on the base branch), and `hive-open-issue comment --blocked-on-pr ` parks an existing one; both write ``. A `blocked` issue is filtered (reason `blocked on owner/repo#N`), derives no stage label, is refused by every automated claim producer (`blocked: owner/repo#N`) including the contributor relay, and never carries `help wanted` ([#13093](https://github.com/hivecommons/hive/issues/13093)). When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker, but only on a review-backlog item (its body carries `filed by Hive review backlog`) that GitHub reports no open "blocked by" link on ([#13364](https://github.com/hivecommons/hive/issues/13364)). A human's marker-less `blocked` stays. A contributor relay's `blocked` verdict that named nothing (GitHub records it as the App bot too, because the relay labels with its hub-minted task token) is turned into a `needs-human` ask instead: `needs-human` is added, `blocked` removed and one note says what is missing. Because the hive put that label on, the repair (write-back kind `blocked_repair`) also runs on an issue older than the hive's write access to the repository and on one without the review opt-in label. Lifting it only sent the issue back to the next agent, which re-applied it, and keeping it hid the issue from every inbox. A blocked verdict whose reason is empty or only the relay footer is booked idle and its label released, like a tooling failure. A marker in a comment counts like one in the body: the row's reason (`blocked on owner/repo#N`) and the band signal name it. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). On a PR, `blocked` holds the merge: the blocked sweep puts it on an open, non-draft PR whose `Depends-on:` line names an open pull request (same repository or another), with one `` comment per open target, and every merge lane refuses a `blocked` PR (`label:blocked`). A person's `blocked` is never lifted; a person who removes the hive's `blocked` is not overridden for the same dependencies. | Hard suppress | Hive (reserved for agents; relay only) | Review backlog filing; blocked-on comment relay | The blocked sweep after each stage-label sync removes it, with one `` note, once the tracked PR is merged or closed, or once a recheck-after deadline passes (`deadline passed; re-triaging`) unless the body also carries ``. On a PR it removes the hive's own `blocked` once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. An ops-tracking issue on the hive's own project keeps `blocked` once its deadline passes and instead gets soak triage: the hub's `/api/hub/hives` (or, with no hub, the spoke's own `/api/version`) is probed for every `stable`-channel hive (fresh heartbeat, no restart loop, running the candidate `sha256:` digest or `v5 ` named in the body), ONE `` comment is posted, only the checklist lines that evidence proves are ticked, and `` is appended to the body so the sweep does not repeat ([#13311](https://github.com/hivecommons/hive/issues/13311)); a human may remove it at any time | Fixed label (`src/pkg/github/issue_blocked.go`, `src/pkg/github/pr_blocked_depends_on.go`, `src/pkg/github/issue_recheck_soak.go`, `src/pkg/github/issue_stage_labels.go`, `src/pkg/github/issue_claim_gate.go`) | | `needs-human` | Issues | Enumeration, task-list sweep, claim escalation gate and un-park sweep | Filters issues from agent kicks and contributor offers; records a human decision requirement. See the [issue reference](#needs-human) and [Maintainer commands](../maintainer-commands.md). | Hard suppress | Hive or human | Enumeration/task-list sweep/kick building | An eligible maintainer's ordinary `/hive approve` or `/hive decision ` lifts it regardless of who applied it; label-driven un-parking remains provenance-sensitive. Keep `hold` if work must stay stopped. Agents may add, never remove it. | Fixed label (`src/pkg/github/client.go:1008-1024`, `src/pkg/github/task_list_sweep.go:611-641`, `src/pkg/github/task_list_sweep.go:874-880`, `src/cmd/hive/claimescalation.go`) | | `needs-spec` from the clarity gate | Issues | Scheduler kick assembly (`runs.triage.clarity_gate`) | With the gate on (default off), a human-filed, unclaimed issue whose clarity score (goal, acceptance criteria, scope, detail, open choices) is under `runs.triage.clarity_gate.threshold` gets one `` comment with concise questions for its author and the `needs-spec` hard-suppress label instead of a claim; it derives `stage/045-needs-decision` and leaves the agent queue ([#13577](https://github.com/hivecommons/hive/issues/13577)). | Hard suppress | Hive | Before kicks are built | The issue author, or a non-hive commenter with write/maintain/admin, replies after the marker and the resume sweep removes the hive's own `needs-spec` and records the un-park so the issue re-derives `stage/010-actionable` ([#13588](https://github.com/hivecommons/hive/issues/13588)); a `needs-spec` a person added is never lifted by the sweep. Or remove `needs-spec` by hand. The marker keeps the gate from asking again | `runs.triage.clarity_gate` (`src/pkg/scheduler/clarity_gate.go`, `src/pkg/classify/clarity.go`, `src/pkg/github/clarity_gate_resume.go`) | | `stage/017-possible-existing` from the prior-art gate | Issues | Scheduler kick assembly (`runs.triage.prior_art`) | With the gate on (default on), an unclaimed feature / RFC / epic issue whose key terms match an existing knowledge bead, setting, merged PR, closed issue, doc or code identifier at `runs.triage.prior_art.threshold` (default 70) gets one `` comment quoting the findings and the `stage/017-possible-existing` label instead of a claim; hive-filed issues get a `` note and are never parked ([#13575](https://github.com/hivecommons/hive/issues/13575)). | Hard suppress | Hive | Before kicks are built | The author or a write+ commenter replies `/hive decision proceed — ` (label lifted, a *Scope vs existing* section appended to the body, re-derives 010) or `/hive decision exists` (closed `not_planned` with `already-exists`); a plain reply lifts the label too. A clanker-filed park with no reply is closed `already-exists` after `runs.triage.prior_art.bot_close_ttl` (default 72h); reopen to continue. A 017 a person added is never lifted by the sweep | `runs.triage.prior_art` (`src/pkg/priorart`, `src/pkg/scheduler/prior_art_gate.go`, `src/pkg/github/prior_art_gate_resume.go`) | @@ -301,6 +301,13 @@ The issue waits on an unmerged PR (or another outside blocker): the review backl When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker on a review-backlog item, leaving a human-applied one alone ([#13364](https://github.com/hivecommons/hive/issues/13364)). Any other hive-applied marker-less `blocked` (a contributor relay's verdict that named no issue or PR) becomes a `needs-human` ask, with one note naming the two ways out: name the dependency, or remove `needs-human` to re-queue it. A verdict with no reason at all is not blocked: the hub books it idle and releases the label. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). +**The repair works on old and unlabelled issues.** Turning a marker-less hive `blocked` into a `needs-human` ask goes through the write-back gate as its own kind, `blocked_repair`. Two of the gate's usual checks do not apply to it, because the hive applied the label being repaired and so has already written to the issue: + +- the backfill guard: the repair runs on an issue created before the hive first saw, or could first write to, the repository, even with `writeback.unsolicited.backfill` off; +- the review opt-in label: the repair runs on an issue that does not carry it. + +The sweep checks who applied the label (the actor of the newest `labeled` event) before it asks the gate, so a person's `blocked` never gets this treatment. Everything else still holds the repair: a paused repository, `writeback.per_repo` mode `off`, `issues-only` or `labels-only` (the repair posts a note, so it is not a label-only write), a repository outside `project.repos`, `own_only` if an operator sets it, and the kind's minimum level (`writeback.kinds.blocked_repair.min_level`, default 1; an unset `acmm_level` refuses). The burst hold applies as well, so a repository with several such issues gets them over a few sweeps. A held repair is logged once per issue and reason (`blocked sweep: needs-human ask held by writeback gate`) and again when the reason changes, not on every sweep. + With the marker (in the body, or in a comment the blocked sweep has read), the band signal reads `blocked on owner/repo#N — lifted when it merges or closes` instead of `the label records no reason`, the activity monitor's pill reads `blocked on #N`, links the PR and carries its cached state (`blocked_on_pr` in `/api/stage-tracker`), and the hive's own removal of the label re-reads the issue live rather than waiting for the next enumeration ([#14952](https://github.com/hivecommons/hive/issues/14952)). With the issue form, the blocker joins `blocked_on` instead and the caption reads `Blocked on issue #N — lifts when it closes` ([#16852](https://github.com/hivecommons/hive/issues/16852)). It is also always skipped by contributor admission and shown in the default waiting band. **On a pull request: `Depends-on:` holds the merge.** A PR body's `Depends-on: #N` or `Depends-on: owner/repo#N` lines (the same lines the dashboard's critical path reads) say the PR must land after another one, for example a change whose tests need a pull request in another repository. The blocked sweep (every 5 minutes, over the configured repositories) puts `blocked` on an open, non-draft PR whose `Depends-on:` names at least one open pull request, and posts one `` comment per open target (the same marker and writer a relay's blocked verdict uses). Every merge lane refuses a `blocked` PR with `label:blocked`: the self-authored and trusted-bot lane, the trusted-author lane, the contributor lane (whatever `exclude_labels` says), the queued merge-request lane, the merge train, the native merge queue (the App's own entry is withdrawn), the dashboard's merge verdict and its human-merge shortcuts. Like a hold it is a gate, not a stage: the PR's stage is its head's progress and the route carries `blocked on owner/repo#N`; the activity monitor row names the target (`blocked_on_pr`, caption `Blocked on PR #N — lifts when it merges`). A target that is an issue, a closed PR, or a PR the App cannot read places nothing (an unreadable one is logged once). The sweep never places over a person's `blocked`, never lifts one, and never puts its own back after a person removed it, until the body names an open pull request it has not marked yet. It lifts its own `blocked`, with one `` note, once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. Both writes go through the write-back gate (`gate_comment`), and the sweep needs the App bot login to tell its label from a person's. Stacked PRs (a base branch that is another open PR's) are not handled. diff --git a/src/docs/operator-reference/sweeps-and-operations.md b/src/docs/operator-reference/sweeps-and-operations.md index d16f5c71f7..f830b3235a 100644 --- a/src/docs/operator-reference/sweeps-and-operations.md +++ b/src/docs/operator-reference/sweeps-and-operations.md @@ -550,7 +550,8 @@ that level, the power the write-back exists to govern? | Adoption close-first (`automerge/automerge_adoption_close_first.go`) | auto-merge sweep, when a supersede adoption opens the hive's copy; close-first poll | close-first note, `hive/superseded` label and close on the original; reopen, copy-failed note and label removal when the copy fails | the contributor's original PR | `contributor_adopt` → L6 + label (an agent-authored original closes without the label, `adoption.close_superseded: on_adopt`); refused: the original stays open until the copy merges, and a due reopen waits for the next poll | | Claims (`prclaims.go`, `claimed_by_label.go`, `cmd/hive/claimswire.go`, `claimescalation.go`) | claim ledger changes | `claimed-by:*` labels, claim/escalation comments | claimed issues | `claim_comment` → L4 + label (hive-claimed issues are own items) | | Issue close / task-list / decomposed parents (`issue_close.go`, `task_list_sweep.go`, `decomposed_parents.go`, `close_on_merge.go`, `tracking_issue.go`) | merges, sweeps | close, body edits (checkbox ticks), `Refs` rewrites | issues linked from hive PRs | `issue_close`, `parent_issue_edit` → L5 + label; hive-filed issues are own items | -| Gate comments (`pkg/scheduler/clarity_gate.go`, `prior_art_gate.go`, `triage.go`, `issue_blocked.go`, `issue_recheck_soak.go`) | scheduler intake | clarity/prior-art/triage questions, `blocked` labels | issues admitted by the issue filter | `gate_comment` → L5 + label; items carrying the intake label are opted in | +| Gate comments (`pkg/scheduler/clarity_gate.go`, `prior_art_gate.go`, `triage.go`, `verdict_blocker.go`, `pr_blocked_depends_on.go`, `issue_recheck_soak.go`) | scheduler intake | clarity/prior-art/triage questions, `blocked` labels | issues admitted by the issue filter | `gate_comment` → L5 + label; items carrying the intake label are opted in | +| Blocked repair (`pkg/github/issue_blocked.go`) | blocked sweep (every 5 min) and the blocked reconcile after each stage-label sync | `needs-human` added, `blocked` removed, one `` note asking a maintainer to name the dependency or re-queue the issue | open issues whose `blocked` the hive's own bot applied and that name nothing they wait on (no blocked-on marker, deadline, ops-tracking marker or native "blocked by" link; not a review-backlog item) | `blocked_repair` → L1, no opt-in label and no backfill guard, because the hive already wrote the label being repaired; a pause, `per_repo.mode` `off` / `issues-only` / `labels-only` and a raised `min_level` still refuse it. Refused: the issue keeps `blocked` and the hold is logged once per issue and reason | | Welcome / outreach (`pkg/gfi`, `gfi_forge.go`, `pr_clanker_requested.go`) | good-first-issue selection | welcome comment, `good first issue` label | issues hive advertises | `welcome_comment` → L5 with label, L6 without | | Issue filing (`pkg/upstreamwatch/github_filer.go`, `advisory.go`, scanner/deps/regression agents, `dedupe_advisory_close.go`) | agents, advisory digest, upstream watch | **new** issues hive authors, pinned advisory issue + comments | `project.repos` | `issue_file` → L3, no label (this is what listing a repo opts in to; `per_repo.mode: issues-only` keeps only this) | | Code scanning sweep (`pkg/github/code_scanning.go`, `pkg/codescanning`) | hourly sweep (`code_scanning`, opt-in per repo) | **new** grouped issues from code scanning alerts, alert-table body edits, one closing comment + close when every alert is fixed or dismissed | the hive's own `hive/code-scanning` issues in repos with `code_scanning` on | `issue_file` → L3, no label (own items); see [code-scanning.md](../code-scanning.md) | @@ -610,9 +611,10 @@ Policy decides whether a kind may write to an item; the content layer under only to comment kinds whose body the caller handed the gate as the preview (`sentinel`, `needs_decision_comment`, `supersession_notice`, `duplicate_fix_notice`, `reporter_trust_hold`, `contributor_adopt`, -`claim_comment`, `gate_comment`, `welcome_comment`); `Allowed()` without a -preview — labels, closes, hold-gate upkeep, task-list edits — stays -policy-only. Three checks run after the policy gate allows a write: +`claim_comment`, `gate_comment`, `welcome_comment`, `blocked_repair`); +`Allowed()` without a preview — labels, closes, hold-gate upkeep, task-list +edits — stays policy-only. Three checks run after the policy gate allows a +write: | Check | Key | Suppression reason | Default | |---|---|---|---| diff --git a/src/pkg/github/issue_blocked.go b/src/pkg/github/issue_blocked.go index a1166c2b29..88ac669c86 100644 --- a/src/pkg/github/issue_blocked.go +++ b/src/pkg/github/issue_blocked.go @@ -471,6 +471,7 @@ func (c *Client) SweepBlockedIssues(ctx context.Context, opts BlockedSweepOption }) } } + c.retainBlockedRepairHeld(repos, targets) return c.reconcileBlockedTargets(ctx, targets, c.logger), nil } @@ -532,6 +533,8 @@ func (c *Client) reconcileBlockedTargets(ctx context.Context, targets []blockedI stripBlockedHelpWanted(ctx, c, t, &res, logger) continue } + // A block that names what it waits on is not one the repair converts. + forgetBlockedRepairHeld(c, t) if onIssue { reconcileBlockedOnIssue(ctx, c, t, prRepo, prNumber, &res, logger) continue @@ -706,8 +709,17 @@ const blockedMarkerlessNote = blockedUnblockedMarker + "\nUnblocked: the `blocke // and puts it in the maintainer inbox with the question. Only a hive-applied // label with no open native "blocked by" link is converted; a person's // marker-less block is theirs. Reports whether the issue was converted. +// +// The write-back gate is asked as blocked_repair with the fact established +// here, that the label is the hive's (Item.HiveAppliedBlock). The hive has +// already written to such an issue, so the gate does not hold the repair back +// for the issue predating the hive's write access or lacking the opt-in +// label; while it was asked as gate_comment both did, every sweep, and the +// issue stayed in the invisible state above for days. A pause, the repo mode +// and the kind's minimum level still refuse it. func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIssueTarget, res *BlockedIssueResult, logger *slog.Logger) bool { if t.reviewBacklog || t.openBlockedBy > 0 { + forgetBlockedRepairHeld(client, t) return false } actor, found, err := client.blockedLabelActor(ctx, t.repo, t.number) @@ -717,6 +729,7 @@ func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIss return false } if !found || (!client.isOwnBotLogin(actor) && !client.IsHiveAuthorLogin(actor)) { + forgetBlockedRepairHeld(client, t) return false } item, err := client.WritebackItemForNumber(ctx, t.repo, t.number) @@ -725,14 +738,23 @@ func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIss logger.Warn("blocked sweep: item lookup failed; marker-less block left alone", "repo", t.repo, "issue", t.number, "error", err) return false } - owner, name := client.splitRepo(t.repo) - if d := client.WritebackAllowed(owner+"/"+name, item, writeback.KindGateComment, blockedMarkerlessAskNote); !d.Allowed { + item.HiveAppliedBlock = true + gateRepo := client.blockedRepairRepo(t.repo) + if d := client.WritebackAllowed(gateRepo, item, writeback.KindBlockedRepair, blockedMarkerlessAskNote); !d.Allowed { // The burst hold lets a couple of asks per repo through per sweep, // and a failed write's retry is a duplicate for same_item_ttl; both - // clear on a later sweep. Say so rather than fall through silently. - logger.Info("blocked sweep: needs-human ask held by writeback gate", "repo", t.repo, "issue", t.number, "reason", d.Reason, "detail", d.Detail) + // clear on a later sweep. Say so rather than fall through silently, + // but once per issue and reason: the sweep asks again every few + // minutes, and a hold that lasts (a pause, the repo mode, the level) + // would repeat the line each time. Repeats go to the debug log. + level := slog.LevelDebug + if client.WritebackGate().NoteHeld(gateRepo, t.number, writeback.KindBlockedRepair, d.Reason) { + level = slog.LevelInfo + } + logger.Log(ctx, level, "blocked sweep: needs-human ask held by writeback gate", "repo", t.repo, "issue", t.number, "reason", d.Reason, "detail", d.Detail) return false } + forgetBlockedRepairHeld(client, t) if !issueHasLabel(t.labels, issueNeedsHumanLabel) { if err := client.AddLabels(ctx, t.repo, t.number, []string{issueNeedsHumanLabel}); err != nil { // writeback:gated res.Errors++ diff --git a/src/pkg/github/issue_blocked_repair_held.go b/src/pkg/github/issue_blocked_repair_held.go new file mode 100644 index 0000000000..51d5c65090 --- /dev/null +++ b/src/pkg/github/issue_blocked_repair_held.go @@ -0,0 +1,51 @@ +package github + +import ( + "strings" + + "github.com/hivecommons/hive/pkg/writeback" +) + +// The blocked sweep asks the write-back gate about every marker-less +// hive-applied block on every pass, and a refusal that lasts would be logged +// each time. The gate remembers the reason it last refused each issue for +// (writeback.Gate.NoteHeld); these helpers keep that memory in step with the +// sweep, so a hold is reported once per issue and reason and again after the +// issue stopped being one the repair converts. + +// blockedRepairRepo is repo as owner/name, the form the gate is asked with. +func (c *Client) blockedRepairRepo(repo string) string { + owner, name := c.splitRepo(repo) + return owner + "/" + name +} + +// forgetBlockedRepairHeld drops the hold remembered for t: the repair went +// through, or t is no longer a block the repair converts (it names what it +// waits on, a person applied it, it is a review-backlog item). A later hold +// is then logged again. +func forgetBlockedRepairHeld(client *Client, t blockedIssueTarget) { + client.WritebackGate().ForgetHeld(client.blockedRepairRepo(t.repo), t.number, writeback.KindBlockedRepair) +} + +// retainBlockedRepairHeld forgets the holds of issues in repos that targets, +// the complete listing of open `blocked` issues in those repos, no longer +// contains: an issue that was closed, or whose label was removed, never +// reaches the repair again. A repo outside this listing keeps its holds, so +// a sweep of some repos does not forget the others. +func (c *Client) retainBlockedRepairHeld(repos []string, targets []blockedIssueTarget) { + gate := c.WritebackGate() + if gate == nil { + return + } + listed := make(map[string]bool, len(repos)) + for _, repo := range repos { + listed[strings.ToLower(c.blockedRepairRepo(repo))] = true + } + open := make(map[string]bool, len(targets)) + for _, t := range targets { + open[blockedOnSeenKey(c.blockedRepairRepo(t.repo), t.number)] = true + } + gate.RetainHeld(writeback.KindBlockedRepair, func(repo string, number int) bool { + return !listed[repo] || open[blockedOnSeenKey(repo, number)] + }) +} diff --git a/src/pkg/github/issue_blocked_repair_test.go b/src/pkg/github/issue_blocked_repair_test.go new file mode 100644 index 0000000000..cf6de8baf8 --- /dev/null +++ b/src/pkg/github/issue_blocked_repair_test.go @@ -0,0 +1,404 @@ +package github + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" + + "github.com/hivecommons/hive/pkg/writeback" +) + +// The blocked sweep turns a `blocked` the hive applied, naming nothing it +// waits on, into a needs-human ask. These tests run that repair through a +// real write-back gate. Asked as gate_comment, the gate refused it on every +// sweep for an issue older than the hive's write access (backfill off) and +// for one without the opt-in label, so the issue stayed blocked, in no inbox, +// for days. Asked as blocked_repair with the fact that the label is the +// hive's, neither check applies; a pause, the repo mode and the level still do. + +const ( + blockedRepairTestRepo = "acme/widgets" + blockedRepairIssue = 80 + blockedRepairHeldLine = "needs-human ask held by writeback gate" + // blockedRepairPerson is a maintainer's login: the issue's author, and + // in some tests the one who applied `blocked`. + blockedRepairPerson = "maintainer" +) + +// blockedRepairFirstSeen is when the gate first saw acme/widgets: an issue +// created before it is one the backfill guard keeps the hive off. +var blockedRepairFirstSeen = time.Date(2026, 6, 1, 12, 0, 0, 0, time.UTC) + +const ( + blockedRepairOldIssue = "2026-01-15T09:00:00Z" // before blockedRepairFirstSeen + blockedRepairNewIssue = "2026-07-01T09:00:00Z" // after it +) + +// blockedRepairServer is acme/widgets with one open issue, #80. Its fields +// are the issue's state on GitHub and may be changed between sweeps. +type blockedRepairServer struct { + mu sync.Mutex + calls []string + // blocked: #80 carries `blocked` and is open (so the sweep lists it). + blocked bool + // actor applied `blocked`, per the issue's timeline. + actor string + // createdAt is the issue's created_at. + createdAt string + // marker: a comment on #80 names the open pull request #7 as its blocker. + marker bool + // blockedBy: GitHub reports an open native "blocked by" link. + blockedBy bool +} + +func (f *blockedRepairServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { + f.mu.Lock() + defer f.mu.Unlock() + w.Header().Set("Content-Type", "application/json") + f.calls = append(f.calls, r.Method+" "+r.URL.Path) + const issues = "/repos/" + blockedRepairTestRepo + "/issues" + const issue = issues + "/80" + switch { + case r.Method == "GET" && r.URL.Path == issues && strings.Contains(r.URL.RawQuery, "labels=blocked"): + items := []map[string]any{} + if f.blocked { + items = append(items, f.issueLocked()) + } + _ = json.NewEncoder(w).Encode(items) + case r.Method == "GET" && r.URL.Path == issue: + _ = json.NewEncoder(w).Encode(f.issueLocked()) + case r.Method == "GET" && r.URL.Path == issue+"/comments": + if f.marker { + _ = json.NewEncoder(w).Encode([]map[string]any{{"id": 3, "body": BlockedOnMarker(blockedRepairTestRepo, 7) + "\nblocked on the shared fix"}}) + return + } + _, _ = io.WriteString(w, `[]`) + case r.Method == "GET" && r.URL.Path == "/repos/"+blockedRepairTestRepo+"/pulls/7": + _, _ = io.WriteString(w, `{"number":7,"state":"open"}`) + case r.Method == "GET" && r.URL.Path == issue+"/timeline": + _ = json.NewEncoder(w).Encode([]map[string]any{{ + "event": "labeled", "created_at": "2026-10-29T04:17:25Z", + "label": map[string]any{"name": "blocked"}, "actor": map[string]any{"login": f.actor}, + }}) + case r.Method == "POST" && r.URL.Path == issue+"/labels": + _, _ = io.WriteString(w, `[{"name":"needs-human"}]`) + case r.Method == "DELETE" && r.URL.Path == issue+"/labels/blocked": + f.blocked = false + _, _ = io.WriteString(w, `[]`) + case r.Method == "POST" && r.URL.Path == issue+"/comments": + _, _ = io.WriteString(w, `{"id":9}`) + default: + w.WriteHeader(http.StatusNotFound) + _, _ = fmt.Fprintf(w, `{"path":%q}`, r.URL.Path) + } +} + +// issueLocked is #80 as GitHub returns it: opened by a person, with no +// review opt-in label. +func (f *blockedRepairServer) issueLocked() map[string]any { + is := map[string]any{ + "number": blockedRepairIssue, "state": "open", + "user": map[string]any{"login": blockedRepairPerson}, + "created_at": f.createdAt, + "labels": []any{map[string]any{"name": "blocked"}}, + "body": "Upgrade the client once the upstream API ships.", + } + if f.blockedBy { + is["issue_dependencies_summary"] = map[string]any{"blocked_by": 1, "total_blocked_by": 1} + } + return is +} + +// set changes the issue's state between sweeps. +func (f *blockedRepairServer) set(change func(*blockedRepairServer)) { + f.mu.Lock() + defer f.mu.Unlock() + change(f) +} + +// wrote reports the calls that change the issue. +func (f *blockedRepairServer) wrote() []string { + f.mu.Lock() + defer f.mu.Unlock() + var out []string + for _, c := range f.calls { + if !strings.HasPrefix(c, "GET ") { + out = append(out, c) + } + } + return out +} + +// blockedRepairPolicy is what the gate reads on each decision; a test +// changes it between sweeps (never during one). +type blockedRepairPolicy struct { + level int + paused bool + cfg writeback.Config +} + +// blockedRepairRig is a client over srv with a gate that first saw +// acme/widgets at blockedRepairFirstSeen, and the client's log. +func blockedRepairRig(t *testing.T, f *blockedRepairServer, p *blockedRepairPolicy) (*Client, *writeback.Gate, *bytes.Buffer) { + t.Helper() + srv := httptest.NewServer(f) + t.Cleanup(srv.Close) + var logs bytes.Buffer + c := NewClientForTest(srv.URL, "acme", []string{"widgets"}, slog.New(slog.NewTextHandler(&logs, nil))) + c.appBotLogin = testBot + gate := writeback.New(writeback.Config{}, writeback.Deps{ + LevelFor: func(string) int { return p.level }, + RepoPause: func(string) (writeback.RepoPauseInfo, bool) { + return writeback.RepoPauseInfo{By: "owner", Reason: "freeze"}, p.paused + }, + Config: func() writeback.Config { return p.cfg }, + Now: func() time.Time { return blockedRepairFirstSeen }, + }, "") + gate.ObserveRepo(blockedRepairTestRepo) + c.SetWritebackGate(gate) + // A sweep that reads a blocked-on comment remembers it process-wide. + t.Cleanup(func() { forgetBlockedOnSeen(blockedRepairTestRepo, blockedRepairIssue) }) + return c, gate, &logs +} + +func blockedRepairSweep(t *testing.T, c *Client) BlockedIssueResult { + t.Helper() + res, err := c.SweepBlockedIssues(context.Background(), BlockedSweepOptions{Repos: []string{blockedRepairTestRepo}}) + if err != nil { + t.Fatalf("SweepBlockedIssues: %v", err) + } + if res.Errors != 0 { + t.Fatalf("res = %+v, want no errors", res) + } + return res +} + +// A hive-applied, marker-less `blocked` is repaired on an issue the gate +// would not let gate_comment touch: one older than the hive's write access +// with backfill off, and one without the opt-in label at a level where +// gate_comment needs it. +func TestBlockedRepairConvertsHiveBlockTheGateUsedToHold(t *testing.T) { + cases := []struct { + name string + createdAt string + // gateComment is the reason the same issue is refused as gate_comment. + gateComment string + }{ + {"issue older than the hive's write access, backfill off", blockedRepairOldIssue, writeback.ReasonBackfill}, + {"issue without the opt-in label", blockedRepairNewIssue, writeback.ReasonNeedsLabel}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: tc.createdAt} + c, gate, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6}) + + created, err := time.Parse(time.RFC3339, tc.createdAt) + if err != nil { + t.Fatal(err) + } + item := writeback.Item{Number: blockedRepairIssue, Author: blockedRepairPerson, CreatedAt: created} + if d := gate.Decide(blockedRepairTestRepo, item, writeback.KindGateComment); d.Allowed || d.Reason != tc.gateComment { + t.Fatalf("gate_comment on this issue = %+v, want refused with %s (the hold this test is about)", d, tc.gateComment) + } + + res := blockedRepairSweep(t, c) + if res.AskedHuman != 1 { + t.Fatalf("res = %+v, want the block turned into a needs-human ask\nlog:\n%s", res, logs.String()) + } + joined := strings.Join(f.wrote(), "\n") + want := []string{ + "POST /repos/acme/widgets/issues/80/labels", + "DELETE /repos/acme/widgets/issues/80/labels/blocked", + "POST /repos/acme/widgets/issues/80/comments", + } + if joined != strings.Join(want, "\n") { + t.Fatalf("writes =\n%s\nwant needs-human added, then blocked removed, then the note:\n%s", joined, strings.Join(want, "\n")) + } + if held := gate.Recent(); len(held) != 0 { + t.Fatalf("gate suppressed %+v, want nothing held", held) + } + if strings.Contains(logs.String(), blockedRepairHeldLine) { + t.Fatalf("a repair that went through logged a hold:\n%s", logs.String()) + } + // The issue is no longer blocked, so the next sweep has nothing to do. + if res := blockedRepairSweep(t, c); res.Items != 0 || res.AskedHuman != 0 { + t.Fatalf("second sweep res = %+v, want nothing left", res) + } + }) + } +} + +// A person's marker-less `blocked` on the same old issue is theirs: nothing +// is written and the gate is not even asked. +func TestBlockedRepairLeavesAPersonsBlockAlone(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: blockedRepairPerson, createdAt: blockedRepairOldIssue} + c, gate, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6}) + for pass := 1; pass <= 2; pass++ { + if res := blockedRepairSweep(t, c); res.Items != 1 || res.AskedHuman != 0 || res.Unblocked != 0 { + t.Fatalf("pass %d: res = %+v, want the issue seen and left alone", pass, res) + } + } + if w := f.wrote(); len(w) != 0 { + t.Fatalf("writes on a person's block: %v", w) + } + if held := gate.Recent(); len(held) != 0 { + t.Fatalf("the gate was asked about a person's block: %+v", held) + } + if strings.Contains(logs.String(), blockedRepairHeldLine) { + t.Fatalf("a person's block logged a gate hold:\n%s", logs.String()) + } +} + +// What the fact does not lift still refuses the repair: a paused repo, repo +// mode off, a level below the kind's floor and an unset level. +func TestBlockedRepairStillRefusedByPauseModeAndLevel(t *testing.T) { + off := writeback.Config{PerRepo: map[string]writeback.RepoOverride{blockedRepairTestRepo: {Mode: writeback.RepoModeOff}}} + three := 3 + raised := writeback.Config{Kinds: map[string]writeback.KindPolicyOverride{string(writeback.KindBlockedRepair): {MinLevel: &three}}} + cases := []struct { + name string + policy blockedRepairPolicy + reason string + }{ + {"paused repo", blockedRepairPolicy{level: 6, paused: true}, writeback.ReasonRepoPaused}, + {"repo mode off", blockedRepairPolicy{level: 6, cfg: off}, writeback.ReasonRepoOff}, + {"below the kind's minimum level", blockedRepairPolicy{level: 2, cfg: raised}, writeback.ReasonBelowLevel}, + {"level unset", blockedRepairPolicy{level: 0}, writeback.ReasonLevelUnknown}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + policy := tc.policy + c, gate, logs := blockedRepairRig(t, f, &policy) + if res := blockedRepairSweep(t, c); res.AskedHuman != 0 || res.Unblocked != 0 { + t.Fatalf("res = %+v, want nothing converted while the gate refuses", res) + } + if w := f.wrote(); len(w) != 0 { + t.Fatalf("writes while the gate refuses: %v", w) + } + held := gate.Recent() + if len(held) != 1 || held[0].Kind != writeback.KindBlockedRepair || held[0].Reason != tc.reason || held[0].Number != blockedRepairIssue { + t.Fatalf("gate suppressed %+v, want one blocked_repair refusal with reason %s", held, tc.reason) + } + if out := logs.String(); !strings.Contains(out, blockedRepairHeldLine) || !strings.Contains(out, "issue=80") || !strings.Contains(out, "reason="+tc.reason) { + t.Fatalf("hold not logged with reason %s:\n%s", tc.reason, out) + } + }) + } +} + +// The sweep runs every few minutes. A hold is logged once per issue and +// reason, again when the reason changes, and again when the issue is held +// anew after it stopped being one the repair converts: it was repaired, it +// left the listing, it named a blocker, or a person took the label over. +func TestBlockedRepairHoldLoggedOncePerIssueAndReason(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + p := &blockedRepairPolicy{level: 6, paused: true} + c, _, logs := blockedRepairRig(t, f, p) + off := writeback.Config{PerRepo: map[string]writeback.RepoOverride{blockedRepairTestRepo: {Mode: writeback.RepoModeOff}}} + + sweeps := func(n int) { + t.Helper() + for i := 0; i < n; i++ { + blockedRepairSweep(t, c) + } + } + wantHeld := func(step string, lines int, lastReason string) { + t.Helper() + out := logs.String() + if got := strings.Count(out, blockedRepairHeldLine); got != lines { + t.Fatalf("%s: %d hold line(s), want %d:\n%s", step, got, lines, out) + } + if i := strings.LastIndex(out, blockedRepairHeldLine); i < 0 || !strings.Contains(out[i:], "reason="+lastReason) { + t.Fatalf("%s: newest hold line does not say reason=%s:\n%s", step, lastReason, out) + } + } + + sweeps(3) + wantHeld("same reason on three sweeps", 1, writeback.ReasonRepoPaused) + + p.paused, p.cfg = false, off + sweeps(2) + wantHeld("reason changed to repo mode off", 2, writeback.ReasonRepoOff) + + p.paused = true + sweeps(2) + wantHeld("reason changed back to the pause", 3, writeback.ReasonRepoPaused) + + // The repair goes through; a relay later blocks the issue again and the + // repo is paused again. + p.paused, p.cfg = false, writeback.Config{} + if res := blockedRepairSweep(t, c); res.AskedHuman != 1 { + t.Fatalf("res = %+v, want the repair to go through once the gate allows it\nlog:\n%s", res, logs.String()) + } + wantHeld("repair went through", 3, writeback.ReasonRepoPaused) + f.set(func(f *blockedRepairServer) { f.blocked = true }) + p.paused = true + sweeps(2) + wantHeld("held anew after a repair", 4, writeback.ReasonRepoPaused) + + // The issue leaves the listing (closed, or the label removed) and comes + // back held for the same reason. + f.set(func(f *blockedRepairServer) { f.blocked = false }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.blocked = true }) + sweeps(2) + wantHeld("held anew after leaving the listing", 5, writeback.ReasonRepoPaused) + + // A comment names a blocker, then is gone again. + f.set(func(f *blockedRepairServer) { f.marker = true }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.marker = false }) + sweeps(2) + wantHeld("held anew after naming a blocker", 6, writeback.ReasonRepoPaused) + + // GitHub reports an open "blocked by" link, then it is gone again. + f.set(func(f *blockedRepairServer) { f.blockedBy = true }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.blockedBy = false }) + sweeps(2) + wantHeld("held anew after a native blocked-by link", 7, writeback.ReasonRepoPaused) + + // A person re-applies the label, then the hive's bot does. + f.set(func(f *blockedRepairServer) { f.actor = blockedRepairPerson }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.actor = testBot }) + sweeps(2) + wantHeld("held anew after a person's label", 8, writeback.ReasonRepoPaused) + + // Only the one repair wrote anything. + if w := f.wrote(); len(w) != 3 { + t.Fatalf("writes = %v, want the single repair's three", w) + } +} + +// A sweep of other repos does not forget this repo's hold, so the next sweep +// of this one does not log it again. +func TestBlockedRepairHoldKeptAcrossSweepOfOtherRepos(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + c, _, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6, paused: true}) + blockedRepairSweep(t, c) + // acme/gadgets has no blocked issue; the server answers its listing 404, + // which the sweep reports as an error, so nothing is forgotten. + if _, err := c.SweepBlockedIssues(context.Background(), BlockedSweepOptions{Repos: []string{"acme/gadgets"}}); err == nil { + t.Fatal("sweep of an unlistable repo returned no error") + } + // Nor does a clean listing of that other repo, with no blocked issue. + c.retainBlockedRepairHeld([]string{"acme/gadgets"}, nil) + blockedRepairSweep(t, c) + if got := strings.Count(logs.String(), blockedRepairHeldLine); got != 1 { + t.Fatalf("%d hold line(s) after a sweep of another repo, want 1:\n%s", got, logs.String()) + } + // With no gate installed there is nothing to keep. + c.SetWritebackGate(nil) + c.retainBlockedRepairHeld([]string{blockedRepairTestRepo}, nil) +} diff --git a/src/pkg/writeback/blocked_repair_test.go b/src/pkg/writeback/blocked_repair_test.go new file mode 100644 index 0000000000..a1e3c82c7d --- /dev/null +++ b/src/pkg/writeback/blocked_repair_test.go @@ -0,0 +1,232 @@ +package writeback + +import ( + "testing" + "time" +) + +// The blocked sweep repairs a `blocked` label the hive applied with nothing +// naming what the issue waits on: needs-human on, blocked off, one note. It +// used to ask the gate as gate_comment, which held the repair back on every +// issue older than the hive's write access (backfill off) and on every issue +// without the opt-in label, although the hive had already labelled each one. +// blocked_repair is the kind for it: the caller states the label is the +// hive's (Item.HiveAppliedBlock) and those two checks stop applying. + +var ( + // oldIssue predates the gate's first sight of org/repo (t0), the + // backfill guard's case; newIssue is newer and carries no opt-in label. + oldIssue = Item{Number: 40, Author: "maintainer", CreatedAt: t0.Add(-48 * time.Hour)} + newIssue = Item{Number: 41, Author: "maintainer", CreatedAt: t0.Add(time.Hour)} + // optedIssue carries the opt-in label. + optedIssue = Item{Number: 42, Author: "maintainer", Labels: []string{"hive-review"}, CreatedAt: t0.Add(time.Hour)} +) + +// hiveBlocked is it with the caller's finding that the hive's own bot +// applied its `blocked` label. +func hiveBlocked(it Item) Item { + it.HiveAppliedBlock = true + return it +} + +func TestBlockedRepairKindDefaults(t *testing.T) { + if !IsKind("blocked_repair") { + t.Fatal("blocked_repair is not a known kind") + } + var c Config + if p := c.Policy(KindBlockedRepair); p.MinLevel != 1 || !p.RequireLabel || p.UnlabeledMinLevel != 0 || p.OwnOnly { + t.Fatalf("blocked_repair default = %+v, want L1 with the opt-in label for a call that does not state the hive applied the block", p) + } + if KindBlockedRepair.IsLabelKind() || KindBlockedRepair.IsIssueFileKind() { + t.Fatal("blocked_repair posts a note on an existing issue: neither a label kind (labels-only refuses it) nor an issue-file kind") + } + if !KindBlockedRepair.ContentDeduped() || KindBlockedRepair.RateCapExempt() { + t.Fatal("blocked_repair keeps the content dedupe, burst hold and hourly cap its note had under gate_comment") + } + listed := false + for _, row := range c.EffectiveKinds() { + listed = listed || row.Kind == KindBlockedRepair + } + if !listed { + t.Fatal("blocked_repair missing from EffectiveKinds (the Settings kind table)") + } + if err := (Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}).Validate(); err != nil { + t.Fatalf("writeback.kinds.blocked_repair override refused: %v", err) + } +} + +// With the fact stated, neither the backfill guard nor the opt-in label +// holds the repair back. Without it the kind is gated like gate_comment for +// the same item, and the fact does nothing for any other kind. +func TestBlockedRepairSkipsBackfillAndOptInOnlyForAHiveAppliedBlock(t *testing.T) { + cases := []struct { + name string + level int + item Item + kind Kind + allow bool + reason string + }{ + {"old issue, repair, hive applied the block", 6, hiveBlocked(oldIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"old issue, repair, fact not stated", 6, oldIssue, KindBlockedRepair, false, ReasonBackfill}, + {"old issue, gate_comment", 6, oldIssue, KindGateComment, false, ReasonBackfill}, + {"old issue, gate_comment ignores the fact", 6, hiveBlocked(oldIssue), KindGateComment, false, ReasonBackfill}, + + {"unlabelled issue, repair, hive applied the block", 6, hiveBlocked(newIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"unlabelled issue, repair, fact not stated", 6, newIssue, KindBlockedRepair, false, ReasonNeedsLabel}, + {"unlabelled issue, gate_comment", 6, newIssue, KindGateComment, false, ReasonNeedsLabel}, + {"unlabelled issue, gate_comment ignores the fact", 6, hiveBlocked(newIssue), KindGateComment, false, ReasonNeedsLabel}, + {"unlabelled issue, stage_label ignores the fact", 6, hiveBlocked(newIssue), KindStageLabel, false, ReasonNeedsLabel}, + + // The label is applied at any level, so the repair's floor is L1. + {"lowest level, hive applied the block", 1, hiveBlocked(oldIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"opted-in issue, hive applied the block", 1, hiveBlocked(optedIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"opted-in issue, fact not stated", 1, optedIssue, KindBlockedRepair, true, ReasonLabelOptIn}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + g, _ := newGate(t, Config{}, tc.level) + d := g.Allowed("org/repo", tc.item, tc.kind) + if d.Allowed != tc.allow || d.Reason != tc.reason { + t.Fatalf("got %+v, want allow=%v reason=%s", d, tc.allow, tc.reason) + } + }) + } +} + +// The fact lifts two checks and no others: a pause, the repo mode, project +// membership, own_only and the kind's minimum level refuse a hive-applied +// block's repair as they refuse any other write. +func TestBlockedRepairStillHeldByPauseModeProjectOwnOnlyAndLevel(t *testing.T) { + item := hiveBlocked(oldIssue) + mode := func(m RepoMode) Config { + return Config{PerRepo: map[string]RepoOverride{"org/repo": {Mode: m}}} + } + cases := []struct { + name string + cfg Config + level int + paused bool + listed bool + allow bool + reason string + }{ + {"paused repo", Config{}, 6, true, true, false, ReasonRepoPaused}, + {"repo mode off", mode(RepoModeOff), 6, false, true, false, ReasonRepoOff}, + {"repo mode issues-only", mode(RepoModeIssuesOnly), 6, false, true, false, ReasonRepoIssues}, + {"repo mode labels-only: the repair posts a note", mode(RepoModeLabelsOnly), 6, false, true, false, ReasonRepoLabels}, + {"repo mode full", mode(RepoModeFull), 6, false, true, true, ReasonRepoFull}, + {"repo not in the project", Config{}, 6, false, false, false, ReasonRepoNotListed}, + {"own_only set by an operator", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {OwnOnly: boolp(true)}}}, 6, false, true, false, ReasonOwnOnly}, + {"level unset", Config{}, 0, false, true, false, ReasonLevelUnknown}, + {"below a raised kind floor", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}, 2, false, true, false, ReasonBelowLevel}, + {"at a raised kind floor", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}, 3, false, true, true, ReasonHiveAppliedBlock}, + {"below the global floor", Config{Unsolicited: UnsolicitedConfig{MinLevel: intp(5)}}, 4, false, true, false, ReasonBelowLevel}, + {"require_labels off changes nothing", Config{Unsolicited: UnsolicitedConfig{RequireLabels: boolp(false)}}, 1, false, true, true, ReasonHiveAppliedBlock}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + g := New(tc.cfg, Deps{ + LevelFor: func(string) int { return tc.level }, + RepoListed: func(string) bool { return tc.listed }, + RepoPause: func(string) (RepoPauseInfo, bool) { return RepoPauseInfo{By: "owner"}, tc.paused }, + Now: func() time.Time { return t0 }, + }, "") + g.ObserveRepo("org/repo") + d := g.Allowed("org/repo", item, KindBlockedRepair) + if d.Allowed != tc.allow || d.Reason != tc.reason { + t.Fatalf("got %+v, want allow=%v reason=%s", d, tc.allow, tc.reason) + } + }) + } +} + +// An allowed repair of an old issue is not one of the "pre-existing items +// left untouched" the backfill banner counts; a refused one still is. +func TestBlockedRepairDoesNotCountAsBackfillSkip(t *testing.T) { + g, _ := newGate(t, Config{}, 6) + if d := g.Allowed("org/repo", hiveBlocked(oldIssue), KindBlockedRepair); !d.Allowed { + t.Fatalf("hive-applied block on an old issue = %+v, want allowed", d) + } + if n := g.Snapshot().Repos["org/repo"].PreExisting; n != 0 { + t.Fatalf("pre-existing untouched = %d after an allowed repair, want 0", n) + } + if d := g.Allowed("org/repo", oldIssue, KindBlockedRepair); d.Allowed || d.Reason != ReasonBackfill { + t.Fatalf("fact not stated = %+v, want refused by backfill", d) + } + if n := g.Snapshot().Repos["org/repo"].PreExisting; n != 1 { + t.Fatalf("pre-existing untouched = %d after a backfill refusal, want 1", n) + } +} + +// The note is one constant body, so the content layer treats it as it did +// under gate_comment: the same issue twice is a duplicate, a second issue in +// the same repo within the burst window is held, and both pass later. +func TestBlockedRepairKeepsDuplicateAndBurstHolds(t *testing.T) { + const note = "\nNeeds a maintainer: the `blocked` label named nothing it waits on." + g, clk, _ := newDedupeGate(t, Config{}, "") + first, second := hiveBlocked(oldIssue), hiveBlocked(newIssue) + if d := g.AllowedPreview("org/repo", first, KindBlockedRepair, note); !d.Allowed || d.Reason != ReasonHiveAppliedBlock { + t.Fatalf("first issue = %+v, want allowed", d) + } + if d := g.AllowedPreview("org/repo", first, KindBlockedRepair, note); d.Allowed || d.Reason != ReasonDuplicate { + t.Fatalf("same issue again = %+v, want held as a duplicate", d) + } + if d := g.AllowedPreview("org/repo", second, KindBlockedRepair, note); d.Allowed || d.Reason != ReasonBurst { + t.Fatalf("second issue in the burst window = %+v, want held as a burst", d) + } + clk.advance(2 * time.Minute) + if d := g.AllowedPreview("org/repo", second, KindBlockedRepair, note); !d.Allowed { + t.Fatalf("second issue after the window = %+v, want allowed", d) + } +} + +// NoteHeld is news once per item and reason; ForgetHeld and RetainHeld make +// a later hold news again. Kinds and repos are remembered apart. +func TestHeldMemory(t *testing.T) { + g, _ := newGate(t, Config{}, 6) + const kind = KindBlockedRepair + if !g.NoteHeld("Org/Repo", 80, kind, ReasonRepoPaused) { + t.Fatal("a first hold must be news") + } + if g.NoteHeld("org/repo", 80, kind, ReasonRepoPaused) { + t.Fatal("the same hold again must not be news, whatever the repo's case") + } + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a changed reason must be news") + } + if g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("the changed reason again must not be news") + } + if !g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) || !g.NoteHeld("org/other", 80, kind, ReasonRepoOff) || !g.NoteHeld("org/repo", 80, KindGateComment, ReasonRepoOff) { + t.Fatal("another issue, repo or kind is its own hold") + } + + g.ForgetHeld("ORG/repo", 80, kind) + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a hold after ForgetHeld must be news again") + } + + // Only org/repo#81 is still a target of this kind. + g.RetainHeld(kind, func(repo string, number int) bool { return repo == "org/repo" && number == 81 }) + if g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) { + t.Fatal("RetainHeld forgot an item it was told to keep") + } + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) || !g.NoteHeld("org/other", 80, kind, ReasonRepoOff) { + t.Fatal("RetainHeld kept an item that is no longer a target") + } + if g.NoteHeld("org/repo", 80, KindGateComment, ReasonRepoOff) { + t.Fatal("RetainHeld for one kind forgot another kind's hold") + } + g.RetainHeld(kind, nil) + if g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) { + t.Fatal("RetainHeld with no keep function must forget nothing") + } + + var none *Gate + if !none.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a nil gate remembers nothing: every hold is news") + } + none.ForgetHeld("org/repo", 80, kind) + none.RetainHeld(kind, func(string, int) bool { return false }) +} diff --git a/src/pkg/writeback/dedupe.go b/src/pkg/writeback/dedupe.go index 1a2651039e..8f5ca6b9a6 100644 --- a/src/pkg/writeback/dedupe.go +++ b/src/pkg/writeback/dedupe.go @@ -84,6 +84,10 @@ var ContentDedupeKinds = map[Kind]bool{ KindClaimComment: true, KindGateComment: true, KindWelcomeComment: true, + // The repair's note is one constant body: the burst hold is what keeps + // a repo with many marker-less blocks from getting them all in one + // sweep, as it did while the note went out under gate_comment. + KindBlockedRepair: true, } // RateCapExemptKinds are never held by the hourly cap: everything on the diff --git a/src/pkg/writeback/gate.go b/src/pkg/writeback/gate.go index c283f9dd17..3c266540e0 100644 --- a/src/pkg/writeback/gate.go +++ b/src/pkg/writeback/gate.go @@ -39,6 +39,13 @@ type Item struct { // caller whose policy closes superseded originals on adopt // (governor.labels.stage.adoption.close_superseded: on_adopt). SupersededBy int + // HiveAppliedBlock is true when the caller has established that the + // item's `blocked` label was most recently applied by the hive's own + // bot (the actor of the newest labeled event for it). Set only by the + // blocked sweep's repair of a block that names nothing it waits on; + // with KindBlockedRepair it lifts the backfill guard and the opt-in + // label requirement (ReasonHiveAppliedBlock). No other kind reads it. + HiveAppliedBlock bool // Labels is the item's current label set. Labels []string // CreatedAt is the item's creation time; the backfill guard compares it @@ -84,6 +91,14 @@ const ( // sign-off, so the close is the hive's own housekeeping, not an // unsolicited write on a person's work. ReasonSupersededAgentPR = "superseded-agent-pr" + // ReasonHiveAppliedBlock allows the blocked-repair kind on an item whose + // `blocked` label the hive itself applied. The hive has already written + // to that item, whatever its age and with no opt-in label, so replacing + // its own label with a needs-human ask is upkeep of the hive's state, + // not a first unsolicited write: the backfill guard and the opt-in + // label do not apply. A pause, the repo mode, project membership, + // own_only and the kind's minimum level still do. + ReasonHiveAppliedBlock = "hive-applied-block" ) // RepoPauseInfo is the provenance of an operator pause, as config records @@ -194,7 +209,10 @@ type Gate struct { recent []Suppressed summary map[string]bool // dedupe is the content layer's table (dedupe.go). - dedupe *dedupeState + dedupe *dedupeState + // held is the last refusal reason reported per item for a writer that + // asks again on every sweep (held.go). + held map[heldKey]string attentionOnce sync.Once attentionStore *attention.Store attentionErr error @@ -460,7 +478,11 @@ func (g *Gate) decideLocked(repo string, item Item, kind Kind) Decision { // carrying the opt-in label was asked for by a human, so the backfill // guard (which exists for items nobody asked about) does not apply. optedIn := g.optedInLocked(repo, item) - if !optedIn { + // Repairing a `blocked` the hive applied acts on an item the hive has + // already written to, so neither the backfill guard here nor the opt-in + // label below applies to it. The level still does. + hiveBlock := kind == KindBlockedRepair && item.HiveAppliedBlock + if !optedIn && !hiveBlock { if backfilled, cut := g.backfillBlockedLocked(repo, item); backfilled { return Decision{Reason: ReasonBackfill, Detail: fmt.Sprintf("#%d was created %s, before hive could write to %s (%s); writeback.unsolicited.backfill is off", item.Number, item.CreatedAt.UTC().Format(time.RFC3339), repo, cut.Format(time.RFC3339))} } @@ -489,6 +511,12 @@ func (g *Gate) decideLocked(repo string, item Item, kind Kind) Decision { if level <= 0 { return Decision{Reason: ReasonLevelUnknown, Detail: "acmm_level is unset; unsolicited writes need a level"} } + if hiveBlock { + if level >= p.MinLevel { + return Decision{Allowed: true, Reason: ReasonHiveAppliedBlock, Detail: fmt.Sprintf("the hive applied the `blocked` label on #%d itself and L%d ≥ L%d", item.Number, level, p.MinLevel)} + } + return Decision{Reason: ReasonBelowLevel, Detail: fmt.Sprintf("%s needs L%d; %s is at L%d", kind, p.MinLevel, repo, level)} + } if !p.RequireLabel { if level >= p.MinLevel { return Decision{Allowed: true, Reason: ReasonLevel, Detail: fmt.Sprintf("L%d ≥ L%d", level, p.MinLevel)} diff --git a/src/pkg/writeback/held.go b/src/pkg/writeback/held.go new file mode 100644 index 0000000000..b58ee2e9fd --- /dev/null +++ b/src/pkg/writeback/held.go @@ -0,0 +1,67 @@ +package writeback + +// A writer that asks the gate again on every sweep for the same item (the +// blocked sweep's repair, every few minutes) would otherwise report the same +// refusal each time. The gate keeps the reason it last refused each such item +// for, so the writer can say so once and again only when the reason changes. +// It lives here rather than with the writer because the gate is the one +// object a process shares across GitHub clients and rebuilds of them. It is +// in memory only: a restart reports each hold once more. + +// heldKey names one item a kind is being held on. +type heldKey struct { + repo string + number int + kind Kind +} + +// NoteHeld records that kind on repo#number was refused for reason and +// reports whether that is news: the item's first hold, or a reason other +// than the one last noted. A nil gate refuses nothing and has nothing to +// remember, so every call on it is news. +func (g *Gate) NoteHeld(repo string, number int, kind Kind, reason string) bool { + if g == nil { + return true + } + g.mu.Lock() + defer g.mu.Unlock() + key := heldKey{repo: repoKey(repo), number: number, kind: kind} + if last, ok := g.held[key]; ok && last == reason { + return false + } + if g.held == nil { + g.held = map[heldKey]string{} + } + g.held[key] = reason + return true +} + +// ForgetHeld drops what NoteHeld kept for kind on repo#number, because the +// write went through or the item is no longer one the writer would write to. +// A later hold of the item is news again. +func (g *Gate) ForgetHeld(repo string, number int, kind Kind) { + if g == nil { + return + } + g.mu.Lock() + defer g.mu.Unlock() + delete(g.held, heldKey{repo: repoKey(repo), number: number, kind: kind}) +} + +// RetainHeld forgets every hold of kind on an item keep does not report, +// which bounds the memory by the writer's current targets: an item that was +// closed, or whose label a person removed, never reaches ForgetHeld. keep is +// given the repo lower-cased and trimmed; it runs under the gate's lock and +// must not call the gate. +func (g *Gate) RetainHeld(kind Kind, keep func(repo string, number int) bool) { + if g == nil || keep == nil { + return + } + g.mu.Lock() + defer g.mu.Unlock() + for key := range g.held { + if key.kind == kind && !keep(key.repo, key.number) { + delete(g.held, key) + } + } +} diff --git a/src/pkg/writeback/kinds.go b/src/pkg/writeback/kinds.go index ba08f092ae..9d3c795a07 100644 --- a/src/pkg/writeback/kinds.go +++ b/src/pkg/writeback/kinds.go @@ -105,6 +105,14 @@ const ( // the file to another PR, a contributor dropped the change). A hold a // person set, or one with another hold source, is never removed. KindHumanMergePathRelease Kind = "human_merge_path_release" + // KindBlockedRepair turns a `blocked` label the hive itself applied, with + // nothing naming what the issue waits on, into a needs-human ask: it + // adds `needs-human`, removes `blocked` and posts one note saying what + // was missing and the two ways out. The caller states that the label is + // the hive's on the item (Item.HiveAppliedBlock); the gate then skips the + // backfill guard and the opt-in label for it (ReasonHiveAppliedBlock). + // A person's `blocked` is never repaired and never reaches the gate. + KindBlockedRepair Kind = "blocked_repair" ) // KindPolicy is the per-kind rule. A write of this kind on an item the hive @@ -167,6 +175,15 @@ func DefaultKinds() map[Kind]KindPolicy { // changes a human-merge path: wherever the hive could say so, it can // take the hold back when that stops being true. KindHumanMergePathRelease: {MinLevel: 5, RequireLabel: true}, + // The label this kind repairs is put on by a contributor relay with + // a hub-minted task token: a write no ACMM level and no write-back + // kind gates, so a marker-less hive `blocked` can sit on an issue at + // any level. The floor is therefore the lowest level there is: + // wherever the hive's label could land, its repair can follow, and + // an unset level still refuses. RequireLabel governs only a call + // that does not state Item.HiveAppliedBlock (none does today), so a + // caller that forgets the fact is held like any other gated kind. + KindBlockedRepair: {MinLevel: 1, RequireLabel: true}, } }