From 9d1f974da39a5f61c28e29c8df9989226ba0079b Mon Sep 17 00:00:00 2001 From: Danathar Date: Thu, 12 Nov 2026 15:50:21 -0500 Subject: [PATCH] =?UTF-8?q?=F0=9F=90=9B=20fix(blocked):=20turn=20a=20hive-?= =?UTF-8?q?applied=20marker-less=20`blocked`=20into=20a=20needs-human=20as?= =?UTF-8?q?k=20on=20old=20and=20unlabelled=20issues=20too?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A contributor relay's blocked verdict that names nothing leaves a `blocked` label applied by the hive's own bot, with no marker, so nothing can lift it. The blocked sweep turns that into a needs-human ask, but it asked the write-back gate as gate_comment, and the gate refused on every sweep for an issue created before the hive could write to the repository (backfill off) and for one without the review opt-in label. Those issues stayed blocked, out of every agent's reach and in no inbox. The hive has already written to such an issue: it applied the label being repaired. So the repair gets its own write-back kind, blocked_repair, and the sweep passes the fact it already established from the timeline (Item.HiveAppliedBlock). With the kind and the fact, the gate skips the backfill guard and the opt-in label and allows with reason hive-applied-block. A pause, the repo mode (off, issues-only, labels-only), project membership, own_only and the kind's minimum level still refuse it, and the content dedupe and burst hold apply as they did under gate_comment. Without the fact the kind is gated like any other, and the fact does nothing for other kinds. Default policy is min_level 1 with require_label: the relay's label write is gated by no level, so the floor is the lowest one, and require_label only matters for a call that does not state the fact. A held repair is now logged once per issue and reason, again when the reason changes or the issue is held anew, instead of on every sweep. The gate keeps that memory (held.go), since it is shared across clients. The dashboard kind matrix and the admin MCP writeback.kind_policy both build from the gate's kind list, so the new kind shows in each. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar --- .../fixed-blocked-repair-writeback-gate.md | 1 + src/docs/acmm-policy-matrix.md | 9 + .../labels-and-control-signals/reference.md | 9 +- .../sweeps-and-operations.md | 10 +- src/pkg/github/issue_blocked.go | 30 +- src/pkg/github/issue_blocked_repair_held.go | 51 +++ src/pkg/github/issue_blocked_repair_test.go | 404 ++++++++++++++++++ src/pkg/writeback/blocked_repair_test.go | 232 ++++++++++ src/pkg/writeback/dedupe.go | 4 + src/pkg/writeback/gate.go | 32 +- src/pkg/writeback/held.go | 67 +++ src/pkg/writeback/kinds.go | 17 + 12 files changed, 855 insertions(+), 11 deletions(-) create mode 100644 changelog.d/fixed-blocked-repair-writeback-gate.md create mode 100644 src/pkg/github/issue_blocked_repair_held.go create mode 100644 src/pkg/github/issue_blocked_repair_test.go create mode 100644 src/pkg/writeback/blocked_repair_test.go create mode 100644 src/pkg/writeback/held.go diff --git a/changelog.d/fixed-blocked-repair-writeback-gate.md b/changelog.d/fixed-blocked-repair-writeback-gate.md new file mode 100644 index 0000000000..a735fa6b4d --- /dev/null +++ b/changelog.d/fixed-blocked-repair-writeback-gate.md @@ -0,0 +1 @@ +- Blocked sweep: a `blocked` label the hive applied itself, with nothing naming what the issue waits on, is now turned into a `needs-human` ask on every issue, including one created before the hive could write to the repository and one without the review opt-in label. The repair went through the write-back gate as `gate_comment`, so the backfill guard (`writeback.unsolicited.backfill` off) and the opt-in label requirement refused it on every sweep, although the hive had already labelled the issue; the issue stayed `blocked`, out of every agent's reach and in no inbox, for days. The repair is now its own write-back kind, `blocked_repair` (`writeback.kinds.blocked_repair`, default minimum level 1, shown in Settings → Features → Write-back containment and settable with the admin MCP `writeback.kind_policy`): when the sweep has established that the hive's own bot applied the label, neither of those two checks applies. A paused repository, `writeback.per_repo` mode `off`, `issues-only` or `labels-only`, a repository outside `project.repos`, `own_only`, the kind's minimum level and the notification burst hold still refuse it, and a person's `blocked` is left alone as before. A held repair is logged once per issue and reason (`blocked sweep: needs-human ask held by writeback gate`) instead of on every sweep; repeats go to the debug log. diff --git a/src/docs/acmm-policy-matrix.md b/src/docs/acmm-policy-matrix.md index 27cef1ea37..e343d26a9f 100644 --- a/src/docs/acmm-policy-matrix.md +++ b/src/docs/acmm-policy-matrix.md @@ -274,6 +274,7 @@ gated. Defaults (override under `writeback.kinds.`): | `claim_comment` | — | opt-in label | opt-in label | opt-in label | | `welcome_comment` | — | — | opt-in label | all items | | `issue_file` (new hive-authored issues) | L3+ | yes | yes | yes | +| `blocked_repair` (a `blocked` label the hive applied itself, with no blocker named) | L1+ | yes | yes | yes | `ci_rerun` also covers the one transient re-run of a red PR from `auto_merge.trusted_bot_authors` (a hive PR is its own item and is not gated): @@ -283,6 +284,14 @@ hands the PR to a person with the reason. `bot_rebase_request` only ticks Renovate's rebase/retry box; a stranded Dependabot PR is handed to a person instead, because Dependabot refuses commands from apps. +`blocked_repair` replaces a marker-less `blocked` the hive applied with a +`needs-human` ask (see +[`blocked`](labels-and-control-signals/reference.md#blocked)). It needs no +opt-in label and ignores the backfill guard, because the hive has already +written to that issue: a contributor relay applies the label with a task +token at any level, so the repair has the lowest floor. On an issue whose +`blocked` the hive did not apply, the kind is never used. + In practice: **L4 means hive files issues and talks only on its own PRs**; **L5 adds comments and labels on PRs a maintainer labelled `hive-review`**; **L6 lets sentinel and welcome comments reach unlabelled PRs**. A repo diff --git a/src/docs/labels-and-control-signals/reference.md b/src/docs/labels-and-control-signals/reference.md index 1a2cb32710..66427af32e 100644 --- a/src/docs/labels-and-control-signals/reference.md +++ b/src/docs/labels-and-control-signals/reference.md @@ -94,7 +94,7 @@ Some hive-authored labels look like provenance but are read as signals, so they | `needs-sha` from SHA-hold | Issues | SHA-hold sweep | Trusted-reporter `kind/bug` without a SHA gets this soft label plus the same `` notice instead of `hold`; the issue stays actionable. The sweep removes it once a SHA appears. | Soft (informational) | Hive | Eval tick SHA sweep | Add SHA evidence, or remove the label | `issues.sha_hold_trusted_label` (default `needs-sha`; `hold` restores the hard hold for everyone) | | `do-not-merge` and `do-not-merge*` | Issues, PRs | Exempt filter, merge sweeps, task-list sweep | Permanently exempt; exact match is case-insensitive but prefix matching follows Go `strings.HasPrefix` on the original label. | Hard suppress | Human | Enumeration/sweeps | Human removes | Built-in `PermanentExemptLabels` (`src/pkg/github/client.go:750-755`, `src/pkg/github/client.go:2035-2053`) | | `governor.labels.exempt` entries | Issues, PRs | Exempt filter | Same exempt behavior as `do-not-merge`; wins over required-label admission. Defaults include `nightly-tests`, `LFX`, `meta-tracker`, `auto-qa-tuning-report`, `adopters`, `changes-requested`, `waiting-on-author`. | Hard suppress | Operator/dashboard | Enumeration/sweeps | Remove label or config entry | `governor.labels.exempt` (`src/pkg/config/governor_config.go:175-184`, `src/pkg/config/defaults.go:260-270`) | -| `blocked` | Issues and PRs | Enumeration, stage derivation, claim gate, review backlog, blocked sweep | The issue waits on an unmerged PR (or another outside blocker): the review backlog files a finding `blocked` when its evidence file exists only on the reviewed PR (absent on the base branch), and `hive-open-issue comment --blocked-on-pr ` parks an existing one; both write ``. A `blocked` issue is filtered (reason `blocked on owner/repo#N`), derives no stage label, is refused by every automated claim producer (`blocked: owner/repo#N`) including the contributor relay, and never carries `help wanted` ([#13093](https://github.com/hivecommons/hive/issues/13093)). When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker, but only on a review-backlog item (its body carries `filed by Hive review backlog`) that GitHub reports no open "blocked by" link on ([#13364](https://github.com/hivecommons/hive/issues/13364)). A human's marker-less `blocked` stays. A contributor relay's `blocked` verdict that named nothing (GitHub records it as the App bot too, because the relay labels with its hub-minted task token) is turned into a `needs-human` ask instead: `needs-human` is added, `blocked` removed and one note says what is missing. Lifting it only sent the issue back to the next agent, which re-applied it, and keeping it hid the issue from every inbox. A blocked verdict whose reason is empty or only the relay footer is booked idle and its label released, like a tooling failure. A marker in a comment counts like one in the body: the row's reason (`blocked on owner/repo#N`) and the band signal name it. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). On a PR, `blocked` holds the merge: the blocked sweep puts it on an open, non-draft PR whose `Depends-on:` line names an open pull request (same repository or another), with one `` comment per open target, and every merge lane refuses a `blocked` PR (`label:blocked`). A person's `blocked` is never lifted; a person who removes the hive's `blocked` is not overridden for the same dependencies. | Hard suppress | Hive (reserved for agents; relay only) | Review backlog filing; blocked-on comment relay | The blocked sweep after each stage-label sync removes it, with one `` note, once the tracked PR is merged or closed, or once a recheck-after deadline passes (`deadline passed; re-triaging`) unless the body also carries ``. On a PR it removes the hive's own `blocked` once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. An ops-tracking issue on the hive's own project keeps `blocked` once its deadline passes and instead gets soak triage: the hub's `/api/hub/hives` (or, with no hub, the spoke's own `/api/version`) is probed for every `stable`-channel hive (fresh heartbeat, no restart loop, running the candidate `sha256:` digest or `v5 ` named in the body), ONE `` comment is posted, only the checklist lines that evidence proves are ticked, and `` is appended to the body so the sweep does not repeat ([#13311](https://github.com/hivecommons/hive/issues/13311)); a human may remove it at any time | Fixed label (`src/pkg/github/issue_blocked.go`, `src/pkg/github/pr_blocked_depends_on.go`, `src/pkg/github/issue_recheck_soak.go`, `src/pkg/github/issue_stage_labels.go`, `src/pkg/github/issue_claim_gate.go`) | +| `blocked` | Issues and PRs | Enumeration, stage derivation, claim gate, review backlog, blocked sweep | The issue waits on an unmerged PR (or another outside blocker): the review backlog files a finding `blocked` when its evidence file exists only on the reviewed PR (absent on the base branch), and `hive-open-issue comment --blocked-on-pr ` parks an existing one; both write ``. A `blocked` issue is filtered (reason `blocked on owner/repo#N`), derives no stage label, is refused by every automated claim producer (`blocked: owner/repo#N`) including the contributor relay, and never carries `help wanted` ([#13093](https://github.com/hivecommons/hive/issues/13093)). When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker, but only on a review-backlog item (its body carries `filed by Hive review backlog`) that GitHub reports no open "blocked by" link on ([#13364](https://github.com/hivecommons/hive/issues/13364)). A human's marker-less `blocked` stays. A contributor relay's `blocked` verdict that named nothing (GitHub records it as the App bot too, because the relay labels with its hub-minted task token) is turned into a `needs-human` ask instead: `needs-human` is added, `blocked` removed and one note says what is missing. Because the hive put that label on, the repair (write-back kind `blocked_repair`) also runs on an issue older than the hive's write access to the repository and on one without the review opt-in label. Lifting it only sent the issue back to the next agent, which re-applied it, and keeping it hid the issue from every inbox. A blocked verdict whose reason is empty or only the relay footer is booked idle and its label released, like a tooling failure. A marker in a comment counts like one in the body: the row's reason (`blocked on owner/repo#N`) and the band signal name it. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). On a PR, `blocked` holds the merge: the blocked sweep puts it on an open, non-draft PR whose `Depends-on:` line names an open pull request (same repository or another), with one `` comment per open target, and every merge lane refuses a `blocked` PR (`label:blocked`). A person's `blocked` is never lifted; a person who removes the hive's `blocked` is not overridden for the same dependencies. | Hard suppress | Hive (reserved for agents; relay only) | Review backlog filing; blocked-on comment relay | The blocked sweep after each stage-label sync removes it, with one `` note, once the tracked PR is merged or closed, or once a recheck-after deadline passes (`deadline passed; re-triaging`) unless the body also carries ``. On a PR it removes the hive's own `blocked` once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. An ops-tracking issue on the hive's own project keeps `blocked` once its deadline passes and instead gets soak triage: the hub's `/api/hub/hives` (or, with no hub, the spoke's own `/api/version`) is probed for every `stable`-channel hive (fresh heartbeat, no restart loop, running the candidate `sha256:` digest or `v5 ` named in the body), ONE `` comment is posted, only the checklist lines that evidence proves are ticked, and `` is appended to the body so the sweep does not repeat ([#13311](https://github.com/hivecommons/hive/issues/13311)); a human may remove it at any time | Fixed label (`src/pkg/github/issue_blocked.go`, `src/pkg/github/pr_blocked_depends_on.go`, `src/pkg/github/issue_recheck_soak.go`, `src/pkg/github/issue_stage_labels.go`, `src/pkg/github/issue_claim_gate.go`) | | `needs-human` | Issues | Enumeration, task-list sweep, claim escalation gate and un-park sweep | Filters issues from agent kicks and contributor offers; records a human decision requirement. See the [issue reference](#needs-human) and [Maintainer commands](../maintainer-commands.md). | Hard suppress | Hive or human | Enumeration/task-list sweep/kick building | An eligible maintainer's ordinary `/hive approve` or `/hive decision ` lifts it regardless of who applied it; label-driven un-parking remains provenance-sensitive. Keep `hold` if work must stay stopped. Agents may add, never remove it. | Fixed label (`src/pkg/github/client.go:1008-1024`, `src/pkg/github/task_list_sweep.go:611-641`, `src/pkg/github/task_list_sweep.go:874-880`, `src/cmd/hive/claimescalation.go`) | | `needs-spec` from the clarity gate | Issues | Scheduler kick assembly (`runs.triage.clarity_gate`) | With the gate on (default off), a human-filed, unclaimed issue whose clarity score (goal, acceptance criteria, scope, detail, open choices) is under `runs.triage.clarity_gate.threshold` gets one `` comment with concise questions for its author and the `needs-spec` hard-suppress label instead of a claim; it derives `stage/045-needs-decision` and leaves the agent queue ([#13577](https://github.com/hivecommons/hive/issues/13577)). | Hard suppress | Hive | Before kicks are built | The issue author, or a non-hive commenter with write/maintain/admin, replies after the marker and the resume sweep removes the hive's own `needs-spec` and records the un-park so the issue re-derives `stage/010-actionable` ([#13588](https://github.com/hivecommons/hive/issues/13588)); a `needs-spec` a person added is never lifted by the sweep. Or remove `needs-spec` by hand. The marker keeps the gate from asking again | `runs.triage.clarity_gate` (`src/pkg/scheduler/clarity_gate.go`, `src/pkg/classify/clarity.go`, `src/pkg/github/clarity_gate_resume.go`) | | `stage/017-possible-existing` from the prior-art gate | Issues | Scheduler kick assembly (`runs.triage.prior_art`) | With the gate on (default on), an unclaimed feature / RFC / epic issue whose key terms match an existing knowledge bead, setting, merged PR, closed issue, doc or code identifier at `runs.triage.prior_art.threshold` (default 70) gets one `` comment quoting the findings and the `stage/017-possible-existing` label instead of a claim; hive-filed issues get a `` note and are never parked ([#13575](https://github.com/hivecommons/hive/issues/13575)). | Hard suppress | Hive | Before kicks are built | The author or a write+ commenter replies `/hive decision proceed — ` (label lifted, a *Scope vs existing* section appended to the body, re-derives 010) or `/hive decision exists` (closed `not_planned` with `already-exists`); a plain reply lifts the label too. A clanker-filed park with no reply is closed `already-exists` after `runs.triage.prior_art.bot_close_ttl` (default 72h); reopen to continue. A 017 a person added is never lifted by the sweep | `runs.triage.prior_art` (`src/pkg/priorart`, `src/pkg/scheduler/prior_art_gate.go`, `src/pkg/github/prior_art_gate_resume.go`) | @@ -301,6 +301,13 @@ The issue waits on an unmerged PR (or another outside blocker): the review backl When a blocked filing folds into an already-open twin, the twin's own state governs: `blocked` is not applied to it, because the folded body (and its marker) is discarded and the twin's earlier evidence stands on its own; the blocked sweep also lifts a hive-applied `blocked` that has no marker, no deadline and no ops-tracking marker on a review-backlog item, leaving a human-applied one alone ([#13364](https://github.com/hivecommons/hive/issues/13364)). Any other hive-applied marker-less `blocked` (a contributor relay's verdict that named no issue or PR) becomes a `needs-human` ask, with one note naming the two ways out: name the dependency, or remove `needs-human` to re-queue it. A verdict with no reason at all is not blocked: the hub books it idle and releases the label. With no blocked-on marker, a body deadline `` files it `blocked until ` and the activity monitor's blocked chip reads `re-check at ` ([#13310](https://github.com/hivecommons/hive/issues/13310)). +**The repair works on old and unlabelled issues.** Turning a marker-less hive `blocked` into a `needs-human` ask goes through the write-back gate as its own kind, `blocked_repair`. Two of the gate's usual checks do not apply to it, because the hive applied the label being repaired and so has already written to the issue: + +- the backfill guard: the repair runs on an issue created before the hive first saw, or could first write to, the repository, even with `writeback.unsolicited.backfill` off; +- the review opt-in label: the repair runs on an issue that does not carry it. + +The sweep checks who applied the label (the actor of the newest `labeled` event) before it asks the gate, so a person's `blocked` never gets this treatment. Everything else still holds the repair: a paused repository, `writeback.per_repo` mode `off`, `issues-only` or `labels-only` (the repair posts a note, so it is not a label-only write), a repository outside `project.repos`, `own_only` if an operator sets it, and the kind's minimum level (`writeback.kinds.blocked_repair.min_level`, default 1; an unset `acmm_level` refuses). The burst hold applies as well, so a repository with several such issues gets them over a few sweeps. A held repair is logged once per issue and reason (`blocked sweep: needs-human ask held by writeback gate`) and again when the reason changes, not on every sweep. + With the marker (in the body, or in a comment the blocked sweep has read), the band signal reads `blocked on owner/repo#N — lifted when it merges or closes` instead of `the label records no reason`, the activity monitor's pill reads `blocked on #N`, links the PR and carries its cached state (`blocked_on_pr` in `/api/stage-tracker`), and the hive's own removal of the label re-reads the issue live rather than waiting for the next enumeration ([#14952](https://github.com/hivecommons/hive/issues/14952)). With the issue form, the blocker joins `blocked_on` instead and the caption reads `Blocked on issue #N — lifts when it closes` ([#16852](https://github.com/hivecommons/hive/issues/16852)). It is also always skipped by contributor admission and shown in the default waiting band. **On a pull request: `Depends-on:` holds the merge.** A PR body's `Depends-on: #N` or `Depends-on: owner/repo#N` lines (the same lines the dashboard's critical path reads) say the PR must land after another one, for example a change whose tests need a pull request in another repository. The blocked sweep (every 5 minutes, over the configured repositories) puts `blocked` on an open, non-draft PR whose `Depends-on:` names at least one open pull request, and posts one `` comment per open target (the same marker and writer a relay's blocked verdict uses). Every merge lane refuses a `blocked` PR with `label:blocked`: the self-authored and trusted-bot lane, the trusted-author lane, the contributor lane (whatever `exclude_labels` says), the queued merge-request lane, the merge train, the native merge queue (the App's own entry is withdrawn), the dashboard's merge verdict and its human-merge shortcuts. Like a hold it is a gate, not a stage: the PR's stage is its head's progress and the route carries `blocked on owner/repo#N`; the activity monitor row names the target (`blocked_on_pr`, caption `Blocked on PR #N — lifts when it merges`). A target that is an issue, a closed PR, or a PR the App cannot read places nothing (an unreadable one is logged once). The sweep never places over a person's `blocked`, never lifts one, and never puts its own back after a person removed it, until the body names an open pull request it has not marked yet. It lifts its own `blocked`, with one `` note, once every marked pull request has merged or closed (the note says when one closed without merging) or is no longer listed under `Depends-on:`. Both writes go through the write-back gate (`gate_comment`), and the sweep needs the App bot login to tell its label from a person's. Stacked PRs (a base branch that is another open PR's) are not handled. diff --git a/src/docs/operator-reference/sweeps-and-operations.md b/src/docs/operator-reference/sweeps-and-operations.md index d16f5c71f7..f830b3235a 100644 --- a/src/docs/operator-reference/sweeps-and-operations.md +++ b/src/docs/operator-reference/sweeps-and-operations.md @@ -550,7 +550,8 @@ that level, the power the write-back exists to govern? | Adoption close-first (`automerge/automerge_adoption_close_first.go`) | auto-merge sweep, when a supersede adoption opens the hive's copy; close-first poll | close-first note, `hive/superseded` label and close on the original; reopen, copy-failed note and label removal when the copy fails | the contributor's original PR | `contributor_adopt` → L6 + label (an agent-authored original closes without the label, `adoption.close_superseded: on_adopt`); refused: the original stays open until the copy merges, and a due reopen waits for the next poll | | Claims (`prclaims.go`, `claimed_by_label.go`, `cmd/hive/claimswire.go`, `claimescalation.go`) | claim ledger changes | `claimed-by:*` labels, claim/escalation comments | claimed issues | `claim_comment` → L4 + label (hive-claimed issues are own items) | | Issue close / task-list / decomposed parents (`issue_close.go`, `task_list_sweep.go`, `decomposed_parents.go`, `close_on_merge.go`, `tracking_issue.go`) | merges, sweeps | close, body edits (checkbox ticks), `Refs` rewrites | issues linked from hive PRs | `issue_close`, `parent_issue_edit` → L5 + label; hive-filed issues are own items | -| Gate comments (`pkg/scheduler/clarity_gate.go`, `prior_art_gate.go`, `triage.go`, `issue_blocked.go`, `issue_recheck_soak.go`) | scheduler intake | clarity/prior-art/triage questions, `blocked` labels | issues admitted by the issue filter | `gate_comment` → L5 + label; items carrying the intake label are opted in | +| Gate comments (`pkg/scheduler/clarity_gate.go`, `prior_art_gate.go`, `triage.go`, `verdict_blocker.go`, `pr_blocked_depends_on.go`, `issue_recheck_soak.go`) | scheduler intake | clarity/prior-art/triage questions, `blocked` labels | issues admitted by the issue filter | `gate_comment` → L5 + label; items carrying the intake label are opted in | +| Blocked repair (`pkg/github/issue_blocked.go`) | blocked sweep (every 5 min) and the blocked reconcile after each stage-label sync | `needs-human` added, `blocked` removed, one `` note asking a maintainer to name the dependency or re-queue the issue | open issues whose `blocked` the hive's own bot applied and that name nothing they wait on (no blocked-on marker, deadline, ops-tracking marker or native "blocked by" link; not a review-backlog item) | `blocked_repair` → L1, no opt-in label and no backfill guard, because the hive already wrote the label being repaired; a pause, `per_repo.mode` `off` / `issues-only` / `labels-only` and a raised `min_level` still refuse it. Refused: the issue keeps `blocked` and the hold is logged once per issue and reason | | Welcome / outreach (`pkg/gfi`, `gfi_forge.go`, `pr_clanker_requested.go`) | good-first-issue selection | welcome comment, `good first issue` label | issues hive advertises | `welcome_comment` → L5 with label, L6 without | | Issue filing (`pkg/upstreamwatch/github_filer.go`, `advisory.go`, scanner/deps/regression agents, `dedupe_advisory_close.go`) | agents, advisory digest, upstream watch | **new** issues hive authors, pinned advisory issue + comments | `project.repos` | `issue_file` → L3, no label (this is what listing a repo opts in to; `per_repo.mode: issues-only` keeps only this) | | Code scanning sweep (`pkg/github/code_scanning.go`, `pkg/codescanning`) | hourly sweep (`code_scanning`, opt-in per repo) | **new** grouped issues from code scanning alerts, alert-table body edits, one closing comment + close when every alert is fixed or dismissed | the hive's own `hive/code-scanning` issues in repos with `code_scanning` on | `issue_file` → L3, no label (own items); see [code-scanning.md](../code-scanning.md) | @@ -610,9 +611,10 @@ Policy decides whether a kind may write to an item; the content layer under only to comment kinds whose body the caller handed the gate as the preview (`sentinel`, `needs_decision_comment`, `supersession_notice`, `duplicate_fix_notice`, `reporter_trust_hold`, `contributor_adopt`, -`claim_comment`, `gate_comment`, `welcome_comment`); `Allowed()` without a -preview — labels, closes, hold-gate upkeep, task-list edits — stays -policy-only. Three checks run after the policy gate allows a write: +`claim_comment`, `gate_comment`, `welcome_comment`, `blocked_repair`); +`Allowed()` without a preview — labels, closes, hold-gate upkeep, task-list +edits — stays policy-only. Three checks run after the policy gate allows a +write: | Check | Key | Suppression reason | Default | |---|---|---|---| diff --git a/src/pkg/github/issue_blocked.go b/src/pkg/github/issue_blocked.go index a1166c2b29..88ac669c86 100644 --- a/src/pkg/github/issue_blocked.go +++ b/src/pkg/github/issue_blocked.go @@ -471,6 +471,7 @@ func (c *Client) SweepBlockedIssues(ctx context.Context, opts BlockedSweepOption }) } } + c.retainBlockedRepairHeld(repos, targets) return c.reconcileBlockedTargets(ctx, targets, c.logger), nil } @@ -532,6 +533,8 @@ func (c *Client) reconcileBlockedTargets(ctx context.Context, targets []blockedI stripBlockedHelpWanted(ctx, c, t, &res, logger) continue } + // A block that names what it waits on is not one the repair converts. + forgetBlockedRepairHeld(c, t) if onIssue { reconcileBlockedOnIssue(ctx, c, t, prRepo, prNumber, &res, logger) continue @@ -706,8 +709,17 @@ const blockedMarkerlessNote = blockedUnblockedMarker + "\nUnblocked: the `blocke // and puts it in the maintainer inbox with the question. Only a hive-applied // label with no open native "blocked by" link is converted; a person's // marker-less block is theirs. Reports whether the issue was converted. +// +// The write-back gate is asked as blocked_repair with the fact established +// here, that the label is the hive's (Item.HiveAppliedBlock). The hive has +// already written to such an issue, so the gate does not hold the repair back +// for the issue predating the hive's write access or lacking the opt-in +// label; while it was asked as gate_comment both did, every sweep, and the +// issue stayed in the invisible state above for days. A pause, the repo mode +// and the kind's minimum level still refuse it. func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIssueTarget, res *BlockedIssueResult, logger *slog.Logger) bool { if t.reviewBacklog || t.openBlockedBy > 0 { + forgetBlockedRepairHeld(client, t) return false } actor, found, err := client.blockedLabelActor(ctx, t.repo, t.number) @@ -717,6 +729,7 @@ func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIss return false } if !found || (!client.isOwnBotLogin(actor) && !client.IsHiveAuthorLogin(actor)) { + forgetBlockedRepairHeld(client, t) return false } item, err := client.WritebackItemForNumber(ctx, t.repo, t.number) @@ -725,14 +738,23 @@ func askHumanBlockedMarkerless(ctx context.Context, client *Client, t blockedIss logger.Warn("blocked sweep: item lookup failed; marker-less block left alone", "repo", t.repo, "issue", t.number, "error", err) return false } - owner, name := client.splitRepo(t.repo) - if d := client.WritebackAllowed(owner+"/"+name, item, writeback.KindGateComment, blockedMarkerlessAskNote); !d.Allowed { + item.HiveAppliedBlock = true + gateRepo := client.blockedRepairRepo(t.repo) + if d := client.WritebackAllowed(gateRepo, item, writeback.KindBlockedRepair, blockedMarkerlessAskNote); !d.Allowed { // The burst hold lets a couple of asks per repo through per sweep, // and a failed write's retry is a duplicate for same_item_ttl; both - // clear on a later sweep. Say so rather than fall through silently. - logger.Info("blocked sweep: needs-human ask held by writeback gate", "repo", t.repo, "issue", t.number, "reason", d.Reason, "detail", d.Detail) + // clear on a later sweep. Say so rather than fall through silently, + // but once per issue and reason: the sweep asks again every few + // minutes, and a hold that lasts (a pause, the repo mode, the level) + // would repeat the line each time. Repeats go to the debug log. + level := slog.LevelDebug + if client.WritebackGate().NoteHeld(gateRepo, t.number, writeback.KindBlockedRepair, d.Reason) { + level = slog.LevelInfo + } + logger.Log(ctx, level, "blocked sweep: needs-human ask held by writeback gate", "repo", t.repo, "issue", t.number, "reason", d.Reason, "detail", d.Detail) return false } + forgetBlockedRepairHeld(client, t) if !issueHasLabel(t.labels, issueNeedsHumanLabel) { if err := client.AddLabels(ctx, t.repo, t.number, []string{issueNeedsHumanLabel}); err != nil { // writeback:gated res.Errors++ diff --git a/src/pkg/github/issue_blocked_repair_held.go b/src/pkg/github/issue_blocked_repair_held.go new file mode 100644 index 0000000000..51d5c65090 --- /dev/null +++ b/src/pkg/github/issue_blocked_repair_held.go @@ -0,0 +1,51 @@ +package github + +import ( + "strings" + + "github.com/hivecommons/hive/pkg/writeback" +) + +// The blocked sweep asks the write-back gate about every marker-less +// hive-applied block on every pass, and a refusal that lasts would be logged +// each time. The gate remembers the reason it last refused each issue for +// (writeback.Gate.NoteHeld); these helpers keep that memory in step with the +// sweep, so a hold is reported once per issue and reason and again after the +// issue stopped being one the repair converts. + +// blockedRepairRepo is repo as owner/name, the form the gate is asked with. +func (c *Client) blockedRepairRepo(repo string) string { + owner, name := c.splitRepo(repo) + return owner + "/" + name +} + +// forgetBlockedRepairHeld drops the hold remembered for t: the repair went +// through, or t is no longer a block the repair converts (it names what it +// waits on, a person applied it, it is a review-backlog item). A later hold +// is then logged again. +func forgetBlockedRepairHeld(client *Client, t blockedIssueTarget) { + client.WritebackGate().ForgetHeld(client.blockedRepairRepo(t.repo), t.number, writeback.KindBlockedRepair) +} + +// retainBlockedRepairHeld forgets the holds of issues in repos that targets, +// the complete listing of open `blocked` issues in those repos, no longer +// contains: an issue that was closed, or whose label was removed, never +// reaches the repair again. A repo outside this listing keeps its holds, so +// a sweep of some repos does not forget the others. +func (c *Client) retainBlockedRepairHeld(repos []string, targets []blockedIssueTarget) { + gate := c.WritebackGate() + if gate == nil { + return + } + listed := make(map[string]bool, len(repos)) + for _, repo := range repos { + listed[strings.ToLower(c.blockedRepairRepo(repo))] = true + } + open := make(map[string]bool, len(targets)) + for _, t := range targets { + open[blockedOnSeenKey(c.blockedRepairRepo(t.repo), t.number)] = true + } + gate.RetainHeld(writeback.KindBlockedRepair, func(repo string, number int) bool { + return !listed[repo] || open[blockedOnSeenKey(repo, number)] + }) +} diff --git a/src/pkg/github/issue_blocked_repair_test.go b/src/pkg/github/issue_blocked_repair_test.go new file mode 100644 index 0000000000..cf6de8baf8 --- /dev/null +++ b/src/pkg/github/issue_blocked_repair_test.go @@ -0,0 +1,404 @@ +package github + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" + + "github.com/hivecommons/hive/pkg/writeback" +) + +// The blocked sweep turns a `blocked` the hive applied, naming nothing it +// waits on, into a needs-human ask. These tests run that repair through a +// real write-back gate. Asked as gate_comment, the gate refused it on every +// sweep for an issue older than the hive's write access (backfill off) and +// for one without the opt-in label, so the issue stayed blocked, in no inbox, +// for days. Asked as blocked_repair with the fact that the label is the +// hive's, neither check applies; a pause, the repo mode and the level still do. + +const ( + blockedRepairTestRepo = "acme/widgets" + blockedRepairIssue = 80 + blockedRepairHeldLine = "needs-human ask held by writeback gate" + // blockedRepairPerson is a maintainer's login: the issue's author, and + // in some tests the one who applied `blocked`. + blockedRepairPerson = "maintainer" +) + +// blockedRepairFirstSeen is when the gate first saw acme/widgets: an issue +// created before it is one the backfill guard keeps the hive off. +var blockedRepairFirstSeen = time.Date(2026, 6, 1, 12, 0, 0, 0, time.UTC) + +const ( + blockedRepairOldIssue = "2026-01-15T09:00:00Z" // before blockedRepairFirstSeen + blockedRepairNewIssue = "2026-07-01T09:00:00Z" // after it +) + +// blockedRepairServer is acme/widgets with one open issue, #80. Its fields +// are the issue's state on GitHub and may be changed between sweeps. +type blockedRepairServer struct { + mu sync.Mutex + calls []string + // blocked: #80 carries `blocked` and is open (so the sweep lists it). + blocked bool + // actor applied `blocked`, per the issue's timeline. + actor string + // createdAt is the issue's created_at. + createdAt string + // marker: a comment on #80 names the open pull request #7 as its blocker. + marker bool + // blockedBy: GitHub reports an open native "blocked by" link. + blockedBy bool +} + +func (f *blockedRepairServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { + f.mu.Lock() + defer f.mu.Unlock() + w.Header().Set("Content-Type", "application/json") + f.calls = append(f.calls, r.Method+" "+r.URL.Path) + const issues = "/repos/" + blockedRepairTestRepo + "/issues" + const issue = issues + "/80" + switch { + case r.Method == "GET" && r.URL.Path == issues && strings.Contains(r.URL.RawQuery, "labels=blocked"): + items := []map[string]any{} + if f.blocked { + items = append(items, f.issueLocked()) + } + _ = json.NewEncoder(w).Encode(items) + case r.Method == "GET" && r.URL.Path == issue: + _ = json.NewEncoder(w).Encode(f.issueLocked()) + case r.Method == "GET" && r.URL.Path == issue+"/comments": + if f.marker { + _ = json.NewEncoder(w).Encode([]map[string]any{{"id": 3, "body": BlockedOnMarker(blockedRepairTestRepo, 7) + "\nblocked on the shared fix"}}) + return + } + _, _ = io.WriteString(w, `[]`) + case r.Method == "GET" && r.URL.Path == "/repos/"+blockedRepairTestRepo+"/pulls/7": + _, _ = io.WriteString(w, `{"number":7,"state":"open"}`) + case r.Method == "GET" && r.URL.Path == issue+"/timeline": + _ = json.NewEncoder(w).Encode([]map[string]any{{ + "event": "labeled", "created_at": "2026-10-29T04:17:25Z", + "label": map[string]any{"name": "blocked"}, "actor": map[string]any{"login": f.actor}, + }}) + case r.Method == "POST" && r.URL.Path == issue+"/labels": + _, _ = io.WriteString(w, `[{"name":"needs-human"}]`) + case r.Method == "DELETE" && r.URL.Path == issue+"/labels/blocked": + f.blocked = false + _, _ = io.WriteString(w, `[]`) + case r.Method == "POST" && r.URL.Path == issue+"/comments": + _, _ = io.WriteString(w, `{"id":9}`) + default: + w.WriteHeader(http.StatusNotFound) + _, _ = fmt.Fprintf(w, `{"path":%q}`, r.URL.Path) + } +} + +// issueLocked is #80 as GitHub returns it: opened by a person, with no +// review opt-in label. +func (f *blockedRepairServer) issueLocked() map[string]any { + is := map[string]any{ + "number": blockedRepairIssue, "state": "open", + "user": map[string]any{"login": blockedRepairPerson}, + "created_at": f.createdAt, + "labels": []any{map[string]any{"name": "blocked"}}, + "body": "Upgrade the client once the upstream API ships.", + } + if f.blockedBy { + is["issue_dependencies_summary"] = map[string]any{"blocked_by": 1, "total_blocked_by": 1} + } + return is +} + +// set changes the issue's state between sweeps. +func (f *blockedRepairServer) set(change func(*blockedRepairServer)) { + f.mu.Lock() + defer f.mu.Unlock() + change(f) +} + +// wrote reports the calls that change the issue. +func (f *blockedRepairServer) wrote() []string { + f.mu.Lock() + defer f.mu.Unlock() + var out []string + for _, c := range f.calls { + if !strings.HasPrefix(c, "GET ") { + out = append(out, c) + } + } + return out +} + +// blockedRepairPolicy is what the gate reads on each decision; a test +// changes it between sweeps (never during one). +type blockedRepairPolicy struct { + level int + paused bool + cfg writeback.Config +} + +// blockedRepairRig is a client over srv with a gate that first saw +// acme/widgets at blockedRepairFirstSeen, and the client's log. +func blockedRepairRig(t *testing.T, f *blockedRepairServer, p *blockedRepairPolicy) (*Client, *writeback.Gate, *bytes.Buffer) { + t.Helper() + srv := httptest.NewServer(f) + t.Cleanup(srv.Close) + var logs bytes.Buffer + c := NewClientForTest(srv.URL, "acme", []string{"widgets"}, slog.New(slog.NewTextHandler(&logs, nil))) + c.appBotLogin = testBot + gate := writeback.New(writeback.Config{}, writeback.Deps{ + LevelFor: func(string) int { return p.level }, + RepoPause: func(string) (writeback.RepoPauseInfo, bool) { + return writeback.RepoPauseInfo{By: "owner", Reason: "freeze"}, p.paused + }, + Config: func() writeback.Config { return p.cfg }, + Now: func() time.Time { return blockedRepairFirstSeen }, + }, "") + gate.ObserveRepo(blockedRepairTestRepo) + c.SetWritebackGate(gate) + // A sweep that reads a blocked-on comment remembers it process-wide. + t.Cleanup(func() { forgetBlockedOnSeen(blockedRepairTestRepo, blockedRepairIssue) }) + return c, gate, &logs +} + +func blockedRepairSweep(t *testing.T, c *Client) BlockedIssueResult { + t.Helper() + res, err := c.SweepBlockedIssues(context.Background(), BlockedSweepOptions{Repos: []string{blockedRepairTestRepo}}) + if err != nil { + t.Fatalf("SweepBlockedIssues: %v", err) + } + if res.Errors != 0 { + t.Fatalf("res = %+v, want no errors", res) + } + return res +} + +// A hive-applied, marker-less `blocked` is repaired on an issue the gate +// would not let gate_comment touch: one older than the hive's write access +// with backfill off, and one without the opt-in label at a level where +// gate_comment needs it. +func TestBlockedRepairConvertsHiveBlockTheGateUsedToHold(t *testing.T) { + cases := []struct { + name string + createdAt string + // gateComment is the reason the same issue is refused as gate_comment. + gateComment string + }{ + {"issue older than the hive's write access, backfill off", blockedRepairOldIssue, writeback.ReasonBackfill}, + {"issue without the opt-in label", blockedRepairNewIssue, writeback.ReasonNeedsLabel}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: tc.createdAt} + c, gate, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6}) + + created, err := time.Parse(time.RFC3339, tc.createdAt) + if err != nil { + t.Fatal(err) + } + item := writeback.Item{Number: blockedRepairIssue, Author: blockedRepairPerson, CreatedAt: created} + if d := gate.Decide(blockedRepairTestRepo, item, writeback.KindGateComment); d.Allowed || d.Reason != tc.gateComment { + t.Fatalf("gate_comment on this issue = %+v, want refused with %s (the hold this test is about)", d, tc.gateComment) + } + + res := blockedRepairSweep(t, c) + if res.AskedHuman != 1 { + t.Fatalf("res = %+v, want the block turned into a needs-human ask\nlog:\n%s", res, logs.String()) + } + joined := strings.Join(f.wrote(), "\n") + want := []string{ + "POST /repos/acme/widgets/issues/80/labels", + "DELETE /repos/acme/widgets/issues/80/labels/blocked", + "POST /repos/acme/widgets/issues/80/comments", + } + if joined != strings.Join(want, "\n") { + t.Fatalf("writes =\n%s\nwant needs-human added, then blocked removed, then the note:\n%s", joined, strings.Join(want, "\n")) + } + if held := gate.Recent(); len(held) != 0 { + t.Fatalf("gate suppressed %+v, want nothing held", held) + } + if strings.Contains(logs.String(), blockedRepairHeldLine) { + t.Fatalf("a repair that went through logged a hold:\n%s", logs.String()) + } + // The issue is no longer blocked, so the next sweep has nothing to do. + if res := blockedRepairSweep(t, c); res.Items != 0 || res.AskedHuman != 0 { + t.Fatalf("second sweep res = %+v, want nothing left", res) + } + }) + } +} + +// A person's marker-less `blocked` on the same old issue is theirs: nothing +// is written and the gate is not even asked. +func TestBlockedRepairLeavesAPersonsBlockAlone(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: blockedRepairPerson, createdAt: blockedRepairOldIssue} + c, gate, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6}) + for pass := 1; pass <= 2; pass++ { + if res := blockedRepairSweep(t, c); res.Items != 1 || res.AskedHuman != 0 || res.Unblocked != 0 { + t.Fatalf("pass %d: res = %+v, want the issue seen and left alone", pass, res) + } + } + if w := f.wrote(); len(w) != 0 { + t.Fatalf("writes on a person's block: %v", w) + } + if held := gate.Recent(); len(held) != 0 { + t.Fatalf("the gate was asked about a person's block: %+v", held) + } + if strings.Contains(logs.String(), blockedRepairHeldLine) { + t.Fatalf("a person's block logged a gate hold:\n%s", logs.String()) + } +} + +// What the fact does not lift still refuses the repair: a paused repo, repo +// mode off, a level below the kind's floor and an unset level. +func TestBlockedRepairStillRefusedByPauseModeAndLevel(t *testing.T) { + off := writeback.Config{PerRepo: map[string]writeback.RepoOverride{blockedRepairTestRepo: {Mode: writeback.RepoModeOff}}} + three := 3 + raised := writeback.Config{Kinds: map[string]writeback.KindPolicyOverride{string(writeback.KindBlockedRepair): {MinLevel: &three}}} + cases := []struct { + name string + policy blockedRepairPolicy + reason string + }{ + {"paused repo", blockedRepairPolicy{level: 6, paused: true}, writeback.ReasonRepoPaused}, + {"repo mode off", blockedRepairPolicy{level: 6, cfg: off}, writeback.ReasonRepoOff}, + {"below the kind's minimum level", blockedRepairPolicy{level: 2, cfg: raised}, writeback.ReasonBelowLevel}, + {"level unset", blockedRepairPolicy{level: 0}, writeback.ReasonLevelUnknown}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + policy := tc.policy + c, gate, logs := blockedRepairRig(t, f, &policy) + if res := blockedRepairSweep(t, c); res.AskedHuman != 0 || res.Unblocked != 0 { + t.Fatalf("res = %+v, want nothing converted while the gate refuses", res) + } + if w := f.wrote(); len(w) != 0 { + t.Fatalf("writes while the gate refuses: %v", w) + } + held := gate.Recent() + if len(held) != 1 || held[0].Kind != writeback.KindBlockedRepair || held[0].Reason != tc.reason || held[0].Number != blockedRepairIssue { + t.Fatalf("gate suppressed %+v, want one blocked_repair refusal with reason %s", held, tc.reason) + } + if out := logs.String(); !strings.Contains(out, blockedRepairHeldLine) || !strings.Contains(out, "issue=80") || !strings.Contains(out, "reason="+tc.reason) { + t.Fatalf("hold not logged with reason %s:\n%s", tc.reason, out) + } + }) + } +} + +// The sweep runs every few minutes. A hold is logged once per issue and +// reason, again when the reason changes, and again when the issue is held +// anew after it stopped being one the repair converts: it was repaired, it +// left the listing, it named a blocker, or a person took the label over. +func TestBlockedRepairHoldLoggedOncePerIssueAndReason(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + p := &blockedRepairPolicy{level: 6, paused: true} + c, _, logs := blockedRepairRig(t, f, p) + off := writeback.Config{PerRepo: map[string]writeback.RepoOverride{blockedRepairTestRepo: {Mode: writeback.RepoModeOff}}} + + sweeps := func(n int) { + t.Helper() + for i := 0; i < n; i++ { + blockedRepairSweep(t, c) + } + } + wantHeld := func(step string, lines int, lastReason string) { + t.Helper() + out := logs.String() + if got := strings.Count(out, blockedRepairHeldLine); got != lines { + t.Fatalf("%s: %d hold line(s), want %d:\n%s", step, got, lines, out) + } + if i := strings.LastIndex(out, blockedRepairHeldLine); i < 0 || !strings.Contains(out[i:], "reason="+lastReason) { + t.Fatalf("%s: newest hold line does not say reason=%s:\n%s", step, lastReason, out) + } + } + + sweeps(3) + wantHeld("same reason on three sweeps", 1, writeback.ReasonRepoPaused) + + p.paused, p.cfg = false, off + sweeps(2) + wantHeld("reason changed to repo mode off", 2, writeback.ReasonRepoOff) + + p.paused = true + sweeps(2) + wantHeld("reason changed back to the pause", 3, writeback.ReasonRepoPaused) + + // The repair goes through; a relay later blocks the issue again and the + // repo is paused again. + p.paused, p.cfg = false, writeback.Config{} + if res := blockedRepairSweep(t, c); res.AskedHuman != 1 { + t.Fatalf("res = %+v, want the repair to go through once the gate allows it\nlog:\n%s", res, logs.String()) + } + wantHeld("repair went through", 3, writeback.ReasonRepoPaused) + f.set(func(f *blockedRepairServer) { f.blocked = true }) + p.paused = true + sweeps(2) + wantHeld("held anew after a repair", 4, writeback.ReasonRepoPaused) + + // The issue leaves the listing (closed, or the label removed) and comes + // back held for the same reason. + f.set(func(f *blockedRepairServer) { f.blocked = false }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.blocked = true }) + sweeps(2) + wantHeld("held anew after leaving the listing", 5, writeback.ReasonRepoPaused) + + // A comment names a blocker, then is gone again. + f.set(func(f *blockedRepairServer) { f.marker = true }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.marker = false }) + sweeps(2) + wantHeld("held anew after naming a blocker", 6, writeback.ReasonRepoPaused) + + // GitHub reports an open "blocked by" link, then it is gone again. + f.set(func(f *blockedRepairServer) { f.blockedBy = true }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.blockedBy = false }) + sweeps(2) + wantHeld("held anew after a native blocked-by link", 7, writeback.ReasonRepoPaused) + + // A person re-applies the label, then the hive's bot does. + f.set(func(f *blockedRepairServer) { f.actor = blockedRepairPerson }) + sweeps(1) + f.set(func(f *blockedRepairServer) { f.actor = testBot }) + sweeps(2) + wantHeld("held anew after a person's label", 8, writeback.ReasonRepoPaused) + + // Only the one repair wrote anything. + if w := f.wrote(); len(w) != 3 { + t.Fatalf("writes = %v, want the single repair's three", w) + } +} + +// A sweep of other repos does not forget this repo's hold, so the next sweep +// of this one does not log it again. +func TestBlockedRepairHoldKeptAcrossSweepOfOtherRepos(t *testing.T) { + f := &blockedRepairServer{blocked: true, actor: testBot, createdAt: blockedRepairOldIssue} + c, _, logs := blockedRepairRig(t, f, &blockedRepairPolicy{level: 6, paused: true}) + blockedRepairSweep(t, c) + // acme/gadgets has no blocked issue; the server answers its listing 404, + // which the sweep reports as an error, so nothing is forgotten. + if _, err := c.SweepBlockedIssues(context.Background(), BlockedSweepOptions{Repos: []string{"acme/gadgets"}}); err == nil { + t.Fatal("sweep of an unlistable repo returned no error") + } + // Nor does a clean listing of that other repo, with no blocked issue. + c.retainBlockedRepairHeld([]string{"acme/gadgets"}, nil) + blockedRepairSweep(t, c) + if got := strings.Count(logs.String(), blockedRepairHeldLine); got != 1 { + t.Fatalf("%d hold line(s) after a sweep of another repo, want 1:\n%s", got, logs.String()) + } + // With no gate installed there is nothing to keep. + c.SetWritebackGate(nil) + c.retainBlockedRepairHeld([]string{blockedRepairTestRepo}, nil) +} diff --git a/src/pkg/writeback/blocked_repair_test.go b/src/pkg/writeback/blocked_repair_test.go new file mode 100644 index 0000000000..a1e3c82c7d --- /dev/null +++ b/src/pkg/writeback/blocked_repair_test.go @@ -0,0 +1,232 @@ +package writeback + +import ( + "testing" + "time" +) + +// The blocked sweep repairs a `blocked` label the hive applied with nothing +// naming what the issue waits on: needs-human on, blocked off, one note. It +// used to ask the gate as gate_comment, which held the repair back on every +// issue older than the hive's write access (backfill off) and on every issue +// without the opt-in label, although the hive had already labelled each one. +// blocked_repair is the kind for it: the caller states the label is the +// hive's (Item.HiveAppliedBlock) and those two checks stop applying. + +var ( + // oldIssue predates the gate's first sight of org/repo (t0), the + // backfill guard's case; newIssue is newer and carries no opt-in label. + oldIssue = Item{Number: 40, Author: "maintainer", CreatedAt: t0.Add(-48 * time.Hour)} + newIssue = Item{Number: 41, Author: "maintainer", CreatedAt: t0.Add(time.Hour)} + // optedIssue carries the opt-in label. + optedIssue = Item{Number: 42, Author: "maintainer", Labels: []string{"hive-review"}, CreatedAt: t0.Add(time.Hour)} +) + +// hiveBlocked is it with the caller's finding that the hive's own bot +// applied its `blocked` label. +func hiveBlocked(it Item) Item { + it.HiveAppliedBlock = true + return it +} + +func TestBlockedRepairKindDefaults(t *testing.T) { + if !IsKind("blocked_repair") { + t.Fatal("blocked_repair is not a known kind") + } + var c Config + if p := c.Policy(KindBlockedRepair); p.MinLevel != 1 || !p.RequireLabel || p.UnlabeledMinLevel != 0 || p.OwnOnly { + t.Fatalf("blocked_repair default = %+v, want L1 with the opt-in label for a call that does not state the hive applied the block", p) + } + if KindBlockedRepair.IsLabelKind() || KindBlockedRepair.IsIssueFileKind() { + t.Fatal("blocked_repair posts a note on an existing issue: neither a label kind (labels-only refuses it) nor an issue-file kind") + } + if !KindBlockedRepair.ContentDeduped() || KindBlockedRepair.RateCapExempt() { + t.Fatal("blocked_repair keeps the content dedupe, burst hold and hourly cap its note had under gate_comment") + } + listed := false + for _, row := range c.EffectiveKinds() { + listed = listed || row.Kind == KindBlockedRepair + } + if !listed { + t.Fatal("blocked_repair missing from EffectiveKinds (the Settings kind table)") + } + if err := (Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}).Validate(); err != nil { + t.Fatalf("writeback.kinds.blocked_repair override refused: %v", err) + } +} + +// With the fact stated, neither the backfill guard nor the opt-in label +// holds the repair back. Without it the kind is gated like gate_comment for +// the same item, and the fact does nothing for any other kind. +func TestBlockedRepairSkipsBackfillAndOptInOnlyForAHiveAppliedBlock(t *testing.T) { + cases := []struct { + name string + level int + item Item + kind Kind + allow bool + reason string + }{ + {"old issue, repair, hive applied the block", 6, hiveBlocked(oldIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"old issue, repair, fact not stated", 6, oldIssue, KindBlockedRepair, false, ReasonBackfill}, + {"old issue, gate_comment", 6, oldIssue, KindGateComment, false, ReasonBackfill}, + {"old issue, gate_comment ignores the fact", 6, hiveBlocked(oldIssue), KindGateComment, false, ReasonBackfill}, + + {"unlabelled issue, repair, hive applied the block", 6, hiveBlocked(newIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"unlabelled issue, repair, fact not stated", 6, newIssue, KindBlockedRepair, false, ReasonNeedsLabel}, + {"unlabelled issue, gate_comment", 6, newIssue, KindGateComment, false, ReasonNeedsLabel}, + {"unlabelled issue, gate_comment ignores the fact", 6, hiveBlocked(newIssue), KindGateComment, false, ReasonNeedsLabel}, + {"unlabelled issue, stage_label ignores the fact", 6, hiveBlocked(newIssue), KindStageLabel, false, ReasonNeedsLabel}, + + // The label is applied at any level, so the repair's floor is L1. + {"lowest level, hive applied the block", 1, hiveBlocked(oldIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"opted-in issue, hive applied the block", 1, hiveBlocked(optedIssue), KindBlockedRepair, true, ReasonHiveAppliedBlock}, + {"opted-in issue, fact not stated", 1, optedIssue, KindBlockedRepair, true, ReasonLabelOptIn}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + g, _ := newGate(t, Config{}, tc.level) + d := g.Allowed("org/repo", tc.item, tc.kind) + if d.Allowed != tc.allow || d.Reason != tc.reason { + t.Fatalf("got %+v, want allow=%v reason=%s", d, tc.allow, tc.reason) + } + }) + } +} + +// The fact lifts two checks and no others: a pause, the repo mode, project +// membership, own_only and the kind's minimum level refuse a hive-applied +// block's repair as they refuse any other write. +func TestBlockedRepairStillHeldByPauseModeProjectOwnOnlyAndLevel(t *testing.T) { + item := hiveBlocked(oldIssue) + mode := func(m RepoMode) Config { + return Config{PerRepo: map[string]RepoOverride{"org/repo": {Mode: m}}} + } + cases := []struct { + name string + cfg Config + level int + paused bool + listed bool + allow bool + reason string + }{ + {"paused repo", Config{}, 6, true, true, false, ReasonRepoPaused}, + {"repo mode off", mode(RepoModeOff), 6, false, true, false, ReasonRepoOff}, + {"repo mode issues-only", mode(RepoModeIssuesOnly), 6, false, true, false, ReasonRepoIssues}, + {"repo mode labels-only: the repair posts a note", mode(RepoModeLabelsOnly), 6, false, true, false, ReasonRepoLabels}, + {"repo mode full", mode(RepoModeFull), 6, false, true, true, ReasonRepoFull}, + {"repo not in the project", Config{}, 6, false, false, false, ReasonRepoNotListed}, + {"own_only set by an operator", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {OwnOnly: boolp(true)}}}, 6, false, true, false, ReasonOwnOnly}, + {"level unset", Config{}, 0, false, true, false, ReasonLevelUnknown}, + {"below a raised kind floor", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}, 2, false, true, false, ReasonBelowLevel}, + {"at a raised kind floor", Config{Kinds: map[string]KindPolicyOverride{"blocked_repair": {MinLevel: intp(3)}}}, 3, false, true, true, ReasonHiveAppliedBlock}, + {"below the global floor", Config{Unsolicited: UnsolicitedConfig{MinLevel: intp(5)}}, 4, false, true, false, ReasonBelowLevel}, + {"require_labels off changes nothing", Config{Unsolicited: UnsolicitedConfig{RequireLabels: boolp(false)}}, 1, false, true, true, ReasonHiveAppliedBlock}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + g := New(tc.cfg, Deps{ + LevelFor: func(string) int { return tc.level }, + RepoListed: func(string) bool { return tc.listed }, + RepoPause: func(string) (RepoPauseInfo, bool) { return RepoPauseInfo{By: "owner"}, tc.paused }, + Now: func() time.Time { return t0 }, + }, "") + g.ObserveRepo("org/repo") + d := g.Allowed("org/repo", item, KindBlockedRepair) + if d.Allowed != tc.allow || d.Reason != tc.reason { + t.Fatalf("got %+v, want allow=%v reason=%s", d, tc.allow, tc.reason) + } + }) + } +} + +// An allowed repair of an old issue is not one of the "pre-existing items +// left untouched" the backfill banner counts; a refused one still is. +func TestBlockedRepairDoesNotCountAsBackfillSkip(t *testing.T) { + g, _ := newGate(t, Config{}, 6) + if d := g.Allowed("org/repo", hiveBlocked(oldIssue), KindBlockedRepair); !d.Allowed { + t.Fatalf("hive-applied block on an old issue = %+v, want allowed", d) + } + if n := g.Snapshot().Repos["org/repo"].PreExisting; n != 0 { + t.Fatalf("pre-existing untouched = %d after an allowed repair, want 0", n) + } + if d := g.Allowed("org/repo", oldIssue, KindBlockedRepair); d.Allowed || d.Reason != ReasonBackfill { + t.Fatalf("fact not stated = %+v, want refused by backfill", d) + } + if n := g.Snapshot().Repos["org/repo"].PreExisting; n != 1 { + t.Fatalf("pre-existing untouched = %d after a backfill refusal, want 1", n) + } +} + +// The note is one constant body, so the content layer treats it as it did +// under gate_comment: the same issue twice is a duplicate, a second issue in +// the same repo within the burst window is held, and both pass later. +func TestBlockedRepairKeepsDuplicateAndBurstHolds(t *testing.T) { + const note = "\nNeeds a maintainer: the `blocked` label named nothing it waits on." + g, clk, _ := newDedupeGate(t, Config{}, "") + first, second := hiveBlocked(oldIssue), hiveBlocked(newIssue) + if d := g.AllowedPreview("org/repo", first, KindBlockedRepair, note); !d.Allowed || d.Reason != ReasonHiveAppliedBlock { + t.Fatalf("first issue = %+v, want allowed", d) + } + if d := g.AllowedPreview("org/repo", first, KindBlockedRepair, note); d.Allowed || d.Reason != ReasonDuplicate { + t.Fatalf("same issue again = %+v, want held as a duplicate", d) + } + if d := g.AllowedPreview("org/repo", second, KindBlockedRepair, note); d.Allowed || d.Reason != ReasonBurst { + t.Fatalf("second issue in the burst window = %+v, want held as a burst", d) + } + clk.advance(2 * time.Minute) + if d := g.AllowedPreview("org/repo", second, KindBlockedRepair, note); !d.Allowed { + t.Fatalf("second issue after the window = %+v, want allowed", d) + } +} + +// NoteHeld is news once per item and reason; ForgetHeld and RetainHeld make +// a later hold news again. Kinds and repos are remembered apart. +func TestHeldMemory(t *testing.T) { + g, _ := newGate(t, Config{}, 6) + const kind = KindBlockedRepair + if !g.NoteHeld("Org/Repo", 80, kind, ReasonRepoPaused) { + t.Fatal("a first hold must be news") + } + if g.NoteHeld("org/repo", 80, kind, ReasonRepoPaused) { + t.Fatal("the same hold again must not be news, whatever the repo's case") + } + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a changed reason must be news") + } + if g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("the changed reason again must not be news") + } + if !g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) || !g.NoteHeld("org/other", 80, kind, ReasonRepoOff) || !g.NoteHeld("org/repo", 80, KindGateComment, ReasonRepoOff) { + t.Fatal("another issue, repo or kind is its own hold") + } + + g.ForgetHeld("ORG/repo", 80, kind) + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a hold after ForgetHeld must be news again") + } + + // Only org/repo#81 is still a target of this kind. + g.RetainHeld(kind, func(repo string, number int) bool { return repo == "org/repo" && number == 81 }) + if g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) { + t.Fatal("RetainHeld forgot an item it was told to keep") + } + if !g.NoteHeld("org/repo", 80, kind, ReasonRepoOff) || !g.NoteHeld("org/other", 80, kind, ReasonRepoOff) { + t.Fatal("RetainHeld kept an item that is no longer a target") + } + if g.NoteHeld("org/repo", 80, KindGateComment, ReasonRepoOff) { + t.Fatal("RetainHeld for one kind forgot another kind's hold") + } + g.RetainHeld(kind, nil) + if g.NoteHeld("org/repo", 81, kind, ReasonRepoOff) { + t.Fatal("RetainHeld with no keep function must forget nothing") + } + + var none *Gate + if !none.NoteHeld("org/repo", 80, kind, ReasonRepoOff) { + t.Fatal("a nil gate remembers nothing: every hold is news") + } + none.ForgetHeld("org/repo", 80, kind) + none.RetainHeld(kind, func(string, int) bool { return false }) +} diff --git a/src/pkg/writeback/dedupe.go b/src/pkg/writeback/dedupe.go index 1a2651039e..8f5ca6b9a6 100644 --- a/src/pkg/writeback/dedupe.go +++ b/src/pkg/writeback/dedupe.go @@ -84,6 +84,10 @@ var ContentDedupeKinds = map[Kind]bool{ KindClaimComment: true, KindGateComment: true, KindWelcomeComment: true, + // The repair's note is one constant body: the burst hold is what keeps + // a repo with many marker-less blocks from getting them all in one + // sweep, as it did while the note went out under gate_comment. + KindBlockedRepair: true, } // RateCapExemptKinds are never held by the hourly cap: everything on the diff --git a/src/pkg/writeback/gate.go b/src/pkg/writeback/gate.go index c283f9dd17..3c266540e0 100644 --- a/src/pkg/writeback/gate.go +++ b/src/pkg/writeback/gate.go @@ -39,6 +39,13 @@ type Item struct { // caller whose policy closes superseded originals on adopt // (governor.labels.stage.adoption.close_superseded: on_adopt). SupersededBy int + // HiveAppliedBlock is true when the caller has established that the + // item's `blocked` label was most recently applied by the hive's own + // bot (the actor of the newest labeled event for it). Set only by the + // blocked sweep's repair of a block that names nothing it waits on; + // with KindBlockedRepair it lifts the backfill guard and the opt-in + // label requirement (ReasonHiveAppliedBlock). No other kind reads it. + HiveAppliedBlock bool // Labels is the item's current label set. Labels []string // CreatedAt is the item's creation time; the backfill guard compares it @@ -84,6 +91,14 @@ const ( // sign-off, so the close is the hive's own housekeeping, not an // unsolicited write on a person's work. ReasonSupersededAgentPR = "superseded-agent-pr" + // ReasonHiveAppliedBlock allows the blocked-repair kind on an item whose + // `blocked` label the hive itself applied. The hive has already written + // to that item, whatever its age and with no opt-in label, so replacing + // its own label with a needs-human ask is upkeep of the hive's state, + // not a first unsolicited write: the backfill guard and the opt-in + // label do not apply. A pause, the repo mode, project membership, + // own_only and the kind's minimum level still do. + ReasonHiveAppliedBlock = "hive-applied-block" ) // RepoPauseInfo is the provenance of an operator pause, as config records @@ -194,7 +209,10 @@ type Gate struct { recent []Suppressed summary map[string]bool // dedupe is the content layer's table (dedupe.go). - dedupe *dedupeState + dedupe *dedupeState + // held is the last refusal reason reported per item for a writer that + // asks again on every sweep (held.go). + held map[heldKey]string attentionOnce sync.Once attentionStore *attention.Store attentionErr error @@ -460,7 +478,11 @@ func (g *Gate) decideLocked(repo string, item Item, kind Kind) Decision { // carrying the opt-in label was asked for by a human, so the backfill // guard (which exists for items nobody asked about) does not apply. optedIn := g.optedInLocked(repo, item) - if !optedIn { + // Repairing a `blocked` the hive applied acts on an item the hive has + // already written to, so neither the backfill guard here nor the opt-in + // label below applies to it. The level still does. + hiveBlock := kind == KindBlockedRepair && item.HiveAppliedBlock + if !optedIn && !hiveBlock { if backfilled, cut := g.backfillBlockedLocked(repo, item); backfilled { return Decision{Reason: ReasonBackfill, Detail: fmt.Sprintf("#%d was created %s, before hive could write to %s (%s); writeback.unsolicited.backfill is off", item.Number, item.CreatedAt.UTC().Format(time.RFC3339), repo, cut.Format(time.RFC3339))} } @@ -489,6 +511,12 @@ func (g *Gate) decideLocked(repo string, item Item, kind Kind) Decision { if level <= 0 { return Decision{Reason: ReasonLevelUnknown, Detail: "acmm_level is unset; unsolicited writes need a level"} } + if hiveBlock { + if level >= p.MinLevel { + return Decision{Allowed: true, Reason: ReasonHiveAppliedBlock, Detail: fmt.Sprintf("the hive applied the `blocked` label on #%d itself and L%d ≥ L%d", item.Number, level, p.MinLevel)} + } + return Decision{Reason: ReasonBelowLevel, Detail: fmt.Sprintf("%s needs L%d; %s is at L%d", kind, p.MinLevel, repo, level)} + } if !p.RequireLabel { if level >= p.MinLevel { return Decision{Allowed: true, Reason: ReasonLevel, Detail: fmt.Sprintf("L%d ≥ L%d", level, p.MinLevel)} diff --git a/src/pkg/writeback/held.go b/src/pkg/writeback/held.go new file mode 100644 index 0000000000..b58ee2e9fd --- /dev/null +++ b/src/pkg/writeback/held.go @@ -0,0 +1,67 @@ +package writeback + +// A writer that asks the gate again on every sweep for the same item (the +// blocked sweep's repair, every few minutes) would otherwise report the same +// refusal each time. The gate keeps the reason it last refused each such item +// for, so the writer can say so once and again only when the reason changes. +// It lives here rather than with the writer because the gate is the one +// object a process shares across GitHub clients and rebuilds of them. It is +// in memory only: a restart reports each hold once more. + +// heldKey names one item a kind is being held on. +type heldKey struct { + repo string + number int + kind Kind +} + +// NoteHeld records that kind on repo#number was refused for reason and +// reports whether that is news: the item's first hold, or a reason other +// than the one last noted. A nil gate refuses nothing and has nothing to +// remember, so every call on it is news. +func (g *Gate) NoteHeld(repo string, number int, kind Kind, reason string) bool { + if g == nil { + return true + } + g.mu.Lock() + defer g.mu.Unlock() + key := heldKey{repo: repoKey(repo), number: number, kind: kind} + if last, ok := g.held[key]; ok && last == reason { + return false + } + if g.held == nil { + g.held = map[heldKey]string{} + } + g.held[key] = reason + return true +} + +// ForgetHeld drops what NoteHeld kept for kind on repo#number, because the +// write went through or the item is no longer one the writer would write to. +// A later hold of the item is news again. +func (g *Gate) ForgetHeld(repo string, number int, kind Kind) { + if g == nil { + return + } + g.mu.Lock() + defer g.mu.Unlock() + delete(g.held, heldKey{repo: repoKey(repo), number: number, kind: kind}) +} + +// RetainHeld forgets every hold of kind on an item keep does not report, +// which bounds the memory by the writer's current targets: an item that was +// closed, or whose label a person removed, never reaches ForgetHeld. keep is +// given the repo lower-cased and trimmed; it runs under the gate's lock and +// must not call the gate. +func (g *Gate) RetainHeld(kind Kind, keep func(repo string, number int) bool) { + if g == nil || keep == nil { + return + } + g.mu.Lock() + defer g.mu.Unlock() + for key := range g.held { + if key.kind == kind && !keep(key.repo, key.number) { + delete(g.held, key) + } + } +} diff --git a/src/pkg/writeback/kinds.go b/src/pkg/writeback/kinds.go index ba08f092ae..9d3c795a07 100644 --- a/src/pkg/writeback/kinds.go +++ b/src/pkg/writeback/kinds.go @@ -105,6 +105,14 @@ const ( // the file to another PR, a contributor dropped the change). A hold a // person set, or one with another hold source, is never removed. KindHumanMergePathRelease Kind = "human_merge_path_release" + // KindBlockedRepair turns a `blocked` label the hive itself applied, with + // nothing naming what the issue waits on, into a needs-human ask: it + // adds `needs-human`, removes `blocked` and posts one note saying what + // was missing and the two ways out. The caller states that the label is + // the hive's on the item (Item.HiveAppliedBlock); the gate then skips the + // backfill guard and the opt-in label for it (ReasonHiveAppliedBlock). + // A person's `blocked` is never repaired and never reaches the gate. + KindBlockedRepair Kind = "blocked_repair" ) // KindPolicy is the per-kind rule. A write of this kind on an item the hive @@ -167,6 +175,15 @@ func DefaultKinds() map[Kind]KindPolicy { // changes a human-merge path: wherever the hive could say so, it can // take the hold back when that stops being true. KindHumanMergePathRelease: {MinLevel: 5, RequireLabel: true}, + // The label this kind repairs is put on by a contributor relay with + // a hub-minted task token: a write no ACMM level and no write-back + // kind gates, so a marker-less hive `blocked` can sit on an issue at + // any level. The floor is therefore the lowest level there is: + // wherever the hive's label could land, its repair can follow, and + // an unset level still refuses. RequireLabel governs only a call + // that does not state Item.HiveAppliedBlock (none does today), so a + // caller that forgets the fact is held like any other gated kind. + KindBlockedRepair: {MinLevel: 1, RequireLabel: true}, } }