From 24a57d5a5c4036a29babe679190c08116fceecd4 Mon Sep 17 00:00:00 2001 From: repires Date: Wed, 23 Sep 2026 17:47:14 +0000 Subject: [PATCH] docs(contributing): add hold-gate PR queue prioritization rubric MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Planning Artifact Adds docs/contributing/hold-gate-rubric.md (new file) — the concrete proposal for the hold-gate PR queue prioritization rubric identified in #1043, incorporating the release-gate expedite policy consolidated from #1082, and scheduled in Phase 0 of the org-wide roadmap (#1073). Contents: - Three-tier prioritization rubric (P0: security/supply-chain/core integrity, P1: release-gate/defects/SSOT blockers, P2: tests/docs/routine refactors) - Release-gate expedite lane (<48h turnaround for approved release/ready PRs per common#1082 consolidation) - Streamlined acceptance criteria for test coverage PRs (~40% of queue backlog) - Factory seven-label contract compliance (tiers as review metadata/sorting signals, preserving canonical labels in docs/skills/label-workflow.md) - Preflight obsolescence check (#1054) to eliminate dead PRs - Operating metrics aligned with Phase 0 targets (<40 PR queue depth) Cross-references added in CONTRIBUTING.md, docs/contributing/reviewer-ladder.md, docs/skills/pr-review/SKILL.md, and docs/skills/pr-review/references/triage-operations.md. Refs #1043 Refs #1082 Refs #1029 Refs #1073 Hive-Run: projectbluefin/common#1043 Hive-Plan: phase-0/prioritization-rubric Hive-Spec: acmm-hold-gate#prioritization-rubric Assisted-by: Gemini 3.8 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: repires --- CONTRIBUTING.md | 3 + docs/contributing/hold-gate-rubric.md | 155 ++++++++++++++++++ docs/contributing/reviewer-ladder.md | 2 +- docs/skills/pr-review/SKILL.md | 4 +- .../pr-review/references/triage-operations.md | 2 + 5 files changed, 164 insertions(+), 2 deletions(-) create mode 100644 docs/contributing/hold-gate-rubric.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3f4a6529..0574036c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -61,3 +61,6 @@ Full layer validation (the `common` behave suite from - [`docs/contributing/reviewer-ladder.md`](docs/contributing/reviewer-ladder.md) — draft proposal for a four-rung contributor ladder (Triager, Domain Reviewer); unadopted until a maintainer decision. +- [`docs/contributing/hold-gate-rubric.md`](docs/contributing/hold-gate-rubric.md) — + draft proposal for hold-gate PR queue prioritization rubric (P0/P1/P2 risk tiers + and release-gate expedite lane); unadopted until a maintainer decision. diff --git a/docs/contributing/hold-gate-rubric.md b/docs/contributing/hold-gate-rubric.md new file mode 100644 index 00000000..7d5abadf --- /dev/null +++ b/docs/contributing/hold-gate-rubric.md @@ -0,0 +1,155 @@ +# Hold-Gate PR Prioritization Rubric — Contribution Strategy (Planning Draft) + +> **Status**: hold-gated planning artifact. Human review required before adoption. +> Filed by the strategist agent. Tracks [common#1043](https://github.com/projectbluefin/common/issues/1043); +> Incorporates the release-gate expedite policy consolidated from [common#1082](https://github.com/projectbluefin/common/issues/1082); +> Aligned with Phase 0 of the org-wide roadmap ([common#1073](https://github.com/projectbluefin/common/pull/1073)). +> +> Nothing below changes GitHub roles, branch protection, CODEOWNERS, workflow behavior, +> or hold-gate enforcement until a maintainer adopts it. + +## Problem + +Human review capacity is the primary binding constraint across the factory. Autonomous agent lanes continuously produce findings and PRs, but review capacity drains the queue at a slower rate, causing hold-gated pull requests to accumulate and compete flatly for human attention. + +Observed queue dynamics from multi-cycle snapshots (2026-08-27 through 2026-09-05): + +- **Rapid accumulation**: Open hold-gated PRs grew from 33 (2026-08-27) → 61 (2026-08-29) → 74 (2026-08-29 kick) → peak of 95 (2026-08-31), with a median age of 10 days and the oldest standing at 69 days (`dakota#962`). +- **Flat competition**: Critical security fixes (e.g. `server#37` k3s kubeconfig 0600 permissions, `dakota-iso#142` cosign verification, `server#36` OTA signatures) sit unordered beside non-urgent test additions (e.g. `fsdk-containers#209`) and formatting tweaks. +- **Release-gate starvation ([common#1082](https://github.com/projectbluefin/common/issues/1082))**: Capacity freed during queue contractions has historically flowed to easy test/refactor PRs rather than release-blocking changes. Approved release promotions (e.g. `bluefin#1115` `ci(promote)`) carrying `release/ready` sat unmerged for 17+ days, contributing to downstream freeze states (Bluefin stable frozen 47 days, Bluefin-LTS 29 days, Common missing monthly release tag `v2026.09`). +- **Test coverage volume (~40% of queue)**: Queue analysis revealed that ~40% of open hold-gated PRs consist of BATS or unit test coverage. Evaluating each test PR as a bespoke high-friction decision stalls the queue; establishing a clear, standardized acceptance bar converts dozens of individual decisions into rapid standard approvals. +- **Silent obsolescence ([common#1054](https://github.com/projectbluefin/common/issues/1054))**: Without queue triage, hold-gated agent PRs sit for weeks and get silently superseded by human fixes merged directly to `main` (e.g. `common#1010` vs `common#1020`/`#1030`), wasting reviewer time on dead work. +- **Multi-repo diffusion**: The hold-gated queue spans 12 repositories and 4 product lines. No individual repository maintainer sees the full shape, so ordering cannot emerge ad-hoc per repo. + +Complementary to the reviewer-scaling ladder ([common#1029](https://github.com/projectbluefin/common/issues/1029), [reviewer-ladder.md](reviewer-ladder.md)), which expands reviewer supply, this rubric addresses **review ordering**: ensuring that existing human review capacity is directed to the highest-risk and highest-impact pull requests first. + +## Proposal: Three-Tier Prioritization Rubric + +Hold-gated PRs are ordered into three priority tiers based on risk, blast radius, and unblocking impact: + +``` +┌────────────────────────────────────────────────────────────────────────┐ +│ Tier P0 — Security, Supply Chain & Core Integrity │ +│ • Permissions, secrets, polkit hardening, CVE fixes │ +│ • Cosign / OTA signature verification, SHA-pinning regressions │ +│ • Org-wide blast radius: system_files/shared/**, root Containerfile │ +├────────────────────────────────────────────────────────────────────────┤ +│ Tier P1 — Release Gates, Defect Fixes & Architectural Blockers │ +│ • Release-Gate Expedite Lane: approved release/ready PRs (<48h) │ +│ • User-facing defects, boot/hardware regressions, broken scanners │ +│ • SSOT / architectural refactors unblocking dependent clusters │ +├────────────────────────────────────────────────────────────────────────┤ +│ Tier P2 — Tests, Documentation & Non-Breaking Refactors │ +│ • BATS and pytest coverage additions (~40% of queue volume) │ +│ • Documentation and skill updates (leverage doc-only push exception) │ +│ • Routine refactors, formatting, and lint cleanliness │ +└────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +### Tier P0: Security, Supply Chain & Core Integrity (Critical) + +**Definition**: Vulnerabilities, security policy regressions, cryptographic verification, or defects with org-wide blast radius. + +- **Examples**: + - Permissions fixes on sensitive configs (`k3s` kubeconfig 0600, polkit rule hardening, sudo policy). + - Supply-chain verification: cosign image signature verification, OTA signature checks, immutable SHA pinning in CI/Containerfiles. + - Critical fixes in `system_files/shared/` or base image definitions that affect all variants (`bluefin`, `bluefin-lts`, `dakota`). +- **Review Precedence**: **Immediate**. P0 items take absolute priority in review sweeps. +- **Target SLA**: First maintainer review within **24–48 hours**. + +--- + +### Tier P1: Release Gates, Defect Fixes & Architectural Blockers (High) + +**Definition**: PRs that directly unblock releases, resolve active defects, or clear architectural dependencies blocking multiple downstream PRs. + +- **Release-Gate Expedite Lane (consolidated from [common#1082](https://github.com/projectbluefin/common/issues/1082))**: + - Any PR carrying the `release/ready` label or automated promotion PR (`ci(promote)`) that has received approval **jumps the hold-gate queue**. + - Reviewers and maintainers must merge or explicitly disposition expedited PRs within **48 hours** to prevent downstream distribution freezes. +- **User-Facing Defects & Regressions**: + - Scanner fixes for crash loops, broken user tooling (`ujust`, `bonedigger-report`), and hardware quirks (audio/video device matching, GPU initialization). +- **Single Source of Truth (SSOT) Blockers**: + - Architectural PRs (e.g. `common#1045`, `bluefin#1148`) that establish canonical definitions and unblock clusters of waiting PRs. +- **Review Precedence**: Reviewed immediately after P0 queue is clear. +- **Target SLA**: Maintainer review within **3–5 days** (Expedite Lane: **48 hours**). + +--- + +### Tier P2: Tests, Documentation & Non-Breaking Refactors (Standard / Batch) + +**Definition**: Changes that add test coverage, enhance documentation, or clean up code without altering core functional behavior. + +- **Scope**: + - BATS and unit test coverage (which represents ~40% of the backlog). + - Documentation and skill updates (note: pure doc changes to `docs/**` or `AGENTS.md` should use the doc-only push exception where applicable, avoiding PR review overhead). + - Code hygiene, variable cleanup, and non-blocking refactors. +- **Standardized Acceptance Bar for Test PRs**: + - To clear the ~40% test backlog efficiently without imposing bespoke review friction, test PRs that: + 1. Pass all automated CI checks (`Validate PR`, `Build`, `Unit Tests`, `PR E2E`), + 2. Contain only additions under `tests/**` following standard BATS/pytest conventions, and + 3. Do not modify production scripts or system files, + - are reviewed via batch sweeps using the "easy-wins mode" (`additions + deletions` ascending) defined in [`docs/skills/pr-review/SKILL.md`](../skills/pr-review/SKILL.md). +- **Review Precedence**: Processed in scheduled batch sweeps once P0 and P1 queues are drained. +- **Target SLA**: Processed during weekly backlog sweeps. + +--- + +## Queue Mechanics and Factory Integration + +### Compliance with the Seven-Label Contract + +The Project Bluefin factory operates strictly on seven canonical labels ([`docs/skills/label-workflow.md`](../skills/label-workflow.md)): +`1-triage`, `2-discussing`, `3-human-queue`, `3-clanker-queue`, `4-review`, `blocked`, and `hold`. + +1. **No New Workflow State Labels**: Priority tiers (`P0`, `P1`, `P2`) are **review metadata and sorting signals**, NOT workflow states. They do not replace or alter `4-review` or the `hold` overlay. +2. **Metadata Signaling**: + - Agent lanes and contributors indicate priority tier in the PR description under a dedicated `## Priority: P0 | P1 | P2` heading, accompanied by evidence. + - Commit messages and trailers may include `Hive-Priority: P0 | P1 | P2`. + - If repository automation surfaces sorting tags (such as `hold-p0`, `hold-p1`, `hold-p2`), these act exclusively as informational reviewer filters, never as workflow state machines. + +### Preflight Obsolescence Check ([common#1054](https://github.com/projectbluefin/common/issues/1054)) + +Before reviewing an aged hold-gated PR: + +1. **Check against `main`**: Inspect `git log` on `main` for the PR's claimed files and functions since the PR creation date. +2. **Detect collisions**: If a merged PR (human or agent) already landed equivalent functionality (e.g. `common#1010` superseded by `#1020` and `#1030`), close the stale PR with a comment pointing to the merged commit. +3. **Prevent queue inflation**: Do not burn reviewer cycles reviewing superseded diffs. + +--- + +## Operating Metrics and Targets + +In coordination with Phase 0 of the org-wide roadmap ([common#1073](https://github.com/projectbluefin/common/pull/1073)): + +| Metric | Measured Baseline (Sept 2026) | Target | +|---|---|---| +| Open hold-gated PR inventory | 88–95 PRs | < 40 and declining | +| Median hold-gated PR age | 10 days (oldest 69d) | < 7 days | +| Tier P0 review latency | Unordered (up to 20d) | < 48 hours | +| Release-gate expedite turnaround | 17 days (`bluefin#1115`) | < 48 hours from approval | +| Stale / superseded PR latency | Open indefinitely | Closed within 7 days of collision | + +--- + +## What this document deliberately does not do + +- **Does not modify branch protection or CODEOWNERS**: Maintainers retain exclusive merge authority. +- **Does not bypass human review**: Hold-gated PRs still require human approval before merge. +- **Does not invent custom workflow states**: Preserves the 7-label contract of `label-workflow.md`. +- **Does not alter automated merge mechanics**: Auto-merge rules and merge queue behavior remain under `hive-automerge.md` and repository rulesets. + +--- + +## Related + +- [common#1043](https://github.com/projectbluefin/common/issues/1043) — Hold-gated PR queue prioritization rubric (this document's tracker) +- [common#1082](https://github.com/projectbluefin/common/issues/1082) — Release-gate expedite lane (consolidated into #1043) +- [common#1029](https://github.com/projectbluefin/common/issues/1029) / [common#1080](https://github.com/projectbluefin/common/pull/1080) — Reviewer-scaling ladder proposal ([reviewer-ladder.md](reviewer-ladder.md)) +- [common#1028](https://github.com/projectbluefin/common/issues/1028) — Release-gate posture across product lines +- [common#1054](https://github.com/projectbluefin/common/issues/1054) — Stale hold-gated PR obsolescence detection +- [common#1058](https://github.com/projectbluefin/common/issues/1058) — Reviewer coverage map / repo concentration data +- [common#1073](https://github.com/projectbluefin/common/pull/1073) — Org-wide roadmap (Phase 0 parent) +- [docs/skills/pr-review/SKILL.md](../skills/pr-review/SKILL.md) — Backlog review procedure +- [docs/skills/label-workflow.md](../skills/label-workflow.md) — Factory seven-label contract diff --git a/docs/contributing/reviewer-ladder.md b/docs/contributing/reviewer-ladder.md index 40d48528..7b59744c 100644 --- a/docs/contributing/reviewer-ladder.md +++ b/docs/contributing/reviewer-ladder.md @@ -104,5 +104,5 @@ draft operating measures: - [common#1073](https://github.com/projectbluefin/common/pull/1073) — org-wide ROADMAP.md (Phase 0 parent) - [common#1058](https://github.com/projectbluefin/common/issues/1058) — reviewer coverage map / repo-concentration data - [common#1067](https://github.com/projectbluefin/common/issues/1067) — user-issue triage starvation -- [common#1043](https://github.com/projectbluefin/common/issues/1043) — hold-gate prioritization rubric +- [common#1043](https://github.com/projectbluefin/common/issues/1043) — hold-gate prioritization rubric ([hold-gate-rubric.md](hold-gate-rubric.md)) - [common#1078](https://github.com/projectbluefin/common/issues/1078) — release cadence breakdown (downstream symptom) diff --git a/docs/skills/pr-review/SKILL.md b/docs/skills/pr-review/SKILL.md index 5ffa7392..85ca6db5 100644 --- a/docs/skills/pr-review/SKILL.md +++ b/docs/skills/pr-review/SKILL.md @@ -67,7 +67,7 @@ For "let's review PRs" the agent assembles the list from three sources, i Default to **streaming**: present one card, take the verdict, execute it immediately, then present the next. The human stays engaged because every answer produces a visible result before the next question arrives. Batching verdicts is the fallback for non-interactive runs only. -**Easy-wins mode.** Sort ascending by `additions + deletions` and present small ones first. Park anything complex in `3-human-queue` with a findings comment. +**Easy-wins mode.** Sort ascending by `additions + deletions` and present small ones first. Park anything complex in `3-human-queue` with a findings comment. For multi-tier hold-gate queues, consult the draft [hold-gate prioritization rubric](../../contributing/hold-gate-rubric.md) to order reviews by risk tier (P0 security/integrity → P1 release-gate/defects → P2 tests/docs). ### 1 — Dossier (one-call fetch) @@ -185,3 +185,5 @@ queue state reading, branch update, and fork PR rebase. - [shell-scripts/SKILL.md](../shell-scripts/SKILL.md) — shell review patterns and bats testing - [ci-tooling/SKILL.md](../ci-tooling/SKILL.md) — CI workflow review and SHA pinning - [lab-testing/SKILL.md](../lab-testing/SKILL.md) — lab verification +- [../../contributing/hold-gate-rubric.md](../../contributing/hold-gate-rubric.md) — draft hold-gate PR queue prioritization rubric and expedite policy +- [../../contributing/reviewer-ladder.md](../../contributing/reviewer-ladder.md) — draft four-rung contributor ladder proposal diff --git a/docs/skills/pr-review/references/triage-operations.md b/docs/skills/pr-review/references/triage-operations.md index 27a3a151..fac295c5 100644 --- a/docs/skills/pr-review/references/triage-operations.md +++ b/docs/skills/pr-review/references/triage-operations.md @@ -29,3 +29,5 @@ Same dossier → verdict → stage → land loop, with issue verdicts: | `Containerfile` | ALL variants | No | | `docs/**`, `AGENTS.md` | Documentation only | N/A (doc-only push) | | `tests/**` | Test suite only | N/A | + +For hold-gate queue risk tiers (P0 security/shared blast radius, P1 release-gate/defects, P2 tests/docs) and the release-gate expedite lane, see the draft [prioritization rubric](../../../contributing/hold-gate-rubric.md).