Skip to content

✨ Reviewer preflight: repo rules, dependency advisories and release age, subtle-defect guidance - #19075

Merged
kubestellar-prow[bot] merged 4 commits into
v5from
feat/review-evidence-19069
Nov 18, 2026
Merged

kubestellar-prow[bot] merged 4 commits into
v5from
feat/review-evidence-19069

Conversation

@clubanderson

Copy link
Copy Markdown
Member

What this adds

A general, configurable reviewer preflight for every hive, repository and language: facts the hive gathers before the reviewer reads the diff, so a review can rest on the repo's own rules and on advisory ids instead of on a hunch. Config lives under review.preflight, with per-repo overrides at repos.<repo>.review_preflight.

  • Repository rules (on by default, no new egress). The hive lists the repo at the PR base commit and tells the reviewer which rule documents exist (AGENTS.md, CLAUDE.md, CONTRIBUTING*, DESIGN*/ARCHITECTURE*/STYLE* docs, docs/design, docs/adr, docs/architecture, copilot and cursor rules, CODEOWNERS). Patterns are configurable (rule_doc_paths, rule_doc_exclude), the byte budget is deterministic (rule_doc_budget_bytes), and the reviewer must name the document and rule it applied. Operators can also define short named rules with a severity that the reviewer enforces.
  • Dependency evidence (off by default). For dependencies the PR adds or bumps, an ecosystem table (manifest patterns, parser, OSV ecosystem id, registry publish date) covers Go, npm/yarn/pnpm, Python, Rust, Java, Ruby, PHP, .NET, GitHub Actions and container images. OSV advisories and release age (min_release_age_days, default 14) become evidence with advisory ids. Unsupported manifests, withheld patches, SHA-pinned or floating action refs and over-limit dependencies are reported as "not checked", never skipped silently. Adding an ecosystem is a table entry plus a test.
  • Control and egress. Per-check flags, offline (or HIVE_REVIEW_PREFLIGHT_OFFLINE=true) giving an explicit "could not check: offline", internal mirror URLs, in-memory cache with TTL, per-request timeout, 429/5xx degrade per check with the reason, upstream proxy honored, no credentials sent, only package name, ecosystem and version. The docs state exactly what leaves the hive.
  • Noise control. Findings need an advisory id, rule citation or concrete evidence. block_severity maps advisory severity to verdict impact, severity_floor hides low noise, and ignore accepts a known advisory with a required reason and an optional expiry (an expired entry stops suppressing and says so). Third-party text never reaches the prompt, only ids, versions, dates and paths.
  • Two reviewer checks in the correctness perspective and combined review: messages and comments that stopped being true after the change, and sentinel values flowing into min/max/sum aggregates.

Decision for the operator: egress default

Dependency checks are off by default because they contact third-party services (OSV and package registries) about repositories that may be private: the services see the hive's egress IP and the dependency names of reviewed PRs. Rule discovery uses only the GitHub API the hive already calls, so it is on by default. Turning dependency checks on is review.preflight.dependencies: true (or per repo).

Motivating evidence

@MikeSpreitzer compared several review tools on one of his PRs. The hive covered the duplicated word, an outdated comment and the PR description accuracy, and missed: repo rule documents, vulnerability databases, dependency release age, outdated failure messages and a sentinel/min bug; the red dots on other tools are why every check here requires evidence. The full table is in #19069.

Table row Covered by
Design/rules documents considered rule-document discovery and named rules
Vulnerability DBs consulted OSV check
Dependency release age considered registry publish date and threshold
Outdated failure messages correctness prompt guidance
Sentinels in total -> bug in min correctness prompt guidance

Not in this PR (listed in #19069): general web search for dependency vulnerabilities, a deterministic outdated-message hint, transitive resolution, container image scanning, release age for Actions and images, Settings dialog controls (allowlisted in the parity file, #19069).

Tests and docs

Table-driven parser tests for 17 manifest cases across all ecosystems, registry age lookups, OSV and registry fakes via httptest, offline and env-offline degradation, outage not read as clean, cache reuse, ignore list with expiry, severity floor and block threshold, rule-doc budget determinism, per-repo override precedence, config validation, prompt wiring. Docs: "Review preflight" in src/docs/review-swarm.md, src/hive.yaml.example, changelog fragment. No local build or test was run (repo rule); CI is the gate.

Fixes #19069

🤖 Generated with Claude Code

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#19073 ✨ Settings UI for per-repo parent auto-close policy CONFLICT src/pkg/config/repo_settings.go
src/pkg/dashboard/settings_parity_allowlist.txt (conflict)
#18873 🐛 needs-sha: exempt owner/detailed issues, overrideable waiver, visible hold on the row same files src/hive.yaml.example

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 18, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@clubanderson clubanderson changed the title 🔎 Reviewer preflight: repo rules, dependency advisories and release age, subtle-defect guidance ✨ Reviewer preflight: repo rules, dependency advisories and release age, subtle-defect guidance Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing

42 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/cmd/hive/review_preflight.go 42 CONDITIONALS_NEGATION
src/cmd/hive/review_preflight.go 62 CONDITIONALS_NEGATION
src/cmd/hive/review_preflight.go 72 CONDITIONALS_NEGATION
src/cmd/hive/review_preflight.go 72 CONDITIONALS_NEGATION
src/cmd/hive/review_preflight.go 81 CONDITIONALS_BOUNDARY
src/cmd/hive/review_preflight.go 81 CONDITIONALS_NEGATION
src/pkg/config/review_preflight_config.go 35 ARITHMETIC_BASE
src/pkg/config/review_preflight_config.go 37 ARITHMETIC_BASE
src/pkg/config/review_preflight_config.go 176 CONDITIONALS_NEGATION
src/pkg/config/review_preflight_config.go 178 CONDITIONALS_NEGATION
src/pkg/config/review_preflight_config.go 255 CONDITIONALS_BOUNDARY
src/pkg/config/review_preflight_config.go 255 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 17 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 17 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 17 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 22 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 22 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 26 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 37 CONDITIONALS_BOUNDARY
src/pkg/github/review_preflight.go 37 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 37 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 37 CONDITIONALS_NEGATION
src/pkg/github/review_preflight.go 42 CONDITIONALS_NEGATION
src/pkg/review/prompts_preflight.go 19 ARITHMETIC_BASE
src/pkg/review/prompts_preflight.go 20 ARITHMETIC_BASE
src/pkg/reviewpreflight/build.go 140 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/build.go 141 CONDITIONALS_BOUNDARY
src/pkg/reviewpreflight/build.go 141 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/build.go 143 CONDITIONALS_BOUNDARY
src/pkg/reviewpreflight/build.go 143 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/build.go 314 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/build.go 316 CONDITIONALS_BOUNDARY
src/pkg/reviewpreflight/build.go 316 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/client.go 19 ARITHMETIC_BASE
src/pkg/reviewpreflight/client.go 20 ARITHMETIC_BASE
src/pkg/reviewpreflight/client.go 94 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/client.go 96 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/client.go 98 CONDITIONALS_BOUNDARY
src/pkg/reviewpreflight/client.go 98 CONDITIONALS_BOUNDARY
src/pkg/reviewpreflight/client.go 98 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/client.go 98 CONDITIONALS_NEGATION
src/pkg/reviewpreflight/rules.go 148 CONDITIONALS_NEGATION

Verdict: uncovered (#17349).

@clubanderson clubanderson added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 18, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 18, 2026
…ge, subtle-defect guidance (#19069)

Adds review.preflight (per-repo override repos.<repo>.review_preflight): the
hive lists the repo's own rule documents and operator-defined rules for the
reviewer, optionally checks added or bumped dependencies against OSV and
package registries across ten ecosystems with offline degradation, cache,
severity thresholds and an expiring ignore list, and the correctness review
now checks outdated messages and sentinel-in-aggregate bugs.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…ver (#19069)

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
….preflight (#19069)

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…d limits to clear the package coverage floor (#19069)

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@clubanderson
clubanderson force-pushed the feat/review-evidence-19069 branch from 1cf629a to 9b626ce Compare November 18, 2026 18:48
@kubestellar-prow kubestellar-prow Bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 18, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔎 Reviewer evidence: repo rules, dependency vulnerabilities and release age, subtle-defect guidance

1 participant