Assign external-PR reviewers deterministically, not randomly - #1155
Open
noahho wants to merge 1 commit into
Open
Assign external-PR reviewers deterministically, not randomly#1155noahho wants to merge 1 commit into
noahho wants to merge 1 commit into
Conversation
Math.random() gave one reviewer several consecutive Dependabot PRs (#368/#369 in tabpfn-extensions). Keying on the PR number makes consecutive numbers land on different reviewers, and makes a rerun idempotent. It is a spread rather than a true round robin - long-run counts stay as uneven as random - which the comment in the workflow now states. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the external/dependabot reviewer-assignment GitHub Actions workflow to select a reviewer deterministically based on the PR number, avoiding clustering that can happen with random selection for consecutive Dependabot PRs.
Changes:
- Replace
Math.random()reviewer selection withpull_request.number % reviewers.length. - Update workflow/step wording to reflect deterministic selection and document the rationale.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+66
to
+68
| // opens its PRs in a run of consecutive numbers, so consecutive numbers | ||
| // must not collide: n % len guarantees that, which random did not (one | ||
| // reviewer drew two of #368/#369 in tabpfn-extensions). It is also |
Comment on lines
+69
to
+72
| // idempotent, so a reopen re-requests the same person rather than adding | ||
| // a second reviewer. Note this is a spread, not a true round robin: | ||
| // long-run counts are as uneven as random, because the numbers that | ||
| // reach this workflow are sparse. Equal counts would need stored state. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the random reviewer pick with a deterministic one keyed on the PR number. The file stays as the source of reviewers.
Why
Dependabot opens its PRs as a run of consecutive numbers, and
Math.random()happily gives the same person several in a row - one reviewer drew both #368 and #369 in tabpfn-extensions. Modulo makes consecutive numbers structurally distinct.Measured against the real eligible-PR history (fork + Dependabot PRs, 8 reviewers):
What this is not: a round robin. Long-run counts are about as uneven as random (modulo spread 4-10 across 8 buckets on TabPFN, 8-16 on tabpfn-extensions; random baseline 4-11 and 8-18), because the PR numbers that reach this workflow are sparse - it fires only for forks and Dependabot, and PR numbers share a counter with issues. Equal counts per reviewer would need stored state or GitHub's own team-based review assignment. This fixes the clustering, not the fairness, and the comment in the workflow says so.
Side benefit: it is idempotent. A reopen or ready-for-review re-request now names the same person instead of possibly adding a second reviewer.
Same change is going into the other copy of this workflow (TabPFN and tabpfn-extensions are the only two repos that have it).
🤖 Generated with Claude Code