Repository navigation
Add pr-triage, one Jev call per pull request, sorted into review tiers - #3
Merged
Merged
Conversation
pr-triage sorts a repository's open pull requests into four review tiers. It asks eleven questions about each one in a single call, weights nine of the answers into a review load from 0 to 1, and maps that load onto a tier. Python owns every threshold on a number, so a size floor raises the tier for a diff too large to send whole, and never lowers one. Two modules, because they want different credentials. fetch_prs.py builds a dataset from the GitHub API and needs no Jev key. pr_triage.py scores a dataset and needs no GitHub token, so a dataset can be replayed as often as you like for the cost of the Jev calls alone. Measured on 25 September 2026: 26 open pull requests from agentic-community/mcp-gateway-registry ran 183,426 input tokens and $0.0077, about four seconds for the batch at 159 ms a call. The distribution is the part worth checking. Those 26 came out 13 high, 12 medium, 1 low and 0 trivial, which reads like a broken threshold until the same cuts run against a repository with mixed traffic. 18 recent open pull requests from apache/airflow came out 3 high, 5 medium, 8 low and 2 trivial. Both reports ship in data/. The datasets they scored stay out, because one mirrors 26 contributors' diffs and one command rebuilds it. Repo wiring: a row in the root README table and a block under "Running a sample", the layout tree and the testing paths in AGENTS.md, step 3 of "Adding a sample" noting that a state with several fields takes one budget per field, and a .gitignore rule that keeps the two committed reports while dropping the datasets. Verified: py_compile, ruff check and ruff format --check clean on both modules. Run live against a key: a fetch by owner/repo, a fetch by URL, --since as a day count and as a date, --all, a --dataset replay, --explain on a number in the dataset and on one absent from it, a 404 repo, a malformed repo name, a malformed --since, and a missing dataset path. The file-list truncation arithmetic was checked against 200 synthetic files, because no pull request in either dataset reaches the 4,000-character file-list budget. Known limits: no tests, matching the rest of the repo, and no Go port or CI wiring, unlike agents-md-readiness. --state closed is implemented and has never run against Jev. Closed pull requests are the obvious calibration set, since their review cost is already known.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third sample. It sorts a repository's open pull requests into four review tiers, asking eleven questions about each one in a single Jev call.
What it does
fetch_prs.pybuilds a dataset from the GitHub API and needs no Jev key.pr_triage.pyscores a dataset and needs no GitHub token, so one dataset replays as often as you like for the cost of the Jev calls alone.Nine of the eleven answers carry a weight and average into a review load from 0 to 1, which maps onto a tier. Python owns every threshold on a number: a size floor raises the tier for a diff too large to send whole, and never lowers one.
Measured
Against 26 open pull requests from
agentic-community/mcp-gateway-registryon 25 September 2026: 183,426 input tokens, $0.0077, about four seconds for the batch at 159 ms a call.Those 26 came out 13 high, 12 medium, 1 low and 0 trivial. Running the same cuts against 18 recent open pull requests from
apache/airflowgave 3 high, 5 medium, 8 low and 2 trivial, so both distributions come from one set of thresholds. Anyone editingLOAD_FLOORSshould re-run both, because a single repository cannot tell a bad cut from a hard queue.Both triage reports ship in
data/. The datasets they scored stay gitignored: one mirrors 26 contributors' diffs, and one command rebuilds it.Repo wiring
README.md: a row in the samples table, and apr-triageblock under "Running a sample" noting it wants a GitHub tokenAGENTS.md: the layout tree, the testing paths for this sample, and step 3 of "Adding a sample" recording that a state with several fields takes one budget per field.gitignore: keeps the two committed reports, drops the datasetsVerified
py_compile,ruff checkandruff format --checkclean on both modules.Run live against a key: a fetch by
owner/repo, a fetch by URL,--sinceas a day count and as a date,--all, a--datasetreplay,--explainon a number in the dataset and on one absent from it, a 404 repo, a malformed repo name, a malformed--since, and a missing dataset path.The file-list truncation arithmetic was checked against 200 synthetic files, because no pull request in either dataset reaches the 4,000-character file-list budget.
Every figure in the sample README was checked against the committed reports, including the worked example, whose nine weighted rows sum to the 0.8247 load the report records.
Known limits
agents-md-readiness--state closedis implemented and has never run against Jev. Closed pull requests are the obvious calibration set, since their review cost is already known