From 1b17bd06b4665ad6859ba164604ecdc33442c6d0 Mon Sep 17 00:00:00 2001 From: Amit Arora Date: Fri, 25 Sep 2026 21:10:25 +0000 Subject: [PATCH 1/2] Port pr-triage to one Go binary, and vend it as a skill One static executable does both halves: it fetches pull requests from the GitHub API and asks Jev eleven questions about each one, so nothing has to run before it. A machine with no Python can sort a review queue from one file. GitHub Enterprise Server works without a patch, because the API base is a variable rather than a glued-together path. Three ways to set it, highest precedence first: -api-base, the host of a URL passed as the repository, or GITHUB_API_URL, which is what GitHub Actions already exports on both github.com and GHES. The progress line names the API it chose, so a wrong guess surfaces before any call. Credentials each read the flag first and the environment second. GitHub takes -token, GITHUB_TOKEN, GH_TOKEN, GH_ENTERPRISE_TOKEN or the gh CLI. TypeSafe takes -jev-key or TYPESAFE_API_KEY. -fetch-only wants the GitHub token and no TypeSafe key, and -dataset wants the reverse, so the half that reads your source and the half that calls an external API can hold different credentials and run in different places. vend/ makes it adoptable: a SKILL.md telling an agent when to reach for a triage and how to read one, and an installer that fetches the binary and drops the skill beside it. It checks for a GitHub and a TypeSafe credential and names the missing one rather than failing halfway through a run. The judgment is unchanged from the Python: the same eleven questions, the same weights, the same tier cuts, the same 0.02 deadband, the same size floor. Datasets and reports match field for field, so a file written by either tool reads in the other. Verified live: - A fetch of apache/airflow by owner/repo, token from the gh CLI, and a dataset whose keys match the Python's field for field. - An 18 pull request replay returning the same tier counts as the Python, 3 high, 5 medium, 8 low, 2 trivial. - -explain printing all eleven questions, its contribution column summing to exactly the 0.3847 load it reported. - GHES resolution all three ways, and -api-base winning over a URL host. - -fail-on-tier low exiting 2 and naming the pull request, -fail-on-tier high exiting 0. - gofmt, go vet, go test clean. ./build.sh 0.1.0 cross-compiles linux, darwin and windows on amd64 and arm64 with SHA256SUMS. Stripped and static, each binary is about 7 MB against 10.4 MB for a plain build. Two bugs fixed in this port. Go's flag package stops parsing at the first non-flag argument, so `pr-triage owner/repo -all` silently ignored -all; the binary now takes the repository and parses what follows it. And main was overriding a gate's exit code with the generic 1. One bug left alone, in the Python payload. design_decisions level 0 reads `- None: the same edit repeated, or a value changed`, which is YAML for a mapping rather than a string, so Jev receives an object as that rubric level and echoes it back in the legend. The Go side reads a legend value as any type, so a released binary keeps working against an old payload. Quoting the string would fix it and would move every recorded number, so it wants its own change. Also adds go/README.md, three .gitignore lines for a plain go build in the new folder, the layout and testing rules in AGENTS.md, and a row in the root README. --- .gitignore | 3 + AGENTS.md | 5 +- README.md | 2 +- samples/pr-triage/README.md | 18 + samples/pr-triage/go/README.md | 119 ++++++ samples/pr-triage/go/build.sh | 55 +++ samples/pr-triage/go/fetch.go | 591 +++++++++++++++++++++++++++ samples/pr-triage/go/go.mod | 5 + samples/pr-triage/go/go.sum | 4 + samples/pr-triage/go/install.sh | 112 +++++ samples/pr-triage/go/main.go | 335 +++++++++++++++ samples/pr-triage/go/output.go | 346 ++++++++++++++++ samples/pr-triage/go/payload.go | 248 +++++++++++ samples/pr-triage/go/payload_test.go | 106 +++++ samples/pr-triage/go/questions.yml | 149 +++++++ samples/pr-triage/go/report.go | 255 ++++++++++++ samples/pr-triage/go/score.go | 512 +++++++++++++++++++++++ samples/pr-triage/vend/SKILL.md | 96 +++++ samples/pr-triage/vend/install.sh | 75 ++++ 19 files changed, 3034 insertions(+), 2 deletions(-) create mode 100644 samples/pr-triage/go/README.md create mode 100755 samples/pr-triage/go/build.sh create mode 100644 samples/pr-triage/go/fetch.go create mode 100644 samples/pr-triage/go/go.mod create mode 100644 samples/pr-triage/go/go.sum create mode 100755 samples/pr-triage/go/install.sh create mode 100644 samples/pr-triage/go/main.go create mode 100644 samples/pr-triage/go/output.go create mode 100644 samples/pr-triage/go/payload.go create mode 100644 samples/pr-triage/go/payload_test.go create mode 100644 samples/pr-triage/go/questions.yml create mode 100644 samples/pr-triage/go/report.go create mode 100644 samples/pr-triage/go/score.go create mode 100644 samples/pr-triage/vend/SKILL.md create mode 100755 samples/pr-triage/vend/install.sh diff --git a/.gitignore b/.gitignore index 2c92cb8..9d24776 100644 --- a/.gitignore +++ b/.gitignore @@ -45,6 +45,9 @@ samples/pr-triage/data/* samples/agents-md-readiness/go/dist/ samples/agents-md-readiness/go/agents-md-readiness samples/agents-md-readiness/go/go +samples/pr-triage/go/dist/ +samples/pr-triage/go/pr-triage +samples/pr-triage/go/go *.exe # OS diff --git a/AGENTS.md b/AGENTS.md index b4c5641..0c73c0d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,9 @@ jev-samples/ fetch_prs.py builds the dataset from the GitHub API, needs no Jev key pr_triage.py the Jev call, the tiers, the tables data/ triage reports, committed; the datasets they scored, gitignored + assets/ the pipeline diagram: HTML source, rendered PNG, renderer + go/ both halves in one static binary, github.com and GHES + vend/ a SKILL.md and an installer, so any repo can use it ``` ## Setup and commands @@ -54,7 +57,7 @@ One check runs in CI: `.github/workflows/agents-md-readiness.yml` scores this fi 2. `uvx ruff check .` and `uvx ruff format --check .` both report clean. 3. The sample runs against a live key, and every code path you changed runs at least once. For readme-check that means a local file, a GitHub repo root, a `/blob/` file page, and an `http://` URL it should refuse. For agents-md-readiness add a repo holding neither AGENTS.md nor CLAUDE.md, and a directory with neither. For pr-triage that means a live fetch, a `--dataset` replay, `--explain` on a number in the dataset and on one that is absent, and a repo whose pull requests include a diff too large to send whole. 4. Any output shown in a README comes from a run you just did, with the date next to it. -5. `samples/agents-md-readiness/go/` compiles that sample into one static binary, so a change to its `questions.yml` or its Go files means `gofmt -l .`, `go vet ./...`, `go test ./...` and `./build.sh ` in that folder, then one live run of the binary you built. The Python stays canonical, and `payload_test.go` fails when the embedded copy of the payload drifts. +5. `samples/agents-md-readiness/go/` and `samples/pr-triage/go/` each compile a sample into one static binary, so a change to a `questions.yml` or to any Go file in those folders means `gofmt -l .`, `go vet ./...`, `go test ./...` and `./build.sh ` in that folder, then one live run of the binary you built. The Python stays canonical, and `payload_test.go` fails when the embedded copy of the payload drifts. Run the whole list before you open a pull request, because CI covers the AGENTS.md check and nothing else. If you add a test suite, use pytest, mock the client rather than calling the API, and wire it into that workflow. diff --git a/README.md b/README.md index cb24794..5938246 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ Every sample is a small self-contained project under [samples/](samples/), and y | --- | --- | | [readme-check](samples/readme-check/) | Reads a README from disk or a GitHub URL, asks five questions about it in one call, and explains every number it prints. Uses all three primitives: `Choice`, `Score` and `Noul`. | | [agents-md-readiness](samples/agents-md-readiness/) | Scores an AGENTS.md or CLAUDE.md against the [agents.md](https://agents.md) format: seventeen questions in one call, a padded markdown table, one weighted readiness score, and a JSON report per document. Ships scored runs for eleven open-source repos in [data/](samples/agents-md-readiness/data/), plus a [Go port](samples/agents-md-readiness/go/) that compiles the same check into one static binary for a CI runner without Python. | -| [pr-triage](samples/pr-triage/) | Sorts a repository's open pull requests into four review tiers, one call each: eleven questions about the title, the description, the file list and the diff, nine of them weighted into a review load, and a size floor in Python for the diffs too large to send whole. Builds its dataset from the GitHub API, prints a triage table grouped by tier with the advice for each one, and ships the scored runs for two repositories in [data/](samples/pr-triage/data/). Twenty-six pull requests took four seconds and $0.0077. | +| [pr-triage](samples/pr-triage/) | Sorts a repository's open pull requests into four review tiers, one call each: eleven questions about the title, the description, the file list and the diff, nine of them weighted into a review load, and a size floor in Python for the diffs too large to send whole. Builds its dataset from the GitHub API, prints a triage table grouped by tier with the advice for each one, and ships the scored runs for two repositories in [data/](samples/pr-triage/data/). Twenty-six pull requests took four seconds and $0.0077. Ships a [Go port](samples/pr-triage/go/) that does both halves in one static binary, against github.com or a GitHub Enterprise Server host, and a [vendable skill](samples/pr-triage/vend/) that installs the binary and drives it. | Each sample keeps its Jev payload in `questions.yml`: the model pin, the state budget, every question with the label it prints under, and what each answer is worth toward the score. The Python reads that file and does the arithmetic, so changing a question or a weight is a data change and changing a threshold is a code change. diff --git a/samples/pr-triage/README.md b/samples/pr-triage/README.md index 37e1afe..ec7d7aa 100644 --- a/samples/pr-triage/README.md +++ b/samples/pr-triage/README.md @@ -57,6 +57,24 @@ uv run pr_triage.py --dataset data/owner-repo-open-all.json --explain 1693 `pr_triage.py` needs a Jev key in `TYPESAFE_API_KEY`, from the environment or from `.env` beside the sample or at the repo root. +### One binary, and a skill that drives it + +A machine with no Python can run the same check from a single static executable. [`go/`](go/) holds a Go port that does both halves in one command, against github.com or a GitHub Enterprise Server host: + +```bash +curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh | sh +pr-triage apache/airflow -all -fail-on-tier high +pr-triage https://ghe.example.com/owner/repo +``` + +[`vend/`](vend/) turns that into something any repository can adopt: a `SKILL.md` telling an agent when to reach for the triage and how to read it, and an installer that fetches the binary and drops the skill beside it. + +```bash +curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/vend/install.sh | sh +``` + +The Python stays canonical. `go/questions.yml` is a copy, `build.sh` refreshes it before every build, and `payload_test.go` fails when the two drift, so the two tools cannot score one pull request differently in silence. + ## What it prints From a run on 25 September 2026 against the twenty-six open pull requests of [agentic-community/mcp-gateway-registry](https://github.com/agentic-community/mcp-gateway-registry), trimmed to the ends of the table: diff --git a/samples/pr-triage/go/README.md b/samples/pr-triage/go/README.md new file mode 100644 index 0000000..93f530e --- /dev/null +++ b/samples/pr-triage/go/README.md @@ -0,0 +1,119 @@ +# pr-triage, as one binary + +The Python sample one directory up is canonical. This is the same check compiled into a single static executable, for a machine with no Python: a CI runner, a bastion host, a developer laptop that should not grow a virtualenv to sort a review queue. + +One binary does both halves. It fetches pull requests from the GitHub API and asks Jev eleven questions about each one, so nothing has to run before it. + +## Install + +```bash +curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh | sh +``` + +The script resolves the newest `pr-triage/` release, picks the build for the platform, checks it against the published `SHA256SUMS`, and drops it in `/usr/local/bin` or `~/.local/bin`. `VERSION` and `BINDIR` override both choices. + +To install the skill alongside the binary, so an agent knows when to reach for it, use [`../vend/install.sh`](../vend/install.sh) instead. + +## Run it + +```bash +pr-triage apache/airflow # the 10 most recent open ones +pr-triage apache/airflow -all # every open one +pr-triage apache/airflow -since 30 # opened in the last 30 days +pr-triage apache/airflow -since 2026-08-01 # opened on or after a date +pr-triage apache/airflow -explain 73713 # one pull request in full +pr-triage apache/airflow -fetch-only # build the dataset, skip Jev +pr-triage -dataset data/apache-airflow-open-all.json +``` + +Flags can sit on either side of the repository, because the binary reorders them before parsing. Go's `flag` package stops at the first non-flag argument on its own, which would leave `-all` unread. + +## Credentials + +Each one reads the flag first and the environment second. + +| What | Flag | Environment | +| --- | --- | --- | +| GitHub | `-token` | `GITHUB_TOKEN`, `GH_TOKEN`, `GH_ENTERPRISE_TOKEN`, then `gh auth token` | +| TypeSafe | `-jev-key` | `TYPESAFE_API_KEY` | + +`-fetch-only` needs the GitHub token and no TypeSafe key. `-dataset` needs the TypeSafe key and no GitHub token. Splitting a run that way lets the half that touches your source code and the half that calls an external API hold different credentials, and run in different places. + +Without a GitHub token the API allows sixty requests an hour, and one pull request costs two of them. + +## GitHub Enterprise Server + +The API base is a variable, not a glued-together path, so an enterprise host needs no patch. Three ways to set it, highest precedence first: + +```bash +pr-triage owner/repo -api-base https://ghe.example.com/api/v3 # the flag +pr-triage https://ghe.example.com/owner/repo # from the URL host +export GITHUB_API_URL=https://ghe.example.com/api/v3 # the environment +``` + +A URL on `github.com` resolves to `https://api.github.com`. A URL on any other host resolves to `https://that-host/api/v3`, which is where GHES serves the v3 API. `GITHUB_API_URL` is the variable GitHub Actions already sets on both, so a workflow needs no flag. + +The progress line says which API it chose, so a wrong guess shows up before any call: + +``` +repository acme/widgets through https://ghe.example.com/api/v3 +``` + +## What it writes + +Two files per run, into `-out` (default `./data`), plus the dataset when it fetched one: + +- `--triage.json`, every answer as Jev sent it, beside the credit, the tier and the size floor +- `--triage.md`, the table and the tier sections, ready to paste into a pull request + +Both match the Python sample's reports field for field, and the dataset matches too, so a dataset or report written by either tool reads in the other. + +## Gating CI + +`-fail-on-tier` exits 2 when any pull request lands in that tier or above: + +```bash +pr-triage owner/repo -all -fail-on-tier high -quiet +``` + +| Exit | Meaning | +| --- | --- | +| 0 | finished, nothing tripped a gate | +| 1 | could not finish: a bad flag, no key, an API that refused | +| 2 | a gate fired | + +`-quiet` drops the progress lines from standard error and leaves the report on standard output, which is what a job that captures output wants. + +## Build it yourself + +```bash +./build.sh 0.1.0 +``` + +That copies the canonical `../questions.yml` in, runs the tests, cross-compiles for linux, darwin and windows on amd64 and arm64, and writes `SHA256SUMS` beside the binaries in `dist/`. Every build is static (`CGO_ENABLED=0`) and stripped (`-s -w`), which takes a binary from 10.4 MB to about 7 MB. + +A plain build works too: + +```bash +go build -o pr-triage . && ./pr-triage -version +``` + +## The payload, and why a test guards it + +`questions.yml` in this folder is a copy. The Python sample owns the canonical one, and `go:embed` cannot read outside its own directory, so `build.sh` copies it in before every build and `payload_test.go` fails when the two drift: + +``` +../questions.yml and the embedded copy differ: run ./build.sh, which copies the canonical file in +``` + +Without that test the two tools would score the same pull request differently, for a reason nobody could see from either side. + +`-questions path.yml` points at a payload on disk instead, which is how you try a different weight without rebuilding. + +## Where this differs from the Python + +Nothing in the judgment: the same eleven questions, the same weights, the same tier cuts, the same 0.02 deadband, the same size floor. Two differences worth knowing: + +**The legend is read loosely.** Jev echoes each rubric level back in a `legend`, and one level in the current payload arrives as an object rather than a string, because `- None: some text` is YAML for a mapping. The Go side accepts any value there, so a released binary keeps working against an older payload. + +**Repeat runs move a load by about 0.02**, the same as the Python. A tier within the deadband of a cut prints `(on a cut)` rather than picking a side, so two runs of one queue agree with each other. diff --git a/samples/pr-triage/go/build.sh b/samples/pr-triage/go/build.sh new file mode 100755 index 0000000..4e19600 --- /dev/null +++ b/samples/pr-triage/go/build.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# Cross-compile pr-triage for every platform install.sh knows about, +# and write SHA256SUMS beside the binaries. +# +# ./build.sh 0.1.0 +# +# The Python sample one directory up is the canonical implementation, and its +# questions.yml is the canonical payload. go:embed cannot reach outside this +# folder, so this script copies that file in before building, and +# payload_test.go fails if the copy ever drifts. +# +# Every binary is static (CGO_ENABLED=0), so it runs on a machine with no Go, no +# Python, and no libc of a particular vintage. +set -euo pipefail + +version="${1:-dev}" +here="$(cd "$(dirname "$0")" && pwd)" +canonical="$here/../questions.yml" +dist="$here/dist" + +if ! cmp -s "$canonical" "$here/questions.yml"; then + echo "syncing questions.yml from the Python sample" + cp "$canonical" "$here/questions.yml" +fi + +go test ./... + +rm -rf "$dist" +mkdir -p "$dist" + +targets=( + "linux amd64" + "linux arm64" + "darwin amd64" + "darwin arm64" + "windows amd64" +) + +for target in "${targets[@]}"; do + read -r os arch <<<"$target" + name="pr-triage_${version}_${os}_${arch}" + [ "$os" = "windows" ] && name="$name.exe" + + CGO_ENABLED=0 GOOS="$os" GOARCH="$arch" go build \ + -trimpath \ + -ldflags "-s -w -X main.version=$version" \ + -o "$dist/$name" \ + "$here" + echo "built $name" +done + +cd "$dist" +sha256sum ./* > SHA256SUMS +echo +cat SHA256SUMS diff --git a/samples/pr-triage/go/fetch.go b/samples/pr-triage/go/fetch.go new file mode 100644 index 0000000..feb7e43 --- /dev/null +++ b/samples/pr-triage/go/fetch.go @@ -0,0 +1,591 @@ +// The GitHub half: turn a repository name into a dataset of pull requests. +// +// This half talks to the GitHub REST API and never to Jev, so it runs with a +// GitHub token and no TypeSafe key. It works against github.com and against a +// GitHub Enterprise Server host, because every endpoint is built from one base +// URL that the caller can set. +// +// The dataset it writes is the same JSON the Python sample writes, field for +// field, so a dataset built by either tool replays through either tool. +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "os" + "os/exec" + "regexp" + "strconv" + "strings" + "time" +) + +const ( + // defaultAPIBase is github.com's API. A GitHub Enterprise Server host puts its + // API under /api/v3 instead, which is why the base is a variable rather than a + // path glued together at each call site. + defaultAPIBase = "https://api.github.com" + // ghesAPIPath is the suffix a GHES host serves the v3 API from. + ghesAPIPath = "/api/v3" + // apiVersion is the version header GitHub asks callers to pin. Without it the + // API is free to change shape under this binary. + apiVersion = "2022-11-28" + // userAgent identifies this tool in GitHub's logs, which the API requires. + userAgent = "jev-samples-pr-triage" + // perPage is GitHub's maximum, and so the fewest round trips per repository. + perPage = 100 + // maxPages stops the walk rather than paginating through a repository with + // 4,000 open pull requests. -all means all of them up to this. + maxPages = 20 + // fetchTimeout caps one API call. + fetchTimeout = 30 * time.Second + // ghTokenTimeout caps the shell out to the gh CLI, which can sit waiting on a + // keyring prompt. + ghTokenTimeout = 10 * time.Second + // maxPatchChars cuts one file's patch as it lands in the dataset. A generated + // lockfile diff runs to hundreds of thousands of characters and would dominate + // the file on disk. The triage half applies its own, smaller budget when it + // builds the state it sends. + maxPatchChars = 40000 + // defaultLimit is how many recent pull requests to take when the caller names + // no selector. + defaultLimit = 10 +) + +// tokenEnvNames lists where the environment carries a GitHub token, in the order +// checked. GH_ENTERPRISE_TOKEN is the one a GHES user sets, and gh reads the same +// name, so a machine already talking to an enterprise host needs no new variable. +var tokenEnvNames = []string{"GITHUB_TOKEN", "GH_TOKEN", "GH_ENTERPRISE_TOKEN"} + +// apiBaseEnvNames lists where the environment carries the API base. GitHub +// Actions sets GITHUB_API_URL on both github.com and GHES, so a workflow needs no +// flag at all. +var apiBaseEnvNames = []string{"GITHUB_API_URL", "GH_API_URL"} + +// repoPattern matches `owner/repo`, which is what the API path wants and what gh +// prints. MustCompile panics on a bad pattern at startup rather than returning an +// error at the first call, which is what you want for a constant pattern. +var repoPattern = regexp.MustCompile(`^[\w.-]+/[\w.-]+$`) + +// slugPattern collapses everything that is not a letter or a digit into one dash, +// so a repository name becomes a filename. +var slugPattern = regexp.MustCompile(`[^a-z0-9]+`) + +// changedFile is one file in a pull request, as the dataset stores it. +// +// The json tags name the keys, and they match the Python sample's dataset exactly +// so the two tools read each other's files. +type changedFile struct { + Path string `json:"path"` + // PreviousPath is set when GitHub reports the file as renamed. It is a pointer + // so it can be null in JSON, which is what the Python writes for a file that + // was not renamed. + PreviousPath *string `json:"previous_path"` + Status string `json:"status"` + Additions int `json:"additions"` + Deletions int `json:"deletions"` + Patch string `json:"patch"` + PatchTruncated bool `json:"patch_truncated"` + HasPatch bool `json:"has_patch"` +} + +// pullRequest is one pull request, as the dataset stores it. +type pullRequest struct { + Number int `json:"number"` + Title string `json:"title"` + Body string `json:"body"` + Author string `json:"author"` + CreatedAt string `json:"created_at"` + UpdatedAt string `json:"updated_at"` + Draft bool `json:"draft"` + Labels []string `json:"labels"` + Base string `json:"base"` + URL string `json:"url"` + ChangedFiles int `json:"changed_files"` + Additions int `json:"additions"` + Deletions int `json:"deletions"` + Files []changedFile `json:"files"` + // FilesTruncated says whether pagination cut the file list short, which GitHub + // also does at 3,000 files. + FilesTruncated bool `json:"files_truncated"` +} + +// selector records which pull requests a dataset holds, so a reader can tell a +// ten-most-recent dataset from an everything-open one. +type selector struct { + State string `json:"state"` + // Limit is a pointer so it can be null, which means "no limit". + Limit *int `json:"limit"` + // Since is a pointer for the same reason: null means no date cutoff. + Since *string `json:"since"` +} + +// dataset is the whole file: which repository, when, which selector, and the pull +// requests themselves. +type dataset struct { + Repo string `json:"repo"` + FetchedAt string `json:"fetched_at"` + Selector selector `json:"selector"` + PullRequests []pullRequest `json:"pull_requests"` +} + +// target names a repository and the API that serves it. +type target struct { + // Repo is `owner/repo`. + Repo string + // APIBase is the root every endpoint hangs off, with no trailing slash. + APIBase string +} + +// parseTarget reads `owner/repo` out of whatever the caller passed, and works out +// which API serves it. +// +// It accepts the bare form and a full URL, so a pasted browser address works. A +// URL on a host other than github.com is taken as a GitHub Enterprise Server +// address, and its API base becomes https://that-host/api/v3. An explicit +// -api-base always wins, because a GHES install can serve its API from somewhere +// else entirely. +func parseTarget(raw, apiBaseFlag string) (target, error) { + cleaned := strings.TrimRight(strings.TrimSpace(raw), "/") + base := "" + + // A URL carries its own host, which is the only hint available about whether + // this is github.com or an enterprise install. + if strings.HasPrefix(cleaned, "https://") || strings.HasPrefix(cleaned, "http://") { + parsed, err := url.Parse(cleaned) + if err != nil { + return target{}, fmt.Errorf("could not read %q as a URL: %w", raw, err) + } + // Path is /owner/repo, possibly with more after it, so the first two + // non-empty segments are the repository. + parts := strings.Split(strings.Trim(parsed.Path, "/"), "/") + if len(parts) < 2 || parts[0] == "" || parts[1] == "" { + return target{}, fmt.Errorf("expected owner/repo in the URL, got %q", raw) + } + cleaned = parts[0] + "/" + parts[1] + + if parsed.Host == "github.com" || parsed.Host == "www.github.com" { + base = defaultAPIBase + } else { + base = parsed.Scheme + "://" + parsed.Host + ghesAPIPath + } + } + + if !repoPattern.MatchString(cleaned) { + return target{}, fmt.Errorf("expected owner/repo, got %q", raw) + } + + // Precedence: the flag, then what the URL implied, then the environment, then + // github.com. The flag wins so a caller can always override a wrong guess. + switch { + case apiBaseFlag != "": + base = apiBaseFlag + case base != "": + default: + base = apiBaseFromEnv() + } + return target{Repo: cleaned, APIBase: strings.TrimRight(base, "/")}, nil +} + +// apiBaseFromEnv reads the API base out of the environment, falling back to +// github.com. +func apiBaseFromEnv() string { + for _, name := range apiBaseEnvNames { + if value := strings.TrimSpace(os.Getenv(name)); value != "" { + return value + } + } + return defaultAPIBase +} + +// resolveToken finds a GitHub token: the flag, then the environment, then the gh +// CLI. It returns an empty string when there is none, which means calling the API +// unauthenticated at sixty requests an hour. +func resolveToken(flagValue string) string { + if strings.TrimSpace(flagValue) != "" { + return strings.TrimSpace(flagValue) + } + for _, name := range tokenEnvNames { + if value := strings.TrimSpace(os.Getenv(name)); value != "" { + logf("read a GitHub token from %s", name) + return value + } + } + return tokenFromGHCLI() +} + +// tokenFromGHCLI asks the gh CLI for the token it already holds. +// +// exec.Command runs a program rather than a shell, so nothing here goes through +// shell quoting and no caller input reaches the command line. +func tokenFromGHCLI() string { + // A context with a deadline stops a gh that sits waiting on a keyring prompt. + ctx, cancel := context.WithTimeout(context.Background(), ghTokenTimeout) + defer cancel() + + out, err := exec.CommandContext(ctx, "gh", "auth", "token").Output() + if err != nil { + logf("no token from the gh CLI, so calling GitHub unauthenticated at 60 requests an hour") + return "" + } + token := strings.TrimSpace(string(out)) + if token != "" { + logf("read a GitHub token from the gh CLI") + } + return token +} + +// getJSON fetches one API URL and decodes the body into out. +// +// out is a pointer to the value to fill, which is how Go's json package writes +// into a caller's variable. +func getJSON(endpoint, token string, out any) error { + ctx, cancel := context.WithTimeout(context.Background(), fetchTimeout) + defer cancel() + + request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil) + if err != nil { + return fmt.Errorf("building a request for %s: %w", endpoint, err) + } + request.Header.Set("Accept", "application/vnd.github+json") + request.Header.Set("X-GitHub-Api-Version", apiVersion) + request.Header.Set("User-Agent", userAgent) + if token != "" { + request.Header.Set("Authorization", "Bearer "+token) + } + + response, err := http.DefaultClient.Do(request) + if err != nil { + return fmt.Errorf("calling %s: %w", endpoint, err) + } + // defer runs when this function returns, whichever path it takes, so the body + // closes even on the error returns below. + defer response.Body.Close() + + body, err := io.ReadAll(response.Body) + if err != nil { + return fmt.Errorf("reading the response from %s: %w", endpoint, err) + } + + // Each of these statuses has one likely cause worth naming, because the raw + // message from GitHub does not say which of them applies. + switch response.StatusCode { + case http.StatusOK: + case http.StatusNotFound: + return fmt.Errorf( + "GitHub returned 404 for %s\n check the repository name, and that this token can see it", + endpoint, + ) + case http.StatusUnauthorized, http.StatusForbidden: + if response.Header.Get("x-ratelimit-remaining") == "0" { + return fmt.Errorf( + "GitHub rate limit exhausted. Set a token with -token or GITHUB_TOKEN, or wait for the window to reset", + ) + } + return fmt.Errorf( + "GitHub returned %d for %s\n the token is missing, expired, or lacks access", + response.StatusCode, endpoint, + ) + default: + return fmt.Errorf("GitHub returned %d for %s", response.StatusCode, endpoint) + } + + if err := json.Unmarshal(body, out); err != nil { + return fmt.Errorf("parsing the response from %s: %w", endpoint, err) + } + return nil +} + +// listEntry is the part of the pulls list endpoint this binary reads. The +// endpoint returns far more, and naming only these fields keeps the shape of the +// dataset in one place. +type listEntry struct { + Number int `json:"number"` + Title string `json:"title"` + Body string `json:"body"` + CreatedAt string `json:"created_at"` + UpdatedAt string `json:"updated_at"` + Draft bool `json:"draft"` + HTMLURL string `json:"html_url"` + User struct { + Login string `json:"login"` + } `json:"user"` + Labels []struct { + Name string `json:"name"` + } `json:"labels"` + Base struct { + Ref string `json:"ref"` + } `json:"base"` +} + +// listPulls lists pull requests newest first, stopping as soon as the selector is +// met. +// +// The pulls endpoint takes no date filter, so the date cut happens here. Newest +// first means the first pull request older than the cutoff ends the walk. +func listPulls(tgt target, token, state string, limit int, since string) ([]listEntry, error) { + var kept []listEntry + + for page := 1; page <= maxPages; page++ { + query := url.Values{} + query.Set("state", state) + query.Set("sort", "created") + query.Set("direction", "desc") + query.Set("per_page", strconv.Itoa(perPage)) + query.Set("page", strconv.Itoa(page)) + + endpoint := fmt.Sprintf("%s/repos/%s/pulls?%s", tgt.APIBase, tgt.Repo, query.Encode()) + logf("listing %s pull requests for %s, page %d", state, tgt.Repo, page) + + var batch []listEntry + if err := getJSON(endpoint, token, &batch); err != nil { + return nil, err + } + + for _, entry := range batch { + if !openedOnOrAfter(entry.CreatedAt, since) { + logf("reached #%d, opened before the cutoff, so stopping", entry.Number) + return kept, nil + } + kept = append(kept, entry) + if limit > 0 && len(kept) >= limit { + return kept, nil + } + } + + if len(batch) < perPage { // the last page + return kept, nil + } + } + + logf("stopped at %d pages, so the dataset may be short", maxPages) + return kept, nil +} + +// openedOnOrAfter says whether a pull request was opened on or after the cutoff. +// An empty cutoff accepts everything. +func openedOnOrAfter(createdAt, since string) bool { + if since == "" { + return true + } + // created_at is RFC 3339, and the first ten characters are the date. Comparing + // two YYYY-MM-DD strings compares the dates, because the format sorts. + if len(createdAt) < 10 { + return true + } + return createdAt[:10] >= since +} + +// apiFile is the part of the files endpoint this binary reads. +type apiFile struct { + Filename string `json:"filename"` + PreviousFilename string `json:"previous_filename"` + Status string `json:"status"` + Additions int `json:"additions"` + Deletions int `json:"deletions"` + Patch string `json:"patch"` +} + +// pullFiles fetches the changed files for one pull request, with their patches. +// +// It returns the files and whether pagination cut the list short. +func pullFiles(tgt target, token string, number int) ([]changedFile, bool, error) { + var files []changedFile + + for page := 1; page <= maxPages; page++ { + query := url.Values{} + query.Set("per_page", strconv.Itoa(perPage)) + query.Set("page", strconv.Itoa(page)) + + endpoint := fmt.Sprintf( + "%s/repos/%s/pulls/%d/files?%s", tgt.APIBase, tgt.Repo, number, query.Encode(), + ) + + var batch []apiFile + if err := getJSON(endpoint, token, &batch); err != nil { + return nil, false, err + } + + for _, entry := range batch { + // A binary file, or one GitHub declined to diff, carries no patch. + patch := entry.Patch + truncated := len(patch) > maxPatchChars + if truncated { + patch = patch[:maxPatchChars] + } + + // A renamed file gets a previous path, and everything else gets null, so + // the field is a pointer and stays nil unless GitHub filled it. + var previous *string + if entry.PreviousFilename != "" { + name := entry.PreviousFilename + previous = &name + } + + files = append(files, changedFile{ + Path: entry.Filename, + PreviousPath: previous, + Status: entry.Status, + Additions: entry.Additions, + Deletions: entry.Deletions, + Patch: patch, + PatchTruncated: truncated, + HasPatch: entry.Patch != "", + }) + } + + if len(batch) < perPage { + return files, false, nil + } + } + + return files, true, nil +} + +// pullDetail carries the three counts the list endpoint leaves out. +type pullDetail struct { + ChangedFiles int `json:"changed_files"` + Additions int `json:"additions"` + Deletions int `json:"deletions"` +} + +// pullRecord turns one list entry into a dataset record, fetching what it lacks. +// +// The list endpoint carries no line counts, so this reads the pull request itself +// for changed_files, additions and deletions, then its files. +func pullRecord(tgt target, token string, entry listEntry) (pullRequest, error) { + logf("fetching #%d: %s", entry.Number, trimTo(entry.Title, 60)) + + var detail pullDetail + endpoint := fmt.Sprintf("%s/repos/%s/pulls/%d", tgt.APIBase, tgt.Repo, entry.Number) + if err := getJSON(endpoint, token, &detail); err != nil { + return pullRequest{}, err + } + + files, truncated, err := pullFiles(tgt, token, entry.Number) + if err != nil { + return pullRequest{}, err + } + + // A JSON array with no elements decodes to a nil slice, which marshals back as + // null rather than []. Building the slice with make keeps the dataset shaped + // like the Python one even for a pull request with no labels. + labels := make([]string, 0, len(entry.Labels)) + for _, label := range entry.Labels { + labels = append(labels, label.Name) + } + + return pullRequest{ + Number: entry.Number, + Title: entry.Title, + Body: entry.Body, + Author: entry.User.Login, + CreatedAt: entry.CreatedAt, + UpdatedAt: entry.UpdatedAt, + Draft: entry.Draft, + Labels: labels, + Base: entry.Base.Ref, + URL: entry.HTMLURL, + ChangedFiles: detail.ChangedFiles, + Additions: detail.Additions, + Deletions: detail.Deletions, + Files: files, + FilesTruncated: truncated, + }, nil +} + +// buildDataset fetches pull requests and returns the dataset. +// +// limit of 0 means no limit, and an empty since means no date cutoff, which is +// how the flags express "everything the other selector allows". +func buildDataset(tgt target, token, state string, limit int, since string) (dataset, error) { + entries, err := listPulls(tgt, token, state, limit, since) + if err != nil { + return dataset{}, err + } + logf("selected %d pull requests, now reading each one", len(entries)) + + records := make([]pullRequest, 0, len(entries)) + for _, entry := range entries { + record, err := pullRecord(tgt, token, entry) + if err != nil { + return dataset{}, err + } + records = append(records, record) + } + + sel := selector{State: state} + if limit > 0 { + kept := limit + sel.Limit = &kept + } + if since != "" { + cutoff := since + sel.Since = &cutoff + } + + return dataset{ + Repo: tgt.Repo, + FetchedAt: time.Now().UTC().Format(time.RFC3339), + Selector: sel, + PullRequests: records, + }, nil +} + +// selectorSlug names the selector, so two datasets for one repository sit in +// separate files. +func selectorSlug(state string, limit int, since string) string { + switch { + case since != "": + return fmt.Sprintf("%s-since-%s", state, since) + case limit <= 0: + return state + "-all" + default: + return fmt.Sprintf("%s-latest-%d", state, limit) + } +} + +// repoSlug turns `owner/repo` into a filename fragment. +func repoSlug(repo string) string { + return strings.Trim(slugPattern.ReplaceAllString(strings.ToLower(repo), "-"), "-") +} + +// writeJSON writes a value as indented JSON with a trailing newline, which is +// what the Python sample writes and what git wants at the end of a file. +func writeJSON(path string, value any) error { + body, err := json.MarshalIndent(value, "", " ") + if err != nil { + return fmt.Errorf("encoding %s: %w", path, err) + } + return os.WriteFile(path, append(body, '\n'), 0o644) +} + +// loadDataset reads a dataset written by either tool. +func loadDataset(path string) (dataset, error) { + raw, err := os.ReadFile(path) + if err != nil { + return dataset{}, fmt.Errorf("reading %s: %w", path, err) + } + + var data dataset + if err := json.Unmarshal(raw, &data); err != nil { + return dataset{}, fmt.Errorf("parsing %s: %w", path, err) + } + if len(data.PullRequests) == 0 { + return dataset{}, fmt.Errorf("%s holds no pull requests", path) + } + return data, nil +} + +// trimTo cuts a string to at most n characters, for a log line that should not +// wrap. +func trimTo(text string, n int) string { + if len(text) <= n { + return text + } + return text[:n] +} diff --git a/samples/pr-triage/go/go.mod b/samples/pr-triage/go/go.mod new file mode 100644 index 0000000..140d908 --- /dev/null +++ b/samples/pr-triage/go/go.mod @@ -0,0 +1,5 @@ +module github.com/aarora79/jev-samples/samples/pr-triage/go + +go 1.25 + +require gopkg.in/yaml.v3 v3.0.1 diff --git a/samples/pr-triage/go/go.sum b/samples/pr-triage/go/go.sum new file mode 100644 index 0000000..a62c313 --- /dev/null +++ b/samples/pr-triage/go/go.sum @@ -0,0 +1,4 @@ +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/samples/pr-triage/go/install.sh b/samples/pr-triage/go/install.sh new file mode 100755 index 0000000..0b6cae5 --- /dev/null +++ b/samples/pr-triage/go/install.sh @@ -0,0 +1,112 @@ +#!/bin/sh +# Install pr-triage: one static binary, no Go and no Python needed. +# +# curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh | sh +# +# Environment: +# VERSION semver release to install, for example 0.1.0, default the newest +# BINDIR where to put the binary, default /usr/local/bin then ~/.local/bin +# +# Piping a script from the internet into a shell is a choice. To read it first: +# curl -fsSLO https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh +# less install.sh && sh install.sh +set -eu + +REPO="aarora79/jev-samples" +TAG_PREFIX="pr-triage/" +BINARY="pr-triage" + +fail() { + echo "install: $1" >&2 + exit 1 +} + +# resolve_version reads the newest tag carrying the tool's prefix. +resolve_version() { + curl -fsSL "https://api.github.com/repos/$REPO/releases" | + grep '"tag_name":' | + sed -e 's/.*"tag_name": *"//' -e 's/".*//' | + grep "^$TAG_PREFIX" | + head -n 1 | + sed -e "s|^$TAG_PREFIX||" +} + +platform() { + os=$(uname -s | tr '[:upper:]' '[:lower:]') + case "$os" in + linux | darwin) ;; + *) fail "no prebuilt binary for $os. Build it with go build, or see go/README.md" ;; + esac + + arch=$(uname -m) + case "$arch" in + x86_64 | amd64) arch="amd64" ;; + aarch64 | arm64) arch="arm64" ;; + *) fail "no prebuilt binary for $arch. Build it with go build, or see go/README.md" ;; + esac + + echo "${os}_${arch}" +} + +checksum() { + if command -v sha256sum >/dev/null 2>&1; then + sha256sum "$1" | cut -d' ' -f1 + elif command -v shasum >/dev/null 2>&1; then + shasum -a 256 "$1" | cut -d' ' -f1 + else + echo "" + fi +} + +command -v curl >/dev/null 2>&1 || fail "curl is required" + +version="${VERSION:-}" +[ -n "$version" ] || version=$(resolve_version) +[ -n "$version" ] || fail "no $TAG_PREFIX release found on $REPO" +# Releases are plain semver, so drop a leading v if someone typed one. +version="${version#v}" + +target=$(platform) +asset="${BINARY}_${version}_${target}" +base="https://github.com/$REPO/releases/download/${TAG_PREFIX}${version}" + +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +echo "downloading $asset ($version)" +curl -fsSL "$base/$asset" -o "$tmp/$BINARY" || fail "could not download $base/$asset" + +if curl -fsSL "$base/SHA256SUMS" -o "$tmp/SHA256SUMS" 2>/dev/null; then + want=$(grep "$asset\$" "$tmp/SHA256SUMS" | cut -d' ' -f1) + got=$(checksum "$tmp/$BINARY") + if [ -z "$got" ]; then + echo "install: no sha256 tool found, skipping the checksum" >&2 + elif [ "$want" != "$got" ]; then + fail "checksum mismatch for $asset: expected $want, got $got" + else + echo "checksum ok" + fi +else + echo "install: no SHA256SUMS published for $version, skipping the checksum" >&2 +fi + +bindir="${BINDIR:-}" +if [ -z "$bindir" ]; then + if [ -w /usr/local/bin ]; then + bindir="/usr/local/bin" + else + bindir="$HOME/.local/bin" + fi +fi +mkdir -p "$bindir" + +install -m 0755 "$tmp/$BINARY" "$bindir/$BINARY" 2>/dev/null || + { cp "$tmp/$BINARY" "$bindir/$BINARY" && chmod 0755 "$bindir/$BINARY"; } + +echo "installed $bindir/$BINARY" +"$bindir/$BINARY" -version + +case ":$PATH:" in + *":$bindir:"*) ;; + *) echo "note: $bindir is not on your PATH" >&2 ;; +esac diff --git a/samples/pr-triage/go/main.go b/samples/pr-triage/go/main.go new file mode 100644 index 0000000..a8f8258 --- /dev/null +++ b/samples/pr-triage/go/main.go @@ -0,0 +1,335 @@ +// pr-triage: sort a repository's pull requests into review tiers, one Jev call +// each. +// +// One binary does both halves. It fetches the pull requests from the GitHub API, +// which can be github.com or a GitHub Enterprise Server host, then asks Jev +// eleven questions about each one and prints the triage. Every input arrives as a +// flag or an environment variable, so a skill or a CI job can hand it everything +// it needs and no config file has to sit next to the binary. +// +// The Python sample one directory up is the canonical implementation. This port +// exists so a machine with no Python can run the same check from one file. +package main + +import ( + "flag" + "fmt" + "os" + "path/filepath" + "strconv" + "strings" + "time" +) + +// Exit codes, so a caller can tell the outcomes apart without parsing text. +const ( + // exitOK means the run finished and nothing tripped a gate. + exitOK = 0 + // exitError means the run could not finish: a bad flag, no key, an API that + // refused, a dataset that would not parse. + exitError = 1 + // exitGate means the run finished and a gate fired, so a CI job should fail. + exitGate = 2 +) + +// jevKeyEnv is where the environment carries the TypeSafe key. +const jevKeyEnv = "TYPESAFE_API_KEY" + +// version is the release this binary was built from. build.sh sets it with +// -ldflags "-X main.version=..." and a plain `go build` leaves it as dev, so +// -version always answers. +var version = "dev" + +// quiet silences the progress lines when -quiet is set. A skill that parses this +// binary's output wants the report and nothing else. +var quiet bool + +// options holds everything the command line and the environment supply. +type options struct { + // target is `owner/repo` or a URL naming one. Empty when -dataset is set. + target string + // dataset replays a dataset file instead of fetching, which needs no GitHub + // token and costs only the Jev calls. + dataset string + // state picks which pull requests to list: open, closed or all. + state string + // limit keeps the most recent N. 0 means no limit. + limit int + // all clears the limit, which reads better in a command line than -limit 0. + all bool + // since keeps pull requests opened on or after a date, given as YYYY-MM-DD or + // as a number of days to go back. + since string + // out is the directory the dataset and the reports land in. + out string + // apiBase overrides the GitHub API root, which is how this talks to a GitHub + // Enterprise Server host. + apiBase string + // token is a GitHub token, for callers that would rather pass it than export + // it. + token string + // jevKey is the TypeSafe key, for the same reason. + jevKey string + // questions points at a payload file to use instead of the baked-in copy. + questions string + // explain prints every question and contribution for one pull request. + explain int + // failUnder exits 2 when any pull request lands in a tier at or above this + // one, so a CI job can fail on a queue that needs attention. + failOnTier string + // fetchOnly stops after writing the dataset, so the GitHub half can run + // somewhere with no TypeSafe key. + fetchOnly bool + // quiet silences progress lines. + quiet bool + // showVersion prints the release and exits, which install.sh calls to confirm + // the download landed. + showVersion bool +} + +// main parses the flags and hands off, so the control flow stays readable and +// every step that can fail returns an error to one place. +func main() { + opts, err := parseFlags() + if err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(exitError) + } + quiet = opts.quiet + + code, err := run(opts) + if err != nil { + fmt.Fprintln(os.Stderr, err) + // A gate that fired returns its own code alongside the reason it fired, and + // that code has to survive. Only a failure with no code of its own becomes + // the generic error exit. + if code == exitOK { + code = exitError + } + } + os.Exit(code) +} + +// run does the work and returns the exit code. +// +// Keeping the body here rather than in main means every failure travels back as +// an error and one place decides what to print and which code to leave with. +func run(opts options) (int, error) { + set, specs, err := loadPayload(opts.questions) + if err != nil { + return exitError, err + } + + data, err := gatherDataset(opts) + if err != nil { + return exitError, err + } + + if opts.fetchOnly { + return exitOK, nil + } + + key := opts.jevKey + if strings.TrimSpace(key) == "" { + key = strings.TrimSpace(os.Getenv(jevKeyEnv)) + } + if key == "" { + return exitError, fmt.Errorf( + "no TypeSafe key: pass -jev-key or set %s", jevKeyEnv, + ) + } + + return triage(opts, data, set, specs, key) +} + +// gatherDataset either replays a dataset file or fetches a fresh one. +// +// A fetch writes the dataset to disk before any Jev call, so a run that dies +// halfway leaves the expensive half of the work behind for a replay. +func gatherDataset(opts options) (dataset, error) { + if opts.dataset != "" { + logf("replaying %s", opts.dataset) + return loadDataset(opts.dataset) + } + + tgt, err := parseTarget(opts.target, opts.apiBase) + if err != nil { + return dataset{}, err + } + logf("repository %s through %s", tgt.Repo, tgt.APIBase) + + since, err := parseSince(opts.since) + if err != nil { + return dataset{}, err + } + + token := resolveToken(opts.token) + data, err := buildDataset(tgt, token, opts.state, opts.limit, since) + if err != nil { + return dataset{}, err + } + if len(data.PullRequests) == 0 { + return dataset{}, fmt.Errorf( + "no %s pull requests matched, so there is nothing to triage", opts.state, + ) + } + + if err := os.MkdirAll(opts.out, 0o755); err != nil { + return dataset{}, fmt.Errorf("making %s: %w", opts.out, err) + } + path := filepath.Join( + opts.out, + fmt.Sprintf("%s-%s.json", repoSlug(tgt.Repo), selectorSlug(opts.state, opts.limit, since)), + ) + if err := writeJSON(path, data); err != nil { + return dataset{}, err + } + logf("wrote %d pull requests to %s", len(data.PullRequests), path) + return data, nil +} + +// parseSince reads an ISO date, or a plain day count to go back from today. +// +// Both forms return YYYY-MM-DD, because that is what the date comparison in +// fetch.go works on and what the dataset records. +func parseSince(value string) (string, error) { + trimmed := strings.TrimSpace(value) + if trimmed == "" { + return "", nil + } + + // A plain number means days. strconv.Atoi fails on anything else, which is how + // the two forms are told apart. + if days, err := strconv.Atoi(trimmed); err == nil { + if days < 0 { + return "", fmt.Errorf("-since wants a positive number of days, got %q", value) + } + // UTC, because the created_at timestamps this gets compared against are + // UTC. A local midnight would move the cutoff by a day either side. + cutoff := time.Now().UTC().AddDate(0, 0, -days) + return cutoff.Format("2006-01-02"), nil + } + + // Go names date layouts by example, and 2006-01-02 is its way of writing + // YYYY-MM-DD. Parse fails on a date that does not exist, such as 2026-02-30. + parsed, err := time.Parse("2006-01-02", trimmed) + if err != nil { + return "", fmt.Errorf("-since wants YYYY-MM-DD or a number of days, got %q", value) + } + return parsed.Format("2006-01-02"), nil +} + +// parseFlags reads the command line and returns the options, or an error the +// caller prints. +func parseFlags() (options, error) { + var opts options + + flag.StringVar(&opts.dataset, "dataset", "", "triage this dataset file instead of fetching") + flag.StringVar(&opts.state, "state", "open", "which pull requests to list: open, closed or all") + flag.IntVar(&opts.limit, "limit", defaultLimit, "keep the most recent N pull requests") + flag.BoolVar(&opts.all, "all", false, "keep every pull request the other selectors allow") + flag.StringVar(&opts.since, "since", "", "keep pull requests opened on or after YYYY-MM-DD, or a number of days back") + flag.StringVar(&opts.out, "out", "data", "directory for the dataset and the reports") + flag.StringVar(&opts.apiBase, "api-base", "", "GitHub API root, for a GitHub Enterprise Server host") + flag.StringVar(&opts.token, "token", "", "GitHub token, otherwise GITHUB_TOKEN, GH_TOKEN, GH_ENTERPRISE_TOKEN or the gh CLI") + flag.StringVar(&opts.jevKey, "jev-key", "", "TypeSafe key, otherwise "+jevKeyEnv) + flag.StringVar(&opts.questions, "questions", "", "payload file to use instead of the copy baked into this binary") + flag.IntVar(&opts.explain, "explain", 0, "print every question and contribution for one pull request number") + flag.StringVar(&opts.failOnTier, "fail-on-tier", "", "exit 2 when any pull request lands in this tier or above: low, medium or high") + flag.BoolVar(&opts.fetchOnly, "fetch-only", false, "write the dataset and stop, which needs no TypeSafe key") + flag.BoolVar(&opts.quiet, "quiet", false, "print the report without the progress lines") + flag.BoolVar(&opts.showVersion, "version", false, "print the version and exit") + + flag.Usage = printUsage + flag.Parse() + + // Go's flag package stops parsing at the first argument that is not a flag, so + // `pr-triage apache/airflow -all` would otherwise leave -all sitting unread as + // a positional argument. Taking the repository and parsing what follows it, for + // as long as anything is left, lets the flags sit on either side of it, which is + // the order people type. + if opts.showVersion { + fmt.Printf("pr-triage %s\n", version) + os.Exit(exitOK) + } + + args := flag.Args() + for len(args) > 0 { + if opts.target != "" { + return opts, fmt.Errorf("expected one repository, got %q as well", args[0]) + } + opts.target = args[0] + if err := flag.CommandLine.Parse(args[1:]); err != nil { + return opts, err + } + args = flag.Args() + } + + // -all is sugar for no limit, and it wins over -limit so the two can appear + // together without the reader having to know the order. + if opts.all { + opts.limit = 0 + } + + switch opts.state { + case "open", "closed", "all": + default: + return opts, fmt.Errorf("-state wants open, closed or all, got %q", opts.state) + } + + if opts.failOnTier != "" && tierIndex(opts.failOnTier) < 0 { + return opts, fmt.Errorf("-fail-on-tier wants trivial, low, medium or high, got %q", opts.failOnTier) + } + + switch { + case opts.dataset != "" && opts.target != "": + return opts, fmt.Errorf("pass a repository or -dataset, not both") + case opts.dataset == "" && opts.target == "": + return opts, fmt.Errorf("name a repository as owner/repo or a URL, or pass -dataset") + } + + return opts, nil +} + +// printUsage prints the help. flag's default listing covers the flags, and the lines +// here cover what a reader cannot guess: the shapes of a command and where the +// credentials come from. +func printUsage() { + fmt.Fprint(os.Stderr, `pr-triage: sort a repository's pull requests into review tiers, one Jev call each. + +Usage: + pr-triage [flags] owner/repo + pr-triage [flags] https://github.com/owner/repo + pr-triage [flags] https://ghe.example.com/owner/repo + pr-triage -dataset data/owner-repo-open-all.json + +Examples: + pr-triage apache/airflow # the 10 most recent open ones + pr-triage apache/airflow -all # every open one + pr-triage apache/airflow -since 30 # opened in the last 30 days + pr-triage apache/airflow -since 2026-08-01 # opened on or after a date + pr-triage apache/airflow -explain 73713 # one pull request in full + pr-triage apache/airflow -fetch-only # build the dataset, skip Jev + pr-triage apache/airflow -fail-on-tier high # exit 2 if anything lands high + +Credentials, each read from the flag first and the environment second: + GitHub -token GITHUB_TOKEN, GH_TOKEN, GH_ENTERPRISE_TOKEN, or the gh CLI + TypeSafe -jev-key TYPESAFE_API_KEY + +GitHub Enterprise Server: + Pass a URL on your host and the API base follows it, or set it outright with + -api-base https://ghe.example.com/api/v3, or export GITHUB_API_URL. + +Flags: +`) + flag.PrintDefaults() +} + +// logf prints one progress line to standard error, so the report on standard +// output stays clean enough to pipe. +func logf(format string, args ...any) { + if quiet { + return + } + fmt.Fprintf(os.Stderr, format+"\n", args...) +} diff --git a/samples/pr-triage/go/output.go b/samples/pr-triage/go/output.go new file mode 100644 index 0000000..1207da0 --- /dev/null +++ b/samples/pr-triage/go/output.go @@ -0,0 +1,346 @@ +// The triage run and everything it prints: the table, the groups, the totals and +// the per-question detail. +// +// The padded tables are valid markdown, so the same text reads in a terminal and +// pastes into a pull request. +package main + +import ( + "fmt" + "path/filepath" + "sort" + "strings" +) + +// triageHeader names the columns of the one-row-per-pull-request table. +var triageHeader = []string{ + "PR", "Files", "Lines", "Kind", "Load", "Tier", "Review focus", "Title", +} + +// maxTitleChars cuts a title so the table stays inside a terminal. +const maxTitleChars = 58 + +// lowCoverageBelow is the coverage that earns a line saying how much of the diff +// Jev read. Above it, the reading covers enough of the change to stand on its own. +const lowCoverageBelow = 0.70 + +// triage asks Jev about every pull request in the dataset, prints the triage, and +// writes the two reports. +// +// It returns the exit code, so -fail-on-tier can turn a queue that needs attention +// into a failed CI job. +func triage(opts options, data dataset, set settings, specs []spec, key string) (int, error) { + results := make([]result, 0, len(data.PullRequests)) + for _, pull := range data.PullRequests { + r, err := ask(key, set, specs, data.Repo, pull) + if err != nil { + return exitError, err + } + logf("#%d: load %.2f, tier %s, %d ms", pull.Number, r.Load, r.Tier, r.LatencyMS) + results = append(results, r) + } + + fmt.Printf("\n## Triage: %s, %d pull requests\n\n", data.Repo, len(results)) + printPadded(triageHeader, triageRows(results)) + fmt.Printf( + "\nLoad is the weighted average of nine questions, 0 to 1. Tier comes from that load, "+ + "raised when size demands it: over %d files or %d lines cannot be trivial, over %d "+ + "files or %d lines is high whatever Jev returned.\n", + sizeFloors[len(sizeFloors)-1].files, sizeFloors[len(sizeFloors)-1].lines, + sizeFloors[0].files, sizeFloors[0].lines, + ) + printGroups(results) + + if opts.explain != 0 { + printDetail(results, specs, opts.explain) + } + + printTotals(results, specs, set) + + jsonPath, mdPath, err := writeReports(opts, data, results, specs, set) + if err != nil { + return exitError, err + } + fmt.Printf("Report: %s\nMarkdown: %s\n", jsonPath, mdPath) + + return gate(opts, results) +} + +// gate turns the results into an exit code, so a pull request job can fail on a +// queue holding work it wants flagged. +func gate(opts options, results []result) (int, error) { + if opts.failOnTier == "" { + return exitOK, nil + } + + bar := tierIndex(opts.failOnTier) + for _, r := range byLoad(results) { + if tierIndex(r.Tier) >= bar { + return exitGate, fmt.Errorf( + "#%d lands in %s at load %.2f, at or above the %s bar", + r.Pull.Number, r.Tier, r.Load, opts.failOnTier, + ) + } + } + return exitOK, nil +} + +// byLoad returns the results heaviest first, without disturbing the caller's +// slice. +func byLoad(results []result) []result { + ordered := make([]result, len(results)) + copy(ordered, results) + sort.SliceStable(ordered, func(i, j int) bool { return ordered[i].Load > ordered[j].Load }) + return ordered +} + +// tierLabel names the tier, marking the two cases a reader should not take at +// face value: a tier the size floor raised, and a load sitting on a cut. +func tierLabel(r result) string { + switch { + case r.RaisedBySize: + return r.Tier + " (size)" + case nearACut(r.Load): + return r.Tier + " (on a cut)" + default: + return r.Tier + } +} + +// triageRows builds one row per pull request, heaviest first. +func triageRows(results []result) [][]string { + rows := make([][]string, 0, len(results)) + for _, r := range byLoad(results) { + rows = append(rows, []string{ + fmt.Sprintf("#%d", r.Pull.Number), + fmt.Sprintf("%d", r.Pull.ChangedFiles), + fmt.Sprintf("+%d/-%d", r.Pull.Additions, r.Pull.Deletions), + r.Answers["change_kind"].Choice, + fmt.Sprintf("%.2f", r.Load), + tierLabel(r), + r.Answers["review_focus"].Choice, + trimTitle(r.Pull.Title), + }) + } + return rows +} + +// trimTitle cuts a title to the column width, ending in an ellipsis so a reader +// can tell a cut title from a short one. +func trimTitle(title string) string { + if len(title) <= maxTitleChars { + return title + } + return title[:maxTitleChars-3] + "..." +} + +// paddedLines builds one table padded to its widest cell per column. +// +// The padding makes it readable in a terminal, and the pipes keep it valid +// markdown, so the same lines go to the screen and into the markdown report. +func paddedLines(header []string, rows [][]string) []string { + widths := make([]int, len(header)) + for index, cell := range header { + widths[index] = len(cell) + } + for _, row := range rows { + for index, cell := range row { + if index < len(widths) && len(cell) > widths[index] { + widths[index] = len(cell) + } + } + } + + line := func(cells []string) string { + padded := make([]string, len(cells)) + for index, cell := range cells { + padded[index] = cell + strings.Repeat(" ", widths[index]-len(cell)) + } + return "| " + strings.Join(padded, " | ") + " |" + } + + rule := make([]string, len(widths)) + for index, width := range widths { + rule[index] = strings.Repeat("-", width) + } + + out := []string{line(header), "| " + strings.Join(rule, " | ") + " |"} + for _, row := range rows { + out = append(out, line(row)) + } + return out +} + +// printPadded prints one padded table. +func printPadded(header []string, rows [][]string) { + for _, text := range paddedLines(header, rows) { + fmt.Println(text) + } +} + +// printGroups prints the pull requests grouped by tier, with the advice for each +// tier and the questions that drove each load. +func printGroups(results []result) { + // Heaviest tier first, because that is the work a reader has to plan for. + for index := len(tiers) - 1; index >= 0; index-- { + tier := tiers[index] + members := make([]result, 0, len(results)) + for _, r := range byLoad(results) { + if r.Tier == tier { + members = append(members, r) + } + } + if len(members) == 0 { + continue + } + + fmt.Printf("\n%s (%d) -> %s\n", strings.ToUpper(tier), len(members), tierAdvice[tier]) + for _, r := range members { + note := "" + if r.RaisedBySize { + note = " [size floor]" + } + fmt.Printf(" #%-6d load %.2f%s %s\n", r.Pull.Number, r.Load, note, r.Pull.Title) + fmt.Printf(" %s\n", driverText(r)) + if r.Coverage < lowCoverageBelow { + fmt.Printf( + " read from %.0f%% of the changed files, so the load is a read on part of the diff\n", + r.Coverage*100, + ) + } + fmt.Printf(" %s\n", r.Pull.URL) + } + } +} + +// driverText names the drivers, or says that nothing cleared the floor. +func driverText(r result) string { + if len(r.Drivers) == 0 { + return "nothing above the driver floor" + } + return strings.Join(r.Drivers, ", ") +} + +// printDetail prints every question for one pull request, and what each one +// contributed. +func printDetail(results []result, specs []spec, number int) { + var found *result + for index := range results { + if results[index].Pull.Number == number { + found = &results[index] + break + } + } + if found == nil { + fmt.Printf("\n#%d is not in this dataset, so there is nothing to explain.\n", number) + return + } + + total := weightedTotal(specs) + header := []string{"Key", "Label", "Type", "Jev returned", "Weight", "Credit", "Adds to load"} + rows := make([][]string, 0, len(specs)) + for _, s := range specs { + a := found.Answers[s.ID] + weight, creditCell, adds := "", "", "" + if s.Q.Weight > 0 { + weight = fmt.Sprintf("%.2f", s.Q.Weight) + creditCell = fmt.Sprintf("%.2f", credit(s, a)) + adds = fmt.Sprintf("%.4f", contribution(s, a, total)) + } else { + adds = "routes the read" + } + label := s.Q.Label + if s.Q.Invert { + label += " (inverted)" + } + rows = append(rows, []string{"`" + s.ID + "`", label, s.Q.Type, returned(a), weight, creditCell, adds}) + } + + fmt.Printf("\n### #%d in full: %s\n\n", found.Pull.Number, found.Pull.Title) + printPadded(header, rows) + fmt.Printf( + "\n**Load %.4f**, the weighted average of the credit column over %.2f of weight. Tier %s.\n", + found.Load, total, tierLabel(*found), + ) +} + +// returned says what Jev sent back for one question, in that question's own +// terms. +func returned(a answer) string { + switch a.Type { + case "noul": + return fmt.Sprintf("%.2f", a.Noul) + case "score": + return fmt.Sprintf("%.2f / %d, confidence %.2f", a.Score, len(a.Legend)-1, a.Confidence) + default: + return fmt.Sprintf("%s, confidence %.2f", a.Choice, a.Confidence) + } +} + +// printTotals prints what the run cost and how long it took. +func printTotals(results []result, specs []spec, set settings) { + tokens, latency, thin := 0, int64(0), 0 + for _, r := range results { + tokens += r.Usage.InputTokens + latency += r.LatencyMS + if r.Coverage < lowCoverageBelow { + thin++ + } + } + calls := len(results) + + fmt.Printf( + "\n%d pull requests, %d questions each, one call apiece. %s input tokens, %s ms total, "+ + "%d ms per call on average, $%.5f at $%v per million input tokens.\n", + calls, len(specs), commas(tokens), commas(int(latency)), + latency/int64(max(calls, 1)), costUSD(results, set), set.InputUSDPerMillion, + ) + if thin > 0 { + fmt.Printf( + "%d of %d had patches too large to send whole, so Jev read part of the diff and the "+ + "paths of the rest. Those are the ones the size floor guards.\n", + thin, calls, + ) + } +} + +// commas groups a number's digits, because a token count reads better as 183,426 +// than as 183426. Go's fmt has no thousands flag. +func commas(value int) string { + text := fmt.Sprintf("%d", value) + if len(text) <= 3 { + return text + } + + var out strings.Builder + lead := len(text) % 3 + if lead > 0 { + out.WriteString(text[:lead]) + } + for index := lead; index < len(text); index += 3 { + if out.Len() > 0 { + out.WriteString(",") + } + out.WriteString(text[index : index+3]) + } + return out.String() +} + +// reportStem names both reports after the repository and the selector, matching +// what the Python sample writes. +func reportStem(opts options, data dataset) string { + state := data.Selector.State + if state == "" { + state = "open" + } + limit := 0 + if data.Selector.Limit != nil { + limit = *data.Selector.Limit + } + since := "" + if data.Selector.Since != nil { + since = *data.Selector.Since + } + name := fmt.Sprintf("%s-%s-triage", repoSlug(data.Repo), selectorSlug(state, limit, since)) + return filepath.Join(opts.out, name) +} diff --git a/samples/pr-triage/go/payload.go b/samples/pr-triage/go/payload.go new file mode 100644 index 0000000..2fe857f --- /dev/null +++ b/samples/pr-triage/go/payload.go @@ -0,0 +1,248 @@ +// Payload loading: the questions, the weights, the model pin, the budgets and +// the price. +// +// The binary bakes in questions.yml, so it runs with no config file beside it. +// Point -questions at your own copy to triage against different questions or +// weights, and the baked copy stays as the default. +// +// The Python sample one directory up owns the canonical payload, at +// ../questions.yml. Go's embed directive reads files inside its own directory +// only, so it cannot reach up a level, and the questions.yml sitting beside this +// file is a copy. build.sh copies the canonical file in before every build, and +// payload_test.go fails when the two drift apart. +package main + +import ( + "embed" + "fmt" + "os" + + "gopkg.in/yaml.v3" +) + +// payloadFS carries questions.yml into the binary. +// +// The line below is a compiler directive, not a comment for a reader. The Go +// compiler acts on comments that begin with go: and this one copies the bytes of +// questions.yml into the executable, handing them to payloadFS, an embed.FS that +// behaves like a tiny read-only filesystem in memory. The directive must sit +// directly above the variable it fills, with no blank line between. +// +// That is what makes this sample one file to ship: the questions travel inside +// the binary, so a CI runner needs no checkout of this repo. +// +//go:embed questions.yml +var payloadFS embed.FS + +// payloadName is the path to read inside payloadFS, and the name build.sh and +// payload_test.go both use for the copied file. +const payloadName = "questions.yml" + +// question is one entry from the questions map in the payload. +// +// The strings in backticks after each field are struct tags, telling the YAML +// library which key fills which field, so max_description_chars can land in a Go +// field named MaxDescriptionChars. Go exports a field whose name starts with a +// capital letter, and the library reads exported fields only, hence the capitals. +// +// A field the file leaves out keeps its zero value: "" for a string, 0 for a +// number, false for a bool, nil for a map. +type question struct { + // Type picks how the binary asks and how it scores the answer: noul for a + // yes-or-no claim, choice for one option out of a named set, score for a + // rubric level. loadPayload rejects anything else. + Type string `yaml:"type"` + // Label is the short name the reports print. orderedQuestions falls back to + // the question id when the file gives no label. + Label string `yaml:"label"` + // Weight is this question's share of the review load. A question with no + // weight still gets asked and printed, and contributes nothing. change_kind + // and review_focus are the two that carry none. + Weight float64 `yaml:"weight"` + // Invert flips the meaning of a yes. mechanical, has_tests and + // description_quality set it, because a repeated diff, a tested change and a + // thorough description each lower the load rather than raising it. + Invert bool `yaml:"invert"` + // Instructions is the question text the binary sends to Jev. + Instructions string `yaml:"instructions"` + // Criteria holds the options a choice question picks from, or the rubric + // levels a score question climbs. The shape differs per type, a map for choice + // and a list for score, so the field takes any: Go's name for a value of any + // type at all. The binary hands it straight to the API. + Criteria any `yaml:"criteria"` + // Credit says what each choice option is worth toward the load, keyed by + // option name. A missing option reads as 0, a Go map's zero value for a + // float64. Only choice questions use it, and only when they carry a weight. + Credit map[string]float64 `yaml:"credit"` +} + +// settings holds everything in the payload except the questions. +// +// This sample sends a state with four named fields rather than one blob, so it +// carries one budget per field that can run long. A pull request description, a +// file list and a diff each grow without limit, and one budget shared between +// them would let a long diff crowd out the description. +type settings struct { + // Model is the pinned model id, jev-1.13.0. Both SDKs default to jev-latest, + // which would move answers under thresholds calibrated on an older version. + Model string `yaml:"model"` + // MaxDescriptionChars caps the author's description. + MaxDescriptionChars int `yaml:"max_description_chars"` + // MaxFileListChars caps the list of changed paths. Past the budget the list + // keeps whole paths and reports how many it dropped. + MaxFileListChars int `yaml:"max_file_list_chars"` + // MaxDiffChars caps the patch text. Past the budget the smallest patches go + // in whole first, because triage asks how far a change reaches and whether one + // edit repeats, and both of those want breadth over depth. + MaxDiffChars int `yaml:"max_diff_chars"` + // InputUSDPerMillion is TypeSafe's published input-token price, which the + // reports use to cost a run. Output tokens are counted and not charged. + InputUSDPerMillion float64 `yaml:"input_usd_per_million"` +} + +// spec pairs a question with its id, so the order in the file survives. +// +// Question order in questions.yml is print order, and a Go map hands its keys +// back in a random order every time it is walked. Carrying the id alongside the +// question keeps them in a slice, which does hold its order. +type spec struct { + // ID is the key from the file, such as blast_radius. + ID string + // Q is the question itself, embedded by value so a spec is self-contained. + Q question +} + +// wholePayload mirrors the file, keeping the questions as a node so the order of +// the keys survives decoding. +// +// The scalar fields decode straight into settings. Questions stops at a +// yaml.Node, the library's raw view of a piece of the document, which records +// the keys in the order the file wrote them. orderedQuestions walks that node. +type wholePayload struct { + Model string `yaml:"model"` + MaxDescriptionChars int `yaml:"max_description_chars"` + MaxFileListChars int `yaml:"max_file_list_chars"` + MaxDiffChars int `yaml:"max_diff_chars"` + InputUSDPerMillion float64 `yaml:"input_usd_per_million"` + Questions yaml.Node `yaml:"questions"` +} + +// loadPayload reads the payload from path, or from the baked copy when path is +// empty. It returns the settings and the questions in file order. +// +// Go functions can return several values at once, and this one returns three: the +// settings, the questions, and an error. A caller checks the error first, and the +// two values before it mean nothing when the error is set. +func loadPayload(path string) (settings, []spec, error) { + var raw []byte + var err error + // origin names the payload in every error message below, so a reader can tell + // a broken file they passed in from a broken baked-in copy. + origin := "the baked-in " + payloadName + + if path == "" { + raw, err = payloadFS.ReadFile(payloadName) + } else { + raw, err = os.ReadFile(path) + origin = path + } + // The Go idiom: a function returns an error alongside its result, and the + // caller tests err != nil right away. %w wraps the original error inside the + // new one, so the printed message keeps the operating system's reason and a + // caller further up can still inspect the cause. + if err != nil { + return settings{}, nil, fmt.Errorf("reading %s: %w", origin, err) + } + + // Unmarshal fills a struct from YAML bytes. &whole passes the address of the + // struct, a pointer, because the library writes into it rather than handing + // back a copy. Without the &, Unmarshal would fill a throwaway. + var whole wholePayload + if err := yaml.Unmarshal(raw, &whole); err != nil { + return settings{}, nil, fmt.Errorf("parsing %s: %w", origin, err) + } + + set := settings{ + Model: whole.Model, + MaxDescriptionChars: whole.MaxDescriptionChars, + MaxFileListChars: whole.MaxFileListChars, + MaxDiffChars: whole.MaxDiffChars, + InputUSDPerMillion: whole.InputUSDPerMillion, + } + // A switch with no value after it runs the first case whose condition holds, + // which reads better than five stacked ifs. Each check catches a key the file + // left out, since a missing key leaves the zero value behind: an empty model + // would send the request to jev-latest, a zero budget would truncate that part + // of the state to nothing, and a zero price would report every run as free. + switch { + case set.Model == "": + return set, nil, fmt.Errorf("%s: missing model", origin) + case set.MaxDescriptionChars <= 0: + return set, nil, fmt.Errorf("%s: missing max_description_chars", origin) + case set.MaxFileListChars <= 0: + return set, nil, fmt.Errorf("%s: missing max_file_list_chars", origin) + case set.MaxDiffChars <= 0: + return set, nil, fmt.Errorf("%s: missing max_diff_chars", origin) + case set.InputUSDPerMillion <= 0: + return set, nil, fmt.Errorf("%s: missing input_usd_per_million", origin) + } + + specs, err := orderedQuestions(whole.Questions, origin) + if err != nil { + return set, nil, err + } + return set, specs, nil +} + +// orderedQuestions walks the questions node and returns one spec per entry, in +// the order the file wrote them. +// +// A yaml.Node for a mapping keeps its children in one flat slice: key, value, +// key, value. The loop below steps two at a time, which is why it reads Content +// at index and index+1. +func orderedQuestions(node yaml.Node, origin string) ([]spec, error) { + if node.Kind != yaml.MappingNode { + return nil, fmt.Errorf("%s: questions is not a mapping", origin) + } + + specs := make([]spec, 0, len(node.Content)/2) + for index := 0; index+1 < len(node.Content); index += 2 { + id := node.Content[index].Value + + var q question + if err := node.Content[index+1].Decode(&q); err != nil { + return nil, fmt.Errorf("%s: question %s: %w", origin, id, err) + } + // A type the scorer cannot read would sail through to the API and come + // back as something this binary has no branch for, so reject it here where + // the message can name the question. + switch q.Type { + case "noul", "choice", "score": + default: + return nil, fmt.Errorf("%s: question %s has unknown type %q", origin, id, q.Type) + } + // The reports print the label, so fall back to the id rather than a blank + // column. + if q.Label == "" { + q.Label = id + } + specs = append(specs, spec{ID: id, Q: q}) + } + + if len(specs) == 0 { + return nil, fmt.Errorf("%s: no questions", origin) + } + return specs, nil +} + +// weightedTotal adds up every weight in the payload. +// +// The load divides by this rather than by a constant 1.0, so editing one weight +// needs no rebalancing of the others. +func weightedTotal(specs []spec) float64 { + total := 0.0 + for _, s := range specs { + total += s.Q.Weight + } + return total +} diff --git a/samples/pr-triage/go/payload_test.go b/samples/pr-triage/go/payload_test.go new file mode 100644 index 0000000..07d9f59 --- /dev/null +++ b/samples/pr-triage/go/payload_test.go @@ -0,0 +1,106 @@ +// The Python sample beside this folder is the canonical implementation, and its +// questions.yml is the canonical payload. go:embed cannot reach outside this +// directory, so the file here is a copy, and this test fails the moment the two +// diverge. +// +// A file ending in _test.go holds tests and stays out of the built binary. Run +// them with: +// +// go test ./... +// +// Each test is a function starting with Test that takes *testing.T, the handle it +// calls to report a failure. t.Fatalf prints a message and stops that test. +package main + +import ( + "bytes" + "os" + "testing" +) + +// canonical is the payload the Python sample reads, one directory up. +const canonical = "../questions.yml" + +// TestEmbeddedPayloadMatchesPython compares the copy compiled into the binary +// against the canonical file on disk. +// +// Without this test the two payloads drift in silence: someone edits a weight in +// the Python sample, the Go binary keeps triaging with the old one, and the two +// tools disagree about the same pull request for a reason nobody can see. The +// test turns that into a build failure with the fix in the message. +func TestEmbeddedPayloadMatchesPython(t *testing.T) { + want, err := os.ReadFile(canonical) + if err != nil { + t.Fatalf("reading %s: %v", canonical, err) + } + + // payloadFS is the embedded filesystem from payload.go. Reading from it reads + // bytes baked into this binary rather than anything on disk. + got, err := payloadFS.ReadFile(payloadName) + if err != nil { + t.Fatalf("reading the embedded %s: %v", payloadName, err) + } + + // Go cannot compare two byte slices with ==, so bytes.Equal does it. + if !bytes.Equal(want, got) { + t.Fatalf("%s and the embedded copy differ: run ./build.sh, which copies the canonical file in", canonical) + } +} + +// TestEmbeddedPayloadLoads parses the embedded payload and checks the fields the +// rest of the program depends on. A payload that parses but arrives empty would +// otherwise surface as a zero load or a division by zero at runtime. +func TestEmbeddedPayloadLoads(t *testing.T) { + // An empty path means "use the embedded copy", which is what the binary does + // when nobody passes -questions. + set, specs, err := loadPayload("") + if err != nil { + t.Fatalf("loading the embedded payload: %v", err) + } + + // %+v prints a struct with its field names, so a failure says which field came + // back empty. Every budget matters: a zero one truncates that part of the + // state to nothing and the load then reads a pull request that is not there. + switch { + case set.Model == "": + t.Fatalf("no model pinned: %+v", set) + case set.MaxDescriptionChars == 0 || set.MaxFileListChars == 0 || set.MaxDiffChars == 0: + t.Fatalf("a state budget came back zero: %+v", set) + case set.InputUSDPerMillion == 0: + t.Fatalf("no price, so every run would report as free: %+v", set) + } + + // The load divides by the total weight, so at least one question has to carry + // weight for the number to mean anything. + if weightedTotal(specs) <= 0 { + t.Fatal("no question carries a weight, so the load would divide by zero") + } +} + +// TestInvertedQuestionsCarryWeight guards the three questions whose meaning is +// flipped. An inverted question with no weight is a silent no-op: it gets asked, +// costs tokens, and changes nothing, which is the kind of mistake that survives +// review because the output still looks right. +func TestInvertedQuestionsCarryWeight(t *testing.T) { + _, specs, err := loadPayload("") + if err != nil { + t.Fatalf("loading the embedded payload: %v", err) + } + + inverted := 0 + for _, s := range specs { + if !s.Q.Invert { + continue + } + inverted++ + if s.Q.Weight <= 0 { + t.Errorf("question %s inverts but carries no weight, so it cannot lower a load", s.ID) + } + } + // mechanical, has_tests and description_quality are the three. A payload that + // loses one of them would still load and still triage, with the load quietly + // reading higher on every tested and repeated change. + if inverted != 3 { + t.Errorf("expected 3 inverted questions, found %d", inverted) + } +} diff --git a/samples/pr-triage/go/questions.yml b/samples/pr-triage/go/questions.yml new file mode 100644 index 0000000..9c7e081 --- /dev/null +++ b/samples/pr-triage/go/questions.yml @@ -0,0 +1,149 @@ +# Everything this sample sends to Jev, plus the weights it triages with: the +# model pin, the state budgets, and all eleven questions. +# +# Nine questions carry a `weight`, and review load is the weighted average of +# what they returned. Load runs 0 to 1, where 1 means a human should read every +# line. The weights here sum to 1, and the sample divides by their total anyway, +# so an edited weight needs no rebalancing. +# +# Weights say what a missed problem costs. Security surface and blast radius +# outrank everything, because a wrong merge there is an incident rather than a +# revert. Design decisions come next, because that is the part review exists for. +# A missing test and a thin description cost least: they slow a reviewer down +# without hiding a fault. +# +# `change_kind` and `review_focus` carry no weight. They route the review rather +# than sizing it. +# +# Adding a question means adding an entry here. Nothing else changes. + +# The SDKs default to jev-latest, and a silent upgrade would move answers under +# thresholds calibrated against an older version. +model: jev-1.13.0 + +# One budget per state field, so a 5,000-line lockfile diff cannot crowd out the +# description and a rambling description cannot crowd out the diff. These sum to +# about 8,000 tokens. Eleven questions each see the whole state plus themselves, +# which keeps every one of them inside the 32,000-token per-question limit. +max_description_chars: 4000 +max_file_list_chars: 4000 +max_diff_chars: 24000 + +# TypeSafe's published price for input tokens, September 2026. Output tokens are +# counted and not charged, and the reports use this to price each run. +input_usd_per_million: 0.042 + +# Each question asks about one thing, so no answer has to cover two claims. The +# order here is the order the sample prints them. +questions: + # No weight: this one names what kind of change it is, which reads next to the + # tier rather than feeding it. A docs change and a schema migration can carry + # the same load for different reasons. + change_kind: + type: choice + label: change kind + instructions: What kind of change this pull request makes + criteria: + docs: Documentation, comments, or README text + tests: Test files, fixtures, or test infrastructure + dependency: Dependency versions, lockfiles, or build tooling + config: Configuration, deployment manifests, or CI workflows + bugfix: A correction to behaviour that was wrong + feature: New behaviour that did not exist before + refactor: Rearranged code that behaves the same way afterward + + # No weight either: this one says where a reviewer should start. + review_focus: + type: choice + label: review focus + instructions: What a reviewer should check first on this pull request + criteria: + correctness: Whether the new logic does what it claims + security: Authentication, authorization, secrets, or input handling + compatibility: Whether existing callers and deployments keep working + tests: Whether the change is covered, and whether the tests assert anything + deployment: Whether this rolls out and rolls back safely + readability: Naming, structure, and whether the next reader follows it + + # Each score rubric runs 0 to 2 with evenly spaced steps, so dividing by the + # top level gives an honest fraction. + blast_radius: + type: score + label: blast radius + weight: 0.15 + instructions: How far the effects of this change reach through the codebase + criteria: + - Confined to one file, or to documentation + - Confined to one module or feature area + - Reaches shared code paths that many callers depend on + + design_decisions: + type: score + label: design decisions + weight: 0.14 + instructions: How many judgment calls a reviewer would have to agree with + criteria: + - None: the same edit repeated, or a value changed + - Local choices inside one function or file + - New abstractions, interfaces, or control flow + + # Inverted: a good description lowers the load, because the reviewer starts + # with the author's reasoning instead of reconstructing it from the diff. + description_quality: + type: score + label: description quality + weight: 0.08 + invert: true + instructions: How well the description accounts for the change + criteria: + - Empty, or says nothing about what changed + - Names what changed + - Explains what changed, why, and how the author verified it + + security_surface: + type: noul + label: security surface + weight: 0.15 + instructions: The change touches authentication, authorization, secrets, tokens, or input validation + + # Inverted: a mechanical diff is the cheapest kind to merge, because checking + # one instance checks them all. + # + # Asking whether the change is "simple" asks Jev to judge difficulty, which + # varies by reader. This asks about the shape of the diff instead, which the + # diff settles. + mechanical: + type: noul + label: mechanical + weight: 0.12 + invert: true + instructions: The diff repeats one kind of edit, with no separate decision per file + + breaking_change: + type: noul + label: breaking change + weight: 0.11 + instructions: The change alters a public API, config key, command-line flag, or stored data shape that existing callers depend on + + # A diff doing more than it says costs a reviewer their bearings, and it is the + # one problem the description cannot warn them about. + scope_creep: + type: noul + label: scope creep + weight: 0.09 + instructions: The diff changes things the title and description never mention + + infra_surface: + type: noul + label: infra surface + weight: 0.08 + instructions: The change touches deployment, infrastructure, container images, or CI configuration + + # Inverted: tests that exercise the change lower the load, because the reviewer + # can read what the author expected instead of guessing at it. + has_tests: + type: noul + label: has tests + weight: 0.08 + invert: true + instructions: The diff adds or updates tests that exercise the behaviour it changes diff --git a/samples/pr-triage/go/report.go b/samples/pr-triage/go/report.go new file mode 100644 index 0000000..c3c4e3d --- /dev/null +++ b/samples/pr-triage/go/report.go @@ -0,0 +1,255 @@ +// The two reports every run writes: a JSON one for a machine and a markdown one +// for a person. +// +// Both land beside the dataset, named after the repository and the selector, and +// both match what the Python sample writes so either tool can read either file. +package main + +import ( + "encoding/json" + "fmt" + "os" + "strings" + "time" +) + +// questionReport is one question's answer and what the binary made of it. +type questionReport struct { + Label string `json:"label"` + Type string `json:"type"` + Weight *float64 `json:"weight"` + Inverted bool `json:"inverted"` + // Returned is Jev's answer, decoded and re-encoded rather than passed through, + // so the report holds one shape whatever the API adds later. + Returned answer `json:"returned"` + // Credit and AddsToLoad are pointers so an unweighted question writes null + // rather than a 0 that reads like a real judgment. + Credit *float64 `json:"credit"` + AddsToLoad *float64 `json:"adds_to_load"` +} + +// pullReport is one pull request in the JSON report. +type pullReport struct { + Number int `json:"number"` + Title string `json:"title"` + URL string `json:"url"` + Author string `json:"author"` + ChangedFiles int `json:"changed_files"` + Additions int `json:"additions"` + Deletions int `json:"deletions"` + ReviewLoad float64 `json:"review_load"` + Tier string `json:"tier"` + SizeFloor string `json:"size_floor"` + RaisedBySize bool `json:"raised_by_size"` + DiffCoverage float64 `json:"diff_coverage"` + Drivers []string `json:"drivers"` + Usage usage `json:"usage"` + LatencyMS int64 `json:"latency_ms"` + Questions map[string]questionReport `json:"questions"` +} + +// triageReport is the whole JSON report. +type triageReport struct { + Repo string `json:"repo"` + Selector selector `json:"selector"` + TriagedAt string `json:"triaged_at"` + Model string `json:"model"` + QuestionsAsked int `json:"questions_asked"` + TierCounts map[string]int `json:"tier_counts"` + CostUSD float64 `json:"cost_usd"` + PullRequests []pullReport `json:"pull_requests"` +} + +// writeReports writes both files and returns their paths. +func writeReports( + opts options, + data dataset, + results []result, + specs []spec, + set settings, +) (string, string, error) { + if err := os.MkdirAll(opts.out, 0o755); err != nil { + return "", "", fmt.Errorf("making %s: %w", opts.out, err) + } + + stem := reportStem(opts, data) + jsonPath := stem + ".json" + mdPath := stem + ".md" + + if err := writeJSON(jsonPath, buildReport(data, results, specs, set)); err != nil { + return "", "", err + } + + body := strings.Join(markdownLines(data, results, specs, set), "\n") + "\n" + if err := os.WriteFile(mdPath, []byte(body), 0o644); err != nil { + return "", "", fmt.Errorf("writing %s: %w", mdPath, err) + } + return jsonPath, mdPath, nil +} + +// buildReport assembles the JSON report. +func buildReport(data dataset, results []result, specs []spec, set settings) triageReport { + total := weightedTotal(specs) + + counts := make(map[string]int, len(tiers)) + for _, tier := range tiers { + counts[tier] = 0 + } + for _, r := range results { + counts[r.Tier]++ + } + + pulls := make([]pullReport, 0, len(results)) + for _, r := range byLoad(results) { + questions := make(map[string]questionReport, len(specs)) + for _, s := range specs { + a := r.Answers[s.ID] + entry := questionReport{ + Label: s.Q.Label, + Type: s.Q.Type, + Inverted: s.Q.Invert, + Returned: a, + } + // Go has no way to take the address of a literal, so each value gets a + // variable first. A weighted question fills all three pointers, and an + // unweighted one leaves them nil, which encodes as null. + if s.Q.Weight > 0 { + weight := s.Q.Weight + creditValue := round4(credit(s, a)) + adds := round4(contribution(s, a, total)) + entry.Weight = &weight + entry.Credit = &creditValue + entry.AddsToLoad = &adds + } + questions[s.ID] = entry + } + + pulls = append(pulls, pullReport{ + Number: r.Pull.Number, + Title: r.Pull.Title, + URL: r.Pull.URL, + Author: r.Pull.Author, + ChangedFiles: r.Pull.ChangedFiles, + Additions: r.Pull.Additions, + Deletions: r.Pull.Deletions, + ReviewLoad: round4(r.Load), + Tier: r.Tier, + SizeFloor: sizeFloor(r.Pull.ChangedFiles, r.Pull.Additions+r.Pull.Deletions), + RaisedBySize: r.RaisedBySize, + DiffCoverage: round4(r.Coverage), + Drivers: r.Drivers, + Usage: r.Usage, + LatencyMS: r.LatencyMS, + Questions: questions, + }) + } + + return triageReport{ + Repo: data.Repo, + Selector: data.Selector, + TriagedAt: time.Now().UTC().Format(time.RFC3339), + Model: set.Model, + QuestionsAsked: len(specs), + TierCounts: counts, + CostUSD: round8(costUSD(results, set)), + PullRequests: pulls, + } +} + +// markdownLines builds the markdown report: the same table and groups the run +// printed, with links a reader can follow. +func markdownLines(data dataset, results []result, specs []spec, set settings) []string { + tokens := 0 + for _, r := range results { + tokens += r.Usage.InputTokens + } + + lines := []string{ + fmt.Sprintf("# Triage: %s", data.Repo), + "", + fmt.Sprintf( + "%d pull requests, %d questions each, one call apiece, on %s with `%s`.", + len(results), len(specs), time.Now().UTC().Format("2 January 2006"), set.Model, + ), + fmt.Sprintf( + "%s input tokens, $%.5f at $%v per million.", + commas(tokens), costUSD(results, set), set.InputUSDPerMillion, + ), + "", + } + lines = append(lines, paddedLines(triageHeader, triageRows(results))...) + lines = append(lines, + "", + fmt.Sprintf( + "Load is the weighted average of nine questions, 0 to 1. Tier comes from that load, "+ + "raised when size demands it: over %d files or %s lines is high whatever Jev returned.", + sizeFloors[0].files, commas(sizeFloors[0].lines), + ), + ) + return append(lines, markdownGroups(results)...) +} + +// markdownGroups builds one section per tier, heaviest tier first. +func markdownGroups(results []result) []string { + var lines []string + + for index := len(tiers) - 1; index >= 0; index-- { + tier := tiers[index] + members := make([]result, 0, len(results)) + for _, r := range byLoad(results) { + if r.Tier == tier { + members = append(members, r) + } + } + if len(members) == 0 { + continue + } + + lines = append(lines, + "", + fmt.Sprintf("## %s (%d)", tier, len(members)), + "", + tierAdvice[tier], + "", + ) + for _, r := range members { + note := "" + if r.RaisedBySize { + note = ", raised by the size floor" + } + lines = append(lines, fmt.Sprintf( + "- [#%d](%s) load %.2f%s: %s", + r.Pull.Number, r.Pull.URL, r.Load, note, r.Pull.Title, + )) + lines = append(lines, fmt.Sprintf(" - drivers: %s", driverText(r))) + if r.Coverage < lowCoverageBelow { + lines = append(lines, fmt.Sprintf( + " - read from %.0f%% of the changed files, so the load is a read on part of the diff", + r.Coverage*100, + )) + } + } + } + return lines +} + +// round4 and round8 trim a float for the report, so a stored number reads the way +// the tables print it rather than carrying seventeen digits of binary noise. +func round4(value float64) float64 { + return parseRounded(fmt.Sprintf("%.4f", value)) +} + +func round8(value float64) float64 { + return parseRounded(fmt.Sprintf("%.8f", value)) +} + +// parseRounded turns a formatted number back into a float. Formatting and +// reparsing is the shortest way to round to a fixed number of places in Go, and +// json.Unmarshal is already the tool for reading a number out of text. +func parseRounded(text string) float64 { + var value float64 + if err := json.Unmarshal([]byte(text), &value); err != nil { + return 0 + } + return value +} diff --git a/samples/pr-triage/go/score.go b/samples/pr-triage/go/score.go new file mode 100644 index 0000000..9060b30 --- /dev/null +++ b/samples/pr-triage/go/score.go @@ -0,0 +1,512 @@ +// The Jev half: ask eleven questions about one pull request, then judge the +// answers. +// +// Every threshold on a number lives here rather than in a question, because Jev +// cannot count. It reads a truncated diff and answers about what it read, and the +// file and line counts stay arithmetic that Go does. +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "sort" + "strings" + "time" +) + +const ( + // endpoint is the one Jev URL. One call carries every question. + endpoint = "https://api.typesafe.ai/v1/systemone" + // jevEndpointEnv lets a caller point at something else, which a proxy or a + // self-hosted reproduction needs. + jevEndpointEnv = "TYPESAFE_API_URL" + // requestTimeout caps one call. Jev answers in well under a second, so a + // minute means something else broke: DNS, a proxy, a hung connection. + requestTimeout = 60 * time.Second + // deadband is how close a load can sit to a tier cut before the tier prints as + // undecided. Repeat calls on one pull request move a load by about this much, + // so a value inside the band would otherwise flip between runs. + deadband = 0.02 + // driverFloor is the smallest share of the load worth naming as a driver. + driverFloor = 0.05 + // driverCount is how many drivers to name. + driverCount = 3 +) + +// tiers names the review tiers, cheapest review first. Index order is what lets +// the size floor raise a tier without a second table. +var tiers = []string{"trivial", "low", "medium", "high"} + +// loadFloors says what review load buys which tier, highest floor first. +// +// The cuts come from the cost of being wrong at each one: calling a medium pull +// request low costs a reviewer an hour, and calling a high one medium costs an +// incident. +var loadFloors = []struct { + floor float64 + tier string +}{ + {0.62, "high"}, + {0.42, "medium"}, + {0.22, "low"}, + {0.00, "trivial"}, +} + +// sizeFloors names the lowest tier a pull request of a given size can land in, +// whatever Jev returned, highest floor first. +// +// Jev sees a truncated diff, so a 54-file change can read as nine repeated edits +// and score as mechanical. Size is arithmetic, Go does it, and it only ever raises +// a tier. +var sizeFloors = []struct { + files int + lines int + tier string +}{ + {30, 1500, "high"}, + {12, 500, "medium"}, + {4, 150, "low"}, +} + +// tierAdvice says what to do with a pull request in each tier. The advice is the +// point of the triage, so it travels with the tier rather than living in a README. +var tierAdvice = map[string]string{ + "trivial": "merge on a glance: read the title, skim the diff, check that CI is green", + "low": "one reviewer, one pass, no meeting", + "medium": "one reviewer who knows this area, reading the whole diff", + "high": "a human reads this line by line, and the author walks them through it", +} + +// answer is one answer from Jev, covering all three question types. +// +// The API returns a different shape per type, and one struct with every field +// costs nothing: the fields a type does not use stay at their zero values. Type +// says which ones to read. +type answer struct { + Type string `json:"type"` + // Noul is the probability a noul question came back with, which is also its + // confidence. + Noul float64 `json:"noul"` + // Choice is the option a choice question picked. + Choice string `json:"choice"` + // Score is where a score question landed on its rubric, between levels. + Score float64 `json:"score"` + // Confidence comes with a choice and a score. A noul has none, because its + // probability is the confidence. + Confidence float64 `json:"confidence"` + // Legend maps each rubric level to the text from questions.yml, keyed by the + // level as a string. Its size is how many levels the rubric has. + // + // The value is any rather than string, because Jev echoes back whatever the + // criteria held. A level written in YAML as `- None: some text` parses as a + // mapping rather than a string, and the API returns that mapping. Decoding + // into a string would fail on a payload this binary should still be able to + // read. + Legend map[string]any `json:"legend"` + // Probabilities is the spread across a choice's options or a score's levels. + Probabilities map[string]float64 `json:"probabilities"` +} + +// usage is what one call cost in tokens. +type usage struct { + InputTokens int `json:"input_tokens"` + OutputTokens int `json:"output_tokens"` +} + +// apiResponse is the whole body Jev returns. +type apiResponse struct { + Model string `json:"model"` + Answers map[string]answer `json:"answers"` + Usage usage `json:"usage"` +} + +// wireQuestion is one question as the API wants it. +type wireQuestion struct { + Type string `json:"type"` + Instructions string `json:"instructions"` + // Criteria is omitted for a noul, which has none. omitempty leaves the key out + // rather than sending null. + Criteria any `json:"criteria,omitempty"` +} + +// result is everything one pull request produced: the answers, the load, the tier +// and what the call cost. +type result struct { + Pull pullRequest + Answers map[string]answer + Load float64 + Tier string + RaisedBySize bool + Coverage float64 + Drivers []string + Usage usage + LatencyMS int64 + Raw json.RawMessage +} + +// buildState assembles the state for one pull request, each part in its own named +// field. +// +// The counts go in as a sentence rather than as numbers to compare, because Jev +// cannot do arithmetic. The sentence is context, and every threshold on a count +// happens in Go. +// +// It returns the state and the coverage: the share of changed files whose patch +// fit the budget, which says how much of the change Jev actually read. +func buildState(repo string, pull pullRequest, set settings) (map[string]string, float64) { + diff, shown := diffText(pull.Files, set.MaxDiffChars) + omitted := len(pull.Files) - shown + + coverage := 1.0 + if len(pull.Files) > 0 { + coverage = float64(shown) / float64(len(pull.Files)) + } + + scope := fmt.Sprintf( + "%d files changed, %d lines added, %d removed", + pull.ChangedFiles, pull.Additions, pull.Deletions, + ) + if omitted > 0 { + scope += fmt.Sprintf( + ". The diff below holds %d of those files; %d are listed without a patch", + shown, omitted, + ) + } + + // Treat the description as untrusted, the same way you treat a prompt. An + // author who writes "trivial, please merge" moves these answers, so the field + // name says what the text is rather than presenting it as fact. + description := pull.Body + if strings.TrimSpace(description) == "" { + description = "(no description)" + } + if len(description) > set.MaxDescriptionChars { + description = description[:set.MaxDescriptionChars] + } + + if diff == "" { + diff = "(no textual diff: binary files, or none GitHub would render)" + } + + return map[string]string{ + "repo": repo, + "base_branch": pull.Base, + "title": pull.Title, + "description_written_by_the_author": description, + "scope": scope, + "changed_files": fileListText(pull.Files, set.MaxFileListChars), + "diff": diff, + }, coverage +} + +// fileListText lists every changed path with its line counts, one per line. +// +// Cheap in tokens and dense in signal: the paths alone say whether a change is +// docs, tests, infrastructure or product code, and they survive a diff that got +// truncated. +func fileListText(files []changedFile, budget int) string { + lines := make([]string, 0, len(files)) + for _, entry := range files { + lines = append(lines, fmt.Sprintf( + "%-9s +%-5d -%-5d %s", entry.Status, entry.Additions, entry.Deletions, entry.Path, + )) + } + + text := strings.Join(lines, "\n") + if len(text) <= budget { + return text + } + + // Cut on a line boundary, so the last entry a reader sees is a whole path + // rather than half of one. + kept := text[:budget] + if cut := strings.LastIndex(kept, "\n"); cut > 0 { + kept = kept[:cut] + } + shown := strings.Count(kept, "\n") + 1 + return fmt.Sprintf("%s\n... and %d more paths", kept, len(files)-shown) +} + +// diffText assembles the patches into one diff, fitting as many whole files as it +// can, smallest patch first. +// +// Triage asks how far a change reaches and whether the same edit repeats, and both +// want breadth: twelve files read in full answer them, where one 5,000-line +// lockfile answers neither. The file list carries the paths that did not fit. +// +// It returns the diff and how many files went in whole. +func diffText(files []changedFile, budget int) (string, int) { + withPatch := make([]changedFile, 0, len(files)) + for _, entry := range files { + if entry.HasPatch { + withPatch = append(withPatch, entry) + } + } + + // sort.SliceStable keeps equal-length patches in the order the dataset listed + // them, so two runs on one dataset build the same state. + sort.SliceStable(withPatch, func(i, j int) bool { + return len(withPatch[i].Patch) < len(withPatch[j].Patch) + }) + + var builder strings.Builder + shown := 0 + for _, entry := range withPatch { + chunk := fmt.Sprintf("diff --git a/%s b/%s\n%s\n", entry.Path, entry.Path, entry.Patch) + if builder.Len()+len(chunk) > budget { + break + } + builder.WriteString(chunk) + shown++ + } + return builder.String(), shown +} + +// ask sends one pull request and every question in a single call. +// +// That is the point of the pattern: the pull request is the expensive part of the +// request, so asking one more question about a diff already on the wire costs a +// few tokens and no extra round trip. +func ask(key string, set settings, specs []spec, repo string, pull pullRequest) (result, error) { + state, coverage := buildState(repo, pull, set) + + questions := make(map[string]wireQuestion, len(specs)) + for _, s := range specs { + questions[s.ID] = wireQuestion{ + Type: s.Q.Type, + Instructions: s.Q.Instructions, + Criteria: s.Q.Criteria, + } + } + + body, err := json.Marshal(map[string]any{ + "model": set.Model, + "state": state, + "questions": questions, + }) + if err != nil { + return result{}, fmt.Errorf("encoding the request for #%d: %w", pull.Number, err) + } + + ctx, cancel := context.WithTimeout(context.Background(), requestTimeout) + defer cancel() + + url := endpoint + if custom := strings.TrimSpace(os.Getenv(jevEndpointEnv)); custom != "" { + url = custom + } + + request, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return result{}, fmt.Errorf("building the request for #%d: %w", pull.Number, err) + } + request.Header.Set("Authorization", "Bearer "+key) + request.Header.Set("Content-Type", "application/json") + + started := time.Now() + response, err := http.DefaultClient.Do(request) + if err != nil { + return result{}, fmt.Errorf("calling Jev for #%d: %w", pull.Number, err) + } + defer response.Body.Close() + + raw, err := io.ReadAll(response.Body) + if err != nil { + return result{}, fmt.Errorf("reading Jev's answer for #%d: %w", pull.Number, err) + } + latency := time.Since(started).Milliseconds() + + if response.StatusCode != http.StatusOK { + return result{}, fmt.Errorf( + "Jev returned %d for #%d: %s", response.StatusCode, pull.Number, trimTo(string(raw), 200), + ) + } + + var parsed apiResponse + if err := json.Unmarshal(raw, &parsed); err != nil { + return result{}, fmt.Errorf("parsing Jev's answer for #%d: %w", pull.Number, err) + } + // A question this binary asked and got no answer for would otherwise surface + // as a zero credit, which reads as a real judgment rather than a gap. + for _, s := range specs { + if _, ok := parsed.Answers[s.ID]; !ok { + return result{}, fmt.Errorf("Jev returned no answer for %s on #%d", s.ID, pull.Number) + } + } + + load := reviewLoad(parsed.Answers, specs) + tier, raised := tierFor(load, pull.ChangedFiles, pull.Additions+pull.Deletions) + + return result{ + Pull: pull, + Answers: parsed.Answers, + Load: load, + Tier: tier, + RaisedBySize: raised, + Coverage: coverage, + Drivers: drivers(parsed.Answers, specs), + Usage: parsed.Usage, + LatencyMS: latency, + Raw: raw, + }, nil +} + +// credit turns one answer into what it contributed toward the review load, 0 to 1. +// +// A score divides by its own top level. A noul is its probability. Either one +// flips when the entry inverts, so a mechanical diff and a thorough description +// lower the load instead of raising it. A choice reads the credit table the entry +// carries, and neither choice carries one today. +func credit(s spec, a answer) float64 { + var value float64 + + switch s.Q.Type { + case "noul": + value = a.Noul + case "score": + // The legend has one entry per level, so its size minus one is the top + // level. Guarding the divisor keeps a malformed answer from panicking. + top := float64(len(a.Legend) - 1) + if top > 0 { + value = a.Score / top + } + default: + value = s.Q.Credit[a.Choice] + } + + if s.Q.Invert { + return 1.0 - value + } + return value +} + +// contribution is one answer's share of the load: its credit times its weight, +// over the total weight in the payload. +func contribution(s spec, a answer, total float64) float64 { + if total <= 0 { + return 0 + } + return s.Q.Weight * credit(s, a) / total +} + +// reviewLoad weights every weighted answer into one number, from 0 to 1. +// +// Dividing by the weights actually present means an edited weight needs no +// rebalancing of the others. +func reviewLoad(answers map[string]answer, specs []spec) float64 { + total := weightedTotal(specs) + if total <= 0 { + return 0 + } + + earned := 0.0 + for _, s := range specs { + if s.Q.Weight <= 0 { + continue + } + earned += s.Q.Weight * credit(s, answers[s.ID]) + } + return earned / total +} + +// drivers names the questions that account for most of the review load. +func drivers(answers map[string]answer, specs []spec) []string { + total := weightedTotal(specs) + + type share struct { + value float64 + label string + } + ranked := make([]share, 0, len(specs)) + for _, s := range specs { + if s.Q.Weight <= 0 { + continue + } + ranked = append(ranked, share{contribution(s, answers[s.ID], total), s.Q.Label}) + } + + sort.SliceStable(ranked, func(i, j int) bool { return ranked[i].value > ranked[j].value }) + + named := make([]string, 0, driverCount) + for _, item := range ranked { + if len(named) == driverCount || item.value < driverFloor { + break + } + named = append(named, item.label) + } + return named +} + +// tierIndex says where a tier sits in the order, cheapest review first, or -1 for +// a name that is not a tier. +func tierIndex(tier string) int { + for index, name := range tiers { + if name == tier { + return index + } + } + return -1 +} + +// sizeFloor names the lowest tier a pull request of this size can land in. +func sizeFloor(changedFiles, lines int) string { + for _, floor := range sizeFloors { + if changedFiles > floor.files || lines > floor.lines { + return floor.tier + } + } + return tiers[0] +} + +// tierFor reads a load as a tier, then lets the size floor raise it. +// +// It returns the tier and whether size raised it, so the output can mark the rows +// where Go overruled Jev instead of hiding the disagreement. +func tierFor(load float64, changedFiles, lines int) (string, bool) { + fromLoad := tiers[0] + for _, band := range loadFloors { + if load >= band.floor { + fromLoad = band.tier + break + } + } + + floor := sizeFloor(changedFiles, lines) + if tierIndex(floor) > tierIndex(fromLoad) { + return floor, true + } + return fromLoad, false +} + +// nearACut says whether a load sits close enough to a tier cut to move between +// runs. +func nearACut(load float64) bool { + for _, band := range loadFloors { + if band.floor <= 0 { + continue + } + delta := load - band.floor + if delta < 0 { + delta = -delta + } + if delta <= deadband+1e-9 { + return true + } + } + return false +} + +// costUSD prices a whole run from its input tokens. TypeSafe bills input tokens +// only, at the rate questions.yml records. +func costUSD(results []result, set settings) float64 { + tokens := 0 + for _, r := range results { + tokens += r.Usage.InputTokens + } + return float64(tokens) * set.InputUSDPerMillion / 1_000_000 +} diff --git a/samples/pr-triage/vend/SKILL.md b/samples/pr-triage/vend/SKILL.md new file mode 100644 index 0000000..d0bcf78 --- /dev/null +++ b/samples/pr-triage/vend/SKILL.md @@ -0,0 +1,96 @@ +--- +name: pr-triage +description: Sort a repository's open pull requests into review tiers, so a maintainer knows which ones a glance clears and which ones need an hour. Use when asked to triage, prioritise, or plan a review queue, to say which pull requests need attention first, or to gate a branch on how much review its queue is carrying. Works against github.com and GitHub Enterprise Server. +license: Apache-2.0 +metadata: + author: aarora79 + requires: pr-triage binary, a GitHub token, a TypeSafe API key +--- + +# pr-triage + +Ask one question per pull request that a reviewer actually has: how much review does this need? The `pr-triage` binary fetches a repository's pull requests, asks Jev eleven questions about each one in a single call, and sorts them into four tiers with the advice for each. + +One pull request costs about a third of a cent to read at 2026 prices, and a queue of twenty-six takes about four seconds. + +## Before you run it + +The binary needs two credentials, and reads each from a flag first and the environment second. + +| What | Flag | Environment | +| --- | --- | --- | +| GitHub | `-token` | `GITHUB_TOKEN`, `GH_TOKEN`, `GH_ENTERPRISE_TOKEN`, or a logged-in `gh` CLI | +| TypeSafe | `-jev-key` | `TYPESAFE_API_KEY` | + +Check the binary is there, and install it if it is not: + +```bash +command -v pr-triage || curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh | sh +``` + +If `TYPESAFE_API_KEY` is unset, ask the user for it rather than guessing. Never print it back. + +## Running it + +```bash +# The ten most recent open pull requests +pr-triage owner/repo + +# Every open one, or a window +pr-triage owner/repo -all +pr-triage owner/repo -since 30 +pr-triage owner/repo -since 2026-08-01 + +# One pull request, every question and what each contributed +pr-triage owner/repo -explain 1693 + +# Build the dataset now, triage it later for the cost of the Jev calls alone +pr-triage owner/repo -all -fetch-only +pr-triage -dataset data/owner-repo-open-all.json +``` + +On GitHub Enterprise Server, pass a URL on the host and the API base follows it: + +```bash +pr-triage https://ghe.example.com/owner/repo +pr-triage owner/repo -api-base https://ghe.example.com/api/v3 +``` + +Add `-quiet` when you are going to read the output rather than show it, which drops the progress lines and leaves the report on standard output. + +## Reading what comes back + +Every run prints a table, then the same pull requests grouped by tier with the advice for each, then what the run cost. It also writes two files into `-out` (default `./data`): a JSON report holding every answer, and a markdown report that pastes into an issue or a pull request without reformatting. + +Four tiers, cheapest review first: + +| Tier | What to do | +| --- | --- | +| `trivial` | merge on a glance: title, skim, green CI | +| `low` | one reviewer, one pass, no meeting | +| `medium` | one reviewer who knows the area, reading the whole diff | +| `high` | a human reads it line by line, and the author walks them through it | + +Three things in that output need reading with care. + +**A tier marked `(size)` came from the file and line counts, not from Jev.** Jev reads a truncated diff, so a 54-file change can read as one repeated edit. The size floor raises a tier and never lowers one, and the mark says where it did. + +**A tier marked `(on a cut)` sits within 0.02 of a band edge.** Repeat calls move a load by about that much, so it could land either side on the next run. Treat it as either of the two tiers it straddles. + +**A line saying it read part of the diff means the patch overflowed the budget.** A load drawn from 44% of a change is a weaker claim than the same number drawn from all of it. + +## Gating a branch + +`-fail-on-tier` exits 2 when any pull request lands in that tier or above, which turns the triage into a check: + +```bash +pr-triage owner/repo -all -fail-on-tier high -quiet +``` + +Exit codes: 0 finished clean, 1 could not finish, 2 a gate fired. + +## What this does not do + +It reads the pull request, and nothing else. It has no view of the repository around the diff, so it cannot tell whether a command the description names really exists, and it never says a change is correct. Nothing here approves, merges or comments. + +The tiers are advice about where to spend attention. TypeSafe reports 67.8% accuracy on their own benchmark, published September 2026, so before anyone trusts a tier, point it at pull requests your team already reviewed and compare the tiers against what those reviews cost. Move the cuts until they match the repository. diff --git a/samples/pr-triage/vend/install.sh b/samples/pr-triage/vend/install.sh new file mode 100755 index 0000000..b93494b --- /dev/null +++ b/samples/pr-triage/vend/install.sh @@ -0,0 +1,75 @@ +#!/bin/sh +# Install the pr-triage skill and the binary it runs. +# +# curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/vend/install.sh | sh +# +# This puts two things on the machine: the pr-triage binary, which fetches pull +# requests and triages them, and a SKILL.md that tells an agent when and how to +# run it. Neither needs Python, Go, or a checkout of this repo. +# +# Environment: +# VERSION the release to install, for example 0.1.0, default the newest +# BINDIR where the binary goes, default /usr/local/bin then ~/.local/bin +# SKILL_DIR where the skill goes, default ~/.claude/skills/pr-triage +# +# Piping a script from the internet into a shell is a choice. To read it first: +# curl -fsSLO https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/vend/install.sh +# less install.sh && sh install.sh +set -eu + +REPO="aarora79/jev-samples" +RAW="https://raw.githubusercontent.com/$REPO/main/samples/pr-triage" +BINARY="pr-triage" +SKILL_DIR="${SKILL_DIR:-$HOME/.claude/skills/pr-triage}" + +fail() { + echo "install: $1" >&2 + exit 1 +} + +command -v curl >/dev/null 2>&1 || fail "curl is required" + +# The binary first, because a skill with nothing to run is worse than no skill. +# go/install.sh resolves the newest pr-triage release, checks the published +# SHA256SUMS, and picks the build for this platform. VERSION and BINDIR pass +# straight through to it. +if command -v "$BINARY" >/dev/null 2>&1; then + echo "found $BINARY on PATH: $($BINARY -version)" +else + echo "installing the $BINARY binary" + curl -fsSL "$RAW/go/install.sh" | sh +fi + +# Then the skill. One file, so a plain download is the whole install. +echo "installing the skill into $SKILL_DIR" +mkdir -p "$SKILL_DIR" +curl -fsSL "$RAW/vend/SKILL.md" -o "$SKILL_DIR/SKILL.md" || + fail "could not download SKILL.md from $RAW/vend/SKILL.md" + +echo +echo "installed:" +echo " skill $SKILL_DIR/SKILL.md" +printf ' binary ' +command -v "$BINARY" || echo "not on PATH yet, see the note above" + +# The skill cannot work without these, and finding out now beats finding out +# halfway through a triage. +echo +missing="" +[ -n "${TYPESAFE_API_KEY:-}" ] || missing="TYPESAFE_API_KEY" +if [ -z "${GITHUB_TOKEN:-}${GH_TOKEN:-}${GH_ENTERPRISE_TOKEN:-}" ] && + ! gh auth token >/dev/null 2>&1; then + missing="$missing GITHUB_TOKEN" +fi + +if [ -n "$missing" ]; then + echo "still needed before the first run:$missing" >&2 + echo " GitHub GITHUB_TOKEN, GH_TOKEN, GH_ENTERPRISE_TOKEN, or gh auth login" >&2 + echo " TypeSafe TYPESAFE_API_KEY" >&2 +else + echo "credentials found for GitHub and TypeSafe" +fi + +echo +echo "try it: $BINARY owner/repo" +echo "on GHES: $BINARY https://ghe.example.com/owner/repo" From d1d912547ba977a9eb8ac3cf3bac1ce1c0a3bf04 Mon Sep 17 00:00:00 2001 From: Amit Arora Date: Fri, 25 Sep 2026 22:39:00 +0000 Subject: [PATCH 2/2] Quote the rubric level YAML was reading as a mapping, in both payloads design_decisions level 0 read: - None: the same edit repeated, or a value changed which is YAML for a mapping, not a string, so Jev received {"None": "the same edit repeated, or a value changed"} as that rubric level and echoed the object back in the legend. Every run recorded until now described level 0 to the model as a JSON object. Quoting the string fixes it and keeps the wording the rubric always intended. Fixed in the Python payload and in the Go copy, which build.sh syncs and payload_test.go guards. A scan of every other criteria entry, list and map alike, found no second instance. The fix changes what the model is asked, so the numbers moved. design_decisions shifted on 17 of the 18 airflow pull requests, mostly downward in score and more sharply in confidence: #73703 from 0.28 to 0.08, #73719 from 0.55 to 0.40, #73722 from 0.41 to 0.26. Loads moved by up to 0.016, and the tier counts held at 3 high, 5 medium, 8 low and 2 trivial. Both committed reports were regenerated and every figure the README pins was re-synced to that run and checked against it: #73704 to 0.50, #73713 to 0.37, #73722 to 0.19. Running the Go binary over the same dataset agrees with the Python inside the documented drift, a mean gap of 0.0093 and a worst of 0.0234, and both now show the quoted text in the legend. Also promotes the skill install out of a subsection into its own "Install it as a Claude Code skill" section, so it can be linked on its own. It answers the whole question in one place: the one curl command, what lands where, the two credentials, and what to say to the agent afterward. The root README row links straight to it. --- README.md | 2 +- samples/pr-triage/README.md | 47 +- .../apache-airflow-open-latest-18-triage.json | 1501 ++++++++--------- .../apache-airflow-open-latest-18-triage.md | 74 +- samples/pr-triage/go/questions.yml | 2 +- samples/pr-triage/questions.yml | 2 +- 6 files changed, 807 insertions(+), 821 deletions(-) diff --git a/README.md b/README.md index 5938246..890fbc8 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ Every sample is a small self-contained project under [samples/](samples/), and y | --- | --- | | [readme-check](samples/readme-check/) | Reads a README from disk or a GitHub URL, asks five questions about it in one call, and explains every number it prints. Uses all three primitives: `Choice`, `Score` and `Noul`. | | [agents-md-readiness](samples/agents-md-readiness/) | Scores an AGENTS.md or CLAUDE.md against the [agents.md](https://agents.md) format: seventeen questions in one call, a padded markdown table, one weighted readiness score, and a JSON report per document. Ships scored runs for eleven open-source repos in [data/](samples/agents-md-readiness/data/), plus a [Go port](samples/agents-md-readiness/go/) that compiles the same check into one static binary for a CI runner without Python. | -| [pr-triage](samples/pr-triage/) | Sorts a repository's open pull requests into four review tiers, one call each: eleven questions about the title, the description, the file list and the diff, nine of them weighted into a review load, and a size floor in Python for the diffs too large to send whole. Builds its dataset from the GitHub API, prints a triage table grouped by tier with the advice for each one, and ships the scored runs for two repositories in [data/](samples/pr-triage/data/). Twenty-six pull requests took four seconds and $0.0077. Ships a [Go port](samples/pr-triage/go/) that does both halves in one static binary, against github.com or a GitHub Enterprise Server host, and a [vendable skill](samples/pr-triage/vend/) that installs the binary and drives it. | +| [pr-triage](samples/pr-triage/) | Sorts a repository's open pull requests into four review tiers, one call each: eleven questions about the title, the description, the file list and the diff, nine of them weighted into a review load, and a size floor in Python for the diffs too large to send whole. Builds its dataset from the GitHub API, prints a triage table grouped by tier with the advice for each one, and ships the scored runs for two repositories in [data/](samples/pr-triage/data/). Twenty-six pull requests took four seconds and $0.0077. Ships a [Go port](samples/pr-triage/go/) that does both halves in one static binary, against github.com or a GitHub Enterprise Server host, and a [Claude Code skill](samples/pr-triage/#install-it-as-a-claude-code-skill) that installs in one command and then drives the binary for you. | Each sample keeps its Jev payload in `questions.yml`: the model pin, the state budget, every question with the label it prints under, and what each answer is worth toward the score. The Python reads that file and does the arithmetic, so changing a question or a weight is a data change and changing a threshold is a code change. diff --git a/samples/pr-triage/README.md b/samples/pr-triage/README.md index ec7d7aa..05d77cf 100644 --- a/samples/pr-triage/README.md +++ b/samples/pr-triage/README.md @@ -57,23 +57,46 @@ uv run pr_triage.py --dataset data/owner-repo-open-all.json --explain 1693 `pr_triage.py` needs a Jev key in `TYPESAFE_API_KEY`, from the environment or from `.env` beside the sample or at the repo root. -### One binary, and a skill that drives it +### The same check without Python -A machine with no Python can run the same check from a single static executable. [`go/`](go/) holds a Go port that does both halves in one command, against github.com or a GitHub Enterprise Server host: +[`go/`](go/) holds a Go port that compiles both halves into one static binary, which is what the skill installs and what a CI runner wants. It works against github.com and a GitHub Enterprise Server host. + +The Python stays canonical. `go/questions.yml` is a copy, `build.sh` refreshes it before every build, and `payload_test.go` fails when the two drift, so the two tools cannot score one pull request differently in silence. + + +## Install it as a Claude Code skill + +One command puts the tool and the skill on the machine. After that the skill does the work, and nobody has to learn the flags. ```bash -curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/go/install.sh | sh -pr-triage apache/airflow -all -fail-on-tier high -pr-triage https://ghe.example.com/owner/repo +curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/vend/install.sh | sh ``` -[`vend/`](vend/) turns that into something any repository can adopt: a `SKILL.md` telling an agent when to reach for the triage and how to read it, and an installer that fetches the binary and drops the skill beside it. +That installs two things: + +| What | Where | Why | +| --- | --- | --- | +| the `pr-triage` binary | `/usr/local/bin`, or `~/.local/bin` | fetches the pull requests and triages them, no Python needed | +| `SKILL.md` | `~/.claude/skills/pr-triage/` | tells the agent when to triage and how to read the result | + +Set `BINDIR` or `SKILL_DIR` to put either somewhere else. The installer checks both credentials and names the one you are missing rather than failing halfway through a run: ```bash -curl -fsSL https://raw.githubusercontent.com/aarora79/jev-samples/main/samples/pr-triage/vend/install.sh | sh +export GITHUB_TOKEN=... # or GH_TOKEN, GH_ENTERPRISE_TOKEN, or run gh auth login +export TYPESAFE_API_KEY=... ``` -The Python stays canonical. `go/questions.yml` is a copy, `build.sh` refreshes it before every build, and `payload_test.go` fails when the two drift, so the two tools cannot score one pull request differently in silence. +Then start a new Claude Code session and ask for a triage in your own words: + +> triage the open pull requests on apache/airflow +> +> which of our open PRs need a real review this week? +> +> triage https://ghe.example.com/platform/gateway and tell me what to read first + +The skill picks the flags, runs the binary, reads the report and tells you which pull requests need an hour and which clear on a glance. It writes a JSON report and a markdown one you can paste into an issue. + +For CI rather than a conversation, [`go/README.md`](go/README.md) covers the binary on its own, including `-fail-on-tier` for failing a job and the three ways to point it at a GitHub Enterprise Server host. ## What it prints @@ -135,17 +158,17 @@ The inverted rows are where a good answer lowers the load. `has_tests` came back The two unweighted labels came back `feature` at 0.99 confidence and `security` at 0.95. They tell a reviewer what kind of change this is and where to start, and neither moves the number. -The size floor changed nothing here. Nineteen files sets a floor of medium, under the high the load already earned. Airflow #73713 is the case where it bites: 54 files of one repeated locale-formatting edit score a load of 0.38, which alone would read low, and the floor raises it to high. +The size floor changed nothing here. Nineteen files sets a floor of medium, under the high the load already earned. Airflow #73713 is the case where it bites: 54 files of one repeated locale-formatting edit score a load of 0.37, which alone would read low, and the floor raises it to high. ## What to notice -**All four tiers get used.** Eighteen recent open pull requests from [apache/airflow](https://github.com/apache/airflow) came out 3 high, 5 medium, 8 low and 2 trivial. The smallest is #73722, a two-line clarification of `max_db_retries` doc wording, which scores 0.20 and lands in trivial. How a queue spreads across the tiers is a fact about the repository, so read your own distribution before you move a cut. +**All four tiers get used.** Eighteen recent open pull requests from [apache/airflow](https://github.com/apache/airflow) came out 3 high, 5 medium, 8 low and 2 trivial. The smallest is #73722, a two-line clarification of `max_db_retries` doc wording, which scores 0.19 and lands in trivial. How a queue spreads across the tiers is a fact about the repository, so read your own distribution before you move a cut. -**Nine files can outrank fifty-four.** #73704 changes 89 lines across 9 files and lands in medium at 0.51, because it fixes socket leaks and adds request timeouts across providers: `security_surface` came back 0.84 and `has_tests` 0.03. #73713 changes 598 lines across 54 files and scores 0.38, because `mechanical` came back 0.80 on one locale-formatting edit repeated through the UI, and the change ships tests. The eleven questions sort by what a review has to catch, and the driver line names the question that did it. +**Nine files can outrank fifty-four.** #73704 changes 89 lines across 9 files and lands in medium at 0.50, because it fixes socket leaks and adds request timeouts across providers: `security_surface` came back 0.85 and `has_tests` 0.03. #73713 changes 598 lines across 54 files and scores 0.37, because `mechanical` came back 0.80 on one locale-formatting edit repeated through the UI, and the change ships tests. The eleven questions sort by what a review has to catch, and the driver line names the question that did it. **Arithmetic stays in Python.** Jev cannot count, so the file and line totals reach it as a sentence for context, and every threshold on a number lives in [pr_triage.py](pr_triage.py). `SIZE_FLOORS` names the lowest tier a change of a given size can land in: over 30 files or 1,500 lines is high whatever Jev returned. The floor only ever raises a tier, and the output marks the rows where it did, so a reader can see the disagreement instead of inheriting it. -**Say how much of the diff the model read.** Five of those eighteen hold more patch than the 24,000-character budget, so Jev read some files whole and the rest as paths and line counts. `_diff_text` fills the budget with the smallest patches first, because triage asks how far a change reaches and whether one edit repeats, and both of those want breadth. Coverage lands in the report as `diff_coverage` and in the grouped output as a line under any pull request below 70%. The 0.38 on #73713 came from 44% of its diff, which is a weaker claim than the same number across all of it, and the size floor guards those rows. +**Say how much of the diff the model read.** Five of those eighteen hold more patch than the 24,000-character budget, so Jev read some files whole and the rest as paths and line counts. `_diff_text` fills the budget with the smallest patches first, because triage asks how far a change reaches and whether one edit repeats, and both of those want breadth. Coverage lands in the report as `diff_coverage` and in the grouped output as a line under any pull request below 70%. The 0.37 on #73713 came from 44% of its diff, which is a weaker claim than the same number across all of it, and the size floor guards those rows. **Weights are a data change.** Raising `security_surface` to 0.25 means editing one line of `questions.yml`. Moving a tier cut means editing `LOAD_FLOORS`. Both sit in a diff, and neither hides inside a question's wording. diff --git a/samples/pr-triage/data/apache-airflow-open-latest-18-triage.json b/samples/pr-triage/data/apache-airflow-open-latest-18-triage.json index 064b13e..f83dab3 100644 --- a/samples/pr-triage/data/apache-airflow-open-latest-18-triage.json +++ b/samples/pr-triage/data/apache-airflow-open-latest-18-triage.json @@ -5,7 +5,7 @@ "limit": 18, "since": null }, - "triaged_at": "2026-09-25T19:44:49+00:00", + "triaged_at": "2026-09-25T21:25:17+00:00", "model": "jev-1.13.0", "questions_asked": 11, "tier_counts": { @@ -14,7 +14,7 @@ "medium": 5, "high": 3 }, - "cost_usd": 0.00367731, + "cost_usd": 0.00367429, "pull_requests": [ { "number": 73698, @@ -24,7 +24,7 @@ "changed_files": 8, "additions": 626, "deletions": 73, - "review_load": 0.6254, + "review_load": 0.6271, "tier": "high", "size_floor": "medium", "raised_by_size": false, @@ -35,10 +35,10 @@ "mechanical" ], "usage": { - "input_tokens": 7154, + "input_tokens": 7150, "output_tokens": 283 }, - "latency_ms": 174, + "latency_ms": 166, "questions": { "change_kind": { "label": "change kind", @@ -50,13 +50,13 @@ "choice": "bugfix", "confidence": 0.99, "probabilities": { - "bugfix": 0.99, + "dependency": 0.0, "config": 0.0, "refactor": 0.0, - "feature": 0.01, "tests": 0.0, - "docs": 0.0, - "dependency": 0.0 + "bugfix": 0.99, + "feature": 0.01, + "docs": 0.0 } }, "credit": null, @@ -70,13 +70,13 @@ "returned": { "type": "choice", "choice": "security", - "confidence": 0.96, + "confidence": 0.97, "probabilities": { - "deployment": 0.0, - "correctness": 0.02, - "security": 0.97, - "tests": 0.01, + "security": 0.98, "readability": 0.0, + "correctness": 0.02, + "deployment": 0.0, + "tests": 0.0, "compatibility": 0.0 } }, @@ -114,11 +114,9 @@ "returned": { "type": "score", "score": 2.0, - "confidence": 0.99, + "confidence": 1.0, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, @@ -139,7 +137,7 @@ "returned": { "type": "score", "score": 1.98, - "confidence": 0.97, + "confidence": 0.96, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -173,10 +171,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.32 + "noul": 0.31 }, - "credit": 0.68, - "adds_to_load": 0.0816 + "credit": 0.69, + "adds_to_load": 0.0828 }, "breaking_change": { "label": "breaking change", @@ -185,10 +183,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.71 + "noul": 0.7 }, - "credit": 0.71, - "adds_to_load": 0.0781 + "credit": 0.7, + "adds_to_load": 0.077 }, "scope_creep": { "label": "scope creep", @@ -209,10 +207,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.66 + "noul": 0.69 }, - "credit": 0.66, - "adds_to_load": 0.0528 + "credit": 0.69, + "adds_to_load": 0.0552 }, "has_tests": { "label": "has tests", @@ -221,10 +219,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.94 + "noul": 0.95 }, - "credit": 0.06, - "adds_to_load": 0.0048 + "credit": 0.05, + "adds_to_load": 0.004 } } }, @@ -236,7 +234,7 @@ "changed_files": 7, "additions": 136, "deletions": 5, - "review_load": 0.5293, + "review_load": 0.5321, "tier": "medium", "size_floor": "low", "raised_by_size": false, @@ -247,10 +245,10 @@ "mechanical" ], "usage": { - "input_tokens": 4565, + "input_tokens": 4561, "output_tokens": 283 }, - "latency_ms": 140, + "latency_ms": 171, "questions": { "change_kind": { "label": "change kind", @@ -260,14 +258,14 @@ "returned": { "type": "choice", "choice": "bugfix", - "confidence": 0.91, + "confidence": 0.9, "probabilities": { - "feature": 0.08, - "bugfix": 0.92, "refactor": 0.0, - "config": 0.0, "tests": 0.0, + "config": 0.0, "docs": 0.0, + "bugfix": 0.92, + "feature": 0.08, "dependency": 0.0 } }, @@ -282,12 +280,12 @@ "returned": { "type": "choice", "choice": "security", - "confidence": 0.85, + "confidence": 0.84, "probabilities": { + "security": 0.86, + "tests": 0.01, + "correctness": 0.12, "deployment": 0.0, - "correctness": 0.1, - "tests": 0.0, - "security": 0.89, "readability": 0.0, "compatibility": 0.01 } @@ -302,8 +300,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.1, - "confidence": 0.85, + "score": 1.09, + "confidence": 0.86, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -311,12 +309,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.9, - "2": 0.1 + "1": 0.91, + "2": 0.09 } }, - "credit": 0.55, - "adds_to_load": 0.0825 + "credit": 0.545, + "adds_to_load": 0.0818 }, "design_decisions": { "label": "design decisions", @@ -325,12 +323,10 @@ "inverted": false, "returned": { "type": "score", - "score": 1.96, - "confidence": 0.94, + "score": 1.95, + "confidence": 0.93, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, @@ -340,8 +336,8 @@ "2": 0.96 } }, - "credit": 0.98, - "adds_to_load": 0.1372 + "credit": 0.975, + "adds_to_load": 0.1365 }, "description_quality": { "label": "description quality", @@ -350,8 +346,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.48, - "confidence": 0.28, + "score": 1.43, + "confidence": 0.35, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -359,12 +355,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.52, - "2": 0.48 + "1": 0.57, + "2": 0.43 } }, - "credit": 0.26, - "adds_to_load": 0.0208 + "credit": 0.285, + "adds_to_load": 0.0228 }, "security_surface": { "label": "security surface", @@ -385,10 +381,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.26 + "noul": 0.24 }, - "credit": 0.74, - "adds_to_load": 0.0888 + "credit": 0.76, + "adds_to_load": 0.0912 }, "breaking_change": { "label": "breaking change", @@ -397,10 +393,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.29 + "noul": 0.28 }, - "credit": 0.29, - "adds_to_load": 0.0319 + "credit": 0.28, + "adds_to_load": 0.0308 }, "scope_creep": { "label": "scope creep", @@ -409,10 +405,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.12 + "noul": 0.13 }, - "credit": 0.12, - "adds_to_load": 0.0108 + "credit": 0.13, + "adds_to_load": 0.0117 }, "infra_surface": { "label": "infra surface", @@ -441,28 +437,28 @@ } }, { - "number": 73704, - "title": "Fix socket leaks and add missing request timeouts across providers", - "url": "https://github.com/apache/airflow/pull/73704", - "author": "TimurRakhmatullin86", - "changed_files": 9, - "additions": 57, - "deletions": 32, - "review_load": 0.5108, - "tier": "medium", - "size_floor": "low", - "raised_by_size": false, - "diff_coverage": 1.0, + "number": 73706, + "title": "Account for AgentOperator spend on failed runs and across retries", + "url": "https://github.com/apache/airflow/pull/73706", + "author": "Lee-W", + "changed_files": 17, + "additions": 2634, + "deletions": 117, + "review_load": 0.5103, + "tier": "high", + "size_floor": "high", + "raised_by_size": true, + "diff_coverage": 0.4706, "drivers": [ + "design decisions", "blast radius", - "security surface", - "has tests" + "breaking change" ], "usage": { - "input_tokens": 5588, + "input_tokens": 7675, "output_tokens": 285 }, - "latency_ms": 143, + "latency_ms": 190, "questions": { "change_kind": { "label": "change kind", @@ -472,15 +468,15 @@ "returned": { "type": "choice", "choice": "bugfix", - "confidence": 1.0, + "confidence": 0.77, "probabilities": { - "feature": 0.0, - "refactor": 0.0, - "bugfix": 1.0, + "bugfix": 0.8, "tests": 0.0, + "dependency": 0.0, + "feature": 0.2, "config": 0.0, "docs": 0.0, - "dependency": 0.0 + "refactor": 0.0 } }, "credit": null, @@ -494,14 +490,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.74, + "confidence": 0.85, "probabilities": { - "compatibility": 0.08, "deployment": 0.0, + "compatibility": 0.07, + "tests": 0.05, "security": 0.0, - "tests": 0.13, - "readability": 0.0, - "correctness": 0.79 + "correctness": 0.88, + "readability": 0.0 } }, "credit": null, @@ -514,8 +510,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.73, - "confidence": 0.59, + "score": 1.44, + "confidence": 0.34, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -523,12 +519,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.27, - "2": 0.73 + "1": 0.56, + "2": 0.44 } }, - "credit": 0.865, - "adds_to_load": 0.1298 + "credit": 0.72, + "adds_to_load": 0.108 }, "design_decisions": { "label": "design decisions", @@ -537,23 +533,21 @@ "inverted": false, "returned": { "type": "score", - "score": 0.95, - "confidence": 0.26, + "score": 2.0, + "confidence": 1.0, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.28, - "1": 0.5, - "2": 0.22 + "0": 0.0, + "1": 0.0, + "2": 1.0 } }, - "credit": 0.475, - "adds_to_load": 0.0665 + "credit": 1.0, + "adds_to_load": 0.14 }, "description_quality": { "label": "description quality", @@ -562,8 +556,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.2, - "confidence": 0.7, + "score": 1.25, + "confidence": 0.62, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -571,12 +565,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.8, - "2": 0.2 + "1": 0.75, + "2": 0.25 } }, - "credit": 0.4, - "adds_to_load": 0.032 + "credit": 0.375, + "adds_to_load": 0.03 }, "security_surface": { "label": "security surface", @@ -585,10 +579,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.84 + "noul": 0.12 }, - "credit": 0.84, - "adds_to_load": 0.126 + "credit": 0.12, + "adds_to_load": 0.018 }, "mechanical": { "label": "mechanical", @@ -597,10 +591,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.52 + "noul": 0.29 }, - "credit": 0.48, - "adds_to_load": 0.0576 + "credit": 0.71, + "adds_to_load": 0.0852 }, "breaking_change": { "label": "breaking change", @@ -609,10 +603,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.1 + "noul": 0.78 }, - "credit": 0.1, - "adds_to_load": 0.011 + "credit": 0.78, + "adds_to_load": 0.0858 }, "scope_creep": { "label": "scope creep", @@ -621,10 +615,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.07 + "noul": 0.41 }, - "credit": 0.07, - "adds_to_load": 0.0063 + "credit": 0.41, + "adds_to_load": 0.0369 }, "infra_surface": { "label": "infra surface", @@ -633,10 +627,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.05 + "noul": 0.04 }, - "credit": 0.05, - "adds_to_load": 0.004 + "credit": 0.04, + "adds_to_load": 0.0032 }, "has_tests": { "label": "has tests", @@ -645,36 +639,36 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.03 + "noul": 0.96 }, - "credit": 0.97, - "adds_to_load": 0.0776 + "credit": 0.04, + "adds_to_load": 0.0032 } } }, { - "number": 73706, - "title": "Account for AgentOperator spend on failed runs and across retries", - "url": "https://github.com/apache/airflow/pull/73706", - "author": "Lee-W", - "changed_files": 17, - "additions": 2634, - "deletions": 117, - "review_load": 0.5047, - "tier": "high", - "size_floor": "high", - "raised_by_size": true, - "diff_coverage": 0.4706, + "number": 73704, + "title": "Fix socket leaks and add missing request timeouts across providers", + "url": "https://github.com/apache/airflow/pull/73704", + "author": "TimurRakhmatullin86", + "changed_files": 9, + "additions": 57, + "deletions": 32, + "review_load": 0.5032, + "tier": "medium", + "size_floor": "low", + "raised_by_size": false, + "diff_coverage": 1.0, "drivers": [ - "design decisions", + "security surface", "blast radius", - "breaking change" + "has tests" ], "usage": { - "input_tokens": 7679, + "input_tokens": 5584, "output_tokens": 285 }, - "latency_ms": 146, + "latency_ms": 155, "questions": { "change_kind": { "label": "change kind", @@ -684,14 +678,14 @@ "returned": { "type": "choice", "choice": "bugfix", - "confidence": 0.68, + "confidence": 1.0, "probabilities": { - "feature": 0.27, + "bugfix": 1.0, "tests": 0.0, - "dependency": 0.0, "refactor": 0.0, + "dependency": 0.0, "docs": 0.0, - "bugfix": 0.73, + "feature": 0.0, "config": 0.0 } }, @@ -706,14 +700,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.81, + "confidence": 0.72, "probabilities": { "readability": 0.0, - "tests": 0.06, - "security": 0.0, + "tests": 0.15, + "compatibility": 0.08, + "security": 0.01, "deployment": 0.0, - "compatibility": 0.1, - "correctness": 0.84 + "correctness": 0.76 } }, "credit": null, @@ -726,8 +720,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.45, - "confidence": 0.33, + "score": 1.7, + "confidence": 0.55, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -735,12 +729,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.55, - "2": 0.45 + "1": 0.29, + "2": 0.71 } }, - "credit": 0.725, - "adds_to_load": 0.1087 + "credit": 0.85, + "adds_to_load": 0.1275 }, "design_decisions": { "label": "design decisions", @@ -749,23 +743,21 @@ "inverted": false, "returned": { "type": "score", - "score": 2.0, - "confidence": 0.99, + "score": 0.85, + "confidence": 0.22, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.0, - "1": 0.0, - "2": 1.0 + "0": 0.33, + "1": 0.48, + "2": 0.19 } }, - "credit": 1.0, - "adds_to_load": 0.14 + "credit": 0.425, + "adds_to_load": 0.0595 }, "description_quality": { "label": "description quality", @@ -774,8 +766,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.29, - "confidence": 0.56, + "score": 1.18, + "confidence": 0.73, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -783,12 +775,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.71, - "2": 0.29 + "1": 0.82, + "2": 0.18 } }, - "credit": 0.355, - "adds_to_load": 0.0284 + "credit": 0.41, + "adds_to_load": 0.0328 }, "security_surface": { "label": "security surface", @@ -797,10 +789,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.11 + "noul": 0.85 }, - "credit": 0.11, - "adds_to_load": 0.0165 + "credit": 0.85, + "adds_to_load": 0.1275 }, "mechanical": { "label": "mechanical", @@ -809,10 +801,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.3 + "noul": 0.54 }, - "credit": 0.7, - "adds_to_load": 0.084 + "credit": 0.46, + "adds_to_load": 0.0552 }, "breaking_change": { "label": "breaking change", @@ -821,10 +813,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.77 + "noul": 0.1 }, - "credit": 0.77, - "adds_to_load": 0.0847 + "credit": 0.1, + "adds_to_load": 0.011 }, "scope_creep": { "label": "scope creep", @@ -833,10 +825,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.39 + "noul": 0.09 }, - "credit": 0.39, - "adds_to_load": 0.0351 + "credit": 0.09, + "adds_to_load": 0.0081 }, "infra_surface": { "label": "infra surface", @@ -857,10 +849,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.96 + "noul": 0.03 }, - "credit": 0.04, - "adds_to_load": 0.0032 + "credit": 0.97, + "adds_to_load": 0.0776 } } }, @@ -872,7 +864,7 @@ "changed_files": 4, "additions": 360, "deletions": 39, - "review_load": 0.4765, + "review_load": 0.4607, "tier": "medium", "size_floor": "low", "raised_by_size": false, @@ -880,13 +872,13 @@ "drivers": [ "design decisions", "mechanical", - "security surface" + "blast radius" ], "usage": { - "input_tokens": 5516, + "input_tokens": 5512, "output_tokens": 284 }, - "latency_ms": 150, + "latency_ms": 185, "questions": { "change_kind": { "label": "change kind", @@ -898,13 +890,13 @@ "choice": "feature", "confidence": 1.0, "probabilities": { - "refactor": 0.0, - "feature": 1.0, - "tests": 0.0, "dependency": 0.0, - "docs": 0.0, "config": 0.0, - "bugfix": 0.0 + "feature": 1.0, + "docs": 0.0, + "bugfix": 0.0, + "refactor": 0.0, + "tests": 0.0 } }, "credit": null, @@ -920,12 +912,12 @@ "choice": "correctness", "confidence": 0.86, "probabilities": { - "readability": 0.0, - "correctness": 0.88, + "compatibility": 0.03, "tests": 0.08, - "deployment": 0.0, - "compatibility": 0.04, - "security": 0.0 + "security": 0.0, + "readability": 0.0, + "correctness": 0.89, + "deployment": 0.0 } }, "credit": null, @@ -939,7 +931,7 @@ "returned": { "type": "score", "score": 1.01, - "confidence": 0.98, + "confidence": 0.99, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -962,18 +954,16 @@ "returned": { "type": "score", "score": 1.97, - "confidence": 0.96, + "confidence": 0.95, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { "0": 0.0, - "1": 0.02, - "2": 0.98 + "1": 0.03, + "2": 0.97 } }, "credit": 0.985, @@ -1009,10 +999,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.52 + "noul": 0.49 }, - "credit": 0.52, - "adds_to_load": 0.078 + "credit": 0.49, + "adds_to_load": 0.0735 }, "mechanical": { "label": "mechanical", @@ -1021,10 +1011,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.23 + "noul": 0.22 }, - "credit": 0.77, - "adds_to_load": 0.0924 + "credit": 0.78, + "adds_to_load": 0.0936 }, "breaking_change": { "label": "breaking change", @@ -1033,10 +1023,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.56 + "noul": 0.48 }, - "credit": 0.56, - "adds_to_load": 0.0616 + "credit": 0.48, + "adds_to_load": 0.0528 }, "scope_creep": { "label": "scope creep", @@ -1045,10 +1035,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.28 + "noul": 0.24 }, - "credit": 0.28, - "adds_to_load": 0.0252 + "credit": 0.24, + "adds_to_load": 0.0216 }, "infra_surface": { "label": "infra surface", @@ -1057,10 +1047,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.03 + "noul": 0.04 }, - "credit": 0.03, - "adds_to_load": 0.0024 + "credit": 0.04, + "adds_to_load": 0.0032 }, "has_tests": { "label": "has tests", @@ -1069,36 +1059,36 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.96 + "noul": 0.97 }, - "credit": 0.04, - "adds_to_load": 0.0032 + "credit": 0.03, + "adds_to_load": 0.0024 } } }, { - "number": 73703, - "title": "[chart/v1-2x-test] Helm: Allow hostAliases and log groomer lifecycle hooks on the Dag processor (#73159)", - "url": "https://github.com/apache/airflow/pull/73703", - "author": "pranayyelugam", - "changed_files": 5, - "additions": 64, - "deletions": 0, - "review_load": 0.4409, + "number": 73723, + "title": "TS SDK: embed one source region per native Dag file", + "url": "https://github.com/apache/airflow/pull/73723", + "author": "pierrejeambrun", + "changed_files": 10, + "additions": 558, + "deletions": 121, + "review_load": 0.4375, "tier": "medium", - "size_floor": "low", + "size_floor": "medium", "raised_by_size": false, - "diff_coverage": 1.0, + "diff_coverage": 0.8, "drivers": [ "design decisions", "blast radius", "mechanical" ], "usage": { - "input_tokens": 2825, + "input_tokens": 9191, "output_tokens": 284 }, - "latency_ms": 116, + "latency_ms": 178, "questions": { "change_kind": { "label": "change kind", @@ -1108,15 +1098,15 @@ "returned": { "type": "choice", "choice": "feature", - "confidence": 0.98, + "confidence": 0.72, "probabilities": { - "dependency": 0.0, - "docs": 0.0, + "feature": 0.76, + "config": 0.0, + "bugfix": 0.24, "tests": 0.0, - "refactor": 0.0, - "bugfix": 0.0, - "config": 0.02, - "feature": 0.98 + "docs": 0.0, + "dependency": 0.0, + "refactor": 0.0 } }, "credit": null, @@ -1130,14 +1120,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.49, + "confidence": 0.75, "probabilities": { - "readability": 0.0, - "tests": 0.33, - "deployment": 0.0, - "compatibility": 0.09, + "correctness": 0.8, "security": 0.0, - "correctness": 0.58 + "tests": 0.11, + "deployment": 0.0, + "readability": 0.0, + "compatibility": 0.09 } }, "credit": null, @@ -1150,8 +1140,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.01, - "confidence": 0.98, + "score": 1.23, + "confidence": 0.66, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -1159,12 +1149,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.99, - "2": 0.01 + "1": 0.77, + "2": 0.23 } }, - "credit": 0.505, - "adds_to_load": 0.0757 + "credit": 0.615, + "adds_to_load": 0.0922 }, "design_decisions": { "label": "design decisions", @@ -1173,23 +1163,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.52, - "confidence": 0.28, + "score": 2.0, + "confidence": 1.0, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.11, - "1": 0.27, - "2": 0.62 + "0": 0.0, + "1": 0.0, + "2": 1.0 } }, - "credit": 0.76, - "adds_to_load": 0.1064 + "credit": 1.0, + "adds_to_load": 0.14 }, "description_quality": { "label": "description quality", @@ -1198,21 +1186,21 @@ "inverted": true, "returned": { "type": "score", - "score": 1.33, - "confidence": 0.48, + "score": 2.0, + "confidence": 1.0, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", "2": "Explains what changed, why, and how the author verified it" }, "probabilities": { - "0": 0.01, - "1": 0.65, - "2": 0.34 + "0": 0.0, + "1": 0.0, + "2": 1.0 } }, - "credit": 0.335, - "adds_to_load": 0.0268 + "credit": 0.0, + "adds_to_load": 0.0 }, "security_surface": { "label": "security surface", @@ -1221,10 +1209,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.2 + "noul": 0.05 }, - "credit": 0.2, - "adds_to_load": 0.03 + "credit": 0.05, + "adds_to_load": 0.0075 }, "mechanical": { "label": "mechanical", @@ -1233,10 +1221,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.37 + "noul": 0.27 }, - "credit": 0.63, - "adds_to_load": 0.0756 + "credit": 0.73, + "adds_to_load": 0.0876 }, "breaking_change": { "label": "breaking change", @@ -1245,10 +1233,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.42 + "noul": 0.78 }, - "credit": 0.42, - "adds_to_load": 0.0462 + "credit": 0.78, + "adds_to_load": 0.0858 }, "scope_creep": { "label": "scope creep", @@ -1257,10 +1245,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.09 + "noul": 0.2 }, - "credit": 0.09, - "adds_to_load": 0.0081 + "credit": 0.2, + "adds_to_load": 0.018 }, "infra_surface": { "label": "infra surface", @@ -1269,10 +1257,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.87 + "noul": 0.05 }, - "credit": 0.87, - "adds_to_load": 0.0696 + "credit": 0.05, + "adds_to_load": 0.004 }, "has_tests": { "label": "has tests", @@ -1289,28 +1277,28 @@ } }, { - "number": 73723, - "title": "TS SDK: embed one source region per native Dag file", - "url": "https://github.com/apache/airflow/pull/73723", - "author": "pierrejeambrun", - "changed_files": 10, - "additions": 558, - "deletions": 121, - "review_load": 0.4308, + "number": 73703, + "title": "[chart/v1-2x-test] Helm: Allow hostAliases and log groomer lifecycle hooks on the Dag processor (#73159)", + "url": "https://github.com/apache/airflow/pull/73703", + "author": "pranayyelugam", + "changed_files": 5, + "additions": 64, + "deletions": 0, + "review_load": 0.4345, "tier": "medium", - "size_floor": "medium", + "size_floor": "low", "raised_by_size": false, - "diff_coverage": 0.8, + "diff_coverage": 1.0, "drivers": [ "design decisions", - "blast radius", - "breaking change" + "mechanical", + "blast radius" ], "usage": { - "input_tokens": 9195, + "input_tokens": 2821, "output_tokens": 284 }, - "latency_ms": 141, + "latency_ms": 134, "questions": { "change_kind": { "label": "change kind", @@ -1320,14 +1308,14 @@ "returned": { "type": "choice", "choice": "feature", - "confidence": 0.75, + "confidence": 0.98, "probabilities": { - "tests": 0.0, - "feature": 0.79, "refactor": 0.0, - "bugfix": 0.21, + "config": 0.02, "dependency": 0.0, - "config": 0.0, + "tests": 0.0, + "bugfix": 0.0, + "feature": 0.98, "docs": 0.0 } }, @@ -1342,14 +1330,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.7, + "confidence": 0.56, "probabilities": { - "tests": 0.15, - "compatibility": 0.1, - "correctness": 0.75, - "deployment": 0.0, "security": 0.0, - "readability": 0.0 + "readability": 0.0, + "deployment": 0.0, + "correctness": 0.64, + "tests": 0.28, + "compatibility": 0.08 } }, "credit": null, @@ -1362,8 +1350,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.2, - "confidence": 0.7, + "score": 1.01, + "confidence": 0.99, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -1371,12 +1359,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.8, - "2": 0.2 + "1": 0.99, + "2": 0.01 } }, - "credit": 0.6, - "adds_to_load": 0.09 + "credit": 0.505, + "adds_to_load": 0.0757 }, "design_decisions": { "label": "design decisions", @@ -1385,23 +1373,21 @@ "inverted": false, "returned": { "type": "score", - "score": 2.0, - "confidence": 1.0, + "score": 1.39, + "confidence": 0.08, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.0, - "1": 0.0, - "2": 1.0 + "0": 0.17, + "1": 0.28, + "2": 0.55 } }, - "credit": 1.0, - "adds_to_load": 0.14 + "credit": 0.695, + "adds_to_load": 0.0973 }, "description_quality": { "label": "description quality", @@ -1410,21 +1396,21 @@ "inverted": true, "returned": { "type": "score", - "score": 2.0, - "confidence": 1.0, + "score": 1.31, + "confidence": 0.51, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", "2": "Explains what changed, why, and how the author verified it" }, "probabilities": { - "0": 0.0, - "1": 0.0, - "2": 1.0 + "0": 0.01, + "1": 0.67, + "2": 0.32 } }, - "credit": 0.0, - "adds_to_load": 0.0 + "credit": 0.345, + "adds_to_load": 0.0276 }, "security_surface": { "label": "security surface", @@ -1433,10 +1419,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.04 + "noul": 0.23 }, - "credit": 0.04, - "adds_to_load": 0.006 + "credit": 0.23, + "adds_to_load": 0.0345 }, "mechanical": { "label": "mechanical", @@ -1445,10 +1431,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.3 + "noul": 0.35 }, - "credit": 0.7, - "adds_to_load": 0.084 + "credit": 0.65, + "adds_to_load": 0.078 }, "breaking_change": { "label": "breaking change", @@ -1457,10 +1443,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.77 + "noul": 0.39 }, - "credit": 0.77, - "adds_to_load": 0.0847 + "credit": 0.39, + "adds_to_load": 0.0429 }, "scope_creep": { "label": "scope creep", @@ -1469,10 +1455,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.21 + "noul": 0.08 }, - "credit": 0.21, - "adds_to_load": 0.0189 + "credit": 0.08, + "adds_to_load": 0.0072 }, "infra_surface": { "label": "infra surface", @@ -1481,10 +1467,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.05 + "noul": 0.86 }, - "credit": 0.05, - "adds_to_load": 0.004 + "credit": 0.86, + "adds_to_load": 0.0688 }, "has_tests": { "label": "has tests", @@ -1493,10 +1479,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.96 + "noul": 0.97 }, - "credit": 0.04, - "adds_to_load": 0.0032 + "credit": 0.03, + "adds_to_load": 0.0024 } } }, @@ -1508,7 +1494,7 @@ "changed_files": 4, "additions": 291, "deletions": 3, - "review_load": 0.4057, + "review_load": 0.4153, "tier": "low", "size_floor": "low", "raised_by_size": false, @@ -1519,10 +1505,10 @@ "blast radius" ], "usage": { - "input_tokens": 7135, + "input_tokens": 7131, "output_tokens": 284 }, - "latency_ms": 155, + "latency_ms": 146, "questions": { "change_kind": { "label": "change kind", @@ -1534,12 +1520,12 @@ "choice": "feature", "confidence": 0.43, "probabilities": { - "tests": 0.0, - "feature": 0.51, + "bugfix": 0.48, "docs": 0.0, - "bugfix": 0.49, "dependency": 0.0, "config": 0.0, + "feature": 0.52, + "tests": 0.0, "refactor": 0.0 } }, @@ -1554,14 +1540,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.76, + "confidence": 0.77, "probabilities": { - "correctness": 0.81, - "readability": 0.0, - "tests": 0.1, + "compatibility": 0.1, + "deployment": 0.0, "security": 0.0, - "compatibility": 0.09, - "deployment": 0.0 + "tests": 0.09, + "correctness": 0.81, + "readability": 0.0 } }, "credit": null, @@ -1597,23 +1583,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.99, + "score": 2.0, "confidence": 0.99, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { "0": 0.0, - "1": 0.01, - "2": 0.99 + "1": 0.0, + "2": 1.0 } }, - "credit": 0.995, - "adds_to_load": 0.1393 + "credit": 1.0, + "adds_to_load": 0.14 }, "description_quality": { "label": "description quality", @@ -1645,10 +1629,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.29 + "noul": 0.32 }, - "credit": 0.29, - "adds_to_load": 0.0435 + "credit": 0.32, + "adds_to_load": 0.048 }, "mechanical": { "label": "mechanical", @@ -1657,10 +1641,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.26 + "noul": 0.27 }, - "credit": 0.74, - "adds_to_load": 0.0888 + "credit": 0.73, + "adds_to_load": 0.0876 }, "breaking_change": { "label": "breaking change", @@ -1669,10 +1653,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.34 + "noul": 0.4 }, - "credit": 0.34, - "adds_to_load": 0.0374 + "credit": 0.4, + "adds_to_load": 0.044 }, "scope_creep": { "label": "scope creep", @@ -1681,10 +1665,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.17 + "noul": 0.16 }, - "credit": 0.17, - "adds_to_load": 0.0153 + "credit": 0.16, + "adds_to_load": 0.0144 }, "infra_surface": { "label": "infra surface", @@ -1720,7 +1704,7 @@ "changed_files": 54, "additions": 467, "deletions": 131, - "review_load": 0.3785, + "review_load": 0.3746, "tier": "high", "size_floor": "high", "raised_by_size": true, @@ -1731,10 +1715,10 @@ "scope creep" ], "usage": { - "input_tokens": 10341, + "input_tokens": 10337, "output_tokens": 284 }, - "latency_ms": 157, + "latency_ms": 150, "questions": { "change_kind": { "label": "change kind", @@ -1744,15 +1728,15 @@ "returned": { "type": "choice", "choice": "feature", - "confidence": 1.0, + "confidence": 0.99, "probabilities": { - "dependency": 0.0, - "tests": 0.0, "bugfix": 0.0, + "tests": 0.0, "docs": 0.0, + "dependency": 0.0, "refactor": 0.0, - "feature": 1.0, - "config": 0.0 + "config": 0.0, + "feature": 1.0 } }, "credit": null, @@ -1766,14 +1750,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.64, + "confidence": 0.7, "probabilities": { - "compatibility": 0.1, "deployment": 0.0, + "tests": 0.16, "readability": 0.0, - "tests": 0.2, + "compatibility": 0.09, "security": 0.0, - "correctness": 0.7 + "correctness": 0.75 } }, "credit": null, @@ -1809,23 +1793,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.93, - "confidence": 0.89, + "score": 1.88, + "confidence": 0.82, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.02, + "0": 0.04, "1": 0.03, - "2": 0.95 + "2": 0.93 } }, - "credit": 0.965, - "adds_to_load": 0.1351 + "credit": 0.94, + "adds_to_load": 0.1316 }, "description_quality": { "label": "description quality", @@ -1857,10 +1839,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.03 + "noul": 0.04 }, - "credit": 0.03, - "adds_to_load": 0.0045 + "credit": 0.04, + "adds_to_load": 0.006 }, "mechanical": { "label": "mechanical", @@ -1881,10 +1863,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.08 + "noul": 0.07 }, - "credit": 0.08, - "adds_to_load": 0.0088 + "credit": 0.07, + "adds_to_load": 0.0077 }, "scope_creep": { "label": "scope creep", @@ -1917,10 +1899,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.95 + "noul": 0.96 }, - "credit": 0.05, - "adds_to_load": 0.004 + "credit": 0.04, + "adds_to_load": 0.0032 } } }, @@ -1932,7 +1914,7 @@ "changed_files": 4, "additions": 209, "deletions": 12, - "review_load": 0.377, + "review_load": 0.3708, "tier": "low", "size_floor": "low", "raised_by_size": false, @@ -1943,10 +1925,10 @@ "mechanical" ], "usage": { - "input_tokens": 5050, + "input_tokens": 5046, "output_tokens": 285 }, - "latency_ms": 186, + "latency_ms": 177, "questions": { "change_kind": { "label": "change kind", @@ -1958,13 +1940,13 @@ "choice": "bugfix", "confidence": 1.0, "probabilities": { - "bugfix": 1.0, - "tests": 0.0, - "docs": 0.0, "dependency": 0.0, - "refactor": 0.0, + "tests": 0.0, "config": 0.0, - "feature": 0.0 + "docs": 0.0, + "bugfix": 1.0, + "feature": 0.0, + "refactor": 0.0 } }, "credit": null, @@ -1978,14 +1960,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.93, + "confidence": 0.94, "probabilities": { + "correctness": 0.95, + "tests": 0.04, + "security": 0.0, "deployment": 0.0, - "tests": 0.06, "readability": 0.0, - "compatibility": 0.01, - "security": 0.0, - "correctness": 0.9299999999999999 + "compatibility": 0.01 } }, "credit": null, @@ -1998,8 +1980,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.33, - "confidence": 0.48, + "score": 1.31, + "confidence": 0.52, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -2007,12 +1989,12 @@ }, "probabilities": { "0": 0.01, - "1": 0.65, - "2": 0.34 + "1": 0.68, + "2": 0.31 } }, - "credit": 0.665, - "adds_to_load": 0.0998 + "credit": 0.655, + "adds_to_load": 0.0983 }, "design_decisions": { "label": "design decisions", @@ -2022,11 +2004,9 @@ "returned": { "type": "score", "score": 1.99, - "confidence": 0.98, + "confidence": 0.99, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, @@ -2046,8 +2026,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.79, - "confidence": 0.69, + "score": 1.81, + "confidence": 0.71, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -2055,12 +2035,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.2, - "2": 0.8 + "1": 0.19, + "2": 0.81 } }, - "credit": 0.105, - "adds_to_load": 0.0084 + "credit": 0.095, + "adds_to_load": 0.0076 }, "security_surface": { "label": "security surface", @@ -2069,10 +2049,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.06 + "noul": 0.05 }, - "credit": 0.06, - "adds_to_load": 0.009 + "credit": 0.05, + "adds_to_load": 0.0075 }, "mechanical": { "label": "mechanical", @@ -2081,10 +2061,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.35 + "noul": 0.37 }, - "credit": 0.65, - "adds_to_load": 0.078 + "credit": 0.63, + "adds_to_load": 0.0756 }, "breaking_change": { "label": "breaking change", @@ -2144,7 +2124,7 @@ "changed_files": 2, "additions": 98, "deletions": 6, - "review_load": 0.3708, + "review_load": 0.3665, "tier": "low", "size_floor": "trivial", "raised_by_size": false, @@ -2155,10 +2135,10 @@ "mechanical" ], "usage": { - "input_tokens": 3412, + "input_tokens": 3408, "output_tokens": 285 }, - "latency_ms": 137, + "latency_ms": 153, "questions": { "change_kind": { "label": "change kind", @@ -2170,12 +2150,12 @@ "choice": "bugfix", "confidence": 1.0, "probabilities": { + "dependency": 0.0, "tests": 0.0, - "bugfix": 1.0, "feature": 0.0, - "refactor": 0.0, "docs": 0.0, - "dependency": 0.0, + "bugfix": 1.0, + "refactor": 0.0, "config": 0.0 } }, @@ -2190,14 +2170,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.82, + "confidence": 0.84, "probabilities": { - "compatibility": 0.01, "deployment": 0.0, + "tests": 0.12, + "correctness": 0.87, "security": 0.0, "readability": 0.0, - "correctness": 0.85, - "tests": 0.14 + "compatibility": 0.01 } }, "credit": null, @@ -2210,8 +2190,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.69, - "confidence": 0.54, + "score": 1.67, + "confidence": 0.51, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -2219,12 +2199,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.3, - "2": 0.7 + "1": 0.32, + "2": 0.68 } }, - "credit": 0.845, - "adds_to_load": 0.1268 + "credit": 0.835, + "adds_to_load": 0.1253 }, "design_decisions": { "label": "design decisions", @@ -2233,12 +2213,10 @@ "inverted": false, "returned": { "type": "score", - "score": 1.91, - "confidence": 0.86, + "score": 1.9, + "confidence": 0.85, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, @@ -2248,8 +2226,8 @@ "2": 0.91 } }, - "credit": 0.955, - "adds_to_load": 0.1337 + "credit": 0.95, + "adds_to_load": 0.133 }, "description_quality": { "label": "description quality", @@ -2258,8 +2236,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.89, - "confidence": 0.83, + "score": 1.87, + "confidence": 0.8, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -2267,12 +2245,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.11, - "2": 0.89 + "1": 0.13, + "2": 0.87 } }, - "credit": 0.055, - "adds_to_load": 0.0044 + "credit": 0.065, + "adds_to_load": 0.0052 }, "security_surface": { "label": "security surface", @@ -2293,10 +2271,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.37 + "noul": 0.4 }, - "credit": 0.63, - "adds_to_load": 0.0756 + "credit": 0.6, + "adds_to_load": 0.072 }, "breaking_change": { "label": "breaking change", @@ -2341,10 +2319,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.98 + "noul": 0.97 }, - "credit": 0.02, - "adds_to_load": 0.0016 + "credit": 0.03, + "adds_to_load": 0.0024 } } }, @@ -2356,7 +2334,7 @@ "changed_files": 4, "additions": 103, "deletions": 8, - "review_load": 0.3598, + "review_load": 0.3551, "tier": "low", "size_floor": "trivial", "raised_by_size": false, @@ -2367,10 +2345,10 @@ "blast radius" ], "usage": { - "input_tokens": 3672, + "input_tokens": 3668, "output_tokens": 285 }, - "latency_ms": 154, + "latency_ms": 141, "questions": { "change_kind": { "label": "change kind", @@ -2383,12 +2361,12 @@ "confidence": 1.0, "probabilities": { "bugfix": 1.0, - "refactor": 0.0, + "tests": 0.0, "docs": 0.0, - "feature": 0.0, - "config": 0.0, "dependency": 0.0, - "tests": 0.0 + "refactor": 0.0, + "config": 0.0, + "feature": 0.0 } }, "credit": null, @@ -2402,14 +2380,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.83, + "confidence": 0.82, "probabilities": { - "readability": 0.0, - "compatibility": 0.01, - "tests": 0.13, + "tests": 0.14, "security": 0.0, - "correctness": 0.86, - "deployment": 0.0 + "deployment": 0.0, + "correctness": 0.84, + "compatibility": 0.02, + "readability": 0.0 } }, "credit": null, @@ -2445,23 +2423,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.7, - "confidence": 0.55, + "score": 1.6, + "confidence": 0.4, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { "0": 0.01, - "1": 0.29, - "2": 0.7 + "1": 0.39, + "2": 0.6 } }, - "credit": 0.85, - "adds_to_load": 0.119 + "credit": 0.8, + "adds_to_load": 0.112 }, "description_quality": { "label": "description quality", @@ -2493,10 +2469,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.19 + "noul": 0.18 }, - "credit": 0.19, - "adds_to_load": 0.0285 + "credit": 0.18, + "adds_to_load": 0.027 }, "mechanical": { "label": "mechanical", @@ -2505,10 +2481,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.28 + "noul": 0.25 }, - "credit": 0.72, - "adds_to_load": 0.0864 + "credit": 0.75, + "adds_to_load": 0.09 }, "breaking_change": { "label": "breaking change", @@ -2517,10 +2493,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.29 + "noul": 0.3 }, - "credit": 0.29, - "adds_to_load": 0.0319 + "credit": 0.3, + "adds_to_load": 0.033 }, "scope_creep": { "label": "scope creep", @@ -2529,10 +2505,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.15 + "noul": 0.14 }, - "credit": 0.15, - "adds_to_load": 0.0135 + "credit": 0.14, + "adds_to_load": 0.0126 }, "infra_surface": { "label": "infra surface", @@ -2561,28 +2537,28 @@ } }, { - "number": 73718, - "title": "Fix merge_dicts crash when overwriting a non-dict value with a dict", - "url": "https://github.com/apache/airflow/pull/73718", - "author": "ccoliu", + "number": 73709, + "title": "Avoid repeated KubernetesExecutor pod deletion for duplicate events", + "url": "https://github.com/apache/airflow/pull/73709", + "author": "Sanjays2402", "changed_files": 2, - "additions": 14, - "deletions": 2, - "review_load": 0.3049, + "additions": 113, + "deletions": 30, + "review_load": 0.2955, "tier": "low", "size_floor": "trivial", "raised_by_size": false, "diff_coverage": 1.0, "drivers": [ + "design decisions", "blast radius", - "mechanical", - "design decisions" + "mechanical" ], "usage": { - "input_tokens": 1831, + "input_tokens": 4404, "output_tokens": 285 }, - "latency_ms": 117, + "latency_ms": 179, "questions": { "change_kind": { "label": "change kind", @@ -2594,13 +2570,13 @@ "choice": "bugfix", "confidence": 1.0, "probabilities": { - "feature": 0.0, - "bugfix": 1.0, "docs": 0.0, "refactor": 0.0, - "tests": 0.0, "config": 0.0, - "dependency": 0.0 + "bugfix": 1.0, + "dependency": 0.0, + "tests": 0.0, + "feature": 0.0 } }, "credit": null, @@ -2614,14 +2590,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.77, + "confidence": 0.88, "probabilities": { "readability": 0.0, - "tests": 0.15, - "compatibility": 0.04, + "compatibility": 0.01, + "tests": 0.09, "security": 0.0, - "deployment": 0.0, - "correctness": 0.81 + "correctness": 0.9, + "deployment": 0.0 } }, "credit": null, @@ -2634,8 +2610,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.81, - "confidence": 0.71, + "score": 1.1, + "confidence": 0.82, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -2643,12 +2619,12 @@ }, "probabilities": { "0": 0.01, - "1": 0.17, - "2": 0.82 + "1": 0.88, + "2": 0.11 } }, - "credit": 0.905, - "adds_to_load": 0.1358 + "credit": 0.55, + "adds_to_load": 0.0825 }, "design_decisions": { "label": "design decisions", @@ -2657,23 +2633,21 @@ "inverted": false, "returned": { "type": "score", - "score": 0.94, - "confidence": 0.82, + "score": 1.47, + "confidence": 0.28, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.09, - "1": 0.88, - "2": 0.03 + "0": 0.01, + "1": 0.52, + "2": 0.47 } }, - "credit": 0.47, - "adds_to_load": 0.0658 + "credit": 0.735, + "adds_to_load": 0.1029 }, "description_quality": { "label": "description quality", @@ -2717,10 +2691,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.35 + "noul": 0.38 }, - "credit": 0.65, - "adds_to_load": 0.078 + "credit": 0.62, + "adds_to_load": 0.0744 }, "breaking_change": { "label": "breaking change", @@ -2729,10 +2703,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.12 + "noul": 0.06 }, - "credit": 0.12, - "adds_to_load": 0.0132 + "credit": 0.06, + "adds_to_load": 0.0066 }, "scope_creep": { "label": "scope creep", @@ -2741,10 +2715,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.05 + "noul": 0.22 }, - "credit": 0.05, - "adds_to_load": 0.0045 + "credit": 0.22, + "adds_to_load": 0.0198 }, "infra_surface": { "label": "infra surface", @@ -2753,10 +2727,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.02 + "noul": 0.04 }, - "credit": 0.02, - "adds_to_load": 0.0016 + "credit": 0.04, + "adds_to_load": 0.0032 }, "has_tests": { "label": "has tests", @@ -2773,28 +2747,28 @@ } }, { - "number": 73709, - "title": "Avoid repeated KubernetesExecutor pod deletion for duplicate events", - "url": "https://github.com/apache/airflow/pull/73709", - "author": "Sanjays2402", + "number": 73718, + "title": "Fix merge_dicts crash when overwriting a non-dict value with a dict", + "url": "https://github.com/apache/airflow/pull/73718", + "author": "ccoliu", "changed_files": 2, - "additions": 113, - "deletions": 30, - "review_load": 0.298, + "additions": 14, + "deletions": 2, + "review_load": 0.2951, "tier": "low", "size_floor": "trivial", "raised_by_size": false, "diff_coverage": 1.0, "drivers": [ - "design decisions", "blast radius", - "mechanical" + "mechanical", + "design decisions" ], "usage": { - "input_tokens": 4408, + "input_tokens": 1827, "output_tokens": 285 }, - "latency_ms": 145, + "latency_ms": 122, "questions": { "change_kind": { "label": "change kind", @@ -2806,13 +2780,13 @@ "choice": "bugfix", "confidence": 1.0, "probabilities": { - "docs": 0.0, + "feature": 0.0, "tests": 0.0, - "config": 0.0, "dependency": 0.0, - "feature": 0.0, "bugfix": 1.0, - "refactor": 0.0 + "docs": 0.0, + "refactor": 0.0, + "config": 0.0 } }, "credit": null, @@ -2826,14 +2800,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.91, + "confidence": 0.76, "probabilities": { - "security": 0.0, - "tests": 0.07, - "correctness": 0.92, + "readability": 0.0, "deployment": 0.0, - "compatibility": 0.01, - "readability": 0.0 + "security": 0.0, + "tests": 0.16, + "compatibility": 0.03, + "correctness": 0.81 } }, "credit": null, @@ -2846,8 +2820,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.12, - "confidence": 0.79, + "score": 1.81, + "confidence": 0.71, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -2855,12 +2829,12 @@ }, "probabilities": { "0": 0.01, - "1": 0.86, - "2": 0.13 + "1": 0.17, + "2": 0.82 } }, - "credit": 0.56, - "adds_to_load": 0.084 + "credit": 0.905, + "adds_to_load": 0.1358 }, "design_decisions": { "label": "design decisions", @@ -2869,23 +2843,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.48, - "confidence": 0.26, + "score": 0.9, + "confidence": 0.75, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.01, - "1": 0.51, - "2": 0.48 + "0": 0.13, + "1": 0.84, + "2": 0.03 } }, - "credit": 0.74, - "adds_to_load": 0.1036 + "credit": 0.45, + "adds_to_load": 0.063 }, "description_quality": { "label": "description quality", @@ -2929,10 +2901,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.37 + "noul": 0.4 }, - "credit": 0.63, - "adds_to_load": 0.0756 + "credit": 0.6, + "adds_to_load": 0.072 }, "breaking_change": { "label": "breaking change", @@ -2941,10 +2913,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.06 + "noul": 0.11 }, - "credit": 0.06, - "adds_to_load": 0.0066 + "credit": 0.11, + "adds_to_load": 0.0121 }, "scope_creep": { "label": "scope creep", @@ -2953,10 +2925,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.21 + "noul": 0.05 }, - "credit": 0.21, - "adds_to_load": 0.0189 + "credit": 0.05, + "adds_to_load": 0.0045 }, "infra_surface": { "label": "infra surface", @@ -2965,10 +2937,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.04 + "noul": 0.02 }, - "credit": 0.04, - "adds_to_load": 0.0032 + "credit": 0.02, + "adds_to_load": 0.0016 }, "has_tests": { "label": "has tests", @@ -2992,7 +2964,7 @@ "changed_files": 3, "additions": 16, "deletions": 6, - "review_load": 0.2948, + "review_load": 0.2914, "tier": "low", "size_floor": "trivial", "raised_by_size": false, @@ -3003,10 +2975,10 @@ "design decisions" ], "usage": { - "input_tokens": 2288, + "input_tokens": 2284, "output_tokens": 284 }, - "latency_ms": 104, + "latency_ms": 137, "questions": { "change_kind": { "label": "change kind", @@ -3016,15 +2988,15 @@ "returned": { "type": "choice", "choice": "feature", - "confidence": 0.94, + "confidence": 0.95, "probabilities": { "refactor": 0.0, - "feature": 0.95, - "tests": 0.0, - "dependency": 0.0, + "bugfix": 0.04, "docs": 0.0, + "feature": 0.96, + "dependency": 0.0, "config": 0.0, - "bugfix": 0.05 + "tests": 0.0 } }, "credit": null, @@ -3038,14 +3010,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.4, + "confidence": 0.41, "probabilities": { "readability": 0.0, - "correctness": 0.51, - "tests": 0.39, - "deployment": 0.0, + "tests": 0.38, "compatibility": 0.1, - "security": 0.0 + "security": 0.0, + "deployment": 0.0, + "correctness": 0.52 } }, "credit": null, @@ -3082,18 +3054,16 @@ "returned": { "type": "score", "score": 0.94, - "confidence": 0.67, + "confidence": 0.71, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.14, - "1": 0.78, - "2": 0.08 + "0": 0.13, + "1": 0.8, + "2": 0.07 } }, "credit": 0.47, @@ -3141,10 +3111,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.35 + "noul": 0.38 }, - "credit": 0.65, - "adds_to_load": 0.078 + "credit": 0.62, + "adds_to_load": 0.0744 }, "breaking_change": { "label": "breaking change", @@ -3153,10 +3123,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.22 + "noul": 0.23 }, - "credit": 0.22, - "adds_to_load": 0.0242 + "credit": 0.23, + "adds_to_load": 0.0253 }, "scope_creep": { "label": "scope creep", @@ -3177,10 +3147,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.08 + "noul": 0.07 }, - "credit": 0.08, - "adds_to_load": 0.0064 + "credit": 0.07, + "adds_to_load": 0.0056 }, "has_tests": { "label": "has tests", @@ -3204,7 +3174,7 @@ "changed_files": 2, "additions": 89, "deletions": 0, - "review_load": 0.2662, + "review_load": 0.2635, "tier": "low", "size_floor": "trivial", "raised_by_size": false, @@ -3215,10 +3185,10 @@ "mechanical" ], "usage": { - "input_tokens": 2683, + "input_tokens": 2679, "output_tokens": 285 }, - "latency_ms": 118, + "latency_ms": 152, "questions": { "change_kind": { "label": "change kind", @@ -3230,13 +3200,13 @@ "choice": "bugfix", "confidence": 0.96, "probabilities": { - "bugfix": 0.97, - "refactor": 0.0, - "feature": 0.03, - "tests": 0.0, "config": 0.0, + "tests": 0.0, "docs": 0.0, - "dependency": 0.0 + "dependency": 0.0, + "refactor": 0.0, + "bugfix": 0.96, + "feature": 0.04 } }, "credit": null, @@ -3250,14 +3220,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.88, + "confidence": 0.84, "probabilities": { - "compatibility": 0.01, "deployment": 0.0, - "security": 0.0, - "tests": 0.09, + "tests": 0.13, "readability": 0.0, - "correctness": 0.9 + "compatibility": 0.01, + "security": 0.0, + "correctness": 0.86 } }, "credit": null, @@ -3270,8 +3240,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.04, - "confidence": 0.92, + "score": 1.06, + "confidence": 0.89, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -3279,12 +3249,12 @@ }, "probabilities": { "0": 0.01, - "1": 0.94, - "2": 0.05 + "1": 0.92, + "2": 0.07 } }, - "credit": 0.52, - "adds_to_load": 0.078 + "credit": 0.53, + "adds_to_load": 0.0795 }, "design_decisions": { "label": "design decisions", @@ -3293,23 +3263,21 @@ "inverted": false, "returned": { "type": "score", - "score": 1.23, - "confidence": 0.63, + "score": 1.16, + "confidence": 0.73, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { "0": 0.01, - "1": 0.75, - "2": 0.24 + "1": 0.82, + "2": 0.17 } }, - "credit": 0.615, - "adds_to_load": 0.0861 + "credit": 0.58, + "adds_to_load": 0.0812 }, "description_quality": { "label": "description quality", @@ -3353,10 +3321,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.43 + "noul": 0.41 }, - "credit": 0.57, - "adds_to_load": 0.0684 + "credit": 0.59, + "adds_to_load": 0.0708 }, "breaking_change": { "label": "breaking change", @@ -3377,10 +3345,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.12 + "noul": 0.11 }, - "credit": 0.12, - "adds_to_load": 0.0108 + "credit": 0.11, + "adds_to_load": 0.0099 }, "infra_surface": { "label": "infra surface", @@ -3389,10 +3357,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.03 + "noul": 0.02 }, - "credit": 0.03, - "adds_to_load": 0.0024 + "credit": 0.02, + "adds_to_load": 0.0016 }, "has_tests": { "label": "has tests", @@ -3416,20 +3384,19 @@ "changed_files": 1, "additions": 2, "deletions": 1, - "review_load": 0.2045, + "review_load": 0.1924, "tier": "trivial", "size_floor": "trivial", "raised_by_size": false, "diff_coverage": 1.0, "drivers": [ - "has tests", - "mechanical" + "has tests" ], "usage": { - "input_tokens": 1329, + "input_tokens": 1325, "output_tokens": 284 }, - "latency_ms": 122, + "latency_ms": 154, "questions": { "change_kind": { "label": "change kind", @@ -3439,15 +3406,15 @@ "returned": { "type": "choice", "choice": "docs", - "confidence": 0.98, + "confidence": 0.97, "probabilities": { - "tests": 0.0, - "config": 0.01, - "bugfix": 0.0, "feature": 0.0, + "bugfix": 0.0, + "docs": 0.98, "refactor": 0.0, "dependency": 0.0, - "docs": 0.99 + "tests": 0.0, + "config": 0.02 } }, "credit": null, @@ -3461,14 +3428,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.47, + "confidence": 0.51, "probabilities": { - "deployment": 0.0, + "readability": 0.36, + "correctness": 0.59, + "compatibility": 0.03, "security": 0.0, - "tests": 0.02, - "correctness": 0.56, - "readability": 0.38, - "compatibility": 0.03 + "deployment": 0.0, + "tests": 0.02 } }, "credit": null, @@ -3504,23 +3471,21 @@ "inverted": false, "returned": { "type": "score", - "score": 0.61, - "confidence": 0.41, + "score": 0.5, + "confidence": 0.26, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.39, - "1": 0.61, + "0": 0.51, + "1": 0.49, "2": 0.0 } }, - "credit": 0.305, - "adds_to_load": 0.0427 + "credit": 0.25, + "adds_to_load": 0.035 }, "description_quality": { "label": "description quality", @@ -3529,8 +3494,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.5, - "confidence": 0.25, + "score": 1.48, + "confidence": 0.28, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -3538,12 +3503,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.5, - "2": 0.5 + "1": 0.52, + "2": 0.48 } }, - "credit": 0.25, - "adds_to_load": 0.02 + "credit": 0.26, + "adds_to_load": 0.0208 }, "security_surface": { "label": "security surface", @@ -3564,10 +3529,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.57 + "noul": 0.62 }, - "credit": 0.43, - "adds_to_load": 0.0516 + "credit": 0.38, + "adds_to_load": 0.0456 }, "breaking_change": { "label": "breaking change", @@ -3612,10 +3577,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.03 + "noul": 0.02 }, - "credit": 0.97, - "adds_to_load": 0.0776 + "credit": 0.98, + "adds_to_load": 0.0784 } } }, @@ -3627,7 +3592,7 @@ "changed_files": 4, "additions": 32, "deletions": 8, - "review_load": 0.1764, + "review_load": 0.1777, "tier": "trivial", "size_floor": "trivial", "raised_by_size": false, @@ -3636,10 +3601,10 @@ "blast radius" ], "usage": { - "input_tokens": 2884, + "input_tokens": 2880, "output_tokens": 285 }, - "latency_ms": 262, + "latency_ms": 274, "questions": { "change_kind": { "label": "change kind", @@ -3651,13 +3616,13 @@ "choice": "bugfix", "confidence": 1.0, "probabilities": { + "docs": 0.0, "refactor": 0.0, + "config": 0.0, "feature": 0.0, - "docs": 0.0, "bugfix": 1.0, - "dependency": 0.0, - "config": 0.0, - "tests": 0.0 + "tests": 0.0, + "dependency": 0.0 } }, "credit": null, @@ -3671,14 +3636,14 @@ "returned": { "type": "choice", "choice": "correctness", - "confidence": 0.73, + "confidence": 0.68, "probabilities": { + "correctness": 0.73, "readability": 0.0, - "correctness": 0.78, "compatibility": 0.01, "security": 0.0, "deployment": 0.0, - "tests": 0.21 + "tests": 0.26 } }, "credit": null, @@ -3691,8 +3656,8 @@ "inverted": false, "returned": { "type": "score", - "score": 1.15, - "confidence": 0.77, + "score": 1.13, + "confidence": 0.8, "legend": { "0": "Confined to one file, or to documentation", "1": "Confined to one module or feature area", @@ -3700,12 +3665,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.85, - "2": 0.15 + "1": 0.87, + "2": 0.13 } }, - "credit": 0.575, - "adds_to_load": 0.0862 + "credit": 0.565, + "adds_to_load": 0.0847 }, "design_decisions": { "label": "design decisions", @@ -3715,17 +3680,15 @@ "returned": { "type": "score", "score": 0.28, - "confidence": 0.57, + "confidence": 0.58, "legend": { - "0": { - "None": "the same edit repeated, or a value changed" - }, + "0": "None: the same edit repeated, or a value changed", "1": "Local choices inside one function or file", "2": "New abstractions, interfaces, or control flow" }, "probabilities": { - "0": 0.73, - "1": 0.25, + "0": 0.74, + "1": 0.24, "2": 0.02 } }, @@ -3739,8 +3702,8 @@ "inverted": true, "returned": { "type": "score", - "score": 1.38, - "confidence": 0.43, + "score": 1.39, + "confidence": 0.41, "legend": { "0": "Empty, or says nothing about what changed", "1": "Names what changed", @@ -3748,12 +3711,12 @@ }, "probabilities": { "0": 0.0, - "1": 0.62, - "2": 0.38 + "1": 0.61, + "2": 0.39 } }, - "credit": 0.31, - "adds_to_load": 0.0248 + "credit": 0.305, + "adds_to_load": 0.0244 }, "security_surface": { "label": "security surface", @@ -3774,10 +3737,10 @@ "inverted": true, "returned": { "type": "noul", - "noul": 0.88 + "noul": 0.87 }, - "credit": 0.12, - "adds_to_load": 0.0144 + "credit": 0.13, + "adds_to_load": 0.0156 }, "breaking_change": { "label": "breaking change", @@ -3786,10 +3749,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.13 + "noul": 0.14 }, - "credit": 0.13, - "adds_to_load": 0.0143 + "credit": 0.14, + "adds_to_load": 0.0154 }, "scope_creep": { "label": "scope creep", @@ -3798,10 +3761,10 @@ "inverted": false, "returned": { "type": "noul", - "noul": 0.07 + "noul": 0.08 }, - "credit": 0.07, - "adds_to_load": 0.0063 + "credit": 0.08, + "adds_to_load": 0.0072 }, "infra_surface": { "label": "infra surface", diff --git a/samples/pr-triage/data/apache-airflow-open-latest-18-triage.md b/samples/pr-triage/data/apache-airflow-open-latest-18-triage.md index dfca989..9a75473 100644 --- a/samples/pr-triage/data/apache-airflow-open-latest-18-triage.md +++ b/samples/pr-triage/data/apache-airflow-open-latest-18-triage.md @@ -1,28 +1,28 @@ # Triage: apache/airflow 18 pull requests, 11 questions each, one call apiece, on 25 September 2026 with `jev-1.13.0`. -87,555 input tokens, $0.00368 at $0.042 per million. +87,483 input tokens, $0.00367 at $0.042 per million. -| PR | Files | Lines | Kind | Load | Tier | Review focus | Title | -| ------ | ----- | ---------- | ------- | ---- | ------------------ | ------------ | ---------------------------------------------------------- | -| #73698 | 8 | +626/-73 | bugfix | 0.63 | high (on a cut) | security | Bind the AWS auth manager SAML response to the browser ... | -| #73696 | 7 | +136/-5 | bugfix | 0.53 | medium | security | Check admin-only views against a dedicated Keycloak res... | -| #73704 | 9 | +57/-32 | bugfix | 0.51 | medium | correctness | Fix socket leaks and add missing request timeouts acros... | -| #73706 | 17 | +2634/-117 | bugfix | 0.50 | high (size) | correctness | Account for AgentOperator spend on failed runs and acro... | -| #73701 | 4 | +360/-39 | feature | 0.48 | medium | correctness | Add durable reconnect option to EmrServerlessStartJobOp... | -| #73703 | 5 | +64/-0 | feature | 0.44 | medium | correctness | [chart/v1-2x-test] Helm: Allow hostAliases and log groo... | -| #73723 | 10 | +558/-121 | feature | 0.43 | medium (on a cut) | correctness | TS SDK: embed one source region per native Dag file | -| #73702 | 4 | +291/-3 | feature | 0.41 | low (on a cut) | correctness | Stop AWS Glue job run when a deferred task is cleared | -| #73713 | 54 | +467/-131 | feature | 0.38 | high (size) | correctness | UI: Group digits of counters according to the selected ... | -| #73724 | 4 | +209/-12 | bugfix | 0.38 | low | correctness | Fix Grid and Graph 500 errors for cyclic TaskGroup depe... | -| #73720 | 2 | +98/-6 | bugfix | 0.37 | low | correctness | [v3-3-test] Raise DeadlockImminentError for sync comms ... | -| #73719 | 4 | +103/-8 | bugfix | 0.36 | low | correctness | Fix masked failure reason for deferrable EMR Serverless... | -| #73718 | 2 | +14/-2 | bugfix | 0.30 | low | correctness | Fix merge_dicts crash when overwriting a non-dict value... | -| #73709 | 2 | +113/-30 | bugfix | 0.30 | low | correctness | Avoid repeated KubernetesExecutor pod deletion for dupl... | -| #73717 | 3 | +16/-6 | feature | 0.29 | low | correctness | add bundle_name to dag processor timeouts metric | -| #73711 | 2 | +89/-0 | bugfix | 0.27 | low | correctness | Emit queued_duration metric when a task enters RUNNING ... | -| #73722 | 1 | +2/-1 | docs | 0.20 | trivial (on a cut) | correctness | Clarify max_db_retries doc wording to avoid off-by-one ... | -| #73725 | 4 | +32/-8 | bugfix | 0.18 | trivial | correctness | Fix Elasticsearch and OpenSearch response wrapper bugs | +| PR | Files | Lines | Kind | Load | Tier | Review focus | Title | +| ------ | ----- | ---------- | ------- | ---- | ----------------- | ------------ | ---------------------------------------------------------- | +| #73698 | 8 | +626/-73 | bugfix | 0.63 | high (on a cut) | security | Bind the AWS auth manager SAML response to the browser ... | +| #73696 | 7 | +136/-5 | bugfix | 0.53 | medium | security | Check admin-only views against a dedicated Keycloak res... | +| #73706 | 17 | +2634/-117 | bugfix | 0.51 | high (size) | correctness | Account for AgentOperator spend on failed runs and acro... | +| #73704 | 9 | +57/-32 | bugfix | 0.50 | medium | correctness | Fix socket leaks and add missing request timeouts acros... | +| #73701 | 4 | +360/-39 | feature | 0.46 | medium | correctness | Add durable reconnect option to EmrServerlessStartJobOp... | +| #73723 | 10 | +558/-121 | feature | 0.44 | medium (on a cut) | correctness | TS SDK: embed one source region per native Dag file | +| #73703 | 5 | +64/-0 | feature | 0.43 | medium (on a cut) | correctness | [chart/v1-2x-test] Helm: Allow hostAliases and log groo... | +| #73702 | 4 | +291/-3 | feature | 0.42 | low (on a cut) | correctness | Stop AWS Glue job run when a deferred task is cleared | +| #73713 | 54 | +467/-131 | feature | 0.37 | high (size) | correctness | UI: Group digits of counters according to the selected ... | +| #73724 | 4 | +209/-12 | bugfix | 0.37 | low | correctness | Fix Grid and Graph 500 errors for cyclic TaskGroup depe... | +| #73720 | 2 | +98/-6 | bugfix | 0.37 | low | correctness | [v3-3-test] Raise DeadlockImminentError for sync comms ... | +| #73719 | 4 | +103/-8 | bugfix | 0.36 | low | correctness | Fix masked failure reason for deferrable EMR Serverless... | +| #73709 | 2 | +113/-30 | bugfix | 0.30 | low | correctness | Avoid repeated KubernetesExecutor pod deletion for dupl... | +| #73718 | 2 | +14/-2 | bugfix | 0.30 | low | correctness | Fix merge_dicts crash when overwriting a non-dict value... | +| #73717 | 3 | +16/-6 | feature | 0.29 | low | correctness | add bundle_name to dag processor timeouts metric | +| #73711 | 2 | +89/-0 | bugfix | 0.26 | low | correctness | Emit queued_duration metric when a task enters RUNNING ... | +| #73722 | 1 | +2/-1 | docs | 0.19 | trivial | correctness | Clarify max_db_retries doc wording to avoid off-by-one ... | +| #73725 | 4 | +32/-8 | bugfix | 0.18 | trivial | correctness | Fix Elasticsearch and OpenSearch response wrapper bugs | Load is the weighted average of nine questions, 0 to 1. Tier comes from that load, raised when size demands it: over 30 files or 1,500 lines is high whatever Jev returned. @@ -32,10 +32,10 @@ a human reads this line by line, and the author walks them through it - [#73698](https://github.com/apache/airflow/pull/73698) load 0.63: Bind the AWS auth manager SAML response to the browser that started the login - drivers: security surface, design decisions, mechanical -- [#73706](https://github.com/apache/airflow/pull/73706) load 0.50, raised by the size floor: Account for AgentOperator spend on failed runs and across retries +- [#73706](https://github.com/apache/airflow/pull/73706) load 0.51, raised by the size floor: Account for AgentOperator spend on failed runs and across retries - drivers: design decisions, blast radius, breaking change - read from 47% of the changed files, so the load is a read on part of the diff -- [#73713](https://github.com/apache/airflow/pull/73713) load 0.38, raised by the size floor: UI: Group digits of counters according to the selected locale +- [#73713](https://github.com/apache/airflow/pull/73713) load 0.37, raised by the size floor: UI: Group digits of counters according to the selected locale - drivers: blast radius, design decisions, scope creep - read from 44% of the changed files, so the load is a read on part of the diff @@ -45,41 +45,41 @@ one reviewer who knows this area, reading the whole diff - [#73696](https://github.com/apache/airflow/pull/73696) load 0.53: Check admin-only views against a dedicated Keycloak resource in multi-team mode - drivers: security surface, design decisions, mechanical -- [#73704](https://github.com/apache/airflow/pull/73704) load 0.51: Fix socket leaks and add missing request timeouts across providers - - drivers: blast radius, security surface, has tests -- [#73701](https://github.com/apache/airflow/pull/73701) load 0.48: Add durable reconnect option to EmrServerlessStartJobOperator - - drivers: design decisions, mechanical, security surface -- [#73703](https://github.com/apache/airflow/pull/73703) load 0.44: [chart/v1-2x-test] Helm: Allow hostAliases and log groomer lifecycle hooks on the Dag processor (#73159) +- [#73704](https://github.com/apache/airflow/pull/73704) load 0.50: Fix socket leaks and add missing request timeouts across providers + - drivers: security surface, blast radius, has tests +- [#73701](https://github.com/apache/airflow/pull/73701) load 0.46: Add durable reconnect option to EmrServerlessStartJobOperator + - drivers: design decisions, mechanical, blast radius +- [#73723](https://github.com/apache/airflow/pull/73723) load 0.44: TS SDK: embed one source region per native Dag file - drivers: design decisions, blast radius, mechanical -- [#73723](https://github.com/apache/airflow/pull/73723) load 0.43: TS SDK: embed one source region per native Dag file - - drivers: design decisions, blast radius, breaking change +- [#73703](https://github.com/apache/airflow/pull/73703) load 0.43: [chart/v1-2x-test] Helm: Allow hostAliases and log groomer lifecycle hooks on the Dag processor (#73159) + - drivers: design decisions, mechanical, blast radius ## low (8) one reviewer, one pass, no meeting -- [#73702](https://github.com/apache/airflow/pull/73702) load 0.41: Stop AWS Glue job run when a deferred task is cleared +- [#73702](https://github.com/apache/airflow/pull/73702) load 0.42: Stop AWS Glue job run when a deferred task is cleared - drivers: design decisions, mechanical, blast radius -- [#73724](https://github.com/apache/airflow/pull/73724) load 0.38: Fix Grid and Graph 500 errors for cyclic TaskGroup dependencies +- [#73724](https://github.com/apache/airflow/pull/73724) load 0.37: Fix Grid and Graph 500 errors for cyclic TaskGroup dependencies - drivers: design decisions, blast radius, mechanical - [#73720](https://github.com/apache/airflow/pull/73720) load 0.37: [v3-3-test] Raise DeadlockImminentError for sync comms calls from a paused event loop thread (#73521) - drivers: design decisions, blast radius, mechanical - [#73719](https://github.com/apache/airflow/pull/73719) load 0.36: Fix masked failure reason for deferrable EMR Serverless jobs - drivers: design decisions, mechanical, blast radius -- [#73718](https://github.com/apache/airflow/pull/73718) load 0.30: Fix merge_dicts crash when overwriting a non-dict value with a dict - - drivers: blast radius, mechanical, design decisions - [#73709](https://github.com/apache/airflow/pull/73709) load 0.30: Avoid repeated KubernetesExecutor pod deletion for duplicate events - drivers: design decisions, blast radius, mechanical +- [#73718](https://github.com/apache/airflow/pull/73718) load 0.30: Fix merge_dicts crash when overwriting a non-dict value with a dict + - drivers: blast radius, mechanical, design decisions - [#73717](https://github.com/apache/airflow/pull/73717) load 0.29: add bundle_name to dag processor timeouts metric - drivers: blast radius, mechanical, design decisions -- [#73711](https://github.com/apache/airflow/pull/73711) load 0.27: Emit queued_duration metric when a task enters RUNNING via the execution API +- [#73711](https://github.com/apache/airflow/pull/73711) load 0.26: Emit queued_duration metric when a task enters RUNNING via the execution API - drivers: design decisions, blast radius, mechanical ## trivial (2) merge on a glance: read the title, skim the diff, check that CI is green -- [#73722](https://github.com/apache/airflow/pull/73722) load 0.20: Clarify max_db_retries doc wording to avoid off-by-one confusion - - drivers: has tests, mechanical +- [#73722](https://github.com/apache/airflow/pull/73722) load 0.19: Clarify max_db_retries doc wording to avoid off-by-one confusion + - drivers: has tests - [#73725](https://github.com/apache/airflow/pull/73725) load 0.18: Fix Elasticsearch and OpenSearch response wrapper bugs - drivers: blast radius diff --git a/samples/pr-triage/go/questions.yml b/samples/pr-triage/go/questions.yml index 9c7e081..542958c 100644 --- a/samples/pr-triage/go/questions.yml +++ b/samples/pr-triage/go/questions.yml @@ -83,7 +83,7 @@ questions: weight: 0.14 instructions: How many judgment calls a reviewer would have to agree with criteria: - - None: the same edit repeated, or a value changed + - "None: the same edit repeated, or a value changed" - Local choices inside one function or file - New abstractions, interfaces, or control flow diff --git a/samples/pr-triage/questions.yml b/samples/pr-triage/questions.yml index 9c7e081..542958c 100644 --- a/samples/pr-triage/questions.yml +++ b/samples/pr-triage/questions.yml @@ -83,7 +83,7 @@ questions: weight: 0.14 instructions: How many judgment calls a reviewer would have to agree with criteria: - - None: the same edit repeated, or a value changed + - "None: the same edit repeated, or a value changed" - Local choices inside one function or file - New abstractions, interfaces, or control flow