From c9780aa4efc31d2f316e0a2353d9091508f80987 Mon Sep 17 00:00:00 2001 From: Nils Lehnen <30603423+iderex@users.noreply.github.com> Date: Sun, 16 Aug 2026 23:45:49 +0200 Subject: [PATCH] Say which contexts arrive on which pull requests (#62) The parity document named the target's context strings and left two of them ambiguous here. `CodeQL` and `Audit workflows (zizmor)` are each two strings on this board: a job reports one and a code-scanning upload reports the other, and only one of the two arrives on a pull request whose token is read-only. A required set assembled from the rows as they stood could have taken the upload's name, which would hold open every pull request the upload condition excludes while nothing on the pull request said why. The same section carries the other half of the arrival question. The command this document hands issue #26 to assemble the set from reads a commit on the default branch, and three workflows here have no push trigger, so that command omits three contexts every pull request reports and offers one that no pull request can report. Both directions are measured and quoted. Measured rather than reasoned about: the upload step is skipped on a pull request the condition excludes, the step that fails on findings runs anyway, and the upload-derived name is absent from that commit's check runs while the job's name is present. The fork route is still unwalked and the section says so. No pull request from a fork has been opened here, and `CodeQL` arrived on the read-only-token pull request measured, so nothing here says what a fork produces for it. Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com> --- docs/quality-parity.md | 132 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 130 insertions(+), 2 deletions(-) diff --git a/docs/quality-parity.md b/docs/quality-parity.md index ab8b208..bec6f22 100644 --- a/docs/quality-parity.md +++ b/docs/quality-parity.md @@ -53,13 +53,13 @@ today no tick does. | `ABI floor build` | Dropped | It exists so a plugin loads against the oldest server it claims to support, and nothing here loads into a host. | | `Package (JPRM) / Build package` | Dropped | This repository ships no plugin, so there is no package to build. | | `Package (JPRM) / Generate SBOM` | Kept in substance, moved out of the gate | A bill of materials is owed for anything downloadable, and this one is produced by the release build rather than by a check on a pull request. Issue #37 builds it and nothing in this tree produces one today. | -| `CodeQL` | Kept, retargeted | Static analysis of the runner's own source, in the language record `0001` chose rather than in C#. | +| `CodeQL` | Kept, retargeted | Static analysis of the runner's own source, in the language record `0001` chose rather than in C#. Two strings arrive here for it, the job's and the code-scanning upload's, and the section on which contexts arrive says which of the two a required set can hold. | | `Analyze (csharp)` | Dropped as a name | The language-specific analysis job is replaced by the equivalent for this language rather than carried across under a name that describes nothing here. | | `DCO sign-off` | Kept unchanged | Already in the tree, asserting the text at `DCO` on every non-merge commit. | | `Deterministic PR-hygiene checks` | Kept, adapted | The class the other checks miss is the pull request itself, and one of the three refusals is this board's own invariant about a record moving with the code it describes. It is in the tree as the `pull request` job, judged in `internal/pullrequest/` and run from `.github/workflows/pull-request.yml`. | | `Enforce greppable invariants` | Kept, different invariants | The invariants are properties of this repository's own tracked text, which is a different set from the target's, and they are in `internal/invariants/`. | | `Reject Trojan Source Unicode` | Kept unchanged | Already in the tree, and the attack it refuses is a property of source rather than of a language. | -| `Audit workflows (zizmor)` | Kept unchanged | Already in the tree. The workflow YAML is the other executable thing here and it runs with write scopes. | +| `Audit workflows (zizmor)` | Kept unchanged | Already in the tree. The workflow YAML is the other executable thing here and it runs with write scopes. The job reports under this name and the code-scanning upload reports under `zizmor`, which is a different context and does not arrive on every pull request. | | `prettier` | Kept, split in two | Records here are Markdown, and a whitespace diff on a record hides the sentence that changed. The runner's own source is held to `gofmt` by a job already in the tree; the prose half is issue #50 and nothing in this tree formats Markdown today. | | `dependency-review` | Kept unchanged | Already in the tree, refusing a newly introduced dependency carrying a known advisory. | @@ -94,6 +94,134 @@ gh api repos/Flowfin/lab/commits/$(git rev-parse origin/main)/check-runs \ That is the command issue #26 assembles the required set from, and it is the one to run before quoting any context name back at this document. +## Which contexts arrive, and on which pull requests + +A required context that does not arrive is not a red tick. It is a pull request +that cannot merge and says nothing about why, so the first response is to wait +and the second is to make the gate smaller. The names above are therefore +separated by what produces them and by the commit they were read from before +any of them goes into a required set. Issue #62 holds this walk. + +### The command above reads a commit no pull request produced + +`git rev-parse origin/main` resolves a commit on the default branch, which is +reached by a push. Three workflows in this tree carry no push trigger: + +``` +git grep -L 'push:' origin/main -- .github/workflows/ +origin/main:.github/workflows/dco.yml +origin/main:.github/workflows/dependency-review.yml +origin/main:.github/workflows/pull-request.yml +``` + +So the two kinds of commit report different sets, and the difference runs in +both directions. Between the head of the default branch and the head of a pull +request that landed on it, on 2026-08-16: + +``` +gh api repos/Flowfin/lab/commits/82c245f/check-runs?per_page=100 \ + --jq '.check_runs[].name' | sort -u > default-branch.txt +gh api repos/Flowfin/lab/commits/7374b6b/check-runs?per_page=100 \ + --jq '.check_runs[].name' | sort -u > pull-request.txt + +comm -23 default-branch.txt pull-request.txt +Scorecard analysis + +comm -13 default-branch.txt pull-request.txt +CodeQL +DCO sign-off +dependency-review +pull request +zizmor +``` + +Three of those five are the pull-request-only workflows above and two are the +subject of the next section. `Scorecard analysis` is the opposite case and is +already written down as a permanent absence in `internal/contexts/contexts.go`, +for the reason the supply-chain paragraph above gives. + +A set assembled from a default-branch commit therefore leaves out three +contexts that every pull request reports, and it offers one that no pull +request can report. Which commit that command is run against decides what the +gate gets. + +### Two of the names come from an upload rather than from a job + +The names on that same pull request head that no job in this tree produced: + +``` +gh api repos/Flowfin/lab/commits/7374b6b/check-runs?per_page=100 \ + --jq '.check_runs[] | select(.app.slug != "github-actions") + | "\(.name) is created by \(.app.slug)"' | sort -u +CodeQL is created by github-advanced-security +zizmor is created by github-advanced-security +``` + +The two rows in the table above carry the target's strings. Here each of them +is two strings rather than one: the analysis job reports `CodeQL (go)` and the +code-scanning upload reports `CodeQL`, the audit job reports +`Audit workflows (zizmor)` and the upload reports `zizmor`. Two of the four are +written in a workflow file and two are written in no file in this tree, which +is how `internal/contexts/contexts.go` holds them. + +### The upload is conditional and the step that fails on findings is not + +``` +git show origin/main:.github/workflows/zizmor.yml | sed -n '76,84p' + - name: Upload SARIF + # Only upload where the GITHUB_TOKEN can write security events: pushes to main + # and same-repo human PRs. Fork and Dependabot pull requests run with a + # read-only token, so the upload is skipped there - the gate step below still + # runs and blocks on findings. continue-on-error keeps the security gate + # independent of the upload: a transient code-scanning upload failure must not + # skip the "Fail on actionable findings" step below. + if: (github.event_name == 'push' && github.ref == 'refs/heads/main') || (github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.user.login != 'dependabot[bot]') + continue-on-error: true +``` + +That condition has a pull request on this board today that it excludes, so the +outcome is measured rather than read off the file. On the head of #135, whose +author the second arm names: + +``` +gh api repos/Flowfin/lab/actions/runs/31929377858/jobs \ + --jq '.jobs[].steps[] | select(.number >= 4 and .number <= 6) + | "step \(.number), \(.name): \(.conclusion)"' +step 4, Audit workflows (SARIF for code scanning): success +step 5, Upload SARIF: skipped +step 6, Fail on actionable findings: failure +``` + +The upload did not run, and only one of the two names reached that commit: + +``` +gh api repos/Flowfin/lab/commits/d18d040/check-runs?per_page=100 \ + --jq '.check_runs[] | select(.name | test("zizmor")) + | "\(.name): \(.conclusion)"' | sort -u +Audit workflows (zizmor): failure +``` + +So the job arrived and reported what the audit found while the upload-derived +context did not arrive at all, which is the separation the comment in the +workflow file argues for. It decides one name: `zizmor` cannot be in the +required set, because requiring it would hold open every pull request the +condition excludes, with nothing on the pull request saying why. That absence is +permanent rather than one issue #26 retires, and the job name beside it is +unaffected. + +### What this section does not settle + +No pull request from a fork has been opened here. The condition above has two +arms and only the second was walked: the branch measured above is in this +repository, and a read-only token is what the two arms have in common rather +than what makes them one route. `CodeQL` did arrive on that pull request, so +nothing here says whether it arrives from a fork, and the fork clause of #62 is +open for it. + +The required set is empty, so nothing above is a report of a required context +that failed to arrive. Every sentence here is about which names a set could +hold, and none of them is about a gate that bit. + ## The rest of the ruleset A required set is one rule out of the four that stand behind a merge at the