Skip to content

Say that the zizmor absence outlives the empty required set (#26) - #143

Merged
iderex merged 1 commit into
mainfrom
parity/the-zizmor-absence-rests-on-a-reason-that-ends
Aug 17, 2026
Merged

Say that the zizmor absence outlives the empty required set (#26)#143
iderex merged 1 commit into
mainfrom
parity/the-zizmor-absence-rests-on-a-reason-that-ends

Conversation

@iderex

@iderex iderex commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Refs #26

What this changes

internal/contexts/contexts.go holds the deliberate-absence list, and every
entry says why the name is outside the required set and what retires it. The
zizmor entry carried the shared reason, which is that the ruleset requires no
status check at all today, and named #26 as what retires it.

docs/quality-parity.md decides that one differently. The zizmor context is
created by the code-scanning upload rather than by the job, that step is skipped
wherever the token cannot write security events, and requiring the name would
hold open every pull request the condition excludes with nothing on the pull
request saying why. So the absence is a decision that outlives the empty set
rather than a debt the set being assembled pays off.

The entry now carries that reason and an empty Until, which is the shape
Scorecard analysis already has. A test refuses the two fields disagreeing
again.

What failure it prevents

The change that assembles the required set reads this list to decide which names
move into the ruleset. An entry marked as retired by that change is one the
change is meant to retire. Moving zizmor in would require a context that never
arrives on a fork or Dependabot pull request, and the pull request would then sit
unmergeable with nothing on it saying why, which is the exact failure the
required set is being assembled carefully to avoid.

The test prevents the smaller version of the same thing later. An absence that
turns out to have its own reason is made permanent by emptying Until, and the
sentence above it is easy to leave as it was. That entry then says its decision
outlives the only condition it rests on.

What was run

The two measurements the reason rests on, taken on the head of a pull request
the upload condition excludes rather than read off the condition:

gh api repos/Flowfin/lab/actions/runs/31929377858/jobs \
  --jq '.jobs[].steps[] | select(.number >= 4 and .number <= 6)
        | "step \(.number), \(.name): \(.conclusion)"'
step 4, Audit workflows (SARIF for code scanning): success
step 5, Upload SARIF: skipped
step 6, Fail on actionable findings: failure

gh api repos/Flowfin/lab/commits/d18d040/check-runs?per_page=100 \
  --jq '.check_runs[] | select(.name | test("zizmor";"i"))
        | "\(.name): \(.conclusion)"' | sort -u
Audit workflows (zizmor): failure

The job arrived and reported what the audit found. The upload-derived name did
not arrive at all.

The new test bites, shown by making the one-field mistake it is written for and
running it. Until was emptied on the entry while the shared reason was left in
place, and the entry was restored afterwards:

go test -count=1 -run TestAPermanentAbsenceDoesNotRestOnAReasonThatEnds ./internal/contexts
--- FAIL: TestAPermanentAbsenceDoesNotRestOnAReasonThatEnds (0.00s)
    contexts_test.go:297: the absence "zizmor" is written as permanent and its reason is that the required set is empty, which is the condition that ends when the set is assembled, so the entry says the absence outlives the only thing it rests on
FAIL
FAIL	github.com/Flowfin/lab/internal/contexts	0.334s

The commands CONTRIBUTING.md asks for, at the commit being pushed:

go build ./cmd/... ./internal/...
(no output)

go vet ./cmd/... ./internal/...
(no output)

gofmt -l cmd internal
(no output)

go test -count=1 -v ./cmd/... ./internal/...
ok  	github.com/Flowfin/lab/cmd/contexts
ok  	github.com/Flowfin/lab/cmd/lab
ok  	github.com/Flowfin/lab/cmd/notices
ok  	github.com/Flowfin/lab/cmd/pullrequest
ok  	github.com/Flowfin/lab/internal/check
ok  	github.com/Flowfin/lab/internal/contexts
ok  	github.com/Flowfin/lab/internal/hardware
ok  	github.com/Flowfin/lab/internal/invariants
ok  	github.com/Flowfin/lab/internal/notices
ok  	github.com/Flowfin/lab/internal/prose
ok  	github.com/Flowfin/lab/internal/pullrequest

The verbose run carried the integration-hardware disclosure, which is what says
the run covered less than everything that exists:

go test -count=1 -v ./internal/hardware
the integration-hardware harness was not asked for and nothing in it ran.
    go test -tags integration_hardware ./internal/hardware
with LAB_INTEGRATION_HARDWARE=1 in the environment. its results are about that machine and are not this suite's results.

That harness was not asked for and was not run here.

What this does not do

It does not assemble the required set, which is what #26 is for, and it changes
no repository setting. The ruleset on the default branch still requires no status
check.

It settles one name and leaves the neighbouring one open. CodeQL is also
reported by an upload, and docs/quality-parity.md records its verdict as open
rather than decided, so its entry keeps the shared reason and keeps naming #26.
Nothing here walks a pull request from a fork, so the fork half of #62 is
untouched.

The test compares two fields of one entry. It cannot tell whether a reason
written out in full is true, and nothing in this tree reads
docs/quality-parity.md, so the agreement between that document and this list is
still held by whoever reads them both.

Nothing but its author has read this change, so the commands above stand in place
of a second reader rather than beside one.

The deliberate-absence list gave `zizmor` the shared reason that every name this
tree declares is outside the required set because the set has no members, and it
named #26 as what retires the entry. `docs/quality-parity.md` decides otherwise.
That name is created by the code-scanning upload rather than by the job, the
upload step is skipped wherever the token cannot write security events, and
requiring it would hold open every pull request the condition excludes with
nothing on the pull request saying why. The absence therefore survives the day
the set is assembled instead of being paid off by it.

The entry now carries its own reason and an empty `Until`, which is the shape
`Scorecard analysis` already has for a decision of the same kind. What that
prevents is the change assembling the required set reading this entry as a debt
and moving the name into the ruleset, which blocks every fork and Dependabot
pull request behind a context that cannot arrive on them.

Measured on the head of a pull request the condition excludes rather than read
off the file:

    gh api repos/Flowfin/lab/actions/runs/31929377858/jobs \
      --jq '.jobs[].steps[] | select(.number >= 4 and .number <= 6)
            | "step \(.number), \(.name): \(.conclusion)"'
    step 4, Audit workflows (SARIF for code scanning): success
    step 5, Upload SARIF: skipped
    step 6, Fail on actionable findings: failure

    gh api repos/Flowfin/lab/commits/d18d040/check-runs?per_page=100 \
      --jq '.check_runs[] | select(.name | test("zizmor";"i"))
            | "\(.name): \(.conclusion)"' | sort -u
    Audit workflows (zizmor): failure

A test refuses the pair rather than leaving it to whoever reads the list next.
An entry written as permanent while carrying the reason that ends when the set
is assembled says its decision outlives the only thing it rests on. That is also
the near miss the test is written for, which is one field: emptying `Until`
because an absence turned out to have its own reason, and leaving the sentence
above it as it was.

Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
@iderex
iderex merged commit 1fc6961 into main Aug 17, 2026
25 checks passed
@iderex
iderex deleted the parity/the-zizmor-absence-rests-on-a-reason-that-ends branch August 17, 2026 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant