Repository navigation
Probe a private repository, or the control verifies the wrong thing - #24
Merged
Merged
Conversation
checks: read is enforced on private repositories only, so the positive control added in #23 can report "readable" against a public repo while the gate is blind to every repository it actually works on. The estate's Dependabot pull requests sit overwhelmingly in private repos, so that is not a corner case. The 2026-11-03 run is the proof, and it was misread when #23 was written. The simtabi job examined one pull request and it passed -- on the public simtabi/cli-kit-js, with the same token that answered 403 for every private repository across four other orgs. So "every pull request the gate examined errored" was wrong: the one in a public repo succeeded, and the discriminator was visibility, not the org. Measured against the control as merged: for simtabi and ichava it picks a public repo, so for those two it would have passed while private repos were unreadable. The other six pass only by luck of which repository was pushed most recently, which changes constantly. Sort private-first over a wide enough page, name the visibility that was verified, and where only a public repository was available say that the read does not exercise checks: read. An account with no private repository has no private blindness to detect, so a public probe is complete for it.
imanimanyara
added a commit
that referenced
this pull request
Nov 8, 2026
#24's control enumerated the errors that are NOT permission failures -- 409 and the word "empty" -- and the enumeration was incomplete. An empty repository whose default_branch is nominally main answers 422 "No commit found for SHA" on commits/{ref}/check-runs while /commits answers 409, so the shape never matched. Measured live after #24 merged: simsange reported ok=False cannot read check runs on simsange/documentation@main: gh: No commit found for SHA: main (HTTP 422) because its newest private repository is the empty simsange/documentation. That is a false permission failure, and the gate would have gone red for simsange every quiet week. Inverted: fail only where GitHub says the permission is missing -- 403, or "Resource not accessible by integration" -- and treat anything else as inconclusive, keep looking, and return None if nothing conclusive is found. One shape to recognise instead of an open-ended list of exemptions. Two existing tests were asserting against the wrong thing and are fixed here: they raised FakeGh's generic "no route for ..." error, which is not a 403, so they passed for a reason unrelated to what they claimed to check. They now raise the real message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The positive control added in #23 can pass while the gate is blind.
checks: readis enforced onprivate repositories only, so a probe that lands on a public repo reports "readable" even when
every private repository answers 403 — and the estate's Dependabot pull requests sit
overwhelmingly in private repos.
This sorts candidates private-first over a wide enough page, names the visibility that was
verified, and — where only a public repository was available — says that the read does not exercise
checks: read.#23 misread the evidence, and this is the correction
I claimed "every pull request the gate examined errored" in run 41713055813. That was wrong. The
simtabijob examined one pull request and it passed:A real policy decision, so
check_runssucceeded — with the same token that answered 403 for fourother orgs. The discriminator is visibility, not the org:
simtabi/cli-kit-jssimbi-prjs-clt/{staging.freeby.io,staging.legalmizer.com,swamari.com,theomaris.com}simbi-prjs-int/creamer.simtabi.comtusente/web-apptayariai/{tayariai,gpt,v2}So the run was five vacuous greens (0 pull requests) plus one genuine pass on a public repo,
not six vacuous ones.
Why that breaks the control as merged
Measured against
main, the repositorychecks_readable()picks per org:simtabisimtabi/go-notifykitichavaichava/icon-sets-synclaranail,tusente,tayariai,simbi-prjs-clt,simbi-prjs-int,brandgulpFor
simtabiandichavait would have reported "readable" under exactly the 2026-11-03conditions. The other six pass only by luck of which repository was pushed most recently, which
changes constantly — so this is a latent false pass, not a fixed one.
Verification
Suite 27 → 30, OK.
Guard proven to have teeth: removing the private-first sort makes
test_control_prefers_a_private_repositoryfail (FAILED (failures=1)); restoring it returns 30OK. Its fixture is the real shape — public repo readable, private repo 403 — and asserts the
control returns
Falserather than passing on the public read.A public-only account still verifies (
test_a_public_only_account_still_verifies): with noprivate repository there is no private blindness to detect.
Live, read-only, against the two that previously picked public: