Repository navigation
Fail the control only on evidence, not on any error - #25
Merged
Merged
Conversation
#24's control enumerated the errors that are NOT permission failures -- 409 and the word "empty" -- and the enumeration was incomplete. An empty repository whose default_branch is nominally main answers 422 "No commit found for SHA" on commits/{ref}/check-runs while /commits answers 409, so the shape never matched. Measured live after #24 merged: simsange reported ok=False cannot read check runs on simsange/documentation@main: gh: No commit found for SHA: main (HTTP 422) because its newest private repository is the empty simsange/documentation. That is a false permission failure, and the gate would have gone red for simsange every quiet week. Inverted: fail only where GitHub says the permission is missing -- 403, or "Resource not accessible by integration" -- and treat anything else as inconclusive, keep looking, and return None if nothing conclusive is found. One shape to recognise instead of an open-ended list of exemptions. Two existing tests were asserting against the wrong thing and are fixed here: they raised FakeGh's generic "no route for ..." error, which is not a 403, so they passed for a reason unrelated to what they claimed to check. They now raise the real message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#24's control enumerated the errors that are not permission failures — 409 and the word
"empty" — and the enumeration was incomplete. An empty repository whose
default_branchisnominally
mainanswers 422 "No commit found for SHA" oncommits/{ref}/check-runs, while/commitsanswers 409. So the shape never matched.Inverted: fail only where GitHub says the permission is missing (403, or
Resource not accessible by integration), treat anything else as inconclusive, keep looking, and returnNoneif nothing conclusive is found. One shape to recognise, instead of an open-ended list of
exemptions.
Found by running it, immediately after #24 merged
simsange/documentationis empty —size=0, 0 branches,/commits→ 409 — and it is that org'smost recently pushed private repository, so #24's private-first sort selected it. A false
permission failure, and the gate would have gone red for
simsangeevery quiet week.After this change, all ten owned orgs verify and none reports a false failure:
simsangefalls past the empty repository to a public one and is labelled(public), so thesummary states that the read does not exercise
checks: readrather than implying it does.Two existing tests were asserting against the wrong thing
test_quiet_run_FAILS_when_checks_are_unreadableandtest_control_reports_absence_not_just_presenceraisedFakeGh's genericno route for ...error, which is not a 403 — so they passed for a reason unrelated to what they claimed to
check, and they are why the 422 case was never caught. Both now raise the real message, and the
fixtures for the real shapes are named constants (
DENIED,NO_COMMIT).Verification
denied()returnTruefor any error — which is what Probe a private repository, or the control verifies the wrong thing #24 effectivelydid — fails three tests (
FAILED (failures=3)), including the new 422 case; restoring returns 32OK.
test_an_unlistable_account_is_inconclusive_unless_deniedassertsNoneon a 404 andFalseon a 403, so the control can still report a genuine denial.Note
This is the third pass on this control (#23 added it, #24 made it probe the right repository, this
makes it fail for the right reason). Each correction came from running it against the real estate
rather than from re-reading it — and each previous version would have reported health while
something was wrong.