Repository navigation
ci: block promotion unless the full Post-Testing E2E is green for the digest - #1338
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hanthor The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Holding this until Post-Testing E2E has passed once. That needs projectbluefin/testsuite#1268, then Utah's testsuite pin bump, then #1337. After that, lift the hold and merge, so the gate starts on a green baseline and doesn't freeze Decided by @hanthor (posted with Claude Code) |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head
No |
… digest Maintainer decision on #1329: a red E2E must block promotion. promote-to-testing now needs the testsuite matrix as well as the LUKS and production-ISO jobs, so a red leg records no evidence and leaves :testing where it is. record also posts e2e/testsuite/<image> for each digest, and the release gate requires it next to e2e/post-testing/<image>. Digests promoted while the testsuite was advisory carry only the old status, so they fail closed until a fully green run moves :testing. There is no override; the repo has none for its other gates. Refs #1329 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: James Reilly <jreilly1821@gmail.com>
697be32 to
ea22051
Compare
|
Hold lifted. Its conditions are met:
Rebased onto main. main had added the Decided on behalf of @hanthor with Claude Code |
There was a problem hiding this comment.
Summary: This change makes sure a new Utah build is only promoted to the testing and stable channels when every automated end-to-end test, including the desktop test suite, passed for that exact build. Since the last review the branch was rebased on top of a separate change that added an upgrade-and-rollback test to the same gate, and the two now combine correctly. One short documentation sentence lists the required tests but leaves out the new upgrade-and-rollback one.
Re-review at head ea22051f (previous: 697be32f, approved, no findings). The 30-file compare is mostly the rebase onto main; the PR's own diff (891d5e46...ea22051f) differs from the old one only in how it merges with #1327's lifecycle gate.
.github/workflows/post-testing-e2e.yml:523-535:testsuiteadded toneeds:and, because the job runs under!cancelled(), also as an explicitneeds.testsuite.result == 'success'term. Without that term, a failed testsuite would no longer skip the job, so adding it is correct.testsuite(line 484) needs onlyresolveand has no deliberate-skip path, so it cannot block a legitimate promotion the way a skippedlifecyclecould.tests/test_promotion_workflows.pyasserts both the newneedsentry and thesuccessterm.scripts/promotion-ratchet.pyand its tests are unchanged since the last review.
Docs: one minor gap, in the inline thread on docs/skills/ci-workflows.md. It does not block the PR.
PR body is stale: #1337 and #1327 have both landed, so the "Merge order" section about them no longer applies. The testsuite pin (projectbluefin/testsuite#1268) is still the item to check before merging, because merging freezes :testing until a run is fully green.
No findings from: correctness, security, intent-alignment, style.
Confidence: 5/5 (safe)
-- hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.327
| Its `promote-to-testing` job needs the LUKS, production ISO and testsuite | ||
| jobs. Only when all pass does it record per-digest evidence and move |
There was a problem hiding this comment.
docs-currency (low): This sentence lists what promote-to-testing needs, but it leaves out lifecycle. After the rebase on #1327, needs: is [resolve, luks, production-iso, lifecycle, testsuite] (.github/workflows/post-testing-e2e.yml:523). This file mentions lifecycle nowhere else. testsuite-triage.md in this same PR lists all five.
| Its `promote-to-testing` job needs the LUKS, production ISO and testsuite | |
| jobs. Only when all pass does it record per-digest evidence and move | |
| Its `promote-to-testing` job needs the LUKS, production ISO, lifecycle | |
| (unless deliberately skipped) and testsuite jobs. Only when all pass does it record per-digest evidence and move |
|
Ready for a human merge: verdict approved @ea22051, 20/20 checks green, soak 30m0s elapsed. The PR touches a human-merge path (auto_merge.human_merge_paths). The hive never merges this lane's changes itself: click Merge (or |
Summary: A new Utah build now becomes
:testing(and later:stable) only if every Post-Testing E2E job passed for that exact image, the desktop testsuite included. Until now the testsuite could fail and the image was promoted anyway.Maintainer decision on #1329: red E2E must block promotion.
What changes
post-testing-e2e.yml:promote-to-testingnow needstestsuiteas well asluksandproduction-iso. A red (or skipped) testsuite leg skips the job, so no evidence is recorded and:testingdoes not move.scripts/promotion-ratchet.py:recordalso postse2e/testsuite/<image>(description = exact digest) next to the existinge2e/post-testing/<image>.release-gate(inexecute-release.yml) requires both statuses for the digest:testingpoints at, on the commit it was built from.e2e/post-testing/<image>, recorded while the testsuite was red. Requiring the new one makes them fail closed.:stablewill not move until a fully green run moves:testing.needs, the extra status, and a release blocked by old LUKS-only evidence.ci-workflows.mdandtestsuite-triage.md, and updatesiso-e2e.mdandrelease-cadence.md.No manual override: the repo has no override pattern for its other gates, so none is added.
Merge order
The E2E is red today, so merging this freezes
:testingand:stableuntil these land and a run goes green:nvidiasuite from CI VMs)#1327 also edits the
promote-to-testingneeds:line; whichever lands second needs a one-line rebase.Checks
just check,actionlint, and the promotion/ISO unit tests pass locally.Refs #1329
🤖 Generated with Claude Code