Skip to content

ci: block promotion unless the full Post-Testing E2E is green for the digest - #1338

Merged
hanthor merged 1 commit into
mainfrom
ci/gate-promotion-on-green-e2e
Nov 19, 2026
Merged

hanthor merged 1 commit into
mainfrom
ci/gate-promotion-on-green-e2e

Conversation

@hanthor

@hanthor hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member

Summary: A new Utah build now becomes :testing (and later :stable) only if every Post-Testing E2E job passed for that exact image, the desktop testsuite included. Until now the testsuite could fail and the image was promoted anyway.

Maintainer decision on #1329: red E2E must block promotion.

What changes

  • post-testing-e2e.yml: promote-to-testing now needs testsuite as well as luks and production-iso. A red (or skipped) testsuite leg skips the job, so no evidence is recorded and :testing does not move.
  • scripts/promotion-ratchet.py:
    • record also posts e2e/testsuite/<image> (description = exact digest) next to the existing e2e/post-testing/<image>.
    • release-gate (in execute-release.yml) requires both statuses for the digest :testing points at, on the commit it was built from.
    • Why a second status: digests promoted before this change carry only e2e/post-testing/<image>, recorded while the testsuite was red. Requiring the new one makes them fail closed. :stable will not move until a fully green run moves :testing.
  • Tests for the new needs, the extra status, and a release blocked by old LUKS-only evidence.
  • Docs: replaces the "testsuite is advisory" text from docs: explain why desktop test failures do not stop promotion #1332 in ci-workflows.md and testsuite-triage.md, and updates iso-e2e.md and release-cadence.md.

No manual override: the repo has no override pattern for its other gates, so none is added.

Merge order

The E2E is red today, so merging this freezes :testing and :stable until these land and a run goes green:

#1327 also edits the promote-to-testing needs: line; whichever lands second needs a one-line rebase.

Checks

just check, actionlint, and the promotion/ISO unit tests pass locally.

Refs #1329

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hanthor

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hanthor hanthor added the hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 18, 2026
@hanthor

hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

Holding this until Post-Testing E2E has passed once. That needs projectbluefin/testsuite#1268, then Utah's testsuite pin bump, then #1337. After that, lift the hold and merge, so the gate starts on a green baseline and doesn't freeze :testing while the fixes are in flight. #1327 touches the same needs: line, so whichever lands second needs a one-line rebase.

Decided by @hanthor (posted with Claude Code)

@hivecommons-hive hivecommons-hive Bot added author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 18, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head ea22051f7d9c matched:

  • sensitive_path: changes 1 sensitive path(s) (.github/workflows/post-testing-e2e.yml)

No sentinel-alert label was added. The finding is recorded in the dashboard Security tab and /api/sentinel; set sentinel.trusted_authors_block: true to block trusted authors too.

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it stage/stalled In its current stage longer than expected and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 18, 2026
… digest

Maintainer decision on #1329: a red E2E must block promotion.

promote-to-testing now needs the testsuite matrix as well as the LUKS
and production-ISO jobs, so a red leg records no evidence and leaves
:testing where it is. record also posts e2e/testsuite/<image> for each
digest, and the release gate requires it next to e2e/post-testing/<image>.
Digests promoted while the testsuite was advisory carry only the old
status, so they fail closed until a fully green run moves :testing.

There is no override; the repo has none for its other gates.

Refs #1329

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
@hanthor
hanthor force-pushed the ci/gate-promotion-on-green-e2e branch from 697be32 to ea22051 Compare November 18, 2026 22:41
@hanthor hanthor removed the hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 18, 2026
@hanthor

hanthor commented Nov 18, 2026

Copy link
Copy Markdown
Member Author

Hold lifted. Its conditions are met:

Rebased onto main. main had added the lifecycle gate, so promote-to-testing now needs [resolve, luks, production-iso, lifecycle, testsuite]. The if: keeps main's lifecycle-skip rule and also requires needs.testsuite.result == 'success'. Tests and the triage doc are updated to match, test_promotion_* passes, and actionlint is clean.

Decided on behalf of @hanthor with Claude Code

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it stage/stalled In its current stage longer than expected labels Nov 18, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: This change makes sure a new Utah build is only promoted to the testing and stable channels when every automated end-to-end test, including the desktop test suite, passed for that exact build. Since the last review the branch was rebased on top of a separate change that added an upgrade-and-rollback test to the same gate, and the two now combine correctly. One short documentation sentence lists the required tests but leaves out the new upgrade-and-rollback one.


Re-review at head ea22051f (previous: 697be32f, approved, no findings). The 30-file compare is mostly the rebase onto main; the PR's own diff (891d5e46...ea22051f) differs from the old one only in how it merges with #1327's lifecycle gate.

  • .github/workflows/post-testing-e2e.yml:523-535: testsuite added to needs: and, because the job runs under !cancelled(), also as an explicit needs.testsuite.result == 'success' term. Without that term, a failed testsuite would no longer skip the job, so adding it is correct. testsuite (line 484) needs only resolve and has no deliberate-skip path, so it cannot block a legitimate promotion the way a skipped lifecycle could.
  • tests/test_promotion_workflows.py asserts both the new needs entry and the success term.
  • scripts/promotion-ratchet.py and its tests are unchanged since the last review.

Docs: one minor gap, in the inline thread on docs/skills/ci-workflows.md. It does not block the PR.

PR body is stale: #1337 and #1327 have both landed, so the "Merge order" section about them no longer applies. The testsuite pin (projectbluefin/testsuite#1268) is still the item to check before merging, because merging freezes :testing until a run is fully green.

No findings from: correctness, security, intent-alignment, style.

Confidence: 5/5 (safe)

-- hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.327

Comment on lines +104 to +105
Its `promote-to-testing` job needs the LUKS, production ISO and testsuite
jobs. Only when all pass does it record per-digest evidence and move

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs-currency (low): This sentence lists what promote-to-testing needs, but it leaves out lifecycle. After the rebase on #1327, needs: is [resolve, luks, production-iso, lifecycle, testsuite] (.github/workflows/post-testing-e2e.yml:523). This file mentions lifecycle nowhere else. testsuite-triage.md in this same PR lists all five.

Suggested change
Its `promote-to-testing` job needs the LUKS, production ISO and testsuite
jobs. Only when all pass does it record per-digest evidence and move
Its `promote-to-testing` job needs the LUKS, production ISO, lifecycle
(unless deliberately skipped) and testsuite jobs. Only when all pass does it record per-digest evidence and move

@hivecommons-hive hivecommons-hive Bot added role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/stalled In its current stage longer than expected stage/090-ready-for-human-merge Hive would merge this but trust/role blocks it; a maintainer's /lgtm /approve is all that is missing and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/stalled In its current stage longer than expected labels Nov 18, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Ready for a human merge: verdict approved @ea22051, 20/20 checks green, soak 30m0s elapsed. The PR touches a human-merge path (auto_merge.human_merge_paths).

The hive never merges this lane's changes itself: click Merge (or /lgtm + /approve for tide) to land it.

@hanthor
hanthor added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit aca9b53 Nov 19, 2026
68 checks passed
@hanthor
hanthor deleted the ci/gate-promotion-on-green-e2e branch November 19, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/090-ready-for-human-merge Hive would merge this but trust/role blocks it; a maintainer's /lgtm /approve is all that is missing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant