From bf69a9490ada7bf9739bf1715153eeab6314b830 Mon Sep 17 00:00:00 2001 From: hanthor Date: Sat, 19 Sep 2026 11:42:46 +0000 Subject: [PATCH] Stop a testing build cancelling the one before it `cancel-in-progress: true` applies to every ref, so two pushes to `testing` close together kill the first build mid-flight. That build is not a redundant one: it produces the images, and it dispatches the ISO end-to-end validation that the release gate reads. Killing it throws the evidence away. This is not theoretical. Today, two testing builds were cancelled within two minutes of each other: 11:09:47 Build Utah workflow_dispatch testing cancelled 11:11:40 Build Utah workflow_dispatch testing cancelled 11:31:30 Build Utah push testing (ran) and the promotion that followed failed on exactly the missing evidence: E2E_LAST_STATUS: skipped READY: false ##[error]Release gate is blocked. So the gate was not wrong and the images were not broken -- the run that would have produced the proof had been cancelled by the next one. A pull request genuinely does supersede itself, so cancellation stays there. The group still includes github.ref, so a testing build does not queue behind an unrelated branch's. This is the uncontroversial half of the concurrency change proposed in #138. I asked for changes on that PR over a different part of it -- taking image builds off pull requests -- and this piece should not wait on that discussion, because it is currently costing the project its ISO validation. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M --- .github/workflows/build.yml | 13 ++++++++++++- tests/test_iso_ci.py | 23 +++++++++++++++++++++++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index e0f4889ad..75ee73274 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -10,9 +10,20 @@ on: permissions: {} +# A pull request supersedes itself: a new push means the previous run is +# answering a question nobody is asking any more. +# +# A run on `testing` does not. It is the run that produces the images, and the +# one that dispatches the ISO end-to-end validation the release gate reads. When +# two land close together the older one is killed mid-build and its E2E +# dispatch dies with it -- which is why the promotion gate keeps reporting +# E2E_LAST_STATUS: skipped and refusing to promote. On 2026-09-19 two testing +# builds were cancelled within two minutes of each other (runs at 11:09 and +# 11:11), and the promotion that followed was blocked for want of the evidence +# they would have produced. concurrency: group: utah-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + cancel-in-progress: ${{ github.event_name == 'pull_request' }} jobs: # Resolve the complete transaction against the same pinned inputs as the diff --git a/tests/test_iso_ci.py b/tests/test_iso_ci.py index b0f35a9aa..f04a67e46 100644 --- a/tests/test_iso_ci.py +++ b/tests/test_iso_ci.py @@ -419,3 +419,26 @@ def test_a_cache_hit_still_reports_the_image_as_available(self): if s.get("id") == "cache") hit = step["run"].index("Cache hit") self.assertIn("available=true", step["run"][hit:hit + 200]) + +class ConcurrencyTests(unittest.TestCase): + """A testing build must not be cancelled by the next one. + + build.yml on `testing` is what produces the images and dispatches the ISO + end-to-end validation the release gate reads. Cancelling it throws away the + evidence: on 2026-09-19 two testing builds were cancelled within two minutes + (11:09 and 11:11) and the promotion that followed failed with + E2E_LAST_STATUS: skipped. + """ + + def test_only_pull_requests_are_cancelled(self): + import yaml + build = yaml.safe_load((ROOT / ".github/workflows/build.yml").read_text()) + self.assertEqual(build["concurrency"]["cancel-in-progress"], + "${{ github.event_name == 'pull_request' }}") + + def test_the_group_still_separates_refs(self): + # Sharing one group across refs would make a testing build queue behind + # an unrelated PR's. + import yaml + build = yaml.safe_load((ROOT / ".github/workflows/build.yml").read_text()) + self.assertIn("github.ref", build["concurrency"]["group"])