Skip to content

ci: derive testing promotion flavor count - #64

Closed
Danathar wants to merge 2 commits into
projectbluefin:mainfrom
Danathar:fix/testing-promotion-flavor-count
Closed

Danathar wants to merge 2 commits into
projectbluefin:mainfrom
Danathar:fix/testing-promotion-flavor-count

Conversation

@Danathar

Copy link
Copy Markdown
Contributor

Summary

  • derive the expected testing promotion image count from config/flavors.json via scripts/flavors.py
  • check out the repository in the promotion job with the required read permission
  • make just check reject numeric promotion-count literals

Verification

  • focused flavor-count and workflow-output checks pass
  • git diff --check passes
  • just check reached the repository validations but stopped because diff is not installed in the environment

Fixes #42

— hive: backend=codex

Signed-off-by: Danathar <Danathar@users.noreply.github.com>

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, well-scoped, and the change itself is right

3 files, +13/-1, and it applies cleanly to current main@636b48e despite the base drift. Replacing the test "$count" -eq 4 literal with a value derived from config/flavors.json is correct and closes a real gap: just check already forbade naming a flavor directly under .github/workflows/, and the count was the one place that escaped it.

The just check guard is the part that makes it stick:

! grep -nE 'count" -eq [0-9]' .github/workflows/post-testing-e2e.yml
grep -q 'flavors.py count' .github/workflows/post-testing-e2e.yml

Both an assertion that the literal is gone and that the derived call is present — good, that's harder to regress than a one-sided check.

Three things to look at before this lands.

1. actions/checkout on a workflow_run trigger checks out the wrong ref

promote-to-testing is triggered by workflow_run, and the new step is a bare checkout:

- name: Checkout
  uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Resolve the expected image count
  id: flavors
  run: echo "count=$(python3 scripts/flavors.py count)" >> "$GITHUB_OUTPUT"

On workflow_run, actions/checkout with no ref: defaults to the repository's default branch, not the commit that produced the images being promoted. The workflow's own if: already pins it to github.event.workflow_run.head_branch == 'testing', so the digests come from testing while the flavor count would come from main.

Today those agree, so this is latent rather than broken. But it becomes wrong precisely when it matters most — during a flavor-set change, when testing and main disagree about how many images exist, which is the exact scenario the count is there to catch. Suggest:

  with:
    ref: ${{ github.event.workflow_run.head_sha }}

That also makes the count trace to the same commit as the digests, which is the property #24's acceptance criteria ask for.

2. It de-literalizes the count but keeps the copy-before-count ordering

The check still runs after the promotion loop:

            skopeo copy --all "docker://${REGISTRY}/${name}@${digest}" "docker://${REGISTRY}/${name}:testing"
            count=$((count + 1))
          done < <(find /tmp/digests -name '*.txt' -exec cat {} +)
-          test "$count" -eq 4
+          test "$count" -eq "$EXPECTED"

So a run that produces 3 of 4 flavors still moves :testing for those 3, then fails on the count, with nothing undoing the partial promotion. Deriving the number is a real improvement over the literal; it does not address the ordering.

#34 identifies exactly this and restructures it to verify the full digest set against scripts/flavors.py images before any copy. That is the more complete fix, and the two PRs overlap on this file. Worth deciding which carries it — if #34 is landing anyway, this PR's flavors.py count addition is still useful groundwork, but the post-testing-e2e.yml edit would be redundant.

3. Conflicts with #44

#44 also adds a count query to scripts/flavors.py and also adds Justfile guards in the same region. Confirmed they collide:

$ git apply /tmp/pr64.diff && git apply --check /tmp/pr44.diff
error: patch failed: Justfile:79
error: Justfile: patch does not apply

#44's description explicitly says it adds flavors.py count as groundwork for #42 (this PR) because the agent token lacked workflows permission to edit the workflow file. So these two were designed as a pair and have since both grown the same half. Landing #44 first and reducing this PR to just the post-testing-e2e.yml + guard change would be the cleaner sequence.

On the CI status

Build Utah here is action_required, not failed — fork PRs need a maintainer to approve the run, so nothing has executed against this branch. Separately, main@636b48e is itself red (install: cannot create regular file '/usr/local/libexec/utah-install-packages', run 34654001732), so even an approved run would fail for reasons unrelated to this change.

The description is refreshingly honest that just check stopped early because diff was missing locally, rather than claiming a pass. That's the right way to report it.


Generated by Claude Code

hanthor's review on projectbluefin#64 found that the workflow_run trigger's checkout
step had no ref:, so it resolved main (the default branch) rather than
the commit that produced the images being promoted to testing. The
if: already gates on head_branch == 'testing', so the flavor count and
the digests being promoted could silently come from two different
commits -- latent today since main and testing agree, but wrong
precisely when a flavor-set change makes them disagree, which is the
scenario this count exists to catch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNgJ8VMdoqzGequbxsP2Kc
@Danathar

Danathar commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor Author

@hanthor — pushed in ffff479: added ref: ${{ github.event.workflow_run.head_sha }} to the checkout step, so the flavor count and the digests being promoted always come from the same commit instead of the count silently resolving against the default branch.

Left items 2 (copy-before-count ordering) and 3 (conflict with #44) as-is per your own framing — those are cross-PR sequencing questions (#34 and #44 aren't mine), not something to resolve unilaterally from this PR.

@hanthor

hanthor commented Sep 18, 2026

Copy link
Copy Markdown
Member

Closing as stale: it conflicts with main's post-testing-e2e.yml and Justfile, which were both rewritten after this opened (the E2E gate now fetches image-digest-testing-* artifacts and promotes from those), and there's been no rebase in a week. Deriving the flavor count from config/flavors.json is still the right idea for #42; a fresh PR against the current workflow would be small.


Generated by Claude Code

@hanthor hanthor closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[architect] :testing promotion gate hardcodes the flavor count — a fifth literal outside config/flavors.json

2 participants