Skip to content

ci: stop fork pull requests failing on a GHCR permission they cannot have - #184

Merged
hanthor merged 1 commit into
mainfrom
claude/kernel-cache-fork-prs
Sep 19, 2026
Merged

hanthor merged 1 commit into
mainfrom
claude/kernel-cache-fork-prs

Conversation

@hanthor

@hanthor hanthor commented Sep 19, 2026

Copy link
Copy Markdown
Member

Why

#154, #157 and #158 are all red for the same reason, and none of it is in their diffs. Each touches scripts/install-ogc-kernel.sh, which rotates the kernel cache tag, so kernel_cache compiles the kernel — twenty minutes — and then dies:

Error: writing blob: initiating layer upload to
/v2/projectbluefin/utah-kernel-cache/blobs/uploads/ in ghcr.io:
denied: installation not allowed to Write organization package
##[error]Process completed with exit code 125

A pull request from a fork gets a read-only GITHUB_TOKEN. That write cannot be granted to a contributor, so every kernel change from a fork is permanently red on a step that says nothing about the change. Three PRs are sitting blocked by CI plumbing.

Worth noting what the log proves: the kernel built. The failure is at the push, so the part that actually validates a kernel change already passes.

What changes

  • The podman build stays unconditional. Only the podman push is gated on whether the token can publish.
  • When it cannot, the job emits a notice — "the kernel compiled, but a fork pull request cannot write to ghcr.io/…; the flavored images are not built for this PR; push the branch to projectbluefin/utah to validate them" — and exits 0.
  • kernel_cache now reports whether the image is pullable, and build_kernel requires it, so the flavored builds are skipped rather than left to fail pulling an image that was never pushed. dispatch-iso already tolerates a skipped build_kernel.
  • The GHCR login is allowed to fail with a warning: only the push needs credentials, and a cache hit can be read anonymously.

CAN_PUBLISH treats an empty head repository as publishable, so pushes, schedules and dispatches are unaffected.

What this deliberately does not do

It does not pretend the flavors were validated. I argued against exactly that trade on #138 — moving image builds off pull requests — and the difference matters: there the loss of signal was a choice, here it is a platform limitation whose only alternative is a job that can never be green. The skip is loud, not silent, and the honest path for a kernel change is a branch in this repository, where the push works. That is why my own kernel PRs are green and these three are not.

Verification

  • Ran 199 tests ... OK (skipped=1), including four new cases in tests/test_iso_ci.py.
  • The tests assert the ordering that carries the meaning: the build is in front of the guard, the push behind it, a cache hit still reports the image as available, and build_kernel requires that output. Removing the build_kernel gate fails the suite — checked.
  • scripts/check_workflow_outputs.py passes with the new job output.

Unblocks #154, #157 and #158.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M


Generated by Claude Code

#154, #157 and #158 are all red for the same reason, and none of it is in
their diffs. Each touches scripts/install-ogc-kernel.sh, which rotates the
kernel cache tag, so kernel_cache compiles the kernel -- twenty minutes --
and then dies:

    Error: writing blob: initiating layer upload to
    /v2/projectbluefin/utah-kernel-cache/blobs/uploads/ in ghcr.io:
    denied: installation not allowed to Write organization package
    ##[error]Process completed with exit code 125

A pull request from a fork gets a read-only GITHUB_TOKEN. No contributor can
be granted that write, so every kernel change from a fork is permanently red
on a step that has nothing to say about the change.

The compile is the part that validates a kernel change, and it already passes.
So the build stays unconditional and only the push is gated: when the token
cannot publish, the job emits a notice saying the kernel compiled and that the
flavored images are not built here, and exits 0. build_kernel is skipped in
that case rather than left to fail pulling an image that was never pushed --
kernel_cache now reports whether the image is available and build_kernel
requires it. dispatch-iso already tolerates a skipped build_kernel.

The GHCR login is also allowed to fail: only the push needs credentials, and
a cache hit can be read anonymously.

What this does not do is pretend the flavors were validated. The notice is
explicit, and the honest fix for a kernel change is a branch in this
repository, where the push works -- which is why my own kernel PRs are green
and these three are not. Trading a red job for a quiet skip would be the
wrong trade if the skip were silent; it is not.

tests/test_iso_ci.py asserts the ordering that matters: the build is in front
of the guard and the push behind it, a cache hit still reports the image as
available, and build_kernel requires that output. Removing the build_kernel
gate fails the suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
@hanthor

hanthor commented Sep 19, 2026

Copy link
Copy Markdown
Member Author

The cache-hit path is confirmed by this PR's own run, which is the case most likely to have been broken by the change.

This branch does not touch any hashed input, so kernel_cache hit the published tag — and build_kernel then started all three flavored images:

kernel_cache                                  completed success
build_kernel / Preflight checks               completed success
build_kernel / image (nvidia, utah-kernel, …)  in_progress
build_kernel / image (gaming, utah-kernel, …)  in_progress
build_kernel / image (nvidia-gaming, …)        in_progress

That only happens if the cache-hit branch emits available=true. Had I set the output only on the push path, a cache hit would have silently skipped every flavored build — the exact failure mode the new test_a_cache_hit_still_reports_the_image_as_available guards, now also demonstrated live rather than only in the unit test.

What this run does not exercise is the fork path, since it is a branch in this repository. That case is covered by the unit tests only, and the honest proof will be #154, #157 or #158 going green after this lands.


Generated by Claude Code

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78

@Danathar Danathar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I checked: this matches the failure I traced on #158/#157/#154 this morning (the kernel builds, then podman push to utah-kernel-cache is denied because a fork PR's token is read-only). The change keeps the compile unconditional, gates only the push on CAN_PUBLISH, and makes build_kernel skip cleanly instead of trying to pull an image that was never pushed. The CAN_PUBLISH expression is right for all three cases: non-PR events (the pull_request object is absent, so the first comparison is true), same-repo PRs (full_name == github.repository), and fork PRs (neither). The tolerated-login change is fine because a cache hit can be read anonymously. The four tests in tests/test_iso_ci.py pin the ordering (build before guard before push) and the skip condition. All 14 checks green.

Heads-up: #185 also edits build.yml and appends a test class to the same file, so the second of the two to merge will need a small rebase.

@hanthor

hanthor commented Sep 19, 2026

Copy link
Copy Markdown
Member Author

This now has a concrete victim rather than a hypothetical one, which is worth recording since it changes the priority.

#158 (kylerankin, the OGC kernel btrfs fix, from a fork) has exactly one failing check, and it is this bug:

Error: writing blob: initiating layer upload to
/v2/projectbluefin/utah-kernel-cache/blobs/uploads/ in ghcr.io:
denied: installation not allowed to Write organization package
##[error]Process completed with exit code 125.

Job 105803941683. The sequence is the bad part: the kernel built successfully — the log runs all the way through the NVIDIA module compile and the objtool warnings — and the job then failed at the final push, roughly 45 minutes in, discarding a result that was correct. The contributor sees a red check on a PR whose diff is fine, and the only remedy available to them is to not be on a fork.

That is the case this PR is for. With kernel_cache gaining outputs.available, the build unconditional and only the push gated by CAN_PUBLISH, a fork PR builds and validates the kernel and then stops cleanly instead of attempting a write it can never be granted. build_kernel requiring needs.kernel_cache.outputs.available == 'true' keeps the downstream honest rather than letting it consume a cache that was never published.

So this is not only a tidiness fix: it is currently the sole thing blocking an outside contribution that fixes a real ISO failure, and it costs 45 minutes of runner time per attempt in the meantime.


Generated by Claude Code

@hanthor

hanthor commented Sep 19, 2026

Copy link
Copy Markdown
Member Author

Second confirmed case, from a different contributor. #157 (mrbobbytables) fails the same way on kernel_cache, job 105802895529:

Error: writing blob: initiating layer upload to /v2/projectbluefin/utah-kernel-…
denied: installation not allowed to Write organization package

So this is now two independent fork pull requests — #158 and #157 — each burning a full kernel build and then discarding a correct result at the push step, with no remedy available to the author.

I checked a third, #154, which also shows kernel_cache red, and it is not this bug: that one genuinely fails to compile, at /usr/src/linux-7.1.8-ogc1/Makefile:2165, which is its own change to the GPU driver config. Worth stating so nobody counts it toward this PR's case — two real victims, not three.


Generated by Claude Code

@hanthor
hanthor merged commit e8f72d3 into main Sep 19, 2026
14 checks passed
@hanthor
hanthor deleted the claude/kernel-cache-fork-prs branch September 19, 2026 13:53
hanthor pushed a commit that referenced this pull request Sep 19, 2026
One conflict, and it is purely additive on both sides: #184 landed
ForkPullRequestKernelCacheTests at the same point in the file where this
branch had added FlatpakRetryTests and OgcKernelConfigGateTests. Nothing
overlaps, so both are kept.

250 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6BYLUD6KBzF8rBFXKjP7M
hanthor pushed a commit that referenced this pull request Sep 19, 2026
…158)

Both gaming flavors fail the LUKS install at installer phase 3 with the same
error, most recently in Post-Testing E2E run 35447837973:

  + mount -t btrfs /dev/mapper/fisherman-root /mnt/fisherman-target
  mount: unknown filesystem type 'btrfs'.
  (blkid: /dev/mapper/fisherman-root: LABEL="root" TYPE="btrfs")

mkfs.btrfs succeeds, blkid reads the filesystem back correctly, and the kernel
cannot mount it: the OGC kernel is a source build from upstream
x86_64_defconfig, which omits CONFIG_BTRFS_FS, so no btrfs module exists. The
non-gaming flavors install to a GNOME login in the same run, because only the
gaming flavors ship that kernel.

BTRFS_FS is enabled as a module and added to required_config, so verify_config
rejects a config missing it at config time rather than after a 45-minute build
and a full ISO compose. Verified in kernel_cache on ee4868d: CC [M]
fs/btrfs/super.o and btrfs.ko produced. Every symbol in required_config is set
somewhere in the script, so the new entry cannot pass vacuously.

luks-e2e.sh also gains a modprobe btrfs pre-check in the live guest, so the next
occurrence is a one-line diagnosis instead of a phase-3 installer failure that
reads as a cryptsetup or filesystem problem.

This unblocks promote-to-testing, which has been skipped for want of a passing
luks matrix, which is why no :testing tag exists for any of the four images and
why promotion PR #181 fails digest resolution.

kernel_cache green and build_kernel skipped is the expected shape for a fork
pull request whose diff changes the kernel cache tag (#184): the kernel is
compiled and validated, and the cache is not published.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants