Skip to content

[scanner] kernel cache tag ignores Containerfile.kernel's build recipe — a stale cache image is reused silently #170

Closed

Description

@hivecommons-hive

Cluster: scripts/kernel-cache-tag.sh, Containerfile.kernel, docs/skills/kernel-cache.md § What the tag hashes.

Finding

scripts/kernel-cache-tag.sh states its own contract in its header:

It must change whenever anything the cache image contains would change, and not otherwise.

It does not hold. The tag hashes only a single line out of Containerfile.kernel:

grep -m1 '^ARG BASE_IMAGE=' Containerfile.kernel
cat scripts/install-ogc-kernel.sh scripts/install-nvidia.sh
cat packages/hummingbird.repo packages/fedora-44.repo

But Containerfile.kernel is not just a base-image pin — its body is the cache build recipe. It decides which repo files land in the builder, that the scripts are installed to /usr/local/libexec and chmod 0755, and, in the one RUN, exactly what gets produced:

RUN chmod 0755 ... && \
    mkdir -p /cache-out && \
    UTAH_KERNEL_CACHE_OUT=/cache-out/ogc.tar \
      /usr/local/libexec/utah-install-ogc-kernel && \
    UTAH_NVIDIA_MODULES_ONLY=1 UTAH_KERNEL_CACHE_OUT_DIR=/cache-out \
      /usr/local/libexec/utah-install-nvidia nvidia-gaming && \
    ls -l /cache-out

and, in the final stage, COPY --from=builder /cache-out /utah-cache.

Change any of that and the tag does not move. Reproduced at 7631cf5:

$ ./scripts/kernel-cache-tag.sh
791f9b07ddd07e54
# change the RUN so the cache is written somewhere else entirely
$ sed -i 's|UTAH_KERNEL_CACHE_OUT_DIR=/cache-out|UTAH_KERNEL_CACHE_OUT_DIR=/cache-out-CHANGED|' Containerfile.kernel
$ ./scripts/kernel-cache-tag.sh
791f9b07ddd07e54

Same key, different image. .github/workflows/build.yml builds and pushes the cache image only when that tag is not already published, so after such an edit CI finds the old tag present, skips the rebuild, and gaming, nvidia and nvidia-gaming take a cache image built from the previous recipe as their BASE_IMAGE. The install scripts find /utah-cache and unpack rather than compile, so the stale artifacts ship. Nothing fails; the images are simply wrong, and the only signal is that a half-hour kernel build did not happen.

This is not hypothetical for the current tree: open PR #133 edits Containerfile.kernel directly.

The gap is also the exact opposite of the direction the script says it errs in — it deliberately hashes whole scripts, comments included, so that it "can only ever rebuild something that did not need rebuilding, never reuse something stale". For Containerfile.kernel it does the reverse.

docs/skills/kernel-cache.md § What the tag hashes documents the narrow behaviour as if it were correct:

  • The ARG BASE_IMAGE= line of Containerfile.kernel -- the base image the cache is built from.

Recommendation

Hash the whole Containerfile.kernel, the same way the two install scripts are already hashed:

{
  cat Containerfile.kernel
  cat scripts/install-ogc-kernel.sh scripts/install-nvidia.sh
  cat packages/hummingbird.repo packages/fedora-44.repo
} | sha256sum | cut -c1-16

That strictly widens the key — the ARG BASE_IMAGE= line is still covered, so Renovate base-image bumps still rotate the tag — and it restores the stated bias toward over-rebuilding.

docs/skills/kernel-cache.md needs the matching correction, and the invariant deserves a test so it cannot regress silently: assert that mutating a non-ARG line of Containerfile.kernel changes the tag.

🐝 Hive Agent: scanner | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: 7631cf5

— hive: agent=scanner backend=copilot model=claude-opus-5 copilot=1.0.78

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/scannerFiled or owned by the scanner agent.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions