Skip to content

Fetch cached OCI charts without contacting the registry - #307

Merged
abrisco merged 1 commit into
periareon:mainfrom
rdark:richard/oci-import-cached-blob-first
Oct 30, 2026
Merged

abrisco merged 1 commit into
periareon:mainfrom
rdark:richard/oci-import-cached-blob-first

Conversation

@rdark

@rdark rdark commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

CI often fails with GET returned 429 Too Many Requests from public.ecr.aws, fetching the Karpenter chart for tests/with_external_deps.

Every job sends helm_import's OCI requests to the registry, even with the repository cache restored. Fetching an oci:// chart makes three requests: a token, the manifest by tag, and the chart blob. Only the blob has a checksum, so it is the only one the repository cache can serve. Eleven jobs start together and make the same anonymous requests, which is enough to trip ECR's rate limit.

Change

A Helm chart's OCI layer is the chart .tgz, so when sha256 is set its blob is blobs/sha256:<sha256>. helm_import_repository now tries that blob first, without a token. If the repository cache has it, the fetch makes no request to the registry. Otherwise it falls back to the token, manifest and blob requests as before. A chart without sha256 is fetched as before.

All four OCI test charts already set sha256, so the test dependencies do not change. Their blobs have always had checksums, so the CI repository cache already holds them. In this PR's CI run, none of the 13 jobs printed a failed download: a cache miss on ECR, Docker Hub or GHCR would print a 401 warning, so those three charts came from the cache. Quay serves the blob without a token, so the log cannot show this for cert-manager.

On a cache miss the first blob request has no token, and a registry that needs one answers 401. Bazel prints one WARNING: Download from ... failed line for it, then the fetch continues as before. ECR, Docker Hub and GHCR do this; Quay serves the blob without a token.

A manifest without a config field, such as an index, now fails with the existing "is not a Helm chart package" message instead of a key error.

Testing

  • With an empty repository cache, all four OCI test charts fetch and pass their checksums.
  • With that cache warm, a fresh output base, --repository_disable_download and --repo_contents_cache= (so the rule runs on Bazel 9), bazel fetch --repo= succeeds for all four on Bazel 9.3.0, and for Karpenter and Grafana on 7.7.1. The same fetch on main fails at the token request in import_authn.bzl.
  • bazel test //... and --config=strict on macOS with Bazel 9.3.0 and 7.7.1: 65 pass, 3 skipped (the Linux-only with_image_deps_img tests).
  • CI passes on macOS, Ubuntu 22.04, Ubuntu 24.04 ARM and Windows, with the latest Bazel and the minimum (7.x) Bazel, with no 429 from any registry.

@rdark
rdark marked this pull request as ready for review October 24, 2026 18:36

@abrisco abrisco left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This looks good but I was wondering, would it make sense to save the data to facts if a sha256 isn't provided to further reduce fetches and to kinda auto-fix gaps?

@rdark

rdark commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

@abrisco thanks. I'd lean towards doing that as a stacked PR as seems like a larger change.

facts is only available to module extensions, and helm_import_repository is a repository rule. Users would need to call it from their own extensions or use_repo_rule. In order to record a missing sha, rules_helm would need an extension tag like helm.chart(name, url). That extension would resolve the digest from the manifest, store it in facts and then pass it into the rule, following the hasattr(module_ctx, "facts") guard pattern that k8s/extensions.bzl has.

A Helm chart's OCI layer is the chart .tgz, so when `sha256` is set its
blob is `blobs/sha256:<sha256>`. Try that blob first without a token, so
a chart in the repository cache needs no token, manifest or blob request.
On a miss, fall back to the token, manifest and blob requests as before.

CI fetched every OCI test chart from its registry on each job, and the
concurrent anonymous requests tripped ECR's rate limit (429).

Also fail with the "is not a Helm chart package" message, rather than a
key error, for a manifest without a `config` field.
@rdark
rdark force-pushed the richard/oci-import-cached-blob-first branch from 4d30912 to b8a2d8f Compare October 30, 2026 10:24
@abrisco
abrisco merged commit 946eb5d into periareon:main Oct 30, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants