[abhinavgautam01]

Closes #264

Adds read-only caching proxy support for Chef Supermarket at /chef, so Berkshelf and Policyfiles can resolve and download cookbooks through the proxy. Chef moves to ✓ in the supported registries table.

Usage

# Berksfile
source "http://localhost:8080/chef"

# Policyfile.rb
default_source :supermarket, "http://localhost:8080/chef"

A private Supermarket is configured with upstream.chef (or PROXY_UPSTREAM_CHEF), which defaults to https://supermarket.chef.io. Credentials go under upstream.auth, scoped to its URL.

How it works

The protocol was checked against the live supermarket.chef.io API:

Route	Behavior
GET /chef/universe	Rewrites each version's location_path and download_url to the proxy. Dependencies and all other fields are kept byte for byte.
GET /chef/api/v1/cookbooks/{name}	Rewrites latest_version and versions[]
GET /chef/api/v1/cookbooks/{name}/versions/{version}	Rewrites cookbook and file. Also accepts 7_5_22, the form Berkshelf uses in URLs.
GET /chef/api/v1/cookbooks/{name}/versions/{version}/download	Follows Supermarket's 302 to its storage host and caches the archive as pkg:chef/{name}@{version}

• Metadata goes through the existing metadata cache, rewrite cache and serveRewrittenMetadata, the same path npm and Composer use. It follows metadata_ttl, falls back to the stored copy when the upstream fails and supports ETag/304. The rewritten universe (about 7.6 MB) is cached rather than rebuilt on every request.
• Downloads are served from the artifact cache once cached, without contacting the upstream. Denylisted versions are refused, as for every artifact.
• Dependencies stay unchanged. JSON is re-encoded without HTML escaping, so constraints such as >= 0.0.0 keep their spelling rather than becoming \u003e= 0.0.0.
• Unreadable upstream documents return 502 instead of being passed through, since passing them through would send clients around the proxy.
• Cache keys include a hash of the upstream URL, so pointing upstream.chef at another Supermarket never serves the previous one's documents.
• Authentication uses the existing upstream.auth transport. Because credentials are scoped by URL, the storage host behind the download redirect never receives them.
• Input validation: cookbook names and versions are validated before any upstream URL is built from them.

Wiring

• Config: upstream.chef with its default, env var and serve -help entry
• Route mount in server.go, plus the chef label in request metrics
• Dashboard: ecosystem filter pill, badge color and install instructions for Berkshelf and Policyfiles
• Docs: README section and API endpoint row, config.example.yaml and docs/configuration.md with a private Supermarket auth example

Tests

New internal/handler/chef_test.go, against a fake Supermarket shaped like the real API, with downloads redirecting to a separate storage host:

• universe rewriting, with every dependency set unchanged and constraints not re-escaped
• cookbook and version metadata rewriting, with unrelated fields unchanged
• both version spellings (7.5.22 and 7_5_22), fetched from upstream once
• the Berkshelf flow (universe, then version at location_path, then file download), covering the redirect, a cache hit and offline reads of both the cookbook and the universe
• upstream authentication, including the storage host not receiving the token
• 400 for invalid names or versions, 404 for unknown cookbooks, 502 for an unreadable universe

Chef is also added to the existing config, serve -help, route-label and missing-artifact 404 tests. Removing the URL rewrite, the underscore handling or the credentials each makes a test fail.

Live check against supermarket.chef.io

Using proxy serve:

• /chef/universe returned 4,074 cookbooks with every entry rewritten. Compared with the upstream universe, all 31,355 versions have identical dependencies and fields.
• apt 7.5.22 downloaded through the redirect as a valid tarball. Its 24,561 bytes match Supermarket's tarball_file_size.

go build, go vet, golangci-lint (v2.13.1), go test -race ./... and Swagger regeneration all pass locally.

Not included

• Storage retention for chef: each ecosystem has received retention in its own PR (Add per-ecosystem artifact retention (age-based eviction) #306). Chef archives are still limited by storage.max_size.
• Cooldown: the universe carries no publish dates.
• Hiding denylisted versions from metadata: like Helm, Conan and Swift, they are blocked at download only.

[andrew]

Changing upstream.chef can serve a cached cookbook archive from the previous Supermarket. Please scope archive cache identity to the upstream and add a regression test covering an upstream switch with the same cookbook name and version.

[abhinavgautam01]

Thanks, fixed. Archives are now cached under a filename that carries the upstream's key (apt-7.5.22-<key>.tgz), the same 16-character digest the metadata cache keys use, so switching upstream.chef refetches from the new Supermarket. The package and version records stay pkg:chef/apt@7.5.22, so the denylist, dashboard and retention are unaffected. TestChefArchiveCacheDependsOnUpstream covers the switch through the download route and it fails with the old filename.

[andrew]

The upstream cache isolation issue is fixed. Archive filenames now include the upstream identity, and the regression test covers switching upstreams and retrieving each cached archive.