From a8a0572a81f945fa1290ace5bc9076f7a81b5503 Mon Sep 17 00:00:00 2001 From: Zoltan Csizmadia Date: Mon, 21 Sep 2026 13:24:51 -0500 Subject: [PATCH 1/3] docker CLI: build the Windows arm64 docker.exe upstream does not ship (#450) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The build and the release plumbing. The manifest still carries arm64 as a placeholder -- that entry needs a published dockercli- release to point at, which needs this on main first, the same order #388 followed. Upstream publishes nothing here: download.docker.com's static tree has exactly one directory, x86_64/, and docker/cli attaches no release assets at all. So `skrog cli install` on Windows on ARM laid down compose, buildx and the credential helper -- all of which DO have real arm64 builds -- and skipped the one command anybody types. third_party/docker-cli/ builds it: docker/cli at a tag pinned to its dereferenced commit, in the pinned Go toolchain the engine uses, cross compiled with CGO off. Cross-compiling is not a compromise here the way it would be for the engine -- this is `go build` with GOARCH set against fully vendored pure Go, not a C-linked tree that needs boot-testing. Two details that cost time to find: - docker/cli ships vendor.mod and vendor.sum rather than go.mod and go.sum, deliberately, so it cannot be consumed as a library. Their own scripts rename them; so does this. - The upstream ldflags set PlatformName to "Docker Engine - Community". It is left UNSET here, and that is the one thing in this change that is about honesty rather than mechanics: this is not Docker's build, and stamping their build string on it would be the single genuinely misleading thing the script could do. Version and commit are set, so `docker version` reports something true rather than "dev". The PE machine type is asserted twice -- in build.sh and again in the workflow, outside it. A cross-compile that silently produced x86-64 would pass every other check and fail only on a user's ARM machine, which is the exact failure #388 was about, and it is the one thing a build on an x86-64 runner cannot feel. Verified against a real local build: PE offset 128, machine bytes 64aa, 28.3 MB. amd64 stays Docker's own bytes. Anyone can re-download that zip and check it against the pinned digest without reference to Skrog; a binary we build can only be checked against us, and building it too would put every user behind our build rather than only the arm64 users with no alternative. It is not about code signing -- Docker's published Windows CLI is NotSigned, measured on the binary `skrog cli install` places. A test fails if the amd64 URL ever stops pointing at download.docker.com. This amends the "ship CLI binaries unmodified" line in PLAN §09, which #450 asked be decided rather than slid past. Building from Apache-2.0 source in our own CI is the treatment that same line already prescribes for the engine, and it is the opposite of repackaging Docker's distribution. Also closes a hole a third tag namespace opens. release.yml and rootfs.yml excluded each other by name -- "not rootfs-" -- which is an open list: a dockercli- tag would have attached skrog.exe to a docker CLI release because nothing said no. All three guards now test for their own prefix positively, which the next namespace cannot break. rootfs.yml's build job also stops rebuilding the engine, twice, for every release that is not its own. CI's shellcheck ran a hardcoded list of paths; it now runs every .sh in the tree, matching what scripts/lint.ps1 already did locally. --- .github/workflows/dockercli.yml | 147 ++++++++++++++++++++++++ .github/workflows/release.yml | 16 ++- .github/workflows/rootfs.yml | 13 ++- CHANGELOG.md | 26 +++++ PLAN.md | 2 +- RELEASING.md | 62 ++++++++-- docs/docker-cli.md | 63 ++++++++-- internal/dockercli/manifest.json | 24 +++- internal/dockercli/versions_env_test.go | 141 +++++++++++++++++++++++ third_party/docker-cli/build.sh | 131 +++++++++++++++++++++ third_party/docker-cli/versions.env | 31 +++++ 11 files changed, 630 insertions(+), 26 deletions(-) create mode 100644 .github/workflows/dockercli.yml create mode 100644 internal/dockercli/versions_env_test.go create mode 100644 third_party/docker-cli/build.sh create mode 100644 third_party/docker-cli/versions.env diff --git a/.github/workflows/dockercli.yml b/.github/workflows/dockercli.yml new file mode 100644 index 0000000..f9da41d --- /dev/null +++ b/.github/workflows/dockercli.yml @@ -0,0 +1,147 @@ +name: docker-cli + +# Builds the Windows arm64 docker.exe that upstream does not publish (#450). +# +# download.docker.com's static tree has exactly one directory, x86_64/, and +# docker/cli publishes no release assets at all. So `skrog cli install` on +# Windows on ARM laid down compose, buildx and the credential helper -- all of +# which DO have arm64 builds -- and skipped the one command anybody types. +# +# amd64 is NOT built here. It stays Docker's own published binary, because +# those bytes are verifiable by anyone against the digest pinned in +# internal/dockercli/manifest.json without reference to Skrog, and building +# them ourselves would put every user behind our build instead of only the +# arm64 users who have no alternative. third_party/docker-cli/build.sh carries +# the full reasoning, including why code signing is not part of it. +# +# The asymmetry ends by itself: when upstream publishes a Windows arm64 +# docker.exe, the manifest points at it and this workflow is deleted. + +on: + push: + branches: [main] + paths: ['third_party/docker-cli/**', '.github/workflows/dockercli.yml'] + pull_request: + paths: ['third_party/docker-cli/**', '.github/workflows/dockercli.yml'] + # A CLI release (tag: dockercli-vX.Y.Z) rebuilds and attaches its assets. + # App and rootfs releases are other namespaces and skip the publish job. + release: + types: [published] + workflow_dispatch: + +permissions: + contents: read + +jobs: + build: + name: build docker.exe (windows/arm64) + # Cross-compiled, and that is not a compromise here the way it would be + # for the engine: docker/cli is pure Go with CGO off and everything + # vendored, so an x86-64 runner produces the same binary an arm64 one + # would. The engine needs native runners because it is C-linked and + # boot-tested; this is `go build` with GOARCH set. + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v7 + + - name: shellcheck + run: shellcheck -x -P SCRIPTDIR third_party/docker-cli/*.sh + + - name: build + run: ./third_party/docker-cli/build.sh "$GITHUB_WORKSPACE/out" + + # build.sh already fails on a non-ARM64 PE header. This repeats the + # check outside the script so a bug in the script's own parsing cannot + # pass its own test -- the binary is going to machines nobody here has, + # and an x86-64 exe named arm64 would only be discovered by the one + # person least able to diagnose it. + - name: it really is an ARM64 binary + run: | + exe=$(ls out/docker-*-windows-arm64.exe) + machine=$(python3 - "$exe" <<'PY' + import struct, sys + with open(sys.argv[1], 'rb') as f: + f.seek(0x3c) + off = struct.unpack('&2; exit 1; } + + - name: report + run: | + { + echo '### docker CLI (windows/arm64)' + echo '' + echo '```' + cat out/*.commit.txt + cat out/*.sha256 + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + - uses: actions/upload-artifact@v7 + with: + name: docker-cli-windows-arm64 + path: out/ + retention-days: 14 + + publish: + name: attach to release + needs: build + # Only docker CLI releases. Without this check every app and rootfs + # release would also get a docker.exe attached, because the release event + # does not distinguish what is being released. + if: github.event_name == 'release' && startsWith(github.event.release.tag_name, 'dockercli-') + runs-on: ubuntu-latest + permissions: + contents: write + # See release.yml: id-token for the OIDC exchange cosign and the + # attestation both need, attestations to record the provenance (#147). + id-token: write + attestations: write + steps: + - uses: actions/download-artifact@v8 + with: + name: docker-cli-windows-arm64 + path: out + + # This is the part that makes a self-built CLI defensible. The amd64 + # binary is Docker's and anyone can re-download it to check the pinned + # digest; this one exists nowhere else, so its provenance has to come + # from here -- tied to this workflow, this repository and this commit. + - uses: actions/attest-build-provenance@v4 + with: + subject-path: 'out/docker-*-windows-arm64.exe' + + - uses: sigstore/cosign-installer@v3 + - name: sign the checksum + run: | + shopt -s nullglob + sumfiles=(out/*.sha256) + if [ ${#sumfiles[@]} -eq 0 ]; then + echo "no .sha256 to sign; the build produced nothing" >&2 + exit 1 + fi + for sums in "${sumfiles[@]}"; do + cosign sign-blob --yes --bundle "$sums.cosign.bundle" "$sums" + cosign verify-blob \ + --bundle "$sums.cosign.bundle" \ + --certificate-identity-regexp "^https://github.com/${GITHUB_REPOSITORY}/" \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + "$sums" + done + + # --repo is required: this job downloads artifacts without checking out + # the code, so gh has no git remote to infer the repository from. + - name: upload release assets + # TAG/REPO via env, not spliced into the command (#87): this job holds + # contents: write. + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ github.event.release.tag_name }} + REPO: ${{ github.repository }} + run: gh release upload "$TAG" out/* --clobber --repo "$REPO" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7e0841b..641377b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -2,8 +2,16 @@ name: release # Builds and attaches the skrog binaries when an app release is published. # -# Rootfs releases are a separate namespace (tag prefix rootfs-) handled by -# rootfs.yml; this workflow skips them so the two never cross-contaminate. +# There are three tag namespaces in this repository -- vX.Y.Z (the app), +# rootfs-vX.Y.Z-N (the engine image) and dockercli-vX.Y.Z (the Windows arm64 +# docker CLI). Each has its own workflow, and each must ignore the others' +# tags or a release ends up carrying assets from a different product. +# +# The guards below are written as "is this an app version" rather than "is +# this not a rootfs", which is how they used to read. That was an open list: +# adding dockercli- would silently have attached skrog.exe to a docker CLI +# release, because nothing said no. A closed test cannot be broken by the +# next namespace. # # Artifacts are signed with cosign keyless (Sigstore, GitHub OIDC) and carry # SLSA build provenance (#147), so a download can be tied to this workflow, this @@ -43,7 +51,7 @@ permissions: jobs: build: name: build binaries - if: github.event_name != 'release' || !startsWith(github.event.release.tag_name, 'rootfs-') + if: github.event_name != 'release' || startsWith(github.event.release.tag_name, 'v') runs-on: windows-latest steps: - uses: actions/checkout@v7 @@ -281,7 +289,7 @@ jobs: publish: name: attach to release needs: build - if: github.event_name == 'release' && !startsWith(github.event.release.tag_name, 'rootfs-') + if: github.event_name == 'release' && startsWith(github.event.release.tag_name, 'v') runs-on: ubuntu-latest permissions: contents: write diff --git a/.github/workflows/rootfs.yml b/.github/workflows/rootfs.yml index 642df3d..68a790a 100644 --- a/.github/workflows/rootfs.yml +++ b/.github/workflows/rootfs.yml @@ -24,8 +24,12 @@ jobs: # Same invocation as scripts/lint.ps1 and the pre-commit hook: -x follows # `source=` directives, -P SCRIPTDIR resolves them next to each script # (versions.env sits beside build.sh, not in the working directory). + # Every .sh in the tree, matching what scripts/lint.ps1 does locally. + # A hardcoded list here means a new script is unchecked in CI until + # someone remembers this line, and the one person who would notice is + # the one who did not add it. - name: shellcheck - run: shellcheck -x -P SCRIPTDIR guest/rootfs/*.sh spike/a/*.sh + run: shellcheck -x -P SCRIPTDIR $(git ls-files '*.sh') build: # One rootfs per architecture, each compiled NATIVELY (#388). @@ -37,6 +41,13 @@ jobs: # GitHub's ubuntu-24.04-arm runners are real arm64 and free for public # repositories, so there is no reason to emulate. name: build rootfs (${{ matrix.arch }}) + # Not on someone else's release. This job runs ~6 minutes per + # architecture, and the release event does not say what is being + # released -- so without this it rebuilt the engine twice for every app + # tag, and would now do it for every docker CLI tag too. The publish job + # was always guarded, so nothing wrong was ever uploaded; this is waste, + # not a correctness fix. + if: github.event_name != 'release' || startsWith(github.event.release.tag_name, 'rootfs-') runs-on: ${{ matrix.runner }} timeout-minutes: 90 strategy: diff --git a/CHANGELOG.md b/CHANGELOG.md index d548b38..6b47da8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,32 @@ useful than saying where the real one is. ### Added +- **Windows on ARM: a `docker` CLI, built here because upstream ships none** + ([#450](https://github.com/wslkit/skrog/issues/450)). `skrog cli install` on + arm64 laid down compose, buildx and the credential helper — all of which + have real arm64 builds — and skipped the one command anybody types. + `download.docker.com`'s static tree has exactly one directory, `x86_64/`, + and `docker/cli` attaches no release assets. + + So Skrog builds it: `docker/cli` at a commit-pinned tag, in the same pinned + Go toolchain as the engine, with SLSA provenance and a cosign-signed + checksum, published as a `dockercli-v*` release of this repository. The + build asserts the PE machine type is really `0xaa64` before anything is + attached — a cross-compile that quietly produced an x86-64 binary would + pass every other check and fail only on a user's ARM machine. + + **amd64 still comes from Docker**, deliberately. Those bytes are verifiable + by anyone against the pinned digest with no reference to Skrog, and building + them too would put every user behind our build rather than only the arm64 + users who have no alternative. It is not about code signing: Docker's + published Windows CLI is not Authenticode-signed either. `docs/docker-cli.md` + has the table and the reasoning, and the asymmetry ends when upstream ships + an arm64 build. This amends the CLI-repackaging line in PLAN §09. + + The arm64 binary reports its version and upstream commit, and deliberately + does **not** claim `Docker Engine - Community` — that is Docker's build + string for Docker's builds. + - **Windows on ARM: `skrog install` works** ([#388](https://github.com/wslkit/skrog/issues/388)). `skrog.exe` has shipped an arm64 build for months while the engine was amd64-only, so diff --git a/PLAN.md b/PLAN.md index 781844a..8ac7be3 100644 --- a/PLAN.md +++ b/PLAN.md @@ -303,7 +303,7 @@ The `wsl` package hides every `wsl.exe` invocation behind an interface so unit t | Risk | Severity | Mitigation | |---|---|---| -| Docker trademark | launch-blocking | No "docker" or whale in name/logo/package IDs (hence Skrog); nominative references only; ship CLI binaries unmodified under Apache-2.0; engine binaries built from source in CI, not repackaged from Docker's distribution channels. Legal sanity check before the repo goes public. | +| Docker trademark | launch-blocking | No "docker" or whale in name/logo/package IDs (hence Skrog); nominative references only; ship CLI binaries unmodified under Apache-2.0; engine binaries built from source in CI, not repackaged from Docker's distribution channels. Legal sanity check before the repo goes public. **Amended 2026-09-21 (#450):** one CLI binary is now *built* rather than shipped unmodified — the Windows arm64 `docker.exe`, because upstream publishes none and `skrog cli install` on ARM was skipping the one command anybody types. Building from Apache-2.0 source in our own CI is the *engine* treatment, which this same line already sanctions, and it is the opposite of the repackaging the trademark concern is about: nothing is redistributed from Docker's channels, and the binary carries no Docker build string (`PlatformName` is deliberately left unset). amd64 stays Docker's own published bytes. The asymmetry is documented in `docs/docker-cli.md` and ends when upstream ships an arm64 build. | | Microsoft's wslc absorbs the niche | strategic | wslc (GA expected fall 2026) mimics the CLI and runs a real Moby engine, but exposes no Docker endpoint — dockerd listens on a guest unix socket with no `-H`, and nothing reaches it from Windows — so compose, Testcontainers and socket-mounting tools cannot use it, and the endpoint request (microsoft/WSL#40976) sits unanswered with no milestone. Note the gap is a missing listener, not a missing engine: Microsoft narrows the surface deliberately, because their registry allowlist and plugin hooks are enforced in `wslcsession.exe` *in front of* dockerd. Ship fast and own the "real Docker API" framing. The pre-thought pivot is now measured rather than hypothetical (#316): Skrog's glue layer (doctor, service lifecycle, fleet policy, path-translating pipe) can sit on top of a wslc session — reached today over vsock (#320), or over Microsoft's own endpoint if #40976 ships. | | ~~Session-0 / no-login WSL~~ — **settled, and negative** | resolved | Spike B (#3) measured it: WSL2 will not create its utility VM outside an interactive session. LocalSystem is refused by name; a dedicated service account fails in HCS with `ERROR_LOGON_TYPE_NOT_GRANTED` holding Service + Batch + Interactive rights and local admin. No longer a risk but a documented constraint (§03, §06). Cost of finding out: one day, in week one, exactly as intended. | | Auto-logon forbidden by policy | medium | Unattended runners need a logged-on session, and auto-logon stores a password in LSA secrets. Fleets that ban it cannot run Skrog unattended — nor any WSL-based engine, so no competitor wins those machines either. Mitigation: state it in the docs, name it in `doctor`, and never automate credential storage on the user's behalf. Theoretical escape (out of scope): run the engine in a plain Hyper-V VM instead of WSL. | diff --git a/RELEASING.md b/RELEASING.md index 1e5f65e..f3e4182 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,25 +1,31 @@ # Releasing -Two independent release streams, deliberately kept apart: +Three independent release streams, deliberately kept apart: | stream | tag | what it publishes | workflow | |---|---|---|---| -| **rootfs** | `rootfs-vX.Y.Z` | one engine tarball **per architecture**, each with its `.sha256` and an SPDX SBOM | `rootfs.yml` | +| **rootfs** | `rootfs-vX.Y.Z-N` | one engine tarball **per architecture**, each with its `.sha256` and an SPDX SBOM | `rootfs.yml` | | **app** | `vX.Y.Z` | `skrog.exe` for amd64 and arm64, zipped, plus `SHA256SUMS` | `release.yml` | +| **docker CLI** | `dockercli-vX.Y.Z` | the Windows **arm64** `docker.exe` upstream does not publish (#450), with its `.sha256` | `dockercli.yml` | -They are separate because the engine and the app version independently: an engine -security patch should not require an app release, and vice versa (PLAN §04, -`skrog engine upgrade`). Both workflows check the tag prefix, so a rootfs -release never gets app binaries attached and an app release never gets a rootfs. +They are separate because they version independently: an engine security patch +should not require an app release, a docker CLI bump is upstream's schedule +rather than ours, and vice versa (PLAN §04, `skrog engine upgrade`). + +**Each workflow tests for its own prefix, positively.** They used to exclude +each other by name — "not `rootfs-`" — which is an open list: adding a third +namespace would silently have attached `skrog.exe` to a docker CLI release +because nothing said no. The guards now ask "is this mine", which the next +namespace cannot break. Deciding *what* to bump — which moby tag, whether containerd/runc/buildkit move with it, where each checksum comes from — is a separate job from cutting the release. That is [docs/bumping-upstream.md](docs/bumping-upstream.md); this file covers publishing once the versions are settled. -## `git describe` sees both streams +## `git describe` sees every stream -The two tag namespaces share one repository, and rootfs releases are cut far +The tag namespaces share one repository, and rootfs releases are cut far more often than app releases — so `git describe --tags` usually resolves to a **rootfs** tag, not an app version: @@ -124,6 +130,46 @@ Until step 2 lands, `skrog install` refuses with a message telling the user to pass `--rootfs-url` and `--rootfs-sha256` explicitly. That is intentional: there is no code path that installs an unverified rootfs. +## Cutting a docker CLI release + +Only needed on Windows arm64, and only until upstream publishes one +([#450](https://github.com/wslkit/skrog/issues/450)). The shape mirrors the +rootfs stream, one step shorter because there is a single artifact. + +1. **Pin the version.** `third_party/docker-cli/versions.env` carries + `DOCKER_CLI_VERSION`, `DOCKER_CLI_TAG` and `DOCKER_CLI_SHA` — the tag's + dereferenced commit, which `build.sh` refuses to build without matching: + + ``` + git ls-remote https://github.com/docker/cli 'refs/tags/vX.Y.Z^{}' + ``` + + It must equal the `docker` component's `version` in + `internal/dockercli/manifest.json`; a test asserts that, because the same + binary is installed on both architectures and two versions is a support + trap. + +2. **Cut the release.** Tag `dockercli-v` — matching + `DOCKER_CLI_VERSION`, no revision suffix, because the artifact is one + binary from one upstream commit and there is nothing else in it to revise. + Publishing triggers `dockercli.yml`, which builds, asserts the PE header is + really ARM64, attests, signs the checksum and attaches both files. + + **`--latest=false`, as always:** + + ``` + gh release edit dockercli-vX.Y.Z --prerelease=false --latest=false + ``` + +3. **Point the manifest at it.** Put the published `.sha256` and URL into + `internal/dockercli/manifest.json` under + `components[docker].arch.arm64`. Merge as a normal PR. + + amd64 is **not** touched — it stays Docker's own published zip. That + asymmetry is deliberate and argued in `docs/docker-cli.md`; a test fails if + the amd64 URL ever stops pointing at `download.docker.com`, so changing it + is a decision rather than a slip. + ## Dry runs Both workflows can be exercised without publishing: diff --git a/docs/docker-cli.md b/docs/docker-cli.md index fb96420..874f9c8 100644 --- a/docs/docker-cli.md +++ b/docs/docker-cli.md @@ -159,11 +159,60 @@ removes the bundled tools and takes the `bin` directory back off your PATH. (Pla `skrog uninstall` removes the engine; the CLI bundle is managed separately so you can keep the CLI while reinstalling the engine.) -## Architecture note - -Windows **amd64** gets all four tools. On Windows **arm64**, compose, buildx, and -the credential helper are available, but upstream does not yet publish a Windows -arm64 `docker.exe`; `skrog cli install` reports it as unavailable and installs -the rest. Use Docker Desktop's `docker` (which is arm64-native) until an upstream -arm64 CLI ships. +## Architecture note: where each binary comes from + +Both architectures get all four tools, but **not from the same place**, and the +difference is worth knowing before you audit them. + +| tool | amd64 | arm64 | +|---|---|---| +| `docker` | Docker's published static build | **built by Skrog** from `docker/cli` source | +| `compose` | upstream release | upstream release | +| `buildx` | upstream release | upstream release | +| `docker-credential-wincred` | upstream release | upstream release | + +### Why one of them is ours + +Upstream publishes no Windows arm64 `docker.exe`. `download.docker.com`'s +static tree contains exactly one directory, `x86_64/`, and `docker/cli` +attaches no binaries to its releases at all. So on Windows on ARM, +`skrog cli install` used to lay down compose, buildx and the credential helper +— all of which *do* have real arm64 builds — and skip the one command anybody +types ([#450](https://github.com/wslkit/skrog/issues/450)). + +`docker/cli` is Apache-2.0, pure Go and fully vendored, so Skrog builds it: +from a tag pinned to an exact commit, in the same pinned Go toolchain as the +engine, with SLSA build provenance and a cosign-signed checksum, published as +a `dockercli-v*` release of this repository. `third_party/docker-cli/` is the +whole of it. + +### Why the other one is not + +amd64 keeps coming from Docker, deliberately: + +- **Anyone can check it.** Those bytes are Docker's own. Re-download the zip + and compare it against the `sha256` pinned in + `internal/dockercli/manifest.json`, with no reference to Skrog. A binary we + build can only be checked against us. +- **Blast radius.** Building it ourselves would put every user behind our + build, rather than only the arm64 users who have no alternative. +- **A bump stays a URL and a hash**, not a rebuild. + +It is **not** about code signing. Docker's published Windows CLI is not +Authenticode-signed either — measured on the binary `skrog cli install` places +— so nothing is being preserved on one side and lost on the other. + +### It is meant to end + +When upstream publishes a Windows arm64 `docker.exe`, the manifest points at +it, `third_party/docker-cli/` is deleted, and both architectures are Docker's +again. This is a stopgap with an exit condition, not a component Skrog has +taken ownership of. + +### What `docker version` shows + +The arm64 build reports the pinned version and the upstream commit it was +built from. It deliberately does **not** claim `Docker Engine - Community`, +which is Docker's build string for Docker's builds — that field is left empty +rather than filled in with something untrue. diff --git a/internal/dockercli/manifest.json b/internal/dockercli/manifest.json index 714a15a..687b4a1 100644 --- a/internal/dockercli/manifest.json +++ b/internal/dockercli/manifest.json @@ -12,11 +12,25 @@ "file (compose: checksums.txt; buildx: checksums-signed.txt, which is the one", "that covers windows; wincred: checksums.txt).", "", - "The docker CLI is the exception: download.docker.com publishes no checksum", - "and no Windows arm64 static build. Its amd64 sha256 is computed from the", - "official download and pinned here, so every install verifies the exact bytes", - "even though upstream ships no digest; arm64 stays a placeholder until a", - "Windows arm64 docker.exe exists upstream.", + "The docker CLI is the exception, twice over (#450).", + "", + "amd64: download.docker.com publishes no checksum, so its sha256 is computed", + "from the official download and pinned here. Every install still verifies the", + "exact bytes; the digest is just ours rather than upstream's.", + "", + "arm64: upstream publishes no Windows arm64 docker.exe at all -- the static", + "tree has one directory, x86_64/, and docker/cli attaches no release assets.", + "So Skrog BUILDS this one, from docker/cli source at a commit-pinned tag,", + "and its URL points at a dockercli- release of this repository rather than", + "at Docker. See third_party/docker-cli/ and docs/docker-cli.md.", + "", + "amd64 deliberately stays Docker's own bytes: anyone can re-download the zip", + "and check the digest below without reference to Skrog, and building it too", + "would put every user behind our build rather than only the arm64 users who", + "have no alternative. A test fails if the amd64 URL stops pointing at", + "download.docker.com, so changing that is a decision rather than a slip.", + "When upstream ships an arm64 build, both point at Docker again and", + "third_party/docker-cli/ is deleted.", "", "Licenses: docker/cli, docker/compose, docker/buildx are Apache-2.0;", "docker-credential-helpers is MIT. Stage() lays each LICENSE beside the", diff --git a/internal/dockercli/versions_env_test.go b/internal/dockercli/versions_env_test.go new file mode 100644 index 0000000..3c084b3 --- /dev/null +++ b/internal/dockercli/versions_env_test.go @@ -0,0 +1,141 @@ +package dockercli_test + +import ( + "bufio" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/wslkit/skrog/internal/dockercli" +) + +// third_party/docker-cli builds the Windows arm64 docker.exe that upstream +// does not publish (#450). Its pinned version and the manifest's `docker` +// component are two hand-edited files describing the same binary, so they +// drift the moment someone bumps one. +// +// The failure that drift produces is nasty and quiet: `skrog cli install` +// would lay down an arm64 docker.exe built from one tag while `skrog version` +// and the manifest both claim another, and the only way to notice is to run +// `docker version` on an ARM machine and compare it against a JSON file. +// +// Mirrors internal/release/versions_env_test.go, which pins the engine the +// same way and for the same reason. +func readCLIVersionsEnv(t *testing.T) map[string]string { + t.Helper() + + dir, err := os.Getwd() + if err != nil { + t.Fatalf("getwd: %v", err) + } + var path string + for i := 0; i < 5; i++ { + candidate := filepath.Join(dir, "third_party", "docker-cli", "versions.env") + if _, err := os.Stat(candidate); err == nil { + path = candidate + break + } + dir = filepath.Dir(dir) + } + if path == "" { + t.Skip("third_party/docker-cli/versions.env not found; skipping cross-check") + } + + f, err := os.Open(path) + if err != nil { + t.Fatalf("opening %s: %v", path, err) + } + defer f.Close() + + out := map[string]string{} + s := bufio.NewScanner(f) + for s.Scan() { + line := strings.TrimSpace(s.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + k, v, ok := strings.Cut(line, "=") + if !ok { + continue + } + out[strings.TrimSpace(k)] = strings.TrimSpace(v) + } + if err := s.Err(); err != nil { + t.Fatalf("reading %s: %v", path, err) + } + return out +} + +func TestDockerCLIVersionMatchesTheBuildPin(t *testing.T) { + pins := readCLIVersionsEnv(t) + + m, err := dockercli.Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + var docker *dockercli.Component + for i := range m.Components { + if m.Components[i].Name == "docker" { + docker = &m.Components[i] + break + } + } + if docker == nil { + t.Fatal("the manifest has no `docker` component") + } + + if want := pins["DOCKER_CLI_VERSION"]; want == "" { + t.Error("versions.env has no DOCKER_CLI_VERSION") + } else if docker.Version != want { + t.Errorf("manifest docker version %q != versions.env DOCKER_CLI_VERSION %q\n"+ + " the same binary is installed on both architectures; two versions is a support trap", + docker.Version, want) + } + + // The tag is the version with a v, and the commit must be pinned at all. + // build.sh only WARNS on an unpinned SHA, because a warning is right for + // someone bisecting locally -- but shipping without one means a moved tag + // silently changes what every docker command on the machine runs (#88). + if tag, ver := pins["DOCKER_CLI_TAG"], pins["DOCKER_CLI_VERSION"]; tag != "v"+ver { + t.Errorf("DOCKER_CLI_TAG %q is not v%s", tag, ver) + } + if sha := pins["DOCKER_CLI_SHA"]; len(sha) != 40 { + t.Errorf("DOCKER_CLI_SHA %q is not a 40-hex commit; build.sh would only warn", sha) + } +} + +// The arm64 asset, once published, must come from this repository's own +// release. It is the one binary Skrog builds rather than fetches, and a URL +// pointing anywhere else would mean the attestation and signature attached to +// the tag describe different bytes than the ones installed. +// +// Skipped while arm64 is still a placeholder, which is the state before the +// first dockercli- release is cut. +func TestArm64DockerCLIComesFromOurOwnRelease(t *testing.T) { + m, err := dockercli.Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + for _, c := range m.Components { + if c.Name != "docker" { + continue + } + a, ok := c.Arch["arm64"] + if !ok || a.URL == "" { + t.Skip("no arm64 docker CLI published yet (#450)") + } + const prefix = "https://github.com/wslkit/skrog/releases/download/dockercli-v" + if !strings.HasPrefix(a.URL, prefix) { + t.Errorf("arm64 docker CLI URL %q does not come from a dockercli- release of this repo.\n"+ + " Skrog builds this binary; its provenance is the tag it was attached to.", a.URL) + } + // And amd64 must still be Docker's, which is the whole point of the + // asymmetry. Building both would put every user behind our build. + if up, ok := c.Arch["amd64"]; ok && !strings.Contains(up.URL, "download.docker.com") { + t.Errorf("amd64 docker CLI URL %q is no longer Docker's own published binary.\n"+ + " If that is deliberate, third_party/docker-cli/build.sh and docs/docker-cli.md\n"+ + " both argue the opposite and need rewriting first.", up.URL) + } + } +} diff --git a/third_party/docker-cli/build.sh b/third_party/docker-cli/build.sh new file mode 100644 index 0000000..58490f2 --- /dev/null +++ b/third_party/docker-cli/build.sh @@ -0,0 +1,131 @@ +#!/usr/bin/env bash +# Build a Windows arm64 docker.exe from docker/cli source (#450). +# +# ./build.sh [output-dir] default: ./out +# +# Output: docker--windows-arm64.exe and its .sha256. +# +# Runs anywhere Docker runs; the compile itself happens in the pinned golang +# image so the toolchain is the same in CI and on a laptop. docker/cli is pure +# Go with everything vendored, so this cross-compiles from x86-64 with no ARM +# hardware and no extra toolchain. +# +# # Why only arm64 +# +# amd64 keeps coming from download.docker.com, and that asymmetry is a choice: +# +# - Those bytes are Docker's. Anyone can re-download the zip and check it +# against the sha256 pinned in internal/dockercli/manifest.json, with no +# reference to Skrog at all. A binary we build can only be checked against +# us. Replacing a verifiable-by-anyone artifact with a trust-us one, for +# users who do not need it, is a real loss. +# - Building it ourselves would make a bad build wrong for every user rather +# than for the arm64 minority who have no alternative. +# - An upstream bump stays a URL and a hash rather than a rebuild. +# +# It is not about code signing. Docker's published Windows CLI is NOT +# Authenticode-signed -- measured on the binary `skrog cli install` places -- +# so there is no signature being preserved on one side and lost on the other. +# +# The asymmetry ends by itself: when upstream publishes a Windows arm64 +# docker.exe, the manifest points at it and this directory is deleted. +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +out="${1:-$here/out}" +# shellcheck source=versions.env +. "$here/versions.env" + +mkdir -p "$out" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +target="docker-${DOCKER_CLI_VERSION}-windows-arm64.exe" + +echo "==> building docker/cli ${DOCKER_CLI_TAG} for windows/arm64" + +# HOST_UID/GID: the build runs as root in the container but writes into a host +# directory, so it hands ownership back before exiting -- otherwise the +# checksum step and cleanup hit permission errors on a CI runner. +docker run --rm \ + -v "$work:/out" \ + -e "CLI_TAG=$DOCKER_CLI_TAG" \ + -e "CLI_VERSION=$DOCKER_CLI_VERSION" \ + -e "CLI_SHA=${DOCKER_CLI_SHA:-}" \ + -e "TARGET=$target" \ + -e "HOST_UID=$(id -u)" \ + -e "HOST_GID=$(id -g)" \ + "golang:${GO_VERSION}-alpine" sh -euc ' + apk add --no-cache git >/dev/null + + git -c advice.detachedHead=false clone --depth 1 --branch "$CLI_TAG" \ + https://github.com/docker/cli.git /src >/dev/null 2>&1 + cd /src + + sha="$(git rev-parse HEAD)" + if [ -n "$CLI_SHA" ] && [ "$CLI_SHA" != "$sha" ]; then + echo "FATAL: docker/cli $CLI_TAG resolved to $sha, expected $CLI_SHA" >&2 + echo " (a moved tag or a compromised source; refusing to build)" >&2 + exit 1 + fi + [ -n "$CLI_SHA" ] || echo " WARNING: no expected SHA pinned; resolved $sha" >&2 + + # docker/cli ships vendor.mod and vendor.sum rather than go.mod and go.sum, + # deliberately, so the repository cannot be consumed as a Go library. Their + # own build scripts rename them; so must we, or the build cannot resolve a + # single import. + cp vendor.mod go.mod + cp vendor.sum go.sum + + # Version and commit, so `docker version` reports something true. Without + # them it says "dev" with an empty commit, which is worse than useless on + # a binary the user did not download from Docker: it makes an unfamiliar + # build look broken as well as unfamiliar. + # + # PlatformName is deliberately NOT set. Upstream stamps it "Docker Engine + # - Community", and this build is not that -- it is docker/cli source + # compiled here because upstream ships no Windows arm64 binary. Claiming + # their build string would be the one genuinely misleading thing this + # script could do. Leaving it empty costs a cosmetic line in + # `docker version` and keeps the output honest. + # + # No BuildTime either: a timestamp is the one input that makes two builds + # of the same commit differ, and a reproducible binary is worth more than + # a date nobody reads. + pkg=github.com/docker/cli/cli/version + GOOS=windows GOARCH=arm64 CGO_ENABLED=0 \ + go build -mod=vendor -trimpath \ + -ldflags "-s -w -X ${pkg}.Version=${CLI_VERSION} -X ${pkg}.GitCommit=${sha}" \ + -o "/out/$TARGET" ./cmd/docker + + [ -s "/out/$TARGET" ] || { echo "the build produced no binary" >&2; exit 1; } + printf "docker/cli %s %s\n" "$CLI_TAG" "$sha" > /out/commit.txt + chown -R "$HOST_UID:$HOST_GID" /out' + +cp "$work/$target" "$out/$target" +cp "$work/commit.txt" "$out/docker-${DOCKER_CLI_VERSION}-windows-arm64.commit.txt" + +# The architecture is asserted, not assumed. A cross-compile that silently +# produced an x86-64 binary would sail through every other check here and fail +# only on a user's ARM machine, which is the exact failure #388 was about -- +# and the one thing a build on an x86-64 runner genuinely cannot feel. +# +# PE: "PE\0\0" at the offset stored in the 4 bytes at 0x3c, then a 2-byte +# little-endian machine type. 0xaa64 is ARM64, 0x8664 is x86-64. +pe_off=$(od -An -tu4 -j60 -N4 "$out/$target" | tr -d ' ') +machine=$(od -An -tx1 -j"$((pe_off + 4))" -N2 "$out/$target" | tr -d ' ') +case "$machine" in + 64aa) arch="ARM64" ;; # 0xaa64, little-endian + 6486) arch="x86-64" ;; # 0x8664 + *) arch="UNKNOWN" ;; +esac +echo "==> PE machine type: $machine ($arch)" +if [ "$machine" != "64aa" ]; then + echo "FATAL: built a $arch binary, expected ARM64" >&2 + exit 1 +fi + +(cd "$out" && sha256sum "$target" > "$target.sha256") + +ls -la "$out" +echo "OK" diff --git a/third_party/docker-cli/versions.env b/third_party/docker-cli/versions.env new file mode 100644 index 0000000..9dc624e --- /dev/null +++ b/third_party/docker-cli/versions.env @@ -0,0 +1,31 @@ +# Pinned inputs for the Windows arm64 docker CLI (#450). +# +# Upstream publishes no Windows arm64 docker.exe -- download.docker.com's +# static tree has exactly one directory, x86_64/, and docker/cli publishes no +# release assets at all. So on Windows on ARM `skrog cli install` laid down +# compose, buildx and the credential helper, all of which DO have arm64 +# builds, and skipped the one command anybody types. +# +# This builds it. amd64 is NOT built here and stays Docker's own published +# binary; see third_party/docker-cli/build.sh for why the asymmetry is +# deliberate rather than an omission. + +# The CLI version, which must match the `docker` component in +# internal/dockercli/manifest.json -- the same binary is installed on both +# architectures and reporting two different versions would be a support +# nightmare. A test pins the two together. +DOCKER_CLI_VERSION=29.8.1 +DOCKER_CLI_TAG=v29.8.1 + +# The commit the tag must resolve to (#88). A git tag can be moved and a +# source repository compromised, and the result is the binary every docker +# command on the machine runs. build.sh fails on a mismatch. +# +# This is the tag's dereferenced commit (refs/tags/^{}), not the tag +# object's own hash. +DOCKER_CLI_SHA=4a63305d74332de5ceba7fcbccbc3cbb7412f5ba + +# Toolchain. Bumping this changes the output, so it is pinned like everything +# else. Kept in step with guest/rootfs/versions.env so the engine and the CLI +# are built by the same compiler. +GO_VERSION=1.26.7 From 83cd4a7b3bd6386572472f4133df7c1896320214 Mon Sep 17 00:00:00 2001 From: Zoltan Csizmadia Date: Mon, 21 Sep 2026 13:26:41 -0500 Subject: [PATCH 2/3] docker CLI: mark build.sh executable `chmod +x` on a Windows working tree does not reach the index, so it landed as 100644 and the workflow died with "Permission denied" (exit 126) before running a line of it. guest/rootfs/build.sh is 100755, which is why nothing else noticed. `git update-index --chmod=+x`, which is the part that actually records it. --- third_party/docker-cli/build.sh | 0 1 file changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 third_party/docker-cli/build.sh diff --git a/third_party/docker-cli/build.sh b/third_party/docker-cli/build.sh old mode 100644 new mode 100755 From cf8a9e3acfcb886b310efabea2503f8b973bd91d Mon Sep 17 00:00:00 2001 From: Zoltan Csizmadia Date: Mon, 21 Sep 2026 13:31:29 -0500 Subject: [PATCH 3/3] docker CLI: record that the build is reproducible Not a change to the build -- a fact about it that is worth writing down, because it is the strongest answer to "why should I trust a binary you built". The output has no timestamp and no build host in it (BuildTime is deliberately unset), so the same commit produces the same bytes anywhere. Confirmed rather than assumed: my local build and this PR's CI build of docker/cli 4a63305d both came out at 910087c0d9a80ac0e7802f8c48b98cdc7fd890c78e0dab5f0472bee856ac50a0 28,319,232 bytes Different machines, different kernels, same binary. The SLSA attestation says we built it; reproducibility means nobody has to take our word for what from. --- docs/docker-cli.md | 11 +++++++++++ third_party/docker-cli/build.sh | 7 +++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/docker-cli.md b/docs/docker-cli.md index 874f9c8..7ae5d81 100644 --- a/docs/docker-cli.md +++ b/docs/docker-cli.md @@ -186,6 +186,17 @@ engine, with SLSA build provenance and a cosign-signed checksum, published as a `dockercli-v*` release of this repository. `third_party/docker-cli/` is the whole of it. +**And you can reproduce it.** The build is deterministic — no timestamp, no +build host in the output — so running `third_party/docker-cli/build.sh` on +any machine with Docker produces a byte-identical binary. First verified +2026-09-21: a local build and the CI build of commit `4a63305d` both came out +at `910087c0d9a80ac0e7802f8c48b98cdc7fd890c78e0dab5f0472bee856ac50a0`, +28,319,232 bytes. + +That is a stronger answer than the provenance attestation on its own. The +attestation says *we* built it; reproducibility means you do not have to take +our word for what from. + ### Why the other one is not amd64 keeps coming from Docker, deliberately: diff --git a/third_party/docker-cli/build.sh b/third_party/docker-cli/build.sh index 58490f2..11e50eb 100755 --- a/third_party/docker-cli/build.sh +++ b/third_party/docker-cli/build.sh @@ -90,8 +90,11 @@ docker run --rm \ # `docker version` and keeps the output honest. # # No BuildTime either: a timestamp is the one input that makes two builds - # of the same commit differ, and a reproducible binary is worth more than - # a date nobody reads. + # of the same commit differ. Leaving it out makes this reproducible, and + # it is -- a local build and the CI build of 4a63305d both came out at + # 910087c0d9a80ac0e7802f8c48b98cdc7fd890c78e0dab5f0472bee856ac50a0. That + # is worth more than a date nobody reads: it means nobody has to take our + # word for what these bytes are. pkg=github.com/docker/cli/cli/version GOOS=windows GOARCH=arm64 CGO_ENABLED=0 \ go build -mod=vendor -trimpath \