Skip to content

Move to bzlmod and support Bazel 9 - #24

Merged
gartnera merged 1 commit into
masterfrom
alexg/bzlmod-bazel-9
Sep 9, 2026
Merged

gartnera merged 1 commit into
masterfrom
alexg/bzlmod-bazel-9

Conversation

@gartnera

@gartnera gartnera commented Sep 9, 2026

Copy link
Copy Markdown
Member

Bazel 9 removed WORKSPACE support outright -- there is no --enable_workspace to turn it back on -- so a WORKSPACE-only toolchain is invisible to it: Bazel synthesizes an empty MODULE.bazel and neither the archives nor @rules_cc resolve. Since the new baseline, 7.7.1, has bzlmod on by default, drop WORKSPACE entirely rather than carry both. deps.bzl becomes extensions.bzl, whose module extension declares the same http_archives, and MODULE.bazel registers the toolchains that native.register_toolchains used to. Consumers move from aarch64_linux_gnu_deps() to a bazel_dep plus archive_override; README.md has the snippet.

Registration is not enough on its own. A dependency's registrations rank below every one the root module makes, and rules_cc registers an auto-detected host cc toolchain, which then wins the native x86_64 and aarch64 builds. That fails quietly: the build succeeds against the host's own gcc. Root modules therefore have to register these toolchains themselves, which tests/MODULE.bazel and the README now do, and CI asserts per configuration that the compiler in the CppCompile action really is the wrapper from this repo.

Three more things break under bzlmod or Bazel 9:

  • The wrapper scripts reached the gcc archive as external/, which only holds in WORKSPACE mode; bzlmod names an extension-generated repo directory ++, derived from the module's place in the graph, so nothing can hardcode it. The wrappers now glob external/ for the name suffix. Only the repos an action declares are linked into its external/, so exactly one directory matches. Glob after the cd to external/, never across it: pathname expansion resolves symlinks where cd keeps the logical path, and a pattern spanning the ".." walks out of the execroot into the real repo location -- which passes under a sandbox and fails without one. See README.hacks.

  • config.bzl used the native cc_common and CcToolchainConfigInfo globals, which Bazel 9 removed; both now load from @rules_cc//cc:defs.bzl, as do ACTION_NAMES and cc_toolchain_config_lib. tests/BUILD loads cc_binary from rules_cc for the same reason.

  • tests/.bazelrc passed both /usr/bin:/bin and a bare /usr/bin to --sandbox_add_mount_pair. Bazel 9 keeps one mount per source path, so the identity mount displaced the mapping, leaving no /bin/bash for the wrappers' shebang and failing every hermetic build with execvp ENOENT on the wrapper itself. Only the mappings are passed now. Also drop --incompatible_enable_cc_toolchain_resolution, which has been the default since Bazel 7 and is gone in 8.

CI runs every job against 7.7.1 and 9.2.0. The QEMU step no longer spells out the repo directory either, and insists on a single match instead of guessing.

Verified on linux/x86_64 against 7.7.1 and 9.2.0: all three cross configs and the native x86_64 build pass from clean under both the default and the hermetic sandbox, the cross binaries run under QEMU, and non-sandboxed (--spawn_strategy=local) builds work too. The arm64 native case needs an arm64 host, so CI's native-arm64 jobs are the check.

Bazel 9 removed WORKSPACE support outright -- there is no --enable_workspace to turn
it back on -- so a WORKSPACE-only toolchain is invisible to it: Bazel synthesizes an
empty MODULE.bazel and neither the archives nor @rules_cc resolve. Since the new
baseline, 7.7.1, has bzlmod on by default, drop WORKSPACE entirely rather than carry
both. deps.bzl becomes extensions.bzl, whose module extension declares the same
http_archives, and MODULE.bazel registers the toolchains that native.register_toolchains
used to. Consumers move from `aarch64_linux_gnu_deps()` to a bazel_dep plus
archive_override; README.md has the snippet.

Registration is not enough on its own. A dependency's registrations rank below every
one the root module makes, and rules_cc registers an auto-detected host cc toolchain,
which then wins the native x86_64 and aarch64 builds. That fails quietly: the build
succeeds against the host's own gcc. Root modules therefore have to register these
toolchains themselves, which tests/MODULE.bazel and the README now do, and CI asserts
per configuration that the compiler in the CppCompile action really is the wrapper from
this repo.

Three more things break under bzlmod or Bazel 9:

- The wrapper scripts reached the gcc archive as external/<repo name>, which only holds
  in WORKSPACE mode; bzlmod names an extension-generated repo directory
  <module>+<extension>+<name>, derived from the module's place in the graph, so nothing
  can hardcode it. The wrappers now glob external/ for the name suffix. Only the repos
  an action declares are linked into its external/, so exactly one directory matches.
  Glob after the cd to external/, never across it: pathname expansion resolves symlinks
  where cd keeps the logical path, and a pattern spanning the ".." walks out of the
  execroot into the real repo location -- which passes under a sandbox and fails without
  one. See README.hacks.

- config.bzl used the native cc_common and CcToolchainConfigInfo globals, which Bazel 9
  removed; both now load from @rules_cc//cc:defs.bzl, as do ACTION_NAMES and
  cc_toolchain_config_lib. tests/BUILD loads cc_binary from rules_cc for the same reason.

- tests/.bazelrc passed both /usr/bin:/bin and a bare /usr/bin to
  --sandbox_add_mount_pair. Bazel 9 keeps one mount per source path, so the identity
  mount displaced the mapping, leaving no /bin/bash for the wrappers' shebang and
  failing every hermetic build with execvp ENOENT on the wrapper itself. Only the
  mappings are passed now. Also drop --incompatible_enable_cc_toolchain_resolution,
  which has been the default since Bazel 7 and is gone in 8.

CI runs every job against 7.7.1 and 9.2.0. The QEMU step no longer spells out the repo
directory either, and insists on a single match instead of guessing.

Verified on linux/x86_64 against 7.7.1 and 9.2.0: all three cross configs and the native
x86_64 build pass from clean under both the default and the hermetic sandbox, the cross
binaries run under QEMU, and non-sandboxed (--spawn_strategy=local) builds work too. The
arm64 native case needs an arm64 host, so CI's native-arm64 jobs are the check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gartnera
gartnera merged commit 21bfd4a into master Sep 9, 2026
7 checks passed
gartnera added a commit that referenced this pull request Sep 9, 2026
* Address review feedback on the bzlmod migration

Four follow-ups from the review of #24:

Make the hermetic-sandbox guard actually execute something. The spawn
strategy is not part of an action's key, so the hermetic build that
followed a default-sandbox build of the same targets was a pure cache
hit -- 0 spawns -- and could not have caught an under-declared toolchain
input. Fork the configuration with --platform_suffix=hermetic so the
actions have distinct keys. In a fresh output base, in CI order, the
hermetic step goes from 0 spawns to 4.

Add a root .bazelignore for tests. Deleting tests/WORKSPACE removed the
pruning that kept package loading out of the tests/ subtree, so
bazel query/build/test '//...' at the repo root followed the bazel-tests
convenience symlink into the output base and failed to load packages.

Fix the archive_override instructions. archive_override has no sha256
attribute, and integrity will not accept a hex sha256sum -- it needs the
SRI spelling. Document how to produce it.

Make strip and objcopy work under the hermetic sandbox and remote
execution. Unlike the ar/as wrappers, these never exported
LD_LIBRARY_PATH, and Ubuntu's binutils need a libbfd that only exists
inside the archives; the native toolchains additionally omitted
host_libraries from strip_files/objcopy_files, so the library was not
declared either. This was invisible because //:all never created a
CcStrip action, so add a filegroup over the implicit .stripped outputs
-- that pulls strip into every existing build step, plain and hermetic,
in all three jobs.

Verified on Bazel 7.7.1 and 9.2.0 across all four configs (native,
aarch64, jetpack_512, jetpack_62), plain and hermetic. objcopy is fixed
by symmetry with strip but stays unexercised: no cc action in the test
package invokes it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stop the toolchain check from failing on a broken pipe

The verify step piped `bazel aquery --output=text` into `grep -qF` under
`set -o pipefail`. The compiler path first appears ~1.8KB from the end of a
~290KB dump, because the huge `Inputs:` line precedes the command line, so
grep matches and exits while bazel is still writing the tail. Bazel treats
the short stdout write as fatal -- "Cannot write to standard output;
exiting..." -- and exits 255, which pipefail then reports as a mis-selected
toolchain even though grep matched. Whether bazel's final flush beats grep's
exit is pure timing, so this failed only on a loaded runner and only for
whichever check lost the race.

Capture aquery to a file and grep that, which removes the pipe the race needs.
The same snippet was triplicated across the three jobs -- both native jobs
carried the identical bug and were only winning the race -- so it moves to
tests/verify_toolchain.sh alongside verify_binary.sh, taking config/expected
pairs. The captured output also serves the failure diagnostic, so a failing
check no longer re-runs aquery to report what it found.

Verified against the load that reproduced the failure 7/25 times: 0/25 with
the fix, on both Bazel 7.7.1 and 9.2.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants