Repository navigation
Move to bzlmod and support Bazel 9 - #24
Merged
Merged
Conversation
Bazel 9 removed WORKSPACE support outright -- there is no --enable_workspace to turn it back on -- so a WORKSPACE-only toolchain is invisible to it: Bazel synthesizes an empty MODULE.bazel and neither the archives nor @rules_cc resolve. Since the new baseline, 7.7.1, has bzlmod on by default, drop WORKSPACE entirely rather than carry both. deps.bzl becomes extensions.bzl, whose module extension declares the same http_archives, and MODULE.bazel registers the toolchains that native.register_toolchains used to. Consumers move from `aarch64_linux_gnu_deps()` to a bazel_dep plus archive_override; README.md has the snippet. Registration is not enough on its own. A dependency's registrations rank below every one the root module makes, and rules_cc registers an auto-detected host cc toolchain, which then wins the native x86_64 and aarch64 builds. That fails quietly: the build succeeds against the host's own gcc. Root modules therefore have to register these toolchains themselves, which tests/MODULE.bazel and the README now do, and CI asserts per configuration that the compiler in the CppCompile action really is the wrapper from this repo. Three more things break under bzlmod or Bazel 9: - The wrapper scripts reached the gcc archive as external/<repo name>, which only holds in WORKSPACE mode; bzlmod names an extension-generated repo directory <module>+<extension>+<name>, derived from the module's place in the graph, so nothing can hardcode it. The wrappers now glob external/ for the name suffix. Only the repos an action declares are linked into its external/, so exactly one directory matches. Glob after the cd to external/, never across it: pathname expansion resolves symlinks where cd keeps the logical path, and a pattern spanning the ".." walks out of the execroot into the real repo location -- which passes under a sandbox and fails without one. See README.hacks. - config.bzl used the native cc_common and CcToolchainConfigInfo globals, which Bazel 9 removed; both now load from @rules_cc//cc:defs.bzl, as do ACTION_NAMES and cc_toolchain_config_lib. tests/BUILD loads cc_binary from rules_cc for the same reason. - tests/.bazelrc passed both /usr/bin:/bin and a bare /usr/bin to --sandbox_add_mount_pair. Bazel 9 keeps one mount per source path, so the identity mount displaced the mapping, leaving no /bin/bash for the wrappers' shebang and failing every hermetic build with execvp ENOENT on the wrapper itself. Only the mappings are passed now. Also drop --incompatible_enable_cc_toolchain_resolution, which has been the default since Bazel 7 and is gone in 8. CI runs every job against 7.7.1 and 9.2.0. The QEMU step no longer spells out the repo directory either, and insists on a single match instead of guessing. Verified on linux/x86_64 against 7.7.1 and 9.2.0: all three cross configs and the native x86_64 build pass from clean under both the default and the hermetic sandbox, the cross binaries run under QEMU, and non-sandboxed (--spawn_strategy=local) builds work too. The arm64 native case needs an arm64 host, so CI's native-arm64 jobs are the check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
aredman-agtonomy
approved these changes
Sep 9, 2026
gartnera
added a commit
that referenced
this pull request
Sep 9, 2026
* Address review feedback on the bzlmod migration Four follow-ups from the review of #24: Make the hermetic-sandbox guard actually execute something. The spawn strategy is not part of an action's key, so the hermetic build that followed a default-sandbox build of the same targets was a pure cache hit -- 0 spawns -- and could not have caught an under-declared toolchain input. Fork the configuration with --platform_suffix=hermetic so the actions have distinct keys. In a fresh output base, in CI order, the hermetic step goes from 0 spawns to 4. Add a root .bazelignore for tests. Deleting tests/WORKSPACE removed the pruning that kept package loading out of the tests/ subtree, so bazel query/build/test '//...' at the repo root followed the bazel-tests convenience symlink into the output base and failed to load packages. Fix the archive_override instructions. archive_override has no sha256 attribute, and integrity will not accept a hex sha256sum -- it needs the SRI spelling. Document how to produce it. Make strip and objcopy work under the hermetic sandbox and remote execution. Unlike the ar/as wrappers, these never exported LD_LIBRARY_PATH, and Ubuntu's binutils need a libbfd that only exists inside the archives; the native toolchains additionally omitted host_libraries from strip_files/objcopy_files, so the library was not declared either. This was invisible because //:all never created a CcStrip action, so add a filegroup over the implicit .stripped outputs -- that pulls strip into every existing build step, plain and hermetic, in all three jobs. Verified on Bazel 7.7.1 and 9.2.0 across all four configs (native, aarch64, jetpack_512, jetpack_62), plain and hermetic. objcopy is fixed by symmetry with strip but stays unexercised: no cc action in the test package invokes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Stop the toolchain check from failing on a broken pipe The verify step piped `bazel aquery --output=text` into `grep -qF` under `set -o pipefail`. The compiler path first appears ~1.8KB from the end of a ~290KB dump, because the huge `Inputs:` line precedes the command line, so grep matches and exits while bazel is still writing the tail. Bazel treats the short stdout write as fatal -- "Cannot write to standard output; exiting..." -- and exits 255, which pipefail then reports as a mis-selected toolchain even though grep matched. Whether bazel's final flush beats grep's exit is pure timing, so this failed only on a loaded runner and only for whichever check lost the race. Capture aquery to a file and grep that, which removes the pipe the race needs. The same snippet was triplicated across the three jobs -- both native jobs carried the identical bug and were only winning the race -- so it moves to tests/verify_toolchain.sh alongside verify_binary.sh, taking config/expected pairs. The captured output also serves the failure diagnostic, so a failing check no longer re-runs aquery to report what it found. Verified against the load that reproduced the failure 7/25 times: 0/25 with the fix, on both Bazel 7.7.1 and 9.2.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bazel 9 removed WORKSPACE support outright -- there is no --enable_workspace to turn it back on -- so a WORKSPACE-only toolchain is invisible to it: Bazel synthesizes an empty MODULE.bazel and neither the archives nor @rules_cc resolve. Since the new baseline, 7.7.1, has bzlmod on by default, drop WORKSPACE entirely rather than carry both. deps.bzl becomes extensions.bzl, whose module extension declares the same http_archives, and MODULE.bazel registers the toolchains that native.register_toolchains used to. Consumers move from
aarch64_linux_gnu_deps()to a bazel_dep plus archive_override; README.md has the snippet.Registration is not enough on its own. A dependency's registrations rank below every one the root module makes, and rules_cc registers an auto-detected host cc toolchain, which then wins the native x86_64 and aarch64 builds. That fails quietly: the build succeeds against the host's own gcc. Root modules therefore have to register these toolchains themselves, which tests/MODULE.bazel and the README now do, and CI asserts per configuration that the compiler in the CppCompile action really is the wrapper from this repo.
Three more things break under bzlmod or Bazel 9:
The wrapper scripts reached the gcc archive as external/, which only holds in WORKSPACE mode; bzlmod names an extension-generated repo directory ++, derived from the module's place in the graph, so nothing can hardcode it. The wrappers now glob external/ for the name suffix. Only the repos an action declares are linked into its external/, so exactly one directory matches. Glob after the cd to external/, never across it: pathname expansion resolves symlinks where cd keeps the logical path, and a pattern spanning the ".." walks out of the execroot into the real repo location -- which passes under a sandbox and fails without one. See README.hacks.
config.bzl used the native cc_common and CcToolchainConfigInfo globals, which Bazel 9 removed; both now load from @rules_cc//cc:defs.bzl, as do ACTION_NAMES and cc_toolchain_config_lib. tests/BUILD loads cc_binary from rules_cc for the same reason.
tests/.bazelrc passed both /usr/bin:/bin and a bare /usr/bin to --sandbox_add_mount_pair. Bazel 9 keeps one mount per source path, so the identity mount displaced the mapping, leaving no /bin/bash for the wrappers' shebang and failing every hermetic build with execvp ENOENT on the wrapper itself. Only the mappings are passed now. Also drop --incompatible_enable_cc_toolchain_resolution, which has been the default since Bazel 7 and is gone in 8.
CI runs every job against 7.7.1 and 9.2.0. The QEMU step no longer spells out the repo directory either, and insists on a single match instead of guessing.
Verified on linux/x86_64 against 7.7.1 and 9.2.0: all three cross configs and the native x86_64 build pass from clean under both the default and the hermetic sandbox, the cross binaries run under QEMU, and non-sandboxed (--spawn_strategy=local) builds work too. The arm64 native case needs an arm64 host, so CI's native-arm64 jobs are the check.