--incompatible_enable_cc_toolchain_resolution + universal tools fails to build

[keith]

With this command:

USE_BAZEL_VERSION=last_green bazel build --@build_bazel_rules_swift//swift:universal_tools --host_platform=@build_bazel_apple_support//platforms:macos_x86_64 tools/worker:worker_wrapper --incompatible_enable_cc_toolchain_resolution examples/xplatform/hello_world

On this branch #1102

The build fails with:

fatal error: /Applications/Xcode-14.3.1-RC1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: bazel-out/darwin_arm64-opt-exec-ST-311b1d6486e8/bin/tools/worker/worker and bazel-out/darwin_x86_64-opt-exec-ST-6f7624269444/bin/tools/worker/worker have the same architectures (x86_64) and can't be in the same fat output file

It doesn't file as long as you have platform_mappings in the case you build the worker directly, so it seems like it must have something to do with it being an exec tool.

I was testing this in preparation for bazelbuild/bazel#19441. If you remove --incompatible_enable_cc_toolchain_resolution it builds again.

[keith]

@comius any thoughts here?

[comius]

You need to have either platform_mappings or the change in transitions to also set platform flag. Otherwise —incompatible_enable_cc_toolchains doesn’t work, because it can’t pick up the right C++ toolchain

[keith]

this is with the platform_mappings from your PR and bzlmod disabled

[comius]

Thanks, I’ll investigate tomorrow.

[comius]

I can't reproduce this.
Bazel build at d57e0a71008c49594872160647756c87f63f5fa1
rules_swift at f4f0ebf
XCode 14.3.1

Command: bazel build --@build_bazel_rules_swift//swift:universal_tools --host_platform=@build_bazel_apple_support//platforms:macos_x86_64 tools/worker:worker_wrapper --incompatible_enable_cc_toolchain_resolution examples/xplatform/hello_world

Result: INFO: Build completed successfully, 97 total actions, no warnings.

I did clean --expunge and shutdown and it still succeeds.

The build succeeds with all 4 combinations of --universal_tools and --host_platform present/not-present.

[comius]

I get a failure using bzlmod and its platform_mappings. This is the simplest build, that's already failing.

bazel build --incompatible_enable_cc_toolchain_resolution --enable_bzlmod --platform_mappings=platform_mappings_bzlmod examples/xplatform/hello_world

ERROR: /private/var/tmp/_bazel_ilist/9cf692febd10e3f926566148e9a74eb8/external/_main~non_module_deps~build_bazel_rules_swift_local_config/BUILD:9:22: in xcode_swift_toolchain rule @_main~non_module_deps~build_bazel_rules_swift_local_config//:toolchain: 

	File "/Users/ilist/src/rules_swift/swift/internal/xcode_swift_toolchain.bzl", line 549, column 29, in _xcode_swift_toolchain_impl
		target_triples.parse(cc_toolchain.target_gnu_system_name),
	File "/Users/ilist/src/rules_swift/swift/internal/target_triples.bzl", line 135, column 28, in _parse
		vendor = components[1],
Error: index out of range (index is 1, but sequence has 1 elements)

This is probably caused by some differences between WORKSPACE/bzlmod mode.

Thanks @keith for bzlmod platform_mapping, I wasn't able to test this yesterday.

[comius]

With #1106, I again get all the builds to pass.

[keith]

i haven't seen that exception (and can't repro with that command on a m1 machine), given the conversation on that linked PR is that something we still need to debug?

[comius]

The exception with bzlmod is being fixed.

I can’t reproduce the bug you’ve reported, but I also don’t have M1.

Can you after the failed build run “bazel config”, search for the two st hashes to get config hashes and then post the output of “bazel config hash1 hash2”?

I can do a bit more investigation on that data, but without M1 machine it’ll be hard to fix. It seems likely the problem is with C++ toolchain setup on M1.

[keith]

30e79b2a8a87725ecb4c74fd0ccbb6e432378a9e81e0dfa4365802fa3aceec1f darwin_x86_64-opt-exec-ST-6f7624269444 (exec)
6ec13afe4a19115726f5fab750f843e53de0eb361f37ae8e04c5814a8b71133e darwin_arm64-opt-exec-ST-311b1d6486e8 (exec)
6fb808dbab8243b3661ce7d11178b1ab77a848bb7061704d0bfc05a5a78dd1ea fastbuild-noconfig
82b6712394747cef0087354f2214849a0cd8e1fb4c2fdea611dc5e9a8ad5eda5 darwin_arm64-fastbuild
ac0f43ee4c342af463337b5535f56b7e1df9585a69c4c73f4b161cb504c0bbe0 darwin_arm64-fastbuild
b66ca617b3e3d583856ac3e33dae9b26104bc9f52ab7a0538f1d0d2b985cf834 darwin_x86_64-fastbuild-ST-6f7624269444

the diff is expected:

Displaying diff between configs 30e79b2a8a87725ecb4c74fd0ccbb6e432378a9e81e0dfa4365802fa3aceec1f and 6ec13afe4a19115726f5fab750f843e53de0eb361f37ae8e04c5814a8b71133e
FragmentOptions com.google.devtools.build.lib.analysis.PlatformOptions {
  platforms: [@build_bazel_apple_support//platforms:macos_x86_64], [@build_bazel_apple_support//platforms:darwin_x86_64]
}
FragmentOptions com.google.devtools.build.lib.analysis.config.CoreOptions {
  cpu: darwin_x86_64, darwin_arm64
}

[keith]

on an intel mac it repros for me too but you have to switch --host_platform to be the opposite arch from what you have. so pass --host_platform=@build_bazel_apple_support//platforms:macos_arm64 instead if you're on intel

[comius]

Thanks, the flag helped me reproduce. The problem was kind of obvious: bazelbuild/apple_support#257

General rule is that if there's a transition on cpu flag, you need to add platforms flag. To get the right C++ toolchain. (This is documented in bazelbuild/bazel#7260, under point 4; but I only recently updated it)

[keith]

thanks for debugging! is there any way we could make bazel more obvious about this type of failure? these seem really tough to spot

[comius]

Maybe. We'd need to hardcode some information, because the failure happens because of cpu and platforms flag mismatch.

But the the flags also won't need to match in the future. After C++ toolchain resolution is flipped on, you could remove cpu flag.

The configurations are different and that's all Bazel can tell. You can't really tell if the rules are using toolchains based on platforms or cpu flag or even a foo flag.