Skip to content

bazel/wasm: upgrade wasmtime from 32.0.0 to 42.0.2#30263

Open
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
travisdowns:td-hermetic-wasmtime42
Open

bazel/wasm: upgrade wasmtime from 32.0.0 to 42.0.2#30263
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
travisdowns:td-hermetic-wasmtime42

Conversation

@travisdowns
Copy link
Copy Markdown
Member

@travisdowns travisdowns commented Apr 23, 2026

Motivation: make the Bazel build of //:redpanda byte-for-byte reproducible. #30187 is related to handles for hwloc, openssl via local rules_foreign_cc build tweaks, but for the wasmtime / cranelift side we had two local reproducibility patches that were already fixed upstream and only required a version bump to pick up. Per the discussion on #30187, the cleaner path is to drop the patches and upgrade wasmtime instead — which is what this PR does.

The minimum wasmtime version that carries both cranelift fixes is v42, so this is a jump from v32.0.0 to v42.0.2. The upgrade brings along the following required adjustments:

  • Bump Rust toolchain from 1.86.0 to 1.91.0 (wasmtime 42 MSRV).
  • Add PULLEY and ALL_ARCH feature flags to the wasmtime.BUILD conf.h template substitutions (new in wasmtime 42).
  • Drop the wasmtime_config_async_support_set() call — async support is implicit now, the separate toggle was deprecated and removed from the C API in bytecodealliance/wasmtime#12371.
  • Bump rules_rust 0.60.0 → 0.70.0 and opt in to experimental_use_allocator_libraries_with_mangled_symbols so the mangled __rustc::* allocator shim symbols (introduced in rustc 1.87+) are provided when a Rust staticlib is linked into a cc_binary via cc_common.link. Without this flag the link fails with undefined references to __rustc::__rust_alloc, __rustc::__rust_dealloc, etc. Background:
  • Mirror the rustc tarball through vectorized-public to reduce dependence on static.rust-lang.org.
  • Incidental: bump bazel_skylib 1.8.1 → 1.8.2 and rules_shell 0.4.1 → 0.6.1 to silence direct-dependency version warnings pulled in by the new rules_rust.

Supersedes #30250, which took the opposite approach of carrying a local rust_alloc_shim workaround. Bumping rules_rust and flipping the flag is cleaner than maintaining a custom shim — per the approach suggested by fhanau in rules_rust#3459.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.3.x
  • v25.2.x
  • v25.1.x

Release Notes

  • none

Copilot AI review requested due to automatic review settings April 23, 2026 13:45
@travisdowns
Copy link
Copy Markdown
Member Author

@claude review

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the Bazel-managed Wasmtime/Cranelift dependency stack to Wasmtime v42.0.2 to eliminate local reproducibility patches and move toward byte-for-byte reproducible //:redpanda builds.

Changes:

  • Bump Wasmtime crates (wasmtime and wasmtime-c-api-impl) from 32.0.0 → 42.0.2 and refresh Cargo.lock accordingly.
  • Update Bazel integration for the new Wasmtime version (new conf.h feature-template substitutions; remove deprecated C API call).
  • Update Bazel module/tooling dependencies (notably rules_rust), bump Rust toolchain version, and add allocator-symbol setting needed for newer rustc.

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/v/wasm/wasmtime.cc Removes the deprecated wasmtime_config_async_support_set call to match Wasmtime v42 C API behavior.
bazel/thirdparty/wasmtime.BUILD Extends conf.h template substitutions for new Wasmtime v42 feature macros.
bazel/thirdparty/Cargo.toml Bumps Wasmtime crate versions to 42.0.2.
bazel/thirdparty/Cargo.lock Updates the resolved Rust dependency graph for Wasmtime 42.0.2.
MODULE.bazel Bumps rules_rust (and other Bazel deps), updates Rust toolchain version, and adds mirrored rustc URL fallback.
.bazelrc Enables the rules_rust allocator-library setting required for rustc 1.87+ allocator shim symbol mangling.

Comment thread MODULE.bazel Outdated
# Prefer our S3 mirror for the rustc tarball (static.rust-lang.org is
# frequently slow from some POPs). Only the `rustc-*` component is
# mirrored; other components (rust-std, cargo, clippy, rustfmt,
# llvm-tools, rust-src) 404 on the mirror and fall through to the
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this refer to whatever below does or how the rust CDN files work?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure I got the question.

Do you refer to the "fall through"? That's a bazel feature. It will try the urls in order.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless I think I should just remove this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this as I think it's more a "me" problem.

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

CI test results

test results on build#83568
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(PASS) UpgradeBackToBackTest test_upgrade_with_all_workloads {"single_upgrade": false} integration https://buildkite.com/redpanda/redpanda/builds/83568#019dbaa1-4c68-488c-8db1-0903433de6f0 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0000, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=UpgradeBackToBackTest&test_method=test_upgrade_with_all_workloads

Additional changes required by the upgrade:

- Bump Rust toolchain from 1.86.0 to 1.91.0 (wasmtime 42 MSRV)
- Add PULLEY and ALL_ARCH feature flags to wasmtime.BUILD conf.h
  template substitutions (new in wasmtime 42)
- Remove wasmtime_config_async_support_set() call: async support is
  now implicit, the separate toggle was deprecated and removed from
  the C API (bytecodealliance/wasmtime#12371)
- Bump rules_rust 0.60.0 -> 0.70.0 and opt in to
  experimental_use_allocator_libraries_with_mangled_symbols so the
  mangled __rustc::* allocator shim symbols (introduced in rustc
  1.87+) are provided when a Rust staticlib is linked into a
  cc_binary via cc_common.link. Without this flag the link fails
  with undefined references to __rustc::__rust_alloc,
  __rustc::__rust_dealloc, etc. Background:
    - Issue:   bazelbuild/rules_rust#3459
    - Fix PR:  bazelbuild/rules_rust#3403
    - Landed in rules_rust 0.63.0 as an opt-in setting.
- Mirror the rustc tarball through vectorized-public to reduce
  dependence on static.rust-lang.org.
- Incidental: bump bazel_skylib 1.8.1 -> 1.8.2 and rules_shell
  0.4.1 -> 0.6.1 to silence direct-dependency version warnings
  pulled in by the new rules_rust.
Comment thread src/v/wasm/wasmtime.cc
//
// See the documentation for more information:
// https://docs.wasmtime.dev/api/wasmtime/struct.Config.html#asynchronous-wasm
wasmtime_config_async_support_set(config, true);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async support is now implicit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants