Skip to content

ci: add riscv64-linux build via cross-compilation on ubuntu-24.04#621

Merged
alexcrichton merged 5 commits intoWebAssembly:mainfrom
gounthar:ci/riscv64-linux-v2
Apr 8, 2026
Merged

ci: add riscv64-linux build via cross-compilation on ubuntu-24.04#621
alexcrichton merged 5 commits intoWebAssembly:mainfrom
gounthar:ci/riscv64-linux-v2

Conversation

@gounthar
Copy link
Copy Markdown
Contributor

@gounthar gounthar commented Apr 5, 2026

Rework of the riscv64-linux CI build to use CMake cross-compilation on a standard ubuntu-24.04 runner, rather than a native RISE runner.

What changed

ci/docker/Dockerfile.riscv64-linux (new):

  • Ubuntu 24.04 base — has crossbuild-essential-riscv64 in its package repos
  • Sets CC=riscv64-linux-gnu-gcc / CXX=riscv64-linux-gnu-g++ so CMake detects cross-compilation and causes LLVM to build a native llvm-tblgen first, then cross-compile the rest of the toolchain
  • Sets CARGO_TARGET_RISCV64_UNKNOWN_LINUX_GNU_LINKER for Rust cross-builds
  • XDG_CACHE_HOME=/tmp/cache avoids write permission issues in the container

ci/docker-build.sh:

  • Select ci/docker/Dockerfile.<artifact> if it exists, fall back to the default ci/docker/Dockerfile
  • Make the wasmtime volume mount conditional on WASI_SDK_CI_SKIP_SYSROOT != 1

.github/workflows/main.yml:

  • New riscv64-linux matrix entry: os: ubuntu-24.04, rust_target: riscv64-unknown-linux-gnu
  • cross_cmake_args: -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=riscv64 -DWASI_SDK_LLDB=OFF
  • WASI_SDK_CI_SKIP_SYSROOT: 1
  • Handle cross_cmake_args in the cmake flags step

Why WASI_SDK_CI_SKIP_SYSROOT

The cross-compiled clang runs on riscv64, not on the x86_64 build host, so the wasm sysroot step is skipped.

Why WASI_SDK_LLDB=OFF

Avoids cross-compiling libedit and libxml2 in this first iteration; can be re-enabled as a follow-up.

Closes #607

@tschneidereit
Copy link
Copy Markdown
Member

wasmtime does not yet publish riscv64 binaries,

I can't comment much on the rest of this PR, but note that Wasmtime does publish releases for the riscv64gc target.

@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 5, 2026

You're right, thanks for the correction. Wasmtime does publish riscv64gc-linux binaries (since v27+).

The actual blocker is that bytecodealliance/actions/wasmtime/setup doesn't handle riscv64 yet. os.arch() returns 'riscv64' and the action throws Unsupported operating system architecture. I've opened a fix at bytecodealliance/actions#20.

Once that lands, the if: runner.arch != 'RISCV64' skip in install-deps can be removed and sysroot tests can run natively. Updating the PR description to reflect the real reason.

@alexcrichton
Copy link
Copy Markdown
Collaborator

Could you expand on why the cross-compilation approach didn't work? Personally I'd prefer to use the stock github actions runners rather than a custom one since we've had trouble with custom providers in the past, and I also think that cross-compilation should be possible to get working.

@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 6, 2026

Cross-compilation wasn't tried and then abandoned; I just didn't go that route. I followed the same pattern as arm64-linux, which uses ubuntu-22.04-arm (a native ARM runner) rather than cross-compiling LLVM from x86_64. Maybe too quick an assumption on my part.

Your concern about custom providers makes sense. The RISE runner is third-party infrastructure, unlike the GitHub-hosted ARM runner.

Cross-compiling LLVM for a riscv64 host from x86_64 is technically possible but needs a two-stage build: first building llvm-tblgen natively, then cross-compiling the rest with a riscv64 sysroot and cross-toolchain. The existing build scripts don't have that path. Docker-based cross-compilation via QEMU would work but would likely be very slow; the native build already takes 8-10 hours on rv64gc hardware.

If you have a preferred direction (CMake cross-compilation with a riscv64 toolchain, or Docker buildx with QEMU), I am happy to try it. Just let me know what the setup should look like.

@alexcrichton
Copy link
Copy Markdown
Collaborator

Ah I originally saw f678512 and got my presumptions from there I think...

Regardless, however, I do think that cross-compilation is the way to go here. We got lucky with the arm64 runners where right as there was a desire to have them GitHub officially added them to its selection of runners meaning we didn't have to deal with the cross-compilation bit for arm64. If you're willing, however, I'd prefer to add CMake configuration/etc to cross-compile LLVM (doing the whole two-stage build thing and whatnot) for riscv64. I suspect cross-compilation will be much faster than native hardware, I agree it'll be way faster than QEMU, and it'll also mean we don't have to rely on third-party runners.

Rework from native RISE runner to standard ubuntu-24.04 runner with
CMake cross-compilation, following alexcrichton's preference.

Changes:
- ci/docker/Dockerfile.riscv64-linux: Ubuntu 24.04 with
  crossbuild-essential-riscv64; sets CC/CXX to riscv64-linux-gnu-gcc
  so CMake detects cross-compilation and LLVM builds a native tablegen
  before cross-compiling the rest of the toolchain.
- ci/docker-build.sh: use artifact-specific Dockerfile if one exists
  (ci/docker/Dockerfile.<artifact>), fall back to default; make the
  wasmtime volume mount conditional on WASI_SDK_CI_SKIP_SYSROOT.
- .github/workflows/main.yml: new riscv64-linux matrix entry on
  ubuntu-24.04 with cross_cmake_args (-DCMAKE_SYSTEM_NAME=Linux
  -DCMAKE_SYSTEM_PROCESSOR=riscv64 -DWASI_SDK_LLDB=OFF) and
  WASI_SDK_CI_SKIP_SYSROOT=1; handle cross_cmake_args in the cmake
  flags step.

WASI_SDK_CI_SKIP_SYSROOT=1: the cross-compiled clang runs on riscv64,
not on the x86_64 build host, so the wasm sysroot step is skipped.
WASI_SDK_LLDB=OFF: avoids cross-compiling libedit/libxml2 in this
first iteration; can be re-enabled as a follow-up.

Closes WebAssembly#607

Signed-off-by: Bruno Verachten <gounthar@gmail.com>
@gounthar gounthar force-pushed the ci/riscv64-linux-v2 branch from 91887df to 0d34ab1 Compare April 7, 2026 16:11
@gounthar gounthar changed the title ci: add riscv64-linux build using native RISE runner ci: add riscv64-linux build via cross-compilation on ubuntu-24.04 Apr 7, 2026
@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 7, 2026

Reworked to use CMake cross-compilation on a standard ubuntu-24.04 runner — no custom runner needed.

The approach: a new ci/docker/Dockerfile.riscv64-linux installs crossbuild-essential-riscv64 and sets CC=riscv64-linux-gnu-gcc / CXX=riscv64-linux-gnu-g++. CMake detects cross-compilation from those and handles the two-stage build (native llvm-tblgen first, then cross-compiles the toolchain). docker-build.sh selects an artifact-specific Dockerfile when one exists. The matrix entry uses ubuntu-24.04 with cross_cmake_args for CMAKE_SYSTEM_NAME and CMAKE_SYSTEM_PROCESSOR.

WASI_SDK_CI_SKIP_SYSROOT=1 and WASI_SDK_LLDB=OFF are kept for the same reasons as before.

Copy link
Copy Markdown
Collaborator

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I suspect there'll be a number of stumbling blocks to overcome in CI, but once that's green happy to take another pass.

Comment thread .github/workflows/main.yml Outdated

- artifact: riscv64-linux
os: ubuntu-24.04
rust_target: riscv64-unknown-linux-gnu
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This'll want to be riscv64gc-unknown-linux-gnu

Copy link
Copy Markdown
Contributor Author

@gounthar gounthar Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch on the triple -- pushed a fix for that.

There was also a second stumbling block: setting CC/CXX as Docker ENV vars meant LLVM's native tblgen sub-build inherited the riscv64 cross-compiler too, which gave an Exec format error when the build tried to run llvm-min-tblgen on the x86_64 host. I moved the compilers to -DCMAKE_C/CXX_COMPILER cmake flags and added WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS with CMAKE_SYSTEM_NAME/CMAKE_SYSTEM_PROCESSOR to trigger LLVM's native/target split -- hopefully that is the right approach, but happy to rework if not.

The correct Rust target triple for riscv64 Linux is
riscv64gc-unknown-linux-gnu, not riscv64-unknown-linux-gnu.
The gc suffix denotes the G (general-purpose) + C (compressed)
ISA extensions that the standard Linux ABI requires.

Also fix the matching CARGO_TARGET_ env var name in the Dockerfile
so Cargo picks up the cross linker for the correct target.

Signed-off-by: Bruno Verachten <gounthar@gmail.com>
@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 8, 2026

The job was failing at the rustup target add step:

error: toolchain 'stable-x86_64-unknown-linux-gnu' does not support target 'riscv64-unknown-linux-gnu'; did you mean 'riscv64gc-unknown-linux-gnu'?

Two places had the wrong triple (riscv64-unknown-linux-gnu instead of riscv64gc-unknown-linux-gnu):

  • rust_target in .github/workflows/main.yml
  • CARGO_TARGET_RISCV64_UNKNOWN_LINUX_GNU_LINKER in ci/docker/Dockerfile.riscv64-linux (renamed to CARGO_TARGET_RISCV64GC_UNKNOWN_LINUX_GNU_LINKER)

The gc suffix is part of the standard Linux ABI target triple and required by rustup. Fixed in the latest commit.

gounthar added 2 commits April 8, 2026 09:56
Setting CC/CXX as Docker ENV vars caused LLVM's native tblgen
sub-build to also use the riscv64 cross-compiler, producing an
riscv64 binary that fails to run on the x86_64 host:
  Exec format error: llvm-min-tblgen

Fix: pass the cross-compiler via CMAKE_C/CXX_COMPILER cmake flags
instead (cmake cache vars are not inherited by subprocess cmake
invocations, so LLVM's native build finds the host compiler).

Also pass CMAKE_SYSTEM_NAME=Linux/CMAKE_SYSTEM_PROCESSOR=riscv64 to
LLVM via WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS so LLVM sets
CMAKE_CROSSCOMPILING=TRUE and triggers the native/target build split,
matching the pattern used by the macOS matrix entries.

Signed-off-by: Bruno Verachten <gounthar@gmail.com>
@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 8, 2026

The next failure is a classic LLVM cross-compilation issue. Setting CC/CXX as Docker ENV vars caused LLVM's native tblgen sub-build to also compile with the riscv64 cross-compiler, producing a binary that can't run on the x86_64 build host:

Exec format error: llvm-min-tblgen

When CMAKE_CROSSCOMPILING=TRUE, LLVM creates a separate native sub-build for tools like llvm-tblgen and llvm-min-tblgen. The native cmake invocation does not explicitly pass -DCMAKE_C_COMPILER, but it does inherit environment variables, so CC=riscv64-linux-gnu-gcc from the Dockerfile gets picked up.

Two changes (following the same pattern as the macOS matrix entries):

  1. Removed ENV CC/CXX from the Dockerfile. The cross-compiler is now passed via -DCMAKE_C_COMPILER/-DCMAKE_CXX_COMPILER cmake flags. CMake cache vars are not inherited by subprocess cmake invocations, so LLVM's native build finds the host compiler.

  2. Added WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS: -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=riscv64 to the matrix env. This tells LLVM it is cross-compiling and triggers the native/target build split.

Signed-off-by: Bruno Verachten <gounthar@gmail.com>
@alexcrichton alexcrichton merged commit 42b7263 into WebAssembly:main Apr 8, 2026
11 checks passed
@alexcrichton
Copy link
Copy Markdown
Collaborator

Thanks!

@gounthar
Copy link
Copy Markdown
Contributor Author

gounthar commented Apr 8, 2026

Thanks to both of you. @alexcrichton, your suggestion to go with cross-compilation was the better direction -- I should have looked at it from the start rather than defaulting to the native runner approach. And @tschneidereit, glad you caught the wasmtime point early, I had that wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add riscv64 host platform support

3 participants