GPU Programming in Rust with cuda-oxide

This course teaches CUDA programming on NVIDIA GPUs using Rust and NVlabs cuda-oxide. Lessons build from thread indexing and first kernels through asynchronous execution, matrix multiplication optimization, and framework comparisons.

Examples use deterministic inputs and verify GPU results against CPU calculations before reporting success.

Configure the Environment

Run the course on a Linux machine with an Ampere-or-newer NVIDIA GPU (sm_80+), either locally or on a remote GPU instance. Start from a machine image that already provides the NVIDIA driver and CUDA Toolkit. On Debian/Ubuntu hosts, the repository Makefile installs LLVM/Clang, the pinned Rust toolchain, and the matching cargo-oxide command.

References:

• https://github.com/NVlabs/cuda-oxide
• https://nvlabs.github.io/cuda-oxide/getting-started/installation.html

After cloning the course on the Linux GPU machine, prepare the environment and run the first lesson:

make setup
make run-vector-add
make test-vector-add

make setup uses the cuda-oxide revision recorded in the lesson lockfiles. A valid first run reports that all vector-add results are correct, and its test target reports no memory errors.

Test the Examples

Each GPU lesson is a standalone Cargo project exposed through Make targets from the repository root:

make list
make run-<lesson>
make test-<lesson>
make run-all
make test-all

The test targets run the lesson's applicable sanitizer checks, including race checking for the tiled matrix multiplication example and initialization checking for the profiling example. Local checks that do not require a GPU are also available:

make check
make run-python-reference

GPU-backed Python comparisons are exposed through make run-python-gpu after their framework dependencies are installed. See python/triton/README.md and python/pytorch/README.md.

For numerical GPU lessons, a successful run means:

• Deterministic inputs were used.
• Every result was compared with its expected value.
• The binary reported PASSED.
• The applicable sanitizer checks reported no errors.

Learning Path

Module	Topics
lessons/01_first_kernel/	Thread indexing and vector addition
lessons/02_memory_sync_and_async/	Atomics, streams, events, and asynchronous transfers
lessons/03_matmul_optimization/	Matrix multiplication, tiling, profiling, and unrolling
lessons/04_vendor_libraries/	Convolution and activation kernels
lessons/05_framework_comparison/	Softmax, Triton, and PyTorch comparisons

Lessons

Lesson	Topic
vector_add	First CUDA vector-add kernel
indexing	Three-dimensional block/thread index flattening
multidimensional_vector_add	One-dimensional versus three-dimensional launches
atomics	Atomic updates and intentional data races
streams	Streams and explicit dependency ordering
advanced_streams	Pinned transfers, events, and host completion
naive_matmul	Naive matrix multiplication with result checking
tiled_matmul	Shared-memory matrix tiling and barriers
profiling_matmul	CUDA event timing around verified matrix multiplication
profiling	Profiler-friendly verified matrix multiplication markers
unrolling	Loop versus explicitly expanded calculation
convolution	NCHW convolution kernel and layout indexing
tanh	Elementwise tanh activation with bounded approximation
softmax	Numerically stable softmax and GPU row ownership
polynomial_activation	Elementwise polynomial activation kernel
Triton examples	Triton and PyTorch comparisons for vector addition and softmax
PyTorch tanh	Stable activation formula compared with PyTorch CUDA

Develop from Another Machine

When the NVIDIA GPU is on a remote Linux host, edit locally and sync course files before running the same Make targets on that host:

export REMOTE_HOST="<GPU_HOST>"
export REMOTE_USER="<USER>"
export REMOTE_DIR="~/cuda-course-rust"
# export REMOTE_PORT="<SSH_PORT>"       # Optional; defaults to 22.
# export SSH_KEY="$HOME/.ssh/id_ed25519" # Optional.

make sync

The sync target excludes build output and local-only planning or editor files.