Skip to content

README.md

Latest commit

 

History

History
310 lines (220 loc) · 6.87 KB

README.md

File metadata and controls

310 lines (220 loc) · 6.87 KB
Raw

🚀 NVIDIA H200 GPU STREAM Triad Benchmarking

Platform CUDA GPU Benchmark Precision

Professional benchmarking guide for measuring memory bandwidth performance of NVIDIA H200 GPUs using the STREAM Triad Benchmark.

📖 Table of Contents

  • Overview
  • STREAM Triad Theory
  • NVIDIA H200 Specifications
  • Requirements
  • Environment Setup
  • Running the Benchmark
  • Understanding the Results
  • Benchmark Results
  • Performance Analysis
  • GPU Topology
  • Best Practices
  • Conclusion

Overview

The STREAM Benchmark is an industry-standard benchmark used to measure sustained memory bandwidth and the corresponding computation rate for CPUs and GPUs.

This documentation demonstrates benchmarking 8× NVIDIA H200 GPUs using the Triad kernel in FP32 (Single Precision).

The STREAM Triad performs the following operation:

[ A[i] = B[i] + scalar \times C[i] ]

Since this operation is memory-bound, it is widely used to evaluate:

  • HPC clusters
  • AI infrastructure
  • Scientific simulations
  • Large-scale memory performance
  • GPU memory subsystem efficiency

STREAM Triad Theory

The Triad kernel performs:

A[i] = B[i] + scalar × C[i]

Each iteration:

  • Reads array B
  • Reads array C
  • Multiplies C by a scalar
  • Adds the result to B
  • Writes the result into A

This benchmark primarily measures:

  • Sustained memory bandwidth
  • Memory subsystem efficiency
  • GPU memory performance
  • Kernel execution time

NVIDIA H200 GPU Specifications

Feature Specification
GPU Architecture Hopper
Compute Capability 9.0
Streaming Multiprocessors 132
CUDA Cores 8,448
Tensor Cores 528
Memory 80 GB HBM3
Memory Clock 3201 MHz
Memory Bus Width 6144-bit
Peak Memory Bandwidth 4,916.7 GB/s
FP32 Performance ~60 TFLOPS
NVLink Generation 4
PCIe Gen5
ECC Enabled
TDP ~700W

Hardware Requirements

  • NVIDIA H200 GPU(s)
  • CUDA Compute Capability ≥ 9.0
  • Minimum 1.5 GB GPU memory for benchmark arrays
  • Linux Server

Software Requirements

  • Ubuntu / Debian Linux
  • CUDA Toolkit 12.x
  • NVIDIA Driver
  • STREAM GPU Benchmark (stream_test_fp32)

Environment Setup

Load CUDA

module load cuda/12

Navigate to Benchmark Directory

cd /scratch/hpc-benchmarks/cuda12/

Verify GPU Availability

nvidia-smi

Expected output:

  • 8× NVIDIA H200 GPUs
  • ECC Enabled
  • Driver Loaded
  • CUDA Available

Benchmark Parameters

Option Description
-d GPU Device ID
-n Number of Array Elements
-t STREAM Test Type

For this benchmark:

Parameter Value
Elements 50,000,000
Precision FP32
Test Triad

Running the Benchmark

Single GPU

/scratch/hpc-benchmarks/cuda12/stream-gpu-linux-x86_64/stream_test_fp32 \
-d 0 \
-n 50000000 \
-t T

Benchmark All GPUs

for i in {0..7}; do
    echo "=================================="
    echo "Running GPU $i"
    echo "=================================="

    /scratch/hpc-benchmarks/cuda12/stream-gpu-linux-x86_64/stream_test_fp32 \
        -d $i \
        -n 50000000 \
        -t T
done

Sample Output

Function      Rate (MB/s)     Avg Time    Min Time    Max Time

Triad:      4,233,461         0.0001      0.0001      0.0001

Benchmark Results

GPU Rate (MB/s) Approx. TB/s Avg Time Min Time Max Time
GPU0 4,233,461 4.23 0.0001 0.0001 0.0001
GPU1 4,205,922 4.21 0.0001 0.0001 0.0001
GPU2 4,176,877 4.18 0.0001 0.0001 0.0002
GPU3 4,102,845 4.10 0.0001 0.0001 0.0002
GPU4 4,117,259 4.12 0.0001 0.0001 0.0002
GPU5 4,113,647 4.11 0.0001 0.0001 0.0002
GPU6 4,111,842 4.11 0.0001 0.0001 0.0002
GPU7 4,226,781 4.23 0.0001 0.0001 0.0001

Performance Analysis

Observations

  • Sustained bandwidth reached 4.10–4.23 TB/s.
  • Performance remained consistent across all eight GPUs.
  • Achieved approximately 85–86% of the H200's theoretical peak memory bandwidth.
  • Kernel execution time remained close to 0.1 ms, indicating excellent memory subsystem utilization.

Memory Bandwidth Visualization

GPU0  ████████████████████████ 4.23 TB/s
GPU1  ███████████████████████  4.21 TB/s
GPU2  ██████████████████████   4.18 TB/s
GPU3  █████████████████████    4.10 TB/s
GPU4  █████████████████████    4.12 TB/s
GPU5  █████████████████████    4.11 TB/s
GPU6  █████████████████████    4.11 TB/s
GPU7  ████████████████████████ 4.23 TB/s

GPU Topology

Host System
│
├── GPU0 (NVIDIA H200)
├── GPU1 (NVIDIA H200)
├── GPU2 (NVIDIA H200)
├── GPU3 (NVIDIA H200)
├── GPU4 (NVIDIA H200)
├── GPU5 (NVIDIA H200)
├── GPU6 (NVIDIA H200)
└── GPU7 (NVIDIA H200)

Best Practices

✅ Run each GPU independently for accurate measurements.

✅ Use large array sizes to fully saturate HBM3 memory bandwidth.

✅ Keep ECC enabled during production benchmarking.

✅ Execute multiple iterations and average the results.

✅ Ensure CUDA Toolkit and NVIDIA drivers are compatible.

✅ Avoid concurrent GPU workloads during benchmarking.

Terminal Output

root@scn90-mn:/scratch/hpc-benchmarks/cuda12#

/scratch/hpc-benchmarks/cuda12/stream-gpu-linux-x86_64/stream_test_fp32 \
-d 0 \
-n 50000000 \
-t T

Complete terminal logs are available in the accompanying text file.

Conclusion

The STREAM Triad benchmark demonstrates that the NVIDIA H200 GPU delivers exceptional sustained memory bandwidth for memory-intensive workloads.

Key Findings

  • Sustained bandwidth of 4.1–4.23 TB/s per GPU.
  • Consistent performance across all 8 NVIDIA H200 GPUs.
  • Near-theoretical HBM3 memory utilization.
  • Excellent suitability for:
    • High Performance Computing (HPC)
    • AI Training
    • AI Inference
    • Scientific Simulations
    • Large-scale Data Analytics

The benchmark confirms that the NVIDIA H200 is highly optimized for memory-bound applications, making it an ideal accelerator for modern HPC and AI infrastructures.