Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
213 changes: 213 additions & 0 deletions TECHNICAL_REPORTS/2099-rocm-ssm-update-hip-port-20261004.en.md

Large diffs are not rendered by default.

213 changes: 213 additions & 0 deletions TECHNICAL_REPORTS/2099-rocm-ssm-update-hip-port-20261004.ko.md

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions benchmarks/logit_traces/rocm_gfx1151_96cbce84/METADATA.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
host: AMD Ryzen AI MAX+ 395 with Radeon 8060S (gfx1151, RDNA 3.5), 96 GiB VRAM carve-out, 31 GiB host RAM
os: Debian GNU/Linux 13 (trixie), kernel 6.18.12+deb13-amd64
rocm: 10.0.0 packages (HIP 7.15.26333), ROCM_PATH=/opt/rocm
mlxcel_commit: 96cbce84 (branch update/issue-2067-rocm-ssm-update-port, on origin/main c0b71344), the first commit of the #2067 PR. The PR's next commit names the A_log, B and C dtypes in the SSM launch's template arguments on every backend instead of on ROCm only; on ROCm the launch, its arguments and its cache key are the same in both commits, so these traces stand for the PR head as well.
mlx_pin: 81ba1c6a0e50a9268b931579c2d4f1158b9aab5a
rocm_overlay_fork: NripeshN/mlx rocm-support @ 75915908dfe5028335d318b10340313744fd3a8d (src/lib/mlx-cpp/patches-rocm/UPSTREAM), plus the local fixes in LOCAL_FIXES.md
logit_trace_sha256: 4c1ecae4dd823f08c6e9aa997cbfcfe535a3aef3a9c85dca63669327a6ee078e
build: cargo build --release --features rocm --example logit_trace
corpus: tests/fixtures/wikitext2_excerpt.txt sha256 16721fc1057d69efc8493718fcad562de9e4554ce05369876cd06dcba431394e
checkpoints: the same local directories as ../rocm_gfx1151_3c9edea0/ (granite-4.0-h-tiny-4bit@02c8783da0cb171942e30a7dde8acce11fe82e0d, NVIDIA-Nemotron-3-Nano-30B-A3B-4bit@832f602eba5d22436c258c1462bdedc5afddb42b)
args: w1ctx512='1 128 8 512' with MLXCEL_TRACE_START_TOKEN=512; w1='1 128 8 0' (CHUNK_TOKENS MAX_CHUNKS TOPK PREFILL)
tags: 'default' = no override, so each w1ctx512 single-token forward meets the prefilled SSM state and takes the HIP ssm_update_kernel (granite through ssm_step_kernel, Nemotron-H through fused_mamba2_forward). 'ssmkernel0' = MLXCEL_SSM_KERNEL=0, the same binary forced onto the SSD graph (ssm_step). w1 chunks have no prefill, so they never meet a state and run the graph on both tags.
env: LD_LIBRARY_PATH unset; MLXCEL_KEEP_BF16 unset; MLXCEL_FUSED_MOE unset; MLXCEL_DEVICE unset (trace header reads '# device GPU')
gpu: correctness traces, not timings; a parallel development unit may have used the GPU during a run
metal_reference: ../metal_m5_3c9edea0/ (w1ctx512); graph references: ../rocm_gfx1151_3c9edea0/ (w1ctx512), ../rocm_gfx1151_c5fe9a16/ (w1)
18 changes: 18 additions & 0 deletions benchmarks/logit_traces/rocm_gfx1151_96cbce84/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# ROCm logit traces with the HIP SSM update kernel (lablup/mlxcel#2067)

Teacher-forced traces from `examples/logit_trace` on the Radeon 8060S (`gfx1151`) after `ssm_update_kernel` gained a HIP port. Before it, ROCm ran the Mamba2 SSD graph for every single-token step, so the `w1ctx512` rows in `../rocm_gfx1151_3c9edea0/` compared Metal's fused kernel with ROCm's graph. These compare kernel with kernel, and the `ssmkernel0` rows give the graph on the same binary for a same-backend A/B. `METADATA.txt` records the host, versions, commit and binary hash; `RUNS.txt` the exit status and row count of every run; `SHA256SUMS` covers every trace. The comparison is in `docs/benchmark_results/rocm-ssm-update-kernel-gfx1151-2026-10-04.md`.

```bash
cargo build --release --features rocm --example logit_trace
T=./target/release/examples/logit_trace
C=tests/fixtures/wikitext2_excerpt.txt
P=rocm_gfx1151_96cbce84
for pair in granite-4.0-h-tiny:granite-4.0-h-tiny-4bit nemotron-3-nano-30b-a3b:NVIDIA-Nemotron-3-Nano-30B-A3B-4bit; do
tag=${pair%%:*}; dir=${pair##*:}
MLXCEL_TRACE_START_TOKEN=512 $T models/mlx/$dir $C 1 128 8 512 > ${P}_${tag}_default_w1ctx512.tsv
MLXCEL_SSM_KERNEL=0 MLXCEL_TRACE_START_TOKEN=512 $T models/mlx/$dir $C 1 128 8 512 > ${P}_${tag}_ssmkernel0_w1ctx512.tsv
$T models/mlx/$dir $C 1 128 8 0 > ${P}_${tag}_default_w1.tsv
done
```

The Nemotron-H traces carry five `[NemotronH]` loader lines on stdout; filter them with `grep -v '^\[NemotronH\] '` on both sides before `scripts/compare_logit_traces.py`.
6 changes: 6 additions & 0 deletions benchmarks/logit_traces/rocm_gfx1151_96cbce84/RUNS.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
rocm_gfx1151_96cbce84_granite-4.0-h-tiny_default_w1ctx512 exit=0 rows=135 elapsed=120s
rocm_gfx1151_96cbce84_granite-4.0-h-tiny_ssmkernel0_w1ctx512 exit=0 rows=135 elapsed=115s
rocm_gfx1151_96cbce84_granite-4.0-h-tiny_default_w1 exit=0 rows=135 elapsed=4s
rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_default_w1ctx512 exit=0 rows=140 elapsed=235s
rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_ssmkernel0_w1ctx512 exit=0 rows=140 elapsed=223s
rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_default_w1 exit=0 rows=140 elapsed=6s
6 changes: 6 additions & 0 deletions benchmarks/logit_traces/rocm_gfx1151_96cbce84/SHA256SUMS
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dea067abe86dde12683271aff5683e1ca231a23a99c3e6d5cd1c5411e0c60063 rocm_gfx1151_96cbce84_granite-4.0-h-tiny_default_w1ctx512.tsv
a6caad4d60d0bca640d0b39065ade10f1e412b36f8204fae56510ad9e3c9dbbe rocm_gfx1151_96cbce84_granite-4.0-h-tiny_default_w1.tsv
70b1b5e7ff6ec79cd61e2437bd3fb26b377f88570554c963d1010a1c5dada116 rocm_gfx1151_96cbce84_granite-4.0-h-tiny_ssmkernel0_w1ctx512.tsv
7cdbb6ad3ac999fc379771060b296e17fb062e6c64b9976007df1a84e37e1a7c rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_default_w1ctx512.tsv
9326e8bde7aaad14c7e57aabc48f4eadfecd5a9b746f82da67dc9b26a2ccdf98 rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_default_w1.tsv
40bb05103bc5c5dffba821328756f346432ff05a73775e4526444f40b062322b rocm_gfx1151_96cbce84_nemotron-3-nano-30b-a3b_ssmkernel0_w1ctx512.tsv
Loading
Loading