update(rocm): port the fused MoE decode kernels to HIP

README.md

@@ -16,7 +16,7 @@ Apple Silicon is the primary target. Linux/CUDA is supported as a secondary targ
 
 The current `main` branch is v0.7.0 plus unreleased work. Install from source to use features that have not reached the latest tagged release; see the [changelog](CHANGELOG.md) for the release-by-release record.
 
-- **Experimental AMD GPU (ROCm) support.** `--features rocm` builds mlxcel on Linux against an MLX ROCm backend vendored into the source tree as the ROCm part of mlxcelverse, applied on the same pinned MLX commit as the Metal and CUDA builds. Affine, mxfp8, and mxfp4 checkpoints, including gpt-oss MoE experts, run on RDNA 3.5 (`gfx1151`); fused kernels fall back to MLX graphs and affine MoE models need `MLXCEL_FUSED_MOE=0` for now. See [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm-experimental) for the build and the open gaps.
+- **Experimental AMD GPU (ROCm) support.** `--features rocm` builds mlxcel on Linux against an MLX ROCm backend vendored into the source tree as the ROCm part of mlxcelverse, applied on the same pinned MLX commit as the Metal and CUDA builds. Affine, mxfp8, and mxfp4 checkpoints, including gpt-oss MoE experts, run on RDNA 3.5 (`gfx1151`); the fused kernels with a HIP port (BitLinear, the SSM update step, the decode-MoE pair) run natively and the rest fall back to MLX graphs. See [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm-experimental) for the build and the open gaps.
 - **Verified `llama-server` compatibility.** A frozen b10621 manifest classifies all 376 pinned options, routes, and native request fields, with no deferred entries. Native completion, embedding, tokenization, template, infill, props, slots, metrics, resumable-stream, router, and LoRA surfaces are implemented or explicitly classified instead of being silently ignored.
 - **OpenAI, Anthropic, and Vertex-compatible serving.** Chat Completions, Completions, Responses, Embeddings, Reranking, Audio, and Anthropic Messages are available alongside the native `llama-server` routes. Optional Vertex AI custom-container routing is supported through the standard `AIP_*` variables.
 - **Multi-model routing and live adapters.** Router mode discovers checkpoints from the model store, a model directory, or INI presets; loads them on demand; and bounds the resident set with LRU eviction. Multiple LoRA adapters can remain unfused for per-request or live scale changes, or be fused for zero decode overhead.

benchmarks/logit_traces/rocm_gfx1151_77afbb1d/METADATA.txt

@@ -0,0 +1,17 @@
+host: AMD Ryzen AI MAX+ 395 with Radeon 8060S (gfx1151, RDNA 3.5), 96 GiB VRAM carve-out, 31 GiB host RAM
+os: Debian GNU/Linux 13 (trixie), kernel 6.18.12+deb13-amd64
+rocm: 10.0.0 packages (HIP 7.15.26333), ROCM_PATH=/opt/rocm
+mlxcel_commit: 77afbb1d (branch update/issue-2065-rocm-fused-moe-ports, lablup/mlxcel#2098, on origin/main 57d8ed29). The HIP ports of moe_gateup and moe_down, the table predicates, and the ROCm SGY default of 2.
+mlx_pin: 81ba1c6a0e50a9268b931579c2d4f1158b9aab5a
+rocm_overlay_fork: NripeshN/mlx rocm-support @ 75915908dfe5028335d318b10340313744fd3a8d (src/lib/mlx-cpp/patches-rocm/UPSTREAM), plus the local fixes in LOCAL_FIXES.md
+logit_trace_sha256: ba41ee7a57f1c4e0999112d071062239241a571eca59aa8aa20751e480fd8a14
+baseline_logit_trace_sha256: 31bd9eb0ab06c232fc0e374a4675d6c92f380f145ffc1c495dce043f05b240f8 (origin/main 57d8ed29, used for the main-vs-branch rows in the results page; those traces are not committed here)
+build: cargo build --release --features rocm --example logit_trace
+corpus: tests/fixtures/wikitext2_excerpt.txt sha256 16721fc1057d69efc8493718fcad562de9e4554ce05369876cd06dcba431394e
+checkpoints: the same local directories as ../rocm_gfx1151_bec64748/ (Qwen3-30B-A3B-4bit@d388dead, Mixtral-8x7B-Instruct-v0.1-4bit@4e8f7045) and ../rocm_gfx1151_96cbce84/ (NVIDIA-Nemotron-3-Nano-30B-A3B-4bit@832f602eba5d22436c258c1462bdedc5afddb42b)
+args: w1='1 128 8 0'; w8='8 80 8 512'; w1ctx512='1 128 8 512' with MLXCEL_TRACE_START_TOKEN=512 (CHUNK_TOKENS MAX_CHUNKS TOPK PREFILL)
+tags: 'default' = no override. 'fused0' = MLXCEL_FUSED_MOE=0 (gather_qmm). The fused MoE kernels run only on a single-token forward, so on Qwen3-30B-A3B only the w1 'default' trace takes the HIP kernels; its w8 'default' and 'fused0' traces are identical. Mixtral-8x7B's experts (Dff 14336) are above the 8192 MLXCEL_FUSED_MOE_MAX_DFF default, so its 'default' runs gather_qmm. Nemotron-H's 'default' runs fused_moe_forward (gather_qmm routed experts) where main ran forward_nonfused.
+filtering: the Nemotron-H files are stored with the five '[NemotronH] ' loader lines removed (grep -v '^\['), so they compare directly; the older Nemotron-H sets keep those lines.
+env: LD_LIBRARY_PATH unset; MLXCEL_KEEP_BF16 unset; MLXCEL_FUSED_MOE unset except on 'fused0'; MLXCEL_FUSED_MOE_SGY unset (ROCm default 2)
+gpu: correctness traces, not timings; another development unit may have used the GPU during a run
+references: ../metal_m1u_bec64748/ (Qwen3, Mixtral), ../rocm_gfx1151_bec64748/ (ROCm gather_qmm 'fused0'), ../rocm_gfx1151_c5fe9a16/ (Nemotron-H w1, w8 on forward_nonfused), ../rocm_gfx1151_96cbce84/ (Nemotron-H w1ctx512 on main), ../metal_m5_d1128266/ (Nemotron-H Metal)

benchmarks/logit_traces/rocm_gfx1151_77afbb1d/README.md

@@ -0,0 +1,20 @@
+# ROCm logit traces with the HIP fused MoE decode kernels (lablup/mlxcel#2065)
+
+Teacher-forced traces from `examples/logit_trace` on the Radeon 8060S (`gfx1151`) after `moe_gateup` and `moe_down` gained HIP ports. The fused MoE path runs only on a single-token forward, so the `w1` rows are the ones that reach the kernels on Qwen3-30B-A3B; the `w8` rows check that the multi-token path is unchanged. `METADATA.txt` records the host, versions, commit and binary hash; `RUNS.txt` the exit status and row count of every run; `SHA256SUMS` covers every trace. The comparisons are in `docs/benchmark_results/rocm-fused-moe-gfx1151-2026-10-05.md`.
+
+```bash
+cargo build --release --features rocm --example logit_trace
+T=./target/release/examples/logit_trace
+C=tests/fixtures/wikitext2_excerpt.txt
+P=rocm_gfx1151_77afbb1d
+for w in "w1:1 128 8 0" "w8:8 80 8 512"; do
+  tag=${w%%:*}; args=${w#*:}
+  $T models/mlx/Qwen3-30B-A3B-4bit $C $args > ${P}_qwen3-30b-a3b_default_$tag.tsv
+  MLXCEL_FUSED_MOE=0 $T models/mlx/Qwen3-30B-A3B-4bit $C $args > ${P}_qwen3-30b-a3b_fused0_$tag.tsv
+  $T models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit $C $args | grep -v '^\[' > ${P}_nemotron-3-nano-30b-a3b_default_$tag.tsv
+done
+MLXCEL_TRACE_START_TOKEN=512 $T models/mlx/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit $C 1 128 8 512 | grep -v '^\[' > ${P}_nemotron-3-nano-30b-a3b_default_w1ctx512.tsv
+$T models/mlx/Mixtral-8x7B-Instruct-v0.1-4bit $C 1 128 8 0 > ${P}_mixtral-8x7b-instruct_default_w1.tsv
+```
+
+Unlike the older Nemotron-H sets, the files here are stored without the five `[NemotronH]` loader lines. Filter the older references with `grep -v '^\[NemotronH\] '` before `scripts/compare_logit_traces.py`.

benchmarks/logit_traces/rocm_gfx1151_77afbb1d/RUNS.txt

@@ -0,0 +1,8 @@
+rocm_gfx1151_77afbb1d_qwen3-30b-a3b_default_w8                     exit=0 rows=647 elapsed=92s
+rocm_gfx1151_77afbb1d_qwen3-30b-a3b_fused0_w8                      exit=0 rows=647 elapsed=89s
+rocm_gfx1151_77afbb1d_qwen3-30b-a3b_default_w1                     exit=0 rows=135 elapsed=13s
+rocm_gfx1151_77afbb1d_qwen3-30b-a3b_fused0_w1                      exit=0 rows=135 elapsed=7s
+rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w8           exit=0 rows=647 elapsed=100s (loader lines filtered)
+rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w1           exit=0 rows=135 elapsed=13s (loader lines filtered)
+rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w1ctx512     exit=0 rows=135 elapsed=not timed (loader lines filtered)
+rocm_gfx1151_77afbb1d_mixtral-8x7b-instruct_default_w1             exit=0 rows=135 elapsed=35s

benchmarks/logit_traces/rocm_gfx1151_77afbb1d/SHA256SUMS

@@ -0,0 +1,8 @@
+48e6537dc7be46fcef35f86fe24ece6778c63d2affcae2d95f098cc47edfccad  rocm_gfx1151_77afbb1d_mixtral-8x7b-instruct_default_w1.tsv
+d70dff2d7a2ef4b67310c7a029282b415ba4465e2260907bfa307f792252ffb8  rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w1ctx512.tsv
+281d6189a3d4ca3f49ee17ac8a51928a0b942edecfcd831d5f331f94d2aa268f  rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w1.tsv
+c7373682196b8425837cb0edb9b2a86c39464bd073bda5fb5c38fd2acc97fb15  rocm_gfx1151_77afbb1d_nemotron-3-nano-30b-a3b_default_w8.tsv
+84f585edd756753803a1e266d7bad8ac4d4573febbc39a9671b2788f89cbd1f4  rocm_gfx1151_77afbb1d_qwen3-30b-a3b_default_w1.tsv
+0c159e49996cc60fa9caf7aa38ff50020bb464b96596569a8926ba336996e7e2  rocm_gfx1151_77afbb1d_qwen3-30b-a3b_default_w8.tsv
+cc0114a1af1b71ca47c7583699b77d645b64de8d6e4c9f0ffe27e4045ed53a44  rocm_gfx1151_77afbb1d_qwen3-30b-a3b_fused0_w1.tsv
+0c159e49996cc60fa9caf7aa38ff50020bb464b96596569a8926ba336996e7e2  rocm_gfx1151_77afbb1d_qwen3-30b-a3b_fused0_w8.tsv