[release/v0.1.12] Cherry-pick #2645 (multi-arch CK GEMM dispatch) + SynchronizedCache backport for v0.1.12.post2

[sunway513]

Summary

This PR brings the validated content of pensun/post2-with-2645 onto release/v0.1.12 so the branch reflects exactly what we shipped as the v0.1.12.post2.dev2645+torch210 wheel that just passed cross-arch DSR1 E2E validation. Ready for tagging as v0.1.12.post2 after review.

Builds on top of #2846 (the C10_HIP_KERNEL_LAUNCH_CHECK macro removal that fixed Greg's build-from-source issue). The macro removal alone was insufficient for vllm/vllm-openai-rocm:v0.19.1's ABI — the dispatch fix from #2645 + the manylinux + torch 2.10 pin combine for the full fix.

What's in this PR

Two commits on top of release/v0.1.12:

1. 7f3e5249 — Cherry-pick of fix(ck_gemm): fix multi-arch build targeting and kernel dispatch across all CK GEMM modules #2645 (Vinay's multi-arch CK GEMM dispatch fix, merged on main as 727253ae)

   • Fixes multi-arch dispatch key collision in CK GEMM modules — the C++ dispatch map is now keyed by (gfx, cu_num, M, N, K) instead of (cu_num, M, N, K), eliminating collisions when the same wheel runs across gfx942 / gfx950.
   • Adds gfx column to all tuned-GEMM CSVs and tuner output keys.
   • Adds chip_info.get_build_targets() / get_gfx_runtime() helpers and PRETUNE_MODULES build flag.
   • Addresses TJ Mok's 0.1.12 fails on DeepSeek R1 on MI300 #2864 DSR1 + MI300 GEMM crash report.

2. b8eab93b — SynchronizedCache template hand-port (subset of Replace unsafe uses of std::unordered_map with SynchronizedCache #2221)

   • PR fix(ck_gemm): fix multi-arch build targeting and kernel dispatch across all CK GEMM modules #2645's csrc/include/gemm_dispatch_utils.h references a SynchronizedCache<Key, T> template that lives in csrc/include/aiter_hip_common.h. That template was added by separate PR Replace unsafe uses of std::unordered_map with SynchronizedCache #2221 ("Replace unsafe uses of std::unordered_map", commit 016ead37) on main.
   • Replace unsafe uses of std::unordered_map with SynchronizedCache #2221 is not on release/v0.1.12. Cherry-picking fix(ck_gemm): fix multi-arch build targeting and kernel dispatch across all CK GEMM modules #2645 alone fails to compile with:

     csrc/include/gemm_dispatch_utils.h:46:12: error: no template named 'SynchronizedCache'
     

   • This commit hand-ports just the 23-line template definition + 2 includes — the minimum needed for fix(ck_gemm): fix multi-arch build targeting and kernel dispatch across all CK GEMM modules #2645 to compile. It deliberately skips Replace unsafe uses of std::unordered_map with SynchronizedCache #2221's other changes (replacing std::unordered_map usages in 12 .cu files), since release/v0.1.12 doesn't have those dispatch sites and pulling them in would expand the diff unnecessarily.

Validation

The wheel built from this exact branch state — amd_aiter-0.1.12.post2.dev2645+rocm7.2.manylinux.2.28.torch210... — has been validated end-to-end:

• Test build run #24800645132 — auditwheel PASS (GLIBCXX ≤ 3.4.29, GLIBC ≤ 2.34), manylinux_2_28 compliant.
• Cross-arch DSR1 E2E: same wheel content serves DeepSeek R1 cleanly on both:
  • MI300X (gfx942) with vllm/vllm-openai-rocm:v0.19.1 + VLLM_ROCM_USE_AITER=1 — coherent inference, no GEMM crash.
  • MI355X (gfx950) with vllm/vllm-openai-rocm:v0.19.1 + VLLM_ROCM_USE_AITER=1 — coherent inference.
• The multi-arch dispatch fix is what unblocks running the same wheel on both targets without the dispatch key collision that 0.1.12 fails on DeepSeek R1 on MI300 #2864 reported.

Issue references

• Resolves Building 0.1.12.post1 with PREBUILD_KERNELS fails #2837 (fully — combination of [release/v0.1.12] Remove C10_HIP_KERNEL_LAUNCH_CHECK (Lingpeng's preferred fix) #2846 + this PR + the manylinux build + torch 2.10 pin)
• Tracks 0.1.12 fails on DeepSeek R1 on MI300 #2864 (DSR1 + MI300 crash) — wheel built from this branch eliminates the crash
• Builds on Trial wheel: v0.1.12.post2 candidate (ROCm 7.2.2 + py3.12) available for smoke test #2843 (release tracking)
• Builds on [release/v0.1.12] Remove C10_HIP_KERNEL_LAUNCH_CHECK (Lingpeng's preferred fix) #2846 (already merged — C10_HIP_KERNEL_LAUNCH_CHECK macro removal)

CI

Applying full label-driven downstream test set:

• ci:vllm
• ci:atom
• ci:sglang
• ci:triton-355

Do NOT merge yet

Leave open for review. Tag v0.1.12.post2 will happen post-merge.

[github-actions]

⚠️ [ruff] <F841> _{reported by reviewdog 🐶}
Local variable prev_task_count is assigned to but never used

Suggested change

	prev_task_count = len(task)
	cu_num, M, N, K, q_dtype_w)

[ChuanLi1101]

FYI — ran auditwheel show + import smoke test on the wheel from run 24800645132 on a ROCm dev box:

Wheel: amd_aiter-0.1.12.post2.dev2645+rocm7.2.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
sha256: 2a70ff7b806049a6a0f8cf2e1be48a9c782ea8a2afd45c1eef5d29849ad5ce7b

auditwheel show verdict:

• Platform: manylinux_2_27_x86_64 (matches the manylinux_2_27 / manylinux_2_28 dual tag in the filename)
• Max required symbols: GLIBCXX_3.4.21 (libstdc++), GLIBC_2.27 (glibc, from libm; libc itself tops at GLIBC_2.17)

Import test inside vllm/vllm-openai-rocm:v0.19.1 (Ubuntu 22.04, ceilings GLIBCXX_3.4.30 / GLIBC_2.35, GPUs mounted):

• pip install + from aiter import flash_attn_varlen_func both succeed.
• Installs as amd-aiter-0.1.12.post2.dev2645+rocm7.2.manylinux.2.28.

For perspective, the earlier post2 candidate (run 24753462414, +rocm7.2.2.ubuntu22 filename tag) was manylinux_2_34 / GLIBCXX_3.4.29 / GLIBC_2.32; this build (+rocm7.2.manylinux.2.28 tag) tightens to manylinux_2_27 / GLIBCXX_3.4.21 / GLIBC_2.27. Base-image switch to manylinux took effect.

Not gating review — just closing the binary-ABI loop.

[valarLip]

#2881 this one also needed