Skip to content

feat(rocm): add end-to-end Vime rollout/training attention ablation matrix - #385

Merged
inaniloquentee merged 3 commits into
testfrom
bench/rocm-pr230-attention-microprobes
Sep 4, 2026
Merged

feat(rocm): add end-to-end Vime rollout/training attention ablation matrix#385
inaniloquentee merged 3 commits into
testfrom
bench/rocm-pr230-attention-microprobes

Conversation

@Flink-ddd

@Flink-ddd Flink-ddd commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add the ROCm end-to-end counterpart of the production/RL-Kernel Attention matrix introduced in feat(alignment): add cross-configuration framework #230.
  • Execute the P/P, P/R, R/P, and R/R matrix through real Vime orchestration, covering both vLLM rollout and Megatron training.
  • Keep FFN and Logp fixed at P/P, so Attention is the only implementation dimension changed between cells.
  • Require runtime readback evidence from both frameworks and fail closed when execution falls back, selects the wrong route, or cannot prove the strict ROCm backend.
  • Remove the previous operator-only micro-probe runner and all checked-in generated result artifacts, including results.json.

Ablation matrix

P selects the framework-native implementation. R selects the RL-Kernel ROCm implementation.

Case Megatron training vLLM rollout
P/P Framework-native Framework-native
P/R Framework-native RL-Kernel AITER/CK
R/P RL-Kernel AITER/CK Framework-native
R/R RL-Kernel AITER/CK RL-Kernel AITER/CK

Every cell starts in a fresh process and uses the same model, checkpoint, prompt data, seeds, token limits, rollout count, and pre-update state.

Implementation

  • Add rl_engine.integrations.rocm_ablation as the end-to-end matrix runner and runtime evidence validator.
  • Add a runnable Qwen3-8B Vime example under examples/vime_qwen3_8b_rocm_ablation.
  • Extend the Attention binding layer with platform-aware strict ROCm runtime selection.
  • Add the Megatron ROCm training route using the strict AITER/CK Attention core and RCCL AG/RS transport.
  • Add the vLLM ROCm rollout route with paged-KV reconstruction and strict AITER/CK execution.
  • Extend runtime and registry metadata so ROCm platform, backend, core, and schedule identities can be validated from executed readbacks.
  • Add CPU-safe end-to-end matrix and framework-adapter contract coverage to the default CI workflow.

Runtime acceptance boundary

A subprocess exit code of zero is not sufficient for a passing cell.

Each cell must emit executed Attention readbacks from both Megatron training and vLLM rollout. For every R side, the runner requires evidence for:

  • runtime_platform=rocm
  • Semantic backend rlkernel.attention.deterministic.v1
  • Actual runtime rlkernel.rocm.attention.aiter_ck_ag_rs.v1
  • The strict AITER/CK core
  • The fixed no-Split-KV ROCm schedule
  • No native, PyTorch-reference, Triton, CUDA, or fallback execution

The runner also rejects missing, malformed, stale, duplicated, or case-inconsistent readbacks.

Replay invariants

The matrix freezes the command and replay-defining environment before the first cell. Only the Attention case selection and case-local output paths may change.

An executable run requires identifiable model, distributed checkpoint, and prompt inputs. It also enforces NUM_ROLLOUT=1, ensuring every cell begins from the same pre-update checkpoint state.

Running the matrix

Review the resolved launch contract without allocating a GPU run:

python examples/vime_qwen3_8b_rocm_ablation/run.py \
  --output-dir /tmp/rocm-attention-ablation \
  -- bash /path/to/vime/scripts/run-qwen3-8B-rocm.sh

Execute all four cells:

python examples/vime_qwen3_8b_rocm_ablation/run.py \
  --run \
  --output-dir /tmp/rocm-attention-ablation \
  -- bash /path/to/vime/scripts/run-qwen3-8B-rocm.sh

Individual cells can be selected with repeatable --case arguments while debugging. The final hardware acceptance run should execute the complete four-cell matrix.

Results and artifacts

Runtime logs and unmodified framework readbacks are written to the requested external output directory. The aggregate report is a human-readable summary.md.

No generated benchmark or result JSON is checked into the repository.

Validation
Added end-to-end matrix, replay-freezing, readback-validation, and failure-boundary tests.
Extended Attention binding and framework-runtime adapter coverage for ROCm.
Added the ROCm end-to-end tests to the default CI workflow.
Source formatting, Python compilation/AST validation, shell syntax validation, and the isolated matrix/readback dry-run contract passed locally.
A full physical MI300X Vime matrix remains the hardware acceptance step before moving the PR out of Draft.

Reference: #230

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: a483ac14-e29c-465a-a1f2-369714b7eb7e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: vensen <vensenmu@gmail.com>
Signed-off-by: vensen <vensenmu@gmail.com>
@Flink-ddd
Flink-ddd force-pushed the bench/rocm-pr230-attention-microprobes branch from 4422132 to d0472ea Compare September 3, 2026 10:46
@Flink-ddd Flink-ddd added the platform: rocm Specific tasks specific to AMD graphics cards (such as CK, bpreshuffle/FA) label Sep 3, 2026
Signed-off-by: Codex <codex@openai.com>
@Flink-ddd
Flink-ddd marked this pull request as draft September 3, 2026 13:01
@Flink-ddd Flink-ddd changed the title bench(rocm): add PR230 Attention taxonomy micro-probes feat(rocm): add end-to-end Vime rollout/training attention ablation matrix Sep 3, 2026
@Flink-ddd
Flink-ddd marked this pull request as ready for review September 3, 2026 14:58
@inaniloquentee
inaniloquentee merged commit 553de61 into test Sep 4, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

platform: rocm Specific tasks specific to AMD graphics cards (such as CK, bpreshuffle/FA)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants