Skip to content

Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD - MI300X)#1

Open
nilapate wants to merge 1 commit into
amd-integrationfrom
nilapate/rocm-inference-overlays
Open

Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD - MI300X)#1
nilapate wants to merge 1 commit into
amd-integrationfrom
nilapate/rocm-inference-overlays

Conversation

@nilapate

@nilapate nilapate commented Jul 3, 2026

Copy link
Copy Markdown

Enable fast ROCm inference via channels_last_3d + bf16 autocast + whole-model torch.compile. Applied as a config overlay on top of the base inference.json;

Usage:
python -m monai.bundle run
--config_file "['configs/inference.json', 'configs/inference_rocm.json']"

Description

Adds inference_rocm.json config overlays for Vista3D and SwinUNETR bundles to enable optimized inference on AMD MI300X GPUs. The overlays apply three optimizations: channels_last_3d memory layout, bf16 autocast, and whole-model torch.compile. These configs extend the existing inference.json without modifying bundle scripts — users opt in by stacking configs at runtime.

Status

Ready

Please ensure all the checkboxes:

  • Codeformat tests passed locally by running ./runtests.sh --codeformat.
  • In-line docstrings updated.
  • Update version and changelog in metadata.json if changing an existing bundle.
  • Please ensure the naming rules in config files meet our requirements (please refer to: CONTRIBUTING.md).
  • Ensure versions of packages such as monai, pytorch and numpy are correct in metadata.json.
  • Descriptions should be consistent with the content, such as eval_metrics of the provided weights and TorchScript modules.
  • Files larger than 25MB are excluded and replaced by providing download links in large_file.yml.
  • Avoid using path that contains personal information within config files (such as use /home/your_name/ for "bundle_root").

Enable fast ROCm/MI300X inference via channels_last_3d + bf16 autocast +
whole-model torch.compile. Applied as a config overlay on top of the base
inference.json; no changes to bundle scripts.

Usage:
  python -m monai.bundle run \
    --config_file "['configs/inference.json', 'configs/inference_rocm.json']" ...
@nilapate nilapate changed the title Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD MI300X) Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD) Jul 3, 2026
@nilapate nilapate changed the title Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD) Add inference_rocm.json overlays for Vista3D and SwinUNETR (AMD - MI300X) Jul 3, 2026

@soumitra-chatterjee-tech soumitra-chatterjee-tech left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, these are not specific to MI300X, right? These would apply equally for MI355X as well? If so, you should update the PR description to make it generic instead of narrowing it to MI300X (you can still clarify that this is only validated on MI300X). If not, we would need to rename the inference_rocm.json to something more MI300X specific, say inference_rocm_mi300x.json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants