Skip to content

feat: GPU enumeration with VRAM reporting (NVIDIA, AMD, Windows)#211

Open
JamesNyeVRGuy wants to merge 1 commit intonikopueringer:mainfrom
JamesNyeVRGuy:feat/gpu-enumeration
Open

feat: GPU enumeration with VRAM reporting (NVIDIA, AMD, Windows)#211
JamesNyeVRGuy wants to merge 1 commit intonikopueringer:mainfrom
JamesNyeVRGuy:feat/gpu-enumeration

Conversation

@JamesNyeVRGuy
Copy link
Copy Markdown
Contributor

GPU enumeration with VRAM reporting (NVIDIA, AMD, Windows)

Adds enumerate_gpus() to device_utils.py — a cross-platform function that detects all available GPUs and reports their VRAM. Useful for status display, VRAM gating before inference, and multi-GPU selection in batch scripts.

Currently device_utils.py can detect which device to use (CUDA/MPS/CPU) but can't tell you what GPUs are available or how much VRAM they have.

What does this change?

Adds to device_utils.py:

  • GPUInfo dataclass: index, name, vram_total_gb, vram_free_gb
  • enumerate_gpus() -> list[GPUInfo] with multi-backend detection:
    • NVIDIA: nvidia-smi CSV query (index, name, total/free memory)
    • AMD Linux: amd-smi (ROCm 6.0+) with live VRAM usage, falls back to rocm-smi (legacy)
    • AMD Windows: Registry lookup via winreg (fixes the Win32_VideoController.AdapterRAM uint32 overflow that reports >4GB GPUs as 0GB)
    • Universal fallback: torch.cuda.get_device_properties() for any GPU torch can see

No new dependencies — uses subprocess, json, winreg (stdlib). torch is only imported in the fallback path.

How was it tested?

  • NVIDIA (Linux Docker + Windows native): nvidia-smi path returns correct GPU name and VRAM
  • AMD (Linux ROCm): rocm-smi path returns correct VRAM for RX 7900 XTX
  • AMD (Windows): registry path correctly reads 64-bit VRAM size for 16GB+ cards
  • No GPU: returns empty list, no errors

Checklist

  • uv run pytest passes
  • uv run ruff check passes
  • uv run ruff format --check passes

Add enumerate_gpus() to device_utils.py for detecting GPUs and
their VRAM across all platforms:

- NVIDIA: nvidia-smi query (index, name, total/free memory)
- AMD Linux: amd-smi (ROCm 6.0+) → rocm-smi (legacy) fallback
- AMD Windows: registry lookup (fixes uint32 VRAM overflow in WMI)
- Universal: torch.cuda fallback for any GPU torch can see

Returns list[GPUInfo] with index, name, vram_total_gb, vram_free_gb.
Useful for CLI status display, VRAM gating, and multi-GPU selection.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant