LLM inference on consumer hardware: multi-GPU AMD Radeon (ROCm, RCCL, vLLM) and the Apple Neural Engine (Core ML, MLX).
-
Technical University of Denmark
- Copenhagen
-
21:15
(UTC +02:00)
Highlights
- Pro
Pinned Loading
-
dual-radeon-vllm
dual-radeon-vllm PublicMulti-GPU tensor-parallel vLLM on AMD Radeon RX 7900 XT / XTX / GRE (7900XT, 7900XTX, RX7900XT, gfx1100, RDNA3, ROCm): root cause and fix for the RCCL hostcall / PCIe atomics (AtomicOps) crash "NCC…
Python 5
-
llms-on-apple-neural-engine
llms-on-apple-neural-engine PublicApple Neural Engine (ANE/NPU) vs GPU for local LLM inference on Apple Silicon / macOS. Core ML (CoreML), Core AI (CoreAI), MLX and Metal benchmarks: prefill, throughput, latency, memory, thermals, …
Python 2
Something went wrong, please refresh the page to try again.
If the problem persists, check the GitHub status page or contact support.
If the problem persists, check the GitHub status page or contact support.


