Repository navigation
[Issue]: RX 7900 XTX over M.2 OCuLink: BACO resume failure leaves ROCm unusable #8232
Description
Problem Description
An RX 7900 XTX connected through a passive M.2-to-OCuLink adapter stopped being usable by ROCm after an amdgpu BACO runtime-resume failure. rocminfo then exited with HSA_STATUS_ERROR_OUT_OF_RESOURCES; a reboot with amdgpu.runpm=0 restored enumeration and real HIP computation.
This is one captured failure episode, not a deterministic reproducer or a proven fix. The logs point to Linux amdgpu/runtime PM. I am following the current ROCm issue-routing notice to this tracker; I would appreciate help identifying the applicable driver/firmware fix or existing driver issue. The drm/amd tracker currently returns an access-denied page from this environment.
Operating System
Ubuntu 24.04, kernel 7.0.0-31-generic (7.0.0-31.31~24.04.1). The driver is the in-tree amdgpu module, not amdgpu-dkms:
/lib/modules/7.0.0-31-generic/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst
CPU
Intel Core i5-12450H, HASEE CV15S laptop, 32 GiB RAM. INSYDE BIOS 1.07.17THZX5, dated 2023-10-04.
GPU
Yeston RX 7900 XTX, gfx1100, 24 GiB VRAM, PCI 1002:744c, subsystem 1ed3:7891, VBIOS 113-APM6264-100.
Connection: M.2 NVMe-to-OCuLink, SFF-8612, PCIe Gen 4 x4. The exact adapter model/revision and cable length have not been established. The host also has Intel integrated graphics and an NVIDIA RTX 4050 Laptop GPU. No display is connected to the RX 7900 XTX.
00:06.0 Intel root port [8086:464d]
02:00.0 AMD upstream bridge [1002:1478]
03:00.0 AMD downstream bridge [1002:1479]
04:00.0 RX 7900 XTX [1002:744c]
04:00.1 AMD audio [1002:ab30]
ROCm Version / Installation Method
ROCm 7.2.1, installed Debian packages. The evidence bundle contains the installed package inventory, including HIP 7.2.53211.70201-81~24.04, HSA runtime 1.18.0.70201-81~24.04, and rocminfo 1.0.0.70201-81~24.04.
Installed linux-firmware package: 20240318.git3b128b60.0ubuntu3.1. This package version alone does not verify the provenance/version of every loaded firmware blob.
ROCm Component
Linux amdgpu runtime PM / device initialization; visible through rocminfo and HIP. No TheRock build failure or regression range has been established.
Observed sequence and diagnostics
- The GPU had been usable before the failure. Runtime PM was at its default,
amdgpu.runpm=-1, and the boot log selected BACO. No amdgpu override was on the kernel command line. - At 2026-09-15 03:30:49 -0700, the kernel logged the failure below. No whole-system suspend/resume event was found in the captured interval. The precise idle duration and triggering operation are unknown.
- Afterwards,
rocminfoexited 8./dev/kfdcould be opened read/write, but opening the AMD render node returnedEINVAL, despite an active-user read/write ACL. - The GPU's
power/runtime_statuswaserror. The AMD bridges reportedD3cold, and their/GPU PCI configuration reads returned allff; the Intel root port remained readable and reportedDLActive-. These post-failure states do not by themselves identify the original cause.
The owner observed continuous external-PSU power and spinning fans during the failed state. That is an observation, not an electrical or signal-integrity test.
Failure log excerpt
2026-09-15T03:30:49-07:00 amdgpu 0000:04:00.0: Failed to exit BACO state!
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: PSP is resuming...
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: psp reg (0x16080) wait timed out, mask: 8000ffff, read: ffffffff exp: 80000000
2026-09-15T03:30:53-07:00 [drm:psp_v13_0_ring_create [amdgpu]] *ERROR* Failed to wait for trust OS ready for ring creation
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: PSP create ring failed!
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: PSP resume failed
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: resume of IP block <psp> failed -62
2026-09-15T03:30:53-07:00 amdgpu 0000:04:00.0: amdgpu_device_ip_resume failed (-62).
rocminfo output in the failed state
ROCk module is loaded
hsa api call failure at: /longer_pathname_so_that_rpms_can_support_packaging_the_debug_info_for_all_os_profiles/src/rocm-systems/projects/rocminfo/rocminfo.cc:1329
Call returned HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.
Recovery attempts and current result
Requesting active runtime power for the root port and first AMD bridge, retraining the root-port link, and one disable/enable cycle of the already inactive link did not restore access. The changed power/control values and Link Control register were restored afterwards.
A normal OS reboot with the same kernel and the added parameter amdgpu.runpm=0 restored the GPU. Because the reboot and parameter change were applied together, this does not establish that the parameter alone repaired the failed state or that it prevents recurrence under all conditions.
| Check | Failed boot | Recovery boot with runpm=0 |
|---|---|---|
| AMD bridges | D3cold, configuration all ff | D0, readable configuration |
| GPU runtime status | error | active |
| External-facing upstream bridge link | inaccessible | 16.0 GT/s, x4 |
| rocminfo | exit 8 | exit 0, gfx1100 enumerated |
| Actual HIP execution | unavailable | 1,048,576 integer results verified |
The bundle includes the HIP check source, its output, and before/after PCI power snapshots. The check allocates device memory, copies inputs, runs a kernel, copies results back and checks every value; it is not just a device-enumeration test. It can be built with:
/opt/rocm/bin/hipcc --offload-arch=gfx1100 -O2 hip-smoke.cpp -o hip-smoke
./hip-smokeSubsequent llama.cpp HIP operator tests and long-context inference also ran successfully. Controlled repeat idle/resume cycles under the original settings, a kernel/firmware comparison, and a power-cost measurement have not yet been done.
Additional Information
The recovered boot also reports:
smu driver if version = 0x0000003d, smu fw if version = 0x00000040, smu fw program = 0, smu fw version = 0x004e8200 (78.130.0)
SMU driver if version not matched
SMU is initialized successfully!
That warning is present in a boot where computation works; it has not been shown to cause this incident.
Linux commit c52feb436539 disables runtime PM for recognized external/removable GPUs. Its applicability to this passive M.2 connection and inclusion in the exact Ubuntu package have not been verified. An existing RX 9070 XT OCuLink investigation uses different GPU/host hardware; a shared root cause is not assumed.
The report text and diagnostic scripts were prepared with AI assistance. Hardware details came from the owner and local system data; the logs and test results are actual observations. No kernel patch has been installed.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
moristeven477-ship-it commentedon Sep 18, 2026
Follow-up: the external PCIe hierarchy became inaccessible again during active HIP inference with
amdgpu.runpm=0already set. This may be a different cause from the original BACO episode; the earlier reboot/workaround has not established prevention of bus loss.Same RX 7900 XTX / passive M.2-to-OCuLink topology, Linux
7.0.0-31-generic, and ROCm 7.2.1 as the initial report. No new kernel or driver patch was installed.On 2026-09-17 at 14:25:57 UTC, a llama.cpp HIP process stopped progressing. It was running the V1 attention patch from ggml-org/llama.cpp#28943, with the Qwen3.8-27B UD-Q4_K_M model, Flash Attention enabled, F16 KV,
-c 98304 -np 2 -no-kvu -b 512 -ub 512, and sequential requests assigned to alternating slots. The first 512-token prompt / 64-token response completed; the next request did not. This identifies the workload at failure, not a deterministic reproducer or an attribution to that patch.The first failure messages and subsequent reset attempt were:
The Intel root port remained readable with
DLActive-. Both AMD bridges and the GPU returned allfffor the first 16 bytes of live PCI configuration reads. Their sysfs runtime status still saidactive; the reported link width of63was an invalid reading, not a negotiated width. One root-port link retrain and one disable/enable cycle did not restore access; Link Control was restored to its original0c42value.The last readable telemetry sample, approximately half a second before the first unreadable sample, reported GPU busy 98%, driver
power1_average291 W, edge 58 C, junction 76 C, and memory 77 C. These are periodic driver readings, not measurements of PSU output or fast electrical transients. They do not establish PSU overload or exclude a connection/power problem. No BACO-exit or system-suspend message was found in the inspected five-minute interval preceding this episode.A normal reboot was requested at 14:39 UTC. The old journal reaches late shutdown at 14:41 UTC. An owner-provided screen photograph shows repeated
MES ring buffer is full, an audio-functionD3cold to D0, device inaccessiblemessage, and a CORB reset timeout. The logs do not establish that this warm reboot completed on its own. A later boot on 2026-09-18 at 12:41 UTC restored readable PCI configuration, the external Gen4 x4 link,rocminfo, and actual HIP execution:Root cause remains undetermined. This update records recurrence despite disabled GPU runtime PM and the failed recovery/shutdown symptoms; it does not claim a driver regression, a defective PSU/cable, or a permanent fix. Guidance on the appropriate amdgpu/PCIe tracker or additional discriminating diagnostics would be useful.
This follow-up was written with AI assistance from captured local logs, test outputs, and the owner's screen photograph. The photograph itself is omitted to avoid publishing unrelated personal surroundings.