ollama-rocm: Ollama-rocm 0.5.4 is broken (at least for me on RX 7900 XTX), while 0.5.1 still works

@abysssol

Nixpkgs version

Unstable (25.05)

Describe the bug

The ollama 0.5.4 from unstable is currently broken from be7b92e, while 0.5.1 still works (01cda5c).

I strongly suspect that this commit broke it: be7b92e (0.5.1 -> 0.5.4 bump), as it contains changes beyond simple version bump. And this revision is broken.

In particular, ollama ps show that it is 100% loaded to GPU:

$ ollama ps
NAME             ID              SIZE     PROCESSOR    UNTIL              
phi4:14b-q8_0    310d366232f4    19 GB    100% GPU     4 minutes from now

But nvtop shows that video memory usage is low.

Because inference is being done on my CPU:

Steps to reproduce

Just install ollama from the latest unstable (e.g. 5df4362) ant try to use ollama with any model. GPU will not be used.

Expected behaviour

Inference is running on GPU, just like for older versions of the package

Screenshots

No response

Relevant log output

No runners beyond CPU are detected for 0.5.4.

січ 20 20:51:24 nucebox ollama[103167]: time=2025-01-20T20:51:24.164+02:00 level=INFO source=images.go:764 msg="total unused blobs removed: 0"
січ 20 20:51:24 nucebox ollama[103167]: time=2025-01-20T20:51:24.165+02:00 level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)"
січ 20 20:51:24 nucebox ollama[103167]: time=2025-01-20T20:51:24.165+02:00 level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners=[cpu]

For 0.5.1 everything is fine:

січ 20 21:05:34 nucebox ollama[108779]: time=2025-01-20T21:05:34.428+02:00 level=INFO source=routes.go:1246 msg="Listening on 127.0.0.1:11434 (version 0.5.1)"
січ 20 21:05:34 nucebox ollama[108779]: time=2025-01-20T21:05:34.429+02:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama2988464704/runners
січ 20 21:05:34 nucebox ollama[108779]: time=2025-01-20T21:05:34.545+02:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu cpu_avx cpu_avx2 rocm]"

Additional context

No response

System metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 6.12.9, NixOS, 24.11 (Vicuna), 24.11.20250119.107d5ef`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.24.11`
 - nixpkgs: `/nix/store/sbnwhhx7zjp45n53bkmnj4whg79aw8pq-source`

Notify maintainers

@abysssol
@dit7ya
@elohmeier
@RoyDubnium

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I assert that this issue is relevant for Nixpkgs

I assert that this is a bug and not a support request.
I assert that this is not a duplicate of an existing issue.
I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

That might be a wild guess, but I suspect that the extra runners are not copied into the package. That is, this might be the culprit:

-  preBuild = ''
+  preBuild =
+    let
+      dist_cmd =
+        if cudaRequested then
+          "dist_cuda_v${cudaMajorVersion}"
+        else if rocmRequested then
+          "dist_rocm"
+        else
+          "dist";
+    in
     # build llama.cpp libraries for ollama
-    make -j $NIX_BUILD_CORES
-  '';
-
-  postInstall = lib.optionalString stdenv.hostPlatform.isLinux ''
-    # copy libggml_*.so and runners into lib
-    # https://github.com/ollama/ollama/blob/v0.4.4/llama/make/gpu.make#L90
-    mkdir -p $out/lib
-    cp -r dist/*/lib/* $out/lib/
-  '';
+    ''
+      make ${dist_cmd} -j $NIX_BUILD_CORES
+    '';

Think this was already fixed in #373234

PR tracker: https://nixpk.gs/pr-tracker.html?pr=373234

Think this was already fixed in #373234

PR tracker: https://nixpk.gs/pr-tracker.html?pr=373234

Thanks! Glad yo hear that!

Waiting for it to arrive to unstable.

Hmm. I still have the issue even with a module from master. How it was on stable 24.11:

Jan 22 17:20:52 desktop ollama[160537]: time=2025-01-22T17:20:52.170+03:00 level=WARN source=amd_linux.go:445 msg="amdgpu detected, but no compatible rocm library found.  Either install rocm v6, or follow manual install instructions at https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install"

How it is on master module:

Jan 22 17:52:59 desktop ollama[196524]: time=2025-01-22T17:52:59.580+03:00 level=WARN source=amd_linux.go:349 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU"

GPU is RX 6900 XT, rocm works in Blender just fine.

$ ollama --version
Warning: could not connect to a running Ollama instance
Warning: client version is 0.5.7

Can confirm that it is fixed with version bump to 0.5.7 (d016996).

Closing the issue.

Navigation Menu

Repository navigation

ollama-rocm: Ollama-rocm 0.5.4 is broken (at least for me on RX 7900 XTX), while 0.5.1 still works #375359

Nixpkgs version

Describe the bug

Steps to reproduce

Expected behaviour

Screenshots

Relevant log output

Additional context

System metadata

Notify maintainers

I assert that this issue is relevant for Nixpkgs

Is this issue important to you?

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

ollama-rocm: Ollama-rocm 0.5.4 is broken (at least for me on RX 7900 XTX), while 0.5.1 still works #375359

Description

Nixpkgs version

Describe the bug

Steps to reproduce

Expected behaviour

Screenshots

Relevant log output

Additional context

System metadata

Notify maintainers

I assert that this issue is relevant for Nixpkgs

Is this issue important to you?

Activity

arbv commented on Jan 20, 2025

LunNova commented on Jan 20, 2025

arbv commented on Jan 21, 2025

voronind-com commented on Jan 22, 2025

arbv commented on Jan 22, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions