nixos/ollama rocm broken for RX 6900 XT.

@abysssol

Nixpkgs version

Unstable (25.05)

Describe the bug

Rocm works fine in blender, but when run with Ollama I get this in logs:

Jan 22 17:52:59 desktop ollama[196524]: time=2025-01-22T17:52:59.580+03:00 level=WARN source=amd_linux.go:349 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU"

$ ollama --version
Warning: could not connect to a running Ollama instance
Warning: client version is 0.5.7

This might be similar to #375359, but that fix didn't solve my issue.

System metadata

system: "x86_64-linux"
host os: Linux 6.12.9, NixOS, 24.11 (Vicuna), 24.11.20250119.107d5ef
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.24.11
channels(root): ""
nixpkgs: /nix/store/sbnwhhx7zjp45n53bkmnj4whg79aw8pq-source

Notify maintainers

@abysssol
@dit7ya
@elohmeier
@RoyDubnium

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I assert that this issue is relevant for Nixpkgs

I assert that this is a bug and not a support request.
I assert that this is not a duplicate of an existing issue.
I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

Try adding services.ollama.environmentVariables.HSA_OVERRIDE_GFX_VERSION = "10.3.0"; to your system configuration. I have a RX 6800 XT, so you may need to adjust the GFX version.

Try adding services.ollama.environmentVariables.HSA_OVERRIDE_GFX_VERSION = "10.3.0"; to your system configuration. I have a RX 6800 XT, so you may need to adjust the GFX version.

I'll try that tomorrow and report. Thank you!

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html

6900XT isn't currently supported

Refer to ROCm/legacy-rocm-build#4276

Unsupported has never stopped anyone from trying :)

But I think this can be closed as wontfix, even if the override works.

Try adding services.ollama.environmentVariables.HSA_OVERRIDE_GFX_VERSION = "10.3.0"; to your system configuration. I have a RX 6800 XT, so you may need to adjust the GFX version.

You could also use the config services.ollama.rocmOverrideGfx instead of the environment variable.

This works for me atm with a RX 6900 XT.

  nixpkgs.config.rocmSupport = true;

  services = {
    ollama = {
      enable = true;
      acceleration = "rocm";
      # Ollama Port 11434/tcp
      host = "0.0.0.0";
      port = 11434;
      openFirewall = true;
      # pin ollama v0.5.7 until nixpkgs update
      # https://github.com/NixOS/nixpkgs/issues/375359
      package = (pinPackage {
        name = "ollama";
        commit = "d0169965cf1ce1cd68e50a63eabff7c8b8959743";
        sha256 = "sha256:1hh0p0p42yqrm69kqlxwzx30m7i7xqw9m8f224i3bm6wsj4dxm05";
      });
      rocmOverrideGfx = "10.3.0";
      # additional environment variables
      # environmentVariables = { HSA_OVERRIDE_GFX_VERSION="10.3.0"; };
    };
  };

My pinPackage Function Definition

      pinPackage =
        {
          name,
          commit,
          sha256,
        }:
        (import (builtins.fetchTarball {
          inherit sha256;
          url = "https://github.com/NixOS/nixpkgs/archive/${commit}.tar.gz";
        }) { system = pkgs.system; }).${name};

Unfortunately, due to the issue Build failure: rocmPackages.llvm.libcxx, I am currently unable to upgrade to the latest nixpkgs available and I continue using this revision.

NixOS/nixpkgs/5df43628fdf08d642be8ba5b3625a6c70731c19c (2025-01-16 21:27:11)

So there seems to be something wrong with ROCm at the moment. But now I'm also stuck on Ollama v0.5.4, so I had to overwrite the package version with services.ollama.package. If your nixpkgs already contain Ollama v0.5.7, you can probably omit the services.ollama.package property.

Very interesting that your nixpkgs seem to contain the new Ollama version, but are not yet affected by #375745.
Maybe you can also tell me the revision of the nixpkgs you are currently using?

Config nixpkgs.config.rocmSupport combined with services.ollama.rocmOverrideGfx = "10.3.0"; should be enough to enable ROCm for our cards.

I kept services.ollama.acceleration from a previous system configuration, but the documentation says that it also automatically follows nixpkgs.config.rocmSupport and you can try without setting it explicitly.

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html

6900XT isn't currently supported

Refer to ROCm/ROCm#4276

Wym? Rocm works great for me in Blender. Also Ollama lists support for my GPU.

I'll try the rocmOverrideGfx = "10.3.0"; today, thank you very much everyone!

Maybe you can also tell me the revision of the nixpkgs you are currently using?

Nixpkgs Master that I use for the ollama module: 2fc5aeb049f44ed4f9e877cda8a1c334612e1d7a
Unstable that I use for the ollama package: 9e4d5190a9482a1fb9d18adf0bdb83c6e506eaab
Stable that I cureently use for the whole system (just in case): 107d5ef05c0b1119749e381451389eded30fb0d5

Also just in case my ollama configuration.

I have the same issue with the same GPU. This used to work and has broken at some point with a nixpkgs update.

I am using services.ollama.package = pkgs.ollama-rocm. If I set acceleration = "rocm" instead, it tries to compile, but it fails due to #375745

I am going to try that again once #375745 is fixed.

rocmOverrideGfx = "10.3.0"; didn't work. Using ollama-rocm also did nothing. I think fazo96 is right and I just need to wait for 375745 to get fixed.

@voronind-com

I agree, @voronind-com, a possible solution has been merged at #375850
https://nixpk.gs/pr-tracker.html?pr=375850
waiting for it to hit unstable, hopefully can close soon

edit: something is still the matter, follow
#376930
and
#375745

Hardware acceleration for both ollama and whisper-cpp works on my AMD RX 6900 XT GPU.

I no longer need to pin the package version (as described in my comment above).

I've seen comments removing the nixpkgs.config.rocmSupport = true;, that is not necessary anymore.

This has been working for me recently. However I noticed that setting services.ollama.pkg = pkgs.ollama-rocm; does not work. Instead I have to set services.ollama.acceleration = "rocm"; and leave the default setting for pkg and then it works.

I have a 6900 XT. I am also setting rocmOverrideGfx = "10.3.0";

All is working now, thanks!

@fazo96

However I noticed that setting services.ollama.pkg = pkgs.ollama-rocm; does not work.

@fazo96 The option is services.ollama.package, not services.ollama.pkg. That is, package not pkg. If you are using the latest stable version of NixOS (25.11), services.ollama.package = pkgs.ollama-rocm; should now use rocm as expected.

Navigation Menu

Repository navigation

nixos/ollama rocm broken for RX 6900 XT. #375910

Nixpkgs version

Describe the bug

System metadata

Notify maintainers

@abysssol
@dit7ya
@elohmeier
@RoyDubnium

I assert that this issue is relevant for Nixpkgs

Is this issue important to you?

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

nixos/ollama rocm broken for RX 6900 XT. #375910

Description

Nixpkgs version

Describe the bug

System metadata

Notify maintainers

@abysssol @dit7ya @elohmeier @RoyDubnium

I assert that this issue is relevant for Nixpkgs

Is this issue important to you?

Activity

SigmaSquadron commented on Jan 22, 2025

voronind-com commented on Jan 22, 2025

fabianhjr commented on Jan 22, 2025

SigmaSquadron commented on Jan 22, 2025

zierf commented on Jan 23, 2025

voronind-com commented on Jan 23, 2025

voronind-com commented on Jan 23, 2025

fazo96 commented on Jan 23, 2025

voronind-com commented on Jan 24, 2025

Dakoteus commented on Jan 25, 2025

zierf commented on Feb 10, 2025

fazo96 commented on Feb 11, 2025

voronind-com commented on Dec 28, 2025

abysssol commented on Dec 29, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

@abysssol
@dit7ya
@elohmeier
@RoyDubnium