ollama: 0.5.4 -> 0.5.5 by liberodark · Pull Request #373234 · NixOS/nixpkgs

liberodark · 2025-01-12T16:49:06Z

Changelog :

https://github.com/ollama/ollama/releases/tag/v0.5.4 > https://github.com/ollama/ollama/releases/tag/v0.5.5

Also is pre-release so probably can wait for release.

Things done

Add a 👍 reaction to pull requests you find important.

niklaskorz · 2025-01-12T19:08:23Z

0.5.4 was just merged, please rebase

GaetanLepage · 2025-01-12T19:54:46Z

Yes, I would prefer if we wait for it to be tagged as a proper release.

mastoca · 2025-01-14T23:54:30Z

looks like 0.5.5 is official now.

GaetanLepage · 2025-01-15T10:00:09Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 373234

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`aarch64-linux`

❌ 3 packages failed to build:

alpaca
chatd
ollama

✅ 1 package built:

ollama-cuda

`x86_64-darwin`

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 1 package failed to build:

ollama

GaetanLepage · 2025-01-15T10:07:11Z

Fails on aarch64-linux:

Running phase: installPhase
cp: missing destination file operand after '/nix/store/4wcrf57b0mavzidx6b9z78k4wcrh3jxa-ollama-0.5.5/lib/'
Try 'cp --help' for more information.

breakds · 2025-01-15T18:10:32Z

Tested and can verify that now 0.5.5 runs well on x86_64 linux + Nvidia GPU. More specifically the postInstall addressed the issue that "since 0.5.2 without copying those libs, ollama will still work but does not start the model on CUDA".

prusnak · 2025-01-15T21:29:42Z

Commit message should read ollama: 0.5.4 -> 0.5.5 (-> not >)

GaetanLepage · 2025-01-16T07:43:06Z

Fails on aarch64-linux:

Running phase: installPhase
cp: missing destination file operand after '/nix/store/4wcrf57b0mavzidx6b9z78k4wcrh3jxa-ollama-0.5.5/lib/'
Try 'cp --help' for more information.

This issue is still making the aarch64-linux build fail.

prusnak · 2025-01-16T07:52:35Z

This issue is still making the aarch64-linux build fail.

Yes, because @liberodark still has not applied the suggestion from my comment #373234 (comment)

Can you try if that fixes the build for you?

GaetanLepage · 2025-01-16T14:09:03Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 373234

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`aarch64-linux`

✅ 4 packages built:

alpaca
chatd
ollama
ollama-cuda

`x86_64-darwin`

❌ 1 package failed to build:

chatd

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 2 packages failed to build:

chatd
ollama

GaetanLepage

LGTM !

LunNova · 2025-01-16T16:22:50Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 373234

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

GaetanLepage · 2025-01-16T16:49:52Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 373234

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`aarch64-linux`

✅ 4 packages built:

alpaca
chatd
ollama
ollama-cuda

`x86_64-darwin`

❌ 1 package failed to build:

chatd

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 2 packages failed to build:

chatd
ollama

GaetanLepage

Good job, thanks !

mastoca

looks good

abysssol

Tested ollama-rocm: it appears to function correctly.

abysssol · 2025-01-16T20:00:35Z

+  postInstall = lib.optionalString (stdenv.hostPlatform.isx86 || enableRocm || enableCuda) ''
+    # copy libggml_*.so and runners into lib
+    # https://github.com/ollama/ollama/blob/v0.4.4/llama/make/gpu.make#L90
+    mkdir -p $out/lib
+    cp -r dist/*/lib/* $out/lib/
+  '';


I think this is correct now, but I want to know @LunNova's thoughts. My previous suggestion on this (replacing the condition with (enableRocm || enableCuda)) turned out to be wrong, as @LunNova helpfully pointed out last time.

Seems good enough!

Oh, someone should check that x86_64-darwin's happy. I can't.

My only lingering worry is that devices like Jetson have Cuda and aarch64.

My only lingering worry is that devices like Jetson have Cuda and aarch64.

The condition (stdenv.hostPlatform.isx86 || enableRocm || enableCuda) only contains || (or), not && (and), so it should work fine if CUDA is requested on ARM (as long as ollama and llama-cpp correctly support that combination). However, enableCuda does require Linux; most programs aren't designed to run on bare metal, though, so that's probably not especially notable.

GaetanLepage · 2025-01-16T21:41:06Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 373234

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`aarch64-linux`

✅ 4 packages built:

alpaca
chatd
ollama
ollama-cuda

`x86_64-darwin`

❌ 1 package failed to build:

chatd

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 2 packages failed to build:

chatd
ollama

gshpychka · 2025-01-17T11:13:42Z

@GaetanLepage but why are the darwin build failures acceptable?

prusnak · 2025-01-17T11:17:01Z

@GaetanLepage but why are the darwin build failures acceptable?

they are spurious - both chatd and ollama build just fine on my aarch64-darwin and also on x86_64-darwin

GaetanLepage · 2025-01-17T12:32:19Z

@GaetanLepage but why are the darwin build failures acceptable?

Yes, in particular, the ollama failure on aarch64-darwin does not occur when building outside of the sandbox. Also, it was not brought by this PR.

HuzaifaTP · 2025-01-22T17:05:01Z

not sure if its related but encountered this on updating unstable and rebuilding:

  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_load_explicit.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store_explicit.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.wait/atomic_notify_all.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.wait/atomic_notify_one.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.wait/atomic_wait.pass.cpp
  llvm-libc++-shared.cfg.in :: std/atomics/atomics.types.operations/atomics.types.operations.wait/atomic_wait_explicit.pass.cpp
  llvm-libc++-shared.cfg.in :: std/input.output/filesystems/class.path/path.nonmember/path.io.unicode_bug.pass.cpp
  llvm-libc++-shared.cfg.in :: std/utilities/function.objects/func.invoke/invoke_r.temporary.verify.cpp

********************
Failed Tests (1):
  llvm-libc++-shared.cfg.in :: libcxx/transitive_includes.sh.cpp


Testing Time: 2782.86s

Total Discovered Tests: 7869
  Unsupported      :  428 (5.44%)
  Passed           : 7400 (94.04%)
  Expectedly Failed:   40 (0.51%)
  Failed           :    1 (0.01%)
FAILED: CMakeFiles/check-runtimes /build/source/runtimes/build/CMakeFiles/check-runtimes
cd /build/source/runtimes/build && /nix/store/inqsga0i9wl1z80xwiq4nijbrnpqz6b4-python3-3.12.8-env/bin/python3.12 /nix/store/9rqf51h9yhz3q2y1in0ql12xl2vibg00-python3.12-lit-18.1.8/bin/.lit-wrapped -sv --show-xfail --show-unsupported /build/source/runtimes/build/libcxx/test
ninja: build stopped: subcommand failed.
error: builder for '/nix/store/35a4dhaa1b27dcscz6qnj97a15qma2bx-rocm-llvm-libcxx-6.0.2.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/wjf41g7sfn87s9s1x0apffcb034s5k6b-rocm-llvm-clang-wrapper-6.0.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/lcsrng48sryasfhzcgm5r3m2zs16ri6a-clr-6.0.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fl29p5hq0a638c791g8big77aaz03wg0-rocm-clang.drv' failed to build
error: 1 dependencies of derivation '/nix/store/hs892zry0nf41pp65xr991w4xjhfh378-rocm-llvm-openmp-6.0.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/1wqp3mwbqg5rz8f5hj26npf9j1xasf18-stdenv-linux.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xs7k2bwasf72rz0mmy4dcvnhgrcp1icd-graphics-drivers.drv' failed to build
error: 1 dependencies of derivation '/nix/store/rnqmkd85jsx7jqj35y5j9j7x2pq7yrx8-ollama-0.5.7.drv' failed to build
error: 1 dependencies of derivation '/nix/store/q31qri5xc3fhvnhwrxwdfnkck89r6y8m-graphics-driver.conf.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xjc78q4p9s5kclzqyl48g18phlmg4nwb-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qlkindfrbccs2zy2ywdnfayjwcsprpc4-unit-ollama.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/r9q92vrslbhmr60yw8xh0iivqg3i2dv8-unit-script-ollama-model-loader-start.drv' failed to build
error: 1 dependencies of derivation '/nix/store/z99gdg97bhi73clfmh33lf6gfrimp7qp-nixos-system-latitude2-25.05beta742084.9e4d5190a948.drv' failed to build

amd gpu + rocm config:

{pkgs,...}:

{
      systemd.packages = with pkgs; [ lact ];
      systemd.services.lactd.wantedBy = ["multi-user.target"];
      hardware.amdgpu.amdvlk.enable = false;
services.ollama = {
  enable = true;
  loadModels = [ "codellama" "llama3.2" ];
  acceleration = "rocm";
  rocmOverrideGfx = "10.1.1";
    environmentVariables = {
       HCC_AMDGPU_TARGET = "gfx1100";
     };
};
      hardware.amdgpu.opencl.enable = true;
      hardware.graphics = {
        enable = true;
        enable32Bit = true;
        extraPackages = with pkgs; [
                 mesa
                 rocmPackages.rocblas
                 rocmPackages.rocm-smi
                 rocmPackages.rocminfo
                 rocmPackages.hipblas
                 rocmPackages.rocm-device-libs
                 rocmPackages.rpp
                 pcre2
                 libselinux
                 libcap
        ];
      };
}

github-actions Bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jan 12, 2025

liberodark marked this pull request as ready for review January 12, 2025 16:57

nix-owners Bot requested review from RoyDubnium, abysssol, dit7ya and elohmeier January 12, 2025 16:57

liberodark marked this pull request as draft January 12, 2025 16:58

niklaskorz mentioned this pull request Jan 12, 2025

ollama: 0.5.1 -> 0.5.4 #365718

Merged

13 tasks

liberodark force-pushed the ollama branch from 97d07d3 to 1192657 Compare January 12, 2025 19:14

liberodark changed the title ~~ollama: 0.5.1 > 0.5.5~~ ollama: 0.5.4 > 0.5.5 Jan 12, 2025

liberodark marked this pull request as ready for review January 12, 2025 19:16

liberodark marked this pull request as draft January 12, 2025 19:58

liberodark marked this pull request as ready for review January 15, 2025 07:58

liberodark force-pushed the ollama branch from 1192657 to d9a9b7d Compare January 15, 2025 08:06

pbsds mentioned this pull request Jan 15, 2025

fix ollama gpu acceleration #373924

Closed

2 of 13 tasks

liberodark force-pushed the ollama branch 2 times, most recently from 22dc139 to ad52a9b Compare January 15, 2025 09:13

prusnak requested changes Jan 15, 2025

View reviewed changes

Comment thread pkgs/by-name/ol/ollama/package.nix Outdated

liberodark force-pushed the ollama branch from ad52a9b to 39860cd Compare January 16, 2025 12:58

liberodark changed the title ~~ollama: 0.5.4 > 0.5.5~~ ollama: 0.5.4 -> 0.5.5 Jan 16, 2025

liberodark force-pushed the ollama branch from 39860cd to 4d546cc Compare January 16, 2025 12:59

GaetanLepage approved these changes Jan 16, 2025

View reviewed changes

abysssol reviewed Jan 16, 2025

View reviewed changes

Comment thread pkgs/by-name/ol/ollama/package.nix Outdated

abysssol requested changes Jan 16, 2025

View reviewed changes

Comment thread pkgs/by-name/ol/ollama/package.nix Outdated

liberodark force-pushed the ollama branch from 4d546cc to a0323d3 Compare January 16, 2025 15:36

prusnak approved these changes Jan 16, 2025

View reviewed changes

prusnak requested a review from abysssol January 16, 2025 17:12

GaetanLepage approved these changes Jan 16, 2025

View reviewed changes

mastoca approved these changes Jan 16, 2025

View reviewed changes

abysssol reviewed Jan 16, 2025

View reviewed changes

Comment thread pkgs/by-name/ol/ollama/package.nix Outdated

ollama: 0.5.4 -> 0.5.5

8b67d6d

liberodark force-pushed the ollama branch from a0323d3 to 8b67d6d Compare January 16, 2025 19:03

mastoca approved these changes Jan 16, 2025

View reviewed changes

abysssol approved these changes Jan 16, 2025

View reviewed changes

abysssol requested a review from LunNova January 16, 2025 20:09

GaetanLepage merged commit 9277281 into NixOS:master Jan 16, 2025

LunNova mentioned this pull request Jan 20, 2025

ollama-rocm: Ollama-rocm 0.5.4 is broken (at least for me on RX 7900 XTX), while 0.5.1 still works #375359

Closed

3 tasks done

ShamrockLee mentioned this pull request Apr 20, 2025

NixOS 25.05 – Feature Freeze & Release Blockers #393359

Closed

37 of 47 tasks

Uh oh!

Conversation

liberodark commented Jan 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

niklaskorz commented Jan 12, 2025

Uh oh!

GaetanLepage commented Jan 12, 2025

Uh oh!

mastoca commented Jan 14, 2025

Uh oh!

GaetanLepage commented Jan 15, 2025

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

Uh oh!

GaetanLepage commented Jan 15, 2025

Uh oh!

breakds commented Jan 15, 2025

Uh oh!

Uh oh!

prusnak commented Jan 15, 2025

Uh oh!

GaetanLepage commented Jan 16, 2025

Uh oh!

prusnak commented Jan 16, 2025

Uh oh!

GaetanLepage commented Jan 16, 2025

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

Uh oh!

GaetanLepage left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LunNova commented Jan 16, 2025

nixpkgs-review result

x86_64-linux

Uh oh!

GaetanLepage commented Jan 16, 2025

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

Uh oh!

GaetanLepage left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mastoca left a comment

Choose a reason for hiding this comment

Uh oh!

abysssol left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abysssol Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

LunNova Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

LunNova Jan 16, 2025 • edited by abysssol Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mastoca Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

abysssol Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

liberodark commented Jan 12, 2025 •

edited

Loading

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

`nixpkgs-review` result

`x86_64-linux`

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

abysssol left a comment •

edited

Loading

LunNova Jan 16, 2025 •

edited by abysssol

Loading

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

prusnak commented Jan 17, 2025 •

edited

Loading

HuzaifaTP commented Jan 22, 2025 •

edited

Loading