Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -147,16 +147,25 @@ jobs:
# MLX sidecar steps below. Shipping llama.cpp alongside it would mean
# compiling it from source on every release for an engine that is
# slower on Apple silicon and that the plug-in no longer offers.
#
# `rust_toolchain` holds Windows at 1.99: Cargo 1.100 moved build-script
# output one directory deeper, and llama-cpp-sys-2's build.rs (up to
# 0.1.161) then panics copying its DLLs, while the DLL staging below
# relies on where that copy used to put them. The `RUST_VERSION` note
# in server-rs/Dockerfile has the details, and what to check before
# moving either on.
- os: windows-latest
os_name: windows
arch: x64
bin_ext: .exe
cargo_features: --features llamacpp
rust_toolchain: '1.99'
- os: macos-latest
os_name: macos
arch: arm64
bin_ext: ''
cargo_features: ''
rust_toolchain: stable

name: Build LrGeniusAI - ${{ matrix.os_name }}-${{ matrix.arch }}

Expand Down Expand Up @@ -198,10 +207,14 @@ jobs:
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

# clippy is named because the action installs with `--profile minimal`.
# The runner's preinstalled stable already has it, but a pinned version
# is a fresh install, and the build step below lints.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: stable
toolchain: ${{ matrix.rust_toolchain }}
components: clippy

# The `llamacpp` feature compiles llama.cpp from source: cmake for the
# build itself, libclang for bindgen. Installed explicitly rather than
Expand Down
20 changes: 18 additions & 2 deletions docs/wiki/Dev-Build-Environment-Setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,29 @@ build panics without it):
# Expected: something like C:\VulkanSDK\1.4.xxx.x
```

Build this feature with Rust **1.99**, not the current stable. Cargo 1.100
moved build-script output one directory deeper, and llama-cpp-sys-2's build
script (up to 0.1.161) then panics while copying its DLLs:

```
panicked at ...\llama-cpp-sys-2-0.1.156\build.rs:1431:56:
called `Result::unwrap()` on an `Err` value: Os { code: 3, kind: NotFound, ... }
```

The release build is held at 1.99 for the same reason (`rust_toolchain` in
`.github/workflows/release.yml`). The baseline build is not affected.

```powershell
rustup toolchain install 1.99 --component clippy
```

Open a **new** terminal so the updated environment variables are picked up,
then:

```powershell
cd server-rs
cargo build --release -p lrg-server --features llamacpp
cargo clippy --workspace --all-targets --features llamacpp
cargo +1.99 build --release -p lrg-server --features llamacpp
cargo +1.99 clippy --workspace --all-targets --features llamacpp
```

This is a from-source llama.cpp compile plus Vulkan shader compilation, so
Expand Down
6 changes: 4 additions & 2 deletions docs/wiki/Dev-Server-README.md
Original file line number Diff line number Diff line change
Expand Up @@ -653,8 +653,10 @@ the variable and answers only to the API call. See the doc comment on
## Docker

`Dockerfile` here builds and ships the compiled binary (multi-stage:
`rust:1-trixie` builder, `debian:trixie-slim` runtime; trixie because the
ONNX Runtime libraries need glibc 2.38+). Model files live in the `/models`
`rust:1.99-trixie` builder, `debian:trixie-slim` runtime; trixie because the
ONNX Runtime libraries need glibc 2.38+, and Rust held at 1.99 because
llama.cpp's build breaks on Cargo 1.100's build-dir layout — see the
`RUST_VERSION` note in the Dockerfile). Model files live in the `/models`
volume: the Dockerfile points every family's env var above there
(`/models/siglip2/...`, `/models/face/...`, `/models/bioclip/...`,
`/models/llm`), which is where the plug-in's *Download AI models* puts them —
Expand Down
13 changes: 12 additions & 1 deletion server-rs/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,19 @@
# Ubuntu 24.04, because the ONNX Runtime libraries ort-sys links need glibc
# 2.38+ (see the trixie note below); 22.04 has 2.35.
ARG UBUNTU_VERSION=24.04
# Held at 1.99 because Cargo 1.100 moved build-script output one directory
# deeper (target/release/build/<crate>/<hash>/out instead of
# build/<crate>-<hash>/out), and two things here assume the old layout:
# llama-cpp-sys-2's build.rs (up to 0.1.161) finds the target dir by walking
# three levels up from OUT_DIR, and then panics hard-linking its shared libs
# into a deps/ directory that is not there; and the library collection in
# the build stage below globs build/llama-cpp-sys-2-*/out. Before moving
# this on, check that llama-cpp-sys-2's get_cargo_target_dir() copes with the
# new layout, and update that glob. .github/workflows/release.yml holds the
# Windows build at the same version for the same reason.
ARG RUST_VERSION=1.99

FROM rust:1-trixie AS rust-toolchain
FROM rust:${RUST_VERSION}-trixie AS rust-toolchain

# --- Build environment, one per LLAMACPP value -----------------------------

Expand Down Expand Up @@ -194,7 +205,7 @@
# The onnxruntime and llama.cpp shared libraries ship next to the binary (see
# the build stage above); it has no rpath pointing here, so the loader needs
# telling. Kept after the base image's own value (the CUDA images set one).
ENV LD_LIBRARY_PATH=/app${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Check warning on line 208 in server-rs/Dockerfile

View workflow job for this annotation

GitHub Actions / Build Docker image - none aarch64

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

Check warning on line 208 in server-rs/Dockerfile

View workflow job for this annotation

GitHub Actions / Build Docker image - none x64

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

Check warning on line 208 in server-rs/Dockerfile

View workflow job for this annotation

GitHub Actions / Build Docker image - vulkan aarch64

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

Check warning on line 208 in server-rs/Dockerfile

View workflow job for this annotation

GitHub Actions / Build Docker image - vulkan x64

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

# Fail the build, not the first start: every library the server and its
# llama.cpp modules link has to be in the image. All but the NVIDIA driver's
Expand Down
6 changes: 4 additions & 2 deletions server-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -649,8 +649,10 @@ the variable and answers only to the API call. See the doc comment on
## Docker

`Dockerfile` here builds and ships the compiled binary (multi-stage:
`rust:1-trixie` builder, `debian:trixie-slim` runtime; trixie because the
ONNX Runtime libraries need glibc 2.38+). Model files live in the `/models`
`rust:1.99-trixie` builder, `debian:trixie-slim` runtime; trixie because the
ONNX Runtime libraries need glibc 2.38+, and Rust held at 1.99 because
llama.cpp's build breaks on Cargo 1.100's build-dir layout — see the
`RUST_VERSION` note in the Dockerfile). Model files live in the `/models`
volume: the Dockerfile points every family's env var above there
(`/models/siglip2/...`, `/models/face/...`, `/models/bioclip/...`,
`/models/llm`), which is where the plug-in's *Download AI models* puts them —
Expand Down
Loading