Skip to content

build: rewrite build-machine paths out of the binaries and gate the shipped artifacts - #549

Merged
bong-water-water-bong merged 3 commits into
mainfrom
buildsecrecy-build-flags
Oct 24, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
buildsecrecy-build-flags

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

build: rewrite build-machine paths out of the binaries and gate the shipped artifacts

Three related packaging properties, all enforced rather than assumed:

  • Path mapping. ONEBIT_PATH_MAP_FLAGS (-ffile-prefix-map for the source tree and the build
    tree) is applied to every engine target and handed to the HRX llama.cpp build through
    cmake/hrx.cmake, so a string an assert or a log bakes in reads . / build instead of the
    machine that produced it. With -DONEBIT_GPU_PRIVATE the add-on checkout is mapped as well.
  • Artifact hygiene gate. pack_artifact_hygiene() (scripts/pack-elf.sh) reports any shipped
    program or library that still carries a compiler identification section, a path into a build
    machine, or an absolute RUNPATH; scripts/package-linux.sh fails the package when it reports
    anything. pack_strip() now also drops .comment (and the gold-version note) after
    strip --strip-unneeded.
  • Relative RUNPATH. Every shipped ELF gets $ORIGIN instead of whatever the build left, so a
    program loads its libraries from beside itself and from the system only — ds4-server included,
    which used to keep an SDK path.

Verified on this box with a Release build of the onebit target:

$ strings -a 1bit | grep -cE '^/(home|Users|root)/'      -> 0
$ readelf -p .comment 1bit                               -> GCC: (Ubuntu 16-…)
$ source scripts/pack-elf.sh; pack_artifact_hygiene …    -> 1bit: .comment names the compiler
$ pack_strip …; patchelf --set-rpath '$ORIGIN' …         -> gate: clean
$ readelf -p .comment 1bit                               -> (no .comment)
$ readelf -d 1bit | grep RUNPATH                         -> Library runpath: [$ORIGIN]

The HRX and NPU binaries take the same flags through the same two CMake entry points but were not
built here (they need the toolchain), so they are covered by the gate on the next package run, not
by this transcript.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit a186574)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Potential Path Exposure

The pack_artifact_hygiene function checks for build-machine paths using a regex that may not be comprehensive enough. Specifically, the pattern ^/(home|Users|root)/[A-Za-z0-9_./-]+ might miss some edge cases or alternative path structures, potentially allowing paths to remain embedded in shipped binaries.

        strings -a "$f" 2>/dev/null | grep -qE "^/(home|Users|root)/[A-Za-z0-9_./-]+" &&
            out="$out
$f: embedded build-machine path"
Compiler Identification Section

The check for compiler identification in .comment section uses a simple grep pattern that may not be robust enough. It could potentially miss cases where the compiler name or version string is formatted differently, leading to false negatives in artifact hygiene checks.

        readelf -p .comment "$f" 2>/dev/null | grep -qE "clang version|GCC:" && out="$out
$f: .comment names the compiler"
RPATH Validation

The pack_artifact_hygiene function validates RUNPATH entries but relies on a specific regex pattern to detect absolute paths. If the toolchain root path is not correctly set or if there are variations in how paths are expressed, this validation might fail to catch violations.

if [ -x "$stage/hrx/llama/bin/llama-server" ]; then
    cp "$src/scripts/lemonade-hrx" "$src/scripts/install-lemonade-ppa.sh" "$stage/"
    mkdir -p "$stage/lemonade"
    cp "$src/config/lemonade/1bit-hrx.conf" "$stage/lemonade/"

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Review (orchestrator): right set - path mapping for the source and build trees, a hygiene gate at pack time that fails on leftover build-machine paths, and $ORIGIN RUNPATH on every shipped ELF. Stays a draft until one package build on the box proves the gate and the packed binaries still load (after the leaderboard releases the lane); then it goes in ahead of the next packaging.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 9658e78

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Package-stage proof on strixhalo (copy of the packed lemonade-onebit-rc-2026-10-23, gpu-hold -l "build pi-issues-b" -t 45, no program run):

Gate, on the artifact as shipped — 21 ELFs checked:

21  .comment names the compiler
16  embedded build-machine path

After pack_strip() + rpath handling: .comment findings → 0 (verified by readelf -p .comment), RUNPATH findings → 0, and the 16 path findings remain, which is the expected split: those are __FILE__ string literals in asserts/logs, so packaging cannot remove them — only a rebuild with the prefix map does, which is what the ryzen onebit build in the first commit shows (0 /home strings).

Loadability, ldd only: checked_elfs=21 unresolved=1, and the one is pre-existing and unrelated: llama-perplexity ships without libllama-perplexity-impl.so (the build produced impl libs for server and bench only). Worth its own fix — either build the perplexity impl lib or stop shipping the binary.

One correction this proof forced (second commit): a blanket $ORIGIN leaves ds4-server, ds4, ds4-bench unloadable — they link libhipblaslt.so.1, libhipblas.so.3, librocblas.so.5, libamdhip64.so.7 from the toolchain, and that closure is 905 MB, so it cannot be bundled. pack_localize already does the right thing ($ORIGIN plus the SDK entries the link needed); only the top-level 1bit, which it does not visit, needed its rpath dropped. The gate now accepts an absolute RUNPATH only under the toolchain root and still fails on one pointing at a build tree or a home directory.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Full build + package stage on strixhalo, from a clean tree with these flags (gpu-hold -l "build pi-issues-b" -t 60 then -t 45, nice -n 19 taskset -c 0-3):

$ ninja -C build -j4 1bit llama_hrx        # HRX ExternalProject rebuilt with the flags
[22/22] Linking CXX executable 1bit
build_rc=0   (253 s)

$ scripts/package-linux.sh build /tmp/pkg-out secproof
  hrx/llama/bin: 25 files
→ 1bit-secproof-linux-x86_64.tar.zst   (packaged tree: 23 files, 41M)

=== hygiene gate on the packaged tree ===
(empty — clean)

=== build-machine paths in the shipped 1bit and the HRX library ===
1bit-secproof-linux-x86_64/1bit                              paths=0  comment=0
1bit-secproof-linux-x86_64/hrx/llama/bin/libggml-hrx.so.0.18.0  paths=0  comment=0

=== runpath ===
1bit                     (none)
hrx/llama/bin/llama-server   Library runpath: [$ORIGIN]

=== load check (ldd, no program run) ===
checked_elfs=15  unresolved=0

So every shipped ELF in this package is clean: no build-machine path (the HRX library previously carried
ggml-hrx.cpp, ../ggml-impl.h, graph-program-cache.cpp, and the add-on's hip-abi-check.cpp paths), no
.comment, and the only absolute string the HRX tree had left — a RUNPATH list into the build tree — is
rewritten by pack_localize into $ORIGIN.

Negative control, same gate on the previous candidate (built without the flags):

21  .comment names the compiler
16  embedded build-machine path

Two limits of this run, stated because they matter: this tree has no add-on checkout and no NPU kernel
sets
, so the package is 41 MB against 147 MB for the real one, and package_checks's assertion that
libggml-hrx.so carries ggml_hrx_hip_addon_register cannot pass here — that check is for the release lane
with -DONEBIT_GPU_PRIVATE. The ds4-server rpath path was confirmed separately on the packed candidate
($ORIGIN + the SDK entry pack_localize keeps; its ROCm closure is 905 MB and is not bundled).

Also in this branch: the gate read $ONEBIT_TOOLCHAIN_ROOT, set only by weekly.sh; under set -u its RUNPATH
rule errored out silently instead of checking, so it now defaults to /opt/rocm-therock.

@bong-water-water-bong
bong-water-water-bong marked this pull request as ready for review October 24, 2026 11:14
bong-water-water-bong and others added 3 commits October 24, 2026 08:14
…hipped artifacts

Three related packaging properties, all enforced rather than assumed:

- **Path mapping.** `ONEBIT_PATH_MAP_FLAGS` (`-ffile-prefix-map` for the source tree and the build
  tree) is applied to every engine target and handed to the HRX llama.cpp build through
  `cmake/hrx.cmake`, so a string an assert or a log bakes in reads `.` / `build` instead of the
  machine that produced it. With `-DONEBIT_GPU_PRIVATE` the add-on checkout is mapped as well.
- **Artifact hygiene gate.** `pack_artifact_hygiene()` (scripts/pack-elf.sh) reports any shipped
  program or library that still carries a compiler identification section, a path into a build
  machine, or an absolute RUNPATH; `scripts/package-linux.sh` fails the package when it reports
  anything. `pack_strip()` now also drops `.comment` (and the gold-version note) after
  `strip --strip-unneeded`.
- **Relative RUNPATH.** Every shipped ELF gets `$ORIGIN` instead of whatever the build left, so a
  program loads its libraries from beside itself and from the system only — `ds4-server` included,
  which used to keep an SDK path.

Verified on this box with a Release build of the `onebit` target:

```
$ strings -a 1bit | grep -cE '^/(home|Users|root)/'      -> 0
$ readelf -p .comment 1bit                               -> GCC: (Ubuntu 16-…)
$ source scripts/pack-elf.sh; pack_artifact_hygiene …    -> 1bit: .comment names the compiler
$ pack_strip …; patchelf --set-rpath '$ORIGIN' …         -> gate: clean
$ readelf -p .comment 1bit                               -> (no .comment)
$ readelf -d 1bit | grep RUNPATH                         -> Library runpath: [$ORIGIN]
```

The HRX and NPU binaries take the same flags through the same two CMake entry points but were not
built here (they need the toolchain), so they are covered by the gate on the next package run, not
by this transcript.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
…n the gate

Measured on the packed candidate: a blanket `$ORIGIN` leaves four programs unloadable —
`ds4-server`, `ds4`, `ds4-bench` need `libhipblaslt.so.1`/`libhipblas.so.3`/`librocblas.so.5`/
`libamdhip64.so.7` from the toolchain, and the transitive closure of those is **905 MB**, so bundling
them beside the binary is not an option. `pack_localize` already resolves this correctly: it gives every
program `$ORIGIN` *plus* whatever SDK entries its link needed, and only the top-level engine binary (which
it does not visit) needed its rpath dropped.

So: revert the blanket override, keep `patchelf --remove-rpath` on `1bit` alone, and teach the hygiene
gate to accept an absolute RUNPATH **only** under the toolchain root while still failing on one that
points anywhere else (a build tree, a home directory).

Also found while checking the packed tree (pre-existing, not touched here): `1bit/hrx/llama/bin/llama-perplexity`
is shipped without `libllama-perplexity-impl.so` — it cannot load. It is the only unresolved ELF out of 21
in the candidate; the other 20 resolve, including `ds4-server` through the SDK entry this commit keeps.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
The gate read $ONEBIT_TOOLCHAIN_ROOT, which only weekly.sh sets; under 'set -u' the RUNPATH rule then errored out silently instead of checking. Default it to /opt/rocm-therock.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Review (orchestrator): proven on the public tree (full build with the flags, package stage, gate clean with a negative control, loads from $ORIGIN incl. the DwarfStar binaries). Merging on green CI; the first package built WITH the private add-on (pi flint's decoy package build on strixhalo, rebased onto this) is the proof for the add-on's code objects, and the gate failing loudly there is the intended behaviour.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit a186574

@bong-water-water-bong
bong-water-water-bong merged commit f709474 into main Oct 24, 2026
7 of 8 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the buildsecrecy-build-flags branch October 24, 2026 11:23
bong-water-water-bong added a commit that referenced this pull request Oct 26, 2026
`os/mini` is the BusyBox image that ships today: everything in RAM, a reflash to update. `os/v2` is the
same appliance - Lemonade Server by AMD on the 1bit engine, our kernel, Secure Boot through the enrolled
MOK - on a minimal Ubuntu 26.04 root filesystem with systemd, where every piece we ship is a `.deb` from
our own signed repository and `apt update && apt upgrade` is the update path. `os/mini` keeps shipping
until v2 passes the same checks on hardware.

Seven packages, built from what a release already produces (no binary is rebuilt): `1bit-engine`,
`1bit-lemonade`, `1bit-kernel-image-<kver>`, `1bit-kernel-modules-<kver>`, `1bit-ryzenadj`,
`1bit-appliance` and `1bit-archive-keyring`. The kernel version is in the package name, so an upgrade keeps
the previous kernel installed - that is the rollback. `mkdebs.sh` refuses a stage whose EC module is
unsigned or built for another kernel, and a kernel image with no signature; it localises the engine's
libraries with `scripts/pack-elf.sh` (the same `pack_localize` a release tarball goes through), because a
package that carries the build machine's RUNPATH is a package that does not load (#549).

The live medium is the root filesystem as a squashfs with a tmpfs overlay: one initramfs-tools script
(`scripts/init-bottom/0-1bit-live`) builds the overlay and hands `rootmnt` back through `/conf/param.conf`,
so userspace has nothing custom at boot and a live boot cannot write the medium. `install.sh` is
`1bit-install`, with the same refusal matrix as #539 and three partitions (ESP, `1BIT-ROOT`, `1BIT-DATA`)
instead of a RAM image; it unpacks the squashfs, writes `fstab` by label, one grub entry per installed
kernel, and leaves `/etc/1bit/live` off the installed disk so `1bit-first-boot.service` writes
`/etc/1bit/installed` after making that machine's own SSH keys.

Two VM tests, both judged from what the guest writes to its own disk rather than from QEMU:
`vmtest-v2.sh` (the appliance, booted through OVMF so the secureboot row answers instead of failing for a
QEMU choice) and `vmtest-install-v2.sh` (builds the medium with a test-only unit inside it, boots it with a
target disk and a USB stick attached, runs the refusals, installs, then boots the installed disk alone).

Bugs this found in the v2 path, all fixed here: the engine package could not load its own libraries
(a dev-built stage carried the build tree's RUNPATH and never shipped libhrx/libloomc/libomp); `1bit`
needed `libgomp1`/`libpcre2-8-0`, which minbase has not; lemond exits at once without `HOME` in a systemd
unit; `ssh.service` enabled while `ssh.socket` was masked starts nothing at all on Ubuntu 26.04 (the socket
is what listens, and it carries RequiredBy=ssh.service) - so the socket is what the package enables; minbase
has no `sfdisk` (Ubuntu's lives in `fdisk`); `root=` in the live entry has to be the squashfs partition, or
`data.mount` fights the initramfs for the data partition; and `1bit-appliance` never shipped the aggregate
`/usr/share/licenses/1bit-os` that release-check reads.

Known gap, named in `os/v2/README.md` rather than papered over: the ROCm userspace (HIP, rocBLAS,
hipBLASLt, the HSA runtime) is the machine's, not the engine package's, and the engine's `Depends` do not
name Ubuntu's packages for it yet - gfx1201 support against the distro's ROCm is a hardware question a VM
cannot settle. `vmtest-v2` therefore asserts the package's own libraries resolve and lets only that set be
missing, which is exactly the check that caught the first bug above.

Evidence: `VMTEST-V2 PASS` (seven packages installed, banner, /data, 1bit --help, our kernel's vmlinuz and
initrd in /boot, sshd on 22, apt from the repository, Lemonade 401 + GGUF + chat, release-check with no
FAIL row but `sealed`, which needs a stage that carries the sealed NPU sets) and `VMTEST-V2-INSTALL PASS`
(six refusals, install, installed disk boots alone: INSTALLED mode, first-boot marker and own SSH keys,
/data on its own partition, Lemonade 401 + GGUF + chat, selftest INSTALLED). Transcripts:
`~/wt/osapt-proto/v2-final.log` and `~/wt/osapt-proto/live-install.log`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant