Repository navigation
build: rewrite build-machine paths out of the binaries and gate the shipped artifacts - #549
Conversation
PR Reviewer Guide 🔍(Review updated until commit a186574)Here are some key observations to aid the review process:
|
|
Review (orchestrator): right set - path mapping for the source and build trees, a hygiene gate at pack time that fails on leftover build-machine paths, and $ORIGIN RUNPATH on every shipped ELF. Stays a draft until one package build on the box proves the gate and the packed binaries still load (after the leaderboard releases the lane); then it goes in ahead of the next packaging. |
|
Persistent review updated to latest commit 9658e78 |
|
Package-stage proof on strixhalo (copy of the packed Gate, on the artifact as shipped — 21 ELFs checked: After Loadability, One correction this proof forced (second commit): a blanket |
|
Full build + package stage on strixhalo, from a clean tree with these flags ( So every shipped ELF in this package is clean: no build-machine path (the HRX library previously carried Negative control, same gate on the previous candidate (built without the flags): Two limits of this run, stated because they matter: this tree has no add-on checkout and no NPU kernel Also in this branch: the gate read |
…hipped artifacts Three related packaging properties, all enforced rather than assumed: - **Path mapping.** `ONEBIT_PATH_MAP_FLAGS` (`-ffile-prefix-map` for the source tree and the build tree) is applied to every engine target and handed to the HRX llama.cpp build through `cmake/hrx.cmake`, so a string an assert or a log bakes in reads `.` / `build` instead of the machine that produced it. With `-DONEBIT_GPU_PRIVATE` the add-on checkout is mapped as well. - **Artifact hygiene gate.** `pack_artifact_hygiene()` (scripts/pack-elf.sh) reports any shipped program or library that still carries a compiler identification section, a path into a build machine, or an absolute RUNPATH; `scripts/package-linux.sh` fails the package when it reports anything. `pack_strip()` now also drops `.comment` (and the gold-version note) after `strip --strip-unneeded`. - **Relative RUNPATH.** Every shipped ELF gets `$ORIGIN` instead of whatever the build left, so a program loads its libraries from beside itself and from the system only — `ds4-server` included, which used to keep an SDK path. Verified on this box with a Release build of the `onebit` target: ``` $ strings -a 1bit | grep -cE '^/(home|Users|root)/' -> 0 $ readelf -p .comment 1bit -> GCC: (Ubuntu 16-…) $ source scripts/pack-elf.sh; pack_artifact_hygiene … -> 1bit: .comment names the compiler $ pack_strip …; patchelf --set-rpath '$ORIGIN' … -> gate: clean $ readelf -p .comment 1bit -> (no .comment) $ readelf -d 1bit | grep RUNPATH -> Library runpath: [$ORIGIN] ``` The HRX and NPU binaries take the same flags through the same two CMake entry points but were not built here (they need the toolchain), so they are covered by the gate on the next package run, not by this transcript. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
…n the gate Measured on the packed candidate: a blanket `$ORIGIN` leaves four programs unloadable — `ds4-server`, `ds4`, `ds4-bench` need `libhipblaslt.so.1`/`libhipblas.so.3`/`librocblas.so.5`/ `libamdhip64.so.7` from the toolchain, and the transitive closure of those is **905 MB**, so bundling them beside the binary is not an option. `pack_localize` already resolves this correctly: it gives every program `$ORIGIN` *plus* whatever SDK entries its link needed, and only the top-level engine binary (which it does not visit) needed its rpath dropped. So: revert the blanket override, keep `patchelf --remove-rpath` on `1bit` alone, and teach the hygiene gate to accept an absolute RUNPATH **only** under the toolchain root while still failing on one that points anywhere else (a build tree, a home directory). Also found while checking the packed tree (pre-existing, not touched here): `1bit/hrx/llama/bin/llama-perplexity` is shipped without `libllama-perplexity-impl.so` — it cannot load. It is the only unresolved ELF out of 21 in the candidate; the other 20 resolve, including `ds4-server` through the SDK entry this commit keeps. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The gate read $ONEBIT_TOOLCHAIN_ROOT, which only weekly.sh sets; under 'set -u' the RUNPATH rule then errored out silently instead of checking. Default it to /opt/rocm-therock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
d23806f to
a186574
Compare
|
Review (orchestrator): proven on the public tree (full build with the flags, package stage, gate clean with a negative control, loads from $ORIGIN incl. the DwarfStar binaries). Merging on green CI; the first package built WITH the private add-on (pi flint's decoy package build on strixhalo, rebased onto this) is the proof for the add-on's code objects, and the gate failing loudly there is the intended behaviour. |
|
Persistent review updated to latest commit a186574 |
`os/mini` is the BusyBox image that ships today: everything in RAM, a reflash to update. `os/v2` is the same appliance - Lemonade Server by AMD on the 1bit engine, our kernel, Secure Boot through the enrolled MOK - on a minimal Ubuntu 26.04 root filesystem with systemd, where every piece we ship is a `.deb` from our own signed repository and `apt update && apt upgrade` is the update path. `os/mini` keeps shipping until v2 passes the same checks on hardware. Seven packages, built from what a release already produces (no binary is rebuilt): `1bit-engine`, `1bit-lemonade`, `1bit-kernel-image-<kver>`, `1bit-kernel-modules-<kver>`, `1bit-ryzenadj`, `1bit-appliance` and `1bit-archive-keyring`. The kernel version is in the package name, so an upgrade keeps the previous kernel installed - that is the rollback. `mkdebs.sh` refuses a stage whose EC module is unsigned or built for another kernel, and a kernel image with no signature; it localises the engine's libraries with `scripts/pack-elf.sh` (the same `pack_localize` a release tarball goes through), because a package that carries the build machine's RUNPATH is a package that does not load (#549). The live medium is the root filesystem as a squashfs with a tmpfs overlay: one initramfs-tools script (`scripts/init-bottom/0-1bit-live`) builds the overlay and hands `rootmnt` back through `/conf/param.conf`, so userspace has nothing custom at boot and a live boot cannot write the medium. `install.sh` is `1bit-install`, with the same refusal matrix as #539 and three partitions (ESP, `1BIT-ROOT`, `1BIT-DATA`) instead of a RAM image; it unpacks the squashfs, writes `fstab` by label, one grub entry per installed kernel, and leaves `/etc/1bit/live` off the installed disk so `1bit-first-boot.service` writes `/etc/1bit/installed` after making that machine's own SSH keys. Two VM tests, both judged from what the guest writes to its own disk rather than from QEMU: `vmtest-v2.sh` (the appliance, booted through OVMF so the secureboot row answers instead of failing for a QEMU choice) and `vmtest-install-v2.sh` (builds the medium with a test-only unit inside it, boots it with a target disk and a USB stick attached, runs the refusals, installs, then boots the installed disk alone). Bugs this found in the v2 path, all fixed here: the engine package could not load its own libraries (a dev-built stage carried the build tree's RUNPATH and never shipped libhrx/libloomc/libomp); `1bit` needed `libgomp1`/`libpcre2-8-0`, which minbase has not; lemond exits at once without `HOME` in a systemd unit; `ssh.service` enabled while `ssh.socket` was masked starts nothing at all on Ubuntu 26.04 (the socket is what listens, and it carries RequiredBy=ssh.service) - so the socket is what the package enables; minbase has no `sfdisk` (Ubuntu's lives in `fdisk`); `root=` in the live entry has to be the squashfs partition, or `data.mount` fights the initramfs for the data partition; and `1bit-appliance` never shipped the aggregate `/usr/share/licenses/1bit-os` that release-check reads. Known gap, named in `os/v2/README.md` rather than papered over: the ROCm userspace (HIP, rocBLAS, hipBLASLt, the HSA runtime) is the machine's, not the engine package's, and the engine's `Depends` do not name Ubuntu's packages for it yet - gfx1201 support against the distro's ROCm is a hardware question a VM cannot settle. `vmtest-v2` therefore asserts the package's own libraries resolve and lets only that set be missing, which is exactly the check that caught the first bug above. Evidence: `VMTEST-V2 PASS` (seven packages installed, banner, /data, 1bit --help, our kernel's vmlinuz and initrd in /boot, sshd on 22, apt from the repository, Lemonade 401 + GGUF + chat, release-check with no FAIL row but `sealed`, which needs a stage that carries the sealed NPU sets) and `VMTEST-V2-INSTALL PASS` (six refusals, install, installed disk boots alone: INSTALLED mode, first-boot marker and own SSH keys, /data on its own partition, Lemonade 401 + GGUF + chat, selftest INSTALLED). Transcripts: `~/wt/osapt-proto/v2-final.log` and `~/wt/osapt-proto/live-install.log`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
build: rewrite build-machine paths out of the binaries and gate the shipped artifacts
Three related packaging properties, all enforced rather than assumed:
ONEBIT_PATH_MAP_FLAGS(-ffile-prefix-mapfor the source tree and the buildtree) is applied to every engine target and handed to the HRX llama.cpp build through
cmake/hrx.cmake, so a string an assert or a log bakes in reads./buildinstead of themachine that produced it. With
-DONEBIT_GPU_PRIVATEthe add-on checkout is mapped as well.pack_artifact_hygiene()(scripts/pack-elf.sh) reports any shippedprogram or library that still carries a compiler identification section, a path into a build
machine, or an absolute RUNPATH;
scripts/package-linux.shfails the package when it reportsanything.
pack_strip()now also drops.comment(and the gold-version note) afterstrip --strip-unneeded.$ORIGINinstead of whatever the build left, so aprogram loads its libraries from beside itself and from the system only —
ds4-serverincluded,which used to keep an SDK path.
Verified on this box with a Release build of the
onebittarget:The HRX and NPU binaries take the same flags through the same two CMake entry points but were not
built here (they need the toolchain), so they are covered by the gate on the next package run, not
by this transcript.
🤖 Generated with Claude Code