Skip to content

Fix musl loader (libc.so) segfault on old guest kernels#8

Merged
lacraig2 merged 1 commit into
mainfrom
fix/musl-loader-4.10
Jun 9, 2026
Merged

Fix musl loader (libc.so) segfault on old guest kernels#8
lacraig2 merged 1 commit into
mainfrom
fix/musl-loader-4.10

Conversation

@lacraig2

@lacraig2 lacraig2 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Problem

Dynamic guest binaries built/staged by penguin-tools (the drop-in compiler's output, gdbserver, strace, python3, …) segfault at startup on older guest kernels (e.g. Linux 4.10), while working fine on 6.13. This blocks penguin's basic_target on every arch that exercises a 4.10 kernel and would break the tools on any 4.10-based rehost.

Root cause

A core dump of a crashing drop-in faults inside musl's __dls2 at the mov %rdi, ldso(%rip) store — writing the loader's ldso global, which lives in the BSS of libc.so's first RW LOAD segment.

mk-arch-bundle runs patchelf --set-rpath /igloo/dylibs on every staged ELF, including libc.so (the musl loader, aliased ld-musl-<arch>.so.1). Stock musl libc has no rpath, so patchelf must grow .dynamic/.dynstr and appends a fresh PT_LOAD, leaving libc.so with two RW LOAD segments. libc.so is the PT_INTERP the kernel maps for every dynamic binary; older kernels mishandle that extra RW segment and never map the loader's BSS → the first ldso write faults → every dynamic binary segfaults.

Stock musl libc.so has a single RW LOAD and no rpath on all arches; only libc.so was mangled (other dylibs/tools already carried /nix/store rpaths that patchelf rewrites in place).

Fix

Skip patchelf --set-rpath on libc.so. The loader needs no rpath — NEEDED libs resolve via each program's own RUNPATH=/igloo/dylibs — so its stock, single-RW-LOAD layout is preserved and old kernels can map it.

Verified: stock cross-musl libc.so is LOADs / RW=1 / rpath='' on every arch; the appended segment came solely from the patchelf rpath edit.

…old kernels

mk-arch-bundle ran 'patchelf --set-rpath /igloo/dylibs' on every staged
ELF, including libc.so (the musl loader, aliased ld-musl-<arch>.so.1).
musl libc ships with no rpath, so patchelf had to grow .dynamic/.dynstr
and appended a fresh PT_LOAD segment. The loader is the PT_INTERP the
kernel maps for every dynamic guest binary, and older guest kernels
(e.g. Linux 4.10) mishandle that extra RW segment: they never map the
loader's BSS, so the first write in musl's __dls2 (storing the 'ldso'
global) faults and every dynamic binary segfaults at startup. The loader
needs no rpath of its own (NEEDED libs resolve via each program's
RUNPATH=/igloo/dylibs), so leave libc.so's stock single-RW-LOAD layout
untouched. Other dylibs/tools already had /nix/store rpaths that patchelf
rewrites in place, so they are unaffected.
@lacraig2 lacraig2 merged commit e37010d into main Jun 9, 2026
1 check passed
@lacraig2 lacraig2 deleted the fix/musl-loader-4.10 branch June 9, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant