Skip to content

fix(ddi): restore the setuid/setgid bits Flatcar ships on sudo, su, passwd, mount, … - #255

Merged
jeefy merged 2 commits into
projectbluefin:mainfrom
jeefy:fix/sudo-setuid
Sep 26, 2026
Merged

jeefy merged 2 commits into
projectbluefin:mainfrom
jeefy:fix/sudo-setuid

Conversation

@jeefy

@jeefy jeefy commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Branch: fix/sudo-setuid (OS payload build; independent of the installer fixes).

Problem

On the installed image:

$ ls -l /usr/bin/sudo
-rwxr-xr-x 1 root root … /usr/bin/sudo
$ sudo -i
sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

Observed in QEMU on the installed installer-v26.08.0 image while integrating with the jeefy/booty PXE server (operator account provisioned via tmpfiles.extra, then sudo refused). su, passwd, mount, umount, pkexec, newuidmap/newgidmap (rootless podman) and the rest of the list below are affected the same way.

Root cause

Flatcar ships the bits (-rwsr-xr-x root/root ./usr/bin/sudo in flatcar-container.tar.gz for 4593.2.5) and tar — running as root inside the BuildStream sandbox — extracts them intact. They are lost one step later: BuildStream artifacts record a single executable bit per file (REAPI FileNode.is_executable; buildbox-fuse keeps the full mode in memory and stores only that bit). When flatcar-usr.bst's install root is captured, every special bit is dropped, so /layer in the bluefin-server-ddi.bst build has sudo at 0755, and mkfs.xfs -p /layer faithfully copies the mode it finds. This is the same limitation freedesktop-sdk works around with chmod 4755 …/sudo in an initial-script run at image assembly (components/sudo.bst), which is why the FSDK-based payload never hit it and the Flatcar cutover (#132) did.

The full list dropped from the 4593.2.5 archive (19 files): setuid — sudo su passwd gpasswd newgrp chsh chfn mount umount mount.nfs ksu pkexec newuidmap newgidmap polkit-agent-helper-1; setgid shadow — unix_chkpwd expiry chage; and dbus-daemon-launch-helper (---s--x---, which is not even executable for its owner and so was staged as 0644).

Fix

Apply the modes at the only point where they can stick — the script element that writes the image — without hardcoding a binary list:

  1. flatcar-usr.bst: while the archive modes are intact — after the element's own rm block, so the list never names a file this element removes — record find … -type f -perm /6000 -printf '%m usr/%P\n' into usr/lib/bluefin-server/setuid-modes (octal mode, install-root-relative path, sorted). The build fails if the list does not contain usr/bin/sudo — i.e. if an extraction ever comes out without its special bits (non-root sandbox, tar flags), we refuse to ship an unusable sudo rather than record an empty list. components/findutils.bst added to build-depends (already used by flatcar-kernel.bst).
  2. bluefin-server-ddi.bst: after depmod and before mkfs.xfs, chmod each recorded mode onto /layer/<path> (failing with a stale-manifest message if a listed path is not in /layer), then fail if the list is missing/empty or /usr/bin/sudo is not 4755; log the restored list and ls -l sudo. A mode without an owner-read bit gets u+r added (dbus-daemon-launch-helper ships as 4110, ---s--x---, and lands as 4510): mkfs.xfs -p has to open every file it copies and buildbox-fuse enforces the mode bits itself — the round-1 CI build failed with mkfs.xfs: cannot open /layer/usr/libexec/dbus-daemon-launch-helper: Permission denied (reproduced locally with a 4110 file). Root reads such a file regardless on a real system.
  3. bluefin-server-ddi.bst: after mkfs.xfs, assert xfs_db -r -c 'path /usr/bin/sudo' -c 'print core.mode' reports 0104755 in the written image. BuildStream only surfaces element output on failure, so this check — not a ls -l — is what makes a green build proof that the mode reached the DDI.

A Flatcar bump that adds or drops a setuid helper is picked up automatically; just validate graph is unchanged apart from the findutils build-dep.

Known remaining flattening (unchanged by this PR): ownership is also lost in the artifact capture, so the three setgid-shadow helpers come back setgid root. With the image's 0600 root:root /etc/shadow that makes no practical difference, but it is why the manifest records modes, not owners. Likewise /usr/libexec/dbus-daemon-launch-helper comes back 4510 root:root instead of ---s--x--- root:messagebus (owner read added so mkfs.xfs can copy it, see fix step 2), so the messagebus user still cannot execute it and non-systemd (classic) D-Bus activation remains unavailable — unchanged from before, when it was staged 0644.

Verification

  • tests/unit/test_flatcar_usr.py — the executed contract now stages sudo/su/umount 4755 and unix_chkpwd 2755 mocks (umount sorts after sudo, so the manifest's last line is not sudo and the loop's last iteration runs under set -e as in production) and asserts the recorded list byte-for-byte and that it names none of the removed files; a test runs the element's install-commands with a 0755 sudo and asserts the build refuses (usr/bin/sudo is not setuid); a test asserts the recording follows the last rm.
  • tests/unit/test_setuid_modes.py (new) — pins both halves to the same path and ordering (depmod → restore → mkfs.xfs → xfs_db check → zstd), runs the DDI restore block against a mock /layer (4755/2755 restored, 4110 → 4510 with a NOTE, unrelated files untouched, restored list logged), and checks the missing-list, empty-list, stale-entry and sudo-not-4755 failures.
  • python3 -m pytest tests/unit -q — 353 passed, 1 xfailed (pre-existing).
  • bash -n + shellcheck -S warning on both extracted scripts (with %{…} substituted) — clean.
  • python3 .github/scripts/docs-checks.py, check-flatcar-version.py (no new literals) and the other check-* scripts — pass.
  • xfs_db invocation verified locally against a mkfs.xfs -p image of a mock tree: core.mode = 0104755 for a 4755 file, 0100755 for 0755, and a 4110 file makes mkfs.xfs fail with the same Permission denied CI hit.
  • Not run: just validate / just build-ddi (no local BuildStream), so the built artefact has not been inspected locally; the round-1 CI build reached mkfs.xfs (i.e. the restore loop and the stat gate passed on the real /layer) and failed only on the unreadable 4110 file. Root cause and fix are established from the pipeline: the tar listing (bits present), BuildStream's _casbaseddirectory.py/cascache.py (only is_executable captured; checkout chmods 0755), buildbox-fuse buildbox.cc (full mode kept in memory, only the executable bit persisted), and xfsprogs mkfs/proto.c (directory population copies st_mode verbatim, so a chmod in the DDI script reaches the image). A ls -l /usr/bin/sudo on the next CI DDI is the remaining proof.

Docs: ddi-installer-build.md gains a Common rationalizations row ("chmod 4755 in install-commands makes the file setuid in the image" → no, and where it must be done instead).

Review round 2

Applied from the line-by-line review and the round-1 CI failure:

  • CI failure fixed: mkfs.xfs: cannot open /layer/usr/libexec/dbus-daemon-launch-helper: Permission denied (run 36259023853) — the restored 4110 mode has no read bit and buildbox-fuse enforces modes itself. The restore loop now adds u+r to any mode lacking owner read (4110 → 4510) and logs a NOTE; reproduced and verified locally with mkfs.xfs -p.
  • Manifest recorded after the rm block in flatcar-usr.bst, so a future Flatcar release shipping a setuid file this element removes cannot produce a stale entry.
  • Post-mkfs.xfs xfs_db … print core.mode assertion in bluefin-server-ddi.bst so CI proves the image, not just /layer; the restored list and ls -l sudo are echoed for failing-run context.
  • test_flatcar_usr mock gains umount 4755 so sudo is not the manifest's last line.
  • Nits: [ … ] && X as last loop command → if … fi; [ -f ] check with a stale-manifest message before chmod; whitespace/newline constraint on paths documented; ddi-installer-build.md row explains why a boot-time tmpfiles z line is the wrong place even though the rootfs is rw.
  • Skipped: begin/end marker comments around the restore block (the test slices on the assignment and the ls -l sudo line, which are the block's natural boundaries).
  • New CI build is running.

jeefy and others added 2 commits September 26, 2026 13:21
…asswd

On the installed image /usr/bin/sudo is -rwxr-xr-x and sudo refuses to run:

  sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

flatcar-container.tar.gz carries sudo as -rwsr-xr-x and tar (running as
root in the sandbox) extracts it that way, but BuildStream artifacts record
a single executable bit per file (REAPI FileNode.is_executable; buildbox-fuse
keeps full modes in memory and stores only that bit), so when
flatcar-usr.bst's install root is captured every special bit is dropped and
/layer in the DDI build has 0755. mkfs.xfs -p copies the mode it finds, so
the image ships 19 binaries without their bits: sudo, su, passwd, gpasswd,
newgrp, chsh, chfn, mount, umount, mount.nfs, ksu, pkexec, newuidmap,
newgidmap, polkit-agent-helper-1 (setuid) and unix_chkpwd, expiry, chage
(setgid) plus dbus-daemon-launch-helper (---s--x---, staged as 0644).
freedesktop-sdk hits the same limitation and re-applies `chmod 4755 sudo`
from an initial-script at image assembly.

Fix it at the only point where the bits can stick: flatcar-usr.bst records
`find -perm /6000` (octal mode, install-root-relative path) into
usr/lib/bluefin-server/setuid-modes while the archive modes are still
intact -- after its own rm block, so the list never names a file the
element removes -- and refuses to build if the list does not contain
usr/bin/sudo (an extraction that lost the bits). bluefin-server-ddi.bst
re-applies the list to /layer after depmod and before mkfs.xfs, failing
with a stale-manifest message if a listed path is not in /layer, if the
list is missing/empty, or if sudo does not end up 4755. A mode without an
owner-read bit gets u+r added (dbus-daemon-launch-helper ships as 4110,
---s--x---, and lands as 4510): mkfs.xfs -p has to open every file it
copies and buildbox-fuse enforces the mode bits itself, so the first CI
build of this change failed with "mkfs.xfs: cannot open
/layer/usr/libexec/dbus-daemon-launch-helper: Permission denied"; root
reads such a file regardless on a real system. It logs the
restored list and `ls -l sudo`, and after mkfs.xfs asserts with
`xfs_db -r -c 'path /usr/bin/sudo' -c 'print core.mode'` that the written
image carries 0104755, so a green build is evidence about the DDI itself
(BuildStream only surfaces element output on failure). No per-binary
hardcoding; a Flatcar bump that adds or drops a setuid helper is picked up
automatically.

Ownership is still flattened to root:root by the same artifact capture, so
the three setgid-shadow helpers come back setgid root. That matches the
image's 0600 root-only /etc/shadow and is unchanged by this fix.

test_flatcar_usr.py extends the executed contract with setuid/setgid mocks
(including umount, so sudo is not the manifest's last line and the loop's
last iteration is exercised under set -e) and asserts the recorded list
names none of the removed files, plus a run that must fail when sudo comes
out 0755 and a check that the recording follows the last rm.
test_setuid_modes.py pins the two halves to the same path and the xfs_db
check to after mkfs.xfs, runs the DDI restore block against a mock /layer
(every mode restored, 4110 -> 4510 with a NOTE, others untouched,
restored list logged) and checks the missing/empty-list, stale-entry and
sudo-not-4755 failures.

Assisted-by: Claude Fable 5.1 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…itten

Add a Common rationalizations row: BuildStream artifacts keep one
executable bit per file, so chmod in an element's install-commands cannot
make a file setuid in the image; point at setuid-modes, the DDI script and
its post-mkfs xfs_db assertion, and say why a boot-time tmpfiles `z` line
is the wrong place even though the rootfs is mounted rw.

Assisted-by: Claude Fable 5.1 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read elements/oci/bluefin-server-ddi.bst and elements/flatcar/flatcar-usr.bst at 5642df3 in full. Looks correct to me, and the design is self-proving.

  • bluefin-server-ddi.bst runs under set -euo pipefail (bst:25), so the (( 8#${mode} & 8#400 )) arithmetic is valid, and a failing xfs_db aborts the build before the case rather than matching the * arm.
  • findutils.bst is already a build-dep of the DDI element (bst:12) and is newly added to flatcar-usr.bst, so find -printf '%m usr/%P' resolves in both sandboxes; %P is relative to the find start point, which is exactly what the usr/ prefix assumes.
  • The restore loop [ -f /layer/${path} ] guard plus recording after the rm block in flatcar-usr.bst closes the stale-manifest window in both directions.
  • The post-mkfs.xfs xfs_db -r -c 'path /usr/bin/sudo' -c 'print core.mode' assertion means a green build job is evidence the mode reached the image, not just /layer — and build (46m) and installer-test / Install and boot in QEMU both pass on this head.

Ownership flattening (setgid-shadow helpers coming back root:root, dbus-daemon-launch-helper at 4510) is called out honestly in the body and is not a regression from the current 0644/0755 state.

Confidence: 5/5 (safe)

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@jeefy
jeefy merged commit 31b4094 into projectbluefin:main Sep 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant