Skip to content

[infra] - Select the bwrap AppArmor CI proof from Ubuntu VERSION_ID #1644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cgfixit merged 2 commits into from
Oct 13, 2026
Merged

[infra] - Select the bwrap AppArmor CI proof from Ubuntu VERSION_ID #1644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7061008
[infra] - Choose the bwrap AppArmor proof from Ubuntu VERSION_ID
cursoragent Oct 13, 2026
87b6b32
Merge remote-tracking branch 'origin/main' into agent/bwrap-apparmor-…
cgfixit Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 30 additions & 235 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,13 +519,12 @@ jobs:
$env:USERPROFILE = $OriginalUserProfile
}

# Non-required Ubuntu 26.04 probe for #1626. ubuntu-latest is still 24.04, so
# the required test leg's name and scripts stay put. continue-on-error keeps
# a red finding from failing the workflow. This job only records host policy
# and the same bwrap command the 24.04 leg runs; it does not write the
# userns sysctl.
ubuntu-26-04-bwrap-diagnose:
name: ubuntu-26-04-bwrap-diagnose
# Same script as the test job's Linux bwrap step (#1626, measured on
# #1641 run 38352458013). continue-on-error keeps the job off the required
# check set. The assertions are the strict 26.04+ branch of that script.
# This job does not write kernel.apparmor_restrict_unprivileged_userns.
ubuntu-26-04-bwrap:
name: ubuntu-26.04 bwrap
runs-on: ubuntu-26.04
continue-on-error: true
timeout-minutes: 15
Expand All @@ -536,163 +535,9 @@ jobs:
with:
persist-credentials: false

- name: Install distro bubblewrap (26.04 has no noble pin)
- name: Version-aware bwrap AppArmor proof
shell: bash
run: |
set -u
sudo apt-get update
echo "--- apt-cache policy bubblewrap ---"
apt-cache policy bubblewrap
if apt-cache show 'bubblewrap=0.9.0-1ubuntu0.3' >/dev/null 2>&1; then
echo "noble_pin=present"
else
echo "noble_pin=absent bubblewrap=0.9.0-1ubuntu0.3"
fi
sudo apt-get install --yes --no-install-recommends bubblewrap apparmor-utils
echo "bwrap_path=$(command -v bwrap)"
bwrap --version
dpkg-query -W -f='installed_bubblewrap=${Version}\n' bubblewrap
dpkg-query -W -f='installed_${Package}=${Version}\n' apparmor apparmor-utils apparmor-profiles 2>/dev/null || true

- name: Record stock AppArmor, then the cyclaw-bwrap load
if: ${{ !cancelled() }}
shell: bash
run: |
set -u
set +e
set +o pipefail

section() {
printf '\n===== %s =====\n' "$1"
}

section "host"
awk -F= '/^(PRETTY_NAME|VERSION_ID|VERSION_CODENAME)=/{gsub(/"/, "", $2); print}' /etc/os-release
uname -a
apparmor_parser --version 2>&1 || true
aa-enabled 2>&1 || true
aa-status --version 2>&1 || true

section "sysctl"
for key in \
kernel.apparmor_restrict_unprivileged_userns \
kernel.apparmor_restrict_unprivileged_unconfined \
kernel.unprivileged_userns_clone \
user.max_user_namespaces
do
val="$(sysctl -n "$key" 2>/dev/null || echo missing)"
printf '%s=%s\n' "$key" "$val"
done

section "bwrap binary"
ls -l /usr/bin/bwrap || true
getcap /usr/bin/bwrap 2>/dev/null || true
bwrap --help 2>&1 | grep -E -n 'unshare-all|unshare-net|share-net|cap-drop' || true

section "apparmor paths"
sudo find /etc/apparmor.d /usr/share/apparmor \
\( -iname '*bwrap*' -o -iname '*userns*' \) -print 2>/dev/null | sort || true

section "profile texts"
while IFS= read -r path; do
[ -n "$path" ] || continue
printf '----- %s -----\n' "$path"
sudo ls -l "$path" || true
if [ -L "$path" ]; then
printf 'symlink_target=%s\n' "$(sudo readlink -f "$path" 2>/dev/null || true)"
fi
sudo awk 'NR<=260 {print}' "$path" || true
done < <(sudo find /etc/apparmor.d /usr/share/apparmor \
\( -iname '*bwrap*' -o -iname '*userns*' \) -print 2>/dev/null | sort)

loaded_profiles() {
section "loaded profiles $1"
sudo awk 'BEGIN{IGNORECASE=1} /bwrap|userns/ {print}' \
/sys/kernel/security/apparmor/profiles 2>/dev/null || true
sudo aa-status 2>&1 | awk 'BEGIN{IGNORECASE=1} /bwrap|userns|profiles are|in enforce|in complain/ {print}' || true
}

dump_denials() {
section "kernel denials $1"
sudo dmesg -T 2>/dev/null | awk 'BEGIN{IGNORECASE=1} /apparmor|bwrap/ {print}' | tail -n 160 || true
sudo journalctl -k --no-pager -n 300 2>/dev/null | awk 'BEGIN{IGNORECASE=1} /apparmor|bwrap/ {print}' | tail -n 160 || true
}

probe() {
name="$1"
shift
err="${RUNNER_TEMP}/bwrap-probe-${name}.err"
out="${RUNNER_TEMP}/bwrap-probe-${name}.out"
section "probe ${name}"
printf 'cmd:'
printf ' %q' "$@"
printf '\n'
"$@" >"$out" 2>"$err"
printf 'exit:%s\n' "$?"
printf 'stdout:\n'
cat "$out" || true
printf 'stderr:\n'
cat "$err" || true
}

loaded_profiles "before"
printf 'runner_attr='
tr '\0' ' ' < /proc/self/attr/current 2>/dev/null || true
printf '\n'
sudo dmesg -c >/dev/null 2>&1 || true

probe stock-full bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
probe stock-no-capdrop bwrap --die-with-parent --new-session --unshare-all \
--ro-bind / / --proc /proc --dev /dev /bin/true
probe stock-share-net bwrap --die-with-parent --new-session --unshare-all --share-net --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
probe stock-user-net bwrap --die-with-parent --unshare-user --unshare-net --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
probe stock-user-only bwrap --die-with-parent --unshare-user --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true

section "child label under stock"
bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev \
/bin/sh -c 'printf "child_attr="; tr "\0" " " < /proc/self/attr/current; printf "\n"; if [ -r /proc/self/attr/apparmor/current ]; then printf "child_apparmor="; tr "\0" " " < /proc/self/attr/apparmor/current; printf "\n"; fi; grep -E "^Cap(Eff|Prm|Bnd|Inh):" /proc/self/status'
printf 'child_label_exit:%s\n' "$?"

section "unprofiled copy"
copy="${RUNNER_TEMP}/bwrap-unprofiled"
cp -a /usr/bin/bwrap "$copy"
chmod 755 "$copy"
probe unprofiled-copy "$copy" --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
if command -v unshare >/dev/null 2>&1; then
probe unshare-user-net unshare --user --map-root-user --net /bin/true
else
echo "unshare_missing=1"
fi
dump_denials "after-stock"

section "load cyclaw-bwrap"
sudo install -m 0644 deploy/apparmor/cyclaw-bwrap /etc/apparmor.d/cyclaw-bwrap
parser_out="${RUNNER_TEMP}/apparmor-parser.out"
sudo apparmor_parser -r /etc/apparmor.d/cyclaw-bwrap 2>&1 | tee "$parser_out"
printf 'apparmor_parser_exit:%s\n' "${PIPESTATUS[0]}"
printf 'parser_output:\n'
cat "$parser_out" || true
loaded_profiles "after-cyclaw-load"
sudo dmesg -c >/dev/null 2>&1 || true
probe after-cyclaw-full bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
dump_denials "after-cyclaw"

section "unload cyclaw-bwrap"
sudo apparmor_parser -R /etc/apparmor.d/cyclaw-bwrap
printf 'apparmor_parser_remove_exit:%s\n' "$?"
sudo rm -f /etc/apparmor.d/cyclaw-bwrap
loaded_profiles "after-unload"
probe after-unload-full bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true

echo "DIAGNOSIS_COMPLETE"
run: bash scripts/ci/bwrap-apparmor-check.sh

test:
name: ${{ matrix.os }}
Expand All @@ -713,17 +558,22 @@ jobs:
# because torch==2.13.0+cpu 404s on https://download.pytorch.org/whl/cpu
# for macOS -- confirmed on this matrix leg's first run, and documented
# in setup-guide.md.
# Native bwrap confinement is proven on both Linux legs. ubuntu-latest
# (24.04) is the required check: stock AppArmor there refuses bwrap's
# user namespaces, so the leg proves the refusal, loads the bwrap-scoped
# profile deploy/apparmor/cyclaw-bwrap, then proves confinement. It never
# relaxes the global userns sysctl. ubuntu-22.04 proves confinement under
# stock policy. Each check is named for its real runner.
# GitHub moves ubuntu-latest to Ubuntu 26 starting 2026-10-19. If the
# stock refusal changes there, the refusal step below fails loudly on
# purpose; re-check it then rather than pinning (a pin renames the check).
# Native bwrap confinement is proven on both Linux legs. Check names
# stay ubuntu-latest and ubuntu-22.04. Which proof runs is VERSION_ID
# from /etc/os-release inside scripts/ci/bwrap-apparmor-check.sh, shared
# with the non-required ubuntu-26.04 job:
# 22.04 keeps the jammy pin and stock confinement.
# 24.04 keeps the noble pin, the stock refusal, cyclaw-bwrap, then
# confinement. It never writes the userns sysctl.
# 26.04+ does not load cyclaw-bwrap and does not install the noble
# pin. It requires the sysctl to read 1, Ubuntu's bwrap profile in
# enforce mode, a non-/usr/bin copy to be refused, and /usr/bin/bwrap
# confinement to pass.
# Any other version fails the job.
# GitHub moves ubuntu-latest to Ubuntu 26 between 2026-10-19 and
# 2026-11-19. The required check name stays ubuntu-latest.
os: [ubuntu-latest, ubuntu-22.04, windows-latest, macos-latest]
# All four legs (Linux 24.04 and 22.04, Windows, macOS) are release gates.
# All four legs (ubuntu-latest, ubuntu-22.04, Windows, macOS) are release gates.
# Platform-specific skips are reported below with `pytest -rs`; a failing
# Windows result is not masked.

Expand All @@ -732,70 +582,15 @@ jobs:
with:
persist-credentials: false

- name: Install Linux verification filesystem/PID/network sandbox
- name: Version-aware bwrap AppArmor proof
# 22.04, 24.04, and 26.04+ share scripts/ci/bwrap-apparmor-check.sh with
# the non-required ubuntu-26.04 job. The script never writes
# kernel.apparmor_restrict_unprivileged_userns. 24.04 still proves the
# stock refusal, loads deploy/apparmor/cyclaw-bwrap, then proves
# confinement. 26.04+ does not load that profile.
if: runner.os == 'Linux'
shell: bash
run: |
# Version pin (apt has no digest pin). ubuntu-latest is 24.04 until
# GitHub's 2026-10-19 move to 26; re-check the noble pin then.
case "${{ matrix.os }}" in
ubuntu-22.04) ver='0.6.1-1ubuntu0.3' ;;
*) ver='0.9.0-1ubuntu0.3' ;; # ubuntu-latest (24.04 noble)
esac
sudo apt-get update
sudo apt-get install --yes --no-install-recommends "bubblewrap=${ver}"
bwrap --version
test "$(bwrap --version)" = "bubblewrap ${ver%%-*}"

- name: Prove stock AppArmor refuses bwrap (24.04 leg)
# Stock 24.04 sets kernel.apparmor_restrict_unprivileged_userns=1, so
# bwrap cannot create its namespaces until a profile grants it userns.
# This is what a stock 24.04 user sees, and CyClaw must fail closed on it.
# If the probe unexpectedly succeeds, the profile below proves nothing.
# Both preconditions are asserted, so neither a relaxed image nor an
# unrelated bwrap failure can pass for "refused".
if: matrix.os == 'ubuntu-latest'
shell: bash
run: |
restrict="$(sysctl -n kernel.apparmor_restrict_unprivileged_userns 2>/dev/null || echo missing)"
echo "kernel.apparmor_restrict_unprivileged_userns=${restrict}"
if [ "${restrict}" != "1" ]; then
echo "::error::expected stock kernel.apparmor_restrict_unprivileged_userns=1, got '${restrict}'"
exit 1
fi
probe_err="${RUNNER_TEMP}/bwrap-stock-probe.err"
if bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true 2>"${probe_err}"; then
echo "::error::bwrap succeeded under stock AppArmor; expected a refusal before the cyclaw-bwrap profile loads"
exit 1
fi
cat "${probe_err}" >&2
if ! grep -Eqi 'permission denied|operation not permitted|no permissions to create new namespace|user namespace' "${probe_err}"; then
echo "::error::bwrap failed, but not with a userns/permission denial; refusing to count it as the stock AppArmor refusal"
exit 1
fi
echo "Stock AppArmor refused bwrap with a userns/permission denial, as expected."

- name: Load the bwrap-scoped AppArmor profile (24.04 leg)
# Grants userns to /usr/bin/bwrap only (deploy/apparmor/cyclaw-bwrap).
# The global sysctl is left at its stock value. These are the exact
# load commands setup-guide.md gives stock 24.04 users, so CI proves
# the documented path, not a CI-only variant.
if: matrix.os == 'ubuntu-latest'
shell: bash
run: |
sudo install -m 0644 deploy/apparmor/cyclaw-bwrap /etc/apparmor.d/cyclaw-bwrap
sudo apparmor_parser -r /etc/apparmor.d/cyclaw-bwrap

- name: Prove bwrap confinement (both Linux legs)
# 22.04 under stock policy; 24.04 under the cyclaw-bwrap profile only.
# The native gate below must also fail if this runner cannot establish
# the required confinement.
if: runner.os == 'Linux'
shell: bash
run: |
bwrap --die-with-parent --new-session --unshare-all --cap-drop ALL \
--ro-bind / / --proc /proc --dev /dev /bin/true
run: bash scripts/ci/bwrap-apparmor-check.sh

- name: macOS install-script smoke test (layout + shim + profile integration)
if: runner.os == 'macOS'
Expand Down
37 changes: 24 additions & 13 deletions deploy/apparmor/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,16 @@ sudo apparmor_parser -R deploy/apparmor/cyclaw-gate
docker compose up
```

## `cyclaw-bwrap`: user namespaces for the agentic sandbox (Ubuntu 24.04+)
## `cyclaw-bwrap`: user namespaces for the agentic sandbox (Ubuntu 24.04)

**Status: opt-in host profile, separate from `cyclaw-gate`.** Stock Ubuntu
24.04 and newer set `kernel.apparmor_restrict_unprivileged_userns=1`, so
bubblewrap cannot create its namespaces and
`scripts/verify_agentic_sandbox.py` refuses (fails closed). The
`deploy/apparmor/cyclaw-bwrap` profile lifts that block for one binary only.
Never set the global sysctl to `0` instead: that removes the restriction for
every program on the host.
24.04 sets `kernel.apparmor_restrict_unprivileged_userns=1`, so bubblewrap
cannot create its namespaces and `scripts/verify_agentic_sandbox.py` refuses
(fails closed). The `deploy/apparmor/cyclaw-bwrap` profile lifts that block
for one binary only. Never set the global sysctl to `0` instead: that removes
the restriction for every program on the host. Ubuntu 26.04 and newer already
ship an enforce-mode `bwrap` profile; do not load this one beside it (see
below).

What the profile grants:

Expand Down Expand Up @@ -113,12 +114,22 @@ To load it at boot, install it to `/etc/apparmor.d/cyclaw-bwrap` first; the
**Why not Ubuntu's `bwrap-userns-restrict`.** AppArmor's extra-profiles ship a
stricter option that confines `bwrap` and stacks every child under
`unpriv_bwrap` (no capabilities inside the namespace). CyClaw defaults to the
one-rule profile because the stricter one is not userns-only (it also allows
`capability`, `mount`, `ptrace`, `dbus`, `network`, and `rwlkm` on `/**`, a
much larger policy that must match the host ABI — upstream declares
`abi <abi/5.0>`), its main gain is what `--cap-drop ALL` already provides
(what it adds is protection against *other* bwrap callers on the host), and CI
proves only the one-rule profile on `ubuntu-latest`.
one-rule profile on Ubuntu 24.04 because the stricter one is not userns-only
(it also allows `capability`, `mount`, `ptrace`, `dbus`, `network`, and
`rwlkm` on `/**`, a much larger policy that must match the host ABI — upstream
declares `abi <abi/5.0>`), its main gain is what `--cap-drop ALL` already
provides (what it adds is protection against *other* bwrap callers on the
host), and CI on Ubuntu 24.04 proves the one-rule profile.

**Ubuntu 26.04 and newer.** Those images already load
`/etc/apparmor.d/bwrap-userns-restrict` (`profile bwrap /usr/bin/bwrap`) in
enforce mode while the sysctl stays 1. `/usr/bin/bwrap` passes confinement.
A copy at another path is refused with
`bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. CI on those
versions does not load `cyclaw-bwrap`. Loading it beside the stock profile
makes `/usr/bin/bwrap` fail the same way. Do not install the 24.04 package pin
`bubblewrap=0.9.0-1ubuntu0.3` there; use the candidate in `apt-cache policy
bubblewrap`'s version table.

**Residual risk.** With `cyclaw-bwrap` loaded, any local user can run
`/usr/bin/bwrap` to get a user namespace — acceptable on a single-user CyClaw
Expand Down
11 changes: 9 additions & 2 deletions deploy/apparmor/cyclaw-bwrap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# CyClaw: let bubblewrap create user namespaces on hosts that enforce
# kernel.apparmor_restrict_unprivileged_userns=1 (stock Ubuntu 24.04 and
# newer) WITHOUT turning that restriction off for the whole system.
# kernel.apparmor_restrict_unprivileged_userns=1 (stock Ubuntu 24.04)
# WITHOUT turning that restriction off for the whole system. Ubuntu 26.04
# and newer already have an enforce-mode bwrap profile; do not load this
# one beside it (see the note below).
# Never use `sysctl kernel.apparmor_restrict_unprivileged_userns=0` instead.
#
# Scope: attaches to exactly /usr/bin/bwrap (the apt `bubblewrap` path). No
Expand All @@ -17,6 +19,11 @@
# restriction. Fine for a single-user CyClaw workstation; on shared or
# multi-user hosts use Ubuntu's stricter `bwrap-userns-restrict` instead.
#
# Ubuntu 26.04 and newer already load that stock profile in enforce mode
# (profile bwrap for /usr/bin/bwrap, sysctl left at 1). Do not load this
# profile beside it: both attach to /usr/bin/bwrap, and confinement then fails
# with "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted".
#
# Why not `bwrap-userns-restrict` (apparmor extra-profiles) by default: it is
# not userns-only (its bwrap profile allows capability, mount, ptrace, dbus,
# network and rwlkm on /**), it must match the host's AppArmor ABI, and its
Expand Down
Loading
Loading