Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,17 @@ and the QEMU command line; `reims-ctl command mac` (inside the container) prints
container's port 22 to the guest, and Docker publishes that as `127.0.0.1:$MACBOX_SSH_PORT`.
reims-ctl's own forward would listen on the container's loopback, which Docker can't publish.
- `-set drive.disk.discard=unmap` (guest TRIM frees space) and `cache=$MACBOX_DISK_CACHE`.
- `--video-acceleration=false` (no VideoToolbox/M2 scaler devices), a borderless window,
- `--video-acceleration=false`, a borderless window,
`REIMS_VGPU_STATS=1` (a line of frame statistics a second in `logs/qemu.log`).
- Before each start: `WAYLAND_DISPLAY` (sway's, or the host's), and the GPU tuning:
`M2V_LIVE=$MACBOX_HOME/run/live.conf` with `host_rt_in_place=1`,
`REIMS_VGPU_PRIVATE_SKIP_PAGE_UP=1`, `M2V_ARL_FACTS` on an Arrow Lake GPU, and with
Coherent (`MACBOX_COHERENT`, on by default with an Arrow Lake GPU) the census, Coherent and the
switches measured with it, in `live.conf` and Reims's environment ([performance.md](performance.md)).
The disk's options in `extra_args` are written again, so a changed `MACBOX_DISK_CACHE` applies.
- With `MACBOX_VIDEO=on`, Reims's VideoToolbox (`apple-videotoolbox-pci`, slot 8) and M2 scaler
(`apple-m2scaler-pci`, slot 9) devices in `extra_args`, where reims-ctl's newer releases put them.
The guest drives them with reims-util's x86-64 ports of Apple's drivers (`macbox video-kexts`).

The system disk starts on SATA, because macOS's installer may not find a disk on NVMe, and moves to
NVMe with `macbox disk nvme` once macOS is installed.
Expand Down
61 changes: 61 additions & 0 deletions docs/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,67 @@ What to know before relying on it:
boot with it (2026-11-14), streamed iPhone screens refused 11 arms, all counted as reads, and the
write-loss gates stayed clean over 2,060 surface reports.

## Hardware video encoding (`MACBOX_VIDEO`)

Without Reims's video devices, macOS in the VM has only VideoToolbox's software H.264 encoder.
Streaming an iPhone 17 Pro simulator at its full 1206x2622 (T3 Code's Device panel) keeps
`VTEncoderXPCService` at about 610 MB and, while the screen scrolls, 75-100% of a core. With
`MACBOX_VIDEO=on` and the video kexts, VideoToolbox picks Reims's hardware encoder, which encodes on
the host's GPU through VA-API (Intel iHD on Arrow Lake: H.264 and HEVC; no AV1 encoder is exposed,
and macOS's VideoToolbox has none either). Measured on Tahoe on 2026-11-13, one boot, the same
simulator, A/B in two rounds:

| | Software encoder | Hardware encoder |
|---|---|---|
| `VTEncoderXPCService` memory | 613-620 MB | 9-18 MB |
| its CPU while scrolling | 73-75% of a core | ~0% |
| encode per frame | 82 ms (p50, scrolling) | 8-9 ms, of which ~7 on the host's GPU |
| host GPU video engine | | 4-10% |

Two things decide whether the hardware path is cheap. VideoToolbox converts anything but 4:2:0
NV12 (`420v`) with 64-byte row and plane alignment on the guest's GPU first, a Metal pass that
waits behind the simulators' own rendering (57 ms against 8 ms), so the frames given to it should
already be NV12 laid out that way. And the bitrate it reaches is close to the target; quality at
the same bitrate, measured on a recorded scroll at full size (SSIM): H.264 0.982 at 4 Mbit/s and
0.998 at 12 Mbit/s; HEVC 0.998 and 0.9996.

### Where a streamed frame's time goes

Measured on 2026-11-13 with the hardware encoder, an iPhone 17 Pro at 1206x2622:

| Stage | Time |
|---|---|
| reading the 12.7 MB framebuffer IOSurface (memcpy) | 5 ms cold, 1.4 ms warm |
| `IOSurfaceLock` before that read | 0.01 ms on a still screen; while the screen changes, until the GPU has finished the frame the simulator just announced (GlassBench ~10 ms, Safari scrolling 42-63 ms) |
| BGRA to NV12 with vImage, including that wait | 18 ms p50 (GlassBench) |
| hardware encode | 7.4 ms p50, 8.4 ms p95 |
| decode in the browser (WebCodecs) | 1.4-5 ms |

The lock's wait is not a copy back from the GPU: an idle simulator's surface locks at once while the GPU is
busy, and the wait follows the simulator's own frame rate. So the stream gets what the simulator draws: GlassBench
45-50 frames a second at `--fps 60` (439 of 456 frames new pictures), Safari's JS-driven pages about 20 (the
compositor redraws 50-70 times a second, mostly with the same pixels). Faster panels come from a faster
simulator, which is the GPU: with Coherent's surfaces off the render engine runs 86% busy with one GlassBench
simulator. Reading with `kIOSurfaceLockAvoidSync` returns stale or torn pictures, and overlapping several synced
reads gains nothing.

With Coherent's surfaces on (2026-11-14) the simulator outran the stream: GlassBench drew 87 frames a second
(p50 10.8 ms) and the stream carried 50-52. The capture's fps throttle measures each frame against the last one
it sent, so with frames 11.5 ms apart at `--fps 60` every other one fell inside the interval. Pacing on a grid
of the interval instead passes one frame per 16.7 ms slot, with a quarter interval's tolerance for jitter.
Measured through Simlock's gateway, the route T3 Code's panel streams from, with one other simulator running:

| | Throttle on the last frame | Throttle on a grid |
|---|---|---|
| GlassBench (simulator at 87 fps) | 50.5-52.0 fps, gaps p50/p95 19-20/25-27 ms | 58.6-58.8 fps, 17.4/26.2 ms |
| Safari scrolling (simulator at 64 fps) | 28.7 fps | 51.5 fps |
| tap to decoded frame, p50 (A/B, alternating, 30 taps each) | 64.9 and 64.2 ms | 60.5 and 61.4 ms |
| copy, encode, capture-to-wire (p50) | 9.5, 9.0, 9.0 ms | 9.2, 9.3, 9.6 ms |

The capture still copies every new frame before the throttle (about 90% of a core at 87 fps); copying only the
frames it sends would save most of that. The throttle is in Simlock's `SimlockCapture/Lifecycle.swift`
(`encodeLoop`), shared with serve-sim.

## Tried, no gain

| | Result |
Expand Down
1 change: 1 addition & 0 deletions docs/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ node), makes an SSH key `~/.ssh/macbox_ed25519`, and adds a `Host macbox` block
| `MACBOX_BUILD_CPUSET` | empty | cores for `macbox build`, e.g. the efficiency cores |
| `MACBOX_DISK_CACHE` | none | `writeback` panicked macOS's NVMe driver under heavy swap; `unsafe` is faster, but a crash of the host itself can corrupt the VM's disk. Applies at the next `macbox up` |
| `MACBOX_COHERENT` | auto | `auto`: on with an Arrow Lake iGPU. Coherent, zero-copy surfaces, and the switches measured with it: a 3x simulator 50 → 120 fps there ([performance.md](performance.md#coherent-and-the-switches-measured-with-it-on-by-default-on-arrow-lake)); `on` with any GPU, `off` |
| `MACBOX_VIDEO` | off | `on`: Reims's hardware VideoToolbox and M2 scaler devices, so the guest's H.264/HEVC encoding (the Device panel's stream) and decoding run on the host GPU through VA-API instead of VideoToolbox's software encoder. Needs the video kexts in OpenCore first (`macbox video-kexts`); macOS 26.7 (25G241) only. See [performance.md](performance.md#hardware-video-encoding-macbox_video) |
| `MACBOX_HOME` | ~/.local/share/macbox | everything macbox keeps |

Then:
Expand Down
26 changes: 25 additions & 1 deletion macbox
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ eval "$_env"
: "${MACBOX_GUEST_USER:=dev}" "${MACBOX_CPUS:=8}" "${MACBOX_MEMORY:=12G}" "${MACBOX_DISK_SIZE:=160G}"
: "${MACBOX_RESOLUTION:=1920x1080}" "${MACBOX_REFRESH:=60}" "${MACBOX_CPUSET:=}"
: "${MACBOX_RENDER_NODE:=/dev/dri/renderD128}" "${MACBOX_DISK_CACHE:=none}" "${MACBOX_GPU_TUNING:=auto}"
: "${MACBOX_COHERENT:=auto}"
: "${MACBOX_COHERENT:=auto}" "${MACBOX_VIDEO:=off}"
: "${MACBOX_SIM_PREFIX:=macbox}" "${MACBOX_POOL:=full slim-1 slim-2 sharp small}"
: "${MACBOX_T3_HOST_ID:=$MACBOX_SSH_HOST}" "${MACBOX_T3_SETTINGS:=$HOME/.t3/userdata/settings.json}"
DOCKER="${MACBOX_DOCKER:-docker}"
Expand Down Expand Up @@ -165,6 +165,9 @@ qemu_extra() { # the network (passt, SSH forwarded by Docker) and the disk's op
-set "drive.disk.cache=$MACBOX_DISK_CACHE" -set drive.disk.discard=unmap; do
out+=("$(printf '%q' "$a")")
done
# Reims's VideoToolbox and M2 scaler devices, in the slots reims-ctl gives them (after the mapper)
[ "$MACBOX_VIDEO" = on ] && out+=(-device apple-videotoolbox-pci,id=video,bus=pcie.0,addr=0x8 \
-device apple-m2scaler-pci,id=scaler,bus=pcie.0,addr=0x9)
echo "${out[*]}"
}

Expand Down Expand Up @@ -209,6 +212,9 @@ MACBOX_GPU_TUNING=$MACBOX_GPU_TUNING
# auto (on with an Arrow Lake iGPU, where it runs day to day), on, off: Coherent, zero-copy
# IOSurfaces, and the switches measured with it (docs/performance.md)
MACBOX_COHERENT=$MACBOX_COHERENT
# on: Reims's hardware VideoToolbox (H.264/HEVC encode and decode on the host's GPU through VA-API)
# and M2 scaler devices; needs the video kexts in OpenCore (macbox video-kexts). off: as before
MACBOX_VIDEO=$MACBOX_VIDEO
# iOS Simulator pool: names are "<device> (<prefix> <label>)"
MACBOX_SIM_PREFIX=$MACBOX_SIM_PREFIX
MACBOX_POOL="$MACBOX_POOL"
Expand Down Expand Up @@ -308,6 +314,22 @@ cmd_build() {
if vm_running; then say "the running VM keeps its QEMU; the new build starts with the next macbox down && macbox up"; fi
}

cmd_video_kexts() { # [reims-util commit]: its kext/ copied into the guest, then opencore-video-kexts.sh
ssh_ok || die "ssh $MACBOX_SSH_HOST doesn't answer"
local rev=${1:-8a8a852} src="$MACBOX_HOME/src/reims-util" t
[ -d "$src/.git" ] || git clone -q https://github.com/steelbrain/reims-util "$src"
Comment on lines +319 to +320

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fetch the video kexts into a user-writable directory

For a non-root user whose installation was built with rootful Docker, macbox video-kexts fails before copying any drivers: src/container/build-reims.sh creates $MACBOX_HOME/src inside the build container, and neither cmd_build nor the Dockerfile selects a non-root user, so that directory is normally root-owned and not writable from this host-side git clone. Run the fetch/archive inside the container, as the existing build does, or use a separate user-writable checkout location.

Useful? React with 👍 / 👎.

git -C "$src" cat-file -e "$rev^{commit}" 2>/dev/null || git -C "$src" fetch -q origin
t=$(mktemp -d)
git -C "$src" archive "$rev" kext/AppleM2ScalerParavirtDriver.kext kext/AppleVideoToolboxParavirtualization.kext \
kext/VtWaitPatch.kext | tar -x -C "$t"
guest 'rm -rf /tmp/macbox-video-kexts && mkdir -p /tmp/macbox-video-kexts'
tar -C "$t/kext" -c . | guest 'tar -x -C /tmp/macbox-video-kexts'
rm -rf "$t"
scp -q "$REPO/src/guest/bin/opencore-video-kexts.sh" "$MACBOX_SSH_HOST:/tmp/macbox-video-kexts/"
ssh -t "$MACBOX_SSH_HOST" 'bash /tmp/macbox-video-kexts/opencore-video-kexts.sh /tmp/macbox-video-kexts'
say "Next: MACBOX_VIDEO=on in $CONF, then macbox down && macbox up"
}

cmd_create() {
need_init
local macos=tahoe offline=()
Expand Down Expand Up @@ -640,6 +662,7 @@ Day to day:
macbox screenshot [out.png] | click X Y [right|double] | move X Y | scroll X Y N
macbox type TEXT|- | key COMBO... | wait SECONDS [out.png]
macbox coherent [status|reset] Coherent's crash guard (MACBOX_COHERENT=auto|on|off)
macbox video-kexts reims-util's video kexts into OpenCore (for MACBOX_VIDEO=on)
macbox backup <name>
EOF
}
Expand Down Expand Up @@ -691,6 +714,7 @@ case "$sub" in
t3) cmd_t3 "$@" ;;
agents) cmd_agents "$@" ;;
coherent) cmd_coherent "$@" ;;
video-kexts) cmd_video_kexts "$@" ;;
backup) cmd_backup "$@" ;;
help|-h|--help) cmd_help ;;
*) cmd_help >&2; exit 2 ;;
Expand Down
94 changes: 94 additions & 0 deletions src/guest/bin/opencore-video-kexts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
#!/bin/bash
# Adds reims-util's three video kexts to the VM's OpenCore image, from inside macOS, so an x86-64
# guest drives Reims's VideoToolbox (106b:1a03) and M2 scaler (106b:1a0b) devices. reims-util
# 8a8a852 (2026-10-29) and later put them in every image they make; an image an older reims-util
# made lacks them, and `reims-ctl update-opencore` would rebuild the whole image (dropping any other
# change to it), so this adds just the three, as reims-util's own entries do:
# AppleVideoToolboxParavirtualization.kext, AppleM2ScalerParavirtDriver.kext: Apple's arm64 guest
# drivers ported to x86-64; they link apv-arm64-port's IOSurface mapper, already in the image;
# VtWaitPatch.kext: a Lilu plugin that keeps VideoToolbox's 10 s waits for the host from expiring
# at once on x86-64 (without it every encode fails with -19092). It patches 25G241 only.
# Each loads on Darwin 25.6.x (macOS 26.7) alone (MinKernel 25.6.0, MaxKernel 25.6.99), and the script
# refuses another macOS build than 25G241 unless FORCE=1.
#
# opencore-video-kexts.sh <dir with the three .kext> [<EFI partition>] (run with sudo rights)
#
# The kexts come from reims-util's kext/ (github.com/steelbrain/reims-util), e.g.
# git -C reims-util archive 8a8a852 kext/AppleM2ScalerParavirtDriver.kext \
# kext/AppleVideoToolboxParavirtualization.kext kext/VtWaitPatch.kext | tar -x
# The EFI partition is OpenCore's own disk (the one whose EFI/OC/config.plist exists); without one
# it is found. config.plist is kept as config.plist.before-video beside it. Takes effect at the next
# boot, with the devices attached (MACBOX_VIDEO=on). Running it again changes nothing.
# An entry already in config.plist for one of the three is set to the values above (an entry that is
# disabled or has other paths or kernel limits would not load). Nothing on the EFI partition changes
# until the edited config.plist passes plutil and the new kexts are copied beside the old ones; then
# both are swapped in by renames, and a failure part way puts the old ones back.
set -euo pipefail
K="$(cd "$1" && pwd)"
NAMES=(AppleM2ScalerParavirtDriver AppleVideoToolboxParavirtualization VtWaitPatch)
for k in "${NAMES[@]}"; do [ -d "$K/$k.kext" ] || { echo "no $K/$k.kext" >&2; exit 1; }; done
build=$(sw_vers -buildVersion)
[ "$build" = 25G241 ] || [ "${FORCE:-}" = 1 ] || {
echo "macOS build $build: VtWaitPatch patches 25G241 (macOS 26.7.1) only; FORCE=1 installs anyway" >&2; exit 1; }
M=$(mktemp -d /tmp/ocmnt.XXXX) T=$(mktemp /tmp/occonfig.XXXX)
O="$M/EFI/OC/Kexts" swapped=()
cleanup() {
for k in "${NAMES[@]}"; do sudo rm -rf "$O/.$k.kext.new" 2>/dev/null || true; done
sudo umount "$M" 2>/dev/null; rmdir "$M"; rm -f "$T"
}
trap cleanup EXIT
found=""
for part in ${2:-$(diskutil list | awk '/EFI/ {print $NF}')}; do
sudo mount -t msdos "/dev/$part" "$M" 2>/dev/null || continue
if [ -f "$M/EFI/OC/config.plist" ]; then found=$part; break; fi
sudo umount "$M"
done
[ -n "$found" ] || { echo "no EFI partition with EFI/OC/config.plist" >&2; exit 1; }
P=/usr/libexec/PlistBuddy C="$M/EFI/OC/config.plist"

# 1. The edited config.plist, staged in /tmp and checked.
cp "$C" "$T"
n=0 # entries; macOS's bash 3.2 has no associative arrays, so an entry is found by a scan
while $P -c "Print :Kernel:Add:$n:BundlePath" "$T" >/dev/null 2>&1; do n=$((n + 1)); done
index_of() { local j=0; while [ $j -lt $n ]; do
[ "$($P -c "Print :Kernel:Add:$j:BundlePath" "$T" 2>/dev/null)" = "$1" ] && { echo $j; return; }; j=$((j + 1)); done; }
for k in "${NAMES[@]}"; do
i=$(index_of "$k.kext")
if [ -n "$i" ]; then what="set in place (Kernel.Add.$i)"
else i=$n; n=$((n + 1)); $P -c "Add :Kernel:Add:$i dict" "$T"; what="added as Kernel.Add.$i"; fi
for kv in "Arch string x86_64" "BundlePath string $k.kext" "Comment string reims-vgpu-video $k" \
"Enabled bool true" "ExecutablePath string Contents/MacOS/$k" "MaxKernel string 25.6.99" \
"MinKernel string 25.6.0" "PlistPath string Contents/Info.plist"; do
key=${kv%% *}
$P -c "Delete :Kernel:Add:$i:$key" "$T" 2>/dev/null || true
$P -c "Add :Kernel:Add:$i:$kv" "$T"
done
echo "$k: $what"
done
plutil -lint "$T" >/dev/null

# 2. The new kexts, copied beside the old ones (a full partition fails here, with nothing changed).
for k in "${NAMES[@]}"; do sudo rm -rf "$O/.$k.kext.new"; sudo cp -R "$K/$k.kext" "$O/.$k.kext.new"; done
[ -f "$C.before-video" ] || sudo cp -p "$C" "$C.before-video"
sudo cp "$T" "$C.new"

# 3. Swap in by renames; on a failure the old kexts and config.plist go back.
rollback() {
echo "failed; putting the previous kexts and config.plist back" >&2
for k in "${swapped[@]}"; do
sudo rm -rf "$O/$k.kext"; [ -d "$O/.$k.kext.prev" ] && sudo mv "$O/.$k.kext.prev" "$O/$k.kext"
done
sudo rm -f "$C.new"
}
trap 'rollback' ERR
for k in "${NAMES[@]}"; do
sudo rm -rf "$O/.$k.kext.prev"
[ -d "$O/$k.kext" ] && sudo mv "$O/$k.kext" "$O/.$k.kext.prev"
swapped+=("$k")
sudo mv "$O/.$k.kext.new" "$O/$k.kext"
done
sudo mv "$C.new" "$C"
trap - ERR
for k in "${NAMES[@]}"; do sudo rm -rf "$O/.$k.kext.prev"; done
sync
echo "OpenCore on $found has the video kexts; they load at the next boot"
3 changes: 2 additions & 1 deletion src/patches/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,14 @@ is in [docs/performance.md](../../docs/performance.md#gpu-tuning-switches).
| 0030 `REIMS_VGPU_PAGED_BUFFER_BY_PAGES` | finds kept paged buffers by their guest pages; off |
| 0031–0033 staged page-ups | `REIMS_VGPU_STAGED_PAGE_UP`, `REIMS_VGPU_STAGED_ZERO`: a new texture's first page-up copies its pages into a staging buffer (one copy per run) when no live view covers them, and pages of zeros write nothing. Before, each page run took its own mapping: 55k a second and a storm of TLB shootdowns; the simulator went from 89 to 101 fps. On with Coherent |

## QEMU (`vendor/qemu` at bd95509, github.com/steelbrain/qemu-reims-vgpu), 13 patches
## QEMU (`vendor/qemu` at bd95509, github.com/steelbrain/qemu-reims-vgpu), 14 patches

| Patches | What they do |
|---|---|
| 0001–0003, 0006–0009 the census | memslots move to a second mapping of guest RAM that only KVM uses, and userfaultfd markers there report the guest CPU's touches of tracked pages (`UFFD_USER_MODE_ONLY`: no capability or `/dev/userfaultfd` needed). QEMU's own loads, stores and DMA reach it too. Off unless `REIMS_VGPU_CENSUS` is set. |
| 0004, 0005 coherent pages | with `REIMS_VGPU_COHERENT=1` the same markers serve metal2vulkan's trap-coherent surfaces. A run that doesn't end well leaves `coherent.running` in the census directory, and every later start boots without coherent pages until it's removed (`macbox coherent reset`), so a fault there costs one crash, not a crash loop. |
| 0010–0013 KVM shortcuts for the GPU device | `x-irqfd` (interrupts through a KVM irqfd), `x-halt-poll-ns`, `x-ioeventfd` (doorbells through KVM ioeventfds): device properties, all off unless set |
| 0014 the video devices | upstream's own 4895365e5b (on `steelbrain/reims-vgpu`, after bd95509), unchanged: `apple-videotoolbox-pci` (106b:1a03) and `apple-m2scaler-pci` (106b:1a0b), the transports of Reims's VideoToolbox and scaler devices. Without it the build can't attach them. Drop it when `QEMU_REF` moves past 4895365e5b. The devices stay detached unless `MACBOX_VIDEO=on` |

## Licenses

Expand Down
Loading
Loading