Skip to content

fix(video): fit FEC, audio and packet overhead inside the client's bitrate - #278

Open
porkloin wants to merge 1 commit into
hgaiser:mainfrom
porkloin:bitrate-overhead
Open

porkloin wants to merge 1 commit into
hgaiser:mainfrom
porkloin:bitrate-overhead

Conversation

@porkloin

Copy link
Copy Markdown
Contributor

Currently, Moonshine encodes video at the full bitrate the user picked in Moonlight and sends FEC, audio and packet headers on top, so a 20 Mb/s setting sends 24–25 Mb/s. On a link without that headroom, the excess is dropped and the stream freezes.

This means that the "real" bitrate being sent to moonlight clients is regularly ~110% of the declared bitrate the user has chosen in moonlight.

This PR fits the whole stream inside the user's declared bitrate, the way Moonlight expects (it sends the setting as x-ml-video.configuredBitrateKbps for this) and Sunshine does in cmd_announce() (src/rtsp.cpp):

  • Video gets what's left after the FEC percentage, then the audio bitrate (at most a fifth of the rest), then 500 kb/s of overhead (at most a tenth). At 20 Mb/s with the default 20% FEC, that's 15.4 Mb/s.
  • On a 22 Mb/s link, main drops to 34–44 fps with hundreds of frames lost; this branch holds 60 fps with none lost.
  • Gated behind setting stream.video.bitrate_includes_overhead, on by default.

Testing

Moonshine's tests

All 199 pass, including a new one for the calculation: 20 Mb/s with 20% FEC and 512 kb/s audio gives 14.99 Mb/s of video, the overhead caps apply at 1 Mb/s, and an FEC percentage above 80 is ignored, as in Sunshine.

Integration testing

Tested with moonshine-test-headless, which streams a test app from Moonshine to a headless Moonlight client and records what the client receives. The app was framestamp, a full-motion test pattern whose frames the client checks against the expected image. 1920x1080 at 60 fps, 20 Mb/s, 20 s per codec. This branch (150dd98) against main (934c5c5).

Link with little headroom. Client on the same machine (RX 7900 XTX), its network limited to 22 Mb/s with a 50 ms router queue and 10 ms of delay. The harness summary below is from this run.

22 Mb/s link main this branch
H.264 39 fps, 391 frames lost, 6.6 s frozen 60 fps, nothing lost
HEVC 44 fps, 185 lost, 5.2 s frozen 60 fps, nothing lost
AV1 34 fps, 479 lost, 8.2 s frozen 60 fps, nothing lost

Peak bitrate fell from 114% to 89% of the setting. On an RX 9070 XT, main got 56, 34 and 32 fps; this branch 60 fps with nothing lost.

No bandwidth limit. The builds match on both GPUs, at 1080p60 and at 3840x2160 120 fps 113 Mb/s: the same missing frames, keyframes and freeze time. Host latency p99 2.59 ms against 2.68 ms.

Over Wi-Fi 7, to an RX 9070 XT client: peak bitrate fell from 114–115% to 91–97%, and neither build lost a frame. Both had freezes, the longest (5–6 s) on this branch's HEVC stream in both runs. Moonshine never went more than 40 ms between frames sent, so these are most likely Wi-Fi interference.

Picture quality. The client's error against the expected image (worst stream, 99th percentile) rose from 0.9–1.8 to 1.1–2.8 at 1080p60, and by 0.1–0.8 at 4K120. No picture was corrupt.

The harness's timing comparison gives no verdict, because it only compares streams encoded at the same bitrate.

Moonshine 150dd98 vs main 934c5c5 · 1920x1080 at 60 fps, link capped at 22 Mb/s

Hardware

Host Client
GPU AMD Radeon RX 7900 XTX (RADV NAVI31) AMD Radeon RX 7900 XTX
Driver radv Mesa 26.2.7-arch1.1 (Vulkan) radeonsi Mesa 26.2.7-arch1.1 (VAAPI)
Other GPUs N/A N/A
CPU AMD Ryzen 9 5950X 16-Core Processor AMD Ryzen 9 5950X 16-Core Processor
Memory 126 GB 126 GB
OS Fedora Linux 44 (Container Image) Arch Linux
Kernel 7.2.11-200.fc44.x86_64 7.2.11-200.fc44.x86_64
Network N/A N/A
Machine X570D4U-2L2T X570D4U-2L2T

Stream

  • local@150dd98 (local moonshine-bitrate, commit 150dd98), 1920x1080 at 60 fps, 20 Mb/s, streams of 20 s each, route loopback, chaos --rate 22000:50 --delay 10
  • workloads (streams): framestamp-full (15); 17865 frames decoded; clock sync ±0.02 ms
  • tools: moonshine-test-headless 848e1f0, gamestream-test-client e0a987a, moonshine-otel v1.4.1

Report card

Metric test baseline Target
Decode errors 🟢 0 🟢 0 ≤ 0
Corrupt pictures 🟢 0 🟢 0 ≤ 0
Corrupt after recovery 🟢 0 🟢 0 ≤ 0
Lost frames 🟢 0 🟢 0 ≤ 0
Frame pacing p99 🟢 1.01 frames (16.8 ms) 🟢 1.02 frames (17.0 ms) ≤ 1.10 frames (18.3 ms)
Delivered frame rate 🟢 100.0% 🟢 100.0% ≥ 99.0%
Host latency p99 🟢 2.59 ms 🟢 2.68 ms ≤ 3.00 ms
Recovery time p95 no loss test 🔴 124.37 frames (2072.8 ms) ≤ 3.00 frames (50.0 ms)
RFI recovery rate no loss test 🟡 67.7% ≥ 95.0%
Input latency p50 no click test no click test ≤ 2.00 frames (33.3 ms)
HDR switch keyframes no HDR switch test no HDR switch test ≤ 0.70
FPS under GPU load no GPU-load test no GPU-load test ≥ 99.0%
Peak bitrate 🟢 89.3% 🟡 114.2% ≤ 110.0%

Client

The client machine's decoding and display. Moonshine doesn't control these, so they aren't scored, but a slow client can cause freezes and keyframes in the report card.

Metric test baseline Target
Decoded to on screen p99 8.50 ms 8.51 ms not scored
Dropped by the display 0.06% 0.09% not scored
Judder 0.00% 0.00% not scored
Client decode p99 0.08 frames (1.3 ms) 0.08 frames (1.3 ms) not scored
Keyframes the client caused 0 0 not scored

Checks

Check Result Notes
Build Moonshine ok
Build baseline Moonshine ok
framestamp, full motion: h264 ok 1118 frames at 60 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1118 checked; video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: h264, --rate 22000:50 --delay 10 ok 1118 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1118 checked; video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: hevc ok 1118 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1118 checked; video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: hevc, --rate 22000:50 --delay 10 ok 1119 frames at 60 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1119 checked; video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: av1 ok 1120 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.7 s frozen); picture: 0 corrupt of 1120 checked, decoded at 1920x1088 (rows the encoder padded, which clients also show); video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: av1, --rate 22000:50 --delay 10 ok 1118 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1118 checked, decoded at 1920x1088 (rows the encoder padded, which clients also show); video encoded at 15404 of the 20000 kb/s asked for
framestamp, full motion: h264, baseline ok 1119 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1119 checked
framestamp, full motion: h264, --rate 22000:50 --delay 10, baseline ok 5891 packets dropped on purpose; 725 frames at 38.9 fps, 391 missing, 10 keyframes, 0 RFI requests, 0 recovery frames, 7 freezes (6.6 s of 18.6 s frozen); picture: 0 corrupt of 725 checked
framestamp, full motion: hevc, baseline ok 1121 frames at 60.1 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.7 s frozen); picture: 0 corrupt of 1121 checked
framestamp, full motion: hevc, --rate 22000:50 --delay 10, baseline ok 5708 packets dropped on purpose; 814 frames at 43.7 fps, 185 missing, 1 keyframes, 303 RFI requests, 12 recovery frames, 13 freezes (5.2 s of 18.6 s frozen); picture: 0 corrupt of 35 checked
framestamp, full motion: av1, baseline ok 1118 frames at 60 fps, 0 missing, 1 keyframes, 0 freezes (0 s of 18.6 s frozen); picture: 0 corrupt of 1118 checked, decoded at 1920x1088 (rows the encoder padded, which clients also show)
framestamp, full motion: av1, --rate 22000:50 --delay 10, baseline ok 5562 packets dropped on purpose; 638 frames at 34.2 fps, 479 missing, 7 keyframes, 464 RFI requests, 9 recovery frames, 12 freezes (8.2 s of 18.6 s frozen); picture: 0 corrupt of 638 checked, decoded at 1920x1088 (rows the encoder padded, which clients also show)
framestamp, full motion: speed h264 vs baseline skipped no verdict (logs/bench/compare-framestamp-full-h264.md)
framestamp, full motion: speed hevc vs baseline skipped no verdict (logs/bench/compare-framestamp-full-hevc.md)
framestamp, full motion: speed av1 vs baseline skipped no verdict (logs/bench/compare-framestamp-full-av1.md)

Results produced by moonshine-test-headless. Results file attached.

To run the same test on your hardware:

podman container runlabel run ghcr.io/porkloin/moonshine-test-headless -- \
    --moonshine porkloin@bitrate-overhead --baseline --iterations 3 \
    --app framestamp-full --chaos --rate 22000:50 --delay 10

To view the attached results:

podman container runlabel run ghcr.io/porkloin/moonshine-test-headless:view \
    path/to/results.tar.gz

@porkloin
porkloin marked this pull request as ready for review November 20, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant