Skip to content

video: encode the H.264 profile the viewer negotiated - #68

Merged
Rust-KVM merged 1 commit into
devfrom
claude/project-thread-u69c0c
Oct 18, 2026
Merged

Rust-KVM merged 1 commit into
devfrom
claude/project-thread-u69c0c

Conversation

@Rust-KVM

Copy link
Copy Markdown
Owner

Requested by 4c

Summary

Before: the encoder always sent H.264 High profile, even when the browser negotiated Constrained Baseline. Linux Chrome offers no High entry, so its session negotiated 42e01f and then got a High 5.2 stream. Lenient decoders cope with that, but a strict decoder such as OpenH264 cannot decode CABAC and would show a black screen.

After: the encoder sends the profile that was negotiated. Browsers that offer High (Chrome or Safari on macOS, for example) still get High; baseline-only browsers get Constrained Baseline (SPS 42 c0).

How

When a viewer's ICE connects, negotiated_h264_profile reads the sender's negotiated codecs and picks the same codec that webrtc-rs binds the track to (the first exact match for 640034, otherwise the first H.264 codec). video::set_h264_profile then switches mpph264enc's profile property at runtime. The switch happens before the new viewer's keyframe is requested, so its first IDR already carries the new SPS. The choice is also applied on every pipeline rebuild, so supervisor restarts keep it. The encoder probe logs the SPS profile_idc whenever it changes, which makes the switch visible in the device log.

The level is still 5.2 while browsers advertise level 3.1 (1f). Every browser accepts that for 4K WebRTC, so this change leaves the level alone.

Testing

  • cargo +nightly fmt --all
  • cargo +nightly clippy -p rustkvm --release --all-targets -- -D warnings
  • cargo +nightly test --workspace: this host has no GStreamer, so I cross-built the new h264_profile test and ran it on the device instead (6/6 passed). CI runs the full suite.
  • Frontend touched: not applicable
  • Tested on an RK3588 device at 3840x2160@60 with view-only headless Chrome (no HID input sent):
    • Log: H.264 encoder profile set to match the viewer, profile: "baseline", then encoder SPS profile, profile_idc: 66, with the IDR delivered 26 ms after the request.
    • The device encoded 59.8 fps with this change and 61.8 fps on the unmodified dev build. Both readings are 60 fps within sampling noise.
    • Bitrate on a static desktop was 3.7 Mbps with Baseline versus 3.4 Mbps with High, about 9% higher.
    • mpph264enc profile=baseline|main|high produces SPS bytes 42 c0, 4d 40 and 64 10 on this device.

Checklist

  • No #[allow(...)] suppressions and no new unwrap() / expect() in production paths
  • Hot paths stay zero-copy; periodic loops use MissedTickBehavior::Delay
  • Shared protocol types live in rkvm-proto
  • RPC method set changed → app/tests/rpc_methods.txt updated (unchanged)
  • Module tree or build invariants changed → AGENTS.md updated (unchanged)

🤖 Generated with Claude Code

The encoder always produced High profile, level 5.2, but a browser whose
offer has no High entry (Chrome without a High-capable decoder, for one)
negotiates Constrained Baseline and was sent High anyway. Lenient decoders
cope; strict ones such as OpenH264 cannot decode CABAC.

Read the negotiated codecs when a viewer connects, mirror the track's
binding choice, and switch mpph264enc's profile at runtime before the
viewer's keyframe is requested. The choice survives pipeline rebuilds.
Log the SPS profile_idc whenever it changes so the switch is visible.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Rust-KVM Rust-KVM self-assigned this Oct 18, 2026
@github-actions github-actions Bot added the area: video GStreamer, MPP encoding, WebRTC media label Oct 18, 2026
@Rust-KVM
Rust-KVM marked this pull request as ready for review October 18, 2026 06:52
@Rust-KVM
Rust-KVM merged commit 11932f3 into dev Oct 18, 2026
8 checks passed
@Rust-KVM
Rust-KVM deleted the claude/project-thread-u69c0c branch October 18, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: video GStreamer, MPP encoding, WebRTC media

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants