Skip to content

feat: add support for B frames in encoding and decoding - #57

Merged
hgaiser merged 13 commits into
mainfrom
feat/b-frames
Nov 13, 2026
Merged

hgaiser merged 13 commits into
mainfrom
feat/b-frames

Conversation

@hgaiser

@hgaiser hgaiser commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Fixes #52

@DatCaptainHorse

Copy link
Copy Markdown
Contributor

You're on a roll 🔥

@hgaiser

hgaiser commented Nov 11, 2026

Copy link
Copy Markdown
Owner Author

You're on a roll 🔥

Haha yeah, onwards to 1.0 I guess :).

Could you try the tests on your GPU and share the results? Maybe also with validation enabled? I'm curious if it runs into any issues.

@DatCaptainHorse

Copy link
Copy Markdown
Contributor

Haha yeah, onwards to 1.0 I guess :).

Could you try the tests on your GPU and share the results? Maybe also with validation enabled? I'm curious if it runs into any issues.

Here:

❯ PIXELFORGE_VALIDATION=1 ANV_DEBUG=video-decode,video-encode cargo test --workspace -- --ignored --nocapture --test-threads=1
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running unittests src/lib.rs (target/debug/deps/pixelforge-63ab83a485f0f200)

running 7 tests
test converter::device_tests::a_replaced_source_image_is_read_afresh ... ok
test converter::device_tests::encode_is_deterministic ... H264 run 1: frames that differ from run 0: []
H264 run 2: frames that differ from run 0: []
H264 run 3: frames that differ from run 0: []
H264 run 4: frames that differ from run 0: []
H265 run 1: frames that differ from run 0: []
H265 run 2: frames that differ from run 0: []
H265 run 3: frames that differ from run 0: []
H265 run 4: frames that differ from run 0: []
AV1 run 1: frames that differ from run 0: []
AV1 run 2: frames that differ from run 0: []
AV1 run 3: frames that differ from run 0: []
AV1 run 4: frames that differ from run 0: []
ok
test converter::device_tests::every_conversion_matches_the_model ... Srgb -> Srgb NV12: worst 1.0, mean 0.0002, VUI 1/1/1 pc
Srgb -> Srgb P010: worst 0.0, mean 0.0000, VUI 1/1/1 pc
Srgb -> Bt2020Pq NV12: worst 1.0, mean 0.0005, VUI 9/16/9 pc
Srgb -> Bt2020Pq P010: worst 1.0, mean 0.0018, VUI 9/16/9 pc
Bt709Linear -> Bt2020Pq NV12: worst 1.0, mean 0.0003, VUI 9/16/9 pc
Bt709Linear -> Bt2020Pq P010: worst 1.0, mean 0.0012, VUI 9/16/9 pc
Bt2020Linear -> Bt2020Pq NV12: worst 1.0, mean 0.0005, VUI 9/16/9 pc
Bt2020Linear -> Bt2020Pq P010: worst 1.0, mean 0.0013, VUI 9/16/9 pc
Bt2020Pq -> Bt2020Pq NV12: worst 0.0, mean 0.0000, VUI 9/16/9 pc
Bt2020Pq -> Bt2020Pq P010: worst 0.0, mean 0.0000, VUI 9/16/9 pc
ok
test converter::device_tests::pipelined_shader_frames_do_not_overwrite_each_other ... ok
test converter::device_tests::quantizers_round_rather_than_truncate ... NV12 Limited: 8177 mismatches if truncating, 0 if rounding, mean signed error -0.0006
NV12 Full: 8259 mismatches if truncating, 4 if rounding, mean signed error +0.0017
P010 Limited: 8176 mismatches if truncating, 0 if rounding, mean signed error -0.0012
P010 Full: 8253 mismatches if truncating, 0 if rounding, mean signed error +0.0040
ok
test converter::device_tests::shader_source_matches_converting_by_hand ... ok
test converter::device_tests::unsupported_conversions_are_refused ... Bt709Linear -> Srgb: Invalid input: no conversion from Bt709Linear to Srgb: converting to SDR would need tone mapping or a gamma curve applied, which the shader does not do
Bt2020Linear -> Srgb: Invalid input: no conversion from Bt2020Linear to Srgb: converting to SDR would need tone mapping or a gamma curve applied, which the shader does not do
Bt2020Pq -> Srgb: Invalid input: no conversion from Bt2020Pq to Srgb: converting to SDR would need tone mapping or a gamma curve applied, which the shader does not do
Srgb -> Bt709Linear: Invalid input: no conversion from Srgb to Bt709Linear: video cannot be encoded in a linear space, so it can only be a source
Srgb -> Bt2020Linear: Invalid input: no conversion from Srgb to Bt2020Linear: video cannot be encoded in a linear space, so it can only be a source
ok

test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 223 filtered out; finished in 1.63s

     Running tests/adopted.rs (target/debug/deps/adopted-dc1dab249f3590ec)

running 3 tests
test decode_on_a_vulkan_1_1_device ... ok
test encode_and_decode_on_one_adopted_device ... ok
test encode_on_a_vulkan_1_1_device ... adopted vs own encode, 0 B-frames, luma PSNR per frame: [inf, inf, inf, inf, inf, inf, inf, inf]; streams bit-identical: true
adopted vs own encode, 2 B-frames, luma PSNR per frame: [inf, inf, inf, inf, inf, inf, inf, inf]; streams bit-identical: true
ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.62s

     Running tests/b_frames.rs (target/debug/deps/b_frames-2a6de517cac7ec22)

running 3 tests
test b_frame_query_matches_encoder_creation ... H264: Flat
H265: Flat
AV1: Pyramid
ok
test control_calls_close_the_group ... [h264 @ 0x5573b4d60fc0] reference count 3 overflow
[h264 @ 0x5573b4db8d80] reference count overflow
[h264 @ 0x5573b4db8d80] decode_slice_header error
[vist#0:0/h264 @ 0x5573b4d603c0] [dec:h264 @ 0x5573b4d90d40] Decoding error: Invalid data found when processing input

thread 'control_calls_close_the_group' (389227) panicked at tests/b_frames.rs:232:9:
assertion `left == right` failed: H264: frames ffmpeg decoded
  left: 40
 right: 41
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAILED
test too_many_b_frames_is_an_error ... ok

failures:

failures:
    control_calls_close_the_group

test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.26s

error: test failed, to rerun pass `-p pixelforge --test b_frames`

Also of note: ANV (Mesa) got updated since last time, it fixed various issues and AV1 has bitrate control support now as well, hopefully getting out of experimental flags soon 👀

@hgaiser

hgaiser commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Thanks @DatCaptainHorse . Seems reference frame invalidation is not (yet) supported on ANV. Not much we can do there but wait 🤷

Can you run once more with --no-fail-fast to see if other tests fail?

@DatCaptainHorse

Copy link
Copy Markdown
Contributor

@hgaiser Ah, forgot about that flag completely. Attached as file since it's quite lengthy.

testrun.txt

@hgaiser

hgaiser commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Cool, that helped :). Could you try again? Most of them should be resolved. AV1 and RFI are still problems though.

@DatCaptainHorse

Copy link
Copy Markdown
Contributor

Here you go @hgaiser

testrun2.txt

If at any point you need it or it would help more, I can passthrough the A310 into a clean VM which you can access for debugging, I'd use the iGPU in the meanwhile (probably just being AFK though).

@hgaiser

hgaiser commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

Thanks for the offer @DatCaptainHorse , I think for now we're good on this feature. The things that I expect to work, work and the things that I expect to be flaky (AV1 and RFI) don't work.

@hgaiser
hgaiser merged commit e384a92 into main Nov 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] B-frame support

2 participants