Repository navigation
media: s5p-mfc: H.264, HEVC, VP8 and VP9 decode on the Pixel 7 (gs201 MFC v15) - #250
angelrenefm wants to merge 21 commits into
Conversation
|
Thank you, Ángel. This is a remarkable piece of work: a mainline driver taught a new MFC generation, bit-exact checks against the software decoders for every codec, real players and browsers tested, and an honest "not checked" list. I especially appreciate that you found the GStreamer pool-flush deadlock and said plainly it's GStreamer's to fix. I checked what I could:
A few things before it can go in, and one I need to decide myself:
Smaller:
For the order: #181 and #195 both add a pool in the same place in |
…E_CHANGE The stateful decoder interface (dev-decoder.rst, "Initialization") has the client learn the stream's coded format from V4L2_EVENT_SOURCE_CHANGE once the header is parsed; s5p-mfc only sent the event for a resolution change in the middle of a stream. Queue it when SEQ_DONE leaves the instance in HEAD_PARSED. Clients that do not subscribe to the event are not affected. Tested on a Pixel 7 (gs201, MFC v15.6) with the rest of this series: a small V4L2 stateful test client that waits for the event, FFmpeg's h264_v4l2m2m and GStreamer's v4l2h264dec all decode H.264 bit-exact against FFmpeg's software decoder. Not tested on any older MFC. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The preallocated-memory allocator marks b->size >> PAGE_SHIFT pages as used, so the tail of a buffer whose size is not a multiple of the page size is left free in the bitmap and can be handed out again. Today the 64 KiB alignment of every allocation hides it for the context buffers (30 KiB, 2 MiB), but not for a buffer that ends just past a 64 KiB boundary. Round the page count up on allocation and release alike. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The Google GS201 SoC (Pixel 7) has an MFC v15.6 (MFC_VERSION reads 0x15060000). Like the Tesla FSD one it takes a single "mfc" clock and sits behind up to two System MMUs. On GS201 that clock is the ACPM firmware's MFC DVFS domain, which sets the voltage with the rate. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The GS201 MFC (IP version 0x15060000) keeps the v8+ register map used by v12, with these differences, measured on a Pixel 7 or taken from Google's exynos-mfc driver (kernel/google-modules/soc/gs, drivers/media/platform/exynos/mfc/) and its device tree (kernel/devices/google/gs201, dts/gs201-mfc.dtsi), both on branch android-gs-pantah-6.1-android16: - The non-secure firmware base is RISC_NONSECURE_BASE_ADDR (0x0068); 0x0054 is the secure one. - MFC_CLOCK_OFF is cleared before RISC_ON, and the firmware status bit (0xF0A4) is polled before SYS_INIT. MFC_VERSION (0xF028) is checked. - The firmware rejects any buffer placed below its own base (ERR_RET with error 0x33, measured). The firmware, the device and instance contexts and also the codec buffers therefore come from one preallocated 64 MiB region with a 32-bit DMA mask, firmware first. An optional "memory-region" makes the vb2 buffers come from the same pool; without it, buffers below the firmware base are reported with a warning. - A missing clock is tolerated: the node's clock is only the ACPM DVFS rate, and the MFC clock controller (CMU_MFC) is not described yet. - The encoder registers differ from v12's (0xF770/0xF774 clash with the decoder's) and nothing programs them yet, so only the decoder node is registered. The firmware is loaded as s5p-mfc-v15.fw; it is the stock mfc_fw.bin from the device's vendor partition, which each owner takes from their own OTA. The vendor driver always runs the decoder with dynamic DPB; that comes in the next patch. This patch was not run on its own: whether the v15 firmware still takes mainline's static DPB is not known. Tested on a Pixel 7 (panther) with the node applied as a runtime DT overlay on the "camera-pool" reserved region, together with the next patch. Not tested: the encoder, VP8/VP9/HEVC/MPEG-4 decode, the System MMU (left in bypass), runtime PM with a real clock, suspend/resume. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
Google's exynos-mfc driver always sets DYNAMIC_DPB (bit 3 of D_INIT_BUFFER_OPTIONS) on v15 and gives the firmware one capture buffer per FRAME_START, through D_DYNAMIC_DPB_FLAG and D_AVAILABLE_DPB_FLAG; D_USED_DPB_FLAG then tells which buffers the firmware still holds as references or for display. Mainline programs every DPB address once at INIT_BUFFERS (static DPB). Do as the vendor driver on v15: skip the DPB addresses at INIT_BUFFERS, and before each frame pick a queued capture buffer that the firmware does not hold (mfc_search_for_dpb() in the vendor driver). If it holds all of them it frees one by displaying it, so any queued buffer may be given then. A frame is not started while no buffer can be given. Tested on a Pixel 7 (panther, MFC v15.6, firmware version 0x44250901), module built from this series on omamobile, node applied as a runtime DT overlay on the "camera-pool" reserved region: - H.264 320x240 Baseline (30 frames), 1920x1080 High (60 frames) and a hard 1920x1080 High stream (300 frames, ~20 Mbit/s): every frame bit-exact against FFmpeg's software decoder. - 1080p at ~70 fps with ~0.09 s of CPU time for the 300 frames, against 8.3 s of CPU time for FFmpeg's software decoder (8 threads). - Clients: a small V4L2 stateful test client, FFmpeg's h264_v4l2m2m (300/300 frames bit-exact with -fps_mode passthrough) and GStreamer's v4l2h264dec. Not tested: VP8/VP9/HEVC/MPEG-4 decode, interlaced or MVC streams, a resolution change in the middle of a stream, seeking (decoder stop and restart in the middle of a stream). Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
Describe the gs201 MFC (v15.6), disabled, from Google's MFC-0 node (kernel/devices/google/gs201, dts/gs201-mfc.dtsi, branch android-gs-pantah-6.1-android16): registers at 0x1c8d0000, GIC SPI 504 level high. Its "mfc" clock is the ACPM firmware's MFC DVFS domain, so that s5p-mfc can raise the rate while it decodes instead of running at the 100 MHz the bootloader leaves; ACPM sets the voltage with it. The two System MMUs in front of it are not described: they stay in bypass, so the MFC sees physical addresses through 32-bit registers, and a board that enables it gives it memory below 4 GiB through memory-region. No power domain and no CMU_MFC gates either: CMU_MFC is not described yet, and the gs201 boards boot with clk_ignore_unused and pd_ignore_unused, which leave them on as the bootloader set them. The node's comment names the CMU_TOP gate that depends on that flag. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The CAPTURE queue is capped at the minimum the stream needs plus MFC_MAX_EXTRA_DPB (5). That fits a fixed DPB, which the firmware takes whole at INIT_BUFFERS. With dynamic DPB (MFC v15) the firmware only holds the frames it still references or has not shown yet, and every other buffer the client keeps, a player's display queue among them, comes out of those five. mpv 0.41 on the Pixel 7 shows the problem: with its default gpu-next output it keeps more decoded frames queued than the five spare ones, the decoder runs out of free buffers after the first frame and the picture freezes. With dynamic DPB, cap the queue at MFC_MAX_BUFFERS (32) only, the driver's own limit for any queue; FFmpeg's h264_v4l2m2m asks for 20 by default. vb2 already falls back to fewer buffers when the memory runs short. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
On the Pixel 7's MFC v15.6 only H.264 has been made to work. The firmware refuses to open an instance for MPEG-1, MPEG-2, MPEG-4, H.263, VP8 and VP9 (OPEN_INSTANCE returns error 0x35, then REQBUFS fails with EIO), and HEVC opens but every frame comes back with a decode error. Each was tried with FFmpeg's *_v4l2m2m decoder on a short clip. Advertising them anyway breaks playback: GStreamer ranks its V4L2 decoders above the software ones, picks v4l2vp9dec for a VP9 file, and the pipeline fails instead of falling back to vp9dec. Leave every compressed format but H.264 out of the v15 list, H.264 MVC included, which wasn't tried. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
S_FMT on the OUTPUT queue caps the bitstream buffer at the variant's CPB size, but hands the application back the size it asked for. The buffers then come out smaller than the format says. GStreamer asks for width * height for a 4K H.264 stream (8294400 bytes), the MFC v15 caps it at 7 MiB, and v4l2h264dec gives up with "buffer size 7340032 is smaller then negotiated size 8294400" and "Failed to allocate required memory". Return the clamped size, as the default branch already does. On the Pixel 7 a 3840x2160 High profile clip then decodes in full and its frames match FFmpeg's software decoder bit for bit. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The decoder's format table sets V4L2_FMT_FLAG_DYN_RESOLUTION and V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM on the coded formats, but vidioc_enum_fmt() only copies the fourcc, so applications never see them. GStreamer's v4l2 decoders read DYN_RESOLUTION to know that the driver sends V4L2_EVENT_SOURCE_CHANGE once the header is parsed. Without it, v4l2h264dec sets up the CAPTURE queue on its own, then takes that first event for a resolution change in the middle of the stream. It stays marked as draining, and the next flush (a seek) throws the CAPTURE pool away and asks for a new one. Copy the flags. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
When a queue is stopped while the context's frame is being decoded, stop_streaming() sets MFCINST_ABORT and waits for FRAME_DONE. The interrupt handler of that frame wakes the context up, releases the hardware and calls try_run, which picks the same context again because its work bit is still set. try_run clears the context's interrupt flags before finding nothing to run in MFCINST_ABORT, so the waiter finds no FRAME_DONE and the STREAMOFF stalls for the full two second timeout, logging "Interrupt (ctx->int_type:0, command:13) timed out". Clear the work bit together with setting MFCINST_ABORT. On the Pixel 7 every seek in GStreamer took two seconds and logged that timeout; with this, three runs of two seeks each logged none and resumed at once. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The bootloader leaves the Pixel 7's MFC at 100 MHz. Decoding only, that gives about 22 fps for 3840x2160 H.264 and 64 fps for 1080p, so 4K cannot play in real time. On gs201 the MFC rate is a DVFS domain of the ACPM firmware, which sets the voltage along with it. Let a variant name the rate to ask of its first clock while the MFC is powered on, and put the previous rate back at power off. MFC v15 asks for 666 MHz, the top of Google's gs201 mfc_freqs table. On the Pixel 7, with the clock below in the device tree: the rate reads 666 MHz while decoding and 100 MHz again afterwards; decoding only, the 4K clip runs at about 70 fps and the 1080p one at about 280 fps; the 4K clip plays on screen through glimagesink at 30 fps with no frame dropped (44 shown and 72 dropped before); and the 300 4K frames and the 1800 frames of a 60 s 1080p clip match FFmpeg's software decoder bit for bit. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The decoder's S_FMT returns EBUSY when either queue is streaming. The stateful decoder interface (dev-decoder.rst, "Capture Setup") has the client set the CAPTURE format after the source change event, while the OUTPUT queue keeps streaming, and Chromium's V4L2 decoder does exactly that: its S_FMT on CAPTURE failed with "vidioc_s_fmt queue busy" and it gave up on the stream. Only refuse a format change on the queue that is streaming. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
With dynamic DPB the plane sizes and strides are only written at INIT_BUFFERS. Under Chromium's V4L2 decoder the first frame decodes, but at the second one those registers read back as 0 and the firmware fails it with error 57, INSUFFICIENT_DPB_SIZE (the name in Google's exynos-mfc driver). The same stream through GStreamer never shows it. Write the sizes and strides again before every frame, next to the frame's DPB addresses. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The decoder only offers multi-planar formats (NV12M and friends), each plane in a buffer of its own. Chromium's V4L2 decoder renders NV12 in one buffer without a copy; for NM12 it sets up an image processor first, and on the Pixel 7 it got no further than that. Offer V4L2_PIX_FMT_NV12 on MFC v15, laid out as V4L2 has it: a 64 byte aligned stride, as Google's NV12N, and the chroma right after the luma's bytesperline * height. The MFC is given the luma address and the chroma one inside the same buffer. The firmware still wants its 64 bytes after each plane (it reports 2089024 and 1044544 bytes as the least for 1920x1088, and fails frames with INSUFFICIENT_DPB_SIZE below that), so the plane sizes it is told include them, and the buffer ends 64 bytes after the chroma. Work the sizes out again when the CAPTURE format changes after the header was parsed, as Chromium does, and count the CAPTURE planes with mem_planes rather than comp_planes. On the Pixel 7, with this format only offered: GStreamer's v4l2h264dec decodes a 20 s and a 60 s 1080p clip (2400 frames) to the same frames as avdec_h264. With all formats offered, GStreamer keeps NV12M and its 2700 frames still match; Chromium 156 (AcceleratedVideoDecoder enabled) plays a 1080p30 clip at 30 fps with no frame dropped and spends 6.8 s of CPU time per 10 s of video, against 9.9 s decoding in software. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
The decoder has no VIDIOC_ENUM_FRAMESIZES. Chromium's V4L2 decoder then assumes it handles up to 1920x1088, and decodes anything larger in software: a 3840x2160 H.264 clip played with the MFC left at its idle clock. Report a stepwise range up to 4096x2304, H.264 level 5.2's largest frame, on MFC v15. On the Pixel 7, Chromium then decodes that 4K clip on the MFC: no frame dropped over the 10 s, seeks included. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
MFC v15 reused v12's instance context sizes, among them 30 KiB for the decoders other than H.264 and HEVC. Google's exynos-mfc driver for gs201 gives those 60 KiB, and 50 KiB to the HEVC and other encoders. With 30 KiB the firmware refuses to open an MPEG-2, VP8 or VP9 instance (OPEN_INSTANCE returns error 0x35). Use Google's sizes on v15. On the Pixel 7 those instances then open. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
After the header is parsed, the driver only checks for H.264 whether the header's buffer holds more than the header, and treats the buffer as used up for every other codec. That buffer usually carries the first picture too: the first HEVC IRAP picture, and in VP8 and VP9 the header is the key frame itself. Dropping it, the firmware fails every HEVC frame with error 190 (SYNC_POINT_NOT_RECEIVED in Google's exynos-mfc driver) and every VP8/VP9 frame with error 34 (NO_KEY_FRAME). Check the consumed bytes for every codec on MFC v15. On the Pixel 7, HEVC, VP8 and VP9 clips then decode to the same frames as FFmpeg's and GStreamer's software decoders. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
With the two previous commits these decode on the Pixel 7. Through FFmpeg's *_v4l2m2m decoders and through GStreamer's v4l2vp8dec, v4l2vp9dec and v4l2h265dec, every frame matches the software decoders (FFmpeg's, vp8dec, vp9dec, avdec_h265): VP9 1920x1080, 450 frames; VP9 3840x2160 (libvpx), 90 frames VP8 1280x720, 450 frames HEVC 1920x1080, 450 frames; HEVC 1920x1080 (x265), 90 frames Chromium 156 decodes the 1080p VP9 clip on the MFC with no frame dropped. MPEG-1/2/4, H.263, VC-1 and H.264 MVC stay off: MPEG-2 and H.263 decode with differences that were not checked further, and MPEG-4 fails to allocate its CAPTURE buffers. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
With dynamic DPB the firmware may keep its whole minimum DPB as references at once. A client that allocates only that minimum plus one or two buffers then has almost none left to display while the next frame decodes. Chromium allocates the driver's minimum plus 2: for a 3840x2160 VP9 clip on the Pixel 7 (minimum 10) it decoded at about 24 fps and dropped 131 of 468 frames at 30 fps. Give such clients at least four buffers more than the minimum. With it Chromium drops 4 of 582 frames of the same clip (looping, the drops are at the loop points), and H.264 3840x2160 still drops none. GStreamer's output stays bit-exact against vp9dec and avdec_h264 at 3840x2160. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
50d6d6c to
525e556
Compare
|
Thank you, Derek. All four points are done, plus two of the smaller ones. The series is now 21 commits:
phone-smoke: unknownKernel
Packages: hyprland 0.57.0-3.4, libcamera 0.7.2-4.23, mesa 1:26.2.4-1.4, omacam 0.3.3-1, omamessages 0.2.2-1, omamobile-apps 0.2.3-1, omamobile-audio 0.5.18-1, omamobile-device-panther 0.1.15-1, omamobile-firstboot 0.1.5-1, omamobile-gnss 0.2.9-1, omamobile-keyring 20261026-1, omamobile-mirror 0.1.1-1, omamobile-omarchy 0.1.41-1, omamobile-owl-widgets 0.1.2-1, omamobile-pixel 0.1.27-1, omamobile-sensors 0.2.1-1, omamobile-settings 0.1.2-1, omamobile-shell 0.2.126-1, omaphone 0.2.3-1, omarchy 4.5.0-3, omatel-agent 0.2.4-2, quickshell 0.3.2.r7.g2b4501a-1 Smaller:
The System MMU question is yours, as you said. I'll update the PR description to match the new order. |
|
My decision on the System MMUs: the MFC stays off by default until its System MMUs are described. Please keep the node That fits the commit split I asked for: the |
Give panther's MFC its own reusable shared-dma-pool of 256 MiB below 4 GiB, and leave the node disabled. The driver takes its 64 MiB for the firmware, contexts and codec buffers from the pool, firmware first, and vb2 serves the OUTPUT and CAPTURE buffers from the rest. The firmware rejects any buffer below its own base, which a single pool rules out. 256 MiB is what a 4K stream needs: a 3840x2160 frame in single-buffer NV12 is 12.4 MiB in one block, and with 128 MiB next to the firmware GStreamer got no frame at all out of a 4K clip. The pool is reusable, so its pages serve movable allocations while nothing is decoded. The node stays disabled until its two System MMUs are described: in bypass, Google's closed firmware parsing untrusted video can reach all memory below 4 GiB. An overlay in omarchy-mobile (device/panther) enables it for owners who want hardware video. Tested on a Pixel 7 booting a DTB with this node and pool enabled, the driver built from this series as a module and the firmware taken from the phone's own OTA. The pool is reserved below 4 GiB and the driver probes from it on its own. H.264, HEVC, VP8 and VP9 decode bit-exact against FFmpeg's and GStreamer's software decoders (1080p and 4K), and Chromium plays 1080p and 4K H.264 and VP9 on the MFC. Assisted-by: LLM Signed-off-by: Ángel René <17342133+angelrenefm@users.noreply.github.com>
525e556 to
da2c033
Compare
|
Thank you, Derek. Done as you decided. The last commit is now The overlay is in omarchy-mobile#811:
|
|
Thank you, Ángel. This is exactly the shape I asked for, and quickly too.
I checked it here:
Thank you also for the smoke report and for saying plainly which tree that boot used, and for the binding and clock-comment fixes. I'll merge this one first. Then #256, once you've rebased it onto |
|
Merged by hand: GitHub's rebase-merge of the 21 commits timed out twice, so they're on |
Hardware H.264, HEVC, VP8 and VP9 decode on the Pixel 7. The gs201 has an MFC v15.6, and its register map is close enough to v12's
for mainline's s5p-mfc. This avoids porting Google's exynos-mfc driver. The MFC firmware is the stock
mfc_fw.binfrom the vendor partition. It is plain and unsigned, and it is loaded ass5p-mfc-v15.fw.Commits:
media: s5p-mfc: signal the parsed stream format with V4L2_EVENT_SOURCE_CHANGE: a general fix. The statefuldecoder spec has the client wait for this event after the header is parsed, and s5p-mfc only sent it on a
resolution change in the middle of a stream.
media: s5p-mfc: round private buffer sizes up to whole pages: a general fix for the preallocated-memory bitmap.dt-bindings: media: s5p-mfc: Add the Google GS201 MFC: the compatible, with Tesla FSD's single "mfc" clock andup to two System MMUs.
media: s5p-mfc: Add support for the MFC v15 of Google GS201: the firmware base goes in 0x68 (0x54 is the secureone), the version and firmware status are checked before SYS_INIT, and the clock is optional. All the buffers come
from one region that starts with the firmware, because the firmware rejects any buffer below its own base (measured:
ERR_RET, error 0x33). Only the decoder node is registered. A missing clock is tolerated.
media: s5p-mfc: decode with dynamic DPB on MFC v15: one capture buffer is given per frame, as Google's driver does.arm64: dts: exynos: google: gs201: Add the MFC: the node in gs201.dtsi, disabled, with its clock (the ACPMfirmware's MFC DVFS domain,
GS201_CLK_ACPM_DVFS_MFC, which sets the voltage with the rate) and no iommus orpower domain. Its comment names the CMU_TOP gate that stays on only through
clk_ignore_unused.media: s5p-mfc: let dynamic DPB clients queue more capture buffers: with dynamic DPB the CAPTURE queue is nolonger capped at the stream's minimum plus 5, only at 32. Players keep decoded frames queued for display, and
five spare buffers were not enough for mpv: with its default gpu-next output the picture froze after the first
frame. Found when testing with real players (below).
media: s5p-mfc: offer only H.264 decode on MFC v15: the firmware refuses to open an instance for MPEG-1/2/4,H.263, VP8 and VP9 (OPEN_INSTANCE error 0x35), and HEVC returns a decode error on every frame. Advertising them
made GStreamer pick
v4l2vp9decfor a VP9 file and fail instead of falling back to software.media: s5p-mfc: report the clamped OUTPUT buffer size: a general fix. S_FMT capped the bitstream buffer at theCPB size but returned the size the client asked for. GStreamer asks for width * height for 4K (8294400 bytes),
gets 7 MiB buffers and gives up with "Failed to allocate required memory". With the fix, a 3840x2160 High
profile clip decodes in full, bit-exact (checked on the first 9 frames), in the same 128 MiB pool.
media: s5p-mfc: report the coded format flags in ENUM_FMT: a general fix. The format table setsV4L2_FMT_FLAG_DYN_RESOLUTION, but ENUM_FMT never copied the flags. Without it, GStreamer takes the firstV4L2_EVENT_SOURCE_CHANGE(commit 1) for a resolution change, and on the next seek it throws the CAPTURE poolaway and fails to get a new one.
media: s5p-mfc: don't lose the FRAME_DONE that stop_streaming waits for: a general fix. A STREAMOFF in themiddle of a frame waited for FRAME_DONE, but try_run cleared the context's interrupt flags first, so every seek
stalled for the 2 s timeout ("Interrupt (ctx->int_type:0, command:13) timed out").
media: s5p-mfc: raise the MFC clock while it is in use on MFC v15: the bootloader leaves the MFC at 100 MHz.A variant can now name a rate for its first clock while the MFC is powered on; the old rate comes back at power
off. v15 asks for 666 MHz, the top of Google's gs201
mfc_freqs.media: s5p-mfc: let the CAPTURE format be set while OUTPUT streams: a general fix. S_FMT returned EBUSY ifeither queue streamed; the stateful decoder interface sets the CAPTURE format with OUTPUT streaming, and
Chromium does.
media: s5p-mfc: set the DPB plane sizes for every frame on MFC v15: under Chromium the sizes written atINIT_BUFFERS read back as 0 at the second frame, which the firmware fails with error 57
(INSUFFICIENT_DPB_SIZE).
media: s5p-mfc: decode to single-buffer NV12 on MFC v15: Chromium renders single-buffer NV12 directly; withNM12 it stopped at setting up an image processor. Laid out as V4L2 NV12 (64 byte stride, chroma right after
the luma), with the 64 byte margins the firmware asks for.
media: s5p-mfc: enumerate the coded frame sizes on MFC v15: up to 4096x2304 (H.264 level 5.2). WithoutVIDIOC_ENUM_FRAMESIZES Chromium assumed 1920x1088 and decoded 4K in software.
media: s5p-mfc: give MFC v15 the context buffer sizes of Google's driver: 60 KiB for the decoders other thanH.264/HEVC (v12's 30 KiB made OPEN_INSTANCE fail with error 0x35), 50 KiB for the HEVC and other encoders.
media: s5p-mfc: keep the frame that came with the header on MFC v15: the header's buffer was dropped for everycodec but H.264, taking the first HEVC IRAP picture and the VP8/VP9 key frame with it (errors 190 and 34).
media: s5p-mfc: offer VP8, VP9 and HEVC decode on MFC v15: they now decode bit-exact (below). Commit 8 hadleft them out.
media: s5p-mfc: leave capture buffers for display with dynamic DPB: a client now gets at least four CAPTUREbuffers more than the minimum. Chromium allocates the driver's minimum plus 2, and with dynamic DPB the firmware
can hold the whole minimum as references. For 4K VP9 (minimum 10) that left Chromium about 24 fps (below).
arm64: dts: exynos: google: gs201-panther: Enable the MFC: panther enables the node with its own reusable256 MiB
mfc-poolbelow 4 GiB. Single-buffer NV12 4K frames don't fit in 128 MiB next to the firmware. This isthe only commit that changes panther's DTB.
Tested on my Pixel 7 (kernel
7.3.0-rc6-omamobile-push1-g44048a9aa795), with s5p-mfc built from these commits as amodule. The decoding was first measured with the node applied as a runtime DT overlay, with
memory-regionon thecamera pool that this kernel carries. Then I booted a DTB with this series' node and pool (only
vendor_kernel_bootchanged, sameImage):mfc-poolwas reserved at 0xc8000000, s5p-mfc loaded on its own from the node, and the 1080p stream belowdecoded bit-exact again; s5p-mfc built from this branch, unchanged, decoded all three clips bit-exact on that boot too.
That boot's DTB was my phone's tree, not
omamobilealone: it also carries the display pool from #181, the camerapool, the GNSS and torch nodes and the overlays my phone uses, so the three pools below 4 GiB (display, camera, MFC)
were reserved together without trouble. The firmware was
s5p-mfc-v15.fw, which ismfc_fw.binfrom my own OTA(FW_VERSION 0x44250901, MFC_VERSION 0x15060000).
Measured before commit 12, with the MFC at 100 MHz:
Players, on screen (Hyprland, 1080p30 High with B-frames, 20 s, made with x264; CPU time over 10 s of playback):
v4l2h264decon its own)gst-launch-1.0 … v4l2h264dec ! glimagesink(zero copy)--hwdec=v4l2m2m-copy,--vo=gpumpv only gets the frames through a copy: Arch's FFmpeg doesn't export DRM_PRIME from h264_v4l2m2m, and reading the
uncached buffers costs most of what the MFC saves, so it isn't worth turning on there.
waylandsinkdoesn'tnegotiate (the compositor offers no NV12 dmabuf here);
glimagesinkand Clapper do.Seeking in Clapper, after commits 10 and 11 and with GStreamer 1.28.9:
clip with AAC sound. In a scripted test (seeks to 7 s, 3 s and 15 s), the 27 frames after the seeks match
avdec_h264bit for bit, and three runs logged no MFC timeout.gst_v4l2_buffer_pool_flush()holds the pool's object lock across streamoff. Releasing the queued buffers therecan drop the last reference to a memory group, and its group-released handler resurrects the buffer and queues it
again, which takes the same lock. My fix blocks that handler during streamoff; the next streamon queues the
missing buffers anyway. It needs fixing in GStreamer itself.
Decode speed, decode only (
v4l2h264dec ! fakesink sync=false):With commit 12 the rate reads 666 MHz while decoding and 100 MHz again afterwards. The 4K clip plays on screen
through
glimagesinkat 30 fps with no frame dropped (before: 44 shown and 72 dropped). At 666 MHz, the 300 4Kframes and the 1800 frames of a 60 s 1080p clip match
avdec_h264bit for bit.Chromium 156 (Arch Linux ARM) uses the MFC with
--enable-features=AcceleratedVideoDecoderand commits 13-15: a1080p30 clip plays at 30 fps with no frame dropped, three seeks work, and Chromium spends 6.8 s of CPU time per
10 s of video against 9.9 s decoding in software. With commits 16 and 21 a 3840x2160 clip plays in Chromium on the
MFC (666 MHz while it plays) with no frame dropped, two seeks included. VP8, VP9 and HEVC (commits 17-19), through FFmpeg's
*_v4l2m2mdecoders and GStreamer'sv4l2vp8dec,v4l2vp9decandv4l2h265dec, every frame equal to the software decoders: VP9 1080p (450 frames) and 4K (libvpx, 90),VP8 720p (450), HEVC 1080p (450, and 90 from x265). Chromium decodes the 1080p VP9 clip on the MFC with no frame
dropped. A 3840x2160 VP9 clip (libvpx, looping) dropped 131 of 468 frames in Chromium before commit 20, and 4 of
582 after it, at the loop points. H.264 3840x2160 still drops none (300/300), and GStreamer's 4K output stays
bit-exact against vp9dec (90 frames) and avdec_h264 (300). Widths that are not a multiple of 16 decode bit-exact too (854x480: 150/150 frames against
FFmpeg, through
h264_v4l2m2mand throughv4l2h264decread without conversion). With all formats offered GStreamer keeps NV12M, and its 2700frames above still match.
Clients:
-c:v h264_v4l2m2m: 300/300 frames bit-exact, with-fps_mode passthrough. Without that option, FFmpegdrops frames from a raw .h264 that has no timestamps;
v4l2h264dec: it decodes, and spends 2.6 s of CPU time copying out of the buffers.tools/kernel-pr-check --build --mergeon the 21 commits, merged ontoomamobile(2eda146):checkpatch --strict is clean, except for the generic "does MAINTAINERS need updating?" on the new
regs-mfc-v15.h. That file is already covered by the s5p-mfc entry.How to try it:
mfc_fw.binfrom your own vendor image (/firmware/mfc_fw.binin vendor.img) to/usr/lib/firmware/s5p-mfc-v15.fw.s5p-mfc.ko). dmesg should showdecoder registered as /dev/videoN.ffmpeg -c:v h264_v4l2m2m -i clip.h264 -fps_mode passthrough -f rawvideo -pix_fmt nv12 out.nv12, and comparethe result with
ffmpeg -i clip.h264 -f rawvideo -pix_fmt nv12 ref.nv12.Not checked:
kUnsupportedCodecfor HEVC (NOTIMPLEMENTED()inv4l2_stateful_video_decoder.cc, still so onmain).MPEG-4 fails REQBUFS on CAPTURE.
V4L2_DEC_CMD_START: not supported, as before. Chromium sends it once per stream and logs an error, but plays on.bootloader leaves them under
clk_ignore_unusedandpd_ignore_unused; only the DVFS rate is set (commit 12).Runtime PM and suspend/resume with the MFC in use.
Conflict with #181: both add a pool to
reserved-memoryings201-panther.dtsat the same place (itsdisplay-pool, thismfc-pool). Keep both nodes; that is the combination booted above.I used AI help (Claude) to work on this.