Can usb_host_uvc support depth cameras (e.g., RealSense D405)? (IEC-511)

[HeyGoda]

A depth camera has multiple video streams; for example, a RealSense D405 will be recognized as 6 devices on a Linux system：

$ lsusb
Bus 004 Device 019: ID 8086:0b5b Intel Corp. Intel(R) RealSense(TM) Depth Camera 405 
$ ls /dev/video*
/dev/video0  /dev/video1  /dev/video2  /dev/video3  /dev/video4  /dev/video5

It contains the following video streams:
``
$ v4l2-ctl --list-formats-ext -d /dev/video0
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'Z16 ' (16-bit Depth)
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)

$ v4l2-ctl --list-formats-ext -d /dev/video2
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'GREY' (8-bit Greyscale)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
[1]: 'UYVY' (UYVY 4:2:2)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
[2]: 'GREY' (8-bit Greyscale)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
[3]: 'Y8I ' (Interleaved 8-bit Greyscale)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
[4]: 'Y12I' (Interleaved 12-bit Greyscale)
	Size: Discrete 1288x808
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.067s (15.000 fps)

$ v4l2-ctl --list-formats-ext -d /dev/video4
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'YUYV' (YUYV 4:2:2)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.200s (5.000 fps)

Can usb_host_uvc support all these video stream formats?

[github-actions]

Hi @HeyGoda! Please be aware that the following suggestions are generated by a bot and haven't been fact-checked by Espressif Systems. We hope that this message will help you until an Espressif Engineer looks at your issue.

---

This is a great question about usb_host_uvc compatibility with depth cameras like the RealSense D405. Based on the available documentation, there are two key limitations to address:

1. Multiple Simultaneous Video Streams

The usb_host_uvc component does support multiple video streams as a feature. [USB Host UVC Driver] However, the knowledge sources do not provide details on whether it can handle the specific multi-function interface topology of a RealSense D405 (which exposes 6 separate video devices on Linux). This would need to be tested.

2. Pixel Format Support

The RealSense D405 exposes several pixel formats that are not standard MJPEG, including:

• Z16 (16-bit Depth) — proprietary/non-standard
• GREY (8-bit Greyscale)
• UYVY / YUYV (YUV 4:2:2)
• Y8I, Y12I (interleaved greyscale) — proprietary Intel formats

Regarding format support:

• For ESP32-S2/S3, the usb_stream component only supports MJPEG output from UVC cameras. [USB Stream Component]
• For ESP32-P4, the usb_host_uvc component supports MJPEG, YUY2, H264, and H265 formats. [USB Host Solutions]

None of the depth-specific formats (Z16, Y8I, Y12I) are listed as supported. The UYVY/YUYV formats may partially overlap with YUY2 support on ESP32-P4, but this is not explicitly confirmed for the RealSense use case.

Summary

Requirement	Supported?
Multiple video streams	Partially (feature exists, RealSense topology untested)
Z16 (Depth) format	❌ Not listed as supported
GREY / Y8I / Y12I formats	❌ Not listed as supported
UYVY/YUYV (YUV 4:2:2)	Possibly on ESP32-P4 only
MJPEG	✅ Supported

In short, usb_host_uvc is unlikely to support all the video stream formats of a RealSense D405 out of the box, particularly the depth-specific (Z16, Y8I, Y12I) and greyscale formats. The knowledge sources do not contain enough information to confirm full compatibility with this device, and it is recommended to open a feature request or check directly with Espressif's engineering team via the esp-usb issues tracker.

[igi540]

Hi @HeyGoda ,
Based on the current code, usb_host_uvc does not fully support RealSense D405-class depth cameras today.

Multiple streams are only partially supported. The driver can search multiple VideoStreaming interfaces inside one UVC function when opening a stream, but uvc_stream_index is a UVC function index, not a Linux /dev/videoX index. Also, uvc_host_get_frame_list() currently parses only the first VS interface in that function, so stream enumeration is incomplete for cameras with several VS interfaces.

More importantly, the format support is hardcoded. The driver currently recognizes only MJPEG, YUY2, H264, and H265. The descriptor parsing and probe/commit negotiation both depend on that mapping, so formats such as Z16, GREY/Y8, Y8I, Y12I, and UYVY are not selectable today and will not be reported by the frame-list API.

The frame receive path itself mostly just reassembles UVC payloads into raw buffers, so the main gap is in descriptor/API support for those depth/IR format GUIDs, not in packet handling.

So the practical answer is: some streams might work if the camera exposes them as MJPEG/YUY2/H264/H265, but D405 depth/IR formats are not supported out of the box, and the driver does not currently support all of the listed stream formats.

[HeyGoda]

Hi @HeyGoda , Based on the current code, usb_host_uvc does not fully support RealSense D405-class depth cameras today.

Multiple streams are only partially supported. The driver can search multiple VideoStreaming interfaces inside one UVC function when opening a stream, but uvc_stream_index is a UVC function index, not a Linux /dev/videoX index. Also, uvc_host_get_frame_list() currently parses only the first VS interface in that function, so stream enumeration is incomplete for cameras with several VS interfaces.

More importantly, the format support is hardcoded. The driver currently recognizes only MJPEG, YUY2, H264, and H265. The descriptor parsing and probe/commit negotiation both depend on that mapping, so formats such as Z16, GREY/Y8, Y8I, Y12I, and UYVY are not selectable today and will not be reported by the frame-list API.

The frame receive path itself mostly just reassembles UVC payloads into raw buffers, so the main gap is in descriptor/API support for those depth/IR format GUIDs, not in packet handling.

So the practical answer is: some streams might work if the camera exposes them as MJPEG/YUY2/H264/H265, but D405 depth/IR formats are not supported out of the box, and the driver does not currently support all of the listed stream formats.

Are there plans to support the depth/IR format?

[tore-espressif]

@HeyGoda Thank you for the request.

At this point, there are no plans to add support for additional formats. We will keep this issue open to track demand and revisit prioritization if there is broader interest.

From an implementation perspective, the UVC driver is responsible only for format negotiation and descriptor parsing. The actual payload handling is left to the application. Therefore, adding support for a new format primarily requires extending:

the format negotiation logic (probe/commit handling), and
the descriptor parsing to recognize the new format.

No fundamental changes to the transfer layer are expected unless the new format introduces different endpoint or bandwidth requirements.