video: MJPEG at 30 fps next to a video stream up to 1920 wide

docs/VIDEO.md

@@ -95,7 +95,7 @@ The H.264 encoder takes only 4:2:0. The capture bridge has its own 4:2:2 → 4:2
 
 The capture mode can only change when the capture starts. It is 4:2:0 while an H.264 stream encodes the whole input at the input rate, without mirror or flip and the setting is on. Any even input width works: the writer stores each row at the pitch the frame description gives, the width rounded up to 64 bytes (1088 for a 1080 wide portrait). When a new video encoder, a codec change (H.264 ↔ H.265), the setting or a lower stream rate asks for the other mode, the capture restarts once in it: a short gap and a new key frame. MJPEG or frame detection joining or leaving never changes the mode; without a video stream the mode stays until the next capture start. A 4:2:0 frame whose layout differs from the expected one (that pitch for both planes, VU plane right after the Y plane) is logged once and H.264 reads the VPSS output until the capture restarts.
 
-Both paths are on by default and only for streams at the input size and rate; MJPEG, scaled streams, a stream rate below the input rate (the VPSS drops the frames in between) and mirrored or flipped capture stay on the VPSS. When MJPEG or frame detection reads the same output (above 1920 pixels the video and MJPEG share one) in 4:2:2 capture, or frame detection in 4:2:0 capture, that output returns to the VPSS until they have been gone for 2 s. MJPEG on its own output (the 4:2:2 choice, without frame detection) leaves the video output straight from capture, with H.265 and with H.264 in 4:2:0 capture alike. Next to a video stream the server sends MJPEG at up to 27 fps and its VPSS output runs at 30; next to a stream wider than 1920 (MJPEG then stays at 1920 × 1080) 25 and 27. The JPEG reader waits for its frame and encodes outside the capture lock, so the video stream reads on meanwhile; video reads leave MJPEG's queued frame in place, and an MJPEG read no longer takes a frame from an output straight from capture (that dropped the video stream's next frame and logged a VI error on every empty read). Each JPEG still costs the video stream about one frame. Measured on the stand at 1080p90 with H.264 and an MJPEG viewer: H.264 78-85 fps next to MJPEG at 27 fps (90 alone). At 2560 × 1440 and 60 fps, H.264 45 fps next to MJPEG at 25 (60 alone): the video stream pre-submits its next frame next to MJPEG on its own output, and the VI keeps three capture surfaces up to that size. The server's MJPEG rate stays below the rate of MJPEG's VPSS output: with the server at 30 over an output at 27 the JPEG reader waited for frames and H.264 fell to 22. When the video output returned to the VPSS it was H.264 68 fps, MJPEG 25.7 fps and 1.61 GB/s; MJPEG without a limit took 53 fps and left H.264 32. With the 4:2:0 choice MJPEG shares the video output: in 4:2:0 capture it takes the H.264 stream's NV21 capture frame in a lease of its own (limited range, as MJPEG on a shared output always is next to video) and the VPSS stays off. **Video settings → Video straight from capture** turns both off; the setting is `/etc/kvm/hevc_vi_direct` (`0` off, the name predates H.264) and applies within a second, with one new key frame (H.264 with a capture restart). The switch shows for H.265 when the codec driver has patch 0011 (`/sys/module/cvi_vc_driver/parameters/nk_packed_src`) and for H.264 when the VI driver has patch 0013 (`nk_yuv420`; GET `/api/vm/screen` reports them as `hevcDirectSupported` and `viDirectH264Supported`).
+Both paths are on by default and only for streams at the input size and rate; MJPEG, scaled streams, a stream rate below the input rate (the VPSS drops the frames in between) and mirrored or flipped capture stay on the VPSS. When MJPEG or frame detection reads the same output (above 1920 pixels the video and MJPEG share one) in 4:2:2 capture, or frame detection in 4:2:0 capture, that output returns to the VPSS until they have been gone for 2 s. MJPEG on its own output (the 4:2:2 choice, without frame detection) leaves the video output straight from capture, with H.265 and with H.264 in 4:2:0 capture alike. Next to a video stream the server sends MJPEG at up to 30 fps and its VPSS output runs at 33; next to a stream wider than 1920 (MJPEG then stays at 1920 × 1080) 25 and 27. With H.265 straight from capture and MJPEG at 30, 1280 × 720 ran at 107-116 fps (120 Hz), 1920 × 1080 at 80-85 (90 Hz) and 57-60 (60 Hz), as with MJPEG at 27; at 2560 × 1440, 30 over 33 was worse than 25 over 27. The JPEG reader waits for its frame and encodes outside the capture lock, so the video stream reads on meanwhile; video reads leave MJPEG's queued frame in place, and an MJPEG read no longer takes a frame from an output straight from capture (that dropped the video stream's next frame and logged a VI error on every empty read). Each JPEG still costs the video stream about one frame. Measured on the stand at 1080p90 with H.264 and an MJPEG viewer: H.264 83-87 fps next to MJPEG at 30 fps (90 alone). At 2560 × 1440 and 60 fps, H.264 45 fps next to MJPEG at 25 (60 alone): the video stream pre-submits its next frame next to MJPEG on its own output, and the VI keeps three capture surfaces up to that size. The server's MJPEG rate stays below the rate of MJPEG's VPSS output: with the server at 30 over an output at 27 the JPEG reader waited for frames and H.264 fell to 22. When the video output returned to the VPSS it was H.264 68 fps, MJPEG 25.7 fps and 1.61 GB/s; MJPEG without a limit took 53 fps and left H.264 32. With the 4:2:0 choice MJPEG shares the video output: in 4:2:0 capture it takes the H.264 stream's NV21 capture frame in a lease of its own (limited range, as MJPEG on a shared output always is next to video) and the VPSS stays off. With H.265 at 1920 × 1080 and 60 fps on that shared output, MJPEG at 30 gave H.265 30 fps and MJPEG 20 (at 27: 33 and 14-17). **Video settings → Video straight from capture** turns both off; the setting is `/etc/kvm/hevc_vi_direct` (`0` off, the name predates H.264) and applies within a second, with one new key frame (H.264 with a capture restart). The switch shows for H.265 when the codec driver has patch 0011 (`/sys/module/cvi_vc_driver/parameters/nk_packed_src`) and for H.264 when the VI driver has patch 0013 (`nk_yuv420`; GET `/api/vm/screen` reports them as `hevcDirectSupported` and `viDirectH264Supported`).
 
 ## Colour
 

server/service/stream/mjpeg/streamer.go

@@ -205,12 +205,14 @@ func (s *Streamer) getClients() []*mjpegClient {
 }
 
 // While an H.264 or H.265 stream has viewers, MJPEG runs at most at this
-// rate, and a little lower next to a stream wider than 1920 (2560x1440): each
-// JPEG costs the video stream about one frame (1080p90 H.264 alone 91 fps,
-// next to MJPEG at 53 fps only 32 fps). The native library runs MJPEG's own
-// VPSS output a little above these (30 and 27).
+// rate, and lower next to a stream wider than 1920 (2560x1440): each JPEG
+// costs the video stream about one frame (1080p90 H.264 alone 91 fps, next
+// to MJPEG at 53 fps only 32 fps). Up to 1920 wide 30 cost nothing measurable
+// against 27 (H.265 1080p90 80-85, 1080p60 57-60, 720p120 107-116); at
+// 2560x1440 30 over 33 was worse than 25 over 27. The native library runs
+// MJPEG's own VPSS output a little above these (33 and 27).
 const (
-	mjpegNextToVideoFPS     = 27
+	mjpegNextToVideoFPS     = 30
 	mjpegNextToWideVideoFPS = 25
 )
 

server/service/stream/mjpeg/streamer_test.go

@@ -236,8 +236,8 @@ func TestMjpegRateNextToVideo(t *testing.T) {
 		wide   bool
 		want   int
 	}{
-		{90, false, false, 90}, {90, true, false, 27}, {60, true, false, 27}, {27, true, false, 27},
-		{25, true, false, 25}, {60, true, true, 25}, {25, true, true, 25}, {20, true, true, 20},
+		{90, false, false, 90}, {90, true, false, 30}, {60, true, false, 30}, {30, true, false, 30},
+		{27, true, false, 27}, {60, true, true, 25}, {25, true, true, 25}, {20, true, true, 20},
 		{60, false, true, 60}, {0, true, false, 1}, {0, false, false, 1},
 	} {
 		if got := mjpegRate(c.screen, c.video, c.wide); got != c.want {

support/sg2002/additional/kvm/src/kvm_vision.cpp

@@ -1242,9 +1242,9 @@ static uint64_t mjpeg_last_video_ms = 0, mjpeg_last_jpeg_ms = 0;
 static unsigned mjpeg_last_video_width = 0, mjpeg_last_jpeg_width = 0;
 static bool mjpeg_resolution_limited = false;
 // MJPEG's own output next to a video stream: a little above the rate the
-// server reads it at (mjpegNextToVideoFPS 27, next to a stream wider than
+// server reads it at (mjpegNextToVideoFPS 30, next to a stream wider than
 // 1920 mjpegNextToWideVideoFPS 25).
-static constexpr int mjpeg_next_to_video_fps = 30;
+static constexpr int mjpeg_next_to_video_fps = 33;
 static constexpr int mjpeg_next_to_wide_video_fps = 27;
 static uint64_t mjpeg_monotonic_ms()
 {