Recorded QuickTime movie video plays too quickly

[ryandesign]

Describe the bug
Using the built in QuickTime movie recording capability, the resulting video plays too quickly.

To Reproduce
Steps to reproduce the behavior:

1. Start recording video.
2. After 15 seconds, stop recording video.
3. Open the video in QuickTime Player.

Expected behavior
The video should be 15 seconds long and play for 15 seconds.

Actual behavior
The video is 15 seconds long but plays at an accelerated rate, finishing after 12 seconds, and then there is nothing for the last 3 seconds.

Versions (please complete the following information):

• OS: macOS 12.7.5
• Mariani: 1.5 (2)

[ryandesign]

The audio portion of the movie however plays at the correct speed, thus the audio and video are not synchronized.

[ryandesign]

My suspicion is:

1. It wants to record at 60 fps.
2. My computer is not fast enough to record at 60 fps, so fewer frames are recorded.
3. The video is nevertheless marked as being 60 fps and plays the frames it did capture back at 60 fps.

When there is a lot of motion, even fewer frames get captured. For example, recording 28 seconds of:

1 PRINT "1234567"; : GOTO 1

gives me only 14 seconds of video and 14 seconds of blank or freezeframe.

[sh95014]

I'm not able to reproduce the issue on my M1 Mac. Can you attach a short sample video exhibiting the issue?

Also, I think a potential fix might be to let the user select the desired frame rate. If I send you a source code patch, are you able to build and see if it solves your issue?

[ryandesign]

I haven't tried building it yet, but I can try.

There is definitely something abnormally slow about my Mac and I'm not surprised you can't reproduce the problem on a modern machine. My Mac here is a 15" 2012 MacBook Pro with Retina Display running macOS 12 via OCLP.

Being able to select the frame rate sounds useful, but I still wouldn't want any frames to be dropped and audio and video to be out of sync, regardless of the frame rate or how slow the computer is.

Is there a possibility to let the emulation slow down to guarantee all frames are always recorded?

[sh95014]

Slowing down the emulation is possible. Right now, there are two vaguely-related timers recordingTimer and runLoopTimer in EmulatorViewController, so your idea would involve synchronizing the two. That could make games suck though, and feels like tail wagging the dog...

The sync issue might be a bug, because I'd expect dropped frames (stuttering) instead if the CPU can't keep up. Maybe I'm computing the presentationTime incorrectly...

            [self.videoWriterAdaptor appendPixelBuffer:pixelBuffer
                                  withPresentationTime:CMTimeMake(self.videoWriterFrameNumber * (CMTIME_BASE / TARGET_FPS), CMTIME_BASE)];

Anyway, if you want to experiment with recording at a lower fps, you should be able to change TARGET_FPS below (in EmulatorViewController) to a lower number:

        self.recordingTimer = [NSTimer scheduledTimerWithTimeInterval:(1.0 / TARGET_FPS) target:self selector:@selector(recordingTimerFired) userInfo:nil repeats:YES];

[ryandesign]

That could make games suck though

My intention is that if the user finds that it is too slow, they would then select a lower frame rate until it's not too slow. But regardless of the frame rate they use, they would end up with an accurate recording. Currently recordings can be corrupted without notification which makes all recordings suspect.

The sync issue might be a bug, because I'd expect dropped frames (stuttering) instead if the CPU can't keep up.

I see that you have a comment about dropped frames too:

AppleWin/source/frontends/mariani/EmulatorViewController.mm

Lines 299 to 302 in a3c51a0

	// if we realize that we've skipped a frame (i.e., videoWriter is
	// not nil but readyForMoreMediaData is false) should we also
	// increment videoWriterFrameNumber?
	self.videoWriterFrameNumber++;

I think I agree with the comment: Currently this code only runs when self.videoWriterInput.readyForMoreMediaData is true and status == kCVReturnSuccess && pixelBuffer != NULL but maybe self.videoWriterFrameNumber should increment even outside of some of those conditions. Maybe I can try playing around with that.

[sh95014]

I believe the inaccuracy of the recording comes from my naive setting of the "presentation time" of each video frame. Basically, it assumes that the timer handler will actually be running at t+1/60s, t+2/60s, t+3/60s, etc, so the video frame it captures is inserted at exactly those times. If the timer handler is actually late as in your case, the current video frame is captured but inserted at an earlier time.

I'm developing a fix (that throws away videoWriterFrameNumber entirely) in the x-screen-recording-thread branch that I'll tweak a bit more before sending you a build to try. That fix should insert the video frame at the correct point in time, and queue up the processing in a lower-priority thread to avoid stalling the emulation. In theory that should get you 60 fps captures, even if it won't complete outputting until sometime after you stop recording...

[sh95014]

@ryandesign Here's a combined build of the fixes from #76 and the video writer frame setting. Rather than relying on two separate timers:

• the main loop should now sleep if it finished emulation for the frame
• when recording, record a frame only if there's time left in the frame and a prior frame isn't already encoding

So on really slow machines, this should still be able to run the emulation at real time and the recording at "best effort". It'd be great if you could try it out and let me know the frame rate of the recording that you can manage (Quicktime Player can toggle between time and frames display from the View menu). If it still can't keep up, I can add an option to sacrifice real-time emulation.

This build should show "Y-Mariani 1.5 (3)" in the About box.

Mariani-screen-recording-1.zip

[sh95014]

I expect this to be fixed with the 1.6 (1) release. Please reopen if the issue persists. Thanks for the report and help reproducing it!

[sh95014]

Doesn't look like the x-screen-recording-thread was ever merged into the macOS branch!

[sh95014]

Please try the "XMariani-17.1-1.zip" attached to #76

(Also interested in how well the three new video quality options do on your computer...)

[sh95014]

Fix also included in https://github.com/sh95014/AppleWin/releases/tag/Mariani-1.7-1 release now.