Movement by a single frame in CMTime and AVFoundation

Question

I'm attempting to play a video with AVFoundation. I am using the following code for a button that advances the playback by one frame.

It works intermittently, on some executions it will do the right thing and advance one frame, but most times I will have to press the button 3 or 4 times before it will advance a frame.

This makes me think it is some kind of precision issue, but I can't figure out what it is. Each time it is run the new CMTime appears to be advancing by the same amount.

My other theory is that it could be caused by the currentTime not being set to an exact frame boundary at my frame rate (caused by seeking through the video). But I don't know how to "snap" to the nearest frame at my frame rate.

AVAssetTrack *videoTrack = ...;
Float64 frameRate = [videoTrack nominalFrameRate];

CMTime currentTime = [self.playerItem currentTime];
CMTime oneFrame = CMTimeMakeWithSeconds(1.0 / frameRate, currentTime.timescale);
CMTime added = CMTimeAdd(currentTime, oneFrame);

[self.player seekToTime:added toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero];

Thanks for your help!

Answer 1

The answer to your question is in the second line of code where you remove the "nominal" from nominalFrameRate.

Your code assumes that the frame rate in your video is constant. This is incorrect.

A video can have a constant fps, but it doesn't have to. E.g. if you shoot a film with your phone in variable lighting conditions then the frame exposure time can change and your frame rate will vary inversely.

To "snap" to frame boundaries, you need to use an AVAssetReader to step through the individual frames. Each frame is tagged with a Presentation Time Stamp (its "t"), but if you're stepping that doesn't matter so much, unless you're updating a position marker.

Answer 2

It isn't very obvious (since strangely it's only available in AVPlayerItem rather than AVPlayer) but if the AVPlayerItem returns YES for canStepForward/canStepBackward then you can call stepByCount:(NSInteger)stepCount: to move forwards or backwards by a certain number of frames.