Get frame numbers in HTML5 Video

Question

I am trying to capture each frame number of the video however it looks like there is no way of achieving it. So I started my own clock to match the frame numbers of the vid

Answer 1

Starting in M83, Chromium has a requestVideoFrameCallback() API, which might solve your issue. You can use the mediaTime to get a consistent timestamp, as outlined in this Github issue. From there, you could try something like this:

var frameCounter = (time, metadata) => {
   let count = metadata.mediaTime * frameRate;
   console.log("Got frame: " + Math.round(count));

   // Capture code here.

   video.requestVideoFrameCallback(frameCounter);
}

video.requestVideoFrameCallback(frameCounter)

This will only fire on new frames, but you may occasionally miss one (which you can detect from a discontinuity in the metadata.presentedFrames count). You might also be slightly late in capturing the frame (usually 16ms, or one call to window.requestAnimationFrame() later than when the video frame is available).

If you're interested in a high level overview of the API, here's a blogpost, or you can take a look at the API's offical github.