How to perfectly sync two or more html5 video tags?

ぐ巨炮叔叔 提交于 2019-12-05 01:54:41

Disclaimer: I'm not a video guy, but here are some thoughts anyway.

If they need to be absolutely perfect...you are fighting several problems at once:

  1. A device might not be powerful enough to acquire, synchronize and render 3 streams at once.

  2. Even if #1 is solved, a device is never totally dedicated to your task. For example, it might pause for garbage collection between processing stream#1 and stream#2--resulting in dropped/unsynchronized frames.

So to give yourself the best chance at perfection, you should first merge your 3 videos into 1 vertical video in the studio (or using studio software).

Then you can use the extended clipping properties of canvas context.drawImage to break each single frame into 2-3 separate frames.

Additionally, buffer a few frames you acquire on the stream (this goes without saying!).

Use requestAnimationFrame (RAF) to control the drawing. RAF does a fairly good job of drawing frames when system resources are available and delaying frames when system resources are lacking.

Your result won't be perfect, but they will be synchronized. You will always have to make the decision whether to drop or delay frames when system resources are unavailable, but at least the frames you do present will be synchronized.

As far as I know it's currently impossible to play HTML5 video frame-by-frame, or seek to a frame accurate time-code. The nearest seek seems to be precise to roughly 1-second.

But you can still get pretty close using the some of the media frameworks:

  1. Popcorn.js library made for synchronizing video with content.
  2. mediagroup.js another library used to add support for mediagroup attributes on HTML5 media elements

The only feature that allowed that is named mediaGroup and it was removed from Chrome(apparently for not being popular enough). It's still present in WebKit. Relevant discussion here and here.

I think you can implement you own "mediagroup"-like tag using wasm though without DOM support it may be tricky.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!