iOS 10.0 - 10.1: AVPlayerLayer doesn't show video after using AVVideoCompositionCoreAnimationTool, only audio

Question

Here is a complete project if you care to run this yourself: https://www.dropbox.com/s/5p384mogjzflvqk/AVPlayerLayerSoundOnlyBug_iOS10.zip?dl=0

This is a new problem

Answer 1

I have met this issue on iOS 10.1 and the issue was fixed on iOS 10.2. One way I found to solve this issue on iOS 10.1 is to delay several seconds to add the playerLayer to your container layer.

Answer 2

The answer for me in this case is to work around the issue with AVVideoCompositionCoreAnimationTool by using a custom video compositing class implementing the AVVideoCompositing protocol, and a custom composition instruction implementing the AVVideoCompositionInstruction protocol. Because I need to overlay a CALayer on top of the video I'm including that layer in the composition instruction instance.

You need to set the custom compositor on your video composition like so:

composition.customVideoCompositorClass = CustomVideoCompositor.self

and then set your custom instructions on it:

let instruction = CustomVideoCompositionInstruction(...) // whatever parameters you need and are required by the instruction protocol
composition.instructions = [instruction]

EDIT: Here is a working example of how to use a custom compositor to overlay a layer on a video using the GPU: https://github.com/samsonjs/LayerVideoCompositor ... original answer continues below

As for the compositor itself you can implement one if you watch the relevant WWDC sessions and check out their sample code. I cannot post the one I wrote here, but I am using CoreImage to do the heavy lifting in processing the AVAsynchronousVideoCompositionRequest, making sure to use an OpenGL CoreImage context for best performance (if you do it on the CPU it will be abysmally slow). You also may need an auto-release pool if you get a memory usage spike during the export.

If you're overlaying a CALayer like me then make sure to set layer.isGeometryFlipped = true when you render that layer out to a CGImage before sending it off to CoreImage. And make sure you cache the rendered CGImage from frame to frame in your compositor.

Answer 3

We had the same issue on iOS 10 and 10.1. Looks fixed as of iOS 10.2 beta 3 though

Answer 4

To expand upon Sami Samhuri's answer, here's a small sample project I worked up that uses a custom AVVideoCompositing class with custom instructions that implement AVVideoCompositionInstructionProtocol

https://github.com/claygarrett/CustomVideoCompositor

The project allows you to place a watermark over a video, but the idea could extend to do whatever you need. This prevents the AVPlayer bug in question from surfacing.

Another interesting solution on a separate thread that might help: AVPlayer playback fails while AVAssetExportSession is active as of iOS 10