问题
I'm trying to figure out how to composite multiple videos (AVAssets) into a single video such that each of the videos goes through its own video composition. However, I can't see a way to accomplish this and was wondering if anyone had any ideas.
Consider the following:
The above picture illustrates what I'm trying to do. I want to take the video track from four different videos and merge them into a single video such that they play in a grid-like layout.
Right now, I'm able to achieve this by doing the following
Create a single
AVMutableCompositionAdd four video tracks to the mutable composition
Create four
AVMutableVideoCompositionLayerInstructioninstances with the appropriatetransformapplied to scale and translate the track.Create an
AVMutableVideoComposition, add the layer instructions to it and then set that video composition on the appropriateAVAssetExportSession
While this works, it doesn't offer me the ability to do any custom effects on each video track before compositing it into the output clip. For example, in the picture above I'd like to add a unique border color around each track. With a single video (and thus no layer instructions), then I can just add the border using something like AVVideoComposition applyingCIFiltersWithHandler, which works great. But I can't use that because I need the video composition for setting the transform and scale instructions.
(applyingCIFiltersWithHandler doesn't appear to offer a way to get access to the tracks that make up the frame being rendered.)
What I think I want is the ability to do some type of custom rendering within a AVMutableVideoCompositionLayerInstruction, but the only options currently available are transform and opacity ones.
(Note that any solution requires that I have access to the current composition time during rendering for each video track because some of the "effects" I'd like to render are a frame counter and a timecode display.)
Any help, ideas or tips would be appreciated.
回答1:
You need to implement a custom compositor that implements the AVVideoCompositing protocol. https://developer.apple.com/reference/avfoundation/avvideocompositing
The startRequest function within that protocol receives an https://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest and this allows you to access the information from each track https://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest/1390379-sourceframe, then combine them with your own custom instructions and composite (put them together) in your own way, returning a final single buffer that has the frame composited from the tracks in the way you like.
来源：https://stackoverflow.com/questions/40153013/how-to-composite-videos-using-multiple-avvideocompositions