Modify AVCaptureSession before saving with AVCaptureMovieFileOutput

Question

Use Case: I want to capture input from the camera, draw on top of the captured frames (and sound) and save the result as a .mov file.

• I see that I can capture input for the camera using AVCaptureSession.
• I can save this to a .mov file using AVCaptureMovieFileOutput.
• AVVideoComposition can be used to add Core Animation for playback. I assume for recording somehow too?

Problem: I can't see how to modify the input before it is saved to file.

Answer 1

The RosyWriter was almost doing what I wanted. Adding the following code to captureOutput:didOutputSampleBuffer:fromConnection: enabled me to draw onto the frame using Quartz.

    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    void *pxdata = CVPixelBufferGetBaseAddress(pixelBuffer);
    NSParameterAssert(pxdata != NULL);

    CGSize frameSize = CGSizeMake(self.videoDimensions.width, self.videoDimensions.height);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, frameSize.width,
                                                 frameSize.height, 8, 4*frameSize.width, rgbColorSpace, 
                                                 kCGImageAlphaNoneSkipFirst);

    CGContextMoveToPoint(context, 100, 100);
    CGContextAddLineToPoint(context, 200, 200);
    CGContextDrawPath(context, kCGPathStroke);

    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);

Answer 2

AVCaptureMovieFileOutput does not let you process captured frames. To do that, replace it with AVCaptureVideoDataOutput and encode the input to a .mov file using an AVAssetWriter.

As for the AVVideoComposition, I think you may have to apply it in a separate pass with an AVAssetExportSession.