How to color manage AVAssetWriter output

Question

I'm having trouble getting a rendered video's colors to match the source content's colors. I'm rendering images into a CGContext, converting the backing data into a CVPixelBuffer and appending that as a frame to an AVAssetWriterInputPixelBufferAdaptor. This causes slight color differences between the images that I'm drawing into the CGContext and the resulting video file.

It seems like there are 3 things that need to be addressed:

1. tell AVFoundation what colorspace the video is in.
2. make the AVAssetWriterInputPixelBufferAdaptor and the CVPixelBuffers I append to it match that color space.
3. use the same colorspace for the CGContext.

The documentation is terrible, so I'd appreciate any guidance on how to do these things or if there is something else I need to do to make the colors be preserved throughout this entire process.

Full code:

AVAssetWriter                        *_assetWriter;
AVAssetWriterInput                   *_assetInput;
AVAssetWriterInputPixelBufferAdaptor *_assetInputAdaptor;

NSDictionary *outputSettings = @{ AVVideoCodecKey :AVVideoCodecH264,
                                  AVVideoWidthKey :@(outputWidth),
                                  AVVideoHeightKey:@(outputHeight)};

_assetInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo
                                                 outputSettings:outputSettings];


NSDictionary *bufferAttributes = @{å(NSString*)kCVPixelBufferPixelFormatTypeKey:@(kCVPixelFormatType_32ARGB)};
_assetInputAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:_assetInput
                                                                                      sourcePixelBufferAttributes:bufferAttributes];


_assetWriter = [AVAssetWriter assetWriterWithURL:aURL fileType:AVFileTypeMPEG4 error:nil];
[_assetWriter addInput:_assetInput];
[_assetWriter startWriting];
[_assetWriter startSessionAtSourceTime:kCMTimeZero];

NSInteger bytesPerRow = outputWidth * 4;
long size = bytesPerRow * outputHeight;
CGColorSpaceRef srgbSpace = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);

UInt8 *data = (UInt8 *)calloc(size, 1);
CGContextRef ctx = CGBitmapContextCreateWithData(data, outputWidth, outputHeight, 8, bytesPerRow, srgbSpace, kCGImageAlphaPremultipliedFirst, NULL, NULL);

// draw everything into ctx

CVPixelBufferRef pixelBuffer;
CVPixelBufferCreateWithBytes(kCFAllocatorSystemDefault,
                                 outputWidth, outputHeight,
                                 k32ARGBPixelFormat,
                                 data,
                                 bytesPerRow,
                                 ReleaseCVPixelBufferForCVPixelBufferCreateWithBytes,
                                 NULL,
                                 NULL,
                             &pixelBuffer);

NSDictionary *pbAttachements = @{(id)kCVImageBufferCGColorSpaceKey : (__bridge id)srgbSpace};
CVBufferSetAttachments(pixelBuffer, (__bridge CFDictionaryRef)pbAttachements, kCVAttachmentMode_ShouldPropagate);
[_assetInputAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:CMTimeMake(0, 60)];

CGColorSpaceRelease(srgbSpace);

[_assetInput markAsFinished];
[_assetWriter finishWritingWithCompletionHandler:^{}];

Answer 1

This is quite a confusing subject and the Apple docs really do not help all that much. I am going to describe the solution I have settled on based on using the BT.709 colorspace, I am sure someone will have an objection based on Colorimetric correctness and the weirdness of various video standards, but this is complex topic. First off, don't use kCVPixelFormatType_32ARGB as the pixel type. Always pass kCVPixelFormatType_32BGRA instead, since BGRA is the native pixel layout on both MacOSX and iPhone hardware and it BGRA is just faster. Next, when you create a CGBitmapContext to render into use the BT.709 colorspace (kCGColorSpaceITUR_709). Also, don't render into a malloc() buffer, render directly into the CoreVideo pixel buffer by creating a bitmap context over the same memory, CoreGraphics will handle the colorspace and gamma conversion from whatever your input image is to BT.709 and its associated gamma. Then you need to tell AVFoundation the colorspace of the pixel buffer, do that by making an ICC profile copy and setting the kCVImageBufferICCProfileKey on the CoreVideo pixel buffer. That takes care of your issues 1 and 2, you do not need to have input images in this same colorspace with this approach. Now, this is of course complex and actual working source code (yes actually working) is hard to come by. Here is a github link to a small project that does these exact steps, the code is BSD licensed, so feel free to use it. Note specifically the H264Encoder class which wraps all this horror up into a reusable module. You can find calling code in encode_h264.m, it is a little MacOSX command line util to encode PNG to M4V. Also attached 3 keys Apple docs related to this subject 1, 2, 3.

MetalBT709Decoder