OpenCV MatToUIImage causes memory leak

Question

My project is working with openCV for iOS(2.4.9). And I found function MatToUIImage which will cause memory leaks, and it only occurs on iOS 10.X.

After I updated this function(2.4.9) to latest(3.2.0) version everything got worked. The only difference is CGBitmapInfo.

So can anyone tell me why?

2.4.9

UIImage* MatToUIImage(const cv::Mat& image) {

    NSData *data = [NSData dataWithBytes:image.data
                                  length:image.elemSize()*image.total()];

    CGColorSpaceRef colorSpace;

    if (image.elemSize() == 1) {
        colorSpace = CGColorSpaceCreateDeviceGray();
    } else {
        colorSpace = CGColorSpaceCreateDeviceRGB();
    }

    CGDataProviderRef provider =
            CGDataProviderCreateWithCFData((__bridge CFDataRef)data);

    // Creating CGImage from cv::Mat
    CGImageRef imageRef = CGImageCreate(image.cols,
                                        image.rows,
                                        8,
                                        8 * image.elemSize(),
                                        image.step.p[0],
                                        colorSpace,
                                        kCGImageAlphaNone|
                                        kCGBitmapByteOrderDefault,
                                        provider,
                                        NULL,
                                        false,
                                        kCGRenderingIntentDefault
                                        );


    // Getting UIImage from CGImage
    UIImage *finalImage = [UIImage imageWithCGImage:imageRef];
    CGImageRelease(imageRef);
    CGDataProviderRelease(provider);
    CGColorSpaceRelease(colorSpace);

    return finalImage;
}

3.2.0

UIImage* MatToUIImage(const cv::Mat& image) {

    NSData *data = [NSData dataWithBytes:image.data
                                  length:image.elemSize()*image.total()];

    CGColorSpaceRef colorSpace;

    if (image.elemSize() == 1) {
        colorSpace = CGColorSpaceCreateDeviceGray();
    } else {
        colorSpace = CGColorSpaceCreateDeviceRGB();
    }

    CGDataProviderRef provider =
            CGDataProviderCreateWithCFData((__bridge CFDataRef)data);

    // Preserve alpha transparency, if exists
    bool alpha = image.channels() == 4;
    CGBitmapInfo bitmapInfo = (alpha ? kCGImageAlphaLast : kCGImageAlphaNone) | kCGBitmapByteOrderDefault;

    // Creating CGImage from cv::Mat
    CGImageRef imageRef = CGImageCreate(image.cols,
                                        image.rows,
                                        8,
                                        8 * image.elemSize(),
                                        image.step.p[0],
                                        colorSpace,
                                        bitmapInfo,
                                        provider,
                                        NULL,
                                        false,
                                        kCGRenderingIntentDefault
                                        );


    // Getting UIImage from CGImage
    UIImage *finalImage = [UIImage imageWithCGImage:imageRef];
    CGImageRelease(imageRef);
    CGDataProviderRelease(provider);
    CGColorSpaceRelease(colorSpace);

    return finalImage;
}

Answer 1

Important update (5.06.2017) Finally, to perform CFRelease manually turned out to be bad idea as it can raise more troubles than solve! Though, it gave me a clue that leaks are somehow connected with NSData (not-)releasing.

I noticed that it's released automatically as expected with ARC when called from block in background thread, like that:

- (void)runInBackgroundWithImageBuffer:(CVImageBufferRef)imageBuffer
                              callback:(void (^)())callback {
    dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
    dispatch_async(queue, ^{
        [self processImageBuffer:imageBuffer];
        if (callback != nil) {
            callback();
        }
    });
}

- (void)previewOpenCVImage:(cv::Mat *)image {
    UIImage *preview = MatToUIImage(*image);

    dispatch_async(dispatch_get_main_queue(), ^{
        // _imagePreview has (UIImageView *) type
        [_imagePreview setImage:preview];
    });
}

I can confirm that for iPhone Simulator. Seems like current MatToUIImage implementation causes memory leaks on simulator. And I can't reproduce it on device.

For some reason they are not detected by profiler but memory usage just blows up after multiple calls.

I've added some tweaks to make it working:

1. Add line CFRelease((CFTypeRef)data) before returning final image

2. When image is not necessary we need to perform CFRelease(image.CGImage) and probably CFRelease((CFTypeRef)image)

Hope that helps. Actually I don't completely understand why this is happening, who is holding references and why do we need to manually perform release.