Converting UIImage to cv::Mat

Question

I have a UIImage which is a pic captured from an iPhone Camera now I want the UIImage to be converted to cv::Mat (OpenCV). I am using the following lines of code to accompli

Answer 1

You should consider using native OpenCV functions to convert forth and back :

#import <opencv2/imgcodecs/ios.h>
...
UIImage* MatToUIImage(const cv::Mat& image);
void UIImageToMat(const UIImage* image,
                         cv::Mat& m, bool alphaExist = false);

Note: if your UIImage comes from the camera, you should 'normalize' it ( iOS UIImagePickerController result image orientation after upload) before converting to cv::Mat since OpenCV does not take into account Exif data. If you don't do that the result should be misoriented.

Answer 2

This will be because the UIImage is not actually portrait. All photos taken with the iPhone camera are landscape in their raw bitmap state, eg 3264 wide x 2488 high. A "portrait" photo is displayed as such by the orientation EXIF flag set in the image, which is honoured, for example, by the photo library app which swivels images according to this flag and the viewing orientation of the camera.

The flag also affects how UIImage reports its width and height properties, transposing them from their bitmap values for images flagged as portrait.

cv::Mat doesn't bother with any of that. This means that (i) when translating to cv::Mat a portrait image will have its size.width and size.height values transposed, and (ii) when translating back from cv::Mat you will have lost the orientation flag.

The simplest way to handle this when going from UIImage to cv::Mat is to swap width and height values if the image is flagged as portrait:

if  (self.imageOrientation == UIImageOrientationLeft
      || self.imageOrientation == UIImageOrientationRight) {
         cols = self.size.height;
         rows = self.size.width;
    }

When translating back from cv::Mat to UIImage, you will want to reinstate the orientation flag. Assuming your cv::Mat -> UIImage code contains this:

self = [self initWithCGImage:imageRef];

you can use this method instead, and reset the orientation as per the original.

self = [self initWithCGImage:imageRef scale:1 orientation:orientation];