Deep Copy of OpenCV cv::Mat

Question

The behaviour of copying cv::Mat is confusing me.

I understand from the documentation that Mat::copyTo() is deep copy while the assignment operator is not. My questions:

1. what should I do to return a cv::Mat from a function, such as: cv::Mat func()?

2. According to the documentation, if I return a cv::Mat it'll have no use, because after the function returns the local copy of the cv::Mat in that function will be destroyed and therefore the one accepting the returned value outside the function should be pointing to some random address. The weird thing is that (most of times) it works correctly. For example, the following works:

   cv::Mat CopyOneImage(const cv::Mat& orgImage)
   {
   
       cv::Mat image;
       orgImage.copyTo(image);
       return image;
   
   }
   
   int main()
   {
   
       std::string orgImgName("a.jpg");        
       cv::Mat orgImage;
       orgImage = cv::imread(orgImgName);
   
       cv::Mat aCopy;
       aCopy = CopyOneImage(orgImage);
   
       return 1;
   }
   

But why? It's not a deep copy.

Question 3. And also sometimes the assignment operator seems to be deep copy, too:

    int main()
    {

        std::string orgImgName("a.jpg");        
        cv::Mat orgImage;
        orgImage = cv::imread(orgImgName);

        cv::Mat aCopy;
        orgImage.copyTo(aCopy);

        cv::Mat copyCopy1;
        copyCopy1 = aCopy;

        cv::namedWindow("smallTest", 1);
        cv::imshow("smallTest", copyCopy1);
        uchar key = (uchar)cv::waitKey();

        cv::Mat orgImage2 = cv::imread("b.jpg");
        orgImage2.copyTo(aCopy);

        cv::imshow("smallTest", copyCopy1);
        return 1;
    }

Then the two displays shows the same image, a.jpg. Why? And some other times it doesn't work. (The original code is too long but it can be also simplified to the above case). In those times the assignment operator seem to be actually 'shallow' copying. Why?

Thanks a lot!

Answer 1

I think, that using assignment is not the best way of matrix copying. If you want new full copy of the matrix, use:

Mat a=b.clone(); 

If you want copy matrix for replace the data from another matrix (for avoid memory reallocation) use:

Mat a(b.size(),b.type());
b.copyTo(a);

When you assign one matrix to another, the counter of references of smart pointer to matrix data increased by one, when you release matrix (it can be done implicitly when leave code block) it decreases by one. When it becomes equal zero the allocated memory deallocated.

If you want get result from the function use references it is faster:

void Func(Mat& input,Mat& output)
{
 somefunc(input,output);
}

int main(void)
{
...
  Mat a=Mat(.....);
  Mat b=Mat(.....);
  Func(a,b);
...
}

Answer 2

I've been using OpenCV for a while now and the cv::Mat confused me too, so I did some reading.

cv::Mat is a header which points to a *data pointer which holds the actual image data. It also implements reference counting. it holds the number of cv::Mat headers currently pointing to that *data pointer. So when you do a regular copy such as:

cv::Mat b; 
cv::Mat a = b;

a will point to b's data and the reference count for it will be incremented. At the same time the reference count for the data previously pointed to by b will be decremented (and the memory will be freed if it is 0 after decrementing).

Question 1: It depends on your program. Please refer to this question for more details: is-cvmat-class-flawed-by-design

Question 2: the function returns by value. That means return image will copy the Mat and increase the ref count(now ref_count = 2) and return the new Mat. When the function ends, image will be destroyed and ref_count will be reduced by one. But the memory will not be freed since the ref_count is not 0. So the returned cv::Mat is not pointing to random memory location.

Question 3: A similar thing happens. When you say orgImage2.copyTo(aCopy); The ref_count for the data pointed to by aCopy will be decreased. Then new memory is allocated to store the new data that will be copied. So That is why copyCopy1 was not modified when you did this.

Answer 3

Take a look at c++11 std::shared_ptr effectively works in the same way, by using a reference counter cv::Mat cleverly remembers every time the pointer is referenced, once the count reaches 0 it is automatically released i.e. memory is deallocated and cv::Mat is no longer available. This is effectively a "shallow copy" and saves resources in allocating/deallocating large amounts of memory.

On the other hand cv::Mat::clone will provide a "deep copy" that allocates a whole new block of memory for the matrix to reside in, this can be useful if you are making transformations to an image that you may want to undo however, more memory allocating/deallocating increases the amount of resources required.

Hope this helps someone.