Garbage Collector too slow when working with large images

Question

I am using Emgu OpenCV to grab images from a webcam and want to visualize them with WPF Image Control.
So I need to convert the image from Mat to something compatible with Image control. So I took this class from the Emgu examples:

public static class BitmapSourceConvert
{
    /// <summary>
    /// Delete a GDI object
    /// </summary>
    /// <param name="o">The poniter to the GDI object to be deleted</param>
    /// <returns></returns>
    [DllImport("gdi32")]
    private static extern int DeleteObject(IntPtr o);

    /// <summary>
    /// Convert an IImage to a WPF BitmapSource. The result can be used in the Set Property of Image.Source
    /// </summary>
    /// <param name="image">The Emgu CV Image</param>
    /// <returns>The equivalent BitmapSource</returns>
    public static BitmapSource ToBitmapSource(IImage image)
    {
        using (System.Drawing.Bitmap source = image.Bitmap)
        {
            IntPtr ptr = source.GetHbitmap(); //obtain the Hbitmap
            BitmapSource bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
                ptr,
                IntPtr.Zero,
                Int32Rect.Empty,
                System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());

            DeleteObject(ptr); //release the HBitmap
            return bs;
        }
    }
}

This works like a charm for small images (640 x 480 for instance). When using the task Manager (I am on Windows 8), I see the used memory increasing and decreasing. Works fine.

But when using larger images like 1920x1080 the application crashes after a short period of time with an exception saying no more memory. When looking at the task manager again, I can see the memory consumption go up, once go down and then go up till the exception is thrown. It feels like the garbage collector works not often enough to free all the space.

So I tried to start the garbage collector manually by adding GC.Collect() somewhere in the function. And it works again. Even with the large images.

I think calling the garbage collector manually is neither good style nor performant. Can anyone please give hints on how to solve this without calling GC.Collect()?

Answer 1

Finally, I think the problem is, that garbage collector has no idea of how big the images are and therefore is not able to plan a reasonable schedule. I found the Methods

GC.AddMemoryPreasure(long bytesAllocated)
GC.RemoveMemoryPreasure(long bytesAllocated)

These Methods tell the garbage collector when large unmanaged objects are allocated and released so the garbage collector can plan his schedule in a better way.

The following code works without any memory problems:

    public static BitmapSource ToBitmapSource(IImage image)
    {
        using (System.Drawing.Bitmap source = image.Bitmap)
        {
            IntPtr ptr = source.GetHbitmap(); //obtain the Hbitmap
            long imageSize = image.Size.Height*image.Size.Width*4; // 4 bytes per pixel
            GC.AddMemoryPressure(imageSize);
            BitmapSource bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
                ptr,
                IntPtr.Zero,
                Int32Rect.Empty,
                System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());

            DeleteObject(ptr); //release the HBitmap
            GC.RemoveMemoryPressure(imageSize);
            return bs;
        }
    }

Answer 2

From where IImage parameter come? Dispose it after you finish with it.

So I tried to start the garbage collector manually by adding GC.Collect() somewhere in the function. And it works again. Even with the large images.

Image implements finalizer, if you don't dispose them. It will make those instances to live more than one GC cycles. Probably that's your issue.

Finalizer is the last point it can release unmanaged (managed too) resources if developer don't call the Dispose. When you call the Dispose it Supress the finalization and it will make them reachable for GC straight away.

can see the memory consumption go up, once go down and then go up till the exception is thrown. It feels like the garbage collector works not often enough to free all the space.

This is not quite right normally. But might be possible when you open/close images frequently and finalization queue is growing up.

Here is a good article for you : The Dangers of the Large Object Heap...

Answer 3

Happens when one uses the wrong tool for the job. A video is not really a set of bitmaps - there are better ways to do it.

What I did last time I had to do that was using Direct3d. There is a WPF integration and it is quite easy to set up a bitmap there. Allows a ton of manipulation in the video stream, too ;) THen you push the image directly into the Direct3d surface. Finished.

No code examples - sorry. It is a couple of years ago and I don't have the code ready.