Access Violation Exception mystery

Question

I\'ve been working with EMGU+OpenCV for quite some time and ran into this AccessViolationException mystery.

First thing first, the code:

Answer 1

Your example doesn't keep a reference to the result image from Image.SmoothBilatral. The input images are rooted in a static array so are fine.

An Emgu.CV Image's data array is pinned to a GCHandle inside the actual image, this is no different from the fact that image contains the array and doesn't prevent collection while the GCHandle's pointer is in use by unmanaged code (in the abscence of a managed root to the image).

Because the Image.SmoothBilatral method doesn't do anything with its temporary result image other than pass its pointer and return it, I think it gets optimised away to the extent that the result image can be collected while the smooth is processing.

Because there's no finalizer for this class, opencv will not get called upon to release it's unmanaged image header (which has a pointer to the managed image data) so opencv still thinks it has a usable image structure.

You can fix it by taking a reference to the result of SmoothBilatral and doing something with it (like disposing it).

This extension method would also work (i.e. allow it to be called successfuly for benchmarking without the result being used):

public static class BilateralExtensionFix
{
    public static Emgu.CV.Image<testchannels, testtype> SmoothBilateral(this Emgu.CV.Image<testchannels, testtype> image, int p1, int p2 , int p3)
    {
        var result = image.CopyBlank();
        var handle = GCHandle.Alloc(result);
        Emgu.CV.CvInvoke.cvSmooth(image.Ptr, result.Ptr, Emgu.CV.CvEnum.SMOOTH_TYPE.CV_BILATERAL, p1, p1, p2, p3);
        handle.Free();
        return result;
    }
}

I think what EmguCV should be doing is only pinning pointers to pass to opencv while making an interop call.

p.s The OpenCv bilateral filter crashes (producing a very similar error to your problem) on any kind of float image passed with zero variation (min() = max()) across all channels. I think because of how it builds it's binned exp() lookup table.

This can be reproduced with:

    // create new blank image
    var zeroesF1 = new Emgu.CV.Image<Rgb, float>(75, 75);
    // uncomment next line for failure
    zeroesF1.Data[0, 0, 0] += 1.2037063600E-035f;
    zeroesF1.SmoothBilatral(15, 50, 50);

This was confusing me as I was actually sometimes getting this error due to a bug in my test code...

Answer 2

What version of Emgu CV are you using? I couldn't find a 2.4.3 version of it.

Pretty sure your code is not the problem.

Seems possible that the Emgu.CV.Image constructor might have a concurrency issue (either in the managed wrapper or the unmanaged code). The way the managed data array is handled in the Emgu CV trunk seems solid, there is some unmanaged data allocated during the image constructor which I suppose might have gone wrong.

What happens if you try:

• Moving Images[i] = GetRandomImage(); outside of the parallel For().
• Slapping a lock() around the Image constructor in GetRandomImage()

I noticed there's a closed bug report of someone having a similar issues (calls to image constructor occuring in parallel but images themselves not shared among threads) here.

[Edit]

Yes this is a strange one. I can reproduce with the stock 2.4.2 version and OpenCV binaries.

It only seems to crash for me if the number of threads in the parallel for exceeds the number of cores which is >2 for me.. would be interesting to know how many cores are on your test system.

Also I only get the crash when the code is not attached to the debugger and Optimize Code is enabled - have you ever observed it in release mode with the debugger attached?

As the SmoothBilateral function is CPU bound, using MaxDegreeOfParallelism more than the number of cores doesn't really add any benefit so there's a perfect workaround assuming what I found about the number if threads vs cores is also true for your rig (sods law predicts: it isn't).

So my guess is there is a concurrency/volatile issue in Emgu that only manifests when JIT optimisation is run, and when the GC is moving managed data around. But, as you say, there are no obvious unpinned-pointer-to-managed-object issues in the Emgu code.

Although I still can't explain it properly, here's what I found so far:

With the GC.Collect + console logs removed, the calls to GetRandomImage() serialised, and the code run outside of MSVC I couldn't reproduce the issue (although this may have just reduced the frequency):

            public static int Width = 750;
            public static int Height = 750;
...
                int N = 500;
                int Threads = 11;
                Images = new Emgu.CV.Image<Rgb, float>[N];
                Console.WriteLine("Start");
                ParallelOptions po = new ParallelOptions();
                po.MaxDegreeOfParallelism = Threads;
                for (int i = 0; i < N; i++)
                {
                    Images[i] = GetRandomImage();
                }
                System.Threading.Tasks.Parallel.For(0, N, po, new Action<int>((i) =>
                {
                    //Console.WriteLine("CallingSmooth");
                    Images[i].SmoothBilatral(15, 50, 50);
                    //Console.WriteLine("SmoothCompleted");
                }));
                Console.WriteLine("End");

I added a timer to fire GC.Collect outside of the parallel for, but still more often than it would fire normally:

        var t = new System.Threading.Timer((dummy) => { 
            GC.Collect(); 
        }, null, 100,100);

And with this change I still can't reproduce the issue, although GC collect is being called less consistently than in your demo as the thread pool is busy, also there are no (or very few) managed allocations occuring in the main loop for it to collect. Uncommenting the console logs around the SmoothBilatral call then repros the error fairly swiftly (by giving GC something to collect I guess).

[Another edit]

The OpenCV 2.4.2 reference manual states that cvSmooth is deprecated AND that "Median and bilateral filters work with 1- or 3-channel 8-bit images and can not process images in-place."... not very encouraging!

I find that using median filter on byte or float images and bilateral on byte images works fine, and I can't see why any CLR/GC issues woudn't affect those cases too.

So despite the strange effects on the C# test program I still reckon this is an Emgu/OpenCV bug.

If you haven't already, you should test with opencv binaries that you've compiled yourself, if it still fails convert your test to C++.

N.b. that OpenCV has its own parallelism implementation which would probably work out faster.