Unsafe Per Pixel access, 30ms access for 1756000 pixels

Question

So I\'ve been sharing some thoughts on the above topic title on my website about fast, unsafe pixel access. A gentlemen gave me a rough example of how he\'d do it in C++, bu

Answer 1

Was able to slice off about 60ms. I think this is going to require the GPU. I'm not seeing any solution to this utilizing the CPU, even by comparing more than one byte/pixel at a time, unless someone can whip up a code sample to show me otherwise. Still sits at about 200-260ms, far too slow for 30fps.

private static BitmapData m_OldData;
private static BitmapData m_NewData;
private static unsafe Byte* m_OldPBase;
private static unsafe Byte* m_NewPBase;
private static unsafe Pixel* m_OldPixel;
private static unsafe Pixel* m_NewPixel;
private static Int32 m_X;
private static Int32 m_Y;
private static Stopwatch m_Watch = new Stopwatch();
private static GraphicsUnit m_GraphicsUnit = GraphicsUnit.Pixel;
private static RectangleF m_OldBoundsF;
private static RectangleF m_NewBoundsF;
private static Rectangle m_OldBounds;
private static Rectangle m_NewBounds;
private static Pixel m_TransparentPixel = new Pixel() { Alpha = 0x00, Red = 0, Green = 0, Blue = 0 };

private Bitmap GetInvalidFrame(Bitmap frame) {
    if (m_FrameBackBuffer == null) {
        return frame;
    }

    m_Watch.Start();

    unsafe {
        m_OldBoundsF = m_FrameBackBuffer.GetBounds(ref m_GraphicsUnit);
        m_OldBounds = new Rectangle((Int32)m_OldBoundsF.X, (Int32)m_OldBoundsF.Y, (Int32)m_OldBoundsF.Width, (Int32)m_OldBoundsF.Height);
        m_OldData = m_FrameBackBuffer.LockBits(m_OldBounds, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);

        m_NewBoundsF = m_FrameBackBuffer.GetBounds(ref m_GraphicsUnit);
        m_NewBounds = new Rectangle((Int32)m_NewBoundsF.X, (Int32)m_NewBoundsF.Y, (Int32)m_NewBoundsF.Width, (Int32)m_NewBoundsF.Height);
        m_NewData = frame.LockBits(m_NewBounds, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);

        m_OldPBase = (Byte*)m_OldData.Scan0.ToPointer();
        m_NewPBase = (Byte*)m_NewData.Scan0.ToPointer();

        do {
            for (m_X = 0; m_X < frame.Width; m_X++) {

                m_OldPixel = (Pixel*)(m_OldPBase + m_Y * m_OldData.Stride + 1 + m_X * sizeof(Pixel));
                m_NewPixel = (Pixel*)(m_NewPBase + m_Y * m_NewData.Stride + 1 + m_X * sizeof(Pixel));

                if (m_OldPixel->Alpha == m_NewPixel->Alpha // AccessViolationException accessing Property in get {}
                    || m_OldPixel->Red == m_NewPixel->Red
                    || m_OldPixel->Green == m_NewPixel->Green
                    || m_OldPixel->Blue == m_NewPixel->Blue) {

                    // Set the transparent pixel
                    *m_NewPixel = m_TransparentPixel;
                }
            }

            m_Y++; //Debug.WriteLine(String.Format("X: {0}, Y: {1}", m_X, m_Y));
        } while (m_Y < frame.Height);
    }

    m_Y = 0;

    m_Watch.Stop();
    Debug.WriteLine("Time elapsed: " + m_Watch.ElapsedMilliseconds.ToString());
    m_Watch.Reset();

    return frame;
}

Answer 2

Yes, you can do so by using unsafe code.

BitmapData d = l.LockBits(new Rectangle(0, 0, l.Width, l.Height), ImageLockMode.ReadOnly,l.PixelFormat);
IntPtr scan = d.Scan0;
unsafe
{
    byte* p = (byte*)(void*)scan;
    //dostuff               
}

Check out http://www.codeproject.com/KB/GDI-plus/csharpgraphicfilters11.aspx for some basic examples of this kind of stuff. My code is based on that.

Note: One of the reasons this will be much faster than yours is that you are separately comparing each channel instead of just comparing the entire byte using one operation. Similarly, changing PixelAt to give you a byte to facilitate this would probably give you an improvement.

Answer 3

Instead of checking each and every pixel, you can just perform a basic memory compare of the 2 bitmaps. In C, something like memcmp().

This would give you a much quicker test to let you know that the images are the same or not. Only when you know they are different do you need to resort to the more expensive code that will help you determine where they are different (if you even need to know that).

I am not a C# person though, so I don't know how easy it is to get access to the raw memory.

Answer 4

Unsafe approach with usage of integers instead of Pixels and single loop:

private static Bitmap GetInvalidFrame(Bitmap oldFrame, Bitmap newFrame)
{
    if (oldFrame.Size != newFrame.Size)
    {
        throw new ArgumentException();
    }
    Bitmap result = new Bitmap(oldFrame.Width, oldFrame.Height, oldFrame.PixelFormat);

    Rectangle lockArea = new Rectangle(Point.Empty, oldFrame.Size);
    PixelFormat format = PixelFormat.Format32bppArgb;
    BitmapData oldData = oldFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);
    BitmapData newData = newFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);
    BitmapData resultData = result.LockBits(lockArea, ImageLockMode.WriteOnly, format);

    int len = resultData.Height * Math.Abs(resultData.Stride) / 4;

    unsafe
    {
        int* pOld = (int*)oldData.Scan0;
        int* pNew = (int*)newData.Scan0;
        int* pResult = (int*)resultData.Scan0;

        for (int i = 0; i < len; i++)
        {
            int oldValue = *pOld++;
            int newValue = *pNew++;
            *pResult++ = oldValue != newValue ? newValue : 0 /* replace with 0xff << 24 if you need non-transparent black pixel */;
            // *pResult++ = *pOld++ ^ *pNew++; // if you can use XORs.
        }
    }

    oldFrame.UnlockBits(oldData);
    newFrame.UnlockBits(newData);
    result.UnlockBits(resultData);

    return result;
}

I think you really can use XORed frames here and I hope that this can have better performance on both sides.

    private static void XorFrames(Bitmap leftFrame, Bitmap rightFrame)
    {
        if (leftFrame.Size != rightFrame.Size)
        {
            throw new ArgumentException();
        }

        Rectangle lockArea = new Rectangle(Point.Empty, leftFrame.Size);
        PixelFormat format = PixelFormat.Format32bppArgb;
        BitmapData leftData = leftFrame.LockBits(lockArea, ImageLockMode.ReadWrite, format);
        BitmapData rightData = rightFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);

        int len = leftData.Height * Math.Abs(rightData.Stride) / 4;

        unsafe
        {
            int* pLeft = (int*)leftData.Scan0;
            int* pRight = (int*)rightData.Scan0;

            for (int i = 0; i < len; i++)
            {
                *pLeft++ ^= *pRight++;
            }
        }

        leftFrame.UnlockBits(leftData);
        rightFrame.UnlockBits(rightData);
    }

You can use this procedure on both sides in following way:
On server side you need to evaluate difference between old and new frame, send it to client and replace old frame by new. The server code should look something like this:

  XorFrames(oldFrame, newFrame); // oldFrame ^= newFrame
  Send(oldFrame); // send XOR of two frames
  oldFrame = newFrame;

On client side you need to update your current frame with xor frame recieved from server:

  XorFrames((Bitmap)uxSurface.Image, e.Frame);

Answer 5

Here: Utilizing the GPU with c# there are mentioned some librarys for using the GPU from C#.