C# Simulating multitouch with Kinect

Question

I have a Kinect application that I can generate 1-4 distinct screen points (Left/Right hands for up to 2 people) and I would like to be able to send each Point to the application with focus as a multi touch message.

I'm currently using SendInput to send mouse move, mouse down, and mouse up messages, but AFAIK, it doesn't support the WM_TOUCH messages.

Does anyone know of an easy way to send multi touch messages in C#? As a test I would like to be able to use the Kinect in MS Paint, and paint with both hands (as well as all the colors of the wind)

Answer 1

What you want is to send messages to the window in question. The message you need to compose is the WM_TOUCH message. You can find a very helpful discussion thread on WM_TOUCH here.

Hope this helps!

Answer 2

I don't think this would work, unless you did something with saving the coordinates of the x and y's of each persons hand, by putting a canvas down, then an image on top of it, then 4 ellipses, like:

, then scaling the position of them to peoples joints, (see Channel 9 for how to do this). Then I would copy the coordinates to a double to then set the pixels of them. Do that like this.

double person1hand1x = Canvas.GetLeft(person1hand1);
double person1hand1y =  Canvas.GetTop(person1hand1);

Then I would change the color of the canvas based on those actions by using the Image control. import the System.Drawing resource into your project, you will need it to set the pixels Then create a Bitmap and set the pixels of it as where the x and y's have been. Do that like this:

        Bitmap b = new Bitmap((int)image1.Width, (int)image1.Height); //set the max height and width

        b.SetPixel(person1hand1x, person1hand1y, person1hand1.Fill); //set the ellipse fill so they can keep track of who drew what


        image1.Source = ToBitmapSource(b); //convert to bitmap source... see https://stackoverflow.com/questions/94456/load-a-wpf-bitmapimage-from-a-system-drawing-bitmap/1470182#1470182 for more details
    }


    /// <summary>
    /// Converts a <see cref="System.Drawing.Bitmap"/> into a WPF <see cref="BitmapSource"/>.
    /// </summary>
    /// <remarks>Uses GDI to do the conversion. Hence the call to the marshalled DeleteObject.
    /// </remarks>
    /// <param name="source">The source bitmap.</param>
    /// <returns>A BitmapSource</returns>
    public static BitmapSource ToBitmapSource(this System.Drawing.Bitmap source)
    {
        BitmapSource bitSrc = null;

        var hBitmap = source.GetHbitmap();

        try
        {
            bitSrc = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
                hBitmap,
                IntPtr.Zero,
                Int32Rect.Empty,
                BitmapSizeOptions.FromEmptyOptions());
        }
        catch (Win32Exception)
        {
            bitSrc = null;
        }
        finally
        {
            NativeMethods.DeleteObject(hBitmap);
        }

        return bitSrc;
    }

    internal static class NativeMethods
    {
        [DllImport("gdi32.dll")]
        [return: MarshalAs(UnmanagedType.Bool)]
        internal static extern bool DeleteObject(IntPtr hObject);
    }

Hope this helps! Note: I got the ToBitmapSource from Load a WPF BitmapImage from a System.Drawing.Bitmap