how to use skeletal joint to act as cursor using bounds (No gestures)

Question

I just want to be able to do something when my skeletal joint (x,y,z) coordinates are over the x,y,z coordinates of the button . . I have the following code but somehow it doesnt work properly . .as soon as my hand moves it will do something without my hand reaching the button

    if (skeletonFrame != null)
                {
                    //int skeletonSlot = 0;
                    Skeleton[] skeletonData = new Skeleton[skeletonFrame.SkeletonArrayLength];
                    skeletonFrame.CopySkeletonDataTo(skeletonData);

                    Skeleton playerSkeleton = (from s in skeletonData where s.TrackingState == SkeletonTrackingState.Tracked select s).FirstOrDefault();

                    if (playerSkeleton != null)
                    {
                        Joint rightHand = playerSkeleton.Joints[JointType.HandRight];

                        handPosition = new Vector2((((0.5f * rightHand.Position.X) + 0.5f) * (640)), (((-0.5f * rightHand.Position.Y) + 0.5f) * (480)));

                        var rightHands = playerSkeleton.Joints[JointType.HandRight];
                        var rightHandsX = rightHands.Position.X;
                        var rightHandsY = rightHands.Position.Y;
                        var rightHandsZ = rightHands.Position.Z;


                        if (Math.Sqrt(Math.Pow(rightHandsX - position.X, 2) + Math.Pow(rightHandsY - position.Y, 2)) < 20)
                        {

                           // Exit();
                        }



if (Math.Sqrt(Math.Pow(rightHandsX - start_bttn.Bounds.X, 1) + Math.Pow(rightHandsY - start_bttn.Bounds.Y, 1)) < 10)
                        {

                            currentGameState = GameState.Selection;
                            // Exit();
                        }

if ((rightHandsX < GraphicsDevice.Viewport.Width / 2 + 150 && rightHandsX > GraphicsDevice.Viewport.Width / 2 - 75) && (rightHandsY > GraphicsDevice.Viewport.Height / 2 && rightHandsY < GraphicsDevice.Viewport.Height / 2 + 50))
                        {
                            currentGameState = GameState.Selection;

                        }

}

Answer 1

Here is my hand tracking function. See if it does what you want, or gets you closer...

    private void TrackHandMovement(Skeleton skeleton)
    {
        Joint leftHand = skeleton.Joints[JointType.HandLeft];
        Joint rightHand = skeleton.Joints[JointType.HandRight];

        Joint leftShoulder = skeleton.Joints[JointType.ShoulderLeft];
        Joint rightShoulder = skeleton.Joints[JointType.ShoulderRight];

        Joint rightHip = skeleton.Joints[JointType.HipRight];

        // the right hand joint is being tracked
        if (rightHand.TrackingState == JointTrackingState.Tracked)
        {
            // the hand is sufficiently in front of the shoulder
            if (rightShoulder.Position.Z - rightHand.Position.Z > 0.4)
            {
                double xScaled = (rightHand.Position.X - leftShoulder.Position.X) / ((rightShoulder.Position.X - leftShoulder.Position.X) * 2) * SystemParameters.PrimaryScreenWidth;
                double yScaled = (rightHand.Position.Y - rightShoulder.Position.Y) / (rightHip.Position.Y - rightShoulder.Position.Y) * SystemParameters.PrimaryScreenHeight;

                // the hand has moved enough to update screen position (jitter control / smoothing)
                if (Math.Abs(rightHand.Position.X - xPrevious) > MoveThreshold || Math.Abs(rightHand.Position.Y - yPrevious) > MoveThreshold)
                {
                    RightHandX = xScaled;
                    RightHandY = yScaled;

                    xPrevious = rightHand.Position.X;
                    yPrevious = rightHand.Position.Y;

                    // reset the tracking timer
                    trackingTimerCounter = 10;
                }
            }
        }
    }

There is a bit of math in there to translate the hand position to the screen position. Different strokes for different folks, but my logic is:

Shoulders = top of screen
Hips = bottom of screen
Left Should = left most on screen

To get the right most screen position, I take the distance between the left & right shoulder and add it to the right shoulder.