How to Track Skeleton Joins using Kinect

In this article I will demonstrate to you on how to track the Skeleton in Kinect.

Download

  Download source code for How to Track Skeleton Joins using Kinect

Introduction

This is the 3^rd article that I have written this year about Kinect. In this article I’m going to introduce you to skeleton tracking.

Objective

The objective of this article is to demonstrate to you on how to track the skeleton in Kinect.

 

Types of Skeleton Tracking Modes

 

Standing (Default)

·       This is the default tracking mode, it tracks all the joints.

·       This mode tracks 20 skeleton joints, reporting their position as "tracked" or "inferred"     

Seated

·       The seated mode only tracks the ten upper-body joints (shoulders, elbows, wrists, arms and head).

Where do I Start

Create a WPF application as I have explained in the previous articles and make sure your xaml looks like this 

<Window xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:Toolbox="clr-namespace:Kinect.Toolbox;assembly=Kinect.Toolbox"x:Class="WpfApplication1.MainWindow"
        Title="MainWindow" Height="514.656" Width="782.122">
 
 
    <Grid>
        <TextBlock x:Name="Message"  HorizontalAlignment="Center" VerticalAlignment="Top" Height="30" Background="Green" Foreground="Red" Margin="0,0,0,462.8" />
 
        <Image Name="Image" Margin="68,27,72,2.8"/>
    </Grid>
 
</Window>

And your code should look like this, I have commented the code, to explain each part or line what it is responsible for. 

This avoids the part that I need to explain between the codes.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Windows;
using System.Windows.Media;
using Microsoft.Kinect;
 
namespace WpfApplication1
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    { 
      /// There can be more than one sensors connected, for now take the first one
        KinectSensor sensor = KinectSensor.KinectSensors[0];
 
        #region "Variables"
        /// <summary>
        /// Thickness of body center ellipse
        /// </summary>
        private const double BodyCenterThickness = 10;
 
        /// <summary>
        /// Thickness of clip edge rectangles
        /// </summary>
        private const double ClipBoundsThickness = 10;
 
        /// <summary>
        /// Brush used to draw skeleton center point
        /// </summary>
        private readonly Brush centerPointBrush = Brushes.Blue;
 
        /// <summary>
        /// Brush used for drawing joints that are currently tracked
        /// </summary>
        private readonly Brush trackedJointBrush = new SolidColorBrush(Color.FromArgb(255, 68, 192, 68));
 
        /// <summary>
        /// Brush used for drawing joints that are currently inferred
        /// </summary>       
        private readonly Brush inferredJointBrush = Brushes.Yellow;
 
        /// <summary>
        /// Pen used for drawing bones that are currently tracked
        /// </summary>
        private readonly Pen trackedBonePen = new Pen(Brushes.Green, 6);
 
        /// <summary>
        /// Pen used for drawing bones that are currently inferred
        /// </summary>       
        private readonly Pen inferredBonePen = new Pen(Brushes.Gray, 1);
 
        /// <summary>
        /// Drawing image that we will display
        /// </summary>
        private DrawingImage imageSource;
 
        /// <summary>
        /// Thickness of drawn joint lines
        /// </summary>
        private const double JointThickness = 3;
 
 
        /// <summary>
        /// Drawing group for skeleton rendering output
        /// </summary>
        private DrawingGroup drawingGroup;
        /// <summary>
        /// Width of output drawing
        /// </summary>
        private const float RenderWidth = 640.0f;
 
        /// <summary>
        /// Height of our output drawing
        /// </summary>
        private const float RenderHeight = 480.0f;
        #endregion
 
 
        public MainWindow()
        {
            InitializeComponent();
            //After Initialization subscribe to the loaded event of the form
            Loaded += MainWindow_Loaded;
 
            //After Initialization subscribe to the unloaded event of the form
            //We use this event to stop the sensor when the application is being closed.
            Unloaded += MainWindow_Unloaded;
 
           
        }
 
        void MainWindow_Unloaded(object sender, RoutedEventArgs e)
        {
            //stop the Sestor
            sensor.Stop();
 
        }
 
    
        void MainWindow_Loaded(object sender, RoutedEventArgs e)
        {
 
            //Create a Drawing Group that will be used for Drawing
            this.drawingGroup = new DrawingGroup();
 
            //Create an image Source that will display our skeleton
            this.imageSource = new DrawingImage(this.drawingGroup);
 
            //Display the Image in our Image control
            Image.Source = imageSource;
 
            try
            {
                //Check if the Sensor is Connected
                if (sensor.Status == KinectStatus.Connected)
                {
                    //Start the Sensor
                    sensor.Start();
//Tell Kinect Sensor to use the Default Mode(Human Skeleton Standing) || Seated(Human Skeleton Sitting Down)
                    sensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Default;
//Subscribe to te  Sensor's SkeletonFrameready event to track the joins and create the joins to display on our image control
                    sensor.SkeletonFrameReady += sensor_SkeletonFrameReady;
                //nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Ready";
                    Message.Background = new SolidColorBrush(Colors.Green);
                    Message.Foreground = new SolidColorBrush(Colors.White);
 
                    // Turn on the skeleton stream to receive skeleton frames
                    this.sensor.SkeletonStream.Enable();
                }
                else if (sensor.Status == KinectStatus.Disconnected)
                {
               //nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Connected";
                    Message.Background = new SolidColorBrush(Colors.Orange);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                    
                }
                else if (sensor.Status == KinectStatus.NotPowered)
                {
//nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Powered";
                    Message.Background = new SolidColorBrush(Colors.Red);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                }
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
 
            }
        }
 
        /// <summary>
        //When the Skeleton is Ready it must draw the Skeleton
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        void sensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
        {
            //declare an array of Skeletons
            Skeleton[] skeletons = new Skeleton[1];
 
             //Opens a SkeletonFrame object, which contains one frame of skeleton data.
                using (SkeletonFrame skeletonframe = e.OpenSkeletonFrame())
                {
                    //Check if the Frame is Indeed open
                    if (skeletonframe != null)
                    {
 
                       skeletons = new Skeleton[skeletonframe.SkeletonArrayLength];
                        
// Copies skeleton data to an array of Skeletons, where each Skeleton contains a collection of the joints.
                        skeletonframe.CopySkeletonDataTo(skeletons);
 
                        //draw the Skeleton based on the Default Mode(Standing), "Seated"
 if (sensor.SkeletonStream.TrackingMode == SkeletonTrackingMode.Default)
  {
                            //Draw standing Skeleton
                            DrawStandingSkeletons(skeletons);
 }
 else if (sensor.SkeletonStream.TrackingMode == SkeletonTrackingMode.Seated)
                        {
                            //Draw a Seated Skeleton with 10 joints
                            DrawSeatedSkeletons(skeletons);
                        }
                    }
              
            }
           
 
        }
 
 
  
        //Thi Function Draws the Standing  or Default Skeleton
        private void DrawStandingSkeletons(Skeleton[] skeletons)
        {
 
            using (DrawingContext dc = this.drawingGroup.Open())
            {
                //Draw a Transparent background to set the render size or our Canvas
                dc.DrawRectangle(Brushes.Black, null, new Rect(0.0, 0.0, RenderWidth, RenderHeight));
 
                //If the skeleton Array has items
                if (skeletons.Length != 0)
                {
                    //Loop through the Skeleton joins
                    foreach (Skeleton skel in skeletons)
                    {
                        RenderClippedEdges(skel, dc);
 
                        if (skel.TrackingState == SkeletonTrackingState.Tracked)
                        {
                            this.DrawBonesAndJoints(skel, dc);
 
 
                        }
  else if (skel.TrackingState == SkeletonTrackingState.PositionOnly)
                        {
                            dc.DrawEllipse(this.centerPointBrush,
                                           null,
                                          this.SkeletonPointToScreen(skel.Position),BodyCenterThickness,BodyCenterThickness);
 
                        }
 
                    }
 
 
                }
 
                //Prevent Drawing outside the canvas
this.drawingGroup.ClipGeometry = new RectangleGeometry(new Rect(0.0, 0.0, RenderWidth, RenderHeight));
 
            }
        }
 
 
        private void DrawSeatedSkeletons(Skeleton[] skeletons)
        {
 
            using (DrawingContext dc = this.drawingGroup.Open())
            {
                //Draw a Transparent background to set the render size
 dc.DrawRectangle(Brushes.Black, null, new Rect(0.0, 0.0, RenderWidth, RenderHeight));
 
                if (skeletons.Length != 0)
                {
                    foreach (Skeleton skel in skeletons)
                    {
                        RenderClippedEdges(skel, dc);
 
                        if (skel.TrackingState == SkeletonTrackingState.Tracked)
                        {
                            this.DrawBonesAndJoints(skel, dc);
 
 
                        }
 else if (skel.TrackingState == SkeletonTrackingState.PositionOnly)
                        {
  dc.DrawEllipse(this.centerPointBrush,  null, this.SkeletonPointToScreen(skel.Position), BodyCenterThickness, BodyCenterThickness);
 
                        }
 
                    }
 
 
                }
 
                //Prevent Drawing outside the canvas
                this.drawingGroup.ClipGeometry = new RectangleGeometry(new Rect(0.0, 0.0, RenderWidth, RenderHeight));
 
            }
        }
 
 
        /// <summary>
        /// Draws indicators to show which edges are clipping skeleton data
        /// </summary>
        /// <param name="skeleton">skeleton to draw clipping information for</param>
        /// <param name="drawingContext">drawing context to draw to</param>
        private static void RenderClippedEdges(Skeleton skeleton, DrawingContext drawingContext)
        {
            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Bottom))
            {
                drawingContext.DrawRectangle(
                    Brushes.Red,
                    null,
                    new Rect(0, RenderHeight - ClipBoundsThickness, RenderWidth, ClipBoundsThickness));
            }
 
            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Top))
            {
                drawingContext.DrawRectangle(
                    Brushes.Red,
                    null,
                    new Rect(0, 0, RenderWidth, ClipBoundsThickness));
            }
 
            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Left))
            {
                drawingContext.DrawRectangle(
                    Brushes.Red,
                    null,
                    new Rect(0, 0, ClipBoundsThickness, RenderHeight));
            }
 
            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Right))
            {
                drawingContext.DrawRectangle(
                    Brushes.Red,
                    null,
                    new Rect(RenderWidth - ClipBoundsThickness, 0, ClipBoundsThickness, RenderHeight));
            }
        }
 
        /// <summary>
        /// Draws a skeleton's bones and joints
        /// </summary>
        /// <param name="skeleton">skeleton to draw</param>
        /// <param name="drawingContext">drawing context to draw to</param>
        private void DrawBonesAndJoints(Skeleton skeleton, DrawingContext drawingContext)
        {
            // Render Torso
            this.DrawBone(skeleton, drawingContext, JointType.Head, JointType.ShoulderCenter);
            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.ShoulderLeft);
            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.ShoulderRight);
            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.Spine);
            this.DrawBone(skeleton, drawingContext, JointType.Spine, JointType.HipCenter);
            this.DrawBone(skeleton, drawingContext, JointType.HipCenter, JointType.HipLeft);
            this.DrawBone(skeleton, drawingContext, JointType.HipCenter, JointType.HipRight);
 
            // Left Arm
            this.DrawBone(skeleton, drawingContext, JointType.ShoulderLeft, JointType.ElbowLeft);
            this.DrawBone(skeleton, drawingContext, JointType.ElbowLeft, JointType.WristLeft);
            this.DrawBone(skeleton, drawingContext, JointType.WristLeft, JointType.HandLeft);
 
            // Right Arm
            this.DrawBone(skeleton, drawingContext, JointType.ShoulderRight, JointType.ElbowRight);
            this.DrawBone(skeleton, drawingContext, JointType.ElbowRight, JointType.WristRight);
            this.DrawBone(skeleton, drawingContext, JointType.WristRight, JointType.HandRight);
 
            // Left Leg
            this.DrawBone(skeleton, drawingContext, JointType.HipLeft, JointType.KneeLeft);
            this.DrawBone(skeleton, drawingContext, JointType.KneeLeft, JointType.AnkleLeft);
            this.DrawBone(skeleton, drawingContext, JointType.AnkleLeft, JointType.FootLeft);
 
            // Right Leg
            this.DrawBone(skeleton, drawingContext, JointType.HipRight, JointType.KneeRight);
            this.DrawBone(skeleton, drawingContext, JointType.KneeRight, JointType.AnkleRight);
            this.DrawBone(skeleton, drawingContext, JointType.AnkleRight, JointType.FootRight);
 
            // Render Joints
            foreach (Joint joint in skeleton.Joints)
            {
                Brush drawBrush = null;
 
                if (joint.TrackingState == JointTrackingState.Tracked)
                {
                    drawBrush = this.trackedJointBrush;
                }
                else if (joint.TrackingState == JointTrackingState.Inferred)
                {
                    drawBrush = this.inferredJointBrush;
                }
 
                if (drawBrush != null)
                {
                    drawingContext.DrawEllipse(drawBrush, null, this.SkeletonPointToScreen(joint.Position), JointThickness, JointThickness);
                }
            }
        }
 
        /// <summary>
        /// Draws a bone line between two joints
        /// </summary>
        /// <param name="skeleton">skeleton to draw bones from</param>
        /// <param name="drawingContext">drawing context to draw to</param>
        /// <param name="jointType0">joint to start drawing from</param>
        /// <param name="jointType1">joint to end drawing at</param>
        private void DrawBone(Skeleton skeleton, DrawingContext drawingContext, JointType jointType0, JointTypejointType1)
        {
            Joint joint0 = skeleton.Joints[jointType0];
            Joint joint1 = skeleton.Joints[jointType1];
 
            // If we can't find either of these joints, exit
            if (joint0.TrackingState == JointTrackingState.NotTracked || joint1.TrackingState ==JointTrackingState.NotTracked)
            {
                return;
            }
 
            // Don't draw if both points are inferred
            if (joint0.TrackingState == JointTrackingState.Inferred && joint1.TrackingState ==JointTrackingState.Inferred)
            {
                return;
            }
 
            // We assume all drawn bones are inferred unless BOTH joints are tracked
            Pen drawPen = this.inferredBonePen;
 
            if (joint0.TrackingState == JointTrackingState.Tracked && joint1.TrackingState ==JointTrackingState.Tracked)
            {
                drawPen = this.trackedBonePen;
            }
 
            drawingContext.DrawLine(drawPen, this.SkeletonPointToScreen(joint0.Position),this.SkeletonPointToScreen(joint1.Position));
        }
 
 
        /// <summary>
        /// Maps a SkeletonPoint to lie within our render space and converts to Point
        /// </summary>
        /// <param name="skelpoint">point to map</param>
        /// <returns>mapped point</returns>
        private Point SkeletonPointToScreen(SkeletonPoint skelpoint)
        {
            // Convert point to depth space. 
            // We are not using depth directly, but we do want the points in our 640x480 output resolution.
            DepthImagePoint depthPoint = this.sensor.CoordinateMapper.MapSkeletonPointToDepthPoint(skelpoint,DepthImageFormat.Resolution640x480Fps30);
            return new Point(depthPoint.X, depthPoint.Y);
        }
 
    }
}
  
  


  
  

Reference

• http://msdn.microsoft.com/en-us/library/hh973077.aspx
• http://www.renauddumont.be/en/2012/kinect-sdk-1-0-3-tracker-les-mouvements-avec-le-skeletonstream
• http://msdn.microsoft.com/en-us/library/windows/desktop/dd162897(v=vs.85).aspx
• http://msdn.microsoft.com/en-us/library/jj131429.aspx
• http://msdn.microsoft.com/en-us/library/jj131024.aspx
• http://msdn.microsoft.com/en-us/library/hh973073.aspx
• http://msdn.microsoft.com/en-us/library/microsoft.kinect.skeletonstream.trackingmode.aspx
• http://msdn.microsoft.com/en-us/library/hh855353.aspx
• http://msdn.microsoft.com/en-us/library/hh855353.aspx

Conclusion

Currently we see a Skeleton, in one of my next articles, I will show you how to cover this skeleton with an avatar. Thank you again for visiting DotNetFunda.com. 

转自：http://www.dotnetfunda.com/articles/article2069-how-to-track-skeleton-joins-using-kinect.aspx