Readme for Kinect for Windows SDK Beta from Microsoft Research

Readme for Kinect for Windows SDK Beta from Microsoft Research

This readme file provides information for Kinect™ for Windows® SDK Beta from Microsoft Research.

For the latest version of this document please see the online version of the Readme.

For help, ask your questions on MSDN Forum for the Kinect for Windows SDK Beta.
Note that you must have a Windows Live ID to log on to the Forums.

In this Readme:

1. System Requirements
2. Installation
3. Kinect Sensor Notes
4. Notes for Developers
5. Files and Samples
6. Known Issues
7. Related Links

1. System Requirements

Supported Operating Systems

• Windows 7 (x86 and x64)

Hardware Requirements

• Computer with a dual-core 2.66-GHz or faster processor
• 2 GB RAM
• Windows 7-compatible graphics card that supports DirectX 9.0c
• A retail Kinect for Xbox 360® sensor, which includes special USB/power cabling

Software Requirements

• Microsoft® Visual Studio® 2010 Express or other Visual Studio 2010 edition
• .NET Framework 4.0  
• For C++ Skeletal Viewer samples:
  Microsoft DirectX® SDK - June 2010 or later version
  Runtime for Microsoft DirectX® 9  
• For Speech sample (x86 only):
  Microsoft Speech Platform Runtime, version 10.2 (x86 edition)
  Microsoft Speech Platform - Software Development Kit, version 10.2 (x86 edition)
  Kinect for Windows Runtime Language Pack, version 0.9
       (acoustic model from Microsoft Speech Platform for the Kinect for Windows SDK Beta)

2. Installation

To install the SDK Beta:

1. Make sure the Kinect device is not plugged in to the USB port on your computer.  
2. Remove any other drivers for the Kinect device, including earlier versions of the Microsoft Kinect drivers.  
3. Close Visual Studio.
   Note: You must close Visual Studio before installing the SDK and then restart it after installation to pick up the MSRKINECTSDK environment variables that the SDK requires.  
4. From the download location for the SDK Beta package, double-click the link for your system: 32-bit or 64-bit.

To load the Microsoft Kinect drivers:

1. Make sure the Kinect device is plugged into an external power source.  
2. After installing the SDK Beta, plug your Kinect device into the PC's USB port.  
3. All the drivers, including audio, will load seamlessly.

If the driver is installed correctly:

1. You should see a blinking green LED on the Kinect sensor.  
2. The Kinect sensor should appear in Device Manager as the following nodes under “Microsoft Kinect”:
   • Microsoft Kinect Audio Array Control
   • Microsoft Kinect Camera
   • Microsoft Kinect Device  
3. The Kinect sensor's microphone array should appear under Sound, Video and Game Controllers in Device Manager as:
   • Kinect USB Audio  

See "Troubleshooting and Installation Notes" in the Programming Guide for more troubleshooting tips for driver installation.

3. Kinect Sensor Notes

General Sensor Notes

1. Important: The Microsoft Kinect drivers might not install correctly and will not work correctly if another driver for the Kinect device is installed on your PC.  
2. Plug the Kinect sensor's power supply into an external power source.
   With USB power only, the sensor will be minimally functional and light the LED, but it must be connected to an external power source to be fully functional.  
3. No tools are required for calibration of audio and video.  
4. Your Kinect sensor should be the only device plugged into the USB hub it uses on your computer. If you have more than one Kinect sensor, make sure that you connect them to different USB hubs.  

Protecting Your Kinect Sensor While Experimenting

Overheating

1. The Kinect sensor is protected from overheating by a fan, which is controlled by the sensor's firmware and turns off the camera at 90 degrees Celsius (194 degrees Fahrenheit).  
2. There is no software interface for applications to control the fan.

Controlling the Sensors

1. All control of the Kinect sensor is through the APIs provided in this SDK Beta.
   Caution: If you use any other mechanisms to control the Kinect sensor, you risk the possibility of irreparable damage to the sensor.  
2. Warning: The tilt mechanism in the sensor array is not rated for frequent use. You should tilt the Kinect sensor as few times as possible, to minimize wear on the camera and to minimize tilting time. The camera motor is not designed for constant or repetitive movement, and attempts to use it that way may cause degradation of motor function.
    
   In this SDK Beta, calls to NuiCameraSetElevation are limited to 1 per second and a maximum of 15 calls in any 20-second period. If the application exceeds these limits, the function imposes a short lockout period during which any further calls will return one of the following error codes:
    
   ERROR_TOO_MANY_CMDS
   The application tried to move the camera more than 15 times within 20 seconds.
    
   ERROR_RETRY
   The application must wait one second between attempts to move the camera.  
3. This SDK Beta does not support selective suspend for power management.

Controlling the Camera Array

1. This SDK Beta allows applications to control camera tilt only, not pan or zoom.  

Skeletal Tracking

1. If moving figures are too close to the camera, skeleton tracking can produce unreliable or odd results. The optimal range is approximately 4 to 11 feet (about 1.2 to 3.5 meters).  
2. This SDK Beta supports the skeleton tracking pipeline, but does not include support for velocity or rotation.  
3. Skeleton tracking supports only two players.  
4. If an application specifies depth data only — and not depth with player index — the returned image data contains depth information in the low-order 12 bits of each 16-bit value.  
5. If you are using multiple Kinect sensors, skeleton tracking works only on the first device that you initialize. To identify the device in your code, you can ask for a NUI device by index.  
6. This SDK Beta does not support the following Xbox 360 features:
   • Initializing ATG simple shaders
   • Implementing Picture-in-Picture (PiP)
   • Gestures — although you can experiment with using depth streams for gesture-like effects
   • Identity

4. Notes for Developers

General Developer Notes

1. Visual Studio Version: Visual Studio 2010 Express (or other Visual Studio 2010 edition) is required.  
2. Developer Environment: For information, see "Configuring the Development Environment" in the Programming Guide.  
3. X86 applications only: This SDK Beta supports only x86 applications. For background information, see the Programming Guide.  
4. Working with other SDKs and frameworks:
   This SDK Beta should not be used to develop prototype applications with the intent of porting those applications to the Microsoft Xbox 360 console. Microsoft recommends that developers use Xbox 360 development kits (XDK hardware and software) to create Xbox 360 applications.  
5. Virtual machines: You must run applications built with the Kinect for Windows SDK Beta in a native Windows environment. Kinect for Windows applications cannot run in a virtual machine, because the Microsoft Kinect drivers and this SDK Beta must be installed on the computer where the application is running.  

Unmanaged Code

1. Platform Target: Unmanaged applications should target Win32, not x64 or Any CPU.  
2. Linking and Path Notes: To implement a C++ application:
    
   • To use the NUI API, include MSR_NuiApi.h, which is in ../Program Files/Microsoft Research KinectSDK/inc.
    
   • To use the Kinect Audio API, include MSRKinectAudio.h, which is in ../Program Files/Microsoft Research KinectSDK/inc.
    
   • Link to MSRKinectNUI.lib, which is in ../Program Files/Microsoft Research KinectSDK/lib.
    
   • Ensure that the Kinect SDK DLLs are on your path when you run your project. They are in ../Program Files/Microsoft Research KinectSDK.  
3. Compile Notes: Do not compile projects with w_char as a separate type.  
4. Player Index: To use player index information, C++ applications must specify both NUI_INITIALIZE_FLAG_USES_DEPTH_AND_PLAYER_INDEX and NUI_INITIALIZE_FLAG_USES_SKELETON in the dwFlags parameter to NuiInitialize.  

Managed Code

1. Platform Target: Managed applications must specify an x86 platform target, not x64 or Any CPU.  
2. References: Managed code applications that use this SDK Beta must reference Microsoft.Research.Kinect.dll. This assembly is in the GAC, and can be found on the .NET tab of the Add Reference dialog box.  
3. Namespaces: Include using directives for the following namespaces:
    
   For the NUI API, add:
    
   using Microsoft.Research.Kinect.Nui;
    
   For the Audio API, add:
    
   using Microsoft.Research.Kinect.Audio;  
4. Player Index: For player index information to work properly, managed code applications that specify RuntimeOptions.UseDepthAndPlayerIndex must also specify RuntimeOptions.UseSkeletalTracking.  

5. Files and Samples

Files

In ../Program Files (x86)/Microsoft Research KinectSDK/lib/:

Microsoft.Research.Kinect.dll
An assembly that supports the managed API.

In ../Program Files (x86)/Microsoft Research KinectSDK/inc/:

MSR_KinectAudio.h
Defines the audio API, including the ISoundSourceLocalizer interface, which supports beamforming and source localization.

MSR_NuiApi.h
Aggregates all NUI API headers and defines basic initialization and access functions—NuiInitialize, NuiShutdown, MSR_NuiXxx, and INuiInstance:
• Enumerate devices
• Access multiple devices

MSR_NuiImageCamera.h
Defines API for the NUI image and camera services—NuiCameraXxx, NuiImageXxx:
• Adjust camera tilt
• Open streams and read image or depth frames

MSR_NuiProps.h
Defines the APIs for the NUI properties enumerator.

MSR_NuiSkeleton.h
Defines API for NUI Skeleton—NuiSkeletonXxx, NuiTransformXxx:
• Enable/disable skeleton tracking
• Get skeleton data
• Transform skeleton data for smoother rendering

NuiImageBuffer.h
Defines the NuiImageBuffer class, which holds image frame data. A NuiImageBuffer object is similar to a Direct3D texture to be coherent with the XDK, but has only one level and does not support mip-mapping.

Samples in This Kit

Security Note: The sample code in this SDK Beta is designed to illustrate concepts and may not illustrate the safest coding practices. Microsoft assumes no liability for incidental or consequential damages should the sample code be used for purposes other than as intended.

The SDK Beta includes the following samples, installed in your /Users/Public/Documents/Microsoft Research KinectSDK Samples/ folder:

Skeletal Viewer—Capturing Data with the NUI API (C++ and C#)
The Kinect sensor includes two cameras: one delivers depth information and the other delivers color data. The NUI API enables applications to access and manipulate this data. The SkeletalViewer sample uses the NUI API to render data from the Kinect sensor’s cameras as images on the screen. The managed sample uses WPF to render captured images, and the native application uses DirectX.

ShapeGame—Creating a Game with Audio and Skeletal Tracking
Displays the tracked skeletons of two players together with shapes falling from the sky. Players can control the shapes by moving and speaking commands.

Audio Capture Raw (C++)
The Kinect sensor’s audio component is a four-element microphone array. The AudioCaptureRaw sample uses the Windows Audio Session API (WASAPI) to capture the raw audio stream from the Kinect sensor’s microphone array and write it to a .wav file.

MicArrayEchoCancellation—Acoustic Echo Cancellation, Beam Forming, and Source Localization (C++)
The primary way for C++ applications to access the Kinect sensor’s microphone array is through the MSRKinectAudio DirectX Media Object (DMO). The MSRKinectAudio DMO supports all standard microphone array functionality, and adds support for beamforming and source localization. The MicArrayEchoCancellation sample shows how to use the KinectAudio DMO as a DirectShow® source to access the Kinect sensor’s microphone array. This sample uses acoustic echo cancellation to record a high-quality audio stream and beamforming and source localization to determine the selected beam and the direction of the sound source.

MFAudioFilter—Media Foundation Audio Filter (C++)
Shows how to capture an audio stream from the Kinect sensor’s microphone array by using the MSRKinectAudio DMO in filter mode in a Windows Media Foundation topology.

RecordAudio—Recording an Audio Stream and Monitoring Direction (C#)
Demonstrates how to capture an audio stream from the Kinect sensor’s microphone array and monitor the currently selected beam and sound source direction.

Speech—Recognizing Voice Commands (C#)
Demonstrates how to use the Kinect sensor’s microphone array with the Microsoft.Speech API to recognize voice commands.

6. Known Issues and Troubleshooting

See "Troubleshooting and Installation Notes" in the Programming Guide for more troubleshooting tips.

Audio

1. When running the MicArrayEchoCancellation sample, you should be streaming data before you start the capture loop — for example, play a music file on Windows Media Player.

Image and Skeletal Data

1. The frame number that is returned with image and skeleton frames increments by 2 for each sequential frame instead of 1.  

Multiple Sensors

1. The current SDK Beta supports the following function to return the number of devices:
    
   NUIAPI HRESULT MSR_NUIGetDeviceCount( int * pCount );
   // note capitalization

Troubleshooting Blue Screens / Deadlocking

1. Make sure that webcams and other high-bandwidth USB devices connected to a different USB bus than the Kinect sensor.

DirectX Runtime Errors in Samples

1. Make sure the driver for your PC graphics card is installed properly.
2. Make sure you have Version 9.0c or later of the DirectX runtime.

7. Related Links

Web site for Kinect for Windows SDK Beta from Microsoft Research

Visual Studio 2010 Notes and Download Information

Latest DirectX Release Notes

© 2011 Microsoft Corporation. All rights reserved. Terms of Use | Trademarks | Privacy