Kinect V1读取图像数据(For Windows)
这篇博客
刚好有一台现成的Kinect V1相机,所以就拿过来学习一下它的数据读取方式和编程方法,毕竟它还能用于跑RGBD-SLAM。Kinect V1,没错就是微软早就停产的那个相机,它仍然能战斗!下面对相机进行简单的介绍。
Kinect V1介绍
Kinect V1相机的长相如下:
Kinect V1由彩色摄像头,深度传感器,和用于语音识别的多点阵列麦克风组成。它本来是给Xbox的体感游戏游戏服务的,但由于它能获得深度图和RGB图,所以它也被研究员拿来做RGBD-SLAM的研究。关于Kinect V1的发展和更详细的介绍,大家可以参考百度百科的内容。
微软为了方便研究人员对Kinect V1进行开发,提供了专门的Kinect for Windows SDK。这个SDK主要是针对Windows7设计(Windows10也能用),内含驱动程序、丰富的原始感测数据流程式开发接口(RawSensorStreamsAPI)、自然用户接口、安装文件以及参考数据。Kinect for Windows SDK可让使用C++、C#或VisualBasic语言搭配MicrosoftVisualStudio2010(VS2019也可使用)工具的程序设计师轻易开发使用。
所以想要开发Kinect V1程序就需要找到对应的SDK和DeveloperToolkit,我使用的是V1.7的版本(SDK下载和DeveloperToolkit下载)。这两个开发工具的选择一定要根据你实际使用的机器型号来选,Kinect V1就选择V1.X版本,Kinect V2则是V2.X,不然驱动对不上。
数据读取的基本流程
下面介绍一些我自己总结的Kinect V1的图像数据读取的代码流程:
(1)先定义一个能指向Kinect对象的指针(好像只能通过指针的形式调用Kinect中的参数),并通过SDK中的函数为指针创建一个Kinect实例对象,帮助代码中的对象能够和现实设备联系上;
(2)然后就能够通过指针调用对象内置的函数,完成初始化,并在初始化时设置你准备初始化Kinect的哪些功能。实验中我初始化了RGB图和深度图两个功能;
(3)完成设备的初始化后,我们通过打开数据流通道(Kinect V1采用的是数据流的形式来传输、读取数据)的方式,开启RGB图和深度图的数据流通道,以便之后的数据获取。在开启数据流通道时,还需要使用一些句柄和事件对象来共同完成对数据流通道的配置;
(4)此时可以从开通的数据流通道中获取数据,并将此数据锁住,防止Kinect修改此数据。获取的数据会被放在一个数组中,需要一个一个地将它们赋值给OpenCV的Mat对象。这时图像就已经被提取出来了;
(5)使用完数据后,一定要记住将数据解锁,否则Kinect会卡住(因为它不能移除掉当前数据),无法传输下一个数据;
(如果还要进行RGB图和深度图的对齐,则再可以先不解锁数据)
上述就是Kinect V1读取数据的大致流程,下面来看具体实现代码。
运行代码和注释
此代码是在Windows上运行的。因为SDK提供的主要是一些头文件,所以还是用C++实现起来方便些。
<span style="color:#000000"><code class="language-cpp"><span style="color:#98c379">#<span style="color:#c678dd">include</span> <span style="color:#669900"><iostream></span></span>
<span style="color:#98c379">#<span style="color:#c678dd">include</span> <span style="color:#669900"><opencv2/opencv.hpp></span></span>
<span style="color:#5c6370">//windows的头文件,必须要,不然NuiApi.h用不了</span>
<span style="color:#98c379">#<span style="color:#c678dd">include</span> <span style="color:#669900"><Windows.h></span></span>
<span style="color:#5c6370">//Kinect for windows 的一些头文件</span>
<span style="color:#98c379">#<span style="color:#c678dd">include</span> <span style="color:#669900"><NuiApi.h></span></span>
<span style="color:#98c379">#<span style="color:#c678dd">include</span> <span style="color:#669900"><d3d11.h></span></span>
<span style="color:#5c6370">//要注意头文件的打开顺序。Windows要放在NuiApi前面。因为NuiApi会用到Windows中的一部分信息</span>
<span style="color:#5c6370">//硬记</span>
<span style="color:#98c379">#<span style="color:#c678dd">define</span> KEY_DOWN(VK_NONAME) ((GetAsyncKeyState(VK_NONAME) & 0x8000) ? 1:0) </span><span style="color:#5c6370">//主要是用来实现按键退出的功能,不需要的话可以不写</span>
<span style="color:#c678dd">using</span> <span style="color:#c678dd">namespace</span> cv<span style="color:#999999">;</span>
<span style="color:#c678dd">using</span> <span style="color:#c678dd">namespace</span> std<span style="color:#999999">;</span>
<span style="color:#5c6370">//深度相机能够探测到的</span>
<span style="color:#5c6370">//最远距离(mm)</span>
<span style="color:#c678dd">const</span> <span style="color:#c678dd">int</span> MAX_DISTANCE <span style="color:#669900">=</span> <span style="color:#98c379">3500</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//最近距离(mm)</span>
<span style="color:#c678dd">const</span> <span style="color:#c678dd">int</span> MIN_DISTANCE <span style="color:#669900">=</span> <span style="color:#98c379">200</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//彩图和深度图的图像大小</span>
<span style="color:#c678dd">const</span> LONG m_depthWidth <span style="color:#669900">=</span> <span style="color:#98c379">640</span><span style="color:#999999">;</span>
<span style="color:#c678dd">const</span> LONG m_depthHeight <span style="color:#669900">=</span> <span style="color:#98c379">480</span><span style="color:#999999">;</span>
<span style="color:#c678dd">const</span> LONG m_colorWidth <span style="color:#669900">=</span> <span style="color:#98c379">640</span><span style="color:#999999">;</span>
<span style="color:#c678dd">const</span> LONG m_colorHeight <span style="color:#669900">=</span> <span style="color:#98c379">480</span><span style="color:#999999">;</span>
<span style="color:#c678dd">const</span> LONG cBytesPerPixel <span style="color:#669900">=</span> <span style="color:#98c379">4</span><span style="color:#999999">;</span>
<span style="color:#c678dd">int</span> <span style="color:#61aeee">main</span><span style="color:#999999">(</span><span style="color:#c678dd">int</span> argc<span style="color:#999999">,</span><span style="color:#c678dd">char</span><span style="color:#669900">*</span><span style="color:#669900">*</span> argv<span style="color:#999999">)</span>
<span style="color:#999999">{</span>
Mat image_rgb<span style="color:#999999">;</span>
Mat image_depth<span style="color:#999999">;</span>
image_rgb<span style="color:#999999">.</span><span style="color:#61aeee">create</span><span style="color:#999999">(</span><span style="color:#98c379">480</span><span style="color:#999999">,</span> <span style="color:#98c379">640</span><span style="color:#999999">,</span> CV_8UC3<span style="color:#999999">)</span><span style="color:#999999">;</span>
image_depth<span style="color:#999999">.</span><span style="color:#61aeee">create</span><span style="color:#999999">(</span><span style="color:#98c379">480</span><span style="color:#999999">,</span> <span style="color:#98c379">640</span><span style="color:#999999">,</span> CV_8UC1<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//一个Kinect实例指针,专门指向一个Kinect类对象</span>
INuiSensor<span style="color:#669900">*</span> m_pNuiSensor <span style="color:#669900">=</span> <span style="color:#98c379">NULL</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//记录当前连接KINECT的数量。也可以不写</span>
<span style="color:#c678dd">int</span> iSensorCount<span style="color:#669900">=</span><span style="color:#669900">-</span><span style="color:#98c379">1</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//获得当前KINECT的数量,数量值保存在iSensorCount变量中</span>
<span style="color:#5c6370">//返回的hr应该是某中状态变量,用于判断相关函数是否运行出错</span>
HRESULT hr <span style="color:#669900">=</span> <span style="color:#61aeee">NuiGetSensorCount</span><span style="color:#999999">(</span><span style="color:#669900">&</span>iSensorCount<span style="color:#999999">)</span><span style="color:#999999">;</span><span style="color:#5c6370">//给函数传指针</span>
std<span style="color:#669900">::</span>cout <span style="color:#669900"><<</span> <span style="color:#669900">"The number of Kinects are "</span> <span style="color:#669900"><<</span> iSensorCount <span style="color:#669900"><<</span> endl<span style="color:#999999">;</span>
<span style="color:#5c6370">//按照序列(0开始)创建一系列KINETC实例</span>
<span style="color:#5c6370">//函数参数:初始化的Kinect的编号,将要指向该编号的实例的指针的地址!二维指针</span>
hr <span style="color:#669900">=</span> <span style="color:#61aeee">NuiCreateSensorByIndex</span><span style="color:#999999">(</span>iSensorCount <span style="color:#669900">-</span> <span style="color:#98c379">1</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>m_pNuiSensor<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//初始化,让其可以接收彩色和深度数据流(即函数传递过去的那个参数)</span>
<span style="color:#5c6370">//对每个创建的实例,使用其自带的初始化函数完成初始化</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiInitialize</span><span style="color:#999999">(</span>NUI_INITIALIZE_FLAG_USES_COLOR <span style="color:#669900">|</span> NUI_INITIALIZE_FLAG_USES_DEPTH<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//判断是否出错。所以hr应该是用来判断函数运行情况的变量类型</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">FAILED</span><span style="color:#999999">(</span>hr<span style="color:#999999">)</span><span style="color:#999999">)</span><span style="color:#5c6370">//主要是Debug时使用,自信的人从不Debug!!!!!!!!!!!!!!!!!!!!!!!!!</span>
<span style="color:#999999">{</span>
std<span style="color:#669900">::</span>cout <span style="color:#669900"><<</span> <span style="color:#669900">"NuiInitialize failed"</span> <span style="color:#669900"><<</span> endl<span style="color:#999999">;</span>
<span style="color:#c678dd">return</span> hr<span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#5c6370">//创建彩色图像获取下一帧事件.创建事件的函数,返回的是事件的句柄!</span>
<span style="color:#5c6370">//CreateEvent的参数可以按照下面这个作为标准</span>
HANDLE nextColorFrameEvent <span style="color:#669900">=</span> <span style="color:#61aeee">CreateEvent</span><span style="color:#999999">(</span><span style="color:#98c379">NULL</span><span style="color:#999999">,</span> TRUE<span style="color:#999999">,</span> FALSE<span style="color:#999999">,</span> <span style="color:#98c379">NULL</span><span style="color:#999999">)</span><span style="color:#999999">;</span><span style="color:#5c6370">//这个事件句柄是构建数据流通道时的输入,可以用来判断相应事件是否发生</span>
<span style="color:#5c6370">//彩色图像数据流通道的事件句柄</span>
HANDLE colorStreamHandle <span style="color:#669900">=</span> <span style="color:#98c379">NULL</span><span style="color:#999999">;</span><span style="color:#5c6370">//这个事件更像是一种指针,即在构建好数据流通道后,该指针直接关联上了数据流通道</span>
<span style="color:#5c6370">//创建(是否获取到了下一帧深度图)事件</span>
HANDLE nextDepthFrameEvent <span style="color:#669900">=</span> <span style="color:#61aeee">CreateEvent</span><span style="color:#999999">(</span><span style="color:#98c379">NULL</span><span style="color:#999999">,</span> TRUE<span style="color:#999999">,</span> FALSE<span style="color:#999999">,</span> <span style="color:#98c379">NULL</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//深度图像数据流的事件句柄</span>
HANDLE depthStreamHandle <span style="color:#669900">=</span> <span style="color:#98c379">NULL</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//为实例构建其与现实设备的数据流通道,这里NUI_IMAGE_TYPE_COLOR表示彩色图像(即构建的是彩图数据流通道)。</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamOpen</span><span style="color:#999999">(</span>NUI_IMAGE_TYPE_COLOR<span style="color:#999999">,</span> NUI_IMAGE_RESOLUTION_640x480
<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#98c379">2</span><span style="color:#999999">,</span> nextColorFrameEvent<span style="color:#999999">,</span> <span style="color:#669900">&</span>colorStreamHandle<span style="color:#999999">)</span><span style="color:#999999">;</span><span style="color:#5c6370">//colorStreamHandle句柄会和这个构建好的流通道相关联</span>
<span style="color:#5c6370">//Kinect实例构建与现实设备间深度图像的数据流通道</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamOpen</span><span style="color:#999999">(</span>NUI_IMAGE_TYPE_DEPTH<span style="color:#999999">,</span> NUI_IMAGE_RESOLUTION_640x480
<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#98c379">2</span><span style="color:#999999">,</span> nextDepthFrameEvent<span style="color:#999999">,</span> <span style="color:#669900">&</span>depthStreamHandle<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//设置用OpenCV的窗口来显示图像</span>
cv<span style="color:#669900">::</span><span style="color:#61aeee">namedWindow</span><span style="color:#999999">(</span><span style="color:#669900">"colorImage"</span><span style="color:#999999">,</span> CV_WINDOW_AUTOSIZE<span style="color:#999999">)</span><span style="color:#999999">;</span>
cv<span style="color:#669900">::</span><span style="color:#61aeee">moveWindow</span><span style="color:#999999">(</span><span style="color:#669900">"colorImage"</span><span style="color:#999999">,</span> <span style="color:#98c379">300</span><span style="color:#999999">,</span> <span style="color:#98c379">600</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
cv<span style="color:#669900">::</span><span style="color:#61aeee">namedWindow</span><span style="color:#999999">(</span><span style="color:#669900">"depthImage"</span><span style="color:#999999">,</span> CV_WINDOW_AUTOSIZE<span style="color:#999999">)</span><span style="color:#999999">;</span>
cv<span style="color:#669900">::</span><span style="color:#61aeee">moveWindow</span><span style="color:#999999">(</span><span style="color:#669900">"depthImage"</span><span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#98c379">200</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//循环读取图像并显示</span>
<span style="color:#c678dd">while</span> <span style="color:#999999">(</span><span style="color:#98c379">1</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//这个用来保存Kinect数据流中的某一时刻的数据</span>
NUI_IMAGE_FRAME pImageFrame_rgb<span style="color:#999999">;</span>
NUI_IMAGE_FRAME pImageFrame_depth<span style="color:#999999">;</span>
<span style="color:#5c6370">//WaitForSingleObject是一种Windows API函数,接受(事件句柄,等待时间ms)两个参数</span>
<span style="color:#5c6370">//该函数会在指定等待时间内,查看事件是否发生。在时间内事件发生,返回0;否则返回其他状态值(依情况而定)</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">WaitForSingleObject</span><span style="color:#999999">(</span>nextColorFrameEvent<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span> <span style="color:#669900">==</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//从指定的数据流通道(由句柄给定)中得到数据,用指针指向获得的数据。0表示等待多久后再获取数据(注意此时获得的数据不只有帧数据)</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamGetNextFrame</span><span style="color:#999999">(</span>colorStreamHandle<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>pImageFrame_rgb<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">FAILED</span><span style="color:#999999">(</span>hr<span style="color:#999999">)</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
std<span style="color:#669900">::</span>cout <span style="color:#669900"><<</span> <span style="color:#669900">"Could not get color image"</span> <span style="color:#669900"><<</span> endl<span style="color:#999999">;</span>
m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiShutdown</span><span style="color:#999999">(</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#c678dd">return</span> <span style="color:#669900">-</span><span style="color:#98c379">1</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#5c6370">//INuiFrameTexture一个容纳图像帧数据的对象(帧数据)</span>
INuiFrameTexture<span style="color:#669900">*</span> pTexture <span style="color:#669900">=</span> pImageFrame_rgb<span style="color:#999999">.</span>pFrameTexture<span style="color:#999999">;</span>
NUI_LOCKED_RECT lockedRect<span style="color:#999999">;</span>
<span style="color:#5c6370">//提取数据帧到LockedRect,它包括两个数据对象:pitch每行字节数,pBits第一个字节地址</span>
<span style="color:#5c6370">//锁定数据,这样当我们读数据的时候,kinect就不会去修改它。之后要记住解锁</span>
pTexture<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">LockRect</span><span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>lockedRect<span style="color:#999999">,</span> <span style="color:#98c379">NULL</span><span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//确认获得的数据是否有效:字节数不为0</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span>lockedRect<span style="color:#999999">.</span>Pitch <span style="color:#669900">!=</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//将数据转换成为OpenCV的Mat格式</span>
<span style="color:#5c6370">//转换的方式可以用扫描来形容,即把相应位置的数据给Mat中相应像素点赋值</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> i <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> i <span style="color:#669900"><</span> image_rgb<span style="color:#999999">.</span>rows<span style="color:#999999">;</span> i<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//第i行指针</span>
uchar<span style="color:#669900">*</span> ptr <span style="color:#669900">=</span> image_rgb<span style="color:#999999">.</span><span style="color:#61aeee">ptr</span><span style="color:#999999">(</span>i<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//每个字节代表一个颜色信息,直接使用uchar</span>
uchar<span style="color:#669900">*</span> pBuffer <span style="color:#669900">=</span> <span style="color:#999999">(</span>uchar<span style="color:#669900">*</span><span style="color:#999999">)</span><span style="color:#999999">(</span>lockedRect<span style="color:#999999">.</span>pBits<span style="color:#999999">)</span> <span style="color:#669900">+</span> i <span style="color:#669900">*</span> lockedRect<span style="color:#999999">.</span>Pitch<span style="color:#999999">;</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> j <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> j <span style="color:#669900"><</span> image_rgb<span style="color:#999999">.</span>cols<span style="color:#999999">;</span> j<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//读到的图像数据是4个字节,0-1-2是BGR,第4个现在未使用</span>
ptr<span style="color:#999999">[</span><span style="color:#98c379">3</span><span style="color:#669900">*</span>j<span style="color:#999999">]</span> <span style="color:#669900">=</span> pBuffer<span style="color:#999999">[</span><span style="color:#98c379">4</span><span style="color:#669900">*</span>j<span style="color:#999999">]</span><span style="color:#999999">;</span>
ptr<span style="color:#999999">[</span><span style="color:#98c379">3</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">1</span><span style="color:#999999">]</span> <span style="color:#669900">=</span> pBuffer<span style="color:#999999">[</span><span style="color:#98c379">4</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
ptr<span style="color:#999999">[</span><span style="color:#98c379">3</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">2</span><span style="color:#999999">]</span> <span style="color:#669900">=</span> pBuffer<span style="color:#999999">[</span><span style="color:#98c379">4</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">2</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#61aeee">imshow</span><span style="color:#999999">(</span><span style="color:#669900">"colorImage"</span><span style="color:#999999">,</span> image_rgb<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//waitKey(1);</span>
<span style="color:#5c6370">//搞定后记得要给数据解除锁定:</span>
pTexture<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">UnlockRect</span><span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//此外还要把获取到的帧释放掉。因为pImage是指针,实际数据还是在那个流通道内。不释放可能会造成通道阻塞吧</span>
m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamReleaseFrame</span><span style="color:#999999">(</span>colorStreamHandle<span style="color:#999999">,</span> <span style="color:#669900">&</span>pImageFrame_rgb<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#c678dd">else</span>
<span style="color:#999999">{</span>
cout <span style="color:#669900"><<</span> <span style="color:#669900">"Buffer length of received texture is bogus\r\n"</span> <span style="color:#669900"><<</span> endl<span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#5c6370">//这两个变量是在进行深度图--彩图对齐时会使用</span>
<span style="color:#5c6370">//如果要对齐的话,对齐部分的操作最好和深度图处理放在一起进行,不然会出现一些奇怪的问题</span>
BOOL nearMode<span style="color:#999999">;</span>
INuiFrameTexture<span style="color:#669900">*</span> pColorToDepthTexture<span style="color:#999999">;</span>
<span style="color:#5c6370">//下面是如何获得深度图</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">WaitForSingleObject</span><span style="color:#999999">(</span>nextDepthFrameEvent<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span> <span style="color:#669900">==</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//从指定的数据流通道内得到数据,pImageFrame指向读取到的数据</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamGetNextFrame</span><span style="color:#999999">(</span>depthStreamHandle<span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>pImageFrame_depth<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//这句话主要是为了之后进行彩图--深度图对齐时使用。如果只是为了获得深度图可以不写这句话</span>
<span style="color:#5c6370">//pColorToDepthTexture应该是彩图和深度图之间像素的偏移度的值。</span>
hr <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageFrameGetDepthImagePixelFrameTexture</span><span style="color:#999999">(</span>
depthStreamHandle<span style="color:#999999">,</span> <span style="color:#669900">&</span>pImageFrame_depth<span style="color:#999999">,</span> <span style="color:#669900">&</span>nearMode<span style="color:#999999">,</span> <span style="color:#669900">&</span>pColorToDepthTexture
<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//以下部分和处理彩图差不多</span>
INuiFrameTexture<span style="color:#669900">*</span> pTexture <span style="color:#669900">=</span> pImageFrame_depth<span style="color:#999999">.</span>pFrameTexture<span style="color:#999999">;</span>
NUI_LOCKED_RECT lockedRect<span style="color:#999999">;</span>
NUI_LOCKED_RECT ColorToDepthLockRect<span style="color:#999999">;</span>
pTexture<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">LockRect</span><span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>lockedRect<span style="color:#999999">,</span> <span style="color:#98c379">NULL</span><span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
pColorToDepthTexture<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">LockRect</span><span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">,</span> <span style="color:#669900">&</span>ColorToDepthLockRect<span style="color:#999999">,</span> <span style="color:#98c379">NULL</span><span style="color:#999999">,</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> i <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> i <span style="color:#669900"><</span> image_depth<span style="color:#999999">.</span>rows<span style="color:#999999">;</span> i<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
uchar<span style="color:#669900">*</span> prt <span style="color:#669900">=</span> image_depth<span style="color:#999999">.</span>ptr<span style="color:#669900"><</span>uchar<span style="color:#669900">></span><span style="color:#999999">(</span>i<span style="color:#999999">)</span><span style="color:#999999">;</span>
uchar<span style="color:#669900">*</span> pBuffer <span style="color:#669900">=</span> <span style="color:#999999">(</span>uchar<span style="color:#669900">*</span><span style="color:#999999">)</span><span style="color:#999999">(</span>lockedRect<span style="color:#999999">.</span>pBits<span style="color:#999999">)</span> <span style="color:#669900">+</span> i <span style="color:#669900">*</span> lockedRect<span style="color:#999999">.</span>Pitch<span style="color:#999999">;</span>
<span style="color:#5c6370">//这里需要转换,因为每个深度数据是2个字节,应将BYTE转成USHORT</span>
USHORT<span style="color:#669900">*</span> pBufferRun <span style="color:#669900">=</span> <span style="color:#999999">(</span>USHORT<span style="color:#669900">*</span><span style="color:#999999">)</span>pBuffer<span style="color:#999999">;</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> j <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> j <span style="color:#669900"><</span> image_depth<span style="color:#999999">.</span>cols<span style="color:#999999">;</span> j<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//先向,将数据归一化处理,对深度距离在300mm-3500mm范围内的像素,映射到【0—255】内,</span>
<span style="color:#5c6370">//超出范围的,都去做是边缘像素</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span>pBufferRun<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900"><<</span> <span style="color:#98c379">3</span> <span style="color:#669900">></span> MAX_DISTANCE<span style="color:#999999">)</span> prt<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900">=</span> <span style="color:#98c379">255</span><span style="color:#999999">;</span><span style="color:#5c6370">//这里的左移3不知道是为什么??</span>
<span style="color:#c678dd">else</span> <span style="color:#c678dd">if</span> <span style="color:#999999">(</span>pBufferRun<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900"><<</span> <span style="color:#98c379">3</span> <span style="color:#669900"><</span> MIN_DISTANCE<span style="color:#999999">)</span> prt<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span>
<span style="color:#c678dd">else</span> prt<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900">=</span> <span style="color:#999999">(</span>BYTE<span style="color:#999999">)</span><span style="color:#999999">(</span><span style="color:#98c379">256</span> <span style="color:#669900">*</span> <span style="color:#999999">(</span>pBufferRun<span style="color:#999999">[</span>j<span style="color:#999999">]</span> <span style="color:#669900"><<</span> <span style="color:#98c379">3</span><span style="color:#999999">)</span> <span style="color:#669900">/</span> MAX_DISTANCE<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#61aeee">imshow</span><span style="color:#999999">(</span><span style="color:#669900">"depthImage"</span><span style="color:#999999">,</span> image_depth<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//waitKey(1);</span>
<span style="color:#5c6370">//接下来是对齐部分,将前景抠出来</span>
<span style="color:#5c6370">//存放深度点的参数</span>
NUI_DEPTH_IMAGE_POINT<span style="color:#669900">*</span> depthPoints <span style="color:#669900">=</span> <span style="color:#c678dd">new</span> NUI_DEPTH_IMAGE_POINT<span style="color:#999999">[</span><span style="color:#98c379">640</span> <span style="color:#669900">*</span> <span style="color:#98c379">480</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span>ColorToDepthLockRect<span style="color:#999999">.</span>Pitch <span style="color:#669900">!=</span> <span style="color:#98c379">0</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
HRESULT hrState <span style="color:#669900">=</span> S_OK<span style="color:#999999">;</span>
<span style="color:#5c6370">//一个能在不同空间坐标转变的类(包括:深度,彩色,骨骼)</span>
INuiCoordinateMapper<span style="color:#669900">*</span> pMapper<span style="color:#999999">;</span>
<span style="color:#5c6370">//设置KINECT实例的空间坐标系</span>
hrState <span style="color:#669900">=</span> m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiGetCoordinateMapper</span><span style="color:#999999">(</span><span style="color:#669900">&</span>pMapper<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">FAILED</span><span style="color:#999999">(</span>hrState<span style="color:#999999">)</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#c678dd">return</span> hrState<span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#5c6370">//重要的一步:从颜色空间映射到深度空间。参数说明:</span>
<span style="color:#5c6370">//【参数1】:彩色图像的类型</span>
<span style="color:#5c6370">//【参数2】:彩色图像的分辨率</span>
<span style="color:#5c6370">//【参数3】:深度图像的分辨率</span>
<span style="color:#5c6370">//【参数4】:深度图像像素点的个数</span>
<span style="color:#5c6370">//【参数5】:深度图所有的像素点</span>
<span style="color:#5c6370">//【参数6】:取内存的大小,个数。类型为NUI_DEPTH_IMAGE_PIXEL</span>
<span style="color:#5c6370">//【参数7】:存放映射结果点的参数</span>
hrState <span style="color:#669900">=</span> pMapper<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">MapColorFrameToDepthFrame</span><span style="color:#999999">(</span>NUI_IMAGE_TYPE_COLOR<span style="color:#999999">,</span> NUI_IMAGE_RESOLUTION_640x480<span style="color:#999999">,</span> NUI_IMAGE_RESOLUTION_640x480<span style="color:#999999">,</span>
<span style="color:#98c379">640</span> <span style="color:#669900">*</span> <span style="color:#98c379">480</span><span style="color:#999999">,</span> <span style="color:#999999">(</span>NUI_DEPTH_IMAGE_PIXEL<span style="color:#669900">*</span><span style="color:#999999">)</span>ColorToDepthLockRect<span style="color:#999999">.</span>pBits<span style="color:#999999">,</span> <span style="color:#98c379">640</span> <span style="color:#669900">*</span> <span style="color:#98c379">480</span><span style="color:#999999">,</span> depthPoints<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//该函数根据之前获得的位移图像(彩图和深度图间偏移量),将彩图上的像素依次投影到深度图上。</span>
<span style="color:#5c6370">//depthPoints保存了彩图中某像素在深度图上的坐标,以及相应的深度值。</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">FAILED</span><span style="color:#999999">(</span>hrState<span style="color:#999999">)</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#c678dd">return</span> hrState<span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#5c6370">//显示的图像</span>
Mat show<span style="color:#999999">;</span>
show<span style="color:#999999">.</span><span style="color:#61aeee">create</span><span style="color:#999999">(</span><span style="color:#98c379">480</span><span style="color:#999999">,</span> <span style="color:#98c379">640</span><span style="color:#999999">,</span> CV_8UC3<span style="color:#999999">)</span><span style="color:#999999">;</span>
show <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//在知道彩图中各像素点的深度值情况下,只显示深度小于一定范围的点</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> i <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> i <span style="color:#669900"><</span> image_rgb<span style="color:#999999">.</span>rows<span style="color:#999999">;</span> i<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#c678dd">for</span> <span style="color:#999999">(</span><span style="color:#c678dd">int</span> j <span style="color:#669900">=</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span> j <span style="color:#669900"><</span> image_rgb<span style="color:#999999">.</span>cols<span style="color:#999999">;</span> j<span style="color:#669900">++</span><span style="color:#999999">)</span>
<span style="color:#999999">{</span>
uchar<span style="color:#669900">*</span> prt_rgb <span style="color:#669900">=</span> image_rgb<span style="color:#999999">.</span><span style="color:#61aeee">ptr</span><span style="color:#999999">(</span>i<span style="color:#999999">)</span><span style="color:#999999">;</span>
uchar<span style="color:#669900">*</span> prt_show <span style="color:#669900">=</span> show<span style="color:#999999">.</span><span style="color:#61aeee">ptr</span><span style="color:#999999">(</span>i<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//在内存中偏移量</span>
<span style="color:#c678dd">long</span> index <span style="color:#669900">=</span> i <span style="color:#669900">*</span> <span style="color:#98c379">640</span> <span style="color:#669900">+</span> j<span style="color:#999999">;</span>
<span style="color:#5c6370">//从保存了映射坐标的数组中获取点</span>
NUI_DEPTH_IMAGE_POINT depthPointAtIndex <span style="color:#669900">=</span> depthPoints<span style="color:#999999">[</span>index<span style="color:#999999">]</span><span style="color:#999999">;</span>
<span style="color:#5c6370">//边界判断</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span>depthPointAtIndex<span style="color:#999999">.</span>x <span style="color:#669900">>=</span> <span style="color:#98c379">0</span> <span style="color:#669900">&&</span> depthPointAtIndex<span style="color:#999999">.</span>x <span style="color:#669900"><</span> image_depth<span style="color:#999999">.</span>cols <span style="color:#669900">&&</span>
depthPointAtIndex<span style="color:#999999">.</span>y <span style="color:#669900">>=</span> <span style="color:#98c379">0</span> <span style="color:#669900">&&</span> depthPointAtIndex<span style="color:#999999">.</span>y <span style="color:#669900"><</span> image_depth<span style="color:#999999">.</span>rows<span style="color:#999999">)</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//深度判断,在MIN_DISTANCE与MAX_DISTANCE之间的当成前景,显示出来</span>
<span style="color:#5c6370">//这个使用也很重要,当使用真正的深度像素点再在深度图像中获取深度值来判断的时候,会出错</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span>depthPointAtIndex<span style="color:#999999">.</span>depth <span style="color:#669900">>=</span> MIN_DISTANCE <span style="color:#669900">&&</span> depthPointAtIndex<span style="color:#999999">.</span>depth <span style="color:#669900"><=</span> MAX_DISTANCE<span style="color:#999999">)</span>
<span style="color:#999999">{</span>
prt_show<span style="color:#999999">[</span><span style="color:#98c379">3</span> <span style="color:#669900">*</span> j<span style="color:#999999">]</span> <span style="color:#669900">=</span> prt_rgb<span style="color:#999999">[</span>j <span style="color:#669900">*</span> <span style="color:#98c379">3</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
prt_show<span style="color:#999999">[</span><span style="color:#98c379">3</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">1</span><span style="color:#999999">]</span> <span style="color:#669900">=</span> prt_rgb<span style="color:#999999">[</span>j <span style="color:#669900">*</span> <span style="color:#98c379">3</span> <span style="color:#669900">+</span> <span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
prt_show<span style="color:#999999">[</span><span style="color:#98c379">3</span> <span style="color:#669900">*</span> j <span style="color:#669900">+</span> <span style="color:#98c379">2</span><span style="color:#999999">]</span> <span style="color:#669900">=</span> prt_rgb<span style="color:#999999">[</span>j <span style="color:#669900">*</span> <span style="color:#98c379">3</span> <span style="color:#669900">+</span> <span style="color:#98c379">2</span><span style="color:#999999">]</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#61aeee">imshow</span><span style="color:#999999">(</span><span style="color:#669900">"show"</span><span style="color:#999999">,</span> show<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#c678dd">delete</span><span style="color:#999999">[</span><span style="color:#999999">]</span>depthPoints<span style="color:#999999">;</span>
pTexture<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">UnlockRect</span><span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiImageStreamReleaseFrame</span><span style="color:#999999">(</span>depthStreamHandle<span style="color:#999999">,</span> <span style="color:#669900">&</span>pImageFrame_depth<span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#c678dd">else</span>
<span style="color:#999999">{</span>
<span style="color:#5c6370">//这里是获得的深度图数据不好时的情况</span>
cout <span style="color:#669900"><<</span> <span style="color:#669900">"Buffer length of received texture is bogus\r\n"</span> <span style="color:#669900"><<</span> endl<span style="color:#999999">;</span>
<span style="color:#999999">}</span>
<span style="color:#999999">}</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">cvWaitKey</span><span style="color:#999999">(</span><span style="color:#98c379">20</span><span style="color:#999999">)</span> <span style="color:#669900">==</span> <span style="color:#98c379">27</span><span style="color:#999999">)</span>
<span style="color:#c678dd">break</span><span style="color:#999999">;</span>
<span style="color:#c678dd">if</span> <span style="color:#999999">(</span><span style="color:#61aeee">KEY_DOWN</span><span style="color:#999999">(</span><span style="color:#669900">'Q'</span><span style="color:#999999">)</span><span style="color:#999999">)</span><span style="color:#5c6370">//如果键盘输入流'Q'则推出循环</span>
<span style="color:#c678dd">break</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
m_pNuiSensor<span style="color:#669900">-</span><span style="color:#669900">></span><span style="color:#61aeee">NuiShutdown</span><span style="color:#999999">(</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#61aeee">destroyAllWindows</span><span style="color:#999999">(</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
<span style="color:#c678dd">return</span> <span style="color:#98c379">0</span><span style="color:#999999">;</span>
<span style="color:#999999">}</span>
</code></span>
结尾
在知道Kinect V1的基础使用后,就能自己编写一些Demo来玩,比如通过OpenCV自带人脸识别工具和Kinect相机,完成摄像头人脸检测跟踪功能等。但这些都是在Windows上实现的,而SLAM在Ubuntu这类Linux系统上运行会更方便一些。所以下一步就是学习如何在Ubuntu和ROS上使用Kinect。