用于双目视觉的程序框架，里面是代码和一些说明

最新推荐文章于 2022-12-16 21:24:44 发布

mengaim_cn

最新推荐文章于 2022-12-16 21:24:44 发布

阅读量7.1k

点赞数

分类专栏：工作日记文章标签：框架 cam filter graph null video

本文链接：https://blog.csdn.net/mengaim_cn/article/details/326970

版权

工作日记专栏收录该内容

25 篇文章 1 订阅

订阅专栏

这是大约一个多星期前完成了大部分的框架，现在为了更好的试验，又进行了些修改。修改，应该还是没有结束的，现在，有个毛病，呆呆的看代码，就是看不进去，所以，打算，一边写这篇文章，一边修改自己的代码。

对于，双目视觉而言，重要的两个部分是摄像机标定和图像中的特征的提取。但是，在这个程序中，这两部分都没有，这个程序只是一个用于实现双目视觉的平台。为双目视觉的实现搭一个架子。

一般双目视觉用于测距，会用到的是带图像采集卡的ccd的摄像头，对于这样的摄像头，图像的采集是没有问题的，在图像采集卡中会有自己带的函数来实现。但是，在这里，我们要用到的是usb的摄像头，也就是普通的用于视频聊天的webCam。对于这样的摄像头，要进行双目视觉的试验，只有自己捕捉视频流了，这需要自己写代码。在普通的windows平台下进行视频流捕捉的方法有两种，一种是vfw，一种是directshow，在下面的程序中，我用到的就是directshow技术。

好了，下面具体对这个程序进行说明。

这个程序具体由视频捕捉，双目视觉算法，数值算法和整个程序的控制这几个部分组成。其中，为了实时的进行双目视觉，而不是一般的只是得到两个图片，然后，再离线的进行处理，这样，这个程序的控制部分是用了好几个线程，这些线程函数都是全局的函数，也就是说，由这些函数控制视频捕捉类，双目视觉算法类和数值算法类中的函数进行具体的处理。

首先，讲一下两个摄像头的视频流捕捉代码。

这也就是一个建立filter graph的过程。这里我是用一个类实现的。

//buildTwoCamFilterGraph.h

#ifndef _BUILDTWOCAMFILTERGRAPH_H_
#define _BUILDTWOCAMFILTERGRAPH_H_

#include <DShow.h>
//ISampleGrabberCB,因为这里面的一个类是继承了directshow基类类库中的类,这一个头文件必须加上。

#include <Qedit.h>
#include <Windows.h>

#include <Streams.h>

#include "dataBuffer.h"

#pragma comment(lib,"strmiids.lib")
#pragma comment(lib,"strmbase.lib")

//下面的这个类是一个辅助的类，当要捕捉相应帧的图像时要用
//

extern DATABUFFER cb1;//for camera 1
extern DATABUFFER cb2;//for camera 2

extern bool g_bOneShot1;
extern bool g_bOneShot2;

extern HANDLE hEventProcessData1;
extern HANDLE hEventProcessData2;

// Note: this object is a SEMI-COM object, and can only be created statically.
// We use this little semi-com object to handle the sample-grab-callback,
// since the callback must provide a COM interface. We could have had an interface
// where you provided a function-call callback, but that's really messy, so we
// did it this way. You can put anything you want into this C++ object, even
// a pointer to a CDialog. Be aware of multi-thread issues though.
//
//

class CSampleGrabberCB : public ISampleGrabberCB
{
public:
// these will get set by the main thread below. We need to
// know this in order to write out the bmp
long lWidth;
long lHeight;

DATABUFFER *pcb;
bool *pbOneShot;

HANDLE *phEventProcessData;

TCHAR m_szCapDir[MAX_PATH];// the directory we want to capture to
TCHAR m_szSnappedName[MAX_PATH];

  //Constructor
CSampleGrabberCB(DATABUFFER *pdataBuffer,bool *pg_bOneShot,HANDLE *phEvent):
pcb(pdataBuffer),pbOneShot(pg_bOneShot),phEventProcessData(phEvent)
{
  ZeroMemory(m_szCapDir,sizeof(m_szCapDir));
  ZeroMemory(m_szSnappedName,sizeof(m_szSnappedName));
  }

// fake out any COM ref counting
STDMETHODIMP_(ULONG) AddRef(){return 2;}
STDMETHODIMP_(ULONG) Release(){return 1;}

// fake out any COM QI'ing
STDMETHODIMP QueryInterface(REFIID riid,void **ppv)
{
  if(riid==IID_ISampleGrabberCB||riid==IID_IUnknown)
  {
   *ppv=(void*)static_cast<ISampleGrabberCB*>(this);
   return NOERROR;
  }
  return E_NOINTERFACE;
}

// we don't implement this interface for this example
STDMETHODIMP SampleCB(double SampleTime,IMediaSample *pSample)
{
return 0;
}

//下面是不用SampleCB的原因

    // The sample grabber is calling us back on its deliver thread.
    // This is NOT the main app thread!
    //
    //           !!!!! WARNING WARNING WARNING !!!!!
    //
    // On Windows 9x systems, you are not allowed to call most of the
    // Windows API functions in this callback. Why not? Because the
    // video renderer might hold the global Win16 lock so that the video
    // surface can be locked while you copy its data. This is not an
    // issue on Windows 2000, but is a limitation on Win95,98,98SE, and ME.
    // Calling a 16-bit legacy function could lock the system, because
    // it would wait forever for the Win16 lock, which would be forever
    // held by the video renderer.
    //
    // As a workaround, copy the bitmap data during the callback,
    // post a message to our app, and write the data later.
//

STDMETHODIMP BufferCB( double dblSampleTime, BYTE * pBuffer, long lBufferSize )
    {
        // this flag will get set to true in order to take a picture
        //
  //每一帧图像传来时，都会进入这个函数的，需要一个判断，只有当需要的时候才
  //把图像的数据提取出来保存下来。
       if( !*pbOneShot )
            return 0;

if (!pBuffer)
return E_POINTER;

        //if( cb.lBufferSize < lBufferSize )
  if(pcb->lBufferSize!=lBufferSize)
        {
            delete [] pcb->pBuffer;
            pcb->pBuffer = NULL;
            pcb->lBufferSize = 0;
        }

        // Since we can't access Windows API functions in this callback, just
        // copy the bitmap data to a global structure for later reference.
        pcb->dblSampleTime = dblSampleTime;

  //-------------------------------------
  //下面是为了保存要保存的帧到达的时间，最后是要显示帧的到达速度
  //注意，下面的代码是这样写了，但是，感觉传给我的数据不对，所以，虽然现在留着，但是也可先不要

  pcb->dblSampleTimeBefore=pcb->dblSampleTimeNow;
  pcb->dblSampleTimeNow=dblSampleTime;
  pcb->dblSampleTimeBeforeAll+=pcb->dblSampleTimeBefore;
  pcb->dblSampleTimeNowAll+=pcb->dblSampleTimeNow;

pcb->dblSampleTimeBetween=pcb->dblSampleTimeNow-pcb->dblSampleTimeBefore;

pcb->cnt++;

//------------------------------------------

        // If we haven't yet allocated the data buffer, do it now.
        // Just allocate what we need to store the new bitmap.
        if (!pcb->pBuffer)
        {
            pcb->pBuffer = new BYTE[lBufferSize];
            pcb->lBufferSize = lBufferSize;
        }

        if( !pcb->pBuffer )
        {
            pcb->lBufferSize = 0;
            return E_OUTOFMEMORY;
        }

// Copy the bitmap data into our global buffer
memcpy(pcb->pBuffer, pBuffer, lBufferSize);

  //set the size of the bitmap
  //
  pcb->lWidth=lWidth;
  pcb->lHeight=lHeight;

  //在这里应该是通知需要的帧已经捕捉好了，来处理吧
  //when the datas have been ready,
        // Post a message to our application, telling it to come back
        // and write the saved data to a bitmap file on the user's disk.
  //激活某个event

  //set the g_bOneShot to be false
  //
  *pbOneShot=false;
  //and single the event
  //
  SetEvent(*phEventProcessData);
        return 0;
    }
};

//捕捉两个摄像头的视频流的类的真正的开始
class BuildTwoCamFilterGraph
{
public:
BuildTwoCamFilterGraph(DATABUFFER *pcb1,DATABUFFER *pcb2);
~BuildTwoCamFilterGraph();

//重新建立filter graph，两个摄像头的
//
void rebuildFilterGraph(void);
//下面是重建一个filter graph的函数
//
void rebuildFilterGraph1(void);
void rebuildFilterGraph2(void);
//解除filter graph之间的连接等等,两个摄像头的
//
void teardownFilterGraph(void);
//解除filter graph之间的连接等等，下面的函数是针对一个摄像头的
//
void teardownFilterGraph1(void);
void teardownFilterGraph2(void);

//the inline functions-----------------------------------
//
//对于第一个camera的filter graph
//void setPinCam1(void);no use now ,this must put to another class
//void setCaptureFilterCam1(void);
IVideoWindow* getIVideoWindow1()
{
  return m_pVidWin1;
}
IMediaControl* getIMediaControl1()
{
  return m_pMediaControl1;
}
IGraphBuilder* getIGraphBuilder1()
{
  return m_pGraph1;
}
IBaseFilter* getCaptureFilter1()
{
  return m_pVCap1;
}
ICaptureGraphBuilder2* getICaptureGraphBuilder21()
{
        return m_pGraphBuilder21;
}

//对于第二个camera的filter graph
//void setPinCam2(void);no use now ,this must put to another class
//void setCaptureFilterCam2(void);
IVideoWindow* getIVideoWindow2()
{
  return m_pVidWin2;
}
IMediaControl* getIMediaControl2()
{
  return m_pMediaControl2;
}
IGraphBuilder* getIGraphBuilder2()
{
  return m_pGraph2;
}
IBaseFilter* getCaptureFilter2()
{
  return m_pVCap2;
}
ICaptureGraphBuilder2* getICaptureGraphBuilder22()
{
        return m_pGraphBuilder22;
}

//set the bool value
void setFilterGraph1Run(bool bValue)
{
  m_bPreview1=bValue;
}
void setFilterGraph2Run(bool bValue)
{
  m_bPreview2=bValue;
}
void setFilterGraph1Build(bool bValue)
{
  m_bFilterGraphCam1=bValue;
}
void setFilterGraph2Build(bool bValue)
{
  m_bFilterGraphCam2=bValue;
}

//are the filter graph have been build?
bool isFilterGraph1Built(void)
{
return m_bFilterGraphCam1;
}
bool isFilterGraph2Built(void)
{
return m_bFilterGraphCam2;
}

//are the filter garph running?
bool isFilterGraph1Run(void)
{
return m_bPreview1;
}

bool isFilterGraph2Run(void)
{
return m_bPreview2;
}

private:
//用于设置显示的一些属性等等
IVideoWindow* m_pVidWin1;
IVideoWindow* m_pVidWin2;

//用于对创建好的filter graph进行控制的，即stop ,run和pause
IGraphBuilder* m_pGraph1;
IGraphBuilder* m_pGraph2;

//在视频流捕捉的时候，可以方便的生成fiter graph
ICaptureGraphBuilder2* m_pGraphBuilder21;
ICaptureGraphBuilder2* m_pGraphBuilder22;

//这是用于截获图像的filter
IBaseFilter *m_pGrabberF1;
IBaseFilter *m_pGrabberF2;

//截取图像的filter上面的接口
ISampleGrabber *m_pGrabber1;
ISampleGrabber *m_pGrabber2;

// the capture filter
IBaseFilter* m_pVCap1;
IBaseFilter* m_pVCap2;

//the render filter
IBaseFilter *m_pRenderer1;
IBaseFilter *m_pRenderer2;

//the media control
IMediaControl* m_pMediaControl1;
IMediaControl* m_pMediaControl2;

//判断是否已经建立了filter graph
//
bool m_bFilterGraphCam1;
bool m_bFilterGraphCam2;

//判断是否在进行预览，也就是是否run了
//
bool m_bPreview1;
bool m_bPreview2;

//一些辅助性的函数----------------------------------
//

//Device enum function for two camera,and set the capture filter directly
//
void deviceEnum(void);

void InitStillGraph(IGraphBuilder **ppGraph,
                  ICaptureGraphBuilder2 **ppGraphBuilder2,
      IBaseFilter **ppVCap,
      IBaseFilter **ppGrabberF,
      IBaseFilter **ppRenderer,
      ISampleGrabber **ppGrabber//,
      //IVideoWindow **ppVidWin,
      //IMediaControl **ppMediaControl
      );

//用于断开filter连接的辅助函数
//
void NukeDownstream(IBaseFilter *pf,IGraphBuilder *pFg);
void TearDownGraph(IGraphBuilder *pFg,IVideoWindow *pVW,IBaseFilter *pVCap);

/*----------------------------------------------------------------------------------
This semi_COM object will receive Sample callbacks for us
---------------------------------------------------------------------------------*/
CSampleGrabberCB mCB1;
CSampleGrabberCB mCB2;

};

#endif

//buildTwoCamFilterGraph.cpp

#include "stdafx.h"

#include "buildTwoCamFilterGraph.h"

#include "dataBuffer.h"//store the definition of the global structure

/*----------------------------------------------------------
此函数用于创建两个filter graph
首先，初始化com组件，枚举硬件设备，然后创建两个filter graph
---------------------------------------------------------*/
BuildTwoCamFilterGraph::BuildTwoCamFilterGraph(DATABUFFER *pcb1,DATABUFFER *pcb2):
m_pVidWin1(NULL),
m_pVidWin2(NULL),
m_pGraph1(NULL),
m_pGraph2(NULL),
m_pGraphBuilder21(NULL),
m_pGraphBuilder22(NULL),
m_pGrabberF1(NULL),
m_pGrabberF2(NULL),
m_pGrabber1(NULL),
m_pGrabber2(NULL),
m_pVCap1(NULL),
m_pVCap2(NULL),
m_pRenderer1(NULL),
m_pRenderer2(NULL),
m_pMediaControl1(NULL),
m_pMediaControl2(NULL),
m_bFilterGraphCam1(false),
m_bFilterGraphCam2(false),
m_bPreview1(false),
m_bPreview2(false),
mCB1(pcb1,&g_bOneShot1,&hEventProcessData1),
mCB2(pcb2,&g_bOneShot2,&hEventProcessData2)
{
//mCB1=CSampleGrabberCB(cb1);
//mCB2=CSampleGrabberCB(cb2);
HRESULT hr;
//step1--------------
// Initialize the COM library.
//
CoInitialize(NULL);

//step2---------------
//enum capture filter
//
deviceEnum();

//step3---------------
//connect filters,build the filter graph
//
//1.for camera 1.................................................
//
InitStillGraph(&m_pGraph1,
             &m_pGraphBuilder21,
       &m_pVCap1,
       &m_pGrabberF1,
       &m_pRenderer1,
       &m_pGrabber1       );

// ask for the connection media type so we know how big
    // it is, so we can write out bitmaps
    //
    AM_MEDIA_TYPE mt;
    hr = m_pGrabber1->GetConnectedMediaType( &mt );
    if ( FAILED( hr) )//something wrong happened
    {
        MessageBox(NULL,TEXT("Someting wrong"), TEXT("Could not read the connected media type"),0);
        //return ;
    }

    VIDEOINFOHEADER * vih = (VIDEOINFOHEADER*) mt.pbFormat;
    mCB1.lWidth = vih->bmiHeader.biWidth;
    mCB1.lHeight = vih->bmiHeader.biHeight;
    FreeMediaType( mt );

// don't buffer the samples as they pass through
    //
    hr = m_pGrabber1->SetBufferSamples( FALSE );
if (SUCCEEDED(hr))
{
  MessageBox(NULL,"setBufferSamples sucess","ok",0);
}
if (FAILED(hr))
{
  MessageBox(NULL,"setBufferSamples FAILE","WRONG",0);
}

    // only grab one at a time, stop stream after
    // grabbing one sample
    //
    hr = m_pGrabber1->SetOneShot( FALSE );
if (SUCCEEDED(hr))
{
  MessageBox(NULL,"SetOneShot sucess","ok",0);
}
if (FAILED(hr))
{
  MessageBox(NULL,"SetOneShot FAILED","WRONG",0);
}

    // set the callback, so we can grab the one sample
    //
    hr = m_pGrabber1->SetCallback(&mCB1, 1);

hr = m_pGrabber1->SetOneShot( FALSE );
if (SUCCEEDED(hr))
{
MessageBox(NULL,"SetCallback sucess","ok",0);
}
if (FAILED(hr))
{
MessageBox(NULL,"SetCallback FAILED","WRONG",0);
}

//get this ,just for the use to show out in the screen
//
hr = m_pGraph1->QueryInterface(IID_IVideoWindow, (void**)&m_pVidWin1);

//get the Media Control object.and run the filter graph
//
hr = m_pGraph1->QueryInterface(IID_IMediaControl, (void **)&m_pMediaControl1);

//2.for camera 2.....................................................
//
InitStillGraph( &m_pGraph2,
     &m_pGraphBuilder22,
     &m_pVCap2,
     &m_pGrabberF2,
     &m_pRenderer2,
     &m_pGrabber2     );

// ask for the connection media type so we know how big
// it is, so we can write out bitmaps
//

    hr = m_pGrabber2->GetConnectedMediaType( &mt );
    if ( FAILED( hr) )
    {
        MessageBox(NULL,TEXT("Someting wrong"), TEXT("Could not read the connected media type"),0);
        return ;
    }

    vih = (VIDEOINFOHEADER*) mt.pbFormat;
    mCB2.lWidth = vih->bmiHeader.biWidth;
    mCB2.lHeight = vih->bmiHeader.biHeight;
    FreeMediaType( mt );

// don't buffer the samples as they pass through
//
hr = m_pGrabber2->SetBufferSamples( FALSE );

    // only grab one at a time, stop stream after
    // grabbing one sample
    //
    hr = m_pGrabber2->SetOneShot( FALSE );

    // set the callback, so we can grab the one sample
    //
    hr = m_pGrabber2->SetCallback(&mCB2, 1);

if(SUCCEEDED(hr))
{
MessageBox(NULL,"SetCallback","ok",0);
}

//get this ,just for the use to show out in the screen
//
hr = m_pGraph2->QueryInterface(IID_IVideoWindow, (void**)&m_pVidWin2);

//get the Media Control object.and run the filter graph
//
hr = m_pGraph2->QueryInterface(IID_IMediaControl, (void **)&m_pMediaControl2);

//now set the flag to be true
//
m_bFilterGraphCam1=true;
m_bFilterGraphCam2=true;
}

/*--------------------------------------------------------------
下面的这个函数是析构函数，进行资源的释放
----------------------------------------------------------------*/
BuildTwoCamFilterGraph::~BuildTwoCamFilterGraph()
{
//step1:stop preview
//
m_pMediaControl1->Stop();
m_pMediaControl2->Stop();

//step2:release resource
//
m_pVidWin1->Release();
m_pGraph1->Release();
m_pGraphBuilder21->Release();
m_pVCap1->Release();
m_pMediaControl1->Release();
m_pGrabberF1->Release();
m_pGrabber1->Release();
m_pRenderer1->Release();

m_pVidWin2->Release();
m_pGraph2->Release();
m_pGraphBuilder22->Release();
m_pVCap2->Release();
m_pMediaControl2->Release();
m_pGrabberF2->Release();
m_pGrabber2->Release();
m_pRenderer2->Release();
//step3:
//
CoUninitialize();

}

/*-----------------------------------------------------------------
这个函数用于重新建立filter graph，因为可能会有需要解除filter graph
之间的连接。这时，应该只有capture filter留下了，所以，重新建立的，
可以调用在constructor里面用到的辅助函数.
^_^这下面用到的就是从constructor里面来的代码，是我封装做得不好
-------------------------------------------------------------------*/
void BuildTwoCamFilterGraph::rebuildFilterGraph(void)
{
HRESULT hr;
    //connect filters,build the filter graph
//
//1.for camera 1.................................................
//
InitStillGraph(&m_pGraph1,
             &m_pGraphBuilder21,
       &m_pVCap1,
       &m_pGrabberF1,
       &m_pRenderer1,
       &m_pGrabber1       );

// ask for the connection media type so we know how big
    // it is, so we can write out bitmaps
    //
    AM_MEDIA_TYPE mt;
    hr = m_pGrabber1->GetConnectedMediaType( &mt );
    if ( FAILED( hr) )
    {
        MessageBox(NULL,TEXT("Someting wrong"), TEXT("Could not read the connected media type"),0);
        return ;
    }

    VIDEOINFOHEADER * vih = (VIDEOINFOHEADER*) mt.pbFormat;
    mCB1.lWidth = vih->bmiHeader.biWidth;
    mCB1.lHeight = vih->bmiHeader.biHeight;
    FreeMediaType( mt );

// don't buffer the samples as they pass through
//
hr = m_pGrabber1->SetBufferSamples( FALSE );

    // only grab one at a time, stop stream after
    // grabbing one sample
    //
    hr = m_pGrabber1->SetOneShot( FALSE );