Tech Spike: How popular video processing libraries/SDK manipulate cameras

最新推荐文章于 2021-05-14 02:25:12 发布

执假以为真

最新推荐文章于 2021-05-14 02:25:12 发布

阅读量2.3k

点赞数

分类专栏： # OpenCV 文章标签： video processing opencv directshow media foundation camera

本文链接：https://blog.csdn.net/nirendao/article/details/50397282

版权

OpenCV 专栏收录该内容

2 篇文章

订阅专栏

对于图像处理，本人是个新手，从来没接触过。近来项目中既然涉及到这一方面的内容，就需做一些研究。项目中有一个要求，就是要搞清楚目前流行的视频或图像处理的库或SDK是如何识别摄像头的，比如说camera id是int型的数字还是string，具体又是如何得到这些id，以及后续会如何应用这些id来获得进一步的处理。本文尝试分析了几个库和SDK。但因为时间有限，这方面知识又不熟悉，有些地方并不算特别透彻。仅此一记，日后再来改进。

Abstraction

In our project, one user story wants to figure out how popular video processing librariesor software development kits manipulate camera in multiple cameras environment.It means, how they identify the cameras, and how to get these identifiers.Actually, this problem may have something to do with the camera driver and theoperating system. This articlemainly analyzed 3 libraries/SDKs, i.e.DirectShow, MS Media Foundation and OpenCV, and also talked about 6 otherpopular libraries. Due to the limited time, there could be something which canstill be improved. Let's improve it further when needed.

Section 1. DirectShow

1.1 Brief Introduction[1][2]

TheMicrosoft DirectShow application programming interface (API) is amedia-streaming architecture for Microsoft Windows. Using DirectShow, yourapplications can perform high-quality video and audio playback or capture. Forinstance, you can useDirectShow to[3]: select a capture device, preview video, capture video to a file,control a capture graph, etc..

1.2 How to get a video capture device[3][4]

Toselect an audio or video capture device, use the SystemDevice Enumerator, described in the topic Usingthe System Device Enumerator. The System Device Enumerator returns acollection of device monikers, selected by device category. Amoniker is a COM objectthat contains information about another object. Monikers enable the applicationto get information about an object without actually creating the object. Later,the application can use the moniker to create the object. For more informationabout monikers, see the documentation for IMoniker.

Thefollowing properties are supported by device monikers.

Property	Description	VARIANT Type
"FriendlyName"	The name of the device.	VT_BSTR
"Description"	A description of the device.	VT_BSTR
"DevicePath"	A unique string that identifies the device. (Video capture devices only.)	VT_BSTR

The"FriendlyName" and "Description" properties are suitablefor displaying in a UI.

The "FriendlyName" property is available for every device. It contains a human-readable name for the device.
The "Description" property is available only for DV and D-VHS/MPEG camcorder devices. For more information, seeMSDV Driver and MSTape Driver. If available, it contains a description of the device which is more specific than the "FriendlyName" property. Typically it includes the vendor name.
The "DevicePath" property is not a human-readable string, but is guaranteed to be unique for each video capture device on the system. You can use this property to distinguish between two or more instances of the same model of device.

1.3 Sample Code [5]

// Create the System Device Enumerator.
HRESULT hr;
ICreateDevEnum *pSysDevEnum = NULL;
hr = CoCreateInstance(CLSID_SystemDeviceEnum, NULL, CLSCTX_INPROC_SERVER,
    IID_ICreateDevEnum, (void **)&pSysDevEnum);
if (FAILED(hr))
{
    return hr;
}

// Obtain a class enumerator for the video compressor category.
IEnumMoniker *pEnumCat = NULL;
hr = pSysDevEnum->CreateClassEnumerator(CLSID_VideoCompressorCategory, &pEnumCat, 0);

if (hr == S_OK) 
{
    // Enumerate the monikers.
    IMoniker *pMoniker = NULL;
    ULONG cFetched;
    while(pEnumCat->Next(1, &pMoniker, &cFetched) == S_OK)
    {
        IPropertyBag *pPropBag;
        hr = pMoniker->BindToStorage(0, 0, IID_IPropertyBag, 
            (void **)&pPropBag);
        if (SUCCEEDED(hr))
        {
            // To retrieve the filter's friendly name, do the following:
            VARIANT varName;
            VariantInit(&varName);
            hr = pPropBag->Read(L"FriendlyName", &varName, 0);
            if (SUCCEEDED(hr))
            {
                // Display the name in your UI somehow.
            }
            VariantClear(&varName);

            // To create an instance of the filter, do the following:
            IBaseFilter *pFilter;
            hr = pMoniker->BindToObject(NULL, NULL, IID_IBaseFilter,
                (void**)&pFilter);
            // Now add the filter to the graph. 
            //Remember to release pFilter later.
            pPropBag->Release();
        }
        pMoniker->Release();
    }
    pEnumCat->Release();
}
pSysDevEnum->Release();

1.4 Conclusion

DirectShow can get “FriendlyName” and “DevicePath” for all the cameras.
“FriendlyName” is human-readable, and mainly for UI; “DevicePath” is for identifying cameras.
For current code base, we already have the codes to fetch “FriendlyName” and “DevicePath” for cameras by DirectShow.
The code locates at FindCameraDevicesDS() function of CameraDeviceIdentifier class in Eric.CVP solution.

3. After getting "DevicePath", we can use it withDirectShow to: capture video to a file, control a capture graph, etc..

Section 2. Microsoft Media Foundation

2.1 Brief Introduction [6][7]

MediaFoundation is the next generation multimedia platform for Windows that enablesdevelopers, consumers, and content providers to embrace the new wave of premiumcontent with enhanced robustness, unparalleled quality, and seamlessinteroperability.It requires Windows Vista or later. It uses the componentobject model (COM) and requires C/C++. Microsoft does not provide a managed APIfor Media Foundation.

TheMedia Foundation APIs are part of the Windows SDK. Todevelop a Media Foundation application, install the latest version of theWindows SDK.

2.1.1 Difference between Media Foundation and DirectShow [8][9]

MicrosoftMedia Foundation was introduced in Windows Vista as thereplacement for DirectShow. Of course, DirectShow is stillsupported in Windows 7, but developers are encouraged to use MediaFoundation in their new digital media applications.

MediaFoundation (MF) is a COM-based multimedia framework pipelineand infrastructure platform for digital media in Windows Vista and later. It is the intended replacement for Microsoft DirectShow, Windows Media SDK, DirectX Media Objects(DMOs) and all other so-called "legacy" multimedia APIs such as Audio CompressionManager (ACM) and Video for Windows (VfW).

2.2 How to get a video capture device [10]

Toenumerate the capture devices on the system, perform the following steps:

Call the MFCreateAttributes function to create an attribute store.
Set the MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE attribute to one of the following values:

Value	Description
MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_AUDCAP_GUID	Enumerate audio capture devices.
MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID	Enumerate video capture devices.

Call the MFEnumDeviceSources function. This function allocates an array of IMFActivate pointers. Each pointer represents an activation object for one device on the system.
Call the IMFActivate::ActivateObject method to create an instance of the media source from one of the activation objects.

2.3 Sample Code

Referto [10].

2.4 Conclusion

The same as Direct Show, MS Media Foundation offers “Friendly Name” and “Symbolic Link” a.k.a. “Device Path” of all the cameras.
For current code base of Sprout, we already have the codes to fetch “Friendly Name” and “Symbolic Link” for cameras by Media Foundation.
The code locates at FindCameraDevicesMF() function of CameraDeviceIdentifier class in Eric.CVP solution.
As the replacement of DirectShow, Media Foundation can also provide the same functions like capture video to a file, control a capture graph, etc.,after selecting a video capture device.

Section 3. OpenCV

3.1 Brief Introduction [11]

OpenCV (OpenSource Computer Vision) is a library of programmingfunctions mainly aimed at real-time computer vision, originallydeveloped by Intel research center in NizhnyNovgorod (Russia), later supported by Willow Garage and nowmaintained by Itseez. The library is cross-platform and free foruse under the open-source BSD license.

3.2 How to get a video capture device [12]

OpenCVmainly uses a class called VideoCapturefor video capturing from video files, image sequences or cameras. Thisclass provides C++ API for capturing video from cameras or for reading videofiles and image sequences.

VideoCapturehas several constructors as below.

VideoCapture();
VideoCapture(const string &filename);
VideoCapture(const string &filename, int apiPreference);
VideoCapture(int index);

The“index” in the last constructor represents the camera id-s. By default, it’szero if there is only one camera. For different SDK-s/drivers which are used to get the cameras, the range of the camera id-s are different. (See the table below or the sample code section.)

Enum name	Value	Comment
CV_CAP_MIL	100	MIL proprietary drivers
CV_CAP_VFW	200	platform native
CV_CAP_FIREWARE	300	IEEE 1394 drivers
CV_CAP_STEREO	400	TYZX proprietary drivers
CV_CAP_QT	500	QuickTime
CV_CAP_UNICAP	600	Unicap drivers
CV_CAP_DSHOW	700	DirectShow (via videoInput)
CV_CAP_PVAPI	800	PvAPI, Prosilica GigE SDK
CV_CAP_OPENNI	900	OpenNI (for Kinect)
CV_CAP_OPENNI_ASUS	910	OpenNI (for Asus Xtion)
CV_CAP_ANDROID	1000	Android
CV_CAP_ANDROID_FRONT	1098	Android front camera
CV_CAP_ANDROID_BACK	1099	Android back camera
CV_CAP_XIAPI	1100	XIMEA Camera API
CV_CAP_AVFOUNDATION	1200	AVFoundation framework for iOS
CV_CAP_GIGANETIX	1300	Smartek Giganetix GigEVisionSDK
CV_CAP_MSMF	1400	Microsoft Media Foundation (via videoInput)
CV_CAP_INTELPERC	1500	Intel Perceptual Computing SDK

3.3 Sample Code

#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>
using namespace cv;
using namespace std;

bool EnumerateCameras(vector<int> &camIdx)
{
    camIdx.clear();
    struct CapDriver{
        int enumValue; string enumName; string comment;
    };
    // list of all CAP drivers (see highgui_c.h)
    vector<CapDriver> drivers;
    drivers.push_back({ CV_CAP_MIL, "CV_CAP_MIL", "MIL proprietary drivers" });
    drivers.push_back({ CV_CAP_VFW, "CV_CAP_VFW", "platform native" });
    drivers.push_back({ CV_CAP_FIREWARE, "CV_CAP_FIREWARE", "IEEE 1394 drivers" });
    drivers.push_back({ CV_CAP_STEREO, "CV_CAP_STEREO", "TYZX proprietary drivers" });
    drivers.push_back({ CV_CAP_QT, "CV_CAP_QT", "QuickTime" });
    drivers.push_back({ CV_CAP_UNICAP, "CV_CAP_UNICAP", "Unicap drivers" });
    drivers.push_back({ CV_CAP_DSHOW, "CV_CAP_DSHOW", "DirectShow (via videoInput)" });
    drivers.push_back({ CV_CAP_MSMF, "CV_CAP_MSMF", "Microsoft Media Foundation (via videoInput)" });
    drivers.push_back({ CV_CAP_PVAPI, "CV_CAP_PVAPI", "PvAPI, Prosilica GigE SDK" });
    drivers.push_back({ CV_CAP_OPENNI, "CV_CAP_OPENNI", "OpenNI (for Kinect)" });
    drivers.push_back({ CV_CAP_OPENNI_ASUS, "CV_CAP_OPENNI_ASUS", "OpenNI (for Asus Xtion)" });
    drivers.push_back({ CV_CAP_ANDROID, "CV_CAP_ANDROID", "Android" });
    drivers.push_back({ CV_CAP_ANDROID_BACK, "CV_CAP_ANDROID_BACK", "Android back camera" }),
    drivers.push_back({ CV_CAP_ANDROID_FRONT, "CV_CAP_ANDROID_FRONT", "Android front camera" }),
    drivers.push_back({ CV_CAP_XIAPI, "CV_CAP_XIAPI", "XIMEA Camera API" });
    drivers.push_back({ CV_CAP_AVFOUNDATION, "CV_CAP_AVFOUNDATION", "AVFoundation framework for iOS" });
    drivers.push_back({ CV_CAP_GIGANETIX, "CV_CAP_GIGANETIX", "Smartek Giganetix GigEVisionSDK" });
    drivers.push_back({ CV_CAP_INTELPERC, "CV_CAP_INTELPERC", "Intel Perceptual Computing SDK" });

    std::string winName, driverName, driverComment;
    int driverEnum;
    Mat frame;
    bool found;
    std::cout << "Searching for cameras IDs..." << endl << endl;
    for (int drv = 0; drv < drivers.size(); drv++)
    {
        driverName = drivers[drv].enumName;
        driverEnum = drivers[drv].enumValue;
        driverComment = drivers[drv].comment;
        std::cout << "Testing driver " << driverName << "...";
        found = false;

        int maxID = 100; //100 IDs between drivers
        if (driverEnum == CV_CAP_VFW)
            maxID = 10; //VWF opens same camera after 10 ?!?
        else if (driverEnum == CV_CAP_ANDROID)
            maxID = 98; //98 and 99 are front and back cam
        else if ((driverEnum == CV_CAP_ANDROID_FRONT) || (driverEnum == CV_CAP_ANDROID_BACK))
            maxID = 1;

        for (int idx = 0; idx <maxID; idx++)
        {
            VideoCapture cap(driverEnum + idx);  // open the camera
            if (cap.isOpened())                  // check if we succeeded
            {
                found = true;
                camIdx.push_back(driverEnum + idx);  // vector of all available cameras
                cap >> frame;
                if (frame.empty())
                    std::cout << endl << driverName << "+" << idx << "\t opens: OK \t grabs: FAIL";
                else
                    std::cout << endl << driverName << "+" << idx << "\t opens: OK \t grabs: OK";
                // display the frame
                // imshow(driverName + "+" + to_string(idx), frame); waitKey(1);
            }
            cap.release();
        }
        if (!found) cout << "Nothing !" << endl;
        cout << endl;
    }
    cout << camIdx.size() << " camera IDs has been found ";
    cout << "Press a key..." << endl; cin.get();

    return (camIdx.size()>0); // returns success
}

int main()
{
    vector<int> cam_vec;
    EnumerateCameras(cam_vec);

    return 0;
}

The output of above code is as below. (My machine has several cameras)

***** VIDEOINPUT LIBRARY - 0.1995 - TFW07 *****

Searching for cameras IDs...

Testing driver CV_CAP_MIL...Nothing !

Testing driver CV_CAP_VFW...Nothing !

Testing driver CV_CAP_FIREWARE...Nothing !

Testing driver CV_CAP_STEREO...Nothing !

Testing driver CV_CAP_QT...Nothing !

Testing driver CV_CAP_UNICAP...Nothing !

Testing driver CV_CAP_DSHOW...SETUP: Setting up device 0
SETUP: USB Camera-OV580
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 1104x828
SETUP: trying specified format RGB24 @ 640x480
SETUP: trying format RGB24 @ 640x480
SETUP: trying format RGB32 @ 640x480
SETUP: trying format RGB555 @ 640x480
SETUP: trying format RGB565 @ 640x480
SETUP: trying format YUY2 @ 640x480
SETUP: trying format YVYU @ 640x480
SETUP: trying format YUYV @ 640x480
SETUP: trying format IYUV @ 640x480
SETUP: trying format UYVY @ 640x480
SETUP: trying format YV12 @ 640x480
SETUP: trying format YVU9 @ 640x480
SETUP: trying format Y411 @ 640x480
SETUP: trying format Y41P @ 640x480
SETUP: trying format Y211 @ 640x480
SETUP: trying format AYUV @ 640x480
SETUP: trying format MJPG @ 640x480
SETUP: trying format Y800 @ 640x480
SETUP: trying format Y800 @ 640x480
SETUP: trying format Y800 @ 640x480
SETUP: trying format I420 @ 640x480
SETUP: couldn't find requested size - searching for closest matching size
SETUP: closest supported size is YUY2 @ 1104 828
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+0   opens: OK       grabs: OK
SETUP: Disconnecting device 0
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter AVI Decompressor...
SETUP: filter removed AVI Decompressor
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter USB Camera-OV580...
SETUP: filter removed USB Camera-OV580
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 0 disconnected and freed

SETUP: Setting up device 1
SETUP: Forward Facing camera
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 320x240
SETUP: trying specified format RGB24 @ 640x480
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+1   opens: OK       grabs: OK
SETUP: Disconnecting device 1
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter Forward Facing camera...
SETUP: filter removed Forward Facing camera
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 1 disconnected and freed

SETUP: Setting up device 2
SETUP: Downward Facing camera
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 320x240
SETUP: trying specified format RGB24 @ 640x480
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+2   opens: OK       grabs: OK
SETUP: Disconnecting device 2
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter Downward Facing camera...
SETUP: filter removed Downward Facing camera
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 2 disconnected and freed

SETUP: Setting up device 3
SETUP: Intel(R) RealSense(TM) 3D Camera (Front F200) RGB
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 320x180
SETUP: trying specified format RGB24 @ 640x480
SETUP: trying format RGB24 @ 640x480
SETUP: trying format RGB32 @ 640x480
SETUP: trying format RGB555 @ 640x480
SETUP: trying format RGB565 @ 640x480
SETUP: trying format YUY2 @ 640x480
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+3   opens: OK       grabs: OK
SETUP: Disconnecting device 3
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter AVI Decompressor...
SETUP: filter removed AVI Decompressor
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter Intel(R) RealSense(TM) 3D Camera (Front F200) RGB...
SETUP: filter removed Intel(R) RealSense(TM) 3D Camera (Front F200) RGB
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 3 disconnected and freed

SETUP: Setting up device 4
SETUP: HP High Definition 1MP Webcam
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 640x480
SETUP: trying specified format RGB24 @ 640x480
SETUP: trying format RGB24 @ 640x480
SETUP: trying format RGB32 @ 640x480
SETUP: trying format RGB555 @ 640x480
SETUP: trying format RGB565 @ 640x480
SETUP: trying format YUY2 @ 640x480
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+4   opens: OK       grabs: OK
SETUP: Disconnecting device 4
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter AVI Decompressor...
SETUP: filter removed AVI Decompressor
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter HP High Definition 1MP Webcam...
SETUP: filter removed HP High Definition 1MP Webcam
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 4 disconnected and freed

SETUP: Setting up device 5
SETUP: Intel(R) RealSense(TM) 3D Camera (Front F200) Depth
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 640x480
SETUP: trying specified format RGB24 @ 640x480
SETUP: trying format RGB24 @ 640x480
SETUP: trying format RGB32 @ 640x480
SETUP: trying format RGB555 @ 640x480
SETUP: trying format RGB565 @ 640x480
SETUP: trying format YUY2 @ 640x480
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.


CV_CAP_DSHOW+5   opens: OK       grabs: OK
SETUP: Disconnecting device 5
SETUP: freeing Grabber Callback
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter AVI Decompressor...
SETUP: filter removed AVI Decompressor
SETUP: removing filter Smart Tee...
SETUP: filter removed Smart Tee
SETUP: removing filter Intel(R) RealSense(TM) 3D Camera (Front F200) Depth...
SETUP: filter removed Intel(R) RealSense(TM) 3D Camera (Front F200) Depth
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 5 disconnected and freed


Testing driver CV_CAP_MSMF...Nothing !

Testing driver CV_CAP_PVAPI...Nothing !

Testing driver CV_CAP_OPENNI...Nothing !

Testing driver CV_CAP_OPENNI_ASUS...Nothing !

Testing driver CV_CAP_ANDROID...Nothing !

Testing driver CV_CAP_ANDROID_BACK...Nothing !

Testing driver CV_CAP_ANDROID_FRONT...Nothing !

Testing driver CV_CAP_XIAPI...Nothing !

Testing driver CV_CAP_AVFOUNDATION...Nothing !

Testing driver CV_CAP_GIGANETIX...Nothing !

Testing driver CV_CAP_INTELPERC...Nothing !

6 camera IDs has been found Press a key...

3.4 Conslution

OpenCV is mainly responsible for video processing but not on camera controlling, e.g. not on getting/setting properties/names of cameras.
OpenCV uses integer numbers as camera id-s for the multiple camera case.
For different SDK-s/drivers which are used to get the cameras, the range of the camera id-s are different. (See the table above or the sample code section.)

Section 4. FFmpeg

4.1 Brief Introduction [14] [15]

FFmpegis a complete, cross-platform solution to record, convert and stream audio andvideo. FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty muchanything that humans and machines have created. It supports the most obscureancient formats up to the cutting edge.

4.2 How to get a video capture device:

Way 1. FFmpeg can take input from "directshow" devices on your Windows computer[16][17].
Way 2. VFW[18].

Sinceway 1 is based on DirectShow which has been described in section 1, and way 2is based on VFW, which has been mentioned that it should be replaced by MSMedia Foundation in section 2, there will be no more description about the 2ways.

Section 5. GStreamer

5.1 Brief Introduction [19][20]

GStreameris a library for constructing graphs of media-handling components. Theapplications it supports range from simple Ogg/Vorbis playback, audio/videostreaming to complex audio (mixing) and video (non-linear editing) processing.

Applicationscan take advantage of advances in codec and filter technology transparently.Developers can add new codecs and filters by writing a simple plugin with aclean, generic interface.

GStreameris released under the LGPL. The 1.x series is API and ABI stable and supersedesthe previous stable 0.10 series. Both can be installed in parallel.

GStreamerworks on all major operating systems such as Linux, Android, Windows, Max OS X,iOS, as well as most BSDs, commercial Unixes, Solaris, and Symbian. It has beenported to a wide range of operating systems, processors and compilers. It runson all major hardware architectures including x86, ARM, MIPS, SPARC andPowerPC, on 32-bit as well as 64-bit, and little endian or big endian.

5.2 Conclusion

It supports v4l cameras, e.g. /dev/video0, /dev/video1, etc.
Have not figured out how GStreamer get video capture device on Windows due to time limitation. If needed, we can do more investigation in the future.