1、OpenCvSharp视频读取、显示小记

写作原因：前段时间一直在使用这个框架，但是没有细看，现在想了解下这个框架的运行流程，固随手记录下，如有不对的地方，请多多指教；

说明：这一篇博客只记录下视频的读取涉及到函数和类，不做延申。

一、整体流程

打开视频→读取→显示→销毁

VideoCapture类

 此类继承了DisposableObject类，可以调用Dispose()销毁对象释放资源。

二、打开视频文件

• 方式1：直接通过带参构造函数

#region 211224 komla 带参构造函数打开视频文件
var capture2 = new VideoCapture(@"F:\\Media\\A.avi");//@"F:\\Media\\A.avi"
bool open2 = capture2.IsOpened();
#endregion
//output
//true

 源码中带参构造函数有多种， 对应不同的需求

 笔者测试使用的函数在源码中如下，从注释中可以看到，通过此构造函数可以打开一个视频文件。

/// <summary>
/// Opens a video file or a capturing device or an IP video stream for video capturing with API Preference
/// </summary>
/// <param name="fileName">it can be:
/// - name of video file (eg. `video.avi`)
/// - or image sequence (eg. `img_%02d.jpg`, which will read samples like `img_00.jpg, img_01.jpg, img_02.jpg, ...`)
/// - or URL of video stream (eg. `protocol://host:port/script_name?script_params|auth`).
/// Note that each video stream or IP camera feed has its own URL scheme. Please refer to the
/// documentation of source stream to know the right URL.</param>
/// <param name="apiPreference">apiPreference preferred Capture API backends to use. Can be used to enforce a specific reader
/// implementation if multiple are available: e.g. cv::CAP_FFMPEG or cv::CAP_IMAGES or cv::CAP_DSHOW.</param>
/// <returns></returns>
public VideoCapture(string fileName, VideoCaptureAPIs apiPreference = VideoCaptureAPIs.ANY)
{
    if (string.IsNullOrEmpty(fileName))
        throw new ArgumentNullException(nameof(fileName));

    NativeMethods.HandleException(
        NativeMethods.videoio_VideoCapture_new2(fileName, (int)apiPreference, out ptr));

    if (ptr == IntPtr.Zero)
        throw new OpenCvSharpException("Failed to create VideoCapture");
            
    captureType = CaptureType.File;
}

• 方式二：通过不带参构造函数VideoCapture()+open()函数读取视频文件

#region 211224 komla 不带参构造函数VideoCapture()+open()函数 打开视频文件
var capture1 = new VideoCapture();//@"F:\\Media\\A.avi"
bool open1 = capture1.IsOpened();
//false
capture1.Open(@"F:\\Media\\A.avi");
open1= capture1.IsOpened();
//true
#endregion

源码中注释说明了使用此函数只是初始化一个空对象，需要配合open函数才能读取视频文件。

/// <summary>
/// Initializes empty capture.
/// To use this, you should call Open. 
/// </summary>
/// <returns></returns>
public VideoCapture()
{
    NativeMethods.HandleException(
        NativeMethods.videoio_VideoCapture_new1(out ptr));
            
        if (ptr == IntPtr.Zero)
            throw new OpenCvSharpException("Failed to create VideoCapture");
            
        captureType = CaptureType.NotSpecified;
}

/// <summary>
/// Opens a video file or a capturing device or an IP video stream for video capturing.
/// </summary>
/// <param name="fileName">it can be:
/// - name of video file (eg. `video.avi`)
/// - or image sequence (eg. `img_%02d.jpg`, which will read samples like `img_00.jpg, img_01.jpg, img_02.jpg, ...`)
/// - or URL of video stream (eg. `protocol://host:port/script_name?script_params|auth`).
/// Note that each video stream or IP camera feed has its own URL scheme. Please refer to the
/// documentation of source stream to know the right URL.</param>
/// <param name="apiPreference">apiPreference preferred Capture API backends to use. Can be used to enforce a specific reader
/// implementation if multiple are available: e.g. cv::CAP_FFMPEG or cv::CAP_IMAGES or cv::CAP_DSHOW.</param>
/// <returns>`true` if the file has been successfully opened</returns>
public bool Open(string fileName, VideoCaptureAPIs apiPreference = VideoCaptureAPIs.ANY)
{
    ThrowIfDisposed();

    NativeMethods.HandleException(
        NativeMethods.videoio_VideoCapture_open1(ptr, fileName, (int)apiPreference, out             var ret));

    GC.KeepAlive(this);
    if (ret == 0) 
        return false;

    captureType = CaptureType.File;
    return true;
}

三、读取帧图像

读取方式也有两种方式，一种是直接通过Read()读取，另一种需要通过两个函数搭配进行读取，

• 方式1：Read()

        /// <summary>
        /// Grabs, decodes and returns the next video frame.
        ///
        /// The method/function combines VideoCapture::grab() and VideoCapture::retrieve() in one call. This is the
        /// most convenient method for reading video files or capturing data from decode and returns the just
        /// grabbed frame. If no frames has been grabbed (camera has been disconnected, or there are no more
        /// frames in video file), the method returns false and the function returns empty image (with %cv::Mat, test it with Mat::empty()).
        /// </summary>
        /// <returns>`false` if no frames has been grabbed</returns>
        public bool Read(Mat image)
        {
            ThrowIfDisposed();
            if(image == null)
                throw new ArgumentNullException(nameof(image));
            image.ThrowIfDisposed();

            NativeMethods.HandleException(
                NativeMethods.videoio_VideoCapture_read_Mat(ptr, image.CvPtr, out var ret));

            GC.KeepAlive(this);
            GC.KeepAlive(image);
            return ret != 0;
        }

 可以看到mat中已经有了数据，当然Read()函数有两种重载类型，笔者使用的是传参为Mat的那一个

• 方式2：Grab()+Retrieve()

 如果只使用Retrieve()是抓不到帧图像的，可以从源码中的定义看，Retrieve()只是解码和返回抓取到的帧，因此在使用Retrieve()之前需要先使用Grab()抓取帧图像。

//
        // 摘要:
        //     Decodes and returns the grabbed video frame. The method decodes and returns the
        //     just grabbed frame. If no frames has been grabbed (camera has been disconnected,
        //     or there are no more frames in video file), the method returns false and the
        //     function returns an empty image (with %cv::Mat, test it with Mat::empty()).
        //
        // 参数:
        //   image:
        //     the video frame is returned here. If no frames has been grabbed the image will
        //     be empty.
        //
        //   flag:
        //     it could be a frame index or a driver specific flag
        public bool Retrieve(Mat image, int flag = 0);

        /// <summary>
        /// Grabs the next frame from video file or capturing device.
        ///
        /// The method/function grabs the next frame from video file or camera and returns true (non-zero) in the case of success.
        ///
        /// The primary use of the function is in multi-camera environments, especially when the cameras do not
        /// have hardware synchronization. That is, you call VideoCapture::grab() for each camera and after that
        /// call the slower method VideoCapture::retrieve() to decode and get frame from each camera. This way
        /// the overhead on demosaicing or motion jpeg decompression etc. is eliminated and the retrieved frames
        /// from different cameras will be closer in time.
        ///
        /// Also, when a connected camera is multi-head (for example, a stereo camera or a Kinect device), the
        /// correct way of retrieving data from it is to call VideoCapture::grab() first and then call
        /// VideoCapture::retrieve() one or more times with different values of the channel parameter.
        /// </summary>
        /// <returns>`true` (non-zero) in the case of success.</returns>
        public bool Grab()
        {
            ThrowIfDisposed();
            NativeMethods.HandleException(
                NativeMethods.videoio_VideoCapture_grab(ptr, out var ret));
            GC.KeepAlive(this);
            return ret != 0;
        }

 当然，Retrieve()根据不同的需求定义了多种重载函数。

 最后一个重载函数在获取到图象时是会返回一个Mat，其他的都是在内部方法中对传参mat进行赋值。

四 显示

• 说明：用到的类是Window，在OpenCvSharp命名空间下。使用的函数是ShowImage()，两个重载。

•  源码

        /// <summary>
        /// Shows the image in this window
        /// </summary>
        /// <param name="img">Image to be shown. </param>
        public void ShowImage(Mat? img)
        {
            if (img != null)
            {
                image = img;
                NativeMethods.HandleException(
                    NativeMethods.highgui_imshow(name, img.CvPtr));
                GC.KeepAlive(img);
            }
        }

• 使用

            #region 211225 komla 显示
            var capture4 = new VideoCapture(@"F:\\Media\\A.avi");//@"F:\\Media\\A.avi" 
            Mat image4 = new Mat(); 
            //初始化显示窗体并命名
            var win4 = new Window("yflbi");
            //循环读取
            while(true)
            {
                capture4.Grab();
                capture4.Retrieve(image4);
                win4.ShowImage(image4);
                //等待时间，单位ms
                Cv2.WaitKey(1000);
            }

五 销毁 

主要是销毁VideoCapture对象

继承关系VideoCapture：DisposableCvObject：DisposableObject：IDisposable，因此VideoCapture可以调用Dispose()销毁对象释放资源。

            #region 211225 komla Dispose() 销毁VideoCapture对象
            var capture5 = new VideoCapture(@"F:\\Media\\A.avi");//@"F:\\Media\\A.avi" 
            bool dis = capture5.IsDisposed;
            //false
            capture5.Dispose();
            bool dis1 = capture5.IsDisposed;
            //true
            #endregion

 笔者还注意到，VideoCapture类中还有一个Release()函数，源码如下：

        /// <summary>
        /// Closes video file or capturing device.
        /// </summary>
        /// <returns></returns>
        public void Release()
        {
            ThrowIfDisposed();
            NativeMethods.HandleException(
                NativeMethods.videoio_VideoCapture_release(ptr));
        }

 注释中解释说是关闭视频文件或抓取视频设备，笔者测试了下，这个会把VideoCapture对象关闭掉，但是并不会销毁VideoCapture对象。

            #region 211225 komla Release() 关闭VideoCapture对象
            var capture6 = new VideoCapture(@"F:\\Media\\A.avi");//@"F:\\Media\\A.avi" 
            bool open6 = capture6.IsOpened();
            //true 
            bool dis6 = capture6.IsDisposed;
            //false 
            capture6.Release();//close VideoCapture对象
            bool open61 = capture6.IsOpened();
            //false 
            bool dis62 = capture6.IsDisposed;//并没有销毁，只是close
            //false 
            #endregion