buildroot+qt+qcamrea+opencv打开摄像头进行拍照录像保存功能

相关参考

如何用FFmpeg API采集摄像头视频和麦克风音频，并实现录制文件的功能_爱写代码的自由人-CSDN博客_ffmpeg 摄像头

红胖子(红模仿)的博文大全：开发技术集合（包含Qt实用技术、树莓派、三维、OpenCV、OpenGL、ffmpeg、OSG、单片机、软硬结合等等）持续更新中..._长沙红胖子-CSDN博客

硬件平台：rk3288核心板，底层驱动是已经调好的，这边主要是用qt做上层应用开发

buildroot版本：2018.02-rc3

qt版本：qt creator 4.11.1   based on qt 5.14.1

opencv版本：opencv3

1、buildroot添加opencv库

1> 进入PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/ 文件夹内，使用 sudo make menuconfig命令进入配置。可以看到一个如下图：

 2>选择 Target packages --> Libraries ---> Graphics ---> opencv3 将光标到opencv3（注：最好是选择最新版本的opencv，旧版本的opencv能会出面问题）的库上，点击键盘上的 'y' 键选择。

3>再按下 “Enter” 键进入opencv3里面进行选择，选择样式如下图（注意和红色框选择的一样即可）：

 4>将配置好的选项进行保存，选择“Save”进行保存，后退出配置。（注：如果全部选中，再去编译好像会出错，目前未找到原因）

5>最后再将配置保存到默认配置中去：sudo make savedefconfig ， 这样配置完后就可以进行编译了。

6>编译完后，会在 PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/host/arm-buildroot-linux-gnueabihf/sysroot/usr/ 这个目录下有一个include目录和一个lib目录，编译出的opencv库头文件与库文件就放在这两个目录里。

 7> 在include目录里可以找到 opencv 与opencv2 两个目录 ，在lib目录里可以找到 libopencv_*开头的所有opencv库文件，说明你已将opencv加入到buildroot系统中。

2、在QT中加入opencv库

打开QT项目中的.pro文件，加入下面代码：

INCLUDEPATH += [前面是你个人系统目录]/PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/host/arm-buildroot-linux-gnueabihf/sysroot/usr/include
               [前面是你个人系统目录]/PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/host/arm-buildroot-linux-gnueabihf/sysroot/usr/include/opencv
                [前面是你个人系统目录]/PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/host/arm-buildroot-linux-gnueabihf/sysroot/usr/include/opencv2

LIBS += [前面是你个人系统目录]/PET_RK3288_P01_Linux/buildroot/output/rockchip_rk3288/host/arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libopencv_* \

所需要添加的头文件如下：

#include <opencv2/opencv.hpp>
#include <opencv/cv.hpp>

using namespace cv;

3、在buildroot中使用opencv出现的一些问题

1>不能使用opencv的以下代码打开摄像头和打开视频文件

cv::VideoCapture capture;
capture.open(0);
或者
cv::VideoCapture capture(0);
不能使用imread()，imwrite()读取和保存图片

2>opencv在保存视频时不支持 H.264编码，目前只测试了MJPG格式的

3>目前测试在保存视频的后缀名也只能使用.avi后缀名进行保存

4、在buildroot中使用QCamrea+opencv进行摄像头处理

1>在.pro文件中需要添加下面代码：

QT       += core gui multimedia multimediawidgets

2>头文件：

#ifndef MAINWIDGET_H
#define MAINWIDGET_H

#include <QWidget>
#include <QTimer>
#include <QCameraInfo>
#include <QCamera>
#include <QCameraImageCapture>
#include <QCameraViewfinder>
#include <QVideoProbe>
#include <QVideoFrame>
#include <QCameraViewfinderSettings>
#include <QCameraInfo>
#include <QFileInfo>
#include <QUrl>
//#include <opencv2/opencv.hpp>
#include <opencv/cv.hpp>

QT_BEGIN_NAMESPACE
namespace Ui { class MainWidget; }
QT_END_NAMESPACE

class MainWidget : public QWidget
{
    Q_OBJECT

    enum RecordStatus{
        STOP_RECORDED,
        RECORDING,
        PAUSE_RECORDING
    };

public:
    MainWidget(QWidget *parent = nullptr);
    ~MainWidget();
    void openCamera(void);
    void savePicture(QString & saveDir);
    void recordStart(QString & saveDir);
    void recording(void);
    void pauseRecord(void);
    void recodEnd(void);

private slots:
    void updateCameraState(QCamera::State state);
    void displayCameraError();
    void onProbeFrameSlot(const QVideoFrame &frame);
    void imageSaved(int, const QString &);
    void displayCaptureError(int, QCameraImageCapture::Error, const QString &);

    void on_pictureBtn_clicked();
    void on_flashBtn_clicked();
    void on_videoBtn_clicked();
    void onTimer1sSlot();

private:
    void initLayout();
    void setCamera(const QCameraInfo &cameraInfo);
    void yuyvToRgb(unsigned char *yuv_buffer,unsigned char *rgb_buffer,int iWidth,int iHeight);
    void nv21ToRgb24(unsigned char *yuyv, unsigned char *rgb, int width, int height);
    QImage mat2Image(const cv::Mat &mat);
    cv::Mat imaget2Mat(QImage image);

private:
    Ui::MainWidget *ui;

    QCamera * m_camera;
    QCameraViewfinder * m_viewFinder;
    QVideoProbe * m_videoProbe;
    cv::VideoWriter outputVideo;
    cv::Mat matData;

    QCameraImageCapture * m_imageCapture;

    enum RecordStatus recordStatus;
    int imageCnt;
    int videoCnt;

    QTimer * time;
};
#endif // MAINWIDGET_H

3>源文件

#include "mainwidget.h"
#include "ui_mainwidget.h"

#include <QDebug>

using namespace cv;

#define CAMERA_PIXEL_WIDE       (1920)
#define CAMERA_PIXEL_HIGH       (1080)
#define DISPLA_PIXEL_WIDE       (1280)
#define DISPLA_PIXEL_HIGH       (720)

//摄像头的帧速率(每秒15帧)
#define CAMERA_FRAME_RATE       (30.0)

MainWidget::MainWidget(QWidget *parent)
    : QWidget(parent)
    , ui(new Ui::MainWidget)
{
    ui->setupUi(this);

    m_viewFinder = new QCameraViewfinder(this);
    m_viewFinder->setWindowFlag(Qt::FramelessWindowHint);
    m_viewFinder->setFixedSize(DISPLA_PIXEL_WIDE, DISPLA_PIXEL_HIGH);

    imageCnt = 0;
    videoCnt = 0;
    recordStatus = RecordStatus::STOP_RECORDED;

    time = new QTimer(this);
    connect(time, &QTimer::timeout, this, &MainWidget::onTimer1sSlot);
    time->start(1000/CAMERA_FRAME_RATE);

    initLayout();
    setCamera(QCameraInfo::defaultCamera());
}

MainWidget::~MainWidget()
{
    delete ui;
}

void MainWidget::initLayout()
{
    QBoxLayout *hlayout = new QHBoxLayout;
    hlayout->setMargin(0);
    hlayout->addWidget(m_viewFinder);

    ui->disWidget->setLayout(hlayout);

#if defined(Q_OS_UNIX)
    this->setWindowState(Qt::WindowFullScreen);
#endif
}

void MainWidget::setCamera(const QCameraInfo &cameraInfo)
{
    //初始化摄像头
    m_camera = new QCamera(cameraInfo);
    connect(m_camera, &QCamera::stateChanged, this, &MainWidget::updateCameraState);
    connect(m_camera, QOverload<QCamera::Error>::of(&QCamera::error), this, &MainWidget::displayCameraError);

    updateCameraState(m_camera->state());

    QCameraViewfinderSettings settings;
    settings.setPixelFormat(QVideoFrame::Format_YUYV); //设置像素格式,Android上只支持NV21格式
    settings.setResolution(QSize(CAMERA_PIXEL_WIDE,CAMERA_PIXEL_HIGH)); //设置摄像头的分辨率
    settings.setMaximumFrameRate(CAMERA_FRAME_RATE);
    settings.setMinimumFrameRate(CAMERA_FRAME_RATE);
    m_camera->setViewfinderSettings(settings);

    m_imageCapture = new QCameraImageCapture(m_camera);
    QImageEncoderSettings m_imageSettings;
    m_imageSettings.setCodec("jpeg");
    m_imageSettings.setQuality(QMultimedia::VeryHighQuality);
    m_imageSettings.setResolution(QSize(CAMERA_PIXEL_WIDE, CAMERA_PIXEL_HIGH));
    m_imageCapture->setEncodingSettings(m_imageSettings);

    connect(m_imageCapture, &QCameraImageCapture::imageSaved, this, &MainWidget::imageSaved);
    connect(m_imageCapture, QOverload<int, QCameraImageCapture::Error, const QString &>::of(&QCameraImageCapture::error),this, &MainWidget::displayCaptureError);

    m_camera->setViewfinder(m_viewFinder);

    m_videoProbe = new QVideoProbe(this);
    m_videoProbe->setSource(m_camera);
    connect(m_videoProbe, &QVideoProbe::videoFrameProbed, this, &MainWidget::onProbeFrameSlot, Qt::QueuedConnection);

    openCamera();
}

void MainWidget::openCamera()
{
    m_camera->unload();
    m_camera->setCaptureMode(QCamera::CaptureStillImage);
    m_camera->start();
}

void MainWidget::updateCameraState(QCamera::State state)
{
    switch (state) {
    case QCamera::ActiveState:
        break;
    case QCamera::UnloadedState:
    case QCamera::LoadedState:
        break;
    }
}

void MainWidget::imageSaved(int id, const QString &fileName)
{
    Q_UNUSED(id);
    Q_UNUSED(fileName);
    qDebug()<<"image saved!";
}

void MainWidget::displayCaptureError(int id, const QCameraImageCapture::Error error, const QString &errorString)
{
    Q_UNUSED(id);
    Q_UNUSED(error);
    qDebug()<<"Image Capture Error:"<<errorString;
}

void MainWidget::displayCameraError()
{
    qDebug()<<"Camera error:"<<m_camera->errorString();
}

void MainWidget::onProbeFrameSlot(const QVideoFrame &frame)
{
    QVideoFrame cloneFrame(frame);
    int w = cloneFrame.width();
    int h = cloneFrame.height();

    if (cloneFrame.map(QAbstractVideoBuffer::ReadOnly) == false){
        qDebug()<<"map error";
        return ;
    }

    matData.release();
    Mat matTemp = Mat(h, w, CV_8UC2, cloneFrame.bits(), static_cast<size_t>(cloneFrame.bytesPerLine()));
    cvtColor(matTemp, matData, CV_YUV2RGB_YUYV);

    cloneFrame.unmap();
}

void MainWidget::savePicture(QString & saveDir)
{
    QString lo = saveDir + "PIC" + QString::number(imageCnt) + ".jpg";
    QFileInfo fi = QFileInfo(lo);

    while(fi.isFile()){
        imageCnt++;
        lo = saveDir + "PIC" + QString::number(imageCnt) + ".jpg";
        fi = QFileInfo(lo);
    }
    m_imageCapture->capture(lo);
    //mat2Image(matData).save(lo, "JPG");
    //imwrite(lo.toStdString(), matData);
}

void MainWidget::recordStart(QString & saveDir)
{
    if (recordStatus != RecordStatus::STOP_RECORDED)
    {
        return ;
    }

    QString lo = saveDir + "VIDEO" + QString::number(videoCnt) + ".avi";
    QFileInfo fi = QFileInfo(lo);

    while(fi.isFile()){
        videoCnt++;
        lo = saveDir + "VIDEO" + QString::number(videoCnt) + ".avi";
        fi = QFileInfo(lo);
    }

    //VideoWriter::fourcc('X', '2', '6', '4')  ('M', 'J', 'P', 'G')
    outputVideo.open(lo.toStdString(), VideoWriter::fourcc('M', 'J', 'P', 'G'), CAMERA_FRAME_RATE, Size(CAMERA_PIXEL_WIDE, CAMERA_PIXEL_HIGH), true);
    if (outputVideo.isOpened() == false)
    {
        qDebug()<<"output video open error";
        return ;
    }
    qDebug()<<"output video open ok";
    recordStatus = RecordStatus::RECORDING;
}

void MainWidget::recording()
{
    //判断存储空间是否足够
    //如果空间不够，可直接调用recodEnd();
    if (recordStatus != RecordStatus::RECORDING){
        return ;
    }
    if (matData.empty()){
        return ;
    }
    outputVideo.write(matData);
}

void MainWidget::pauseRecord()
{
    switch (recordStatus) {
    case RECORDING: recordStatus = RecordStatus::PAUSE_RECORDING;break;
    case PAUSE_RECORDING: recordStatus = RecordStatus::PAUSE_RECORDING;break;
    default:break;
    }
}

void MainWidget::recodEnd()
{
    if (recordStatus != RecordStatus::STOP_RECORDED)
    {
        outputVideo.release();
        recordStatus = RecordStatus::STOP_RECORDED;
    }
}

void MainWidget::yuyvToRgb(unsigned char *yuv_buffer,unsigned char *rgb_buffer,int iWidth,int iHeight)
{
    int x;
    int z=0;
    unsigned char *ptr = rgb_buffer;
    unsigned char *yuyv= yuv_buffer;
    for (x = 0; x < iWidth*iHeight; x++)
    {
        int r, g, b;
        int y, u, v;

        if (!z)
            y = yuyv[0] << 8;
        else
            y = yuyv[2] << 8;
        u = yuyv[1] - 128;
        v = yuyv[3] - 128;

        r = (y + (359 * v)) >> 8;
        g = (y - (88 * u) - (183 * v)) >> 8;
        b = (y + (454 * u)) >> 8;

        *(ptr++) = (r > 255) ? 255 : ((r < 0) ? 0 : r);
        *(ptr++) = (g > 255) ? 255 : ((g < 0) ? 0 : g);
        *(ptr++) = (b > 255) ? 255 : ((b < 0) ? 0 : b);

        if(z++)
        {
            z = 0;
            yuyv += 4;
        }
    }
}
/*
 * 初始化代码默认设置输出的图像格式是YUYV，在windows和Linux系统上是支持的，这个可能与摄像头有关，实际需要测试调整；Android系统上只支持NV21格式,如果是Android系统上运行，要记得修改格式
 */
void MainWidget::nv21ToRgb24(unsigned char *yuyv, unsigned char *rgb, int width, int height)
{
    const int nv_start = width * height ;
    int rgb_index = 0; //index = 0,
    uint8_t y, u, v;
    int r, g, b, nv_index = 0,i, j;

    for(i = 0; i < height; i++){
        for(j = 0; j < width; j ++){
            //nv_index = (rgb_index / 2 - width / 2 * ((i + 1) / 2)) * 2;
            nv_index = i / 2  * width + j - j % 2;

            y = yuyv[rgb_index];
            u = yuyv[nv_start + nv_index ];
            v = yuyv[nv_start + nv_index + 1];

            r = y + (140 * (v-128))/100;  //r
            g = y - (34 * (u-128))/100 - (71 * (v-128))/100; //g
            b = y + (177 * (u-128))/100; //b

            if(r > 255)   r = 255;
            if(g > 255)   g = 255;
            if(b > 255)   b = 255;
            if(r < 0)     r = 0;
            if(g < 0)     g = 0;
            if(b < 0)     b = 0;

            //index = rgb_index % width + (height - i - 1) * width;
            //rgb[index * 3+0] = b;
            //rgb[index * 3+1] = g;
            //rgb[index * 3+2] = r;

            //颠倒图像
            //rgb[height * width * 3 - i * width * 3 - 3 * j - 1] = b;
            //rgb[height * width * 3 - i * width * 3 - 3 * j - 2] = g;
            //rgb[height * width * 3 - i * width * 3 - 3 * j - 3] = r;

            //正面图像
            rgb[i * width * 3 + 3 * j + 0] = b;
            rgb[i * width * 3 + 3 * j + 1] = g;
            rgb[i * width * 3 + 3 * j + 2] = r;

            rgb_index++;
        }
    }
}

QImage MainWidget::mat2Image(const cv::Mat &mat)
{
   if (mat.type() == CV_8UC1)                 // 单通道
   {
       QImage image(mat.cols, mat.rows, QImage::Format_Indexed8);
       image.setColorCount(256);              // 灰度级数256
       for (int i = 0; i < 256; i++)
       {
           image.setColor(i, qRgb(i, i, i));
       }
       uchar *pSrc = mat.data;                // 复制mat数据
       for (int row = 0; row < mat.rows; row++)
       {
           uchar *pDest = image.scanLine(row);
           memcpy(pDest, pSrc, mat.cols);
           pSrc += mat.step;
       }
       return image;
   }
   else if (mat.type() == CV_8UC3)            // 3通道
   {
       const uchar *pSrc = (const uchar*)mat.data;        // 复制像素
       QImage image(pSrc, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);   // R, G, B 对应 0,1,2
       return image.rgbSwapped();             // rgbSwapped是为了显示效果色彩好一些。
   }
   else if (mat.type() == CV_8UC4)
   {
       const uchar *pSrc = (const uchar*)mat.data;        // 复制像素

       // Create QImage with same dimensions as input Mat
       QImage image(pSrc, mat.cols, mat.rows, mat.step, QImage::Format_ARGB32);       // B,G,R,A 对应 0,1,2,3
       return image.copy();
   }
   else
   {
       return QImage();
   }
}

cv::Mat MainWidget::imaget2Mat(QImage image)
{
    cv::Mat mat;
    switch (image.format())
    {
    case QImage::Format_ARGB32:
    case QImage::Format_RGB32:
    case QImage::Format_ARGB32_Premultiplied:
        mat = cv::Mat(image.height(), image.width(), CV_8UC4, (void*)image.constBits(), image.bytesPerLine());
        break;
    case QImage::Format_RGB888:
        mat = cv::Mat(image.height(), image.width(), CV_8UC3, (void*)image.constBits(), image.bytesPerLine());
        cv::cvtColor(mat, mat, cv::COLOR_BGR2RGB);
        break;
    case QImage::Format_Indexed8:
        mat = cv::Mat(image.height(), image.width(), CV_8UC1, (void*)image.constBits(), image.bytesPerLine());
        break;
    default: break;
    }
    return mat;
}

void MainWidget::on_pictureBtn_clicked()
{
    QString path("/usr/app/");

    savePicture(path);
}

void MainWidget::on_flashBtn_clicked()
{
    openCamera();
}

void MainWidget::on_videoBtn_clicked()
{
    static bool videoFlag = false;

    if (videoFlag == false)
    {
        QString path("/usr/app/");
        recordStart(path);

        if (outputVideo.isOpened())
        {
            videoFlag = true;
            ui->videoBtn->setText(tr("videoing"));
        }
    }else{
        recodEnd();
        ui->videoBtn->setText(tr("录像"));
        videoFlag = false;
    }
}

void MainWidget::onTimer1sSlot()
{
    recording();
}