有时候需要在视频上画图,所以需要能获取到每一帧视频数据。
以前从视频文件或视频流中得到帧,一般都是使用qt + ffmpeg或qt + vlc。
qt对显示处理视频大体有以下方法:
1. QMediaPlayer + QVideoWidget
这种方法只适合简单的显示视频功能,不适合对视频进行处理(比如画图)
2. QMediaPlayer + QGraphicsVideoItem + QGraphicsScene + QGraphicsView
这种方法功能强大,除了显示视频功能,还可以做复杂的图形处理(具体可以查看QGraphicsScene的使用)
3. QMediaPlayer + QAbstractVideoSurface
这种方法比较简单,是我下面要介给的。可以获取到每一帧视频数据,基本可以实现与qt + ffmpeg或qt + vlc相同的效果。
自定义VideoSurface:
class VideoSurface : public QAbstractVideoSurface
-
{
-
Q_OBJECT
-
public:
-
VideoSurface(QObject *parent = Q_NULLPTR);
-
~VideoSurface();
-
QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const;
-
bool present(const QVideoFrame &frame);
-
signals:
-
void frameAvailable(QVideoFrame &frame);
-
};
实现如下:
-
VideoSurface::VideoSurface(QObject *parent)
-
: QAbstractVideoSurface(parent)
-
{
-
}
-
VideoSurface::~VideoSurface()
-
{
-
}
-
QList<QVideoFrame::PixelFormat> VideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
-
{
-
QList<QVideoFrame::PixelFormat> listPixelFormats;
-
listPixelFormats << QVideoFrame::Format_ARGB32
-
<< QVideoFrame::Format_ARGB32_Premultiplied
-
<< QVideoFrame::Format_RGB32
-
<< QVideoFrame::Format_RGB24
-
<< QVideoFrame::Format_RGB565
-
<< QVideoFrame::Format_RGB555
-
<< QVideoFrame::Format_ARGB8565_Premultiplied
-
<< QVideoFrame::Format_BGRA32
-
<< QVideoFrame::Format_BGRA32_Premultiplied
-
<< QVideoFrame::Format_BGR32
-
<< QVideoFrame::Format_BGR24
-
<< QVideoFrame::Format_BGR565
-
<< QVideoFrame::Format_BGR555
-
<< QVideoFrame::Format_BGRA5658_Premultiplied
-
<< QVideoFrame::Format_AYUV444
-
<< QVideoFrame::Format_AYUV444_Premultiplied
-
<< QVideoFrame::Format_YUV444
-
<< QVideoFrame::Format_YUV420P
-
<< QVideoFrame::Format_YV12
-
<< QVideoFrame::Format_UYVY
-
<< QVideoFrame::Format_YUYV
-
<< QVideoFrame::Format_NV12
-
<< QVideoFrame::Format_NV21
-
<< QVideoFrame::Format_IMC1
-
<< QVideoFrame::Format_IMC2
-
<< QVideoFrame::Format_IMC3
-
<< QVideoFrame::Format_IMC4
-
<< QVideoFrame::Format_Y8
-
<< QVideoFrame::Format_Y16
-
<< QVideoFrame::Format_Jpeg
-
<< QVideoFrame::Format_CameraRaw
-
<< QVideoFrame::Format_AdobeDng;
-
//qDebug() << listPixelFormats;
-
// Return the formats you will support
-
return listPixelFormats;
-
}
-
bool VideoSurface::present(const QVideoFrame &frame)
-
{
-
// Handle the frame and do your processing
-
if (frame.isValid())
-
{
-
QVideoFrame cloneFrame(frame);
-
emit frameAvailable(cloneFrame);
-
return true;
-
}
-
return false;
-
}
看了上面的代码就知道,只需要外部连接frameAvailable信号就可以获取到每一帧数据。
具体使用:
-
QMediaPlayer *mediaPlayer = new QMediaPlayer;
-
VideoSurface *videoSurface = new VideoSurface;
-
mediaPlayer->setVideoOutput(videoSurface);
-
mediaPlayer->setMedia(QUrl("rtsp://admin:admin@192.168.1.112"));
-
mediaPlayer->play();
把frameAvailable信号与显示窗口的槽连接,比如:
connect(videoSurface, SIGNAL(frameAvailable(QVideoFrame &)), this, SLOT(ProcessFrame(QVideoFrame &)));
-
void QtVideoTest::ProcessFrame(QVideoFrame &frame)
-
{
-
qDebug() << "=============ProcessFrame===============";
-
qDebug() << "width : " << frame.width() << " height : " << frame.height();
-
qDebug() << "start time : " << frame.startTime()/1000 << "ms";
-
qDebug() << "end time : " << frame.endTime()/1000 << "ms";
-
qDebug() << "pixelFormat :" << frame.pixelFormat();
-
frame.map(QAbstractVideoBuffer::ReadOnly);
-
QImage recvImage(frame.bits(), frame.width(), frame.height(), QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat()));
-
qDebug() << "frame data size :" << frame.mappedBytes();
-
frame.unmap();
-
}
如上,QVideoFrame转QImage,拿QImage进行画图操作就简单了。