有时候需要在视频上画图,所以需要能获取到每一帧视频数据。
以前从视频文件或视频流中得到帧,一般都是使用qt + ffmpeg或qt + vlc。
qt对显示处理视频大体有以下方法:
1. QMediaPlayer + QVideoWidget
这种方法只适合简单的显示视频功能,不适合对视频进行处理(比如画图)
2. QMediaPlayer + QGraphicsVideoItem + QGraphicsScene + QGraphicsView
这种方法功能强大,除了显示视频功能,还可以做复杂的图形处理(具体可以查看QGraphicsScene的使用)
3. QMediaPlayer + QAbstractVideoSurface
这种方法比较简单,是我下面要介给的。可以获取到每一帧视频数据,基本可以实现与qt + ffmpeg或qt + vlc相同的效果。
自定义VideoSurface:
class VideoSurface : public QAbstractVideoSurface
{
Q_OBJECT
public:
VideoSurface(QObject *parent = Q_NULLPTR);
~VideoSurface();
QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const;
bool present(const QVideoFrame &frame);
signals:
void frameAvailable(QVideoFrame &frame);
};
实现如下:
VideoSurface::VideoSurface(QObject *parent)
: QAbstractVideoSurface(parent)
{
}
VideoSurface::~VideoSurface()
{
}
QList<QVideoFrame::PixelFormat> VideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
QList<QVideoFrame::PixelFormat> listPixelFormats;
listPixelFormats << QVideoFrame::Format_ARGB32
<< QVideoFrame::Format_ARGB32_Premultiplied
<< QVideoFrame::Format_RGB32
<< QVideoFrame::Format_RGB24
<< QVideoFrame::Format_RGB565
<< QVideoFrame::Format_RGB555
<< QVideoFrame::Format_ARGB8565_Premultiplied
<< QVideoFrame::Format_BGRA32
<< QVideoFrame::Format_BGRA32_Premultiplied
<< QVideoFrame::Format_BGR32
<< QVideoFrame::Format_BGR24
<< QVideoFrame::Format_BGR565
<< QVideoFrame::Format_BGR555
<< QVideoFrame::Format_BGRA5658_Premultiplied
<< QVideoFrame::Format_AYUV444
<< QVideoFrame::Format_AYUV444_Premultiplied
<< QVideoFrame::Format_YUV444
<< QVideoFrame::Format_YUV420P
<< QVideoFrame::Format_YV12
<< QVideoFrame::Format_UYVY
<< QVideoFrame::Format_YUYV
<< QVideoFrame::Format_NV12
<< QVideoFrame::Format_NV21
<< QVideoFrame::Format_IMC1
<< QVideoFrame::Format_IMC2
<< QVideoFrame::Format_IMC3
<< QVideoFrame::Format_IMC4
<< QVideoFrame::Format_Y8
<< QVideoFrame::Format_Y16
<< QVideoFrame::Format_Jpeg
<< QVideoFrame::Format_CameraRaw
<< QVideoFrame::Format_AdobeDng;
//qDebug() << listPixelFormats;
// Return the formats you will support
return listPixelFormats;
}
bool VideoSurface::present(const QVideoFrame &frame)
{
// Handle the frame and do your processing
if (frame.isValid())
{
QVideoFrame cloneFrame(frame);
emit frameAvailable(cloneFrame);
return true;
}
return false;
}
看了上面的代码就知道,只需要外部连接frameAvailable信号就可以获取到每一帧数据。
具体使用:
QMediaPlayer *mediaPlayer = new QMediaPlayer;
VideoSurface *videoSurface = new VideoSurface;
mediaPlayer->setVideoOutput(videoSurface);
mediaPlayer->setMedia(QUrl("rtsp://admin:admin@192.168.1.112"));
mediaPlayer->play();
把frameAvailable信号与显示窗口的槽连接,比如:
connect(videoSurface, SIGNAL(frameAvailable(QVideoFrame &)), this, SLOT(ProcessFrame(QVideoFrame &)));
void QtVideoTest::ProcessFrame(QVideoFrame &frame)
{
qDebug() << "=============ProcessFrame===============";
qDebug() << "width : " << frame.width() << " height : " << frame.height();
qDebug() << "start time : " << frame.startTime()/1000 << "ms";
qDebug() << "end time : " << frame.endTime()/1000 << "ms";
qDebug() << "pixelFormat :" << frame.pixelFormat();
frame.map(QAbstractVideoBuffer::ReadOnly);
QImage recvImage(frame.bits(), frame.width(), frame.height(), QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat()));
qDebug() << "frame data size :" << frame.mappedBytes();
frame.unmap();
}
如上,QVideoFrame转QImage,拿QImage进行画图操作就简单了。