原来Qt从视频中获取每一帧数据如此简单

有时候需要在视频上画图，所以需要能获取到每一帧视频数据。

以前从视频文件或视频流中得到帧，一般都是使用qt + ffmpeg或qt + vlc。

qt对显示处理视频大体有以下方法：

1. QMediaPlayer + QVideoWidget

这种方法只适合简单的显示视频功能，不适合对视频进行处理（比如画图）

2. QMediaPlayer + QGraphicsVideoItem + QGraphicsScene + QGraphicsView

这种方法功能强大，除了显示视频功能，还可以做复杂的图形处理(具体可以查看QGraphicsScene的使用)

3. QMediaPlayer + QAbstractVideoSurface

这种方法比较简单，是我下面要介给的。可以获取到每一帧视频数据，基本可以实现与qt + ffmpeg或qt + vlc相同的效果。

 

自定义VideoSurface：

class VideoSurface : public QAbstractVideoSurface
{
	Q_OBJECT

public:
	VideoSurface(QObject *parent = Q_NULLPTR);
	~VideoSurface();

	QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const;
	bool present(const QVideoFrame &frame);

signals:
	void frameAvailable(QVideoFrame &frame);

};

实现如下：

VideoSurface::VideoSurface(QObject *parent)
	: QAbstractVideoSurface(parent)
{
}

VideoSurface::~VideoSurface()
{
}

QList<QVideoFrame::PixelFormat> VideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
	QList<QVideoFrame::PixelFormat> listPixelFormats;

	listPixelFormats << QVideoFrame::Format_ARGB32
		<< QVideoFrame::Format_ARGB32_Premultiplied
		<< QVideoFrame::Format_RGB32
		<< QVideoFrame::Format_RGB24
		<< QVideoFrame::Format_RGB565
		<< QVideoFrame::Format_RGB555
		<< QVideoFrame::Format_ARGB8565_Premultiplied
		<< QVideoFrame::Format_BGRA32
		<< QVideoFrame::Format_BGRA32_Premultiplied
		<< QVideoFrame::Format_BGR32
		<< QVideoFrame::Format_BGR24
		<< QVideoFrame::Format_BGR565
		<< QVideoFrame::Format_BGR555
		<< QVideoFrame::Format_BGRA5658_Premultiplied
		<< QVideoFrame::Format_AYUV444
		<< QVideoFrame::Format_AYUV444_Premultiplied
		<< QVideoFrame::Format_YUV444
		<< QVideoFrame::Format_YUV420P
		<< QVideoFrame::Format_YV12
		<< QVideoFrame::Format_UYVY
		<< QVideoFrame::Format_YUYV
		<< QVideoFrame::Format_NV12
		<< QVideoFrame::Format_NV21
		<< QVideoFrame::Format_IMC1
		<< QVideoFrame::Format_IMC2
		<< QVideoFrame::Format_IMC3
		<< QVideoFrame::Format_IMC4
		<< QVideoFrame::Format_Y8
		<< QVideoFrame::Format_Y16
		<< QVideoFrame::Format_Jpeg
		<< QVideoFrame::Format_CameraRaw
		<< QVideoFrame::Format_AdobeDng;

	//qDebug() << listPixelFormats;

	// Return the formats you will support
	return listPixelFormats;
}

bool  VideoSurface::present(const QVideoFrame &frame)
{
	// Handle the frame and do your processing
	if (frame.isValid())
	{
		QVideoFrame cloneFrame(frame);
		emit frameAvailable(cloneFrame);

		return true;
	}

	return false;
}

看了上面的代码就知道，只需要外部连接frameAvailable信号就可以获取到每一帧数据。

具体使用：

QMediaPlayer *mediaPlayer = new QMediaPlayer;
VideoSurface *videoSurface = new VideoSurface;
mediaPlayer->setVideoOutput(videoSurface);
mediaPlayer->setMedia(QUrl("rtsp://admin:admin@192.168.1.112"));
mediaPlayer->play();

把frameAvailable信号与显示窗口的槽连接，比如：

connect(videoSurface, SIGNAL(frameAvailable(QVideoFrame &)), this, SLOT(ProcessFrame(QVideoFrame &)));

void QtVideoTest::ProcessFrame(QVideoFrame &frame)
{
	qDebug() << "=============ProcessFrame===============";
	qDebug() << "width : " << frame.width() << " height : " << frame.height();
	qDebug() << "start time : " << frame.startTime()/1000 << "ms";
	qDebug() << "end time : " << frame.endTime()/1000 << "ms";
	qDebug() << "pixelFormat :" << frame.pixelFormat();

	frame.map(QAbstractVideoBuffer::ReadOnly);
	QImage recvImage(frame.bits(), frame.width(), frame.height(), QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat()));

	qDebug() << "frame data size :" << frame.mappedBytes();
	frame.unmap();
}

如上，QVideoFrame转QImage，拿QImage进行画图操作就简单了。