一、简介
Qt对音视频的播放和控制，相机拍照，收音机等多媒体应用提供了强大的支持。Qt5使用了全新的Qt Multimedia模块来实现多媒体应用，而原来Qt4中用于实现多媒体功能的Phonon模块已经被移除。
新的Qt Multimedia模块提供了丰富的接口，使读者可以轻松地使用平台的多媒体功能，例如，进行多媒体播放，使用相机和收音机等。该模块还分别提供了一组QML类型和一组C++类来处理多媒体内容。
C++ 中多媒体模块可以实现的功能，对应的示例程序以及需要使用的C++类如：
Qt的多媒体接口建立在底层平台的多媒体框架之上，这就意味着对于各种编解码器的支持依赖于使用的平台。如果要访问一些平台相关的设置，或者将Qt多媒体接口移植到新的平台，则可以参考Qt帮助中的Multimedia Backend Development文档。
另外，如果要使用多媒体模块中的内容，则需要在.pro项目文件中添加如下代码，引入多媒体模块：
QT += multimedia二、几个主要类介绍
QMediaPlayer
提供给外部应用程序的主要API，应用程序可以通过调用其成员函数play，setVolume，setPosition等控制视频文件的播放。大部分成员函数都是通过调用QMediaPlayerControl类型指针的方法来实现的。
QMediaControl
控制媒体的抽象类，包含大量控制媒体的成员函数。
QMediaServiceProvider
提供媒体服务的抽象类，主要功能是requestService得到QMediaService对象。
QPluginServiceProvider
QPluginServiceProvider继承QMediaServiceProvider类，通过requestService方法给QMediaPlayer提供QMediaService。
QMediaService
媒体服务的抽象类，主要功能是requestControl得到QMediaControl对象。
QMediaServiceProviderPlugin
所有提供媒体服务的plugin都必须继承这个抽象类。create成员函数用来得到实现后的QMediaService派生类实例的指针，key成员函数用来得到一个QStringList，里面包含这个plugin中能提供的所有媒体服务的id。
注：由于一个plugin可能包含几个QMediaServiceProvider的实现，一个QMediaServiceProvider的实现又可能提供几个QMediaService的实现，一个QMediaService的实现也可能提供几个QMediaControl的实现...所以他们的每个派生类都有一个id来识别。
名字 | QMediaServiceProviderPlugin | QMediaService |
directshow | DSServicePlugin | DSCameraService、DirectShowPlayerService |
windowsmediafoundation | WMFServicePlugin | MFPlayerService/MFAudioDecoderService |
gstreamer | QGstreamerPlayerServicePlugin | QGstreamerPlayerService |
三、QMediaServiceProvider
提供媒体服务的抽象类，主要功能是requestService得到QMediaService对象
class QMediaService;
class Q_MULTIMEDIA_EXPORT QMediaServiceProvider : public QObject
{
Q_OBJECT
public:
virtual QMediaService* requestService(const QByteArray &type, const QMediaServiceProviderHint &hint = QMediaServiceProviderHint()) = 0;
virtual void releaseService(QMediaService *service) = 0;
virtual QMediaServiceProviderHint::Features supportedFeatures(const QMediaService *service) const;
virtual QMultimedia::SupportEstimate hasSupport(const QByteArray &serviceType,
const QString &mimeType,
const QStringList& codecs,
int flags = 0) const;
virtual QStringList supportedMimeTypes(const QByteArray &serviceType, int flags = 0) const;
virtual QByteArray defaultDevice(const QByteArray &serviceType) const;
virtual QList<QByteArray> devices(const QByteArray &serviceType) const;
virtual QString deviceDescription(const QByteArray &serviceType, const QByteArray &device);
virtual QCamera::Position cameraPosition(const QByteArray &device) const;
virtual int cameraOrientation(const QByteArray &device) const;
static QMediaServiceProvider* defaultServiceProvider();
static void setDefaultServiceProvider(QMediaServiceProvider *provider);
};四、QPluginServiceProvider
QPluginServiceProvider继承QMediaServiceProvider类，在requestService方法中相应的加载QMediaServiceProviderPlugin实现类（windows下的DSServicePlugin、WMFServicePlugin、Linux下的QGstreamerPlayerServicePlugin），在QMediaServiceProviderPlugin实现类中会创建对应的QMediaService。
class QPluginServiceProvider : public QMediaServiceProvider
{
struct MediaServiceData {
QByteArray type;
QMediaServiceProviderPlugin *plugin;
MediaServiceData() : plugin(nullptr) { }
};
QMap<const QMediaService*, MediaServiceData> mediaServiceData;
public:
// type: "org.qt-project.qt.mediaplayer"
QMediaService* requestService(const QByteArray &type, const QMediaServiceProviderHint &hint) override {
// ...
// 选择合适的QMediaServiceProviderPlugin，有DSServicePlugin、WMFServicePlugin、QGstreamerPlayerServicePlugin
// 相应的Plugin中创建对应的QMediaService
}
void releaseService(QMediaService *service) override {
// ...
}
QMediaServiceProviderHint::Features supportedFeatures(const QMediaService *service) const override {
// ...
}
QMultimedia::SupportEstimate hasSupport(const QByteArray &serviceType,
const QString &mimeType,
const QStringList& codecs,
int flags) const override {
// ...
}
QStringList supportedMimeTypes(const QByteArray &serviceType, int flags) const override {
// ...
}
QByteArray defaultDevice(const QByteArray &serviceType) const override {
// ...
}
QList<QByteArray> devices(const QByteArray &serviceType) const override {
// ...
}
QString deviceDescription(const QByteArray &serviceType, const QByteArray &device) override {
// ...
}
QCamera::Position cameraPosition(const QByteArray &device) const override {
// ...
}
int cameraOrientation(const QByteArray &device) const override {
// ...
}
};五、QMediaServiceProviderPlugin
class Q_MULTIMEDIA_EXPORT QMediaServiceProviderPlugin : public QObject, public QMediaServiceProviderFactoryInterface
{
Q_OBJECT
Q_INTERFACES(QMediaServiceProviderFactoryInterface)
public:
QMediaService* create(const QString& key) override = 0;
void release(QMediaService *service) override = 0;
};5.1 DSServicePlugin
class DSServicePlugin
: public QMediaServiceProviderPlugin
, public QMediaServiceSupportedDevicesInterface
, public QMediaServiceDefaultDeviceInterface
, public QMediaServiceFeaturesInterface
{
#if QT_CONFIG(directshow_player)
Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "directshow.json")
#else
Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "directshow_camera.json")
#endif
// ...
}5.2 WMFServicePlugin
class WMFServicePlugin
: public QMediaServiceProviderPlugin
, public QMediaServiceSupportedDevicesInterface
, public QMediaServiceDefaultDeviceInterface
, public QMediaServiceFeaturesInterface
{
#if QT_CONFIG(wmf_player)
Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "wmf.json")
#else
Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "wmf_audiodecode.json")
#endif
// ...
}5.3 QGstreamerPlayerServicePlugin
class QGstreamerPlayerServicePlugin
: public QMediaServiceProviderPlugin
, public QMediaServiceFeaturesInterface
, public QMediaServiceSupportedFormatsInterface
{
Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "mediaplayer.json")
// ...
}六 gstreamer
gstreamer，其插件类为QGstreamerPlayerServicePlugin，其Service类为QGstreamerPlayerService。
#linux下播放插件为libgstmediaplayer.so，其链接到了libgstreamer-1.0.so.0上，最终使用了gstreamer
ldd libgstmediaplayer.so | grep libgstreamer*
libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007f66d65b6000)ubuntu下的qt和gstreamer1.0环境报错问题解决
ubuntu16.04 18.04 Qt5.11安装Gstreamer
【FFMPEG】gstreamer插件调用ffmpeg 详解
七 ffmpeg
八 QMediaPlayer初始化过程
1、QMediaPlayer实例化
// playerService
QMediaPlayer::QMediaPlayer(QObject *parent, QMediaPlayer::Flags flags):
QMediaObject(*new QMediaPlayerPrivate,
parent,
playerService(flags)) // 详见1.1
{
Q_D(QMediaPlayer);
d->provider = QMediaServiceProvider::defaultServiceProvider();
if (d->service == nullptr) {
d->error = ServiceMissingError;
} else {
// 此处control为QGstreamerPlayerService中创建的QGstreamerPlayerControl
d->control = qobject_cast<QMediaPlayerControl*>(d->service->requestControl(QMediaPlayerControl_iid));
#ifndef QT_NO_BEARERMANAGEMENT
QT_WARNING_PUSH
QT_WARNING_DISABLE_DEPRECATED
d->networkAccessControl = qobject_cast<QMediaNetworkAccessControl*>(d->service->requestControl(QMediaNetworkAccessControl_iid));
QT_WARNING_POP
#endif
if (d->control != nullptr) {
connect(d->control, SIGNAL(mediaChanged(QMediaContent)), SLOT(_q_handleMediaChanged(QMediaContent)));
connect(d->control, SIGNAL(stateChanged(QMediaPlayer::State)), SLOT(_q_stateChanged(QMediaPlayer::State)));
connect(d->control, SIGNAL(mediaStatusChanged(QMediaPlayer::MediaStatus)),
SLOT(_q_mediaStatusChanged(QMediaPlayer::MediaStatus)));
connect(d->control, SIGNAL(error(int,QString)), SLOT(_q_error(int,QString)));
connect(d->control, &QMediaPlayerControl::durationChanged, this, &QMediaPlayer::durationChanged);
connect(d->control, &QMediaPlayerControl::positionChanged, this, &QMediaPlayer::positionChanged);
connect(d->control, &QMediaPlayerControl::audioAvailableChanged, this, &QMediaPlayer::audioAvailableChanged);
connect(d->control, &QMediaPlayerControl::videoAvailableChanged, this, &QMediaPlayer::videoAvailableChanged);
connect(d->control, &QMediaPlayerControl::volumeChanged, this, &QMediaPlayer::volumeChanged);
connect(d->control, &QMediaPlayerControl::mutedChanged, this, &QMediaPlayer::mutedChanged);
connect(d->control, &QMediaPlayerControl::seekableChanged, this, &QMediaPlayer::seekableChanged);
connect(d->control, &QMediaPlayerControl::playbackRateChanged, this, &QMediaPlayer::playbackRateChanged);
connect(d->control, &QMediaPlayerControl::bufferStatusChanged, this, &QMediaPlayer::bufferStatusChanged);
d->state = d->control->state();
d->status = d->control->mediaStatus();
if (d->state == PlayingState)
addPropertyWatch("position");
if (d->status == StalledMedia || d->status == BufferingMedia)
addPropertyWatch("bufferStatus");
d->hasStreamPlaybackFeature = d->provider->supportedFeatures(d->service).testFlag(QMediaServiceProviderHint::StreamPlayback);
d->audioRoleControl = qobject_cast<QAudioRoleControl*>(d->service->requestControl(QAudioRoleControl_iid));
if (d->audioRoleControl) {
connect(d->audioRoleControl, &QAudioRoleControl::audioRoleChanged,
this, &QMediaPlayer::audioRoleChanged);
d->customAudioRoleControl = qobject_cast<QCustomAudioRoleControl *>(
d->service->requestControl(QCustomAudioRoleControl_iid));
if (d->customAudioRoleControl) {
connect(d->customAudioRoleControl,
&QCustomAudioRoleControl::customAudioRoleChanged,
this,
&QMediaPlayer::customAudioRoleChanged);
}
}
}
#ifndef QT_NO_BEARERMANAGEMENT
if (d->networkAccessControl != nullptr) {
QT_WARNING_PUSH
QT_WARNING_DISABLE_DEPRECATED
connect(d->networkAccessControl, &QMediaNetworkAccessControl::configurationChanged,
this, &QMediaPlayer::networkConfigurationChanged);
QT_WARNING_POP
}
#endif
}
}1.1 playerService
// 通过静态方法获取QMediaService
static QMediaService *playerService(QMediaPlayer::Flags flags)
{
// 获取默认的QMediaServiceProvider ==> 1.1.1
QMediaServiceProvider *provider = QMediaServiceProvider::defaultServiceProvider();
if (flags) {
QMediaServiceProviderHint::Features features;
if (flags & QMediaPlayer::LowLatency)
features |= QMediaServiceProviderHint::LowLatencyPlayback;
if (flags & QMediaPlayer::StreamPlayback)
features |= QMediaServiceProviderHint::StreamPlayback;
if (flags & QMediaPlayer::VideoSurface)
features |= QMediaServiceProviderHint::VideoSurface;
return provider->requestService(Q_MEDIASERVICE_MEDIAPLAYER,
QMediaServiceProviderHint(features));
}
// ==> 1.1.2 QPluginServiceProvider->requestService
return provider->requestService(Q_MEDIASERVICE_MEDIAPLAYER);
}1.1.1 defaultServiceProvider
QMediaServiceProvider *QMediaServiceProvider::defaultServiceProvider()
{
// qt_defaultMediaServiceProvider默认为空，
// pluginProvider()是由Q_GLOBAL_STATIC(QPluginServiceProvider, pluginProvider)定义，
// 返回类型为QPluginServiceProvider类型的对象
return qt_defaultMediaServiceProvider != nullptr
? qt_defaultMediaServiceProvider
: static_cast<QMediaServiceProvider *>(pluginProvider());
}
// 定义命名空间为pluginProvider的QPluginServiceProvider对象 ？
Q_GLOBAL_STATIC(QPluginServiceProvider, pluginProvider);1.1.2 QPluginServiceProvider::requestService
class QPluginServiceProvider : public QMediaServiceProvider
{
struct MediaServiceData {
QByteArray type;
QMediaServiceProviderPlugin *plugin;
MediaServiceData() : plugin(nullptr) { }
};
QMap<const QMediaService*, MediaServiceData> mediaServiceData;
public:
QMediaService* requestService(const QByteArray &type, const QMediaServiceProviderHint &hint) override
{
// key 的值为 "org.qt-project.qt.mediaplayer"
QString key(QLatin1String(type.constData()));
QList<QMediaServiceProviderPlugin *>plugins;
// loader()是由Q_GLOBAL_STATIC_WITH_ARGS(QMediaPluginLoader, loader,(QMediaServiceProviderFactoryInterface_iid, QLatin1String("mediaservice"), Qt::CaseInsensitive))定义，
// 其类型是QMediaPluginLoader对象, ==> 1.1.2.1 , 1.1.2.2
// instances值在linux下为QGstreamerPlayerServicePlugin
const auto instances = loader()->instances(key);
for (QObject *obj : instances) {
QMediaServiceProviderPlugin *plugin =
qobject_cast<QMediaServiceProviderPlugin*>(obj);
if (plugin)
plugins << plugin;
}
if (!plugins.isEmpty()) {
QMediaServiceProviderPlugin *plugin = nullptr;
// 此处hint.type()值为QMediaServiceProviderHint::Null
switch (hint.type()) {
case QMediaServiceProviderHint::Null:
plugin = plugins[0];
//special case for media player, if low latency was not asked,
//prefer services not offering it, since they are likely to support
//more formats
if (type == QByteArray(Q_MEDIASERVICE_MEDIAPLAYER)) {
for (QMediaServiceProviderPlugin *currentPlugin : qAsConst(plugins)) {
QMediaServiceFeaturesInterface *iface =
qobject_cast<QMediaServiceFeaturesInterface*>(currentPlugin);
if (!iface || !(iface->supportedFeatures(type) &
QMediaServiceProviderHint::LowLatencyPlayback)) {
plugin = currentPlugin;
break;
}
}
}
break;
case QMediaServiceProviderHint::SupportedFeatures:
// ...
break;
case QMediaServiceProviderHint::Device:
// ...
break;
case QMediaServiceProviderHint::CameraPosition: {
// ...
}
break;
case QMediaServiceProviderHint::ContentType: {
// ...
}
break;
}
if (plugin != nullptr) {
// 返回类型为QGstreamerPlayerService ==> 1.1.2.3
QMediaService *service = plugin->create(key);
if (service != nullptr) {
MediaServiceData d;
d.type = type;
d.plugin = plugin;
mediaServiceData.insert(service, d);
}
return service;
}
}
qWarning() << "defaultServiceProvider::requestService(): no service found for -" << key;
return nullptr;
}
// ...
}1.1.2.1 loader()->instances(key)
1.1.2.1.1 QMediaPluginLoader
Q_GLOBAL_STATIC_WITH_ARGS(QMediaPluginLoader, loader,
(QMediaServiceProviderFactoryInterface_iid, QLatin1String("mediaservice"), Qt::CaseInsensitive))相应的宏为：
# define Q_GLOBAL_STATIC_INTERNAL_DECORATION Q_DECL_HIDDEN inline
#define Q_GLOBAL_STATIC_INTERNAL(ARGS) \
Q_GLOBAL_STATIC_INTERNAL_DECORATION Type *innerFunction() \
{ \
struct HolderBase { \
~HolderBase() noexcept \
{ if (guard.loadRelaxed() == QtGlobalStatic::Initialized) \
guard.storeRelaxed(QtGlobalStatic::Destroyed); } \
}; \
static struct Holder : public HolderBase { \
Type value; \
Holder() \
noexcept(noexcept(Type ARGS)) \
: value ARGS \
{ guard.storeRelaxed(QtGlobalStatic::Initialized); } \
} holder; \
return &holder.value; \
}
// this class must be POD, unless the compiler supports thread-safe statics
template <typename T, T *(&innerFunction)(), QBasicAtomicInt &guard>
struct QGlobalStatic
{
typedef T Type;
bool isDestroyed() const { return guard.loadRelaxed() <= QtGlobalStatic::Destroyed; }
bool exists() const { return guard.loadRelaxed() == QtGlobalStatic::Initialized; }
operator Type *() { if (isDestroyed()) return nullptr; return innerFunction(); }
Type *operator()() { if (isDestroyed()) return nullptr; return innerFunction(); }
Type *operator->()
{
Q_ASSERT_X(!isDestroyed(), "Q_GLOBAL_STATIC", "The global static was used after being destroyed");
return innerFunction();
}
Type &operator*()
{
Q_ASSERT_X(!isDestroyed(), "Q_GLOBAL_STATIC", "The global static was used after being destroyed");
return *innerFunction();
}
};
#define Q_GLOBAL_STATIC_WITH_ARGS(TYPE, NAME, ARGS) \
namespace { namespace Q_QGS_ ## NAME { \
typedef TYPE Type; \
QBasicAtomicInt guard = Q_BASIC_ATOMIC_INITIALIZER(QtGlobalStatic::Uninitialized); \
Q_GLOBAL_STATIC_INTERNAL(ARGS) \
} } \
static QGlobalStatic<TYPE, \
Q_QGS_ ## NAME::innerFunction, \
Q_QGS_ ## NAME::guard> NAME;
#define Q_GLOBAL_STATIC(TYPE, NAME) \
Q_GLOBAL_STATIC_WITH_ARGS(TYPE, NAME, ())
其扩展后变成：
namespace {
namespace Q_QGS_loader {
typedef QMediaPluginLoader Type;
// 初始化 guard 原子性变量
QBasicAtomicInt guard = Q_BASIC_ATOMIC_INITIALIZER(QtGlobalStatic::Uninitialized);
//Q_GLOBAL_STATIC_INTERNAL(ARGS)
inline QMediaPluginLoader *innerFunction() { \
struct HolderBase {
~HolderBase() noexcept
{ if (guard.loadRelaxed() == QtGlobalStatic::Initialized)
guard.storeRelaxed(QtGlobalStatic::Destroyed); }
};
static struct Holder : public HolderBase {
QMediaPluginLoader value;
Holder()
noexcept(noexcept(QMediaPluginLoader(QMediaServiceProviderFactoryInterface_iid, QLatin1String("mediaservice"), Qt::CaseInsensitive)))
: value(QMediaServiceProviderFactoryInterface_iid, QLatin1String("mediaservice"), Qt::CaseInsensitive)
{ guard.storeRelaxed(QtGlobalStatic::Initialized); }
} holder;
return &holder.value;
}
}
}
//static QGlobalStatic<QMediaPluginLoader, Q_QGS_loader::innerFunction,
// Q_QGS_loader::guard> loader;
static struct QGlobalStatic<QMediaPluginLoader, Q_QGS_loader::innerFunction,
Q_QGS_loader::guard> {
typedef QMediaPluginLoader Type;
bool isDestroyed() const { return Q_QGS_loader::guard.loadRelaxed() <= QtGlobalStatic::Destroyed; }
bool exists() const { return Q_QGS_loader::guard.loadRelaxed() == QtGlobalStatic::Initialized; }
operator Type *() { if (isDestroyed()) return nullptr; return Q_QGS_loader::innerFunction(); }
Type *operator()() { if (isDestroyed()) return nullptr; return Q_QGS_loader::innerFunction(); }
Type *operator->() {
Q_ASSERT_X(!isDestroyed(), "Q_GLOBAL_STATIC", "The global static was used after being destroyed");
return Q_QGS_loader::innerFunction();
}
Type &operator*() {
Q_ASSERT_X(!isDestroyed(), "Q_GLOBAL_STATIC", "The global static was used after being destroyed");
return *Q_QGS_loader::innerFunction();
}
} loader;从上可知loader()是operator()()的重载，其返回QMediaPluginLoader*对象
1.1.2.1.2 QMediaPluginLoader实例化
/* 实例化的时，传入的参数
* iid: QMediaServiceProviderFactoryInterface_iid, 值为
* "org.qt-project.qt.mediaserviceproviderfactory/5.0"
* location: QLatin1String("mediaservice")
* caseSensitivity: Qt::CaseInsensitive
*/
QMediaPluginLoader::QMediaPluginLoader(const char *iid, const QString &location, Qt::CaseSensitivity caseSensitivity):
m_iid(iid)
{
#if defined(Q_OS_ANDROID)
m_location = QString(location).replace(QLatin1Char('/'), QLatin1Char('_'));
#else
m_location = QString::fromLatin1("/%1").arg(location);
#endif
m_factoryLoader = new QFactoryLoader(m_iid, m_location, caseSensitivity);
loadMetadata();
}1.1.2.1.2.1 QFactoryLoader实例化
QMediaPluginLoader实例化时，会创建QFactoryLoader对象，其里面调用update方法更新Metadata信息，然后通过loadMetadata()中使用。
update方法中会查找Qt plugins/mediaservice下的插件，并加载成QLibraryPrivate对象，
然后读取QLibraryPrivate中元数据（qt plugins 插件框架），通过元数据判断是否是想要的插件，是则放入把QLibraryPrivate放入到QFactoryLoaderPrivate->libraryList中，以供loadMetadata()读取。
QFactoryLoader::QFactoryLoader(const char *iid,
const QString &suffix,
Qt::CaseSensitivity cs)
: QObject(*new QFactoryLoaderPrivate)
{
moveToThread(QCoreApplicationPrivate::mainThread());
Q_D(QFactoryLoader);
d->iid = iid;
#if QT_CONFIG(library)
d->cs = cs;
d->suffix = suffix;
# ifdef Q_OS_ANDROID
if (!d->suffix.isEmpty() && d->suffix.at(0) == QLatin1Char('/'))
d->suffix.remove(0, 1);
# endif
QMutexLocker locker(qt_factoryloader_mutex());
update();
qt_factory_loaders()->append(this);
#else
Q_UNUSED(suffix);
Q_UNUSED(cs);
#endif
}
// 查找Qt plugins/mediaservice下的插件，并加载成QLibraryPrivate对象，
// 读取QLibraryPrivate中元数据，如果是想要的插件，则放入QFactoryLoaderPrivate->libraryList中，
// 以供loadMetadata()读取
void QFactoryLoader::update()
{
#ifdef QT_SHARED
Q_D(QFactoryLoader);
// paths的值为：
// "/home/Qt/5.15.2/gcc_64/plugins"
// "/home/Qt/Examples/Qt-5.15.2/multimediawidgets/build-player-Desktop_Qt_5_15_2_GCC_64bit-Debug"
QStringList paths = QCoreApplication::libraryPaths();
for (int i = 0; i < paths.count(); ++i) {
const QString &pluginDir = paths.at(i);
// Already loaded, skip it...
if (d->loadedPaths.contains(pluginDir))
continue;
d->loadedPaths << pluginDir;
QString path = pluginDir + d->suffix;
if (!QDir(path).exists(QLatin1String(".")))
continue;
QStringList plugins = QDir(path).entryList(
#if defined(Q_OS_WIN)
QStringList(QStringLiteral("*.dll")),
#elif defined(Q_OS_ANDROID)
QStringList(QLatin1String("libplugins_%1_*.so").arg(d->suffix)),
#endif
QDir::Files);
QLibraryPrivate *library = nullptr;
for (int j = 0; j < plugins.count(); ++j) {
QString fileName = QDir::cleanPath(path + QLatin1Char('/') + plugins.at(j));
#ifdef Q_OS_MAC
const bool isDebugPlugin = fileName.endsWith(QLatin1String("_debug.dylib"));
const bool isDebugLibrary =
#ifdef QT_DEBUG
true;
#else
false;
#endif
// Skip mismatching plugins so that we don't end up loading both debug and release
// versions of the same Qt libraries (due to the plugin's dependencies).
if (isDebugPlugin != isDebugLibrary)
continue;
#elif defined(Q_PROCESSOR_X86)
if (fileName.endsWith(QLatin1String(".avx2")) || fileName.endsWith(QLatin1String(".avx512"))) {
// ignore AVX2-optimized file, we'll do a bait-and-switch to it later
continue;
}
#endif
if (qt_debug_component()) {
qDebug() << "QFactoryLoader::QFactoryLoader() looking at" << fileName;
}
Q_TRACE(QFactoryLoader_update, fileName);
library = QLibraryPrivate::findOrCreate(QFileInfo(fileName).canonicalFilePath());
// ==> 1.1.2.1.2.1.1
if (!library->isPlugin()) {
if (qt_debug_component()) {
qDebug() << library->errorString << Qt::endl
<< " not a plugin";
}
library->release();
continue;
}
QStringList keys;
bool metaDataOk = false;
QString iid = library->metaData.value(QLatin1String("IID")).toString();
if (iid == QLatin1String(d->iid.constData(), d->iid.size())) {
QJsonObject object = library->metaData.value(QLatin1String("MetaData")).toObject();
metaDataOk = true;
QJsonArray k = object.value(QLatin1String("Keys")).toArray();
for (int i = 0; i < k.size(); ++i)
keys += d->cs ? k.at(i).toString() : k.at(i).toString().toLower();
}
if (qt_debug_component())
qDebug() << "Got keys from plugin meta data" << keys;
if (!metaDataOk) {
library->release();
continue;
}
int keyUsageCount = 0;
for (int k = 0; k < keys.count(); ++k) {
// first come first serve, unless the first
// library was built with a future Qt version,
// whereas the new one has a Qt version that fits
// better
const QString &key = keys.at(k);
QLibraryPrivate *previous = d->keyMap.value(key);
int prev_qt_version = 0;
if (previous) {
prev_qt_version = (int)previous->metaData.value(QLatin1String("version")).toDouble();
}
int qt_version = (int)library->metaData.value(QLatin1String("version")).toDouble();
if (!previous || (prev_qt_version > QT_VERSION && qt_version <= QT_VERSION)) {
d->keyMap[key] = library;
++keyUsageCount;
}
}
if (keyUsageCount || keys.isEmpty()) {
library->setLoadHints(QLibrary::PreventUnloadHint); // once loaded, don't unload
QMutexLocker locker(&d->mutex);
d->libraryList += library;
} else {
library->release();
}
}
}
#else
Q_D(QFactoryLoader);
if (qt_debug_component()) {
qDebug() << "QFactoryLoader::QFactoryLoader() ignoring" << d->iid
<< "since plugins are disabled in static builds";
}
#endif
}1.1.2.1.2.1.1 library->isPlugin()
bool QLibraryPrivate::isPlugin()
{
if (pluginState == MightBeAPlugin)
updatePluginState();
return pluginState == IsAPlugin;
}
void QLibraryPrivate::updatePluginState()
{
QMutexLocker locker(&mutex);
errorString.clear();
if (pluginState != MightBeAPlugin)
return;
bool success = false;
#if defined(Q_OS_UNIX) && !defined(Q_OS_MAC)
if (fileName.endsWith(QLatin1String(".debug"))) {
// refuse to load a file that ends in .debug
// these are the debug symbols from the libraries
// the problem is that they are valid shared library files
// and dlopen is known to crash while opening them
// pretend we didn't see the file
errorString = QLibrary::tr("The shared library was not found.");
pluginState = IsNotAPlugin;
return;
}
#endif
if (!pHnd.loadRelaxed()) {
// scan for the plugin metadata without loading
// 没有加载，则读入文件内容进行分析 ==> 1.1.2.1.2.1.1.1
success = findPatternUnloaded(fileName, this);
} else {
// library is already loaded (probably via QLibrary)
// simply get the target function and call it.
// 有加载则直接调用qt_plugin_query_metadata函数 ==> 1.1.2.1.2.1.1.2
success = qt_get_metadata(this, &errorString);
}
if (!success) {
if (errorString.isEmpty()){
if (fileName.isEmpty())
errorString = QLibrary::tr("The shared library was not found.");
else
errorString = QLibrary::tr("The file '%1' is not a valid Qt plugin.").arg(fileName);
}
pluginState = IsNotAPlugin;
return;
}
pluginState = IsNotAPlugin; // be pessimistic
uint qt_version = (uint)metaData.value(QLatin1String("version")).toDouble();
bool debug = metaData.value(QLatin1String("debug")).toBool();
if ((qt_version & 0x00ff00) > (QT_VERSION & 0x00ff00) || (qt_version & 0xff0000) != (QT_VERSION & 0xff0000)) {
if (qt_debug_component()) {
qWarning("In %s:\n"
" Plugin uses incompatible Qt library (%d.%d.%d) [%s]",
QFile::encodeName(fileName).constData(),
(qt_version&0xff0000) >> 16, (qt_version&0xff00) >> 8, qt_version&0xff,
debug ? "debug" : "release");
}
errorString = QLibrary::tr("The plugin '%1' uses incompatible Qt library. (%2.%3.%4) [%5]")
.arg(fileName)
.arg((qt_version&0xff0000) >> 16)
.arg((qt_version&0xff00) >> 8)
.arg(qt_version&0xff)
.arg(debug ? QLatin1String("debug") : QLatin1String("release"));
#ifndef QT_NO_DEBUG_PLUGIN_CHECK
} else if(debug != QLIBRARY_AS_DEBUG) {
//don't issue a qWarning since we will hopefully find a non-debug? --Sam
errorString = QLibrary::tr("The plugin '%1' uses incompatible Qt library."
" (Cannot mix debug and release libraries.)").arg(fileName);
#endif
} else {
pluginState = IsAPlugin;
}
}1.1.2.1.2.1.1.1 findPatternUnloaded
没有加载，则读入文件内容进行分析
static bool findPatternUnloaded(const QString &library, QLibraryPrivate *lib)
{
QFile file(library);
if (!file.open(QIODevice::ReadOnly)) {
return false;
}
// Files can be bigger than the virtual memory size on 32-bit systems, so
// we limit to 512 MB there. For 64-bit, we allow up to 2^40 bytes.
constexpr qint64 MaxMemoryMapSize =
Q_INT64_C(1) << (sizeof(qsizetype) > 4 ? 40 : 29);
QByteArray data;
qsizetype fdlen = qMin(file.size(), MaxMemoryMapSize);
const char *filedata = reinterpret_cast<char *>(file.map(0, fdlen));
if (filedata == nullptr) {
// Try reading the data into memory instead (up to 64 MB).
data = file.read(64 * 1024 * 1024);
filedata = data.constData();
fdlen = data.size();
}
/*
ELF and Mach-O binaries with GCC have .qplugin sections.
*/
bool hasMetaData = false;
qsizetype pos = 0;
char pattern[] = "qTMETADATA ";
pattern[0] = 'Q'; // Ensure the pattern "QTMETADATA" is not found in this library should QPluginLoader ever encounter it.
const ulong plen = qstrlen(pattern);
#if defined (Q_OF_ELF) && defined(Q_CC_GNU)
int r = QElfParser().parse(filedata, fdlen, library, lib, &pos, &fdlen);
if (r == QElfParser::Corrupt || r == QElfParser::NotElf) {
if (lib && qt_debug_component()) {
qWarning("QElfParser: %ls", qUtf16Printable(lib->errorString));
}
return false;
} else if (r == QElfParser::QtMetaDataSection) {
qsizetype rel = qt_find_pattern(filedata + pos, fdlen, pattern, plen);
if (rel < 0)
pos = -1;
else
pos += rel;
hasMetaData = true;
}
#elif defined (Q_OF_MACH_O)
{
QString errorString;
int r = QMachOParser::parse(filedata, fdlen, library, &errorString, &pos, &fdlen);
if (r == QMachOParser::NotSuitable) {
if (qt_debug_component())
qWarning("QMachOParser: %ls", qUtf16Printable(errorString));
if (lib)
lib->errorString = errorString;
return false;
}
// even if the metadata section was not found, the Mach-O parser will
// at least return the boundaries of the right architecture
qsizetype rel = qt_find_pattern(filedata + pos, fdlen, pattern, plen);
if (rel < 0)
pos = -1;
else
pos += rel;
hasMetaData = true;
}
#else
pos = qt_find_pattern(filedata, fdlen, pattern, plen);
if (pos > 0)
hasMetaData = true;
#endif // defined(Q_OF_ELF) && defined(Q_CC_GNU)
bool ret = false;
if (pos >= 0 && hasMetaData) {
const char *data = filedata + pos;
QString errMsg;
QJsonDocument doc = qJsonFromRawLibraryMetaData(data, fdlen, &errMsg);
if (doc.isNull()) {
qWarning("Found invalid metadata in lib %ls: %ls",
qUtf16Printable(library), qUtf16Printable(errMsg));
} else {
lib->metaData = doc.object();
if (qt_debug_component())
qWarning("Found metadata in lib %s, metadata=\n%s\n",
library.toLocal8Bit().constData(), doc.toJson().constData());
ret = !doc.isNull();
}
}
if (!ret && lib)
lib->errorString = QLibrary::tr("Failed to extract plugin meta data from '%1'").arg(library);
file.close();
return ret;
}
1.1.2.1.2.1.1.2 qt_get_metadata
有加载则直接调用qt_plugin_query_metadata函数
static bool qt_get_metadata(QLibraryPrivate *priv, QString *errMsg)
{
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
auto getMetaData = [](QFunctionPointer fptr) {
auto f = reinterpret_cast<const char * (*)()>(fptr);
return qMakePair<const char *, size_t>(f(), INT_MAX);
};
#else
auto getMetaData = [](QFunctionPointer fptr) {
auto f = reinterpret_cast<QPair<const char *, size_t> (*)()>(fptr);
return f();
};
#endif
QFunctionPointer pfn = priv->resolve("qt_plugin_query_metadata");
if (!pfn)
return false;
auto metaData = getMetaData(pfn);
QJsonDocument doc = qJsonFromRawLibraryMetaData(metaData.first, metaData.second, errMsg);
if (doc.isNull())
return false;
priv->metaData = doc.object();
return true;
}最终获取到的metaData数据为：
QFactoryLoader::QFactoryLoader() looking at "/home/xx/Qt/5.15.2/gcc_64/plugins/mediaservice/libgstmediaplayer.so"
Found metadata in lib /home/xx/Qt/5.15.2/gcc_64/plugins/mediaservice/libgstmediaplayer.so, metadata=
{
"IID": "org.qt-project.qt.mediaserviceproviderfactory/5.0",
"MetaData": {
"Keys": [
"gstreamermediaplayer"
],
"Services": [
"org.qt-project.qt.mediaplayer"
]
},
"archreq": 0,
"className": "QGstreamerPlayerServicePlugin",
"debug": false,
"version": 331520
}
1.1.2.1.2.2 loadMetadata() 加载Metadata到m_metadata中
void QMediaPluginLoader::loadMetadata()
{
if (!m_metadata.isEmpty()) {
return;
}
// QFactoryLoader::metaData() ==> 1.1.2.1.2.2.1
QList<QJsonObject> meta = m_factoryLoader->metaData();
for (int i = 0; i < meta.size(); i++) {
QJsonObject jsonobj = meta.at(i).value(QStringLiteral("MetaData")).toObject();
jsonobj.insert(QStringLiteral("index"), i);
#if !defined QT_NO_DEBUG
if (showDebug)
qDebug() << "QMediaPluginLoader: Inserted index " << i << " into metadata: " << jsonobj;
#endif
QJsonArray arr = jsonobj.value(QStringLiteral("Services")).toArray();
// Preserve compatibility with older plugins (made before 5.1) in which
// services were declared in the 'Keys' property
if (arr.isEmpty())
arr = jsonobj.value(QStringLiteral("Keys")).toArray();
for (const QJsonValue &value : qAsConst(arr)) {
QString key = value.toString();
if (!m_metadata.contains(key)) {
#if !defined QT_NO_DEBUG
if (showDebug)
qDebug() << "QMediaPluginLoader: Inserting new list for key: " << key;
#endif
m_metadata.insert(key, QList<QJsonObject>());
}
m_metadata[key].append(jsonobj);
}
}
}1.1.2.1.2.2.1 m_factoryLoader->metaData()
QList<QJsonObject> QFactoryLoader::metaData() const
{
Q_D(const QFactoryLoader);
QList<QJsonObject> metaData;
#if QT_CONFIG(library)
QMutexLocker locker(&d->mutex);
for (int i = 0; i < d->libraryList.size(); ++i)
metaData.append(d->libraryList.at(i)->metaData);
#endif
const auto staticPlugins = QPluginLoader::staticPlugins();
for (const QStaticPlugin &plugin : staticPlugins) {
const QJsonObject object = plugin.metaData();
if (object.value(QLatin1String("IID")) != QLatin1String(d->iid.constData(), d->iid.size()))
continue;
metaData.append(object);
}
return metaData;
}
1.1.2.2 QMediaPluginLoader::instances(key)
参数key值为"org.qt-project.qt.mediaplayer"，m_metadata.value(key)值为下图：
QList<QObject*> QMediaPluginLoader::instances(QString const &key)
{
if (!m_metadata.contains(key))
return QList<QObject*>();
QList<QString> keys;
QList<QObject *> objects;
const auto list = m_metadata.value(key);
for (const QJsonObject &jsonobj : list) {
int idx = jsonobj.value(QStringLiteral("index")).toDouble();
if (idx < 0)
continue;
// 此处idx为3， ==> 1.1.2.2.1
QObject *object = m_factoryLoader->instance(idx);
if (!objects.contains(object)) {
QJsonArray arr = jsonobj.value(QStringLiteral("Keys")).toArray();
keys.append(!arr.isEmpty() ? arr.at(0).toString() : QStringLiteral(""));
objects.append(object);
}
}
// 运行到此处 keys 值为"gstreamermediaplayer"
// objects值为QGstreamerPlayerServicePlugin对象
static const bool showDebug = qEnvironmentVariableIntValue("QT_DEBUG_PLUGINS");
// preferredPlugins 默认为空
static const QStringList preferredPlugins =
qEnvironmentVariable("QT_MULTIMEDIA_PREFERRED_PLUGINS").split(QLatin1Char(','), Qt::SkipEmptyParts);
for (int i = preferredPlugins.size() - 1; i >= 0; --i) {
auto name = preferredPlugins[i];
bool found = false;
for (int j = 0; j < keys.size(); ++j) {
if (!keys[j].startsWith(name))
continue;
auto obj = objects[j];
objects.removeAt(j);
objects.prepend(obj);
auto k = keys[j];
keys.removeAt(j);
keys.prepend(k);
found = true;
break;
}
}
return objects;
}1.1.2.2.1 m_factoryLoader->instance(idx)
// 此处index为3
QObject *QFactoryLoader::instance(int index) const
{
Q_D(const QFactoryLoader);
if (index < 0)
return nullptr;
#if QT_CONFIG(library)
QMutexLocker lock(&d->mutex);
if (index < d->libraryList.size()) {
// library 为 "plugins/mediaservice/libgstmediaplayer.so"
QLibraryPrivate *library = d->libraryList.at(index);
// library->pluginInstance() 实例化libgstmediaplayer.so，
// 其类型为QGstreamerPlayerServicePlugin
if (QObject *obj = library->pluginInstance()) {
if (!obj->parent())
obj->moveToThread(QCoreApplicationPrivate::mainThread());
return obj;
}
return nullptr;
}
index -= d->libraryList.size();
lock.unlock();
#endif
QVector<QStaticPlugin> staticPlugins = QPluginLoader::staticPlugins();
for (int i = 0; i < staticPlugins.count(); ++i) {
const QJsonObject object = staticPlugins.at(i).metaData();
if (object.value(QLatin1String("IID")) != QLatin1String(d->iid.constData(), d->iid.size()))
continue;
if (index == 0)
return staticPlugins.at(i).instance();
--index;
}
return nullptr;
}
1.1.2.3 plugin->create(key)
QGstreamerPlayerServicePlugin插件其原信息为：Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "mediaplayer.json")
mediaplayer.json在Src/qtmultimedia/src/plugins/gstreamer/mediaplayer下，文件内容为：
{
"Keys": ["gstreamermediaplayer"],
"Services": ["org.qt-project.qt.mediaplayer"]
}// Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "mediaplayer.json")
// key 为 "org.qt-project.qt.mediaplayer"
QMediaService* QGstreamerPlayerServicePlugin::create(const QString &key)
{
// 初始化gsstreamer库？
QGstUtils::initializeGst();
if (key == QLatin1String(Q_MEDIASERVICE_MEDIAPLAYER))
return new QGstreamerPlayerService;
qWarning() << "Gstreamer service plugin: unsupported key:" << key;
return 0;
}1.1.2.3.1 QGstreamerPlayerService
QGstreamerPlayerService::QGstreamerPlayerService(QObject *parent)
: QMediaService(parent)
{
m_session = new QGstreamerPlayerSession(this);
m_control = new QGstreamerPlayerControl(m_session, this);
m_metaData = new QGstreamerMetaDataProvider(m_session, this);
m_streamsControl = new QGstreamerStreamsControl(m_session,this);
m_availabilityControl = new QGStreamerAvailabilityControl(m_control->resources(), this);
m_videoRenderer = new QGstreamerVideoRenderer(this);
m_videoWindow = new QGstreamerVideoWindow(this);
// If the GStreamer video sink is not available, don't provide the video window control since
// it won't work anyway.
if (!m_videoWindow->videoSink()) {
delete m_videoWindow;
m_videoWindow = 0;
}
#if defined(HAVE_WIDGETS)
m_videoWidget = new QGstreamerVideoWidgetControl(this);
// If the GStreamer video sink is not available, don't provide the video widget control since
// it won't work anyway.
// QVideoWidget will fall back to QVideoRendererControl in that case.
if (!m_videoWidget->videoSink()) {
delete m_videoWidget;
m_videoWidget = 0;
}
#endif
}1.1.2.3.1.1 QGstreamerPlayerSession
其位于：qtmultimedia/src/gsttools/qgstreamerplayersession.cpp
#include <gst/gst.h>
QGstreamerPlayerSession::QGstreamerPlayerSession(QObject *parent)
: QObject(parent)
{
initPlaybin();
}从上面可见，其引用了GStreamer头文件。
1.1.2.3.1.2 QGstreamerPlayerControl
其位于：qtmultimedia/src/gsttools/qgstreamerplayercontrol.cpp
QGstreamerPlayerControl::QGstreamerPlayerControl(QGstreamerPlayerSession *session, QObject *parent)
: QMediaPlayerControl(parent)
, m_session(session)
{
m_resources = QMediaResourcePolicy::createResourceSet<QMediaPlayerResourceSetInterface>();
Q_ASSERT(m_resources);
connect(m_session, &QGstreamerPlayerSession::positionChanged, this, &QGstreamerPlayerControl::positionChanged);
connect(m_session, &QGstreamerPlayerSession::durationChanged, this, &QGstreamerPlayerControl::durationChanged);
connect(m_session, &QGstreamerPlayerSession::mutedStateChanged, this, &QGstreamerPlayerControl::mutedChanged);
connect(m_session, &QGstreamerPlayerSession::volumeChanged, this, &QGstreamerPlayerControl::volumeChanged);
connect(m_session, &QGstreamerPlayerSession::stateChanged, this, &QGstreamerPlayerControl::updateSessionState);
connect(m_session, &QGstreamerPlayerSession::bufferingProgressChanged, this, &QGstreamerPlayerControl::setBufferProgress);
connect(m_session, &QGstreamerPlayerSession::playbackFinished, this, &QGstreamerPlayerControl::processEOS);
connect(m_session, &QGstreamerPlayerSession::audioAvailableChanged, this, &QGstreamerPlayerControl::audioAvailableChanged);
connect(m_session, &QGstreamerPlayerSession::videoAvailableChanged, this, &QGstreamerPlayerControl::videoAvailableChanged);
connect(m_session, &QGstreamerPlayerSession::seekableChanged, this, &QGstreamerPlayerControl::seekableChanged);
connect(m_session, &QGstreamerPlayerSession::error, this, &QGstreamerPlayerControl::error);
connect(m_session, &QGstreamerPlayerSession::invalidMedia, this, &QGstreamerPlayerControl::handleInvalidMedia);
connect(m_session, &QGstreamerPlayerSession::playbackRateChanged, this, &QGstreamerPlayerControl::playbackRateChanged);
connect(m_resources, &QMediaPlayerResourceSetInterface::resourcesGranted, this, &QGstreamerPlayerControl::handleResourcesGranted);
//denied signal should be queued to have correct state update process,
//since in playOrPause, when acquire is call on resource set, it may trigger a resourcesDenied signal immediately,
//so handleResourcesDenied should be processed later, otherwise it will be overwritten by state update later in playOrPause.
connect(m_resources, &QMediaPlayerResourceSetInterface::resourcesDenied,
this, &QGstreamerPlayerControl::handleResourcesDenied, Qt::QueuedConnection);
connect(m_resources, &QMediaPlayerResourceSetInterface::resourcesLost, this, &QGstreamerPlayerControl::handleResourcesLost);
}1.1.2.3.1.3 QGstreamerMetaDataProvider
其位于：qtmultimedia/src/plugins/gstreamer/mediaplayer/qgstreamermetadataprovider.cpp
1.1.2.3.1.4 QGstreamerStreamsControl
其位于：qtmultimedia/src/plugins/gstreamer/mediaplayer/qgstreamerstreamscontrol.cpp
1.1.2.3.1.5 QGStreamerAvailabilityControl
其位于：qtmultimedia/src/plugins/gstreamer/mediaplayer/qgstreameravailabilitycontrol.cpp
1.1.2.3.1.6 QGstreamerVideoRenderer
其位于：qtmultimedia/src/gsttools/qgstreamervideorenderer.cpp
1.1.2.3.1.7 QGstreamerVideoWindow
其位于：qtmultimedia/src/gsttools/qgstreamervideowindow.cpp
1.1.2.3.1.8 QGstreamerVideoWidgetControl
其位于：qtmultimedia/src/gsttools/qgstreamervideowidget.cpp
参考
Qt Multimedia::QMediaPlayer框架源码分析
2万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
