上文我们介绍到通过不同的playerType创建不同的player。
StagefrightPlayer:
sp<MediaPlayerBase> MediaPlayerService::Client::createPlayer(player_type playerType)
{
// determine if we have the right player type
sp<MediaPlayerBase> p = mPlayer;
if ((p != NULL) && (p->playerType() != playerType)) {
ALOGV("delete player");
p.clear();
}
if (p == NULL) {
p = MediaPlayerFactory::createPlayer(playerType, this, notify, mPid);
}
if (p != NULL) {
p->setUID(mUID);
}
return p;
}
virtual sp<MediaPlayerBase> createPlayer(pid_t /* pid */) {
ALOGV(" create StagefrightPlayer");
return new StagefrightPlayer();
}
当我们打开StagefrightPlayer,发现其只是一个空壳:
kernel\android\marshmallow\frameworks\av\media\libmediaplayerservice\StagefrightPlayer.cpp
status_t StagefrightPlayer::setDataSource(
const char *url, const KeyedVector<String8, String8> *headers) {
return mPlayer->setDataSource(url, headers);
}
status_t StagefrightPlayer::prepare() {
return mPlayer->prepare();
}
status_t StagefrightPlayer::start() {
ALOGV("start");
return mPlayer->play();
}
status_t StagefrightPlayer::stop() {
ALOGV("stop");
return pause(); // what's the difference?
}
status_t StagefrightPlayer::pause() {
ALOGV("pause");
return mPlayer->pause();
}
实际起作用的是AwesomePlayer *mPlayer;
AwesomePlayer:
视频处理过程中有很多都是十分耗时的,如果都放在一个大的线程空间中。用户体验的效果可想而知。所以通常都是做异步操作。
AwesomePlayer是通过event事件调度来实现这些功能之间的驱动和调用的。
AwesomePlayer中的内部变量:
TimedEventQueue mQueue;
这个mQueue就是AwesomePlayer的事件队列,也是事件调度器。从他类型的名字上就能很清楚的看出他是以时间为基础事件队列。接下来看看它是怎么玩转的。
1.先来看TimedEventQueue的内部结构,TimedEventQueue内部有一个 List<QueueItem>,每个QueueItem包含enent和时间:
struct QueueItem {
sp<Event> event;
int64_t realtime_us;
};
有一个独立线程threadEntry是在TimedEventQueue::start被创建,TimedEventQueue::stop被销毁的。
void TimedEventQueue::start() {
if (mRunning) {
return;
}
mStopped = false;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_create(&mThread, &attr, ThreadWrapper, this);
pthread_attr_destroy(&attr);
mRunning = true;
}
void TimedEventQueue::stop(bool flush) {
if (!mRunning) {
return;
}
if (flush) {
postEventToBack(new StopEvent);
} else {
postTimedEvent(new StopEvent, INT64_MIN);
}
void *dummy;
pthread_join(mThread, &dummy);
mQueue.clear();
mRunning = false;
}
2.List<QueueItem>目的就是按照延时时间维护一个event事件队列,threadEntry线程就是不断的从队列的头取出一个event,然后通过 event->fire(this, now_us); 回调到这个event事件提前注册好的相对应功能函数。
void TimedEventQueue::threadEntry() {
prctl(PR_SET_NAME, (unsigned long)"TimedEventQueue", 0, 0, 0);
for (;;) {
int64_t now_us = 0;
sp<Event> event;
bool wakeLocked = false;
{
Mutex::Autolock autoLock(mLock);
if (mStopped) {
break;
}
while (mQueue.empty()) {
mQueueNotEmptyCondition.wait(mLock);
}
event_id eventID = 0;
for (;;) {
if (mQueue.empty()) {
// The only event in the queue could have been cancelled
// while we were waiting for its scheduled time.
break;
}
List<QueueItem>::iterator it = mQueue.begin();
eventID = (*it).event->eventID();
now_us = ALooper::GetNowUs();
int64_t when_us = (*it).realtime_us;
int64_t delay_us;
if (when_us < 0 || when_us == INT64_MAX) {
delay_us = 0;
} else {
delay_us = when_us - now_us;
}
if (delay_us <= 0) {
break;
}
static int64_t kMaxTimeoutUs = 10000000ll; // 10 secs
bool timeoutCapped = false;
if (delay_us > kMaxTimeoutUs) {
ALOGW("delay_us exceeds max timeout: %" PRId64 " us", delay_us);
// We'll never block for more than 10 secs, instead
// we will split up the full timeout into chunks of
// 10 secs at a time. This will also avoid overflow
// when converting from us to ns.
delay_us = kMaxTimeoutUs;
timeoutCapped = true;
}
status_t err = mQueueHeadChangedCondition.waitRelative(
mLock, delay_us * 1000ll);
if (!timeoutCapped && err == -ETIMEDOUT) {
// We finally hit the time this event is supposed to
// trigger.
now_us = ALooper::GetNowUs();
break;
}
}
// The event w/ this id may have been cancelled while we're
// waiting for its trigger-time, in that case
// removeEventFromQueue_l will return NULL.
// Otherwise, the QueueItem will be removed
// from the queue and the referenced event returned.
event = removeEventFromQueue_l(eventID, &wakeLocked);
}
if (event != NULL) {
// Fire event with the lock NOT held.
event->fire(this, now_us);
if (wakeLocked) {
Mutex::Autolock autoLock(mLock);
releaseWakeLock_l();
}
}
}
}
3.然后看看AwesomePlayer是怎么用TimedEventQueue,AwesomePlayer会定义很多类型的event事件,并把和这些事件相关的功能函数一定绑定起来。
mVideoEvent = new AwesomeEvent(this, &AwesomePlayer::onVideoEvent);
mVideoEventPending = false;
mStreamDoneEvent = new AwesomeEvent(this, &AwesomePlayer::onStreamDone);
mStreamDoneEventPending = false;
mBufferingEvent = new AwesomeEvent(this, &AwesomePlayer::onBufferingUpdate);
mBufferingEventPending = false;
mVideoLagEvent = new AwesomeEvent(this, &AwesomePlayer::onVideoLagUpdate);
mVideoEventPending = false;
mCheckAudioStatusEvent = new AwesomeEvent(this, &AwesomePlayer::onCheckAudioStatus);
原因之前也说了,因为好多音视频处理的功能是十分耗时间的,假如AwesomePlayer 想用某个功能,他并不是直线去调用它,而是抽象成一种AwesomeEvent,将想要调用的功能函数与事件捆绑。通过TimedEventQueue::postTimedEvent(),按照延时的优先顺序把它放到TimedEventQueue的队列之中。然后AwesomePlayer就不管了。TimedEventQueue start之后,自己内部的线程会从队列中依次取出这些事件,然后通过event->fire回调事件的功能函数。这样就达到了AwesomePlayer的目的。
4.之前也介绍过mediaPlayer大致流程就是
mediaPlayer.setDataSource(path);
mediaPlayer.prepare();
mediaPlayer.start();
在AwesomePlayer 也是这种流程,在AwesomePlayer prepare()相关函数中。
status_t AwesomePlayer::prepareAsync_l() {
if (mFlags & PREPARING) {
return UNKNOWN_ERROR; // async prepare already pending
}
if (!mQueueStarted) {
mQueue.start();
mQueueStarted = true;
}
modifyFlags(PREPARING, SET);
mAsyncPrepareEvent = new AwesomeEvent(
this, &AwesomePlayer::onPrepareAsyncEvent);
mQueue.postEvent(mAsyncPrepareEvent);
return OK;
}
并没有实际的调用onPrepareAsyncEvent()真正的功能函数,他只是把mQueue start之后,然后创建个mAsyncPrepareEvent事件,把它插入到mQueue之中就不管了,具体调用是由mQueue中的threadEntry线程来做。
1.通过setDataSource 指定播放器的数据源。可以是URI或者fd.可以是http:// 、rtsp://、本地地址或者本地文件描述符fd。其最终调用是将上层传递来的参数转化为DataSource,为下一步的demux提供数据支持。
2.在真正Prepare功能函数onPrepareAsyncEvent()会调用finishSetDataSource_l。通过第一步产生的DataSource来生成extractor,因为封装的格式很多,所以需要通过DataSource的信息,去创建不同的extractor。
kernel\android\marshmallow\frameworks\av\media\libstagefright\MediaExtractor.cpp
extractor = MediaExtractor::Create(
dataSource, sniffedMIME.empty() ? NULL : sniffedMIME.c_str());
if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG4)
|| !strcasecmp(mime, "audio/mp4")) {
ret = new MPEG4Extractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_MPEG)) {
ret = new MP3Extractor(source, meta);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AMR_NB)
|| !strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AMR_WB)) {
ret = new AMRExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_FLAC)) {
ret = new FLACExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_WAV)) {
ret = new WAVExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_OGG)) {
ret = new OggExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MATROSKA)) {
ret = new MatroskaExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG2TS)) {
ret = new MPEG2TSExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_WVM)) {
// Return now. WVExtractor should not have the DrmFlag set in the block below.
return new WVMExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AAC_ADTS)) {
ret = new AACExtractor(source, meta);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG2PS)) {
ret = new MPEG2PSExtractor(source);
}
3.得到extractor之后,通过setVideoSource() setAudioSource()产生独立的mVideoTrack(视频)、mAudioTrack(音频)数据流,分别为音视频解码器提供有各自需要的数据流。
其实extractor和mVideoTrack、mAudioTrack就组成了播放器模型中的demuxer部分。把封装格式里面的音视频流拆分出来,分别的送给音视频解码器。
void AwesomePlayer::setVideoSource(sp<MediaSource> source) {
CHECK(source != NULL);
mVideoTrack = source;
}
void AwesomePlayer::setAudioSource(sp<MediaSource> source) {
CHECK(source != NULL);
mAudioTrack = source;
}
4.接下来就是initVideoDecoder() initAudioDecoder().依赖上面产生的mVideoTrack(视频)、mAudioTrack(音频)数据流。生成了mVideoSource和mAudioSource这两个音视频解码器。不同类型匹配不同的解码器。
mVideoSource = OMXCodec::Create(
mClient.interface(), mVideoTrack->getFormat(),
false, // createEncoder
mVideoTrack,
NULL, flags, USE_SURFACE_ALLOC ? mNativeWindow : NULL);
mAudioSource = OMXCodec::Create(
mClient.interface(), mAudioTrack->getFormat(),
false, // createEncoder
mAudioTrack);
}
mVideoSource、mAudioSource组成了播放器模型中的decoder部分。
android系统中的编解码器部分用的是openmax,以后会深入了解。openma x是一套标准接口,各家硬件厂商都可以遵循这个标准来做自己的实现,发挥自己芯片特性。然后提供给android系统来用。因为大部分的机顶盒芯片产品硬件的编解码是它的优势,可以把这种优势完全融入到android平台中。以后手机高清视频硬解码也会是个趋势。
5.解码完之后的数据就要输出了。AwesomePlayer分别用了mVideoRenderer做视频输出、mAudioPlayer做音频输出。他们分别调用android图像和音频的相关服务。这俩部分是android平台中十分重要的2块,以后会深入了解。
mVideoRenderer和mAudioPlayer就组成了播放器中output的部分。
OMX:待续