http://wangshh03.blog.163.com/blog/static/49103415201282611152692/
ICS4.0.3中播放RTSP流媒体使用的播放器框架是NuPlayer,现在就分析创建NuPlayer的处理流程。
1.创建Nulayer所做的处理
在NuPlayerDriver::NuPlayerDriver()构造函数中:
1.1 创建了ALooper对象mLooper,并调用了 mLooper->start函数,优先级为PRIORITY_AUDIO。
在ALooper::start函数中循环执行ALooper::loop()函数,在loop函数中调用gLooperRoster.deliverMessage函数,
然后在ALooperRoster::deliverMessag函数中调用handler->onMessageReceived函数,最终执行了AHandler的子struct中的onMessageReceived函数,
处理具体消息;例如MyHandler的onMessageReceived函数,用于处理rtsp的消息。
1.2 创建了NuPlayer对象mPlayer,并调用 mLooper->registerHandler,把mPlayer注册到全局变量gLooperRoster的mHandlers成员vector中。
所有的AHandler对象都需要注册到全局变量gLooperRoster的mHandlers成员中。
2. NuPlayer::start处理的流程为以下顺序
2.1 MediaPlayer::start() -> MediaPlayerService::Client::start() ->
2.2 NuPlayerDriver::start()
在NuPlayerDriver::setDataSource中mState被设置为STOPPED状态,所以在start函数中执行了mPlayer->start(),调用到NuPlayer::start()函数
2.3 NuPlayer::start()
发送kWhatStart消息
2.4 NuPlayer::onMessageReceived对kWhatStart消息的处理
2.4.1 接收到kWhatStart消息后,先对一些变量赋初始值,并调用mSource->start()函数,对于rtsp流来说,即调用NuPlayer::RTSPSource::start()函数
2.4.1.1 NuPlayer::RTSPSource::start()
(1)创建了ALooper对象mLooper,并调用了mLooper->setName("rtsp")和mLooper->start()函数。
(2)创建了AHandlerReflector<RTSPSource>对象mReflector,并通过调用 mLooper->registerHandler,把mReflector注册到全局变量gLooperRoster的mHandlers成员中
AHandlerReflector的作用暂时还不清楚,貌似是指定RTSPSource接收消息
(3)创建消息kWhatNotify,并做为MyHandler的参数,在NuPlayer::RTSPSource::onMessageReceived函数中处理了kWhatNotify消息
(4)创建MyHandler对象mHandler,并注册到gLooperRoster的mHandlers成员中
(5)执行mHandler->connect(),链接到服务器,并把mState设置为CONNECTING状态
调用ARTSPConnection的connect函数,进行链接服务器的处理。
2.4.2 新建Renderer对象mRenderer,并发送kWhatRendererNotify消息,把mRenderer注册到全局变量gLooperRoster的mHandlers成员中
Renderer构造函数只是简单的对变量赋默认值,并没有做其他处理
2.4.3 调用postScanSources()函数,进行初始化audio/video的解码器decoder
(1)NuPlayer::postScanSources()函数中发送了kWhatScanSources消息
2.5 NuPlayer::onMessageReceived对kWhatScanSources消息的处理
2.5.1 调用instantiateDecoder函数执行初始化video和audio解码器
[cpp] view plaincopy
- instantiateDecoder(false, &mVideoDecoder);
- if (mAudioSink != NULL) {
- instantiateDecoder(true, &mAudioDecoder);
- }
2.5.2 NuPlayer::instantiateDecoder
(1)当初始化audio解码器时,发送kWhatAudioNotify消息
当初始化video解码器时,发送kWhatVideoNotify消息
(2)初始解码器Decoder对象decoder,并注册到全局变量gLooperRoster的mHandlers成员中
[cpp] view plaincopy
- *decoder = audio ? new Decoder(notify) :
- new Decoder(notify, mNativeWindow); // video时,设置NativeWindowWrapper参数
- looper()->registerHandler(*decoder);
- (*decoder)->configure(meta); // 调用decoder的configure方法,初始化ACodec解码器
2.5.3 NuPlayer::Decoder::configure
(1)创建消息kWhatCodecNotify,用于ACodec对象mCodec的通知消息
(2)调用makeFormat函数,根据meta类型,创建对应的message format
(3)创建ACodec对象mCodec,
(4)video解码时,创建ALooper对象mCodecLooper,用于释放主消息event队列,并注册到全局变量gLooperRoster的mHandlers成员中
mCodecLooper的name设置为"NuPlayerDecoder"
2.6 NuPlayer::onMessageReceived
case kWhatVideoNotify:
case kWhatAudioNotify:
取得AMessage消息"codec-request"对应的"what"值,分别对以下情况处理
2.6.1 ACodec::kWhatFillThisBuffer
执行feedDecoderInputData函数,取得一个有效的buffer
2.6.2 ACodec::kWhatEOS
向render发送EOS消息
mRenderer->queueEOS(audio, err);
2.6.3 ACodec::kWhatFlushCompleted
(1)把mFlushingAudio或mFlushingVideo设置为FLUSHED或者SHUTTING_DOWN_DECODER状态
(2)调用finishFlushIfPossible()函数,执行一些flush后的操作,例如:
mRenderer->signalTimeDiscontinuity();
mAudioDecoder->signalResume();
mVideoDecoder->signalResume();
finishReset();
如果mAudioDecoder == NULL || mVideoDecoder == NULL,则会执行postScanSources()函数,进行初始化audio/video的解码器decoder
2.6.4 ACodec::kWhatOutputFormatChanged
2.6.5 ACodec::kWhatShutdownCompleted
2.6.6 ACodec::kWhatError
2.6.7 ACodec::kWhatDrainThisBuffer //在ACodec::BaseState::onOMXFillBufferDone中发送的
调用renderBuffer(audio, codecRequest), 把需要解码的数据放入render队列
2.7 NuPlayer::renderBuffer
先做一些检查,丢弃需要跳过的数据
调用mRenderer->queueBuffer(audio, buffer, reply);,把真正需要解码的数据放入renderer队列
2.8 NuPlayer::Renderer::queueBuffer
发送kWhatQueueBuffer消息
[cpp] view plaincopy
- sp<AMessage> msg = new AMessage(kWhatQueueBuffer, id());
- msg->setInt32("audio", static_cast<int32_t>(audio));
- msg->setObject("buffer", buffer);
- msg->setMessage("notifyConsumed", notifyConsumed);
- msg->post();
2.9 NuPlayer::Renderer::onMessageReceived
case kWhatQueueBuffer:
--->
NuPlayer::Renderer::onQueueBuffer
在video时,调用postDrainVideoQueue函数
--->
NuPlayer::Renderer::postDrainVideoQueue()
发送kWhatDrainVideoQueue消息,消息延迟时间为delayUs。
+++++++++++++++变量注解+++++++++++++++
[cpp] view plaincopy
- mAnchorTimeMediaUs表示audio的媒体时间mediaTimeUs,初始值为-1
- mAnchorTimeMediaUs = mediaTimeUs;
- int64_t realTimeOffsetUs =
- (mAudioSink->latency() / 2 /* XXX */
- + numFramesPendingPlayout
- * mAudioSink->msecsPerFrame()) * 1000ll;
- // LOGI("realTimeOffsetUs = %lld us", realTimeOffsetUs);
- mAnchorTimeRealUs =
- ALooper::GetNowUs() + realTimeOffsetUs;
- mAnchorTimeRealUs表示audio的已经处理完(包括已经播放和正要播放)的数据的时间。如果没有audio,则为当前系统时间
- mAnchorTimeRealUs = ALooper::GetNowUs();
++++++++++++++++++++++++++++++
--->
NuPlayer::Renderer::onMessageReceived
case kWhatDrainVideoQueue:
调用onDrainVideoQueue()处理
继续调用postDrainVideoQueue函数,循环处理下一个buffer
--->
[cpp] view plaincopy
- NuPlayer::Renderer::onDrainVideoQueue()
- // 发送消息表示此buffer已经处理
- entry->mNotifyConsumed->setInt32("render", !tooLate);
- entry->mNotifyConsumed->post();
- mVideoQueue.erase(mVideoQueue.begin());
- entry = NULL;
- notifyPosition();
--------------------------------------------------------------------------------------------------
**************** 发送kWhatDrainThisBuffer消息的处理流程 Start ****************
NuPlayer::instantiateDecoder
创建Decoder对象,然后调用
(*decoder)->configure(meta);
--->
NuPlayer::Decoder::configure
调用mCodec->initiateSetup(format);
--->
ACodec::initiateSetup
发送kWhatSetup消息
--->
ACodec::UninitializedState::onMessageReceived
case ACodec::kWhatSetup:
调用onSetup(msg);
--->
[cpp] view plaincopy
- ACodec::UninitializedState::onSetup
- status_t err = omx->allocateNode(componentName.c_str(), observer, &node);
--->
OMX::allocateNode
创建了OMXNodeInstance对象,并注册了OMXNodeInstance::kCallbacks回调
--->
[cpp] view plaincopy
- // static
- OMX_CALLBACKTYPE OMXNodeInstance::kCallbacks = {
- &OnEvent, &OnEmptyBufferDone, &OnFillBufferDone // 注册了回调函数OnFillBufferDone
- };
--->
[cpp] view plaincopy
- OMX_ERRORTYPE OMXNodeInstance::OnFillBufferDone(
- OMX_IN OMX_HANDLETYPE hComponent,
- OMX_IN OMX_PTR pAppData,
- OMX_IN OMX_BUFFERHEADERTYPE* pBuffer) {
- OMXNodeInstance *instance = static_cast<OMXNodeInstance *>(pAppData);
- if (instance->mDying) {
- return OMX_ErrorNone;
- }
- return instance->owner()->OnFillBufferDone(instance->nodeID(), pBuffer); // OMXNodeInstance含有OMX成员变量OMX *mOwner;
- }
--->
OMX::OnFillBufferDone
设置msg.type = omx_message::FILL_BUFFER_DONE;并发送
--->
[cpp] view plaincopy
- ACodec::BaseState::onOMXMessage
- case omx_message::FILL_BUFFER_DONE:
- return onOMXFillBufferDone(
- bufferID,
- (size_t)rangeOffset, (size_t)rangeLength,
- (OMX_U32)flags,
- timeUs,
- platformPrivate,
- dataPtr);
--->
[cpp] view plaincopy
- ACodec::BaseState::onOMXFillBufferDone
- PortMode mode = getPortMode(kPortIndexOutput);
- switch (mode) {
- case KEEP_BUFFERS:
- break;
- case RESUBMIT_BUFFERS:
- 设置发送消息notify->setInt32("what", ACodec::kWhatDrainThisBuffer);
**************** 发送kWhatDrainThisBuffer消息的处理流程 End ****************
NuPlayer流媒体播放器中从网络上取得的rtp包在解码和显示时所用buffer的填充和清空的机制,与stagefright框架播放本地视频时的处理流程类似。
都是通过回调函数fillbuffer及emptybuffer来实现的。
NuPlayer中的ACodec与Stagefright中的OMXCodec的作用相似。
以下是rtsp流媒体中填充待解码的buffer,以及把解码后的buffer送给显示并清空的处理过程:
1.ACodec::UninitializedState::onSetup
调用status_t err = omx->allocateNode(componentName.c_str(), observer, &node);
2.OMX::allocateNode
[cpp] view plaincopy
- status_t OMX::allocateNode(
- const char *name, const sp<IOMXObserver> &observer, node_id *node) {
- Mutex::Autolock autoLock(mLock);
- *node = 0;
- OMXNodeInstance *instance = new OMXNodeInstance(this, observer); // OMXNodeInstance是在OMX类中创建的
- OMX_COMPONENTTYPE *handle;
- OMX_ERRORTYPE err = mMaster->makeComponentInstance(
- name, &OMXNodeInstance::kCallbacks, // 注册了kCallbacks回调函数,
- instance, &handle);
3.OMXNodeInstance::kCallbacks
[cpp] view plaincopy
- // static
- OMX_CALLBACKTYPE OMXNodeInstance::kCallbacks = {
- &OnEvent, &OnEmptyBufferDone, &OnFillBufferDone // 回调函数OnEmptyBufferDone,OnFillBufferDone
- };
4.OMXNodeInstance::OnEmptyBufferDone // 数据解码完,可以取走数据用于显示了,然后清空buffer
调用instance->owner()->OnEmptyBufferDone(instance->nodeID(), pBuffer) // owner()返回的是OMX指针对象mOwner
处理过程如下:
(1)在OMX::OnEmptyBufferDone中发送omx_message::EMPTY_BUFFER_DONE消息,并设置了解码后的buffer
(2)首先ACodec.cpp文件中的CodecObserver结构体的onMessage方法接收到omx_message::EMPTY_BUFFER_DONE消息。
(3)然后ACodec::BaseState::onOMXMessage再接收到omx_message::EMPTY_BUFFER_DONE消息,调用onOMXEmptyBufferDone继续处理。
(4)在ACodec::BaseState::onOMXEmptyBufferDone 函数中,PortMode为的值为RESUBMIT_BUFFERS,则调用postFillThisBuffer函数
(5)在ACodec::BaseState::postFillThisBuffer 函数中,发送了kWhatInputBufferFilled消息,设置"what"参数为ACodec::kWhatFillThisBuffer
(6)接收到kWhatInputBufferFilled消息后,调用onInputBufferFilled 函数
(7)在ACodec::BaseState::onInputBufferFilled 函数中
1)PortMode为的值为RESUBMIT_BUFFERS,并且buffer != info->mData,则执行内存拷贝,把解码后的数据从buffer->data()拷贝到info->mData->data()中
memcpy(info->mData->data(), buffer->data(), buffer->size());
2)然后调用mCodec->mOMX->emptyBuffer函数,最终调用的是OMXNodeInstance::emptyBuffer 函数
3)再调用getMoreInputDataIfPossible函数,取得下一个解码完的数据。但在执行到eligible == NULL时,执行了返回动作,没有调用postFillThisBuffer(eligible)函数继续处理。
5.OMXNodeInstance::OnFillBufferDone // 数据已经准备好,可以送给解码器解码
调用instance->owner()->OnFillBufferDone(instance->nodeID(), pBuffer)
---------------------------------------------------------------------------------
在OMX::allocateNode函数中,创建了CallbackDispatcher对象,即
在CallbackDispatcher构造函数中创建了CallbackDispatcherThread对象,并调用了run函数,在run中会调用Thread::_threadLoop,又调用threadLoop
->
bool OMX::CallbackDispatcherThread::threadLoop() {
return mDispatcher->loop();
}
->
OMX::CallbackDispatcher::loop()
->
OMX::CallbackDispatcher::dispatch
->
OMXNodeInstance::onMessage
mObserver->onMessage(msg);// IOMXObserver是构造函数OMXNodeInstance中传递过来的,即OMX::allocateNode中传递的参数observer,
// 而OMX::allocateNode又是在ACodec::UninitializedState::onSetup函数中被调用的,即observer的类型是CodecObserver。
CodecObserver继承自BnOMXObserver类,而BnOMXObserver又是IOMXObserver的子类。
所以,OMX::CallbackDispatcher::dispatch中执行mOwner->onMessage(msg),
最终是调用的CodecObserver的onMessage方法,在CodecObserver的onMessage方法中发送ACodec::kWhatOMXMessage消息。