Android Video System[2] --- MediaExtractor, Decoder and Renderer Source Code Analysis

At the beginning, Let’s take a look at the catalogue of this chapter：

（一）、多媒体文件解析 - MediaExtractor分离音视频
1.1、mMediaPlayer.prepareAsync()
1.1.1、MediaPlayer.setVideoSurfaceTexture()
1.1.2、MediaPlayer.prepareAsync()
1.1.3、GenericSource.initFromDataSource()
1.1.4、MediaExtractor::Create()
1.1.5、IMediaExtractor->getTrack()
（二）、多媒体文件 - 音视频解码（Decoder）
2.1、NuPlayer::start()
2.2、postScanSources()
2.3、instantiateDecoder()
2.3.1、创建音视频解码器new Decoder()
2.3.2、初始化Decoder->init()
2.3.3、配置Decoder->configure()
2.3.4、MediaCodec::init(…)
2.3.4.1、GetCodecBase
2.3.4.2、MediaCodecList::getInstance()
2.3.5、MediaCodec->configure()
2.3.6、MediaCodec->start()
（三）、音视频解码数据处理
3.1、音视频解码数据处理-emptyBuffer
3.1.1、ACodec::ExecutingState::resume()
3.1.1.1、kWhatFillThisBuffer消息处理
3.1.1.2、kWhatInputBufferFilled消息处理
3.2、音视频解码数据处理-fillBuffer
3.2.1、ACodec::ExecutingState::resume()
（四）、多媒体文件 - 音视频渲染（Renderer）
（五）、视频解码输出到SurfaceFlinger
5.1、Surfaceflinger 视频解码缓存申请
5.1.1、allocateOutputBuffersFromNativeWindow()的实现
5.1.1.1、首先为视频编码输出准备Surface
5.1.1.2、Surface->dequeueBuffer
5.2、Surface->queueBuffer()
（六）、参考资料(特别感谢各位前辈的分析和图示)：

code path:

V4l2 Driver Code
☯ kernel/drivers/media/v4l2-core/（File prefix with videobuf2）

☯ MSM  video driver Code
☯ kernel/drivers/media/platform/msm/vidc/

☯ device-tree#include <binder/IPCThreadState.h>
#include <gui/BufferItem.h>
#include <gui/BufferQueueCore.h>
#include <gui/BufferQueueProducer.h>
#include <gui/GLConsumer.h>
#include <gui/IConsumerListener.h>
#include <gui/IProducerListener.h>

☯ /kernel/arch/arm/boot/dts/qcom（Venus Rgister base adress，Clock frequency）

☯ Stagefright、libmedia、libmediaplayerservice、mediaserver
☯ /frameworks/av/media/

☯ OMX
☯ /hardware/qcom/media/mam8996/mm-video-v4l2/vidc/

☯ OMX Core
☯ /hardware/qcom/media/mm-core

☯ Software codec path
☯ /vendor/qcom/proprietary/mm-video/omx_vpp(?) →  Decoder code
☯ /vendor/qcom/proprietary/mm-video/omx_vpp(?) →  Encoder code

Preface Question

Before analysing the code, we put forward several questions:
(1) How to detect and parse multimedia files in different formats? Where is the audio and video data buffer? (Source)
(2) Where are audio decoding threads and video decoding threads? (DecoderBase)
(3) How does the video display? How does the Audio play? where is audio and Video sychronization? (Renderer)

Chapter 1: Multimedia file Resolution — MediaExtractor separation of Audio and Video

1.1、mMediaPlayer.prepareAsync()

[->\frameworks\base\media\java\android\media\MediaPlayer.java]
public native void prepareAsync() throws IllegalStateException;

（Second）、MultiMedia - Audio and Video Decoding（Decoder）

print display log：
/frameworks/native/libs/gui/BufferQueueProducer.cpp

--- a/libs/gui/BufferQueueProducer.cpp
+++ b/libs/gui/BufferQueueProducer.cpp
@@ -18,7 +18,7 @@
 
 #define LOG_TAG "BufferQueueProducer"
 #define ATRACE_TAG ATRACE_TAG_GRAPHICS
-//#define LOG_NDEBUG 0
+#define LOG_NDEBUG 0  //ciellee
 
 #if DEBUG_ONLY_CODE

waiting …