前言:
正文
一:SoftAAC2解码流程
01-07 01:39:14.250 E/SoftAAC2( 151): in SoftAAC2 constructor....
首先会进入SoftAAC2的构造函数
01-07 01:39:14.250 E/SoftAAC2( 151): in SoftAAC2 initPorts....
为这个解码器初始化两个端口(输入和输出,portIndex分别为0和1),看下port对应的结构体:
port对应的结构体中包含了三部分的内容
1:关于port所有信息的结构体OMX_PARAM_PORTDEFINITIONTYPE,定义在omx_core.h标准中,需要好好看下;
2:每个port都会对应分配几块buffer(对应的结构体为bufferInfo,后面详解),用于放置解码前后的数据
3:port也维护了一个状态
初始化过程就是初始化OMX_PARAM_PORTDEFINITIONTYPEdef变量,最后调用addPort加入到一个容器中去管理
void SimpleSoftOMXComponent::addPort(const OMX_PARAM_PORTDEFINITIONTYPE &def) {
}
01-07 01:39:14.250 E/SoftAAC2( 151): in SoftAAC2 initDecoder before call aacDecoder_open...
01-07 01:39:14.250 E/SoftAAC2( 151): in SoftAAC2 initDecoder before call aacDecoder_GetStreamInfo...
其次是在initDecoder函数初始化解码器,调用aacDecoder_Open接口获得解码库的句柄mAACDecoder,以及调用aacDecoder_GetStreamInfo获取aac的一些信息,保存在结构体CStreamInfo中。
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocating 4 buffers of size 8192 on input port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x40069f48 on input port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x401fd3b8 on input port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x400a7008 on input port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x400a70c0 on input port
为输入端口分配四个buffer(个数是在初始化port过程中kNumInputBuffers指定的),这四个buffer的作用就是让omx client往里面放原始的aac数据,并且调用emptyBuffer,告诉解码器,将这些buffer拿去解码,这些buffer都是通过共享内存实现的,也就是omx client和omx component共享这段内存,client往里面写,component读取里面的数据,看下buffer对应的数据结构:
包含了两部分内容:
1:buffer中包含的数据的结构信息OMX_BUFFERHEADERTYPE,定义在openmax头文件中;
2:buffer当前被谁占有,client(mOwnedByUs为false)或者component(mOwnedByUs为true)。
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocating 4 buffers of size 16384 on output port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x401fe6a8 on output port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x404d09c8 on output port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x408bedd8 on output port
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent useBufferWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent useBuffer....
01-07 01:39:14.270 V/OMXCodec( 151): [OMX.google.aac.decoder] allocated buffer 0x408bef00 on output port
同理,为输出端口分配四块buffer,用于client和component共享,这四个buffer用于存放解码器解码后的pcm数据,解码器往里面写,omx client读取里面的数据,其他和input port对应的buffer一致。
上面八个buffer都是通过android中Shared Memory机制实现,内存是在OMXCodec中分配的
01-07 01:39:14.270 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent GetParameterWrapper..
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent getParameter ....
01-07 01:39:14.270 E/SoftAAC2( 151): huangbangbang in SoftAAC2 internalGetParameter index is 33554433
01-07 01:39:14.270 E/SoftAAC2( 151): huangbangbang in SoftAAC2 internalGetParameter in default...
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent internalGetParameter....
01-07 01:39:14.270 E/SimpleSoftOMXComponent( 151): huangbangbang in internalGetParameter in case OMX_IndexParamPortDefinition ....
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent msgType = 0 (kWhatSendCommand: 0, kWhatEmptyFillThisBuffer : 1, kWhatFillThisBuffer:2)
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onSendcommand.
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onSendcommand in onChangeState.
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onChangeStage state=2, component_cur_state=1
01-07 01:39:14.280 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent notify..
01-07 01:39:14.280 V/OMXCodec( 151): [OMX.google.aac.decoder] onStateChange 2
01-07 01:39:14.280 V/OMXCodec( 151): [OMX.google.aac.decoder] Now Idle.
01-07 01:39:14.280 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent SendCommandWrapper..
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent sendCommand....
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent msgType = 0 (kWhatSendCommand: 0, kWhatEmptyFillThisBuffer : 1, kWhatFillThisBuffer:2)
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onSendcommand.
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onSendcommand in onChangeState.
01-07 01:39:14.280 E/SimpleSoftOMXComponent( 151): huangbangbang in SimpleSoftOMXComponent onChangeStage state=3, component_cur_state=2
01-07 01:39:14.280 E/SoftOMXComponent( 151): huangbangbang in SoftOMXcomponent notify..
01-07 01:39:14.280 V/OMXCodec( 151): [OMX.google.aac.decoder] onStateChange 3
01-07 01:39:14.280 V/OMXCodec( 151): [OMX.google.aac.decoder] Now Executing.
上面的log表示,client调用getParameter和setParameter获取或者设置解码器的一些参数,changeState改变解码器当前状态等函数,解码器本身也维护了一个状态机,每个状态下面需要或者不能做哪些事情是有规定的,可以在openMAX文档中查看,如下图,
从log中可以看出,我们的aac解码器状态机的变化是:
UNLOADED------->LOADED------->IDLE--------->EXECUTING
到达EXECUTING状态之后,就能够开始读写数据编解码了
谷歌实现的软解码的逻辑类采用了多态机制,因为谷歌继承的不只是aac decoder一个component,还有很多其他类型文件的解码器,所以为了达到代码的最大共用,谷歌实现先了两个父类,对于和具体解码器无关的一些操作都放在父类中完成,继承关系如下:
01-07 01:39:14.460 E/SoftAAC2( 151): in SoftAAC2 onQueueFilled.....
01-07 01:39:14.460 E/SoftAAC2( 151): in SoftAAC2 onQueueFilled in portIndex is 0 mInputBufferCount == 0.....
01-07 01:39:14.460 E/SoftAAC2( 151): in SoftAAC2 onQueueFilled in portIndex is 0 before call aacDecoder_ConfigRaw.....
解码器,输入输出端口,每个端口需要的buffer都分配好了,下面就是真正调用解码库进行数据的解码了,真正音频数据前面都会有一些configure数据,所以来到的第一个input buffer我们没有去调用aacDecoder_DecodeFrame函数直接去解码,而是调用aacDecoder_ConfigRaw 去重新获取aac的一些信息,如sampleRate和numChannels,保存在CStreamInfo结构体中, 如果sampleRate和numChannels参数有变化,可能还需要disable output port,并且重新初始化output port和对应的四个buffer,对于CStreamInfo结构体将在第二部分讲解解码库的时候进行
01-07 01:39:14.460 E/SoftAAC2( 151): in SoftAAC2 onQueueFilled.....
01-07 01:39:14.460 E/SoftAAC2( 151): in SoftAAC2 onQueueFilled in while looper.....
01-07 01:39:14.460 E/SoftAAC2( 151): in onQueueFilled in while before call aacDecoder_Fill and aacDecoder_DecodeFrame bytesvalid is 416
所有准备工作都做好了之后就开始真正aac数据的解码了,解码都是在onQueueFilled函数中进行,剔除一些参数的初始化,主要过程就是下面的while循环:
要明白while循环里面aacDecoder_Fill和aacDecoder_DecodeFrame的作用,需要明白解码库的buffer机制:
解码过程中除了前面已经分配的input buffer和output buffer,解码库还会分配一个过渡性的decoder-internal input buffer,这个buffer大小又RANSPORTDEC_INBUF_SIZE规定,可以任意设定但必须满足两个条件:
1:each input channel requires 768 bytes
2:the whole buffer must be of size 2^n
So for example a stereo decoder:
aacDecoder_Fill就是从input buffer往ecoder-internal input buffer里面拷贝数据,返回的是input buffer中还剩下多少没有被拷贝的数据,如果input buffer中的数据全部拷贝完毕了,就需要re-fill,下图是input buffer的变化图(from aacDecoder.pdf)
aacDecoder_DecodeFrame用来解码internal buffer中的数据,如果数据不足以解码,则返回AAC_DEC_NOT_ENOUGH_BITS,继续调用aacDecoder_Fill填充数据。
这样解码一直进行下去,知道OMXCodec中读取aac原始数据过程中发现已经到该track的结尾,就会给这个mediabuffer打上一个标签,在onQueueFilled函数中检测的这个flag,解码完这一帧之后,就停止解码,并且做所有的release操作
OMXCodec::drainInputBuffer
SoftAAC2::onQueueFilled
最后就是free所有的buffer,disable端口等
二:解码库学习 Advanced Audio Coding DecoderLibrary
1,术语:
MPEG-2 and MPEG-4
AAC Low-Complexity (AAC-LC),
High-Eficiency AAC v2 (HE-AAC v2),
AAC Low-Delay (AAC-LD), andAAC Enhanced Low-Delay (AAC-ELD) decoder
2,主要头文件:#include
3,常用的数据和数据结构:
struct CStreamInfo
• INT sampleRate
• INT frameSize
• INT numChannels
• AUDIO_CHANNEL_TYPE pChannelType
• UCHAR pChannelIndices
• INT aacSampleRate
• INT profile
• AUDIO_OBJECT_TYPE aot
• INT channelConfig
• INT bitRate
• INT aacSamplesPerFrame
• AUDIO_OBJECT_TYPE extAot
• INT extSamplingRate
• UINT flags
• SCHAR epConfig
• INT numLostAccessUnits
• UINT numTotalBytes
• UINT numBadBytes
• UINT numTotalAccessUnits
• UINT numBadAccessUnits
Defines:
• #define IS_INIT_ERROR(err) ( (((err)>=aac_dec_init_error_start) && ((err)<=aac_dec_init_-
error_end)) ? 1 : 0)
• #define IS_DECODE_ERROR(err) ( (((err)>=aac_dec_decode_error_start) && ((err)<=aac_dec_-
decode_error_end)) ? 1 : 0)
• #define IS_OUTPUT_VALID(err) ( ((err) == AAC_DEC_OK) jj IS_DECODE_ERROR(err) )
• #define AACDEC_CONCEAL 1
• #define AACDEC_FLUSH 2
• #define AACDEC_INTR 4
• #define AACDEC_CLRHIST 8
Typedefs:
Enumerations:
Functions:
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_AncDataInit (HANDLE_AACDECODER self, UCHAR buffer, int size)
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_AncDataGet (HANDLE_AACDECODER self, int index, UCHAR ptr, int size)
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_SetParam (const HANDLE_-AACDECODER self, const AACDEC_PARAM param, const INT value)
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_GetFreeBytes (const HANDLE_-AACDECODER self, UINT pFreeBytes)
• LINKSPEC_H HANDLE_AACDECODER aacDecoder_Open (TRANSPORT_TYPE trans-portFmt, UINT nrOfLayers)
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_ConfigRaw (HANDLE_AACDECODER self, UCHAR conf[ ], const UINT length[ ])
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_Fill (HANDLE_AACDECODER self,
UCHAR pBuffer[ ], const UINT bufferSize[ ], UINT bytesValid)
• LINKSPEC_H AAC_DECODER_ERROR aacDecoder_DecodeFrame (HANDLE_-AACDECODER self, INT_PCM pTimeData, const INT timeDataSize, const UINT flags)
• LINKSPEC_H void aacDecoder_Close (HANDLE_AACDECODER self)
• LINKSPEC_H CStreamInfo aacDecoder_GetStreamInfo (HANDLE_AACDECODER self)
• LINKSPEC_H INT aacDecoder_GetLibInfo (LIB_INFO info)
4:function call sequence
5:buffer system
still unprocessed bytes (bytesValid is unequal 0), you would have to additionally perform a memcpy(), so that just means unnecessary computational overhead and therefore we recommend to re-fill the buffer only when bytesValid is 0.
over,go and have lunch