随着Android系统手机性能的不断提升,现阶段大部分手机都自带GPU(承担图形显示的专门硬件),大幅度提高手机显示性能,在视频显示、游戏画面刷新,和高分辨图像显示方面必须使用GPU。GOOGLE在API 16 -4.1版本中增加MediaCodec类,专用于编解码多媒体数据,
二、MediaCodec使用方式
MediaCodec总共有三种使用方法,如下图所示:
同步数据处理(使用buffer arrays) 从Android4.1 api 16即可以使用;
同步数据处理 (使用buffers ) 从Android5.0 api 21 即可以使用;
异步数据处理(使用buffers ) 从Android5.0 api 21 即可以使用;
同步与异步处理数据的主要不同点是:对于同步处理是循环的将待处理数据交给编解码器(不用理会编解码器是否已经准备好接收数据),而异步处理则是每次都会去等待编解码器已经能够处理数据时,才将待处理数据送给相应编解码器。
MediaCodec is typically used like this in asynchronous mode:
- MediaCodec codec = MediaCodec.createByCodecName(name);
- MediaFormat mOutputFormat; // member variable
- codec.setCallback(new MediaCodec.Callback() {
- @Override
- void onInputBufferAvailable(MediaCodec mc, int inputBufferId) {
- ByteBuffer inputBuffer = codec.getInputBuffer(inputBufferId);
- // fill inputBuffer with valid data
- …
- codec.queueInputBuffer(inputBufferId, …);
- }
- @Override
- void onOutputBufferAvailable(MediaCodec mc, int outputBufferId, …) {
- ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
- MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId); // option A
- // bufferFormat is equivalent to mOutputFormat
- // outputBuffer is ready to be processed or rendered.
- …
- codec.releaseOutputBuffer(outputBufferId, …);
- }
- @Override
- void onOutputFormatChanged(MediaCodec mc, MediaFormat format) {
- // Subsequent data will conform to new format.
- // Can ignore if using getOutputFormat(outputBufferId)
- mOutputFormat = format; // option B
- }
- @Override
- void onError(…) {
- …
- }
- });
- codec.configure(format, …);
- mOutputFormat = codec.getOutputFormat(); // option B
- codec.start();
- // wait for processing to complete
- codec.stop();
- codec.release();
Synchronous Processing using Buffers
- MediaCodec codec = MediaCodec.createByCodecName(name);
- codec.configure(format, …);
- MediaFormat outputFormat = codec.getOutputFormat(); // option B
- codec.start();
- for (;;) {
- int inputBufferId = codec.dequeueInputBuffer(timeoutUs);
- if (inputBufferId >= 0) {
- ByteBuffer inputBuffer = codec.getInputBuffer(…);
- // fill inputBuffer with valid data
- …
- codec.queueInputBuffer(inputBufferId, …);
- }
- int outputBufferId = codec.dequeueOutputBuffer(…);
- if (outputBufferId >= 0) {
- ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
- MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId); // option A
- // bufferFormat is identical to outputFormat
- // outputBuffer is ready to be processed or rendered.
- …
- codec.releaseOutputBuffer(outputBufferId, …);
- } else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
- // Subsequent data will conform to new format.
- // Can ignore if using getOutputFormat(outputBufferId)
- outputFormat = codec.getOutputFormat(); // option B
- }
- }
- codec.stop();
- codec.release();
三、MediaCodec能够硬编解码的类型
其中对于视频格式,MediaCodec支持的有vp8 、VP9 、H.264、H.265、MPEG4、H.263基本上涵盖了现今流行的所有视频格式,
对于音频来说,MediaCodec支持的音频格式有3gpp、amr-wb、amr-wb、amr-wb、g711-A、g711-U 、AAC(not packaged in LATM)
- "video/x-vnd.on2.vp8" - VP8 video (i.e. video in .webm)
- "video/x-vnd.on2.vp9" - VP9 video (i.e. video in .webm)
- "video/avc" - H.264/AVC video
- "video/hevc" - H.265/HEVC video
- "video/mp4v-es" - MPEG4 video
- "video/3gpp" - H.263 video
- "audio/3gpp" - AMR narrowband audio
- "audio/amr-wb" - AMR wideband audio
- "audio/amr-wb" - MPEG1/2 audio layer III
- "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!)
- "audio/amr-wb" - vorbis audio
- "audio/g711-alaw" - G.711 alaw audio
- "audio/g711-mlaw" - G.711 ulaw audio
Format | Suitable key frame |
---|---|
VP9/VP8 | a suitable intraframe where no subsequent frames refer to frames prior to this frame. (There is no specific name for such key frame.) |
H.265 HEVC | IDR or CRA |
H.264 AVC | IDR |
MPEG-4 H.263 MPEG-2 | a suitable I-frame where no subsequent frames refer to frames prior to this frame. (There is no specific name for such key frame.) |