Mediacodec 如何硬件解码到纹理的

最新推荐文章于 2024-06-19 20:51:19 发布

程龙-音视频

最新推荐文章于 2024-06-19 20:51:19 发布

阅读量2.8k

点赞数 3

分类专栏： android 文章标签： android java 开发语言

本文链接：https://blog.csdn.net/c553110519/article/details/126896431

版权

本文深入解析Android Mediacodec如何将硬件解码后的数据渲染到纹理，通过源码跟踪，从releaseOutputBuffer到最终的纹理更新，详细阐述了数据从解码器到Surface，再到纹理的整个过程，帮助理解外部纹理的工作原理。

摘要由CSDN通过智能技术生成

Mediacodec 如何硬件解码到纹理的

背景：

网上很多关于mediacodec，surface ，surfacetexture的源码分析，以及内部原理，但是都局限于各自的内容，今天我们就从mediacodec硬件解码到纹理，以及外部纹理到底是什么角度的来分析源码。

mediacodec

这是googl官网提供的一张图https://developer.android.com/reference/android/media/MediaCodec，

大体上是生产者消费者之间的关系。

解码过程如下：

Java
MediaCodec codec = MediaCodec.createByCodecName(name);
codec.configure(format, …);
MediaFormat outputFormat = codec.getOutputFormat(); // option B
codec.start();
for (;;) {
int inputBufferId = codec.dequeueInputBuffer(timeoutUs);
if (inputBufferId >= 0) {
    ByteBuffer inputBuffer = codec.getInputBuffer(…);
    // fill inputBuffer with valid data
    …
    codec.queueInputBuffer(inputBufferId, …);
}
int outputBufferId = codec.dequeueOutputBuffer(…);
if (outputBufferId >= 0) {
  ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
    MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId); // option A
    // bufferFormat is identical to outputFormat
    // outputBuffer is ready to be processed or rendered.
    …
    codec.releaseOutputBuffer(outputBufferId, …);
} else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
    // Subsequent data will conform to new format.
    // Can ignore if using getOutputFormat(outputBufferId)
    outputFormat = codec.getOutputFormat(); // option B
}
}
codec.stop();
codec.release();

因为我们要分析渲染到纹理的过程，我就着重看下releaseOutputBuffer这个方法

releaseOutputBuffer

Java
public void releaseOutputBuffer (int index,
boolean render)
/*
If you are done with a buffer, use this call to return the buffer to the codec or to render it on the output surface. If you configured the codec with an output surface, setting render to true will first send the buffer to that output surface. The surface will release the buffer back to the codec once it is no longer used/displayed. Once an output buffer is released to the codec, it MUST NOT be used until it is later retrieved by getOutputBuffer(int) in response to a dequeueOutputBuffer(MediaCodec.BufferInfo, long) return value or a Callback#onOutputBufferAvailable callback.

Parameters
indexint: The index of a client-owned output buffer previously returned from a call to dequeueOutputBuffer(MediaCodec.BufferInfo, long).

renderboolean: If a valid surface was specified when configuring the codec, passing true renders this output buffer to the surface.
*/

可以看的到当render=true的时候就会渲染到surface，那我结合android framework的源码来跟下这个代码到底做了些什么

https://android.googlesource.com/platform/frameworks/base/+/master/media/jni/android_media_MediaCodec.cpp

C++
status_t JMediaCodec::releaseOutputBuffer(
        size_t index, bool render, bool updatePTS, int64_t timestampNs) {
    if (updatePTS) {
        return mCodec->renderOutputBufferAndRelease(index, timestampNs);
    }
    return render
        ? mCodec->renderOutputBufferAndRelease(index)
        : mCodec->releaseOutputBuffer(index);
}

可以看的出来当render=true的时候，调用的是renderOutputBufferAndRelease，那接下来我们来看下renderOutputBufferAndRelease他的源码

renderOutputBufferAndRelease

C++
status_t MediaCodec::renderOutputBufferAndRelease(size_t index, int64_t timestampNs) {
    sp<AMessage> msg = new AMessage(kWhatReleaseOutputBuffer, this);
    msg->setSize("index", index);
    msg->setInt32("render", true);
    msg->setInt64("timestampNs", timestampNs);

    sp<AMessage> response;
    return PostAndAwaitResponse(msg, &response);
}

mediacodec在start的时候会起一个线程，这里会这个解码线程发送一个消息，并设置msg->setInt32("render", true);那我们就看下它的接收方是怎么做的

C++
void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {
    switch (msg->what()) {
    ....
    case kWhatReleaseOutputBuffer:
        {
            sp<AReplyToken> replyID;
            CHECK(msg->senderAwaitsResponse(&replyID));
            status_t err = onReleaseOutputBuffer(msg);

            PostReplyWithError(replyID, err);
            break;
        }
        .........
    }
}