OpenHarmony VideoEncoder js接口开发

OpenHarmony 添加VideoEncoder的ts接口开发记录

一、背景

​ 由于项目中需要将camera导出的yuv数据进行推流，所以需要VideoEncoder来编码yuv数据程h264视频流，但是查看官方网站发现openharmony关于编码和解码这部分只有native接口。所以需要使用native接口实现视频编码并通过napi导出ts接口提供给app使用。而且在3.2release版本中 video encoder的native接口中没有buffer模式的入口，只能通过surface接口把数据传到video encoder中，而surface相关的接口并没有开放到ndk中。所以想要在3.2 release分支下开发video encoder的ts接口，只能将surface相关的接口导出到ndk中才能在app中开发。但是我在查看官方master分支中发现，在master分支上，官方将codec相管的接口都从/foundation/multimedia/player_framework/这个仓库移动到了/foundation/multimedia/av_codec/这个仓库下，并且增加了buffer模式的入口。，而且在3.2 release版本中虽然buffer模式的接口没有开放，但是底层其实还是实现了对应的接口的。所以现在有两个方案：

方案一、使用现成的surface接口，将surface相关的接口导出来即可开发

方案二、使用buffer接口，需要将最新的buffer模式的接口加入到目前的代码中

综合考虑我选择了方案二 项目地址：https://gitee.com/yw2219247615/camera-demo_-ohos

二、定义视频编码的接口

由于本人对于视频编码的知识了解较少，所以提供给app使用的接口主要参考native接口来设计，接口文件index.d.ts如下

//index.d.ts
import media from '@ohos.multimedia.media';

declare namespace avcodec {
    /**
     * Creates a video encoder instance.
     * @since 9
     * @syscap SystemCapability.Multimedia.Media.VideoEncoder
     * @return Returns VideoEncoder interface
     */
    function createVideoEncoder():VideoEncoder
    /**
     * Manages video encoder. Before calling an VideoEncoder method, you must use createVideoEncoder()
     * to create an VideoEncoder instance.
     * @since 9
     * @syscap SystemCapability.Multimedia.Media.VideoEncoder
     */
    interface VideoEncoder {
        /**
         * To prepare the internal resources of the encoder, the Configure interface must be called before
         * calling this interface.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        prepare():CodecErrCode
        /**
         * To configure the video encoder, typically, you need to configure the description information of the
         * encoded video track. This interface must be called before Prepare is called.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @param  an AvFormat instance {@link AvFormat}
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        configure(format:AvFormat):CodecErrCode
        /**
         * Start the encoder, this interface must be called after the Prepare is successful.
         * calling this interface.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        start():CodecErrCode
        /**
         * Stop the encoder. After stopping, you can re-enter the Started state through Start.
         * calling this interface.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        stop():CodecErrCode
        /**
         * Clear output data buffered in the encoder. After this interface is called, all the Buffer
         * indexes previously reported through the asynchronous callback will be invalidated, make sure not to access the
         * Buffers corresponding to these indexes. this interface must be called before stop interface.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        flush():CodecErrCode
        /**
         * Reset the encoder. To continue coding, you need to call the configure interface again to
         * configure the encoder instance.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        reset():CodecErrCode
        /**
         * Get the description information of the output data of the encoder, refer to {@link AvFormat} for details.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns a string obj about AvFormat
         */
        getOutputDescription():string
        /**
         * Clear the internal resources of the encoder and destroy the encoder instance
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @return Returns AV_ERR_OK if the execution is successful,
         * otherwise returns a specific error code, refer to {@link CodecErrCode}
         */
        release():CodecErrCode
        /**
         * Send frame data to video encoder, this interface must be called after the start is successful.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @param data Image data buffer
         * @param dataSize Image data byte size, Indicates the size of valid data in the buffer
         * @param timeStamp Image data timestamp
         */
        sendFrameData(data:Uint8Array,dataSize:number,timeStamp:number):void
        /**
         * Register or unregister listens for video encoder events.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @param type Type of the video encoder event to listen for.
         * @param callback Callback used to listen for the video encoder output buffer event.
         */
        on(type:"OnOutputBufferAvailable",callback:(data: ArrayBuffer, dataSize: number,pts:number,flags:number) => void):void
        /**
         * Register or unregister listens for video encoder events.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @param type Type of the video encoder event to listen for.
         * @param callback Callback used to listen for the video encoder error event.
         */
        on(type:"onError",callback:(errCoder: number) => void):void
        /**
         * Register or unregister listens for video encoder events.
         * @since 9
         * @syscap SystemCapability.Multimedia.Media.VideoEncoder
         * @param type Type of the video encoder event to listen for.
         * @param callback Callback used to listen for the video encoder output format change event.
         */
        on(type:"OnOutputFormatChanged",callback:(format: AvFormat) => void):void
    }

    interface AvFormat {
        /**设置编码器的类型目前只支持视频编码器*/
        mime:media.CodecMimeType;
        /**图像帧的宽度 */
        width:number;
        /**图像帧的高度 */
        height:number;
        /**输入图像的格式 */
        pixelFormat:VideoPixelFormat;
        /**图像编码的帧率*/
        frameRate:number;
        /**图像编码的码率*/
        encodeBitRate:number;
        /** I 帧采集的间隔 单位：毫秒*/
        iFrameIntervalMs:number;
        /**编码质量:0 - 100*/
        codecQuality:number;
        /**编码的profile:
         *AVC_PROFILE_BASELINE = 0,
         *AVC_PROFILE_HIGH = 4,
         *AVC_PROFILE_MAIN = 8,
         */
        codecProfile:number;

        videoEncodeBitRateMode: VideoEncodeBitrateMode;
    }

    enum VideoPixelFormat {
        /**
         * yuv 420 planar.
         */
        YUVI420 = 1,
        /**
         *  NV12. yuv 420 semiplanar.
         */
        NV12 = 2,
        /**
         *  NV21. yvu 420 semiplanar.
         */
        NV21 = 3,
        /**
         * format from surface.
         */
        SURFACE_FORMAT = 4,
        /**
         * RGBA.
         */
        RGBA = 5,
    }

    enum CodecErrCode {
        /**
         * the operation completed successfully.
         */
        AV_ERR_OK = 0,
        /**
         * no memory.
         */
        AV_ERR_NO_MEMORY = 1,
        /**
         * opertation not be permitted.
         */
        AV_ERR_OPERATE_NOT_PERMIT = 2,
        /**
         * invalid argument.
         */
        AV_ERR_INVALID_VAL = 3,
        /**
         * IO error.
         */
        AV_ERR_IO = 4,
        /**
         * network timeout.
         */
        AV_ERR_TIMEOUT = 5,
        /**
         * unknown error.
         */
        AV_ERR_UNKNOWN = 6,
        /**
         * media service died.
         */
        AV_ERR_SERVICE_DIED = 7,
        /**
         * the state is not support this operation.
         */
        AV_ERR_INVALID_STATE = 8,
        /**
         * unsupport interface.
         */
        AV_ERR_UNSUPPORT = 9,
        /**
         * extend err start.
         */
        AV_ERR_EXTEND_START = 100,
    }

    enum VideoEncodeBitrateMode {
        /**
         * constant bit rate mode.
         */
        CBR = 0,
        /**
         * variable bit rate mode.
         */
        VBR = 1,
        /**
         * constant quality mode.
         */
        CQ = 2,
    }
}

export default avcodec;

上面的文件主要时定义了一些encoder的行为和相关的枚举类，基本也都是从native层拷贝过来的，AvFormat主要用来配置videoencoder的参数。

三、建立NAPI层

这里我定义VideoEncoder的napi层为video_encoder_napi.cpp

首先需要先定义和上层ts接口对应的napi接口如下：

//video_encoder_napi.cpp
napi_value VideoEncoderNapi::Init(napi_env env, napi_value exports) {
    AVCODEC_LOGD("%s ", __FUNCTION__);
    napi_property_descriptor properties[] = {
        //这里的这些宏定义都被我导入到了common/native_common.h中其实都是系统源码中的
        DECLARE_NAPI_FUNCTION("configure", Configure),
        DECLARE_NAPI_FUNCTION("prepare", Prepare),
        DECLARE_NAPI_FUNCTION("start", Start),
        DECLARE_NAPI_FUNCTION("stop", Stop),
        DECLARE_NAPI_FUNCTION("flush", Flush),
        DECLARE_NAPI_FUNCTION("reset", Reset),
        DECLARE_NAPI_FUNCTION("getOutputDescription", GetOutputDescription),
        DECLARE_NAPI_FUNCTION("sendFrameData", SendFrameData),
        DECLARE_NAPI_FUNCTION("release", Release),
        DECLARE_NAPI_FUNCTION("on", On)};

    
    napi_property_descriptor staticProperty[] = {
        DECLARE_NAPI_STATIC_FUNCTION("createVideoEncoder", CreateVideoEncoder),
    };

    napi_value constructor = nullptr;
    napi_status status = napi_define_class(env, CLASS_NAME.c_str(), NAPI_AUTO_LENGTH, Constructor, nullptr,
                                           sizeof(properties) / sizeof(properties[0]), properties, &constructor);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, nullptr, "Failed to define VideoEncoder class");

    status = napi_create_reference(env, constructor, 1, &constructor_);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, nullptr, "Failed to create reference of constructor");

    status = napi_set_named_property(env, exports, CLASS_NAME.c_str(), constructor);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, nullptr, "Failed to set constructor");

    status = napi_define_properties(env, exports, sizeof(staticProperty) / sizeof(staticProperty[0]), staticProperty);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, nullptr, "Failed to define static function");

    AVCODEC_LOGD("Init success");

    return exports;
}

在上面的接口中CreateVideoEncoder会触发整个video_encoder_napi的创建，当调用configure时才会通过上层传下来的avformat中的mime真正的创建encoder，并配置相关的参数，剩下的操作基本就是一个搬运工将ts的指令传递到native的encoder即可。

3.1 CreateVideoEncoder

napi_value VideoEncoderNapi::CreateVideoEncoder(napi_env env, napi_callback_info info) {
    AVCODEC_LOGD("%s ", __FUNCTION__);
    napi_value result = nullptr;
    napi_get_undefined(env, &result);
    //创建napi对象
    napi_value ctor;
    napi_status status;
    status = napi_get_reference_value(env, constructor_, &ctor);
    if (status != napi_ok) {
        AVCODEC_LOGE("napi_get_reference_value fail");
        return result;
    }
    status = napi_new_instance(env, ctor, 0, nullptr, &result);
    if (status == napi_ok) {
        AVCODEC_LOGD("New instance success");
    } else {
        AVCODEC_LOGD("New instance could not be obtained");
    }
    return result;
}

CreateVideoEncoder函数非常简单就是创建napi对象，其中会调用到VideoEncoderNapi::Constructor函数，如下；

napi_value VideoEncoderNapi::Constructor(napi_env env, napi_callback_info info) {
    AVCODEC_LOGD("%s ", __FUNCTION__);
    napi_value result = nullptr;
    napi_get_undefined(env, &result);
    napi_value jsThis = nullptr;
    napi_status status = napi_get_cb_info(env, info, nullptr, nullptr, &jsThis, nullptr);
    if (status != napi_ok) {
        AVCODEC_LOGE("Failed to retrieve details about the callback");
        return result;
    }
    if (jsThis != nullptr) {
        VideoEncoderNapi *videoEncoderNapi = new (std::nothrow) VideoEncoderNapi();
        if (videoEncoderNapi != nullptr) {
            videoEncoderNapi->env_ = env;
            status = napi_wrap(env, jsThis, reinterpret_cast<void *>(videoEncoderNapi),
                               VideoEncoderNapi::Destructor, nullptr, nullptr);
            if (status != napi_ok) {
                delete videoEncoderNapi;
                AVCODEC_LOGE("Failed to warp native instance!");
                return result;
            }
        }
    }
    AVCODEC_LOGD("Constructor success");
    return jsThis;
}

上面的这些基本都属于模板了。

3.2 Configure

napi_value VideoEncoderNapi::Configure(napi_env env, napi_callback_info info) {
    AVCODEC_LOGD("%s ", __FUNCTION__);
    napi_value result = nullptr;
    napi_get_undefined(env, &result);
    //获取arkts传过来参数信息
    napi_value thisVar = nullptr;
    napi_value args[ARGS1] = {nullptr};
    size_t argCount = ARGS1;
    napi_status status;
    status = napi_get_cb_info(env, info, &argCount, args, &thisVar, nullptr);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, result, "failed to napi_get_cb_info");
    //获取AvFormat
    AvFormat format = getAvFormat(env, args[PARAM0]);
    //获取napi实例
    VideoEncoderNapi *videoEncoderNapi = nullptr;
    getVideoEncoderNapi(env, thisVar, &videoEncoderNapi);
    if (videoEncoderNapi == nullptr)
        return result;
    int32_t ret;
    videoEncoderNapi->videoEncoder = VideoEncoderFactory::CreateByMime(format.codec_mime);
    ret = videoEncoderNapi->videoEncoder->configure(&format);
    //创建回调接口
    videoEncoderNapi->codecCallback = std::make_shared<AvCodecCallbackNapi>();
    videoEncoderNapi->videoEncoder->SetCallback(videoEncoderNapi->codecCallback);
    AvCodecNapiUtils::createNapiInt32Value(env, ret, result);
    return result;
}

在napi层当调用了configure后才会真正创建native层的video encoder可以看到上面通过调用VideoEncoderFactory::CreateByMime创建一个videoEncoder对象并保存在了videoEncoderNapi的videoEncoder指针里，后面我们的操作都会通过这个指针进行。

3.3 Start

napi_value VideoEncoderNapi::Start(napi_env env, napi_callback_info info) {
    AVCODEC_LOGD("%s ", __FUNCTION__);
    napi_value result = nullptr;
    napi_value thisVar = nullptr;
    napi_get_undefined(env, &result);
    napi_status status;
    AVCODEC_NAPI_GET_JS_OBJ_WITH_ZERO_ARGS(env, info, status, thisVar);
    CHECK_AND_RETURN_RET_LOG(status == napi_ok, result, "failed to napi_get_cb_info");
    VideoEncoderNapi *videoEncoderNapi = nullptr;
    getVideoEncoderNapi(env, thisVar, &videoEncoderNapi);
    if (videoEncoderNapi == nullptr)
        return result;
    int32_t ret;
    ret = videoEncoderNapi->videoEncoder->start();
    AvCodecNapiUtils::createNapiInt32Value(env, ret, result);
    return result;
}

可以看到正如上面所说都是通过获取napi实例，接着通过它的videoEncoder来操作video encoder的，其他的流程控制比如stop reset release flush等等都是一个模板。这里就不再细说了

四、封装video encoder

由于video encoder的ndk中都只是一些接口，所以我这边将这些接口封装成了一个类VideoEncoder，在上面的3.2小节中我们知道videoencoder是通过VideoEncoderFactory创建的，当它初始化时就会创建native video encoder并设置callback

VideoEncoder::VideoEncoder(std::string mime){
    AVCODEC_LOGD("VideoEncoder create mime =%s",mime.c_str());
    venc_ = OH_VideoEncoder_CreateByMime(mime.c_str());
    if(venc_ == nullptr){
        AVCODEC_LOGD("venc_ is null");
    }
    setEncoderCallback();
    setVideoEncoderMap(venc_,this);
}
int32_t VideoEncoder::setEncoderCallback(){
    AVCODEC_CHECK_AND_RETURN_RET_LOG(venc_ != nullptr, AV_ERR_UNKNOWN,"Fatal: OH_VideoEncoder_CreateByMime fail");
    struct OH_AVCodecAsyncCallback callback;
                callback.onError = VideoEncoder::OnError;
                callback.onStreamChanged = VideoEncoder::OnStreamChanged;
                callback.onNeedInputData = VideoEncoder::OnNeedInputData;
                callback.onNeedOutputData = VideoEncoder::OnNewOutputData;
    return OH_VideoEncoder_SetCallback(venc_, callback,nullptr);
}
void VideoEncoder::setVideoEncoderMap(OH_AVCodec* codec,VideoEncoder* encoder){
    std::lock_guard<std::mutex> lock(mutex_);
    encoderMap_[codec] = encoder;
}

setEncoderCallback主要是设置一些接口给native video encoder来接收输入的buffer(用来填充待编码的数据)和已经编码的buffer数据以及一些错误和变化。setVideoEncoderMap则是将当前创建的native vidoe encoder和我们的VideoEncoder对象对应起来，这是因为OH_AVCodecAsyncCallback的成员函数要求是静态函数，但是如果我们的VideoEncoder对象创建多个，那么无法找到对应的VideoEncoder对象去回调，所以我这边使用了一个map去存储对应关系，这个map的key是OH_AVCodec指针，value是VideoEncoder指针。

4.1 数据流转

当video encoder启动后底层会回调待填充数据的buffer上来，我这里会将这个buffer存储到一个队列中，当上层ts传入数据下来时，我从这个队列中拿出一个buffer将数据拷贝进去，接着调用OH_VideoEncoder_PushInputData将数据发送给native video encoder。代码实现如下：

//native video encoder回调可用的buffer
void VideoEncoder::OnNeedInputData(OH_AVCodec *codec, uint32_t index, OH_AVMemory *data, void *userData){
    //从map中获取到encoder对象
    VideoEncoder* encoder = GetVideoEncoder(codec);
    if (encoder != nullptr) {
        //将buffer存放到队列中
        encoder->pushInputBufferToQueue(index,data);
    }
}

int32_t VideoEncoder::sendFrameData(uint8_t *data, int32_t size, int32_t timestamp){
    //当上层的待编码的数据传过来时
    AVCODEC_LOGD("sendFrameData data = %d",data);
    AVCODEC_CHECK_AND_RETURN_RET_LOG(venc_ != nullptr, AV_ERR_UNKNOWN,"Fatal: OH_VideoEncoder_CreateByMime fail");
    std::lock_guard<std::mutex> lock(queueMutex_);
    if(indexQueue.size() <= 0){
        AVCODEC_LOGE("indexQueue is empty");
        return AV_ERR_NO_MEMORY;
    }
    uint32_t index = indexQueue.front();
    auto buffer = inBufferQueue.front();
    //从队列中拿出buffer
    OH_AVCodecBufferAttr info;
    if(size == 0){
        info.size = 0;
        info.offset = 0;
        info.pts = 0;
        info.flags = AVCODEC_BUFFER_FLAGS_EOS;
        OH_VideoEncoder_PushInputData(venc_, index, info);
        AVCODEC_LOGE("send data len == 0");
    }
    info.size = size;
    info.offset = 0;
    info.pts = timestamp;
    //拷贝数据
    memcpy(OH_AVMemory_GetAddr(buffer), data, OH_AVMemory_GetSize(buffer));
    //将buffer 推给native video encoder去编码 //这里调用的是buffer模式的发送数据3.2 release分支需要增加此接口并导出
    int32_t ret = OH_VideoEncoder_PushInputData(venc_,index,info);
    indexQueue.pop();
    inBufferQueue.pop();
    return ret;
}

当数据编码后会通过VideoEncoder::OnNewOutputData接收到

void VideoEncoder::OnNewOutputData(OH_AVCodec *codec, uint32_t index, OH_AVMemory *data,OH_AVCodecBufferAttr *attr, void *userData){
    VideoEncoder* encoder = GetVideoEncoder(codec);
    if (encoder != nullptr) {
        encoder->avcodecCb->OnOutputBufferAvailable(OH_AVMemory_GetAddr(data), attr->size, attr->pts, attr->flags);
    }
    OH_VideoEncoder_FreeOutputData(codec,index);
}

这里通过回调接口将数据回调到上层后我们需要调用OH_VideoEncoder_FreeOutputData将buffer归还给native video encoder来维持数据流转。

五、增加buffer模式的发送数据接口

From 72ee572a520604d23dd2aee9064d4fc252462df4 Mon Sep 17 00:00:00 2001
From: yuwei <yuw@guideir.com>
Date: Fri, 2 Jun 2023 11:06:36 +0800
Subject: [PATCH] =?UTF-8?q?feat:=E5=A2=9E=E5=8A=A0video=20encoder=20buffer?=
 =?UTF-8?q?=E6=A8=A1=E5=BC=8F=E5=B9=B6=E5=AF=BC=E5=87=BAndk=E6=8E=A5?=
 =?UTF-8?q?=E5=8F=A3?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

---
 .../capi/avcodec/native_video_encoder.cpp     | 67 ++++++++++++++++++-
 .../kits/c/native_avcodec_videoencoder.h      | 27 ++++++++
 .../libnative_media_venc.ndk.json             |  4 +-
 .../codec/hdi_plugins/hdi_codec_util.cpp      |  2 +
 4 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/player_framework/frameworks/native/capi/avcodec/native_video_encoder.cpp b/player_framework/frameworks/native/capi/avcodec/native_video_encoder.cpp
index 07b9989..8ac87ea 100644
--- a/player_framework/frameworks/native/capi/avcodec/native_video_encoder.cpp
+++ b/player_framework/frameworks/native/capi/avcodec/native_video_encoder.cpp
@@ -39,10 +39,12 @@ struct VideoEncoderObject : public OH_AVCodec {
     const std::shared_ptr<AVCodecVideoEncoder> videoEncoder_;
     std::list<OHOS::sptr<OH_AVMemory>> memoryObjList_;
     OHOS::sptr<OH_AVFormat> outputFormat_ = nullptr;
+    OHOS::sptr<OH_AVFormat> inputputFormat_ = nullptr;
     std::shared_ptr<NativeVideoEncoderCallback> callback_ = nullptr;
     std::atomic<bool> isFlushing_ = false;
     std::atomic<bool> isStop_ = false;
     std::atomic<bool> isEOS_ = false;
+    bool isInputSurfaceMode_ = false;
 };
 
 class NativeVideoEncoderCallback : public AVCodecCallback {
@@ -82,7 +84,12 @@ public:
                 MEDIA_LOGD("At flush, eos or stop, no buffer available");
                 return;
             }
-            callback_.onNeedInputData(codec_, index, nullptr, userData_);
+            OH_AVMemory *data = nullptr;
+            if (!videoEncObj->isInputSurfaceMode_) {
+                data = GetInputData(codec_, index);
+                CHECK_AND_RETURN_LOG(data != nullptr, "Data is nullptr, get input data failed");
+            }
+            callback_.onNeedInputData(codec_, index, data, userData_);
         }
     }
 
@@ -115,6 +122,30 @@ public:
     }
 
 private:
+    OH_AVMemory *GetInputData(struct OH_AVCodec *codec, uint32_t index)
+    {
+        CHECK_AND_RETURN_RET_LOG(codec != nullptr, nullptr, "Codec is nullptr!");
+        CHECK_AND_RETURN_RET_LOG(codec->magic_ == AVMagic::MEDIA_MAGIC_VIDEO_ENCODER, nullptr, "Codec magic error!");
+
+        struct VideoEncoderObject *videoEncObj = reinterpret_cast<VideoEncoderObject *>(codec);
+        CHECK_AND_RETURN_RET_LOG(videoEncObj->videoEncoder_ != nullptr, nullptr, "Video decoder is nullptr!");
+
+        std::shared_ptr<AVSharedMemory> memory = videoEncObj->videoEncoder_->GetInputBuffer(index);
+        CHECK_AND_RETURN_RET_LOG(memory != nullptr, nullptr, "Memory is nullptr, get input buffer failed!");
+
+        for (auto &memoryObj : videoEncObj->memoryObjList_) {
+            if (memoryObj->IsEqualMemory(memory)) {
+                return reinterpret_cast<OH_AVMemory *>(memoryObj.GetRefPtr());
+            }
+        }
+
+        OHOS::sptr<OH_AVMemory> object = new (std::nothrow) OH_AVMemory(memory);
+        CHECK_AND_RETURN_RET_LOG(object != nullptr, nullptr, "AV memory create failed");
+
+        videoEncObj->memoryObjList_.push_back(object);
+        return reinterpret_cast<OH_AVMemory *>(object.GetRefPtr());
+    }
+
     OH_AVMemory *GetOutputData(struct OH_AVCodec *codec, uint32_t index)
     {
         CHECK_AND_RETURN_RET_LOG(codec != nullptr, nullptr, "input codec is nullptr!");
@@ -319,6 +350,7 @@ OH_AVErrCode OH_VideoEncoder_GetSurface(OH_AVCodec *codec, OHNativeWindow **wind
     *window = CreateNativeWindowFromSurface(&surface);
 
     CHECK_AND_RETURN_RET_LOG(*window != nullptr, AV_ERR_INVALID_VAL, "CreateNativeWindowFromSurface failed!");
+    videoEncObj->isInputSurfaceMode_ = true;
 
     return AV_ERR_OK;
 }
@@ -402,3 +434,36 @@ OH_AVErrCode OH_VideoEncoder_SetCallback(
 
     return AV_ERR_OK;
 }
+OH_AVErrCode OH_VideoEncoder_PushInputData(struct OH_AVCodec *codec, uint32_t index, OH_AVCodecBufferAttr attr)
+{
+    CHECK_AND_RETURN_RET_LOG(codec != nullptr, AV_ERR_INVALID_VAL, "Codec is nullptr!");
+    CHECK_AND_RETURN_RET_LOG(codec->magic_ == AVMagic::MEDIA_MAGIC_VIDEO_ENCODER, AV_ERR_INVALID_VAL,
+        "Codec magic error!");
+    CHECK_AND_RETURN_RET_LOG(attr.size >= 0, AV_ERR_INVALID_VAL, "Invalid buffer size!");
+
+    struct VideoEncoderObject *videoEncObj = reinterpret_cast<VideoEncoderObject *>(codec);
+    CHECK_AND_RETURN_RET_LOG(videoEncObj->videoEncoder_ != nullptr, AV_ERR_INVALID_VAL, "Video encoder is nullptr!");
+
+    struct AVCodecBufferInfo bufferInfo;
+    bufferInfo.presentationTimeUs = attr.pts;
+    bufferInfo.size = attr.size;
+    bufferInfo.offset = attr.offset;
+    enum AVCodecBufferFlag bufferFlag = static_cast<enum AVCodecBufferFlag>(attr.flags);
+
+    int32_t ret = videoEncObj->videoEncoder_->QueueInputBuffer(index, bufferInfo, bufferFlag);
+    CHECK_AND_RETURN_RET_LOG(ret == MSERR_OK, AV_ERR_OPERATE_NOT_PERMIT, "Video encoder push input data failed!");
+    if (bufferFlag == AVCODEC_BUFFER_FLAG_EOS) {
+        videoEncObj->isEOS_.store(true);
+    }
+    return AV_ERR_OK;
+}
+
+OH_AVErrCode OH_VideoEncoder_IsValid(OH_AVCodec *codec, bool *isValid)
+{
+    CHECK_AND_RETURN_RET_LOG(codec != nullptr, AV_ERR_INVALID_VAL, "Codec is nullptr!");
+    CHECK_AND_RETURN_RET_LOG(codec->magic_ == AVMagic::MEDIA_MAGIC_VIDEO_ENCODER, AV_ERR_INVALID_VAL,
+        "Codec magic error!");
+    CHECK_AND_RETURN_RET_LOG(isValid != nullptr, AV_ERR_INVALID_VAL, "Input isValid is nullptr!");
+    *isValid = true;
+    return AV_ERR_OK;
+}
diff --git a/player_framework/interfaces/kits/c/native_avcodec_videoencoder.h b/player_framework/interfaces/kits/c/native_avcodec_videoencoder.h
index a429309..2610f00 100644
--- a/player_framework/interfaces/kits/c/native_avcodec_videoencoder.h
+++ b/player_framework/interfaces/kits/c/native_avcodec_videoencoder.h
@@ -207,6 +207,33 @@ OH_AVErrCode OH_VideoEncoder_FreeOutputData(OH_AVCodec *codec, uint32_t index);
  */
 OH_AVErrCode OH_VideoEncoder_NotifyEndOfStream(OH_AVCodec *codec);
 
+/**
+ * @brief Submit the input buffer filled with data to the video encoder.
+ * @syscap SystemCapability.Multimedia.Media.VideoEncoder
+ * @param codec Pointer to an OH_AVCodec instance
+ * @param index Enter the index value corresponding to the Buffer
+ * @param attr Information describing the data contained in the Buffer
+ * @return Returns AV_ERR_OK if the execution is successful,
+ * otherwise returns a specific error code, refer to {@link OH_AVErrCode}
+ * @since 10
+ * @version 4.0
+ */
+OH_AVErrCode OH_VideoEncoder_PushInputData(OH_AVCodec *codec, uint32_t index, OH_AVCodecBufferAttr attr);
+
+/**
+ * @brief Check whether the current codec instance is valid. It can be used fault recovery or app
+ * switchback from the background
+ * @syscap SystemCapability.Multimedia.Media.VideoEncoder
+ * @param codec Pointer to an OH_AVCodec instance
+ * @param isValid Pointer to an bool instance, true: the codec instance is valid, false: the codec
+ * instance is invalid
+ * @return Returns AV_ERR_OK if the execution is successful,
+ * otherwise returns a specific error code, refer to {@link OH_AVErrCode}
+ * @since 10
+ * @version 4.0
+ */
+OH_AVErrCode OH_VideoEncoder_IsValid(OH_AVCodec *codec, bool *isValid);
+
 /**
  * @brief The bitrate mode of video encoder.
  * @syscap SystemCapability.Multimedia.Media.VideoEncoder
diff --git a/player_framework/interfaces/kits/c/video_encoder/libnative_media_venc.ndk.json b/player_framework/interfaces/kits/c/video_encoder/libnative_media_venc.ndk.json
index e2264a4..a62e0c1 100644
--- a/player_framework/interfaces/kits/c/video_encoder/libnative_media_venc.ndk.json
+++ b/player_framework/interfaces/kits/c/video_encoder/libnative_media_venc.ndk.json
@@ -13,5 +13,7 @@
     { "name": "OH_VideoEncoder_SetParameter" },
     { "name": "OH_VideoEncoder_GetSurface" },
     { "name": "OH_VideoEncoder_FreeOutputData" },
-    { "name": "OH_VideoEncoder_NotifyEndOfStream" }
+    { "name": "OH_VideoEncoder_NotifyEndOfStream" },
+    { "name": "OH_VideoEncoder_PushInputData" },
+    { "name": "OH_VideoEncoder_IsValid" }
 ]

2.25.1

六、存在的问题

1.在持续的编码的数据过程中发现内存一直在持续增加，可能出现了内存泄漏

加入官方的patch：https://gitee.com/openharmony/arkui_napi/commit/420a9434606c2b1b6be726eb1b2f6c34b2faff94

并且加入如下的代码

--- a/player_framework/frameworks/js/avcodec/avcodec_callback_napi.cpp
+++ b/player_framework/frameworks/js/avcodec/avcodec_callback_napi.cpp
@@ -38,6 +38,9 @@ void AvCodecCallbackNapi::OnOutputBufferAvailableAsync(BufferInfo &info) const{
     int32_t flags = info.flags;
     if(onOutputBufferAvailableCb_env == nullptr || onOutputBufferAvailableCb == nullptr) return;
     if(data != nullptr && size > 0){
+        napi_handle_scope scope;
+        napi_open_handle_scope(onOutputBufferAvailableCb_env,&scope);
+        if(scope == nullptr) return;
         napi_value args[ARGS4] = {0};
         napi_create_int32(onOutputBufferAvailableCb_env,size,&args[PARAM1]);
         napi_create_int32(onOutputBufferAvailableCb_env,pts,&args[PARAM2]);
@@ -50,6 +53,7 @@ void AvCodecCallbackNapi::OnOutputBufferAvailableAsync(BufferInfo &info) const{
             AVCODEC_LOGE("AvCodecCallbackNapi::%s CreateArrayBuffer fail",__FUNCTION__);
             free(napiArrBuf);
             napiArrBuf = nullptr;
+            napi_close_handle_scope(onOutputBufferAvailableCb_env, scope);
             return;
         }
         napi_value callbackRet = nullptr;
@@ -59,6 +63,7 @@ void AvCodecCallbackNapi::OnOutputBufferAvailableAsync(BufferInfo &info) const{
             AVCODEC_LOGE("AvCodecCallbackNapi::%s napi_get_reference_value err[%{public}d]",__FUNCTION__, status);
             free(napiArrBuf);
             napiArrBuf = nullptr;
+            napi_close_handle_scope(onOutputBufferAvailableCb_env, scope);
             return;
         }
         status = napi_call_function(onOutputBufferAvailableCb_env, nullptr, callback, ARGS4, args, &callbackRet);
@@ -69,6 +74,7 @@ void AvCodecCallbackNapi::OnOutputBufferAvailableAsync(BufferInfo &info) const{
         }
         free(napiArrBuf);
         napiArrBuf = nullptr;
+        napi_close_handle_scope(onOutputBufferAvailableCb_env, scope);
     }
 }

可以降低内存增加的速度，但是无法彻底解决内存增长的问题，目前怀疑是napi层的接口存在内存泄漏已经向官方提了问题单，需要持续跟踪。
2023.07.26 拉取openharmony的master分支编译验证发现此问题已经解决。