Android平台RTSP|RTMP播放器如何实现TextureView渲染

技术背景

自2015年我们发布Android平台RTSP、RTMP直播播放模块以来,渲染这块,支持SurfaceView或GlSurfaceView,当然如果开发者需要TextureView渲染,可以把RTSP、RTMP流数据解码回调YUV或RGB数据上来,上层自己渲染。本文主要介绍,如何实现RTSP、RTMP播放器TextureView渲染。在此之前,我们先看看TextureView优缺点:

先说优点:

  1. 更高的性能:TextureView使用基于硬件加速的渲染管道,可以在GPU中进行图像处理和渲染,这比SurfaceView的软件渲染方式更高效。因此,TextureView在图像和视频渲染方面具有更好的性能。
  2. 更强的功能:TextureView可以与其他控件进行自由组合,可以在布局中灵活地放置和调整大小。而SurfaceView只能全屏显示,无法与其他控件混合使用。
  3. 更灵活的绘制方式:TextureView允许开发者直接在其上面绘制图像,通过Canvas和OpenGL ES等API,可以实现更丰富的渲染效果。
  4. 支持动画和截图:TextureView支持移动、旋转、缩放等动画,并且支持截图功能。

再说缺点:

  1. 必须在硬件加速的窗口中使用:TextureView必须在硬件加速的窗口中使用,如果设备不支持硬件加速或者硬件加速被禁用,TextureView可能无法正常工作。
  2. 占用内存较高:与SurfaceView相比,TextureView占用内存更高,这可能会影响到应用的性能,特别是在处理大型图像或视频时。
  3. 渲染线程问题:在Android 5.0以前,TextureView在主线程进行渲染,这可能导致UI卡顿。虽然在Android 5.0及以后版本中,TextureView有了单独的渲染线程,但在高GPU负荷的场景下,可能存在帧率下降的问题。
  4. 同步问题:TextureView需要在多个线程之间进行写读同步,包括CPU和GPU的同步。当同步失调时,可能会出现掉帧或吞帧导致的卡顿和抖动现象。

技术实现

本文以大牛直播SDK的Android平台SmartPlayerV2工程demo为例:

开始播放之前,CreateView()实现如下:

    /*
     * SmartPlayer.java
     * Author: daniusdk.com
     * Create rendering with different type
     */
    private boolean CreateView() {
        if (sSurfaceView != null)
            return true;

        Log.i(TAG, "CreateView");

        if (SURFACE_TYPE_NULL == surface_type_) {
            String manufacturer = Build.MANUFACTURER;
            Log.i(TAG, "CreateView, current manufacturer: " + manufacturer);

            if (is_enable_hardware_render_mode) {
                //hardware render模式,第二个参数设置为false
                sSurfaceView = NTRenderer.CreateRenderer(this, false);
            } else {
                //这个字符串可以自己定义,例如判断华为就填写huawei,魅族就填写meizu
                if ("huawei".equalsIgnoreCase(manufacturer)) {
                    sSurfaceView = NTRenderer.CreateRenderer(this, true);
                } else {
                    /*
                     * useOpenGLES2: If with true: Check if system supports openGLES, if
                     * supported, it will choose openGLES. If with false: it will set
                     * with default surfaceView;
                     */
                    sSurfaceView = NTRenderer.CreateRenderer(this, true);
                }
            }
        } else {
            if (SURFACE_TYPE_TEXTURE_VIEW == surface_type_) {
                TextureView texture_view = new TextureView(this);
                texture_view.setSurfaceTextureListener(this);
                sSurfaceView = texture_view;
            } else
                sSurfaceView= new SurfaceView(this);
        }

        if (sSurfaceView == null) {
            Log.i(TAG, "Create render failed..");
            return false;
        }

        if (is_enable_hardware_render_mode || SURFACE_TYPE_SURFACE_VIEW == surface_type_) {
            if (sSurfaceView instanceof SurfaceView) {
                SurfaceHolder surfaceHolder = ((SurfaceView)sSurfaceView).getHolder();
                if (surfaceHolder == null)
                    Log.e(TAG, "CreateView, surfaceHolder with null..");
                 else
                	surfaceHolder.addCallback(this);
            }
        }

        return true;
    }

视频流开始播放后,我们会把视频宽高信息回调上来(EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO),然后,根据获取到的宽高信息,调用adjustTextureViewAspectRatio()按比例显示窗口,如果需要铺满显示,不调用比例显示即可。

    class EventHandeV2 implements NTSmartEventCallbackV2 {
        @Override
        public void onNTSmartEventCallbackV2(long handle, int id, long param1,
                                             long param2, String param3, String param4, Object param5) {

            //Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);

            String player_event = "";

            switch (id) {
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_STARTED:
                    player_event = "开始..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTING:
                    player_event = "连接中..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED:
                    player_event = "连接失败..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTED:
                    player_event = "连接成功..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED:
                    player_event = "连接断开..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_STOP:
                    player_event = "停止播放..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO:
                    player_event = "分辨率信息: width: " + param1 + ", height: " + param2;
                    handler.post(new OnNTPlayerVideoSize((int)param1, (int)param2));
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED:
                    player_event = "收不到媒体数据,可能是url错误..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL:
                    player_event = "切换播放URL..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE:
                    player_event = "快照: " + param1 + " 路径:" + param3;

                    if (param1 == 0)
                        player_event = player_event + ", 截取快照成功";
                     else
                        player_event = player_event + ", 截取快照失败";

                    if (param4 != null && !param4.isEmpty())
                        player_event += (", user data:" + param4);

                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE:
                    player_event = "[record]开始一个新的录像文件 : " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED:
                    player_event = "[record]已生成一个录像文件 : " + param3;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING:
                    Log.i(TAG, "Start Buffering");
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_BUFFERING:
                    Log.i(TAG, "Buffering:" + param1 + "%");
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING:
                    Log.i(TAG, "Stop Buffering");
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED:
                    player_event = "download_speed:" + param1 + "Byte/s" + ", "
                            + (param1 * 8 / 1000) + "kbps" + ", " + (param1 / 1024)
                            + "KB/s";
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE:
                    Log.e(TAG, "RTSP error code received, please make sure username/password is correct, error code:" + param1);
                    player_event = "RTSP error code:" + param1;
                    break;
                 ....
            }

            if (player_event.length() > 0) {
                Log.i(TAG, player_event);
                Message message = new Message();
                message.what = PLAYER_EVENT_MSG;
                message.obj = player_event;
                handler.sendMessage(message);
            }
        }
    }

OnNTPlayerVideoSize实现如下:

    class OnNTPlayerVideoSize implements Runnable{
        int width_;
        int height_;

        OnNTPlayerVideoSize(int w, int h ) {
            this.width_ = w;
            this.height_ = h;
        }

        public void run() {
            if (this.width_ < 1 || this.height_ < 1)
                return;

            video_width_ = this.width_;
            video_height_ = this.height_;

            if (null == sSurfaceView)
                return;

            if (SURFACE_TYPE_TEXTURE_VIEW == surface_type_ && sSurfaceView instanceof TextureView)
                adjustTextureViewAspectRatio((TextureView)sSurfaceView, this.width_, this.height_);
            else if (((isHardwareDecoder&&is_enable_hardware_render_mode) || SURFACE_TYPE_SURFACE_VIEW == surface_type_)
                && sSurfaceView instanceof SurfaceView )
                adjustSurfaceViewAspectRatio((SurfaceView)sSurfaceView, this.width_, this.height_);
        }
    }

针对TextureView处理如下:

   /**
     * Invoked when a {@link TextureView}'s SurfaceTexture is ready for use.
     *
     * @param surface The surface returned by
     *                {@link android.view.TextureView#getSurfaceTexture()}
     * @param width The width of the surface
     * @param height The height of the surface
     */
    public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
        Log.i(TAG, "TextureView onSurfaceTextureAvailable w:" + width + ", h:" + height);

        if (texture_view_surface_ != null) {
            texture_view_surface_.release();
            texture_view_surface_ = null;
        }

        texture_view_surface_ = new Surface(surface);

        if (isPlaying && SURFACE_TYPE_TEXTURE_VIEW == surface_type_) {
            libPlayer.SetSurface(playerHandle, texture_view_surface_, 0, disable_codec_render_surface_, disable_sdk_render_surface_);

            if (video_width_ > 0 && video_height_ > 0 && sSurfaceView != null && (sSurfaceView instanceof TextureView))
                adjustTextureViewAspectRatio((TextureView)sSurfaceView, video_width_, video_height_);
        }
    }

    /**
     * Invoked when the {@link SurfaceTexture}'s buffers size changed.
     *
     * @param surface The surface returned by
     *                {@link android.view.TextureView#getSurfaceTexture()}
     * @param width The new width of the surface
     * @param height The new height of the surface
     */
    public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
        Log.i(TAG, "TextureView onSurfaceTextureSizeChanged w:" + width + ", h:" + height);

        if(isPlaying && SURFACE_TYPE_TEXTURE_VIEW == surface_type_) {
            if (sSurfaceView != null && (sSurfaceView instanceof TextureView) && video_width_ > 0 && video_height_ > 0)
                adjustTextureViewAspectRatio((TextureView) sSurfaceView, video_width_, video_height_);
        }
    }

    /**
     * Invoked when the specified {@link SurfaceTexture} is about to be destroyed.
     * If returns true, no rendering should happen inside the surface texture after this method
     * is invoked. If returns false, the client needs to call {@link SurfaceTexture#release()}.
     * Most applications should return true.
     *
     * @param surface The surface about to be destroyed
     */
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        Log.i(TAG, "TextureView onSurfaceTextureDestroyed");

        if(isPlaying && SURFACE_TYPE_TEXTURE_VIEW == surface_type_)
            libPlayer.SetSurface(playerHandle, null, 0, 0, 0);

        if (texture_view_surface_ != null) {
            texture_view_surface_.release();
            texture_view_surface_ = null;
        }

        return true;
    }

    /**
     * Invoked when the specified {@link SurfaceTexture} is updated through
     * {@link SurfaceTexture#updateTexImage()}.
     *
     * @param surface The surface just updated
     */
    public void onSurfaceTextureUpdated(SurfaceTexture surface) {
        // Log.i(TAG, "TextureView onSurfaceTextureUpdated");
    }

至此,RTSP|RTMP播放,我们是实现的功能如下(如不特别说明,代表Windows、Linux、Android、iOS平台均支持):

  • [支持播放协议]高稳定、超低延迟、业内首屈一指的RTSP|RTMP直播播放器SDK;
  •  [多实例播放]支持多实例播放;
  •  [事件回调]支持网络状态、buffer状态等回调;
  •  [视频格式]支持H.265、H.264,此外,还支持RTSP MJPEG播放;
  •  [音频格式]RTSP支持AAC/PCMA/PCMU、RTMP支持AAC/PCMA/PCMU/Speex;
  •  [H.264/H.265软解码]支持H.264/H.265软解,支持Enhanced RTMP H.265
  •  [H.264硬解码]Windows/Android/iOS支持特定机型H.264硬解;
  •  [H.265硬解]Windows/Android/iOS支持特定机型H.265硬解;
  •  [H.264/H.265硬解码]Android支持设置Surface模式硬解和普通模式硬解码;
  •  [RTSP模式设置]支持RTSP TCP/UDP模式设置;
  •  [RTSP TCP/UDP自动切换]支持RTSP TCP、UDP模式自动切换;
  •  [RTSP超时设置]支持RTSP超时时间设置,单位:秒;
  •  [RTSP 401认证处理]支持上报RTSP 401事件,如URL携带鉴权信息,会自动处理;
  •  [缓冲时间设置]支持buffer time设置;
  •  [首屏秒开]支持首屏秒开模式;
  •  [复杂网络处理]支持断网重连等各种网络环境自动适配;
  •  [快速切换URL]支持播放过程中,快速切换其他URL,内容切换更快;
  •  [音视频多种render机制]Android平台,视频:surfaceview/OpenGL ES,音频:AudioTrack/OpenSL ES;
  •  [实时静音]支持播放过程中,实时静音/取消静音;
  •  [实时音量调节]支持播放过程中实时调节音量;
  •  [实时快照]支持播放过程中截取当前播放画面;
  •  [只播关键帧]Windows平台支持实时设置是否只播放关键帧;
  •  [渲染角度]支持0°,90°,180°和270°四个视频画面渲染角度设置;
  •  [渲染镜像]支持水平反转、垂直反转模式设置;
  •  [等比例缩放]支持图像等比例缩放绘制(Android设置surface模式硬解模式不支持);
  •  [实时下载速度更新]支持当前下载速度实时回调(支持设置回调时间间隔);
  •  [解码前视频数据回调]支持H.264/H.265数据回调;
  •  [解码后视频数据回调]支持解码后YUV/RGB数据回调;
  •  [解码前音频数据回调]支持AAC/PCMA/PCMU数据回调;
  •  [音视频自适应]支持播放过程中,音视频信息改变后自适应;
  •  [扩展录像功能]完美支持和录像SDK组合使用。

总结

做播放器不难,做高稳定低延迟低资源占用的RTMP|RTSP直播播放器还是有点儿难度,以上是大牛直播SDK针对Android平台RTMP|RTSP播放器TextureView渲染相关的技术交流,感兴趣的开发者,也可以找我单独沟通。

  • 17
    点赞
  • 26
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值