如何在Android实现录屏直播

流媒体 专栏收录该内容
107 篇文章 2 订阅

许多开发者在做智慧教室同屏亦或会议同屏时,基于Andriod平台采集屏幕并编码推送,往往遇到各种各样的问题,以下就我们开发过程中的一些技术考量做个分享,权当抛砖引玉:

协议选择、数据来源和处理

1. 内网环境下,组播还是RTMP?

回答:这个问题,被无数的开发者问到,为此,单独写了篇博客论证:https://blog.csdn.net/renhui1112/article/details/86741428,感兴趣的可以参考下,简单来说,能RTMP的,就RTMP,如果真是内网环境下,没有并发瓶颈的同屏,可以启动内置RTSP服务(走单播),然后,其他终端拉流也不失为一个好的方案。

2. 推送分辨率如何设定或缩放?

回答:一般来说,好多Android设备,特别是高分屏,拿到的视频原始宽高非常大,如果推原始分辨率,编码和上行压力大,所以,一般建议,适当缩放,比如宽高缩放至2/3,缩放一般建议等比例缩放,此外,缩放宽高建议16字节对齐。

废话不多说,上实例代码:

    private void createScreenEnvironment() {

        sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();
        screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();

        Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "
                + screenWindowHeight);

        if (sreenWindowWidth > 800)
        {
            if (screenResolution == SCREEN_RESOLUTION_STANDARD)
            {
                scale_rate = SCALE_RATE_HALF;
                sreenWindowWidth = align(sreenWindowWidth / 2, 16);
                screenWindowHeight = align(screenWindowHeight / 2, 16);
            }
            else if(screenResolution == SCREEN_RESOLUTION_LOW)
            {
                scale_rate = SCALE_RATE_TWO_FIFTHS;
                sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);

            }
        }

        Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);

        int pf = mWindowManager.getDefaultDisplay().getPixelFormat();
        Log.i(TAG, "display format:" + pf);

        DisplayMetrics displayMetrics = new DisplayMetrics();
        mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);
        mScreenDensity = displayMetrics.densityDpi;

        mImageReader = ImageReader.newInstance(sreenWindowWidth,
                screenWindowHeight, 0x1, 6);

        mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);
    }

3. 横竖屏自动适配

回答:因为横竖屏状态下,采集的屏幕宽高不一样,如果横竖屏切换,这个时候,需要考虑到横竖屏适配问题,确保比如竖屏状态下,切换到横屏时,推拉流两端可以自动适配,横竖屏自动适配,编码器需要重启,拉流端,需要能自动适配宽高变化,自动播放。

4. 一定的补帧策略

回答:好多人不理解为什么要补帧,实际上,屏幕采集的时候,屏幕不动的话,不会一直有数据下去,这个时候,比较好的做法是,保存最后一帧数据,设定一定的补帧间隔,确保不会因为帧间距太大,导致播放端几秒都收不到数据,当然,如果服务器可以缓存GOP,这个问题迎刃而解。

5. 异常网络处理、事件回调机制

回答:如果是走RTMP,网络抖动或者其他网络异常,需要有好重连机制和状态回馈机制。

    class EventHandeV2 implements NTSmartEventCallbackV2 {
        @Override
        public void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {

            Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);

            String publisher_event = "";

            switch (id) {
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:
                    publisher_event = "开始..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:
                    publisher_event = "连接中..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:
                    publisher_event = "连接失败..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:
                    publisher_event = "连接成功..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:
                    publisher_event = "连接断开..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:
                    publisher_event = "关闭..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:
                    publisher_event = "开始一个新的录像文件 : " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:
                    publisher_event = "已生成一个录像文件 : " + param3;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:
                    publisher_event = "发送时延: " + param1 + " 帧数:" + param2;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:
                    publisher_event = "快照: " + param1 + " 路径:" + param3;

                    if (param1 == 0) {
                        publisher_event = publisher_event + "截取快照成功..";
                    } else {
                        publisher_event = publisher_event + "截取快照失败..";
                    }
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:
                    publisher_event = "RTSP服务URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:
                    publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:
                    publisher_event ="服务器不支持RTSP推送, 推送的RTSP URL: " + param3;
                    break;
            }

            String str = "当前回调状态:" + publisher_event;

            Log.i(TAG, str);

            Message message = new Message();
            message.what = PUBLISHER_EVENT_MSG;
            message.obj = publisher_event;
            handler.sendMessage(message);
        }
    }

6. 部分屏幕数据采集

回答:我们遇到的好多场景下,教室端,会拿出来3/4的区域用来投递给学生看,1/4的区域,用来做一些指令等操作,这个时候,就需要考虑屏幕区域裁剪,接口可做如下设计:

	/**
	 * 投递裁剪过的RGBA数据
	 *
	 * @param data: RGBA data
	 *
	 * @param rowStride: stride information
	 *
	 * @param width: width
	 *
	 * @param height: height
	 *
	 * @param clipedLeft: 左;  clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle,  ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);
                        //实际裁剪比例,可酌情自行调整
                        int left = 100;
                        int cliped_left = 0;

                        int top = 0;
                        int cliped_top = 0;

                        int cliped_width = width_;
                        int cliped_height = height_;

                        if(scale_rate == SCALE_RATE_HALF)
                        {
                            cliped_left = left / 2;
                            cliped_top = top / 2;

                            //宽度裁剪后,展示3/4比例
                            cliped_width = (width_ *3)/4;
                            //高度不做裁剪
                            cliped_height = height_;
                        }
                        else if(scale_rate == SCALE_RATE_TWO_FIFTHS)
                        {
                            cliped_left = left * 2 / 5;
                            cliped_top = top * 2 / 5;

                            //宽度裁剪后,展示3/4比例
                            cliped_width = (width_ *3)/4;
                            //高度不做裁剪
                            cliped_height = height_;
                        }

                        if(cliped_width % 2 != 0)
                        {
                            cliped_width = cliped_width + 1;
                        }

                        if(cliped_height % 2 != 0)
                        {
                            cliped_height = cliped_height + 1;
                        }

                        if ( (cliped_left + cliped_width) > width_)
                        {
                            Log.e(TAG, " invalid cliped region settings, cliped_left: " + cliped_left + " cliped_width:" + cliped_width + " width:" + width_);

                            return;
                        }

                        if ( (cliped_top + cliped_height) > height_)
                        {
                            Log.e(TAG, "invalid cliped region settings, cliped_top: " + cliped_top + " cliped_height:" + cliped_height + " height:" + height_);

                            return;
                        }

                        //Log.i(TAG, " clipLeft: " + cliped_left + " clipTop: " + cliped_top +  " clipWidth: " + cliped_width + " clipHeight: " + cliped_height);

                        libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_,
                                width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );

7. 文字、图片水印

回答:好多场景下,同屏者会把公司logo,和一定的文字信息展示在推送端,这个时候,需要考虑到文字和图片水印问题,具体可参考如下接口设置:

   /**
     * Set Text water-mark(设置文字水印)
     * 
     * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting font water-mark when publishing stream. </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);
    
    
    /**
     * Set Text water-mark font file name(设置文字水印字体路径)
	 *
     * @param fontFileName:  font full file name, e.g: /system/fonts/DroidSansFallback.ttf
	 *
	 * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);
	
    /**
     * Set picture water-mark(设置png图片水印)
     * 											
     * @param picPath: the picture working path, e.g: /sdcard/logo.png
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param picWidth, picHeight: picture width & height
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);

如何通过RTMP或RTSP协议发送屏幕数据

RTMP或RTSP方案,有很多开源的选择,不过大多是demo级,如果产品化,自(填)研(坑)需要很大的精力,以下以大牛直播SDK为例,介绍下如何对接数据,好多开发者提到,为什么大牛直播SDK的Android平台RTMP推送接口怎么这么多?不像一些开源或者商业RTMP推送一样,就几个接口,简单明了。废话不多说,感兴趣的可以参考下 Github

1. 初始化Publisher接口,返回推送实例句柄:

	/**
	 * Open publisher(启动推送实例)
	 *
	 * @param ctx: get by this.getApplicationContext()
	 * 
	 * @param audio_opt:
	 * if 0: 不推送音频
	 * if 1: 推送编码前音频(PCM)
	 * if 2: 推送编码后音频(aac/pcma/pcmu/speex).
	 * 
	 * @param video_opt:
	 * if 0: 不推送视频
	 * if 1: 推送编码前视频(YUV420SP/YUV420P/RGBA/ARGB)
	 * if 2: 推送编码后视频(H.264)
	 *
	 * @param width: capture width; height: capture height.
	 *
	 * <pre>This function must be called firstly.</pre>
	 *
	 * @return the handle of publisher instance
	 */
    public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt,  int width, int height);

2. 相关Event事件回调,如网络状态、实时快照、录像状态等回调:

	 /**
	  * Set callback event(设置事件回调)
	  * 
	  * @param callbackv2: callback function
	  * 
	 * @return {0} if successful
	  */
    public native int SetSmartPublisherEventCallbackV2(long handle, NTSmartEventCallbackV2 callbackv2);
    class EventHandeV2 implements NTSmartEventCallbackV2 {
        @Override
        public void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {

            Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);

            String publisher_event = "";

            switch (id) {
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:
                    publisher_event = "开始..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:
                    publisher_event = "连接中..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:
                    publisher_event = "连接失败..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:
                    publisher_event = "连接成功..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:
                    publisher_event = "连接断开..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:
                    publisher_event = "关闭..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:
                    publisher_event = "开始一个新的录像文件 : " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:
                    publisher_event = "已生成一个录像文件 : " + param3;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:
                    publisher_event = "发送时延: " + param1 + " 帧数:" + param2;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:
                    publisher_event = "快照: " + param1 + " 路径:" + param3;

                    if (param1 == 0) {
                        publisher_event = publisher_event + "截取快照成功..";
                    } else {
                        publisher_event = publisher_event + "截取快照失败..";
                    }
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:
                    publisher_event = "RTSP服务URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:
                    publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:
                    publisher_event ="服务器不支持RTSP推送, 推送的RTSP URL: " + param3;
                    break;
            }

            String str = "当前回调状态:" + publisher_event;

            Log.i(TAG, str);

            Message message = new Message();
            message.what = PUBLISHER_EVENT_MSG;
            message.obj = publisher_event;
            handler.sendMessage(message);
        }
    }

3. 检测/设置软硬编码:

	 /**
	  * Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)
	  * 
	  * @param kbps: the kbps of different resolution(25 fps).
	  * 
	  * @return {0} if successful
	  */
   public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);

	/**
	 * Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)
	 *
	 * @param kbps: the kbps of different resolution(25 fps).
	 *
	 * @return {0} if successful
	 */
	public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);

4. 设置文字水印、PNG图片水印:

    /**
     * Set Text water-mark(设置文字水印)
     * 
     * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting font water-mark when publishing stream. </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);
    
    
    /**
     * Set Text water-mark font file name(设置文字水印字体路径)
	 *
     * @param fontFileName:  font full file name, e.g: /system/fonts/DroidSansFallback.ttf
	 *
	 * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);
	
    /**
     * Set picture water-mark(设置png图片水印)
     * 											
     * @param picPath: the picture working path, e.g: /sdcard/logo.png
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param picWidth, picHeight: picture width & height
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);

5. 软编码可变码率设置(如摄像头采集编码,在设备性能允许的情况下,可以考虑软编码可变码率):

	/**
	 * Set software encode vbr mode(软编码可变码率).
	 *
	 * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
	 *
	 * is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr
	 *
	 * video_quality: vbr video quality, range with (1,50), default 23
	 *
	 * vbr_max_kbitrate: vbr max encode bit-rate(kbps)
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);

6. 帧率、软编码码率、关键帧间隔,编码速度、软编码profile等基础设定:

    /**
     * Set gop interval(设置I帧间隔)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * gopInterval: encode I frame interval, the value always > 0
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetGopInterval(long handle, int gopInterval);
    
    /**
     * Set software encode video bit-rate(设置视频软编码bit-rate)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * avgBitRate: average encode bit-rate(kbps)
     * 
     * maxBitRate: max encode bit-rate(kbps)
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);
    
    /**
     * Set fps(设置帧率)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * fps: the fps of video, range with (1,25).
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetFPS(long handle, int fps);
    
	/**
     * Set software video encoder profile(设置视频编码profile).
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * profile: the software video encoder profile, range with (1,3).
     * 
     * 1: baseline profile
     * 2: main profile
     * 3: high profile
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);
    
    
    /**
     * Set software video encoder speed(设置视频软编码编码速度)
     * 
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     * 
     * @param speed: range with(1, 6), the default speed is 6. 
     * 
     * if with 1, CPU is lowest.
     * if with 6, CPU is highest.
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);

7. 设置audio编码类型,如AAC编码、Speex编码,并设置如AAC编码码率等参数:

    /**
     * Set audio encoder type(设置音频编码类型)
     * 
     * @param type: if with 1:AAC, if with 2: SPEEX
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetAudioCodecType(long handle, int type);


	/**
	 * Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效
	 *
	 * @param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);
    
    /**
     * Set speex encoder quality(设置speex编码质量)
     * 
     * @param quality: range with (0, 10), default value is 8
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetSpeexEncoderQuality(long handle, int quality);

8. 音频处理,如自动增益控制、噪音抑制:

    /**
     * Set Audio Noise Suppression(设置音频噪音抑制)
     * 
     * @param isNS: if with 1:suppress, if with 0: does not suppress
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetNoiseSuppression(long handle, int isNS);
    
    
    /**
     * Set Audio AGC(设置音频自动增益控制)
     * 
     * @param isAGC: if with 1:AGC, if with 0: does not AGC
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetAGC(long handle, int isAGC);

9. 音频混音接口设置:

	/**
	 * 设置混音,目前支持两路音频混音
	 *
	 * @param is_mix: 1混音, 0不混音, 默认不混音
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetAudioMix(long handle, int is_mix);

10. 实时静音设置:

    /**
     * Set mute or not during publish stream(设置实时静音)
     * 
     * @param isMute: if with 1:mute, if with 0: does not mute
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetMute(long handle, int isMute);

11. 音量调整:

	/**
	 * 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量
	 *
	 * @param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量
	 *
	 * @param volume: 音量,默认是1.0,范围是[0.0, 5.0], 设置成0静音, 1音量不变
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);

12. 前置摄像头镜像:

    /**
     * Set mirror(设置前置摄像头镜像)
     * 
     * @param isMirror: if with 1:mirror mode, if with 0: normal mode
     * 
     * Please note when with "mirror mode", the publisher and player with the same echo direction
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetMirror(long handle, int isMirror);

13. 录像相关配置,如只录制音频或视频,单个文件最大size,录像存放目录,开始/暂停/停止录像:

	/**
	 * 音频录制开关, 目的是为了更细粒度的去控制录像, 一般不需要调用这个接口, 这个接口使用场景比如同时推送音视频,但只想录制视频,可以调用这个接口关闭音频录制
	 *
	 * @param is_recoder: 0: do not recorder; 1: recorder; sdk默认是1
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetRecorderAudio(long handle, int is_recoder);

	/**
	 * 视频录制开关, 目的是为了更细粒度的去控制录像, 一般不需要调用这个接口, 这个接口使用场景比如同时推送音视频,但只想录制音频,可以调用这个接口关闭视频录制
	 *
	 * @param is_recoder: 0: do not recorder; 1: recorder; sdk默认是1
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetRecorderVideo(long handle, int is_recoder);
    
    /**
     * Create file directory(创建录像存放目录)
     * 
     * @param path,  E.g: /sdcard/daniulive/rec
     * 
     * <pre> The interface is only used for recording the stream data to local side. </pre> 
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherCreateFileDirectory(String path);
    
    /**
     * Set recorder directory(设置录像存放目录)
     * 
     * @param path: the directory of recorder file.
     * 
     * <pre> NOTE: make sure the path should be existed, or else the setting failed. </pre>
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetRecorderDirectory(long handle, String path);
    
    /**
     * Set the size of every recorded file(设置单个录像文件大小,如超过最大文件大小,自动切换到下个文件录制)
     * 
     * @param size: (MB), (5M~500M), if not in this range, set default size with 200MB.
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetRecorderFileMaxSize(long handle, int size);

    /**
	* Start recorder(开始录像)
	*
	* @return {0} if successful
	*/
    public native int SmartPublisherStartRecorder(long handle);

	/**
	 * Pause recorder(暂停/恢复录像)
	 *
	 * is_pause: 1表示暂停, 0表示恢复录像, 输入其他值将调用失败
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherPauseRecorder(long handle, int is_pause);

    /**
   	* Stop recorder(停止录像)
   	*
   	* @return {0} if successful
   	*/
    public native int SmartPublisherStopRecorder(long handle);

14. 实时快照:

	 /**
	  * Set if needs to save image during publishing stream(设置是否启用快照)
	  *
	  * @param is_save_image: if with 1, it will save current image via the interface of SmartPlayerSaveImage(), if with 0: does not it
	  *
	  * @return {0} if successful
	  */
	 public native int SmartPublisherSaveImageFlag(long handle,  int is_save_image);

	 /**
	  * Save current image during publishing stream(实时快照)
	  *
	  * @param imageName: image name, which including fully path, "/sdcard/daniuliveimage/daniu.png", etc.
	  *
	  * @return {0} if successful
	  */
	 public native int SmartPublisherSaveCurImage(long handle,  String imageName);

15. 设置推送的RTMP URL:

    /**
	* Set rtmp publish stream url(设置推送的RTMP url)
	*
	* @param url: rtmp publish url.
	*
	* @return {0} if successful
	*/
    public native int SmartPublisherSetURL(long handle,  String url);

16. Android摄像头前后camera通过OnPreviewFrame()回调的数据接口:

    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        frameCount++;
        if (frameCount % 3000 == 0) {
            Log.i("OnPre", "gc+");
            System.gc();
            Log.i("OnPre", "gc-");
        }

        if (data == null) {
            Parameters params = camera.getParameters();
            Size size = params.getPreviewSize();
            int bufferSize = (((size.width | 0x1f) + 1) * size.height * ImageFormat.getBitsPerPixel(params.getPreviewFormat())) / 8;
            camera.addCallbackBuffer(new byte[bufferSize]);
        } else {
            if (isRTSPPublisherRunning || isPushingRtmp || isRecording || isPushingRtsp) {
                libPublisher.SmartPublisherOnCaptureVideoData(publisherHandle, data, data.length, currentCameraType, currentOrigentation);
            }

            camera.addCallbackBuffer(data);
        }
    }

 对应接口定义:

/**
    * Set live video data(no encoded data).
    *
    * @param cameraType: CAMERA_FACING_BACK with 0, CAMERA_FACING_FRONT with 1
    * 
    * @param curOrg:
         * PORTRAIT = 1;    //竖屏
         * LANDSCAPE = 2;    //横屏 home键在右边的情况
         * LANDSCAPE_LEFT_HOME_KEY = 3; //横屏 home键在左边的情况
    *
    * @return {0} if successful
    */
    public native int SmartPublisherOnCaptureVideoData(long handle, byte[] data, int len, int cameraType, int curOrg);

17. 部分定制设备,只支持YV12的数据:

    /**
     * YV12数据接口
     *
     * @param data: YV12 data
     *
     * @param width: 图像宽
     *
     * @param height: 图像高
     *
     * @param y_stride:  y面步长
     *
     * @param v_stride: v面步长
     *
     * @param u_stride: u面步长
     *
     * rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnYV12Data(long handle, byte[] data, int width, int height, int y_stride,  int v_stride, int u_stride, int rotation_degree);

18. 支持NV21数据接口:

nv21数据接口,除了用于常规的camera数据接入外,部分定制摄像头出来的数据发生翻转,这个接口也支持。

    /**
     * NV21数据接口
     *
     * @param data: nv21 data
     *
     * @param len: data length
     *
     * @param width: 图像宽
     *
     * @param height: 图像高
     *
     * @param y_stride:  y面步长
     *
     * @param uv_stride:  uv面步长
     *
     * rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnNV21Data(long handle, byte[] data, int len, int width, int height, int y_stride,  int uv_stride, int rotation_degree);


    /**
     * NV21数据接口
     *
     * @param data: nv21 data
     *
     * @param len: data length
     *
     * @param width: 图像宽
     *
     * @param height: 图像高
     *
     * @param y_stride:  y面步长
     *
     * @param uv_stride:  uv面步长
     *
     * rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270
     *
     * @param  is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转
     *
     * @param  is_horizontal_flip:是否水平翻转, 0不翻转, 1翻转
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnNV21DataV2(long handle, byte[] data, int len, int width, int height, int y_stride,  int uv_stride, int rotation_degree,
                                                 int is_vertical_flip, int is_horizontal_flip);

19. 支持YUV数据接入:

    /**
    * Set live video data(no encoded data).
    *
    * @param data: I420 data
    * 
    * @param len: I420 data length
    * 
    * @param yStride: y stride
    * 
    * @param uStride: u stride
    * 
    * @param vStride: v stride
    *
    * @return {0} if successful
    */
    public native int SmartPublisherOnCaptureVideoI420Data(long handle,  byte[] data, int len, int yStride, int uStride, int vStride);


20. 支持RGBA数据接入(支持裁剪后数据接入,主要用于同屏场景):

    /**
    * Set live video data(no encoded data).
    *
    * @param data: RGBA data
    * 
    * @param rowStride: stride information
    * 
    * @param width: width
    * 
    * @param height: height
    *
    * @return {0} if successful
    */
    public native int SmartPublisherOnCaptureVideoRGBAData(long handle,  ByteBuffer data, int rowStride, int width, int height);

    /**
     * 投递裁剪过的RGBA数据
     *
     * @param data: RGBA data
     *
     * @param rowStride: stride information
     *
     * @param width: width
     *
     * @param height: height
     *
     * @param clipedLeft: 左;  clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle,  ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);

    /**
     * Set live video data(no encoded data).
     *
     * @param data: ABGR flip vertical(垂直翻转) data
     *
     * @param rowStride: stride information
     *
     * @param width: width
     *
     * @param height: height
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnCaptureVideoABGRFlipVerticalData(long handle,  ByteBuffer data, int rowStride, int width, int height);

21. 支持RGB565数据接入(主要用于同屏场景):

    /**
     * Set live video data(no encoded data).
     *
     * @param data: RGB565 data
     *
     * @param row_stride: stride information
     *
     * @param width: width
     *
     * @param height: height
     *
     * @return {0} if successful
     */
    public native int SmartPublisherOnCaptureVideoRGB565Data(long handle,ByteBuffer data, int row_stride, int width, int height);

22. 支持camera数据接入(主要用于camera2接口对接):
    

/*
    *  专门为android.media.Image的android.graphics.ImageFormat.YUV_420_888格式提供的接口
    *
    * @param  width: 必须是8的倍数
    *
    * @param  height: 必须是8的倍数
    *
    * @param  crop_left: 剪切左上角水平坐标, 一般根据android.media.Image.getCropRect() 填充
    *
    * @param  crop_top: 剪切左上角垂直坐标, 一般根据android.media.Image.getCropRect() 填充
    *
    * @param  crop_width: 必须是8的倍数, 填0将忽略这个参数, 一般根据android.media.Image.getCropRect() 填充
    *
    * @param  crop_height: 必须是8的倍数, 填0将忽略这个参数,一般根据android.media.Image.getCropRect() 填充
    *
    * @param y_plane 对应android.media.Image.Plane[0].getBuffer()
    *
    * @param y_row_stride 对应android.media.Image.Plane[0].getRowStride()
    *
    * @param u_plane 对应android.media.Image.Plane[1].getBuffer()
    *
    * @param v_plane 对应android.media.Image.Plane[2].getBuffer()
    *
    * @param uv_row_stride 对应android.media.Image.Plane[1].getRowStride()
    *
    * @param uv_pixel_stride 对应android.media.Image.Plane[1].getPixelStride()
    *
    * @param  rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270
    *
    * @param  is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转
    *
    * @param  is_horizontal_flip:是否水平翻转, 0不翻转, 1翻转
    *
    * @param  scale_width: 缩放宽,必须是8的倍数, 0不缩放
    *
    * @param  scale_height: 缩放高, 必须是8的倍数, 0不缩放
    *
    * @param  scale_filter_mode: 缩放质量, 范围必须是[1,3], 传0使用默认速度
    *
    * @return {0} if successful
    */
    public native int SmartPublisherOnImageYUV420888(long handle, int width, int height,
                                                     int crop_left, int crop_top, int crop_width, int crop_height,
                                                     ByteBuffer y_plane, int y_row_stride,
                                                     ByteBuffer u_plane, ByteBuffer v_plane, int uv_row_stride, int uv_pixel_stride,
                                                     int rotation_degree, int is_vertical_flip, int is_horizontal_flip,
                                                     int scale_width, int scale_height, int scale_filter_mode);

23. 支持PCM数据接入:
 

    /**
     * 传递PCM音频数据给SDK, 每10ms音频数据传入一次
     * 
     *  @param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.
     *  @param size: pcm数据大小
     *  @param sample_rate: 采样率,当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100
     *  @param channel: 通道, 当前通道支持单通道(1)和双通道(2),推荐单通道(1)
     *  @param per_channel_sample_number: 这个请传入的是 sample_rate/100
     */
    public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);


    /**
     * 传递PCM音频数据给SDK, 每10ms音频数据传入一次
     *
     *  @param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.
     *  @param offset: pcmdata的偏移
     *  @param size: pcm数据大小
     *  @param sample_rate: 采样率,当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100
     *  @param channel: 通道, 当前通道支持单通道(1)和双通道(2),推荐单通道(1)
     *  @param per_channel_sample_number: 这个请传入的是 sample_rate/100
     */
    public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);


    /**
     * 传递PCM音频数据给SDK, 每10ms音频数据传入一次
     *
     *  @param pcm_short_array: pcm数据, short是native endian order
     *  @param offset: 数组偏移
     *  @param len: 数组项数
     *  @param sample_rate: 采样率,当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100
     *  @param channel: 通道, 当前通道支持单通道(1)和双通道(2),推荐单通道(1)
     *  @param per_channel_sample_number: 这个请传入的是 sample_rate/100
     */
    public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);

24. 支持远端PCM数据接入和混音后PCM数据接入(主要用于一对一互动):    

/**
     * Set far end pcm data
     * 
     * @param pcmdata : 16bit pcm data
     * @param sampleRate: audio sample rate
     * @param channel: auido channel
     * @param per_channel_sample_number: per channel sample numbers
     * @param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency
     * @return {0} if successful
     */
    public native int SmartPublisherOnFarEndPCMData(long handle,  ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);


    /**
     * 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次
     *
     *  @param stream_index: 当前只能传1, 传其他返回错误
     *  @param pcm_data: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.
     *  @param offset: pcmdata的偏移
     *  @param size: pcm数据大小
     *  @param sample_rate: 采样率,当前只支持{44100, 8000, 16000, 24000, 32000, 48000}
     *  @param channels: 通道, 当前通道支持单通道(1)和双通道(2)
     *  @param per_channel_sample_number: 这个请传入的是 sample_rate/100
     */
    public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);


    /**
     * 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次
     *
     *  @param stream_index: 当前只能传1, 传其他返回错误
     *  @param pcm_short_array: pcm数据, short是native endian order
     *  @param offset: 数组偏移
     *  @param len: 数组项数
     *  @param sample_rate: 采样率,当前只支持{44100, 8000, 16000, 24000, 32000, 48000}
     *  @param channels: 通道, 当前通道支持单通道(1)和双通道(2)
     *  @param per_channel_sample_number: 这个请传入的是 sample_rate/100
     */
    public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);

25. H264扩展SEI消息:

	/**
	 * 设置发送队列大小,为保证实时性,默认大小为3, 必须设置一个大于0的数
	 *
	 * @param max_size: 队列最大长度
	 *
	 * @param reserve: 保留字段
	 *
	 * NOTE: 1. 如果数据超过队列大小,将丢掉队头数据; 2. 这个接口请在 StartPublisher 之前调用
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetPostUserDataQueueMaxSize(long handle, int max_size, int reserve);

	/**
	 * 清空用户数据队列, 有些情况可能会用到,比如发送队列里面有4条消息再等待发送,又想把最新的消息快速发出去, 可以先清除掉正在排队消息, 再调用PostUserXXX
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherClearPostUserDataQueue(long handle);

	/**
	 * 发送二进制数据
	 *
	 * NOTE:
	 * 1.目前数据大小限制在256个字节以内,太大可能会影响视频传输,如果有特殊需求,需要增大限制,请联系我们
	 * 2. 如果积累的数据超过了设置的队列大小,之前的队头数据将被丢弃
	 * 3. 必须再调用StartPublisher之后再发送数据
	 *
	 * @param data: 二进制数据
	 *
	 * @param size: 数据大小
	 *
	 * @param reserve: 保留字段
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherPostUserData(long handle, byte[] data, int size, int reserve);

	/**
	 * 发送utf8字符串
	 *
	 * NOTE:
	 * 1. 字符串长度不能超过256, 太大可能会影响视频传输,如果有特殊需求,需要增大限制,请联系我们
	 * 2. 如果积累的数据超过了设置的队列大小,之前的队头数据将被丢弃
	 * 3. 必须再调用StartPublisher之后再发送数据
	 *
	 * @param utf8_str: utf8字符串
	 *
	 * @param reserve: 保留字段
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherPostUserUTF8StringData(long handle, String utf8_str, int reserve);

26. 关闭推送实例:

    /**
     * 关闭推送实例,结束时必须调用close接口释放资源
	 *
	 * @return {0} if successful
	 */
    public native int SmartPublisherClose(long handle);
  • 0
    点赞
  • 0
    评论
  • 0
    收藏
  • 一键三连
    一键三连
  • 扫一扫,分享海报

相关推荐
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值