移动端视频开发通过什么方式实现直播？十分钟带你快速了解

琦靓影芳

于 2024-04-02 12:18:58 发布

阅读量995

点赞数 23

分类专栏： 2024年程序员学习文章标签：音视频

本文链接：https://blog.csdn.net/adamjzy/article/details/137265485

版权

2024年程序员学习专栏收录该内容

239 篇文章 0 订阅

订阅专栏

@Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
}

相机的初始化就在这里啦：

public void InitCamera() {
Camera.Parameters p = _mCamera.getParameters();

Size prevewSize = p.getPreviewSize();
showlog(“Original Width:” + prevewSize.width + “, height:” + prevewSize.height);

List PreviewSizeList = p.getSupportedPreviewSizes();
List PreviewFormats = p.getSupportedPreviewFormats();
showlog(“Listing all supported preview sizes”);
for (Camera.Size size : PreviewSizeList) {
showlog(" w: " + size.width + ", h: " + size.height);
}

showlog(“Listing all supported preview formats”);
Integer iNV21Flag = 0;
Integer iYV12Flag = 0;
for (Integer yuvFormat : PreviewFormats) {
showlog(“preview formats:” + yuvFormat);
if (yuvFormat == android.graphics.ImageFormat.YV12) {
iYV12Flag = android.graphics.ImageFormat.YV12;
}
if (yuvFormat == android.graphics.ImageFormat.NV21) {
iNV21Flag = android.graphics.ImageFormat.NV21;
}
}

if (iNV21Flag != 0) {
_iCameraCodecType = iNV21Flag;
} else if (iYV12Flag != 0) {
_iCameraCodecType = iYV12Flag;
}
p.setPreviewSize(HEIGHT_DEF, WIDTH_DEF);
p.setPreviewFormat(_iCameraCodecType);
p.setPreviewFrameRate(FRAMERATE_DEF);

showlog(“_iDegrees=”+_iDegrees);
_mCamera.setDisplayOrientation(_iDegrees);
p.setRotation(_iDegrees);
_mCamera.setPreviewCallback(_previewCallback);
_mCamera.setParameters§;
try {
_mCamera.setPreviewDisplay(_mSurfaceView.getHolder());
} catch (Exception e) {
return;
}
_mCamera.cancelAutoFocus();//只有加上了这一句，才会自动对焦。
_mCamera.startPreview();
}

还记得之前初始化完成之后开始推流函数吗？

private void RtmpStartMessage() {
Message msg = new Message();
msg.what = ID_RTMP_PUSH_START;
Bundle b = new Bundle();
b.putInt(“ret”, 0);
msg.setData(b);
mHandler.sendMessage(msg);
}

Handler处理：

public Handler mHandler = new Handler() {
public void handleMessage(android.os.Message msg) {
Bundle b = msg.getData();
int ret;
switch (msg.what) {
case ID_RTMP_PUSH_START: {
Start();
break;
}
}
}
};
}

真正的推流实现原来在这里：

private void Start() {
if (DEBUG_ENABLE) {
File saveDir = Environment.getExternalStorageDirectory();
String strFilename = saveDir + “/aaa.h264”;
try {
if (!new File(strFilename).exists()) {
new File(strFilename).createNewFile();
}
_outputStream = new DataOutputStream(new FileOutputStream(strFilename));
} catch (Exception e) {
e.printStackTrace();
}
}
//_rtmpSessionMgr.Start(“rtmp://192.168.0.110/live/12345678”);
_rtmpSessionMgr = new RtmpSessionManager();
_rtmpSessionMgr.Start(_rtmpUrl); //------point 1

int iFormat = _iCameraCodecType;
_swEncH264 = new SWVideoEncoder(WIDTH_DEF, HEIGHT_DEF, FRAMERATE_DEF, BITRATE_DEF);
_swEncH264.start(iFormat); //------point 2

_bStartFlag = true;

_h264EncoderThread = new Thread(_h264Runnable);
_h264EncoderThread.setPriority(Thread.MAX_PRIORITY);
_h264EncoderThread.start(); //------point 3

_AudioRecorder.startRecording();
_AacEncoderThread = new Thread(_aacEncoderRunnable);
_AacEncoderThread.setPriority(Thread.MAX_PRIORITY);
_AacEncoderThread.start(); //------point 4
}

里面主要的函数有四个，我分别标出来了，现在我们逐一看一下。首先是point 1，这已经走到SDK里面了

public int Start(String rtmpUrl){
int iRet = 0;

_rtmpUrl = rtmpUrl;
_rtmpSession = new RtmpSession();

_bStartFlag = true;
_h264EncoderThread.setPriority(Thread.MAX_PRIORITY);
_h264EncoderThread.start();

return iRet;
}

其实就是启动了一个线程，这个线程稍微有点复杂

private Thread _h264EncoderThread = new Thread(new Runnable() {

private Boolean WaitforReConnect(){
for(int i=0; i < 500; i++){
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
if(_h264EncoderThread.interrupted() || (!_bStartFlag)){
return false;
}
}
return true;
}
@Override
public void run() {
while (!_h264EncoderThread.interrupted() && (_bStartFlag)) {
if(_rtmpHandle == 0) {
_rtmpHandle = _rtmpSession.RtmpConnect(_rtmpUrl);
if(_rtmpHandle == 0){
if(!WaitforReConnect()){
break;
}
continue;
}
}else{
if(_rtmpSession.RtmpIsConnect(_rtmpHandle) == 0){
_rtmpHandle = _rtmpSession.RtmpConnect(_rtmpUrl);
if(_rtmpHandle == 0){
if(!WaitforReConnect()){
break;
}
continue;
}
}
}

if((_videoDataQueue.size() == 0) && (_audioDataQueue.size()==0)){
try {
Thread.sleep(30);
} catch (InterruptedException e) {
e.printStackTrace();
}
continue;
}
//Log.i(TAG, “VideoQueue length=”+_videoDataQueue.size()+“, AudioQueue length=”+_audioDataQueue.size());
for(int i = 0; i < 100; i++){
byte[] audioData = GetAndReleaseAudioQueue();
if(audioData == null){
break;
}
//Log.i(TAG, “###RtmpSendAudioData:”+audioData.length);
_rtmpSession.RtmpSendAudioData(_rtmpHandle, audioData, audioData.length);
}

byte[] videoData = GetAndReleaseVideoQueue();
if(videoData != null){
//Log.i(TAG, “$$$RtmpSendVideoData:”+videoData.length);
_rtmpSession.RtmpSendVideoData(_rtmpHandle, videoData, videoData.length);
}
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
_videoDataQueueLock.lock();
_videoDataQueue.clear();
_videoDataQueueLock.unlock();
_audioDataQueueLock.lock();
_audioDataQueue.clear();
_audioDataQueueLock.unlock();

if((_rtmpHandle != 0) && (_rtmpSession != null)){
_rtmpSession.RtmpDisconnect(_rtmpHandle);
}
_rtmpHandle = 0;
_rtmpSession = null;
}
});

看18行，主要就是一个while循环，每隔一段时间去_audioDataQueue和_videoDataQueue两个缓冲数组中取数据发送给服务器，发送方法_rtmpSession.RtmpSendAudioData和_rtmpSession.RtmpSendVideoData都是Native方法，通过jni调用so库文件的内容，每隔一段时间，这个时间是多少呢？看第4行，原来是5秒钟，也就是说我们的视频数据会在缓冲中存放5秒才被取出来发给服务器，所有直播会有5秒的延时，我们可以修改这块来控制直播延时。

上面说了我们会从_audioDataQueue和_videoDataQueue两个Buffer里面取数据，那么数据是何时放进去的呢？看上面的point 2,3,4。首先是point 2，同样走进了SDK：

public boolean start(int iFormateType){
int iType = OpenH264Encoder.YUV420_TYPE;

if(iFormateType == android.graphics.ImageFormat.YV12){
iType = OpenH264Encoder.YUV12_TYPE;
}else{
iType = OpenH264Encoder.YUV420_TYPE;
}
_OpenH264Encoder = new OpenH264Encoder();
_iHandle = _OpenH264Encoder.InitEncode(_iWidth, _iHeight, _iBitRate, _iFrameRate, iType);
if(_iHandle == 0){
return false;
}

_iFormatType = iFormateType;
return true;
}

其实这是初始化编码器，具体的初始化过程也在so文件，jni调用。point 3,4其实就是开启两个线程，那我们看看线程中具体实现吧。

private Thread _h264EncoderThread = null;
private Runnable _h264Runnable = new Runnable() {
@Override
public void run() {
while (!_h264EncoderThread.interrupted() && _bStartFlag) {
int iSize = _YUVQueue.size();
if (iSize > 0) {
_yuvQueueLock.lock();
byte[] yuvData = _YUVQueue.poll();
if (iSize > 9) {
Log.i(LOG_TAG, “###YUV Queue len=” + _YUVQueue.size() + “, YUV length=” + yuvData.length);
}

_yuvQueueLock.unlock();
if (yuvData == null) {
continue;
}

if (_bIsFront) {
_yuvEdit = _swEncH264.YUV420pRotate270(yuvData, HEIGHT_DEF, WIDTH_DEF);
} else {
_yuvEdit = _swEncH264.YUV420pRotate90(yuvData, HEIGHT_DEF, WIDTH_DEF);
}
byte[] h264Data = _swEncH264.EncoderH264(_yuvEdit);
if (h264Data != null) {
_rtmpSessionMgr.InsertVideoData(h264Data);
if (DEBUG_ENABLE) {
try {
_outputStream.write(h264Data);
int iH264Len = h264Data.length;
//Log.i(LOG_TAG, “Encode H264 len=”+iH264Len);
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
try {
Thread.sleep(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
_YUVQueue.clear();
}
}

也是一个循环线程，第9行，从_YUVQueue中取出摄像头获取的数据，然后进行视频旋转，第24行，对数据进行编码，然后执行26行，InsertVideoData：

public void InsertVideoData(byte[] videoData){
if(!_bStartFlag){
return;
}
_videoDataQueueLock.lock();
if(_videoDataQueue.size() > 50){
_videoDataQueue.clear();
}
_videoDataQueue.offer(videoData);
_videoDataQueueLock.unlock();
}

果然就是插入之前提到的_videoDataQueue的Buffer。这里插入的是视频数据，那么音频数据呢？在另外一个线程，内容大致相同

private Runnable _aacEncoderRunnable = new Runnable() {
@Override
public void run() {
DataOutputStream outputStream = null;
if (DEBUG_ENABLE) {
File saveDir = Environment.getExternalStorageDirectory();
String strFilename = saveDir + “/aaa.aac”;
try {
if (!new File(strFilename).exists()) {
new File(strFilename).createNewFile();
}
outputStream = new DataOutputStream(new FileOutputStream(strFilename));
} catch (Exception e1) {
e1.printStackTrace();
}
}

long lSleepTime = SAMPLE_RATE_DEF * 16 * 2 / _RecorderBuffer.length;

while (!_AacEncoderThread.interrupted() && _bStartFlag) {
int iPCMLen = _AudioRecorder.read(_RecorderBuffer, 0, _RecorderBuffer.length); // Fill buffer
if ((iPCMLen != _AudioRecorder.ERROR_BAD_VALUE) && (iPCMLen != 0)) {
if (_fdkaacHandle != 0) {
byte[] aacBuffer = _fdkaacEnc.FdkAacEncode(_fdkaacHandle, _RecorderBuffer);
if (aacBuffer != null) {
long lLen = aacBuffer.length;

_rtmpSessionMgr.InsertAudioData(aacBuffer);
//Log.i(LOG_TAG, “fdk aac length=”+lLen+" from pcm="+iPCMLen);
if (DEBUG_ENABLE) {
try {
outputStream.write(aacBuffer);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
} else {
Log.i(LOG_TAG, “######fail to get PCM data”);
}
try {
Thread.sleep(lSleepTime / 10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Log.i(LOG_TAG, “AAC Encoder Thread ended …”);
}
};
private Thread _AacEncoderThread = null;
}

这就是通过循环将音频数据插入_audioDataQueue这个Buffer。以上就是视频采集和推流的代码分析，Demo中并没有对视频进行任何处理，只是摄像头采集，编码后推流到服务器端。
###三.第二部分：Nginx服务器搭建
流媒体服务器有诸多选择，如商业版的Wowza。但我选择的是免费的Nginx（nginx-rtmp-module）。Nginx本身是一个非常出色的HTTP服务器，它通过nginx的模块nginx-rtmp-module可以搭建一个功能相对比较完善的流媒体服务器。这个流媒体服务器可以支持RTMP和HLS。 Nginx配合SDK做流媒体服务器的原理是: Nginx通过rtmp模块提供rtmp服务, SDK推送一个rtmp流到Nginx, 然后客户端通过访问Nginx来收看实时视频流。 HLS也是差不多的原理,只是最终客户端是通过HTTP协议来访问的,但是SDK推送流仍然是rtmp的。下面是一款已经集成rtmp模块的windows版本的Nginx。下载后，即可直接使用下载链接：https://github.com/illuspas/nginx-rtmp-win32

1、rtmp端口配置配置文件在/conf/nginx.conf RTMP监听 1935 端口，启用live 和hls 两个application