前言
前些时候由于接到公司一个视频直播的研发任务,对视频直播领域的知识进行系统学习了一段时间,开发完成了基于RTMP协议的视频推拉流功能,涉及到的终端程序主要有3个版本,分别是微信小程序版、PC版和Web版。应用过程为微信小程序与PC程序可以创建直播房间操作,对方可以用以上的任何终端程序加入房间进行视频对讲。
架构
Nginx 作为视频流中转服务。
Client1 作为主播终端,发起推流并创建直播房间等待其它人加入。
Client2 作为客播终端,加入正在直播的房间并主动推送自己的流。
这样就可以完成双终端对讲的业务逻辑,Clent1 推送自己的视频流,拉取Client2的视频流,界面可以实现同时观看自己和对方的视频图像。Client2也是一样的逻辑推送自己的视频流,拉取Client1的视频流。这样双方都可以完成视频对接功能。
技术
视频直播其实就是把终端设备的硬件产生的模拟信号数据化的过程。
本实例主要是应用JavaCV中的OpenCVFrameGrabber对电脑端摄像头进行数据采集,通过FrameRecorder将视频流数据以RTMP协义推送到网络媒体服务器。接入终端通过VLC播放器EmbeddedMediaPlayerComponent 对服务器上的视频流进行拉取并渲染播放。
实现
1. 创建SpringBoot工程
不要以为SpringBoot只是开发web的框架,其实它的功能非常强大,不仅能开发B/S程序,还可以辅助开发C/S程序的。本实例就是一个纯Java的C/S程序,创建过程如下。
在Idea开发环境下创建SpringBoot的详细过程不是我的重点就不细说了,不懂的可以自已百度一下教程。
2. 安装流媒体插件
下载安装VLC播放器程序插件,此插件可以使用VLC播放器内嵌入JFrame窗口中,作为视频流拉取的容器。下载完成后我是把它放到程序根目录下,这不是必须的,看自己意愿吧。
下载地址:
https://download.csdn.net/download/xxxlllbbb/12589061
3. 增加Maven引用
由于篇幅的原因这里我只贴出关键性的的Maven引用,如果想要完成POM文件可以给我留言。
<dependency>
<groupId>org.creation</groupId>
<artifactId>common.stream</artifactId>
<version>1.0.0</version>
</dependency>
<!-- javavc引用 -->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv-platform</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.56</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.56</version>
</dependency>
<!-- VLC播放器 -->
<dependency>
<groupId>uk.co.caprica</groupId>
<artifactId>vlcj</artifactId>
<version>3.12.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.ini4j</groupId>
<artifactId>ini4j</artifactId>
<version>0.5.4</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.4</version>
</dependency>
视频流推拉
视频流主要拉取窗口代码如下:
package com.rtmp.jframe;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.rtmp.Application;
import com.rtmp.jframe.event.impl.HomeCloseEventImpl;
import com.rtmp.jframe.event.impl.RootWindowCloseEventImpl;
import com.rtmp.utils.ControlUtils;
import com.rtmp.utils.Person;
import com.rtmp.utils.Result;
import com.rtmp.utils.Room;
import com.rtmp.view.JImagePanel;
import com.sun.jna.NativeLibrary;
import okhttp3.*;
import org.creation.common.stream.rtmp.*;
import org.creation.common.string.StringUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.ApplicationContext;
import org.springframework.stereotype.Component;
import uk.co.caprica.vlcj.binding.LibVlc;
import uk.co.caprica.vlcj.component.EmbeddedMediaPlayerComponent;
import uk.co.caprica.vlcj.discovery.NativeDiscovery;
import uk.co.caprica.vlcj.player.embedded.EmbeddedMediaPlayer;
import uk.co.caprica.vlcj.runtime.RuntimeUtil;
import javax.sound.midi.Soundbank;
import javax.swing.*;
import java.awt.*;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.util.List;
@Component
public class HomeView extends JFrame {
@Value("${rtmp.get}")
private String rtmpGet;
@Value("${rtmp.remove}")
private String rtmpRemove;
@Value("${rtmp.delete}")
private String rtmpDelete;
@Value("${rtmp.push}")
private String rtmpPushUrl;
@Value("${rtmp.pull}")
private String rtmpPullUrl;
private JImagePanel pushPlayer;
private EmbeddedMediaPlayerComponent pullPlayer;
private String playerPath = "D:\\\\workspaceIdea\\com.rtmp.app\\player";
private Pusher pusher;
JButton btn1;
JButton btnBg;
private int width = 815;
private int height = 600;
private JLayeredPane layeredPanel;
@Autowired
SystemView systemView;
@Autowired
ListView listView;
@Autowired
HomeView homeView;
private int k = 0;
private JLabel jz;
private String pushId;
private String pullId;
private String roomId;
private String roomType;
//重写这个方法
@Override
protected void processWindowEvent(WindowEvent e) {
if (e.getID() == WindowEvent.WINDOW_CLOSING) {
listView.setVisible(true);
homeView.setVisible(false);
try {
if (pusher != null) {
pusher.close();
}
if (pullPlayer != null) {
getMediaPlayer().stop();
}
if (roomType.equals("add")) {
Request.Builder reqBuild = new Request.Builder();
HttpUrl.Builder urlBuilder = HttpUrl.parse(systemView.serviceUrl + rtmpRemove).newBuilder();
urlBuilder.addQueryParameter("roomid", roomId);
urlBuilder.addQueryParameter("id", pushId);
reqBuild.url(urlBuilder.build());
Request request = reqBuild.build();
OkHttpClient okHttpClient = new OkHttpClient();
Response response1 = okHttpClient.newCall(request).execute();
} else if (roomType.equals("create")) {
Request.Builder reqBuild = new Request.Builder();
HttpUrl.Builder urlBuilder = HttpUrl.parse(systemView.serviceUrl + rtmpDelete).newBuilder();
urlBuilder.addQueryParameter("id", roomId);
reqBuild.url(urlBuilder.build());
Request request = reqBuild.build();
OkHttpClient okHttpClient = new OkHttpClient();
Response response1 = okHttpClient.newCall(request).execute();
}
} catch (Exception c) {
}
return; //直接返回,阻止默认动作,阻止窗口关闭
}
super.processWindowEvent(e); //该语句会执行窗口事件的默认动作(如:隐藏)
}
public HomeView() {
super("Cre视频直播组件");
setResizable(false);
// setUndecorated(true);//去除标题栏
// getRootPane().setWindowDecorationStyle(JRootPane.PLAIN_DIALOG); //采用指定的窗口装饰风格
setDefaultCloseOperation(JFrame.DO_NOTHING_ON_CLOSE);
enableEvents(AWTEvent.WINDOW_EVENT_MASK);
getContentPane().setLayout(null);
RootWindowCloseEventImpl rootCloseEvt = new RootWindowCloseEventImpl();
addWindowListener(rootCloseEvt);
Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
setBounds((screenSize.width - width) / 2, (screenSize.height - height) / 2, width, height);
initializing();
}
private void initializing() {
layeredPanel = new JLayeredPane();
layeredPanel.setLayout(null);
layeredPanel.setBounds(0, 0, width, height);
layeredPanel.setOpaque(false);
getContentPane().add(layeredPanel);
/**
* 增加拉流器面板
*/
initializingMediaPlayer();
btnBg = new JButton();
btnBg.setBounds(0, 0, width, height);
btnBg.setBorderPainted(false);
btnBg.setBackground(Color.black);
btnBg.setIcon(new ImageIcon(ControlUtils.class.getResource(String.format("/image/%s", "bg.jpg"))));
layeredPanel.add(btnBg, new Integer(101));
/**
* 增加推流器面板
*/
pushPlayer = ControlUtils.addPanelOfContentPane(0, 0, 200, 200, null, layeredPanel, JLayeredPane.MODAL_LAYER);
/**
* 关闭推流板
*/
btn1 = new JButton();
btn1.setBounds(180, 2, 20, 20);
btn1.setBorderPainted(false);
btn1.setContentAreaFilled(false);
btn1.setIcon(new ImageIcon(ControlUtils.class.getResource(String.format("/image/%s", "5.png"))));
layeredPanel.add(btn1, new Integer(400));
btn1.addActionListener(e -> {
layeredPanel.remove(pushPlayer);
btn1.setVisible(false);
});
/**
* @增加开始按钮
*/
ControlUtils.addButtonOfContentPane("1.png", layeredPanel, e -> {
if (!StringUtil.isEmpty(pushId)) {
// invoke(pushId);
invokePusher();
invokePuller();
btnBg.setVisible(false);
} else {
// btn1.setVisible(false);
String pushUrl = systemView.push_text.getText();
String pullUrl = systemView.pull_text.getText();
rtmpPushUrl = pushUrl + "/" + pushId;
rtmpPullUrl = pullUrl + "/" + pullId;
invokePusher();
invokePuller();
btnBg.setVisible(false);
}
}, null);
/**
* @增加结束按钮
*/
ControlUtils.addButtonOfContentPane("0.png", layeredPanel, e -> {
try {
if (pusher != null) {
pusher.close();
}
// if (pullPlayer != null) {
// getMediaPlayer().stop();
// }
} catch (Exception ex) {
ex.printStackTrace();
}
}, btn -> {
btn.setBounds(250, 450, 64, 64);
});
}
public void invoke2(String pushId, String roomId, String pullId) {
if (pusher != null) {
pusher = null;
}
this.roomId = roomId;
roomType = "add";
btn1.setVisible(false);
rtmpPushUrl = systemView.pushUrl;
rtmpPullUrl = systemView.pullUrl;
this.pushId = pushId;
this.pullId = pullId;
jz = new JLabel("加载中", JLabel.CENTER);
Font font = new Font("宋体", Font.BOLD, 25);//创建1个字体实例
jz.setForeground(Color.RED);
jz.setFont(font);
jz.setBounds(0, 0, width, height);
layeredPanel.add(jz, new Integer(900));
// 线程的另一种实现方法,也可以使用匿名的内部类
Thread2 thread2 = new Thread2();
thread2.start();
}
class Thread2 extends Thread {
@Override
public void run() {
rtmpPushUrl = rtmpPushUrl + "/" + pushId;
invokePusher();
btn1.setVisible(true);
jz.setVisible(false);
rtmpPullUrl = rtmpPullUrl + "/" + pullId;
invokePuller();
btnBg.setVisible(false);
}
}
class Thread1 extends Thread {
@Override
public void run() {
rtmpPushUrl = rtmpPushUrl + "/" + pushId;
invokePusher();
btn1.setVisible(true);
if (!StringUtil.isEmpty(pushId)) {
asyncSend();
}
}
}
public void invoke(String id) {
if (pusher != null) {
pusher = null;
}
btnBg.setVisible(true);
k = 0;
roomId = "room_rtmp_" + id;
roomType = "create";
btn1.setVisible(false);
rtmpPushUrl = systemView.pushUrl;
rtmpPullUrl = systemView.pullUrl;
pushId = id;
jz = new JLabel("加载中", JLabel.CENTER);
Font font = new Font("宋体", Font.BOLD, 25);//创建1个字体实例
jz.setForeground(Color.RED);
jz.setFont(font);
jz.setBounds(0, 0, width, height);
layeredPanel.add(jz, new Integer(900));
// 线程的另一种实现方法,也可以使用匿名的内部类
Thread1 thread1 = new Thread1();
thread1.start();
}
private void initializingMediaPlayer() {
String path = getAppPath(HomeView.class);
if (path.indexOf("BOOT-INF") > -1) {
playerPath = path;
playerPath += "/classes/player";
playerPath = playerPath.substring(0, playerPath.substring(0, playerPath.indexOf(".jar")).lastIndexOf("/")) + "/player";
}
NativeLibrary.addSearchPath(RuntimeUtil.getLibVlcLibraryName(), playerPath);
LibVlc.INSTANCE.libvlc_get_version();
new NativeDiscovery().discover();
pullPlayer = new EmbeddedMediaPlayerComponent();
pullPlayer.setBounds(0, 0, width, height);
layeredPanel.add(pullPlayer, JLayeredPane.DEFAULT_LAYER);
}
private EmbeddedMediaPlayer getMediaPlayer() {
return pullPlayer.getMediaPlayer();
}
/**
* 开始拉流
*/
private void invokePuller() {
getMediaPlayer().playMedia(rtmpPullUrl);
}
/**
* 开始推流
*/
private void invokePusher() {
//System.out.println(rtmpPushUrl);
try {
if (pusher == null) {
pusher = new Pusher(rtmpPushUrl);
pusher.invoke(bg -> {
pushPlayer.setBackground(bg);
pushPlayer.validate();
pushPlayer.repaint();
});
}
pusher.start();
} catch (java.lang.Exception ex) {
ex.printStackTrace();
}
}
private void asyncSend() {
Request.Builder reqBuild = new Request.Builder();
HttpUrl.Builder urlBuilder = HttpUrl.parse(systemView.serviceUrl + rtmpGet).newBuilder();
urlBuilder.addQueryParameter("id", roomId);
reqBuild.url(urlBuilder.build());
Request request = reqBuild.build();
OkHttpClient okHttpClient = new OkHttpClient();
Call call = okHttpClient.newCall(request);
//异步处理
call.enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
// Log.e("onFailure", "onFailure");
}
@Override
public void onResponse(Call call, Response response) throws IOException {
Result result = JSON.parseObject(response.body().string(), Result.class);
Room room = JSON.parseObject(result.getResult().toString(), Room.class);
if (k == 0) {
List<Person> personList = room.getPersons();
if (personList.size() > 1) {
k = 1;
layeredPanel.remove(jz);
for (int j = 0; j < personList.size(); j++) {
Person person = personList.get(j);
if (!person.getId().equals(pushId)) {
pullId = person.getId();
}
}
rtmpPullUrl += "/" + pullId;
invokePuller();
btnBg.setVisible(false);
} else {
asyncSend();
}
}
}
});
}
public static String getAppPath(Class cls) {
//检查用户传入的参数是否为空
if (cls == null)
throw new java.lang.IllegalArgumentException("参数不能为空!");
ClassLoader loader = cls.getClassLoader();
//获得类的全名,包括包名
String clsName = cls.getName() + ".class";
//获得传入参数所在的包
Package pack = cls.getPackage();
String path = "";
//如果不是匿名包,将包名转化为路径
if (pack != null) {
String packName = pack.getName();
//此处简单判定是否是Java基础类库,防止用户传入JDK内置的类库
if (packName.startsWith("java.") || packName.startsWith("javax."))
throw new java.lang.IllegalArgumentException("不要传送系统类!");
//在类的名称中,去掉包名的部分,获得类的文件名
clsName = clsName.substring(packName.length() + 1);
//判定包名是否是简单包名,如果是,则直接将包名转换为路径,
if (packName.indexOf(".") < 0) path = packName + "/";
else {//否则按照包名的组成部分,将包名转换为路径
int start = 0, end = 0;
end = packName.indexOf(".");
while (end != -1) {
path = path + packName.substring(start, end) + "/";
start = end + 1;
end = packName.indexOf(".", start);
}
path = path + packName.substring(start) + "/";
}
}
//调用ClassLoader的getResource方法,传入包含路径信息的类文件名
java.net.URL url = loader.getResource(path + clsName);
//从URL对象中获取路径信息
String realPath = url.getPath();
//去掉路径信息中的协议名"file:"
int pos = realPath.indexOf("file:");
if (pos > -1) realPath = realPath.substring(pos + 5);
//去掉路径信息最后包含类文件信息的部分,得到类所在的路径
pos = realPath.indexOf(path + clsName);
realPath = realPath.substring(0, pos - 1);
//如果类文件被打包到JAR等文件中时,去掉对应的JAR等打包文件名
if (realPath.endsWith("!"))
realPath = realPath.substring(0, realPath.lastIndexOf("/"));
/*------------------------------------------------------------
ClassLoader的getResource方法使用了utf-8对路径信息进行了编码,当路径
中存在中文和空格时,他会对这些字符进行转换,这样,得到的往往不是我们想要
的真实路径,在此,调用了URLDecoder的decode方法进行解码,以便得到原始的
中文及空格路径
-------------------------------------------------------------*/
try {
realPath = java.net.URLDecoder.decode(realPath, "utf-8");
} catch (Exception e) {
throw new RuntimeException(e);
}
return realPath;
}//getAppPath定义结束
}
playerPath是VLC播放器程序插件安装路径。
EmbeddedMediaPlayerComponent就是VLC播放器管理组件,通过它可以调用播放器对RTMP流进行拉取。
rtmpPushUrl设置程序运行以后的视频流推送地址。
rtmpPullUrl设置程序运行以后的视频流拉取地址。
运行
实例程序下载
https://download.csdn.net/download/xxxlllbbb/12589559