前言
在音视频通信领域,本人也算是个老兵了。从早先的H.323到SIP、以及自定义协议,从G.729、MJEPG、MPEG-4到iLBC、GIPS、H.264、VP8,从点对点通话到多媒体会议,从PC客户端、专用硬件设备到移动端、网页端......在这二十多年的变迁中,我认为,WebRTC应该是最具影响力的技术变革——正是它的出现,极大地降低了音视频通信的入门门槛。稍不留神,差点要动手写成一篇历史考古文章了。
代码
言归正传,下面分享的是一段React项目中的WebRTC点对点音视频通话实现代码。
1)本地音视频输入设备
首先,需要获取到本地的音视频输入设备,即麦克风、摄像头。通过MediaTrackConstraints可以设置相关参数,包括优先值、最大值、最小值等。需要注意的是,不仅是不同的硬件,而且不同的操作系统、不同的浏览器所支持的参数值范围也是不相同的,所以兼容性测试是不可忽视的。
async function getLocalStream(constraints: MediaStreamConstraints) {
let c: MediaStreamConstraints = {
audio: false,
video: false,
};
const audio: MediaTrackConstraints = {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
};
const video: MediaTrackConstraints = {
aspectRatio: 4 / 3,
width: { ideal: 640, max: 800 },
frameRate: { ideal: 15, max: 20 },
};
// 检查音视频输入设备
(await navigator.mediaDevices.enumerateDevices()).forEach((d) => {
switch (d.kind) {
case 'audioinput':
if (constraints.audio) {
c.audio = audio;
}
break;
case 'videoinput':
if (constraints.video) {
c.video = video;
}
break;
}
});
return await navigator.mediaDevices.getUserMedia(c);
}
2)媒体流操作
包括本地媒体流与远程对方媒体流。
/** 关闭媒体流 */
function stopStream(stream: MediaStream | undefined | null) {
stream?.getTracks().forEach((track) => track.stop());
}
3)点对点连接
点对点连接的建立,最关键的就是搞清楚主叫方与被叫方的ICECandidate与SessionDescription交换过程,也就是一般所说的信令部分,以及媒体流Track建立。WebRTC没有提供统一的信令通道的实现,需要大家自己去实现,本文就不展开了。
let peerConnection: RTCPeerConnection | null = null;
let localStream: MediaStream | null = null;
let remoteStream: MediaStream | null = null;
async function initPeerConnection(
constraints: { audio: boolean; video: boolean },
ice: RTCIceServer,
offer: RTCSessionDescription | undefined,
onCandidate: (candidate: RTCIceCandidate) => void,
) {
peerConnection = new RTCPeerConnection({ iceServers: [ice] });
localStream = await getLocalStream(constraints);
localMedia?.getTracks().forEach((t) => peerConnection!.addTrack(t));
remoteStream = new MediaStream();
peerConnection.onicecandidate = ({ candidate }) => {
candidate && onCandidate(candidate);
};
peerConnection.ontrack = async ({ track }) => {
track.onunmute = () => {
remoteMedia.addTrack(track);
};
};
if (offer) {
// 被叫方
await peerConnection.setRemoteDescription(offer);
const answerInit = await peerConnection.createAnswer();
await peerConnection.setLocalDescription(answerInit);
} else {
// 主叫方
const offerInit = await peerConnection.createOffer({
offerToReceiveAudio: constraints.audio,
offerToReceiveVideo: constraints.video,
});
await peerConnection.setLocalDescription(offerInit);
}
}
function getLocalDescription() {
return peerConnection?.localDescription;
}
async function addRemoteDescription({ description }: { description: RTCSessionDescription }) {
try {
if (peerConnection) {
if (description) {
await peerConnection.setRemoteDescription(description);
}
}
} catch (err: any) {}
}
async function addRemoteCandidate({ candidate }: { candidate: RTCIceCandidate }) {
try {
if (peerConnection) {
if (candidate) {
await peerConnection.addIceCandidate(candidate);
}
}
} catch (err: any) {}
}
function closePeerConnection() {
stopStream(remoteStream)
stopStream(localStream)
if (peerConnection) {
peerConnection.close();
peerConnection = null;
}
}
4)前端页面展示
这里以React项目为例,摘取前端页面中与点对点通话的音视频播放的相关实现代码。
const localVideoRef = useRef<HTMLVideoElement | null>(null)
const remoteVideoRef = useRef<HTMLVideoElement | null>(null)
useEffect(() => {
const init = async () => {
try {
await initPeerConnection(
{ audio: true, video: true },
ice,
offer, // 主叫方,为空;被叫方,为信令通道接收到的主叫方SessionDescription
(candidate) => {
// 信令通道发送ICECandidate
},
)
if (!offer) { // 主叫方
const sd = peerConnection.getLocalDescription()
// 信令通道发送SessionDescription
}
localVideoRef.current!.srcObject = localStream
remoteVideoRef.current!.srcObject = remoteStream
} catch (err: any) {
if (err === 'PermissionDeniedError') {
// 浏览器权限请求未被接受
}
}
}
init()
return () => {
closePeerConnection()
}
}, [])
return (
// ...
<video
ref={remoteVideoRef}
width={640}
height={480}
muted={false}
autoPlay
playsInline
/>
<video
ref={localVideoRef}
width={144}
height={108}
muted={true}
autoPlay
playsInline
/>
// ...
)
结束语
WebRTC确实使得多媒体通信领域的开发变得更简单了。这里通过点对点音视频通话,分享了一点点WebRTC的开发经验。实际上,在多媒体通信领域,还有很多的应用场景与需求,后续还会继续分享。