MediaSoup代码走读之房间创建

MediaSoup代码走读之房间创建

---

一、Mediasoup房间的创建流程

server.js 中 protoowsWebSocketServer.on(‘connectionrequest’, (info, accept, reject) 中获取wss登录信息

wss://169.254.119.31:4443/?roomId=chensong&peerId=jzey0uxt

获取房间名和peerid

然后放到异步队列中查询是否有该房间

queue.push(async () =>
{
	const room = await getOrCreateRoom({ roomId });

	// Accept the protoo WebSocket connection.
	const protooWebSocketTransport = accept();

	room.handleProtooConnection({ peerId, protooWebSocketTransport });
})

1、 没有该房间就创建该房间, 使用MediasoupWork去创建一个房间调用Room类create方法

1. 调用mediasoupwork中方法createRouter 创建Router

js 代码的逻辑

在[ mediasoup-demo/server/node_modules/mediasoup/lib]目录下Worker类中createRouter方法

internal使用uuid分配一个然后所有发送请求创建一个Router命令 worker.createRouter

  /**
     * Create a Router.
     */
    async createRouter({ mediaCodecs, appData = {} } = {}) {
        logger.debug('createRouter()');
        if (appData && typeof appData !== 'object')
            throw new TypeError('if given, appData must be an object');
        // This may throw.
        const rtpCapabilities = ortc.generateRouterRtpCapabilities(mediaCodecs);
        const internal = { routerId: uuid_1.v4() };
        await this._channel.request('worker.createRouter', internal);
        const data = { rtpCapabilities };
		// js逻辑中也创建Router对象与C++逻辑所对应的
        const router = new Router_1.Router({
            internal,
            data,
            channel: this._channel,
            payloadChannel: this._payloadChannel,
            appData
        });
        this._routers.add(router);
        router.on('@close', () => this._routers.delete(router));
        // Emit observer event.
        this._observer.safeEmit('newrouter', router);
        return router;
    }

C++代码的逻辑

在Worker类中实现非常简单 就是拿到internal唯一id作为key创建Router放到mapRouters中

case Channel::ChannelRequest::MethodId::WORKER_CREATE_ROUTER:
{
	std::string routerId;

	try
	{
		SetNewRouterIdFromInternal(request->internal, routerId);
	}
	catch (const MediaSoupError& error)
	{
		MS_THROW_ERROR("%s [method:%s]", error.buffer, request->method.c_str());
	}

	auto* router = new RTC::Router(routerId);

	this->mapRouters[routerId] = router;

	MS_DEBUG_DEV("Router created [routerId:%s]", routerId.c_str());

	request->Accept();

	break;
}

2 、 在Router中创建音频的基本设置

js 代码的逻辑

在调用Router类中方法createAudioLevelObserver 中随机一个新rtpObserverId的值 看看这个是干啥的 然后把Routerid和刚刚新创建rtpObserverId转到C++层

 /**
     * Create an AudioLevelObserver.
     */
    async createAudioLevelObserver({ maxEntries = 1, threshold = -80, interval = 1000, appData = {} } = {}) {
        logger.debug('createAudioLevelObserver()');
        if (appData && typeof appData !== 'object')
            throw new TypeError('if given, appData must be an object');
		// RouterId 和 音频通道rtpObserverId唯一值进行设置
        const internal = { ...this._internal, rtpObserverId: uuid_1.v4() };
        const reqData = { maxEntries, threshold, interval /*计时器秒数 */};
        await this._channel.request('router.createAudioLevelObserver', internal, reqData);
        const audioLevelObserver = new AudioLevelObserver_1.AudioLevelObserver({
            internal,
            channel: this._channel,
            payloadChannel: this._payloadChannel,
            appData,
            getProducerById: (producerId) => (this._producers.get(producerId))
        });
        this._rtpObservers.set(audioLevelObserver.id, audioLevelObserver);
        audioLevelObserver.on('@close', () => {
            this._rtpObservers.delete(audioLevelObserver.id);
        });
        // Emit observer event.
        this._observer.safeEmit('newrtpobserver', audioLevelObserver);
        return audioLevelObserver;
    }

C++层代码逻辑

分三层

ChannelSocket -> Worker -> Router

ChannelSocket： 是负责与js通信的模块

流程

Worker：中查找Router唯一id 后带调用 Router中HandleRequest方法的才是真正操作的动作哈 ^_

其实在Worker中OnChannelRequest方法 进行过滤了到最后调用 默认的方法哈 就是

// Any other request must be delivered to the corresponding Router.
default:
{
	try
	{
		RTC::Router* router = GetRouterFromInternal(request->internal);

		router->HandleRequest(request);
	}
	catch (const MediaSoupTypeError& error)
	{
		MS_THROW_TYPE_ERROR("%s [method:%s]", error.buffer, request->method.c_str());
	}
	catch (const MediaSoupError& error)
	{
		MS_THROW_ERROR("%s [method:%s]", error.buffer, request->method.c_str());
	}

	break;
}

从上面的可以直接看Router拿到 rtpObserverId 唯一id new出新一个对象RTC::AudioLevelObserver 中在js代码中interval是音频 在AudioLevelObserver类中Update更新的哈
非常好玩 C++发送给 js一些通知的指令哈事件

两个指令

1. volumes

2. silence

AudioLevelObserver::AudioLevelObserver(const std::string& id, json& data) : RTC::RtpObserver(id)
	{
		MS_TRACE();

		auto jsonMaxEntriesIt = data.find("maxEntries");

		// clang-format off
		if (
			jsonMaxEntriesIt == data.end() ||
			!Utils::Json::IsPositiveInteger(*jsonMaxEntriesIt)
		)
		// clang-format on
		{
			MS_THROW_TYPE_ERROR("missing maxEntries");
		}

		this->maxEntries = jsonMaxEntriesIt->get<uint16_t>();

		if (this->maxEntries < 1)
			MS_THROW_TYPE_ERROR("invalid maxEntries value %" PRIu16, this->maxEntries);

		auto jsonThresholdIt = data.find("threshold");

		if (jsonThresholdIt == data.end() || !jsonThresholdIt->is_number())
			MS_THROW_TYPE_ERROR("missing threshold");

		this->threshold = jsonThresholdIt->get<int8_t>();

		if (this->threshold < -127 || this->threshold > 0)
			MS_THROW_TYPE_ERROR("invalid threshold value %" PRIi8, this->threshold);

		auto jsonIntervalIt = data.find("interval");

		if (jsonIntervalIt == data.end() || !jsonIntervalIt->is_number())
			MS_THROW_TYPE_ERROR("missing interval");

		this->interval = jsonIntervalIt->get<uint16_t>();

		if (this->interval < 250)
			this->interval = 250;
		else if (this->interval > 5000)
			this->interval = 5000;

		this->periodicTimer = new Timer(this);
		//启动计时器鸭 Update检查的哈 ^_^
		this->periodicTimer->Start(this->interval, this->interval);
	}
	
	// Update 给 js发送信息哈 看看发一些啥东西
	void AudioLevelObserver::Update()
	{
		MS_TRACE();

		std::map<int8_t, RTC::Producer*> mapDBovsProducer;

		for (auto& kv : this->mapProducerDBovs)
		{
			auto* producer = kv.first;
			auto& dBovs    = kv.second;

			if (dBovs.count < 10)
				continue;

			auto avgDBov = -1 * static_cast<int8_t>(std::lround(dBovs.totalSum / dBovs.count));

			if (avgDBov >= this->threshold)
				mapDBovsProducer[avgDBov] = producer;
		}

		// Clear the map.
		ResetMapProducerDBovs();

		if (!mapDBovsProducer.empty())
		{
			this->silence = false;

			uint16_t idx{ 0 };
			auto rit  = mapDBovsProducer.crbegin();
			json data = json::array();

			for (; idx < this->maxEntries && rit != mapDBovsProducer.crend(); ++idx, ++rit)
			{
				data.emplace_back(json::value_t::object);

				auto& jsonEntry = data[idx];

				jsonEntry["producerId"] = rit->second->id;
				jsonEntry["volume"]     = rit->first;
			}

			Channel::ChannelNotifier::Emit(this->id, "volumes", data);
		}
		else if (!this->silence)
		{
			this->silence = true;

			Channel::ChannelNotifier::Emit(this->id, "silence");
		}
	}
	//封装好的发送到js中的代码的
	void ChannelNotifier::Emit(const std::string& targetId, const char* event, json& data)
	{
		MS_TRACE();

		MS_ASSERT(ChannelNotifier::channel, "channel unset");

		json jsonNotification = json::object();

		jsonNotification["targetId"] = targetId;
		jsonNotification["event"]    = event;
		jsonNotification["data"]     = data;

		ChannelNotifier::channel->Send(jsonNotification);
	}

下面我们就看看js接受C++业务层接受信令的处理 EnhancedEventEmitter 类是啥玩法呢

发现js中event事件的玩法就是一个监听一个发送 使用只要找到监听的地方就好了哈 那这些注册事件在那 在[mediasoup-demo/server/lib]Room类中_handleAudioLevelObserver方法中

//引入events模块
const EventEmitter = require("events");
//myEmitter继承EventEmitter类
class myEmitter extends EventEmitter{};
//实例化对象
var even=new myEmitter();
//自定义一个函数
function ev1(){
    console.log("我很帅！");
}
//通过实例化对象event的on方法，自定义一个start的监听事件
even.on("start",ev1);
//触发名字叫做start的自定义事件
even.emit("start");
 

[AudioLevelObserver extends RtpObserver_1.RtpObserver]

  _handleWorkerNotifications() {
        this._channel.on(this._internal.rtpObserverId, (event, data) => {
            switch (event) {
                case 'volumes':
                    {
                        // Get the corresponding Producer instance and remove entries with
                        // no Producer (it may have been closed in the meanwhile).
                        const volumes = data
                            .map(({ producerId, volume }) => ({
                            producer: this._getProducerById(producerId),
                            volume
                        }))
                            .filter(({ producer }) => producer);
                        if (volumes.length > 0) {
                            this.safeEmit('volumes', volumes);
                            // Emit observer event.
                            this._observer.safeEmit('volumes', volumes);
                        }
                        break;
                    }
                case 'silence':
                    {
                        this.safeEmit('silence');
                        // Emit observer event.
                        this._observer.safeEmit('silence');
                        break;
                    }
                default:
                    {
                        logger.error('ignoring unknown event "%s"', event);
                    }
            }
        });
    }

Room 注册音频的事件哈

通知客户端干嘛呢 要好好看看哈 通知客户端现在谁在发言哈

特别搞笑哈 可以发言哈volumes

不可以发信 silence

_handleAudioLevelObserver()
	{
		this._audioLevelObserver.on('volumes', (volumes) =>
		{
			const { producer, volume } = volumes[0];

			// logger.debug(
			// 	'audioLevelObserver "volumes" event [producerId:%s, volume:%s]',
			// 	producer.id, volume);

			// Notify all Peers.
			for (const peer of this._getJoinedPeers())
			{
				peer.notify(
					'activeSpeaker',
					{
						peerId : producer.appData.peerId,
						volume : volume
					})
					.catch(() => {});
			}
		});

		this._audioLevelObserver.on('silence', () =>
		{
			// logger.debug('audioLevelObserver "silence" event');

			// Notify all Peers.
			for (const peer of this._getJoinedPeers())
			{
				peer.notify('activeSpeaker', { peerId: null })
					.catch(() => {});
			}
		});
	}

看看客户端代码的操作哈

[app/lib/] RoomClient类中调用setRoomActiveSpeaker方法的实现在[app/lib/redux] stateActions类中setRoomActiveSpeaker

真正实现在[app/lib/redux/reducers]room类中 SET_ROOM_ACTIVE_SPEAKER

// 1. RoomClient类
case 'activeSpeaker':
{
	const { peerId } = notification.data;

	store.dispatch(
		stateActions.setRoomActiveSpeaker(peerId));

	break;
}

// 2. stateActions类中setRoomActiveSpeaker
export const setRoomActiveSpeaker = (peerId) =>
{
	return {
		type    : 'SET_ROOM_ACTIVE_SPEAKER',
		payload : { peerId }
	};
};

// 3. Room.js 

const room = (state = initialState, action) =>
{
	switch (action.type)
	{
		case 'SET_ROOM_URL':
		{
			const { url } = action.payload;

			return { ...state, url };
		}

		case 'SET_ROOM_STATE':
		{
			const roomState = action.payload.state;

			if (roomState === 'connected')
				return { ...state, state: roomState };
			else
				return { ...state, state: roomState, activeSpeakerId: null, statsPeerId: null };
		}

		case 'SET_ROOM_ACTIVE_SPEAKER':
		{
			const { peerId } = action.payload;
			//音频状态变化 那个玩家正在说说的
			return { ...state, activeSpeakerId: peerId };
		}

		case 'SET_ROOM_STATS_PEER_ID':
		{
			const { peerId } = action.payload;

			if (state.statsPeerId === peerId)
				return { ...state, statsPeerId: null };

			return { ...state, statsPeerId: peerId };
		}

		case 'SET_FACE_DETECTION':
		{
			const flag = action.payload;

			return { ...state, faceDetection: flag };
		}

		case 'REMOVE_PEER':
		{
			const { peerId } = action.payload;
			const newState = { ...state };

			if (peerId && peerId === state.activeSpeakerId)
				newState.activeSpeakerId = null;

			if (peerId && peerId === state.statsPeerId)
				newState.statsPeerId = null;

			return newState;
		}

		default:
			return state;
	}
};

export default room;

客户端生产者的创建 的代码

// Create mediasoup Transport for sending (unless we don't want to produce).
if (this._produce)
{
	const transportInfo = await this._protoo.request(
		'createWebRtcTransport',
		{
			forceTcp         : this._forceTcp,
			producing        : true,
			consuming        : false,
			sctpCapabilities : this._useDataChannel
				? this._mediasoupDevice.sctpCapabilities
				: undefined
		});

	const {
		id,
		iceParameters,
		iceCandidates,
		dtlsParameters,
		sctpParameters
	} = transportInfo;
	// 进行 new peerconnect 进行ICE信令交换鸭 - > 给mediasoup-clieint封装的非常好了鸭 可以看看  当是我现在没有找到回调 produce的回调函数 -> 有点苦逼哎 -> 找呗
	this._sendTransport = this._mediasoupDevice.createSendTransport(
		{
			id,
			iceParameters,
			iceCandidates,
			dtlsParameters,
			sctpParameters,
			iceServers             : [],
			proprietaryConstraints : PC_PROPRIETARY_CONSTRAINTS,
			additionalSettings 	   :
				{ encodedInsertableStreams: this._e2eKey && e2e.isSupported() }
		});
	// 创建offer    
	this._sendTransport.on(
		'connect', ({ dtlsParameters }, callback, errback) => // eslint-disable-line no-shadow
		{
			this._protoo.request(
				'connectWebRtcTransport',
				{
					transportId : this._sendTransport.id,
					dtlsParameters
				})
				.then(callback)
				.catch(errback);
		});
	// 什么地方回调这个方法鸭 
	this._sendTransport.on(
		'produce', async ({ kind, rtpParameters, appData }, callback, errback) =>
		{
			try
			{
				// eslint-disable-next-line no-shadow
				const { id } = await this._protoo.request(
					'produce',
					{
						transportId : this._sendTransport.id,
						kind,
						rtpParameters,
						appData
					});

				callback({ id });
			}
			catch (error)
			{
				errback(error);
			}
		});

	this._sendTransport.on('producedata', async (
		{
			sctpStreamParameters,
			label,
			protocol,
			appData
		},
		callback,
		errback
	) =>
	{
		logger.debug(
			'"producedata" event: [sctpStreamParameters:%o, appData:%o]',
			sctpStreamParameters, appData);

		try
		{
			// eslint-disable-next-line no-shadow
			const { id } = await this._protoo.request(
				'produceData',
				{
					transportId : this._sendTransport.id,
					sctpStreamParameters,
					label,
					protocol,
					appData
				});

			callback({ id });
		}
		catch (error)
		{
			errback(error);
		}
	});
}

客户端 RoomClient.js -> mediasoup-clinet/lib/Device.js

mediasoup-client 做适配每个中浏览器的接口 在 mediasoup-client/lib/handlers中有浏览器适配

非常有意思哈 ->run方法 中我没有找到 produce回调 只是找到connect回调 暂时不知道为啥

看一下Chrome55.js中run方法的实现哈

 run({ direction, iceParameters, iceCandidates, dtlsParameters, sctpParameters, iceServers, iceTransportPolicy, additionalSettings, proprietaryConstraints, extendedRtpCapabilities }) {
        logger.debug('run()');
        this._direction = direction;
        this._remoteSdp = new RemoteSdp_1.RemoteSdp({
            iceParameters,
            iceCandidates,
            dtlsParameters,
            sctpParameters,
            planB: true
        });
        this._sendingRtpParametersByKind =
            {
                audio: ortc.getSendingRtpParameters('audio', extendedRtpCapabilities),
                video: ortc.getSendingRtpParameters('video', extendedRtpCapabilities)
            };
        this._sendingRemoteRtpParametersByKind =
            {
                audio: ortc.getSendingRemoteRtpParameters('audio', extendedRtpCapabilities),
                video: ortc.getSendingRemoteRtpParameters('video', extendedRtpCapabilities)
            };
		//  是不是webrtc有种熟悉的感觉了鸭 peerconnect 类
        this._pc = new RTCPeerConnection(Object.assign({ iceServers: iceServers || [], iceTransportPolicy: iceTransportPolicy || 'all', bundlePolicy: 'max-bundle', rtcpMuxPolicy: 'require', sdpSemantics: 'plan-b' }, additionalSettings), proprietaryConstraints);
		// 进行ICE的交换的 然后回调上面connect方法 
        // Handle RTCPeerConnection connection status.
        this._pc.addEventListener('iceconnectionstatechange', () => {
            switch (this._pc.iceConnectionState) {
                case 'checking':
                    this.emit('@connectionstatechange', 'connecting');
                    break;
                case 'connected':
                case 'completed':
                    this.emit('@connectionstatechange', 'connected');
                    break;
                case 'failed':
                    this.emit('@connectionstatechange', 'failed');
                    break;
                case 'disconnected':
                    this.emit('@connectionstatechange', 'disconnected');
                    break;
                case 'closed':
                    this.emit('@connectionstatechange', 'closed');
                    break;
            }
        });
    }

二、mediasoup中Transport 类说明

Transport

1. WebRtcTransport
2. PlainTransport
3. PlainRtpTransport
4. PipeTransport
5. DirectTransport

js

AudioLevelObserver.js  

          用于检测声音的大小， 通过C++检测音频声音返回应用层，通过Observer接收并展示音频大小

Channel.js

          主要用于与C++部分信令通讯

Consume.js

      消费媒体数据，音频或视频

EnhancedEventEmitter.js

      EventEmitter的封装，C++底层向上层发送事件

Logger.js

      用于写日志

PipeTransport.js

      Router之间的转发

PlainRtpTransport.js

      普通的rtp传输通道，如FFmpeg等不经过浏览器rtp协议的数据传输

Producer.js

     生产媒体数据，音频或视频

Routers.js

     代表一个房间或者一个路由器

RtpObserver.js

     Rtp数据的观察者 回调用的

Transport.js

     所有传输的的基类(父类)

WebRtcRtpTransport.js

  浏览器使用的传输

Worker.js

    一个节点或者一个进程，实际应该是进程，代码中根据CPU核数启动相对   应的Worker数量;一个房间只能在一个Worker里。

Errors.js

     错误信息的定义

Index.js

     Mediasoup的库，上层引入Mediasoup最先导入的库，也为库的索引。

Ortc.js

     其与SDP相对应，以对象的形式标识SDP，如编解码参数，编解码器，帧   率等，以对象方式去存储。

ScalabilityModes.js

  一般不关心，略过

SupportedRtpCapabilities.js

  对通讯能力的支持，实际上是媒体协商相关的东西，如你支持的帧率， 码率，编解码器是什么等

Utils.js

       一些常见的工具函数

C++

SimpleConsumer

        普通RTP数据的消费者，比如有音频流和视频流每个都是SimpleConsumer，没有按类型区分，音视频流都是一样的，最简单的consumer

PipeConsumer

        不同Worker之间Router之间的数据流转，则其为接收或者消费从另外一个Worker中的Router传过来的数据

SvcConsumer

        传输时一般分为3层(核心层、拓展层、边缘层)进行传输，则其处理消费多层数据

SimulcastConsumer

        当共享者使用的是多路流时，则使用其来接收

Consumer

        为上述模块的基类(父类)

WebRtcTransport

        主要用于浏览器之间的或者浏览器与其他终端进行通讯的，这种传输数据一般是进行加密的，为了保证数据安全，它有很多安全机制，安全机制较为复杂。

PlainRtpTransport

用于普通或者自定义的rtp数据传输

PipeTransport

不同Worker之间Router之间的数据传输

TransportTuple

包括了本地的Socket，远端的Soucket ，使用的是TCP还是UDP ， 传输协议等信息存储地方

Transport

        各种传输的基类(父类)