webrtc服务器_服务器WebRTC over TCP的通道质量指标

webrtc服务器

发布和播放 (Publish and Play)

There exist two main functions of WebRTC operation on the server side in the field of streaming video: publishing and playing. In the case of publishing, the video stream is captured from the web camera and moves from the browser to the server. In the case of playing, the stream moves in the opposite direction, from the server to the browser, is decoded and played in the browser’s HTML5 <video> element on the device’s screen.

在流视频领域，服务器端有WebRTC操作的两个主要功能：发布和播放。 在发布的情况下，视频流是从网络摄像机捕获的，并从浏览器移动到服务器。 在播放的情况下，流从服务器到浏览器以相反的方向移动，并在设备屏幕上的浏览器HTML5 <video>元素中解码并播放。

UDP和TCP (UDP and TCP)

Video can move via two transport protocols: TCP or UDP. In the case of UDP, NACK RTCP feedbacks work actively, carrying the information about the lost packets, due to which it is quite a simple task to detect the UDP channel deterioration, it is boiled down to counting the NACK (Negative ACK) to determine the quality. The more NACK and PLI (Picture Loss Indicator) feedbacks there are, the more real losses there are, and the lower the channel quality is.

视频可以通过两种传输协议移动：TCP或UDP。 对于UDP，NACK RTCP反馈会主动工作，携带有关丢失数据包的信息，由于这是检测UDP信道恶化的一项非常简单的任务，因此归结为对NACK(负ACK)进行计数以确定质量。 NACK和PLI(图片丢失指示符)反馈越多，实际丢失就越多，并且信道质量越低。

不带NACK的TCP (TCP without NACK)

In this article, we are going to focus more on the TCP protocol. When WebRTC is used over TCP, NACK RTCP feedbacks are not sent, and even if they are sent they do not reflect the real picture of the losses, and it does not seem possible to determine the channel quality by the feedbacks. As it is commonly known, TCP is a transport protocol with guaranteed delivery. For this reason, in case that the channel quality deteriorates the remainder of the packets that there are in the network will be sent on the level of the transport protocol. Sooner or later they will be delivered, but a NACK will not be generated for those losses because in fact there were no losses. As the result, the packets will reach their destination with a delay. The delayed packets simply will not gather into video frames and will be discarded by the depacketizer, as a result of which the user will see a picture like this, full of artifacts:

在本文中，我们将重点放在TCP协议上。 在TCP上使用WebRTC时，不会发送NACK RTCP反馈，即使发送了NACK RTCP反馈，也不能反映出损耗的真实情况，并且似乎无法通过反馈来确定信道质量。 众所周知，TCP是具有保证传递的传输协议。 因此，在信道质量恶化的情况下，网络中存在的其余数据包将在传输协议级别上发送。 迟早会将其交付，但不会为这些损失生成NACK，因为实际上没有损失。 结果，分组将延迟到达其目的地。 延迟的数据包根本不会聚集到视频帧中，并且会被拆包器丢弃，结果用户将看到这样的图片，其中充满了伪像：

The feedbacks will show that everything is well, but the picture will contain artifacts. Below you can see screenshots of Wireshark traffic which are illustrative of the behavior of the stream being published on compressed TCP and UDP channels, as well as screenshots of Google Chrome statistics. In the screenshots you can see that in the case of TCP, the NACK metric does not grow unlike the UDP, even though the channel’s state is very bad.

反馈将显示一切正常，但是图片将包含伪影。 您可以在下面看到Wireshark流量的屏幕截图，这些屏幕截图说明了在压缩的TCP和UDP通道上发布的流的行为，以及Google Chrome统计信息的屏幕截图。 在屏幕快照中，您可以看到在TCP情况下，即使通道状态非常糟糕，NACK指标也不会像UDP一样增长。

TCP

TCP协议

UDP

UDP协议

如果存在UDP，为什么还要通过TCP进行流传输 (Why stream over TCP at all if there is UDP)

This is a reasonable question to ask. The answer is, for pushing big resolutions through the channel. For example, in the case of VR (Virtual Reality) streaming the resolutions may begin from 4k. It does not seem possible to push a stream with such a resolution and with a bitrate of about 10 Mbps into a regular channel without losses, the server spits out the UDP packets and they begin to be lost in the network in bunches, then the remainder of them begins to be sent, and so on. Big amounts of discarded video packets corrupt the video, and the net result is that the quality becomes very poor. This is the reason why WebRTC over TCP is used for delivering the video in general purpose networks and with high resolutions, such as Full HD and 4k, in order to rule out network packet losses at the expense of a slight increase in the communication latency.

这是一个合理的问题。 答案是，要通过渠道推动更高的分辨率。 例如，在VR(虚拟现实)流传输的情况下，分辨率可能从4k开始。 似乎不可能将具有这种分辨率和约10 Mbps比特率的流无损地推送到常规通道，服务器吐出UDP数据包，然后开始在网络中成束丢失它们，然后剩下的他们开始被发送，依此类推。 大量丢弃的视频数据包会破坏视频，最终结果是质量会很差。 这就是为什么使用TCP上的WebRTC来在通用网络中以高分辨率(例如Full HD和4k)传送视频，以便排除网络数据包丢失的情况，以稍微增加通信延迟为代价。

RTT用于确定信道质量 (RTT for determining the channel quality)

So, there is no metric to tell you for sure that the channel is in a very bad state. Some developers try to rely on the RTT metric but it is far from being able to work in all browsers, and does not give accurate results.

因此，没有任何指标可以确保频道处于非常不良的状态。 一些开发人员试图依靠RTT指标，但是它远不能在所有浏览器中都能使用，并且无法给出准确的结果。

Below you can find an illustration of the dependence of channel quality on RTT according to the callstat project

您可以在下面找到根据callstat项目的信道质量对RTT依赖性的图示

基于REMB的解决方案 (REMB-based solution)

We have decided to take a slightly different approach to this problem. There is the REMB working on the server side, which calculates the incoming bitrate for all incoming streams, calculates its deviation from the mean, and suggests that the browser lowers the bitrate in the case of a significant dispersion, sending specialized REMB commands over the RTCP protocol. The browser receives such a message and lowers the bitrate of the video encoder for the recommended values: this is how the protection against channel overload and incoming stream degradation works. In this way, the bitrate calculation mechanism has already been implemented on the server side. The averaging and determining the dispersion has been are realized via the Kalman filter. This allows for getting the current bitrate at any moment of time with high accuracy and for filtering any significant deviations.

我们已决定对这个问题采取稍微不同的方法。 REMB在服务器端工作，它计算所有传入流的传入比特率，计算其与均值的偏差，并建议浏览器在明显分散的情况下降低比特率，并通过RTCP发送专门的REMB命令协议。 浏览器会收到这样的消息，并将视频编码器的比特率降低为建议值：这是防止通道过载和传入流降级的方法。 这样，比特率计算机制已经在服务器端实现。 色散的平均值和确定是通过卡尔曼滤波器实现的。 这允许在任何时间以高精度获得当前比特率，并过滤任何明显的偏差。

The reader will certainly have this question, “How will it help me to know the bitrate that the server can see for the stream coming into it?” This will only let you understand that there is video coming into the server with a bitrate the value of which it was possible to determine. To evaluate the channel quality, one more component will be required.

读者肯定会有这个问题，“如何帮助我知道服务器可以看到流中的比特率？” 这只会让您了解到有视频以可能确定的比特率进入服务器。 为了评估信道质量，将需要一个以上的组件。

即将到来的比特率及其重要性 (The outcoming bitrate, and why it is important)

The statistics for the publishing WebRTC stream shows with what bitrate the video stream comes out of the browser. As an old joke goes, a site admin says when checking his assault rifle, “On my side, the bullets have flown out. The problems are on your side...” The idea of checking the channel quality involves comparing two bitrates: 1) the bitrate sent by the browser, 2) the bitrate actually received by the server.

发布WebRTC流的统计信息显示了视频流从浏览器输出的比特率。 就像一个老笑话那样，一位站点管理员在检查他的突击步枪时说：“在我这一边，子弹已经飞了出来。 问题就在您这边……”检查通道质量的想法涉及比较两个比特率：1)浏览器发送的比特率，2)服务器实际接收的比特率。

The site admin fires the bullets and calculates the speed they fly out at on his side. The server calculates the speed they are received at on its side. There is one more participant of this event, TCP, this is a superhero who is situated in the middle between the admin and the server and can stop bullets at random. For example, it can stop 10 random bullets of 100 for 2 seconds, and then let them fly again. That’s the Matrix we see here.

站点管理员发射子弹并计算它们在他身边飞出的速度。 服务器计算在其一侧接收它们的速度。 TCP还有另一个参与者，TCP，这是一个超级英雄，位于管理员和服务器之间的中间，可以随意停止项目符号。 例如，它可以停止10枚100枚随机子弹，持续2秒，然后让它们再次飞行。 这就是我们在这里看到的矩阵。

On the browser side, we take the current bitrate from the WebRTC statistics, then smooth the graph with the Kalman filter in the JavaScript implementation, and get a smoothed version of the client browser bitrate at the end of the process. Now we have practically all that we need: the client bitrate tells us how the traffic goes out of the browser, and the server bitrate tells us how that traffic is seen by the server, and with what bitrate it is received. It is obvious that if the client bitrate remains high and the server bitrate begins to shrink in relation to the client bitrate, it means that not all of the bullets have “reached the target”, and the server actually cannot see a part of the traffic that was sent to it. On this basis, we can conclude that something is wrong with the channel and it’s time to change the indicator color to red.

在浏览器端，我们从WebRTC统计信息中获取当前比特率，然后在JavaScript实现中使用Kalman过滤器对图形进行平滑处理，并在流程结束时获得客户端浏览器比特率的平滑版本。 现在我们几乎满足了所有需要：客户端比特率告诉我们流量如何从浏览器中流出，服务器比特率告诉我们服务器如何看到流量，以及接收到的比特率如何。 显而易见，如果客户端比特率仍然很高，并且服务器比特率相对于客户端比特率开始缩小，则意味着并非所有项目符号都“到达目标”，并且服务器实际上看不到一部分流量发送给它。 在此基础上，我们可以得出结论，通道出了点问题，是时候将指示器颜色更改为红色了。

还有更多 (And there’s more)

The graphs correlate but they are slightly time-shifted in relation to each other. For full correlation it is necessary to time-match the graphs in order to compare the client and server bitrate in the same point of time against historical data. The desynchronization looks approximately like this:

这些图相互关联，但是它们之间的时间偏移很小。 为了完全相关，有必要对图形进行时间匹配，以便将同一时间点的客户端和服务器比特率与历史数据进行比较。 去同步看起来像这样：

And this is what a time-synchronized graph looks like.

这就是时间同步图的样子。

让我们测试一下 (Let’s test it)

We have a little left to do, we only have to test it. Let’s publish a video stream, open it, and look at the graph of the published bitrates: on the browser side and on the server side. The graphs demonstrate practically a perfect match. And this is the name of the indicator, PERFECT.

我们还有点要做，我们只需测试一下。 让我们发布视频流，打开它，然后查看发布的比特率的图形：在浏览器端和服务器端。 这些图显示出几乎完美的匹配。 这就是指标的名称，即PERFECT。

Then, let’s begin to corrupt the channel. To do that, we can use the following free tools: winShaper for Windows or Network Link Conditioner for MacOS. They allow for compressing the channel to the preset value. For example, if we know that a stream of 640x480 can reach a speed of 1 Mbps, let’s compress it to 300 kbs. When doing so we must not forget that we are working with TCP. Let’s check the result of our test: there is inverse correlation in the graphs, and the indicator is slipping to BAD. Indeed, the browser continues to send data and is even increasing the bitrate attempting to push a new portion of traffic into the network. This data accumulates in the network in the form of retransmits and fails to reach the server, as a result the server demonstrates an inverse picture and says that the bitrate it can see has dropped. Indeed, it’s BAD.

然后，让我们开始破坏频道。 为此，我们可以使用以下免费工具：Windows的winShaper或MacOS的网络链接调节器。 它们允许将通道压缩到预设值。 例如，如果我们知道640x480的流可以达到1 Mbps的速度，那么让我们将其压缩为300 kbs。 这样做时，我们一定不要忘记我们正在使用TCP。 让我们检查一下测试结果：图表中存在反相关关系，指标滑向BAD。 确实，浏览器继续发送数据，甚至在增加比特率以试图将新的流量推入网络。 该数据以重传的形式累积在网络中，并且无法到达服务器，结果服务器显示了相反的图像，并说它可以看到的比特率下降了。 确实，这很糟糕。

We have performed quite many tests that show the correct operation of the indicator. As a result of that we have a feature that makes it possible to reliably and promptly inform the user about any problems with the channel both for stream publishing and playing (working by the same principle). Yeah, all this fuss was for this green and red lamp, PERFECT-BAD. But practice shows that this indicator is very important, and its absence, along with failure to understand the current status, can create big problems for an ordinary user of a WebRTC video streaming service.

我们已经进行了大量测试，这些测试表明指示器的正确操作。 因此，我们具有一项功能，可以可靠且Swift地将流发布和播放的频道问题通知用户(按照相同的原理)。 是的，所有这些大惊小怪都是为了这个绿色和红色的灯，完美无瑕。 但是实践表明，该指标非常重要，缺少该指标以及无法了解当前状态会给WebRTC视频流服务的普通用户带来很大的问题。

链接 (Links)

WCS 5.2 is a streaming media server for web and mobile application development

WCS 5.2是用于Web和移动应用程序开发的流媒体服务器

Publisher and player channel quality control

发布者和播放者渠道质量控制

REMB — Receiver Estimated Maximum Bitrate used for bandwidth control NACK — Negative Acknowledgement used for packet lost control and re-transmits

REMB —用于带宽控制的接收器估计的最大比特率NACK —用于丢包控制和重新发送的否定确认

翻译自: https://habr.com/en/company/flashphoner/blog/480006/

webrtc服务器