NetEq

NetEq

NetEq

NetEq is the audio jitter buffer and packet loss concealer. The jitter buffer is an adaptive jitter buffer, meaning that the buffering delay is continuously optimized based on the network conditions. Its main goal is to ensure a smooth playout of incoming audio packets from the network with a low amount of audio artifacts (alterations to the original content of the packets) while at the same time keep the delay as low as possible.

NetEq是音频抖动缓冲和数据包丢失遮瑕膏。抖动缓冲器是一种自适应抖动缓冲器，这意味着缓冲延迟会根据网络条件不断优化。它的主要目标是确保来自网络的传入音频分组的平稳播放，同时保持尽可能低的延迟，同时具有少量的音频伪影（对分组的原始内容的更改）。

API

At a high level, the NetEq API has two main functions: InsertPacket and GetAudio.

​在高级别上，NetEq API有两个主要功能：InsertPacket和GetAudio。

InsertPacket

InsertPacket delivers an RTP packet from the network to NetEq where the following happens:

InsertPacket将RTP数据包从网络传递到NetEq，发生以下情况：

1.The packet is discarded if it is too late for playout (for example if it was reordered). Otherwize it is put into the packet buffer where it is stored until it is time for playout. If the buffer is full, discard all the existing packets (this should be rare).

1.如果数据包太晚无法播放（例如，如果它被重新排序），则会丢弃它。否则，它将被放入数据包缓冲区，在那里存储它，直到播放时间结束。如果缓冲区已满，则丢弃所有现有的数据包（这种情况应该很少见）。

2.The interarrival time between packets is analyzed and statistics is updated which is used to derive a new target playout delay. The interarrival time is measured in the number of GetAudio ‘ticks’ and thus clock drift between the sender and receiver can be accounted for.

2.分析分组之间的到达时间，并更新统计数据，该统计数据用于导出新的目标播放延迟。到达时间以GetAudio“滴答”的数量来衡量，因此可以考虑发送器和接收器之间的时钟漂移。

GetAudio

GetAudio pulls 10 ms of audio from NetEq for playout. A much simplified decision logic is as follows:

GetAudio从NetEq中提取10毫秒的音频进行播放。一个非常简化的决策逻辑如下：

1.If there is 10 ms audio in the sync buffer then return that.

1.如果在同步缓冲器中有10ms的音频，则返回该音频。

2.If the next packet is available (based on RTP timestamp) in the packet buffer then decode it and append the result to the sync buffer.

2.如果下一个分组在分组缓冲器中可用（基于RTP时间戳），则对其进行解码并将结果附加到同步缓冲器。

• Compare the current delay estimate (filtered buffer level) with the target delay and time stretch (accelerate or decelerate) the contents of the sync buffer if the buffer level is too high or too low.
• 如果缓冲区电平过高或过低，将当前延迟估计（滤波缓冲区电平）与目标延迟和同步缓冲区内容的时间拉伸（加速或减速）进行比较。
• Return 10 ms of audio from the sync buffer.
• 从同步缓冲区返回10毫秒的音频。

3.If the last decoded packet was a discontinuous transmission (DTX) packet then generate comfort noise.

3.如果最后解码的分组是不连续传输（DTX）分组，则产生舒适噪声。

4.If there is no available packet for decoding due to the next packet having not arrived or been lost then generate packet loss concealment by extrapolating the remaining audio in the sync buffer or by asking the decoder to produce it.

4.如果由于下一个分组尚未到达或丢失而没有可用于解码的分组，则通过外推同步缓冲器中的剩余音频或通过要求解码器产生它来产生分组丢失隐藏。

In summary, the output is the result one of the following operations:

总之，输出是以下操作之一的结果：

• Normal: audio decoded from a packet.
• 正常：从数据包中解码音频。
• Acceleration: accelerated playout of a decoded packet.
• 加速：解码数据包的加速播放。
• Preemptive expand: decelerated playout of a decoded packet.
• 抢占式扩展：对解码后的数据包进行减速播放。
• Expand: packet loss concealment generated by NetEq or the decoder.
• 扩展：由NetEq或解码器生成的丢包隐藏。
• Merge: audio stitched together from packet loss concealment to decoded data in case of a loss.
• 合并：在丢失的情况下，从数据包丢失隐藏到解码数据的音频拼接在一起。
• Comfort noise (CNG): comfort noise generated by NetEq or the decoder between talk spurts due to discontinuous transmission of packets (DTX).
• 舒适噪声（CNG）：由于数据包（DTX）的不连续传输，在通话突发之间由NetEq或解码器产生的舒适噪声。

Statistics

统计

There are a number of functions that can be used to query the internal state of NetEq, statistics about the type of audio output and latency metrics such as how long time packets have waited in the buffer.

有许多函数可用于查询NetEq的内部状态、有关音频输出类型的统计信息以及延迟度量，例如数据包在缓冲区中等待的时间。

• NetworkStatistics: instantaneous values or stats averaged over the duration since last call to this function.
• NetworkStatistics：自上次调用此函数以来，瞬时值或统计数据在持续时间内的平均值。
• GetLifetimeStatistics: cumulative stats that persist over the lifetime of the class.
• GetLifetimeStatistics：在类的生命周期中持续存在的累积统计数据。
• GetOperationsAndState: information about the internal state of NetEq (is only inteded to be used for testing and debugging).
• GetOperationsAndState：关于NetEq内部状态的信息（仅用于测试和调试）。

Tests and tools

测试和工具

• neteq_rtpplay: Simulate NetEq behavior based on either an RTP dump, a PCAP file or an RTC event log. A replacement audio file can also be used instead of the original payload. Outputs aggregated statistics and optionally an audio file to listen to.
• neteq_rtpplay：基于RTP转储、PCAP文件或RTC事件日志模拟neteq行为。也可以使用替换音频文件来代替原始有效载荷。输出聚合的统计信息，并可选地输出要收听的音频文件。
• neteq_speed_test: Measure performance of NetEq, used on perf bots.
• neteq_speed_test：测量性能机器人上使用的neteq的性能。
• Unit tests including bit exactness tests where RTP file is used as an input to NetEq, the output is concatenated and a checksum is calculated and compared against a reference.
• 单元测试，包括比特精确性测试，其中RTP文件被用作NetEq的输入，输出被连接，校验和被计算并与参考进行比较。

Other responsibilities

其他职责

• Dual-tone multi-frequency signaling (DTMF): receive telephone events and produce dual tone waveforms.
• 双音多频信号（DTMF）：接收电话事件并产生双音波形。
• Forward error correction (RED or codec inband FEC): split inserted packets and prioritize the payloads.
• 前向纠错（RED或编解码器带内FEC）：分割插入的数据包并对有效载荷进行优先级排序。
• NACK (negative acknowledgement): keep track of lost packets and generate a list of packets to NACK.
• NACK（否定确认）：跟踪丢失的数据包，并为NACK生成数据包列表。
• Audio/video sync: NetEq can be instructed to increase the latency in order to keep audio and video in sync.
• 音频/视频同步：可以指示NetEq增加延迟，以保持音频和视频同步。