WebRTC音频引擎NetEQ在处理音频数据时,主要用到了4块内存区域用于暂存处理过程中间的临时数据,分别是抖动缓冲区,解码缓冲区,DSP算法缓冲区和语音缓冲区。
抖动缓冲区用于暂存网络中到来的未解码的音频数据包,然后这些数据包会通过解码器解码成为pcm原始音频数据,这些pcm数据存放的位置就是解码缓冲区,在需要经过音频信号处理的情况下,NetEQ会将解码缓冲区中的数据进行拉伸、压缩或者是平滑处理,将处理结果放到算法缓冲区中,最后算法缓冲区中的数据会被塞到语音缓冲区中,声卡每隔10ms会从语音缓冲区中提取长度为10ms的语音数据拿去播放。接下来对这几种缓冲区的实现源码做分别的介绍。
抖动缓冲区
// A list of packets.
typedef std::list<Packet> PacketList;
// This is the actual buffer holding the packets before decoding.
class PacketBuffer {
public:
enum BufferReturnCodes {
kOK = 0,
kFlushed,
kNotFound,
kBufferEmpty,
kInvalidPacket,
kInvalidPointer
};
// Constructor creates a buffer which can hold a maximum of
// |max_number_of_packets| packets.
PacketBuffer(size_t max_number_of_packets, const TickTimer* tick_timer);
// Deletes all packets in the buffer before destroying the buffer.
virtual ~PacketBuffer();
// Flushes the buffer and deletes all packets in it.
virtual void Flush();
// Returns true for an empty buffer.
virtual bool Empty() const;
// Inserts |packet| into the buffer. The buffer will take over ownership of
// the packet object.
// Returns PacketBuffer::kOK on success, PacketBuffer::kFlushed if the buffer
// was flushed due to overfilling.
virtual int InsertPacket(Packet&& packet, StatisticsCalculator* stats);
// Inserts a list of packets into the buffer. The buffer will take over
// ownership of the packet objects.
// Returns PacketBuffer::kOK if all packets were inserted successfully.
// If the buffer was flushed due to overfilling, only a subset of the list is
// inserted, and PacketBuffer::kFlushed is returned.
// The last three parameters are included for legacy compatibility.
// TODO(hlundin): Redesign to not use current_*_payload_type and
// decoder_database.
virtual int InsertPacketList(
PacketList* packet_list,
const DecoderDatabase& decoder_database,
rtc::Optional<uint8_t>* current_rtp_payload_type,
rtc::Optional<uint8_t>* current_cng_rtp_payload_type,
StatisticsCalculator* stats);
// Gets the timestamp for the first packet in the buffer and writes it to the
// output variable |next_timestamp|.
// Returns PacketBuffer::kBufferEmpty if the buffer is empty,
// PacketBuffer::kOK otherwise.
virtual int NextTimestamp(uint32_t* next_timestamp) const;
// Gets the timestamp for the first packet in the buffer with a timestamp no
// lower than the input limit |timestamp|. The result is written to the output
// variable |next_timestamp|.
// Returns PacketBuffer::kBufferEmpty if the buffer is empty,
// PacketBuffer::kOK otherwise.
virtual int NextHigherTimestamp(uint32_t timestamp,
uint32_t* next_timestamp) const;
// Returns a (constant) pointer to the first packet in the buffer. Returns
// NULL if the buffer is empty.
virtual const Packet* PeekNextPacket() const;
// Extracts the first packet in the buffer and returns it.
// Returns an empty optional if the buffer is empty.
virtual rtc::Optional<Packet> GetNextPacket();
// Discards the first packet in the buffer. The packet is deleted.
// Returns PacketBuffer::kBufferEmpty if the buffer is empty,
// PacketBuffer::kOK otherwise.
virtual int DiscardNextPacket(StatisticsCalculator* stats);
// Discards all packets that are (strictly) older than timestamp_limit,
// but newer than timestamp_limit - horizon_samples. Setting horizon_samples
// to zero implies that the horizon is set to half the timestamp range. That
// is, if a packet is more than 2^31 timestamps into the future compared with
// timestamp_limit (including wrap-around), it is considered old.
virtual void DiscardOldPackets(uint32_t timestamp_limit,
uint32_t horizon_samples,
StatisticsCalculator* stats);
// Discards all packets that are (strictly) older than timestamp_limit.
virtual void DiscardAllOldPackets(uint32_t timestamp_limit,
StatisticsCalculator* stats);
// Removes all packets with a specific payload type from the buffer.
virtual void DiscardPacketsWithPay