WebRTC Transport-CC协议介绍

1. 产生背景

Transport-CC全称为Transport-wide Congestion Control，WebRTC最新的拥塞控制算法基于Transport-CC在发送端做带宽估计，其原理是发送端对RTP报文做记录，接收端根据RTP报文的到达时间构造RTCP报文，然后反馈给发送端，后者根据Feedbacks实现丢包率和RTT的统计。

Transport CC和 REMB(Receiver Estimated Maximum Bitrate，接收端估计的最大比特率）是WebRTC中用于带宽估计和拥塞控制的两种机制，它们各自在设计理念和工作方式上有所不同。特别地，Transport CC在发送端进行带宽估计，而REMB在接收端进行。相比REMB，Transport CC的优点在于：

1. 更加及时的反馈：Transport CC通过发送端接收来自接收端的传输报告，可以更快速地做出调整。这是因为在Transport CC机制下，每个发送的RTP数据包都会由接收端确认，从而允许发送端直接和实时地监控每个包的传输情况，包括丢包和到达时间。这种及时的报告机制使得带宽估计更加灵敏和准确。
2. 适应性和可伸缩性：Transport CC设计为可以自适应不同的网络状况，并且由于其控制机制直接集成在发送端，使得调节算法可以更灵活地适应网络状况的变化。此外，它为多流和多接收者场景下的带宽共享提供了更好的机制。
3. 提升包的传输效率：因为Transport CC工作机制是基于发送端的决策，所以它能更有效地管理和调节每个数据包的发送速率。这有助于减少丢包现象并降低延迟，尤其是当网络条件频繁变化时。
4. 降低接收端的CPU开销：Transport CC的带宽估计是在发送端完成，在不增加接收端计算负担的情况下，为接收端提供更优化的数据流。这对于计算资源有限的接收设备来说是一个重要优势。

尽管Transport CC提供了许多优点，但实际应用中两种机制选择的优劣往往依赖于具体的应用场景、网络环境以及实现的具体细节。在某些情况下，两者甚至可以结合使用，以实现更优的网络流量控制和带宽利用。

2. RTP&Transport-CC Feedback交互图

3. 协议格式

Transport-CC协议格式的完整描述在IETF的官网上：https://datatracker.ietf.org/doc/html/draft-holmer-rmcat-transport-wide-cc-extensions-01。以下内容直接参考了该原始文档。WebRTC中为了使用Transport-CC，需要用到RTP报头扩展以及增加新的RTCP类型。这里我们介绍下Transport-CC相关的RTP Header Extension以及新的RTCP类型。

3.1. 发送端启用RTP Transport Sequence Number

3.1.1. Transport Sequence Number的价值

RFC(https://datatracker.ietf.org/doc/html/rfc3550#section-5.1)定义了RTP的固定头部：

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       sequence number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

其中的Sequence Number字段是RTP报文的序列号，一般情况下一个传输通道只包含一路视频流，这个Sequence Number能满足大多数需求。但是在一些情况下，我们需要传输多个视频流，这些视频流复用一个传输通道，例如我们通常说的推送大小流，此时两路视频的RTP Sequence Number是单独计数的。例如此时大流为B(Big)，小流为S(Small)，它们的RTP包记为B(n)，S(n)，n表示Sequence Number，视频流按如下形式传输：

B(1),B(2),S(1),S(2),B(3),B(4,S(3),S(4)

我们需要对传输通道做整体的带宽估计和带宽分配，例如带宽估计为2M，为大流分配1M码率，为小流分配200K码率，此时需要统计通道维度的丢包率和RTT，于是Sequence Number就不够用了。Transport-CC为解决这个问题，使用了RTP报头扩展，用于记录Transport Sequence Number，同一个传输通道下的所有流使用统一的Transport Sequence Number进行计数，例如：

我们使用前面的例子，视频流B与S，它们的RTP包记为B(n，m)，S(n,m)，N表示Sequence Number，m表示Transport Sequence Number，视频流按如下形式传输：
B(1,1),B(2,2),S(1,3),S(2,4),B(3,5),B(4,6),S(3,7),S(4,8)

这样进行带宽估计时，通过Transport Sequence Number我们就能得到这条传输通道下所有数据包的情况了。

RTP Transport Sequence Number对应的扩展头部定义如下(以0xBEDE固定字段开头)：

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       0xBE    |    0xDE       |           length=1            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  ID   | L=1   |transport-wide sequence number | zero padding  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Transport Sequence Number占两个字节，对于一个刚初始化的传输通道，该字段从一个base sequence number开始递增(在到达0xFFFF后会出现翻转)。由于按4字节对齐，所以还有值为0的填充数据。

这里我们看下Wireshark中对带Transport Sequence Number的RTP报头扩展的解析：

这里的Extension data字段为0x0028，可知该RTP包的Transport Sequence Number为0x0028(大端，网络字节序)。

3.1.2. Enable Transport Sequence Number

RTP的Transport Sequence Number需要通过SDP协商才能开启，默认情况下是关闭的。

SDP的对应选项为：

a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01

即使SDP显式开启了Transport Sequence Number，WebRTC也不会要求每个RTP报文都被接收端应答，在保证对丢包率、RTT的计算具备统计学意义的基础上，WebRTC选取了部分RTP报文作为样本，对这部分样本WebRTC开启了TransportSequenceNumber这个扩展项。因此，每个RTP报文在发送的时候都会被检查是否已开启TransportSequenceNumber，开启了才会在RTP Header中加Transport Sequence Number的扩展，接收端看到该扩展才会在TCC Feedback中加入该RTP报文的应答信息，否则该RTP报文的应答信息就不会在接收端的TCC Feedback中出现。(Note：哪些报文被Enable，哪些不被Enable，这个怎么决定呢？)

3.2. 接收端构造Transport-CC Feedback

3.2.1. 格式定义

Transport CC Feedback的协议格式如下，头12字节是RTCP报文的标准头部，从base sequence number开始是Transport CC Feedback的协议内容。

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|  FMT=15 |    PT=205     |           length              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     SSRC of packet sender                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      SSRC of media source                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|      base sequence number     |      packet status count      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 reference time                | fb pkt. count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          packet chunk         |         packet chunk          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.                                                               .
.                                                               .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         packet chunk          |  recv delta   |  recv delta   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.                                                               .
.                                                               .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           recv delta          |  recv delta   | zero padding  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

• FMT：5bits，Feedback message type(FMT)固定为15；
• PT：8bits，属于传输层的Feedback Messages，payload type(PT)为205；
• base sequence number：2字节，Transport Feedback包中记录的第一个RTP包的transport sequence number，该字段会一直递增直到0xFFFF，然后出现翻转从0开始重新递增。所以，在反馈的各个TransportFeedback RTCP包中，这个字段有可能比之前的RTCP报文要小；
• packet status count：2字节，表示这个Transport Feedback包记录了多少个RTP报文。这些RTP的transport sequence number以base sequence number为基准。例如，第一个RTP报文的transport sequence number为base sequence number，那么第二个RTP报文的transport sequence number为base sequence number+1；
• reference time：3字节，表示参考时间，以64ms为单位，RTCP包记录的RTP包到达时间信息以这个reference time为基准进行计算；
• feedback packet count：1字节，用于计数发送的每个Transport Feedback报文，相当于RTCP包的序列号，用于检测Transport Feedback的丢包情况；
• packet chunk：2字节，记录RTP报文的到达状态，记录的这些RTP包transport sequence number通过base sequence number计算得到；
• recv delta: 8bits或者16bits，对于"packet received"状态的包，也就是收到的RTP包，在recv delta列表中添加对应的的到达时间间隔信息，用于记录RTP报文的到达时间。通过前面的reference time以及recv delta信息，我们就可以得到RTP报文到达的绝对时间。

3.2.2. RTP Packet Status

RTP报文状态有如下四种情况，二进制对应的编号如下：

• 00：Packet not received （包未收到）
• 01：Packet received, small delta （包收到，间隔时间很小 ）
• 10：Packet received, large or negative delta（ 包收到，间隔时间很大或者为负数）
• 11：[Reserved], packet received, w/o recv delta (包收到了，但是没有间隔时间)

3.2.3. Packet Chunk

packet chunk用2个字节描述RTP报文的到达状态，它有两种表示方式：

• Run Length Chunk
• Status Vector Chunk

这两种表示方式通过第一个比特位来标识：

• 0 ：Run Length Chunk
• 1 ：Status Vector Chunk

3.2.4. Run Length Chunk

Run Length Chunk用到了行程编码的思想，其编码原理是把数据看成一个线性序列，对于连续的重复数据块采用的压缩策略是用一个数字表示数据块重复的次数，然后在这个数字后面存储对应的数据本身。举个例子，原始数据为A-A-A-A-A-B-B-C-D，

• 压缩前：A-A-A-A-A-B-B-C-D（0x41-0x41-0x41-0x41-0x41-0x42-0x42-0x43-0x44）
• 压缩后： 5-A-2-B-1-C-1-D（0x05-0x41-0x02-0x42-0x01-0x43-0x01-0x44）

Run Length Chunk格式为：

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|T| S |       Run Length        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

各字段含义如下：

• T (1 bit) ：chunk type，取值为0
• S (2 bit) ：packet status symbol，2bits ，表示包的到达状态
• Run Length (13 bit)：表示有多少个连续报文处于S字段所描述的状态

例1：如下图，0表示这是Run Length Chunk的表示方式，00代表包未收到"packet not received"，后面13个比特值为221，表示共有221连续报文处于未收到的状态。

    0                   1
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|0 0|0 0 0 0 0 1 1 0 1 1 1 0 1|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

例2：如下图，0表示这是Run Length Chunk的表示方式，11代表"packet received, w/o recv delta"，共有24个连续报文处于“收到，但没有间隔时间“的状态

    0                   1
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|1 1|0 0 0 0 0 0 0 0 1 1 0 0 0|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

3.2.5. Status Vector Chunk

Status Vector Chunk格式如下：

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|T|S|       symbol list         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

各字段的含义如下：

• T(1 bit): chunk type，取值为1；
• S(1 bit): symbol size符号长度，0表示用1个比特标识报文状态，1表示用2个比特标识报文状态；
• Symbol list(14 bits)：符号表，描述了N个包的到达状态，N的数量取决于S 的值，当 S = 0 时 N = 14，每个符号为1个比特，0代表没收到，1代表收到了；当 S = 1 时 N = 7，每个符号为 2 个比特，二进制取值分别为00，01，10，11，表示包的4种状态。

例1：如下图，第一比特的1表示这是Status Vector Chunk的表示方式，第二比特的0表示接下来用1个比特标识报文状态，后面14个比特的含义是：第一个包状态为"packet not received"（0），接着后面5个包状态为"packet received"（1），再接着三个包状态为"packet not received"，再接着三个包状态为"packet received"，最后两个包状态为"packet not received"。

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1|0|0 1 1 1 1 1 0 0 0 1 1 1 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

例2：如下图，symbol size为1，表示接下来用2个比特标识报文状态。第一个包为"packet not received"（00)状态，第二个包为 "packet received, w/o timestamp"（11）状态，再接着三个包为"Packet received, small delta"（01）状态，最后两个包为"packet not received"（00）状态。

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1|1|0 0 1 1 0 1 0 1 0 1 0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

3.2.6. Receive Delta

Receive Delta长度为一个字节或两个字节，是一个带符号的整数, 记录每个报文与之前收到的报文的间隔时间, 它是250us的倍数。注意这个Delta是当前报文和最近一个报文的间隔时间，第一个报文以reference time为参考，第二个报文以第一个报文为参考，也就是：

1st arrive_time = reference_time + receive_delta
2nd arrtive_time = 1st arrive_time + receive_delta

根据Delta的大小，分三种情况处理：

• 当状态是 “Packet received, small delta”，用 8-bit unsigned 存储 delta, 附加在 packet status list 之后， 此时 delta 取值为 [0,255] * 250 , 表示范围为 [0, 63.75] ms；
• 当状态是 “Packet received, large or negative delta”，用 16-bit signed 存储 delta，附加在 packet status list 之后, 此时 delta 取值为 [-32767, 32768] * 250, 表示范围为 [-8192.0, 8191.75] ms；
• 如果间隔时间太大,就需要启用使用新的 RTCP feedback 包了，不过一般也不会有这么大的延迟，除非网络中断了。

4. Transport-CC的应用场景

4.1. 丢包率/RTT统计及BWE

Transport CC使得流媒体的发送端可以根据Feedback实时计算物理链路的丢包率和RTT，侦测网络变化，并针对性调整带宽估计和发包行为。具体而言，WebRTC根据Transport CC Feedback计算rtt和丢包率，更新Delay Based BWE算法和Loss Based BWE算法的评估结果，同时更新基于真实发送的媒体数据得到的ACK码率，并修正带宽探测行为和媒体发送的Pacing Rate。

4.2. ACK Bitrate统计

WebRTC做码率评估时，有一个重要的参考因子是Ack Bitrate，也就是网络中的真实发送带宽，这里重点介绍Ack Bitrate的统计数据来源。

Ack Bitrate用一个时间窗口内的RTP Packet Size的累计值来衡量，该累计值来自两部分：一是Transport-CC Feedback对应的RTP报文在历史缓存中的记录，二是没有开启TransportSequenceNumber的RTP Packet Size的累加值。前者参考AcknowledgedBitrateEstimator::IncomingPacketFeedbackVector()，这里重点分析下第二部分数据是如何读取的。

4.2.1. 将发包记录存入历史缓存

发送端在发送RTP报文时，需要将一些信息以Key-Value的形式存入Map表中。当发送端收到Transport-CC Feedback之后，从Map表中取出该RTP报文的相关信息，用作丢包、RTT的统计。下面我们对这部分代码做分析。

发送RTP报文的调用堆栈请参考***。当调用堆栈进入RtpSenderEgress::SendPacket()->RtpSenderEgress::CompleteSendPacket()时，函数会判断该RTP报文是否启用了TransportSequenceNumber，若启用了则调用AddPacketToTransportFeedback()。

void RtpSenderEgress::SendPacket(std::unique_ptr<RtpPacketToSend> packet,
                                 const PacedPacketInfo& pacing_info) {
  ...
  auto compound_packet = Packet{std::move(packet), pacing_info, now};
  if (enable_send_packet_batching_ && !is_audio_) {
    //std::vector<Packet> packets_to_send_ RTC_GUARDED_BY(worker_queue_);
    packets_to_send_.push_back(std::move(compound_packet));
  } else {
    CompleteSendPacket(compound_packet, false);
  }
}

void RtpSenderEgress::CompleteSendPacket(const Packet& compound_packet,
                                         bool last_in_batch) {
  RTC_DCHECK_RUN_ON(worker_queue_);
  auto& [packet, pacing_info, now] = compound_packet;

  const bool is_media = packet->packet_type() == RtpPacketMediaType::kAudio ||
                        packet->packet_type() == RtpPacketMediaType::kVideo;

  PacketOptions options;
  options.included_in_allocation = force_part_of_allocation_;

  // Downstream code actually uses this flag to distinguish between media and
  // everything else.
  options.is_retransmit = !is_media;
  if (auto packet_id = packet->GetExtension<TransportSequenceNumber>()) {
    options.packet_id = *packet_id;
    options.included_in_feedback = true;
    options.included_in_allocation = true;
    AddPacketToTransportFeedback(*packet_id, *packet, pacing_info);
  }
  ...
  const bool send_success = SendPacketToNetwork(*packet, options, pacing_info);
  ...
}

AddPacketToTransportFeedback()将RTP报文的所有数据存入packet_info中，包括Transport Sequence Number/Timestamp/RTP Packet Length/Pacing Info/Packet Type(数据包/重传包/Padding/FEC)，然后以packet_info为参数调用RtpTransportControllerSend::OnAddPacket()。

void RtpSenderEgress::AddPacketToTransportFeedback(
    uint16_t packet_id,
    const RtpPacketToSend& packet,
    const PacedPacketInfo& pacing_info) {
  if (transport_feedback_observer_) {
    RtpPacketSendInfo packet_info;
    packet_info.transport_sequence_number = packet_id;
    packet_info.rtp_timestamp = packet.Timestamp();
    packet_info.length = packet.size();
    packet_info.pacing_info = pacing_info;
    packet_info.packet_type = packet.packet_type();

    switch (*packet_info.packet_type) {
      case RtpPacketMediaType::kAudio:
      case RtpPacketMediaType::kVideo:
        packet_info.media_ssrc = ssrc_;
        packet_info.rtp_sequence_number = packet.SequenceNumber();
        break;
      case RtpPacketMediaType::kRetransmission:
        // For retransmissions, we're want to remove the original media packet
        // if the retransmit arrives - so populate that in the packet info.
        packet_info.media_ssrc = ssrc_;
        packet_info.rtp_sequence_number =
            *packet.retransmitted_sequence_number();
        break;
      case RtpPacketMediaType::kPadding:
      case RtpPacketMediaType::kForwardErrorCorrection:
        // We're not interested in feedback about these packets being received
        // or lost.
        break;
    }

    transport_feedback_observer_->OnAddPacket(packet_info);
  }
}

RtpTransportControllerSend::OnAddPacket()根据packet_info创建PacketFeedback，作为对将要发送的RTP报文的本地记录，然后以(RTP Transport Sequence Number， PacketFeedback)为键值对存入history_中。当发送端收到TCC RTCP Feedback时，可以根据RTCP报文中记录的RTP Transport Sequence Number检索history_中匹配的PacketFeedback，找出对应的RTP报文的原始信息。

void RtpTransportControllerSend::OnAddPacket(
    const RtpPacketSendInfo& packet_info) {
  RTC_DCHECK_RUN_ON(&sequence_checker_);
  Timestamp creation_time = Timestamp::Millis(clock_->TimeInMilliseconds());
  feedback_demuxer_.AddPacket(packet_info);
  transport_feedback_adapter_.AddPacket(
      packet_info, transport_overhead_bytes_per_packet_, creation_time);
}

void TransportFeedbackAdapter::AddPacket(const RtpPacketSendInfo& packet_info,
                                         size_t overhead_bytes,
                                         Timestamp creation_time) {
  PacketFeedback packet;
  packet.creation_time = creation_time;
  packet.sent.sequence_number =
      seq_num_unwrapper_.Unwrap(packet_info.transport_sequence_number);
  packet.sent.size = DataSize::Bytes(packet_info.length + overhead_bytes);
  packet.sent.audio = packet_info.packet_type == RtpPacketMediaType::kAudio;
  packet.network_route = network_route_;
  packet.sent.pacing_info = packet_info.pacing_info;

  while (!history_.empty() &&
         creation_time - history_.begin()->second.creation_time >
             kSendTimeHistoryWindow/*60s*/) {
    // TODO(sprang): Warn if erasing (too many) old items?
    /*将map表中所有停留时间超过60s的记录全部清除*/
    if (history_.begin()->second.sent.sequence_number > last_ack_seq_num_)
      in_flight_.RemoveInFlightPacketBytes(history_.begin()->second);
    history_.erase(history_.begin());
  }
  history_.insert(std::make_pair(packet.sent.sequence_number, packet));
}

4.2.2. 记录RTP报文的发送数据量

发送RTP报文时，WebRTC会调用TransportForMediaChannels::SendRtp()，后者有个参数options(来自RtpSenderEgress::CompleteSendPacket())，有3个值和Transport-CC相关：

options.packet_id >= 0; /*记录Transport Sequence Number，是一个非负整数*/

options.included_in_feedback = true; /*显式记录该RTP报文需要被接收端以TCC Feedback应答*/

options.included_in_allocation = true; /*该RTP报文需要计入到发送总带宽里面*/

bool MediaChannelUtil::TransportForMediaChannels::SendRtp(
    const uint8_t* data,
    size_t len,
    const webrtc::PacketOptions& options) {
  auto send =
      [this, packet_id = options.packet_id,
       included_in_feedback = options.included_in_feedback,
       included_in_allocation = options.included_in_allocation,
       batchable = options.batchable,
       last_packet_in_batch = options.last_packet_in_batch,
       packet = rtc::CopyOnWriteBuffer(data, len, kMaxRtpPacketLen/*2048*/)]() mutable {
        rtc::PacketOptions rtc_options;
        rtc_options.packet_id = packet_id;
        if (DscpEnabled()) {
          rtc_options.dscp = PreferredDscp();
        }
        rtc_options.info_signaled_after_sent.included_in_feedback =
            included_in_feedback;
        rtc_options.info_signaled_after_sent.included_in_allocation =
            included_in_allocation;
        rtc_options.batchable = batchable;
        rtc_options.last_packet_in_batch = last_packet_in_batch;
        DoSendPacket(&packet, false, rtc_options);
      };

  // TODO(bugs.webrtc.org/11993): ModuleRtpRtcpImpl2 and related classes (e.g.
  // RTCPSender) aren't aware of the network thread and may trigger calls to
  // this function from different threads. Update those classes to keep
  // network traffic on the network thread.
  if (network_thread_->IsCurrent()) {
    send();
  } else {
    network_thread_->PostTask(SafeTask(network_safety_, std::move(send)));
  }
  return true;
}

在网络层发送完RTP数据之后，会有自下而上的回调。TransportFeedbackAdapter::ProcessSentPacket()是底层网络库AsyncUdpSocket::SendTo()在发送完数据之后的回调，sent_packet记录的rtc options来自TransportForMediaChannels::SendRtp()创建的rtc options。根据当前RTP报文是否启用Transport Sequence Number，分为两种情况：

1. 如果刚刚发送的RTP报文开启了Transport Sequence Number且在历史缓存中，则将未开启Transport Sequence Number的RTP报文的累计Size(存于变量pending_untracked_size_中)加到prior_unacked_data字段中，用于统计实际发送数据，进而确认流媒体数据真实的发送速度(Acknowledge BitRate)；
2. 若刚刚发送的RTP报文未开启Transport Sequence Number，则将RTP报文的大小计入pending_untracked_size_中，用于第一种情况下的数据统计。

absl::optional<SentPacket> TransportFeedbackAdapter::ProcessSentPacket(
    const rtc::SentPacket& sent_packet) {
  auto send_time = Timestamp::Millis(sent_packet.send_time_ms);
  // TODO(srte): Only use one way to indicate that packet feedback is used.
  if (sent_packet.info.included_in_feedback || sent_packet.packet_id != -1) {
    int64_t unwrapped_seq_num =
        seq_num_unwrapper_.Unwrap(sent_packet.packet_id);
    auto it = history_.find(unwrapped_seq_num);
    if (it != history_.end()) {
      bool packet_retransmit = it->second.sent.send_time.IsFinite();
      it->second.sent.send_time = send_time;
      last_send_time_ = std::max(last_send_time_, send_time);
      // TODO(srte): Don't do this on retransmit.
      if (!pending_untracked_size_.IsZero()) {
        if (send_time < last_untracked_send_time_)
          RTC_LOG(LS_WARNING)
              << "appending acknowledged data for out of order packet. (Diff: "
              << ToString(last_untracked_send_time_ - send_time) << " ms.)";
        it->second.sent.prior_unacked_data += pending_untracked_size_;
        pending_untracked_size_ = DataSize::Zero();
      }
      if (!packet_retransmit) {
        if (it->second.sent.sequence_number > last_ack_seq_num_)
          in_flight_.AddInFlightPacketBytes(it->second);
        it->second.sent.data_in_flight = GetOutstandingData();
        return it->second.sent;
      }
    }
  } else if (sent_packet.info.included_in_allocation) {
    if (send_time < last_send_time_) {
      RTC_LOG(LS_WARNING) << "ignoring untracked data for out of order packet.";
    }
    pending_untracked_size_ +=
        DataSize::Bytes(sent_packet.info.packet_size_bytes);
    last_untracked_send_time_ = std::max(last_untracked_send_time_, send_time);
  }
  return absl::nullopt;
}

5. 接收端发送Transport-CC Feedback

5.1. RemoteEstimatorProxy及关联类介绍

5.2. 调用关系图

接下来解释下各个类及其成员变量之间的关系，通过梳理其相关关系，明确函数之间的调用关系。

在创建Call实例时，会跟着创建RtpTransportControllerSend实例(由transport_send_指向该实例)，并对receive_side_cc_进行初始化，将函数指针receive_side_cc_->remote_estimate_proxy_.feedback_sender_指向transport_send_->packet_router_的成员函数PacketRouter::SendCombinedRtcpPacket(同时receive_side_cc_->remote_estimate_proxy_.network_state_estimator_被置为NULL)。Call实例初始化的最后，会创建定时任务receive_side_cc_periodic_task_，该任务定时执行receive_side_cc->MaybeProcess()。

Call* Call::Create(const Call::Config& config) {
  Clock* clock = Clock::GetRealTimeClock();
  return Create(config, clock,
                RtpTransportControllerSendFactory().Create(
                    config.ExtractTransportConfig(), clock));
}

Call::Call(Clock* clock,
           const Call::Config& config,
           std::unique_ptr<RtpTransportControllerSendInterface> transport_send,
           TaskQueueFactory* task_queue_factory)
    : clock_(clock),
      task_queue_factory_(task_queue_factory),
      worker_thread_(GetCurrentTaskQueueOrThread()),
      network_thread_(config.network_task_queue_ ? config.network_task_queue_
      ...
      receive_side_cc_(clock,
                       absl::bind_front(&PacketRouter::SendCombinedRtcpPacket,
                                        transport_send->packet_router()),
                       absl::bind_front(&PacketRouter::SendRemb,
                                        transport_send->packet_router()),
                       /*network_state_estimator=*/nullptr),
      ...
      transport_send_ptr_(transport_send.get()),
      transport_send_(std::move(transport_send)) {
  ...
  call_stats_->RegisterStatsObserver(&receive_side_cc_);

  ReceiveSideCongestionController* receive_side_cc = &receive_side_cc_;
  receive_side_cc_periodic_task_ = RepeatingTaskHandle::Start(
      worker_thread_,
      [receive_side_cc] { return receive_side_cc->MaybeProcess(); },
      TaskQueueBase::DelayPrecision::kLow, clock_);
}

TimeDelta ReceiveSideCongestionController::MaybeProcess() {
  Timestamp now = clock_.CurrentTime();
  mutex_.Lock();
  TimeDelta time_until_rbe = rbe_->Process();
  mutex_.Unlock();
  TimeDelta time_until_rep = remote_estimator_proxy_.Process(now);
  TimeDelta time_until = std::min(time_until_rbe, time_until_rep);
  return std::max(time_until, TimeDelta::Zero());
}

以下是ReceiveSideCongestionController的构造函数，remote_estimator_proxy_.feedback_sender_(函数指针)被赋值为feedback_sender，remote_estimator_proxy_.network_state_estimator_被赋值为network_state_estimator，这个代码可以和Call的构造函数对照着看。

ReceiveSideCongestionController::ReceiveSideCongestionController(
    Clock* clock,
    RemoteEstimatorProxy::TransportFeedbackSender feedback_sender,
    RembThrottler::RembSender remb_sender,
    NetworkStateEstimator* network_state_estimator)
    : clock_(*clock),
      remb_throttler_(std::move(remb_sender), clock),
      remote_estimator_proxy_(std::move(feedback_sender),
                              network_state_estimator),
      rbe_(new RemoteBitrateEstimatorSingleStream(&remb_throttler_, clock)),
      using_absolute_send_time_(false),
      packets_since_absolute_send_time_(0) {}

下面解释下PacketRouter::SendCombinedRtcpPacket()和更底层的函数之间的调用关系。为解释清楚这个问题，我们需要从RtpTransportControllerSend这里说起。

RtpTransportControllerSend通过CreateRtpVideoSender()创建了video_rtp_senders_的每个成员变量，也就是RtpVideoSender实例，在RtpVideoSender的构造函数中又通过调用CreateRtpStreamSenders()创建了rtp_streams的每个成员变量，也就是RtpStreamSender实例。

RtpVideoSenderInterface* RtpTransportControllerSend::CreateRtpVideoSender(
    const std::map<uint32_t, RtpState>& suspended_ssrcs,
    const std::map<uint32_t, RtpPayloadState>& states,
    const RtpConfig& rtp_config,
    int rtcp_report_interval_ms,
    Transport* send_transport,
    const RtpSenderObservers& observers,
    RtcEventLog* event_log,
    std::unique_ptr<FecController> fec_controller,
    const RtpSenderFrameEncryptionConfig& frame_encryption_config,
    rtc::scoped_refptr<FrameTransformerInterface> frame_transformer) {
  RTC_DCHECK_RUN_ON(&sequence_checker_);
  video_rtp_senders_.push_back(std::make_unique<RtpVideoSender>(
      clock_, suspended_ssrcs, states, rtp_config, rtcp_report_interval_ms,
      send_transport, observers,
      // TODO(holmer): Remove this circular dependency by injecting
      // the parts of RtpTransportControllerSendInterface that are really used.
      this, event_log, &retransmission_rate_limiter_, std::move(fec_controller),
      frame_encryption_config.frame_encryptor,
      frame_encryption_config.crypto_options, std::move(frame_transformer),
      field_trials_, task_queue_factory_));
  return video_rtp_senders_.back().get();
}

RtpVideoSender::RtpVideoSender(
    ...,
    Transport* send_transport,
    const RtpSenderObservers& observers,
    RtpTransportControllerSendInterface* transport,
    ...)
    : ...,
      rtp_streams_(CreateRtpStreamSenders(...)),
      ...
      transport_(transport),
      ... {
  ...
}

RtpVideoSender::SetActiveModulesLocked()通过调用PacketRouter::AddSendRtpModule()向PacketRouter::send_modules_map_中注册了RTP模块(指向RtpVideoSender::rtp_streams_[i].rtp_rtcp)，实际上就是ModuleRtpRtcpImpl2模块。注意，这个地方是有问题的，因为SetActiveModulesLocked的调用者本身就有疑问，也许这个调用路径是不对的，后面再更新下这部分。

void RtpVideoSender::SetActiveModulesLocked(
    const std::vector<bool>& active_modules) {
  RTC_DCHECK_RUN_ON(&transport_checker_);
  RTC_DCHECK_EQ(rtp_streams_.size(), active_modules.size());
  active_ = false;
  for (size_t i = 0; i < active_modules.size(); ++i) {
    if (active_modules[i]) {
      active_ = true;
    }

    //call ModuleRtpRtcpImpl2
    RtpRtcpInterface& rtp_module = *rtp_streams_[i].rtp_rtcp;
    const bool was_active = rtp_module.Sending();
    const bool should_be_active = active_modules[i];

    // Sends a kRtcpByeCode when going from true to false.
    rtp_module.SetSendingStatus(active_modules[i]);

    if (was_active && !should_be_active) {
      // Disabling media, remove from packet router map to reduce size and
      // prevent any stray packets in the pacer from asynchronously arriving
      // to a disabled module.
      transport_->packet_router()->RemoveSendRtpModule(&rtp_module);

      // Clear the pacer queue of any packets pertaining to this module.
      transport_->packet_sender()->RemovePacketsForSsrc(rtp_module.SSRC());
      if (rtp_module.RtxSsrc().has_value()) {
        transport_->packet_sender()->RemovePacketsForSsrc(
            *rtp_module.RtxSsrc());
      }
      if (rtp_module.FlexfecSsrc().has_value()) {
        transport_->packet_sender()->RemovePacketsForSsrc(
            *rtp_module.FlexfecSsrc());
      }
    }

    // If set to false this module won't send media.
    rtp_module.SetSendingMediaStatus(active_modules[i]);

    if (!was_active && should_be_active) {
      // Turning on media, register with packet router.
      transport_->packet_router()->AddSendRtpModule(&rtp_module,
                                                    /*remb_candidate=*/true);
    }
  }
  if (!active_) {
    auto* feedback_provider = transport_->GetStreamFeedbackProvider();
    if (registered_for_feedback_) {
      feedback_provider->DeRegisterStreamFeedbackObserver(this);
      registered_for_feedback_ = false;
    }
  } else if (!registered_for_feedback_) {
    auto* feedback_provider = transport_->GetStreamFeedbackProvider();
    feedback_provider->RegisterStreamFeedbackObserver(rtp_config_.ssrcs, this);
    registered_for_feedback_ = true;
  }
}

void PacketRouter::AddSendRtpModule(RtpRtcpInterface* rtp_module,
                                    bool remb_candidate) {
  RTC_DCHECK_RUN_ON(&thread_checker_);

  AddSendRtpModuleToMap(rtp_module, rtp_module->SSRC());
  if (absl::optional<uint32_t> rtx_ssrc = rtp_module->RtxSsrc()) {
    AddSendRtpModuleToMap(rtp_module, *rtx_ssrc);
  }
  if (absl::optional<uint32_t> flexfec_ssrc = rtp_module->FlexfecSsrc()) {
    AddSendRtpModuleToMap(rtp_module, *flexfec_ssrc);
  }

  if (rtp_module->SupportsRtxPayloadPadding()) {
    last_send_module_ = rtp_module;
  }

  if (remb_candidate) {
    AddRembModuleCandidate(rtp_module, /* media_sender = */ true);
  }
}

void PacketRouter::AddSendRtpModuleToMap(RtpRtcpInterface* rtp_module,
                                         uint32_t ssrc) {
  RTC_DCHECK_RUN_ON(&thread_checker_);
  RTC_DCHECK(send_modules_map_.find(ssrc) == send_modules_map_.end());

  // Signal to module that the pacer thread is attached and can send packets.
  rtp_module->OnPacketSendingThreadSwitched();

  // Always keep the audio modules at the back of the list, so that when we
  // iterate over the modules in order to find one that can send padding we
  // will prioritize video. This is important to make sure they are counted
  // into the bandwidth estimate properly.
  if (rtp_module->IsAudioConfigured()) {
    send_modules_list_.push_back(rtp_module);
  } else {
    send_modules_list_.push_front(rtp_module);
  }
  send_modules_map_[ssrc] = rtp_module;
}

PacketRouter::SendCombinedRtcpPacket()在执行时，send_modules_list_中存有ModuleRtpRtcpImpl2这个RTP Module，所以函数第11行实际调用了ModuleRtpRtcpImpl2::SendCombinedRtcpPacket()，后者又调用了RTCPSender::SendCombinedRtcpPacket()。

void PacketRouter::SendCombinedRtcpPacket(
    std::vector<std::unique_ptr<rtcp::RtcpPacket>> packets) {
  RTC_DCHECK_RUN_ON(&thread_checker_);

  // Prefer send modules.
  for (RtpRtcpInterface* rtp_module : send_modules_list_) {
    if (rtp_module->RTCP() == RtcpMode::kOff) {
      continue;
    }
    /*Call ModuleRtpRtcpImpl2::SendCombinedRtcpPacket*/
    rtp_module->SendCombinedRtcpPacket(std::move(packets));
    return;
  }

  if (rtcp_feedback_senders_.empty()) {
    return;
  }
  auto* rtcp_sender = rtcp_feedback_senders_[0];
  rtcp_sender->SendCombinedRtcpPacket(std::move(packets));
}

void ModuleRtpRtcpImpl2::SendCombinedRtcpPacket(
    std::vector<std::unique_ptr<rtcp::RtcpPacket>> rtcp_packets) {
  /*Call RTCPSender::SendCombinedRtcpPacket*/
  rtcp_sender_.SendCombinedRtcpPacket(std::move(rtcp_packets));
}

5.3. RemoteEstimatorProxy核心函数介绍

WebRTC发送Transport-CC Feedback的时机有两处：

• 定时发送；
• 收到RTP报文。

下面按这两种情况分别介绍。需要注意的是，如果接收端收到了一个开启了Transport Sequence Number V2的RTP报文，则在当前通道上永久禁用了定时发送Feedback报文的任务。(Why？)

5.3.1. 定时激发控制报文发送

WebRTC每隔100ms会触发一次Transport-CC Feedback的构造和发送，这个是由定时任务控制的。我们回顾下Call的构造函数，在该函数中创建了一个定时任务receive_side_cc_periodic_task_，该任务在Worker线程(worker_thread_)中定时执行receive_side_cc->MaybeProcess()，后者调用了RemoteEstimatorProxy::Process()完成真正的RTCP报文创建和发送。在我们的实践中，我们不会在接收端做带宽估计，所以我们只考虑在接收端发送Transport-CC报文的场景。

TimeDelta ReceiveSideCongestionController::MaybeProcess() {
  Timestamp now = clock_.CurrentTime();
  mutex_.Lock();
  TimeDelta time_until_rbe = rbe_->Process();
  mutex_.Unlock();
  TimeDelta time_until_rep = remote_estimator_proxy_.Process(now);
  TimeDelta time_until = std::min(time_until_rbe, time_until_rep);
  return std::max(time_until, TimeDelta::Zero());
}

RemoteEstimatorProxy::Process()的行为如下：

1. 先做判断，如果在此之前收到了开启TransportSequenceNumberV2的报文，则永久禁用定时反馈Transport-CC的行为；
2. 否则判断当前时间是否已到定时反馈Transport-CC的时间，

1. 1. 若未到，则返回距离触发定时反馈的剩余时间；
   2. 若已到，则执行定时反馈的动作(SendPeriodicFeedbacks())，同时返回下次反馈的时间间隔(send_interval_）。

TimeDelta RemoteEstimatorProxy::Process(Timestamp now) {
  MutexLock lock(&lock_);
  if (!send_periodic_feedback_) {
    // If TransportSequenceNumberV2 has been received in one packet,
    // PeriodicFeedback is disabled for the rest of the call.
    return TimeDelta::PlusInfinity();
  }
  Timestamp next_process_time = last_process_time_ + send_interval_/*100ms*/;
  if (now >= next_process_time) {
    last_process_time_ = now;
    SendPeriodicFeedbacks();
    return send_interval_;
  }

  return next_process_time - now;
}

RemoteEstimatorProxy::SendPeriodicFeedbacks()先构造Transport-CC Feedback，然后在第39行实际调用PacketRouter::SendCombinedRtcpPacket()将报文发送出去。由于network_state_estimator_被设置为NULL，所以这部分的代码都可以直接忽略。

void RemoteEstimatorProxy::SendPeriodicFeedbacks() {
  // `periodic_window_start_seq_` is the first sequence number to include in
  // the current feedback packet. Some older may still be in the map, in case
  // a reordering happens and we need to retransmit them.
  if (!periodic_window_start_seq_)
    return;

  std::unique_ptr<rtcp::RemoteEstimate> remote_estimate;
  if (network_state_estimator_) {/*=NULL*/
    absl::optional<NetworkStateEstimate> state_estimate =
        network_state_estimator_->GetCurrentEstimate();
    if (state_estimate) {
      remote_estimate = std::make_unique<rtcp::RemoteEstimate>();
      remote_estimate->SetEstimate(state_estimate.value());
    }
  }

  int64_t packet_arrival_times_end_seq =
      packet_arrival_times_.end_sequence_number();
  while (periodic_window_start_seq_ < packet_arrival_times_end_seq) {
    auto feedback_packet = MaybeBuildFeedbackPacket(
        /*include_timestamps=*/true, *periodic_window_start_seq_,
        packet_arrival_times_end_seq,
        /*is_periodic_update=*/true);

    if (feedback_packet == nullptr) {
      break;
    }

    RTC_DCHECK(feedback_sender_ != nullptr);

    std::vector<std::unique_ptr<rtcp::RtcpPacket>> packets;
    if (remote_estimate) {/*=NULL*/
      packets.push_back(std::move(remote_estimate));
    }
    packets.push_back(std::move(feedback_packet));

    /*Call PacketRouter::SendCombinedRtcpPacket*/
    feedback_sender_(std::move(packets));
  }
}

5.3.2. 收到RTP报文后激发控制报文发送

媒体接收端在收到RTP报文之后，从底层的AsyncUDPSocket开始，逐层向上投递报文，进入Call::DeliverRtpPacket()之后，调用了NotifyBweOfReceivedPacket()。由于我们只关心Send Side BWE的行为，所以最终进入RemoteEstimatorProxy::IncomingPacket()，后者如果遇到Transport Sequence Number V2类型的Transport-CC Feedback，则立即发送Feedback，否则将这些报文对应的Transport Sequence Number存入一个数组中，待定时任务被激发时统一发送。

void Call::DeliverRtpPacket(
    MediaType media_type,
    RtpPacketReceived packet,
    OnUndemuxablePacketHandler undemuxable_packet_handler) {
  if (receive_time_calculator_) {
    int64_t packet_time_us = packet.arrival_time().us();
    // Repair packet_time_us for clock resets by comparing a new read of
    // the same clock (TimeUTCMicros) to a monotonic clock reading.
    packet_time_us = receive_time_calculator_->ReconcileReceiveTimes(
        packet_time_us, rtc::TimeUTCMicros(), clock_->TimeInMicroseconds());
    packet.set_arrival_time(Timestamp::Micros(packet_time_us));
  }

  NotifyBweOfReceivedPacket(packet, media_type);
  ...
}

void Call::NotifyBweOfReceivedPacket(const RtpPacketReceived& packet,
                                     MediaType media_type) {
  RTC_DCHECK_RUN_ON(worker_thread_);

  ReceivedPacket packet_msg;
  packet_msg.size = DataSize::Bytes(packet.payload_size());
  packet_msg.receive_time = packet.arrival_time();
  uint32_t time_24;
  if (packet.GetExtension<AbsoluteSendTime>(&time_24)) {
    packet_msg.send_time = AbsoluteSendTime::ToTimestamp(time_24);
  }
  /*Call RtpTransportControllerSend::OnReceivedPacket*/
  transport_send_->OnReceivedPacket(packet_msg);

  /*Call ReceiveSideCongestionController::OnReceivedPacket*/
  receive_side_cc_.OnReceivedPacket(packet, media_type);
}

void ReceiveSideCongestionController::OnReceivedPacket(
    const RtpPacketReceived& packet,
    MediaType media_type) {
  bool has_transport_sequence_number =
      packet.HasExtension<TransportSequenceNumber>() ||
      packet.HasExtension<TransportSequenceNumberV2>();
  if (media_type == MediaType::AUDIO && !has_transport_sequence_number) {
    // For audio, we only support send side BWE.
    return;
  }

  if (has_transport_sequence_number) {
    // Send-side BWE.
    /*Call RemoteEstimatorProxy::IncomingPacket*/
    remote_estimator_proxy_.IncomingPacket(packet);
  } else {
    // Receive-side BWE.
    MutexLock lock(&mutex_);
    RTPHeader header;
    packet.GetHeader(&header);
    PickEstimator(packet.HasExtension<AbsoluteSendTime>());
    rbe_->IncomingPacket(packet.arrival_time().ms(),
                         packet.payload_size() + packet.padding_size(), header);
  }
}

RemoteEstimatorProxy::IncomingPacket()先检查当前RTP报文是否开启了Transport Sequence Number，如果未开启则直接Return，此时当前RTP报文就直接略过了。否则：

• 如果开启的是Transport Sequence Number V2，则直接禁用定时反馈Feedback的任务(将send_periodic_feedback_设置为False，在RemoteEstimatorProxy::Process()禁用该任务)，同时直接发送Feedback；
• 如果开启的是Transport Sequence Number V1，则将当前报文的Transport Sequence Number和到达时间设置到数组中，待定时任务统一发送。为了提高Feedback的有效性，会将定时任务反馈的报文序号范围压缩到数组的最小序号和最大序号之间。

void RemoteEstimatorProxy::IncomingPacket(const RtpPacketReceived& packet) {
  if (packet.arrival_time().IsInfinite()) {
    RTC_LOG(LS_WARNING) << "Arrival time not set.";
    return;
  }

  /*检查RTP报文的扩展头部是否开启了Transport Sequence Number
   *若未开启则直接Return,此时该报文直接被忽略
   *若开启的是TransportSequenceNumberV2,则Disable定期发送Feedback的任务
   */

  uint16_t seqnum = 0;
  absl::optional<FeedbackRequest> feedback_request;
  if (!packet.GetExtension<TransportSequenceNumber>(&seqnum) &&
      !packet.GetExtension<TransportSequenceNumberV2>(&seqnum,
                                                      &feedback_request)) {
    return;
  }

  MutexLock lock(&lock_);
  send_periodic_feedback_ = packet.HasExtension<TransportSequenceNumber>();

  media_ssrc_ = packet.Ssrc();
  int64_t seq = unwrapper_.Unwrap(seqnum);

  if (send_periodic_feedback_) {
    /*若seq比当前队列中最大的序列号都大,
     *则将序列号很小且停留时间达到上限的报文全部淘汰掉
     */
    MaybeCullOldPackets(seq, packet.arrival_time());

    if (!periodic_window_start_seq_ || seq < *periodic_window_start_seq_) {
      periodic_window_start_seq_ = seq;
    }
  }

  // We are only interested in the first time a packet is received.
  if (packet_arrival_times_.has_received(seq)) {
    return;
  }

  /*将报文的seq和arrival_time存入数组中
   *将Feedback对应的报文序列范围压缩到数组的最小序号和最大序号之间
   */

  packet_arrival_times_.AddPacket(seq, packet.arrival_time());

  // Limit the range of sequence numbers to send feedback for.
  if (periodic_window_start_seq_ <
      packet_arrival_times_.begin_sequence_number()) {
    periodic_window_start_seq_ = packet_arrival_times_.begin_sequence_number();
  }

  if (feedback_request.has_value()) {
    /*如果RTP报文的扩展头部开启了Transport Sequence Number V2
     *则立即发送Feedback,此时会Disable定期反馈Feedback的任务
     */
    // Send feedback packet immediately.
    SendFeedbackOnRequest(seq, *feedback_request);
  }
  ...
}

RemoteEstimatorProxy::SendFeedbackOnRequest()为开启了Transport Sequence Number V2标识的所有RTP报文做一个统一的Transport CC Feedback，并通过网络发送给媒体发送端。这里的RTP报文的Transport Sequence Number落在[sequence_number - sequence_count + 1, sequence_number]这个闭区间，本函数将这部分RTP报文的Feedback记在了同一个Feedback中。

void RemoteEstimatorProxy::SendFeedbackOnRequest(
    int64_t sequence_number,
    const FeedbackRequest& feedback_request) {
  if (feedback_request.sequence_count == 0) {
    return;
  }

  /*Transport Sequence Number落在[first_sequence_number, sequence_num]
   *之间的RTP报文需要接收端在收到后一次给出Feedback
   *包括first_sequence_number & sequence_num
   */
  int64_t first_sequence_number =
      sequence_number - feedback_request.sequence_count + 1;

  /*为落在[first_sequence_number, sequence_num]区间的RTP报文
   *创建一个Transport-CC RTCP作为统一的Feedback
   */
  auto feedback_packet = MaybeBuildFeedbackPacket(
      feedback_request.include_timestamps, first_sequence_number,
      sequence_number + 1, /*is_periodic_update=*/false);

  // This is called when a packet has just been added.
  RTC_DCHECK(feedback_packet != nullptr);

  // Clear up to the first packet that is included in this feedback packet.
  packet_arrival_times_.EraseTo(first_sequence_number);

  /*将Transport-CC Feedback发送出去*/

  RTC_DCHECK(feedback_sender_ != nullptr);
  std::vector<std::unique_ptr<rtcp::RtcpPacket>> packets;
  packets.push_back(std::move(feedback_packet));
  /*Call PacketRouter::SendCombinedRtcpPacket*/
  feedback_sender_(std::move(packets));
}

5.4. 创建Transport-CC报文

5.4.1. 确定Feedback要反馈的报文序列范围

RemoteEstimatorProxy::MaybeBuildFeedbackPacket()根据入参begin_sequence_number_inclusive &end_sequence_number_exclusive取出本次Transport-CC Feedback要处理的RTP Transport Sequence Number的范围，然后遍历这些序列号，将ssrc/base sequence number/feedback sequence number/收到报文的时间间隔/中间丢失的报文等信息存入结构体rtcp::TransportFeedback::received_packets_维护的内存向量中。

本函数在确定Feedback要反馈的报文序号范围时，遵从如下约束：

• 当前序号的报文与之前收到的报文的间隔时间太大(16bits signed无法表示该Recv Delta)，需要新起一个feedback报文；
• 当前反馈的报文序列号不小于上次已反馈的报文序列号，feedback报文反馈最新的报文接收情况；
• 一次反馈的报文不要太多，太多则需要新起一个feedback报文。

std::unique_ptr<rtcp::TransportFeedback>
RemoteEstimatorProxy::MaybeBuildFeedbackPacket(
    bool include_timestamps,
    int64_t begin_sequence_number_inclusive,
    int64_t end_sequence_number_exclusive,
    bool is_periodic_update) {
  RTC_DCHECK_LT(begin_sequence_number_inclusive, end_sequence_number_exclusive);

  /*取出本次Transport-CC Feedback要处理的RTP Transport Sequence Number的范围
   *(落在[start_seq,end_seq-1]这个闭区间)
   */

  int64_t start_seq =
      packet_arrival_times_.clamp(begin_sequence_number_inclusive);

  int64_t end_seq = packet_arrival_times_.clamp(end_sequence_number_exclusive);

  // Create the packet on demand, as it's not certain that there are packets
  // in the range that have been received.
  std::unique_ptr<rtcp::TransportFeedback> feedback_packet;

  int64_t next_sequence_number = begin_sequence_number_inclusive;

  for (int64_t seq = start_seq; seq < end_seq; ++seq) {
    PacketArrivalTimeMap::PacketArrivalTime packet =
        packet_arrival_times_.FindNextAtOrAfter(seq);
    seq = packet.sequence_number;
    if (seq >= end_seq) {
      break;
    }

    if (feedback_packet == nullptr) {

      /*如果还没有创建feedback,则创建一个新的,
       *同时设置ssrc/base sequence number/feedback sequence number
       *再将当前seq对应的收包信息存入feedback中
       */

      feedback_packet =
          std::make_unique<rtcp::TransportFeedback>(include_timestamps);
      feedback_packet->SetMediaSsrc(media_ssrc_);

      // It should be possible to add `seq` to this new `feedback_packet`,
      // If difference between `seq` and `begin_sequence_number_inclusive`,
      // is too large, discard reporting too old missing packets.
      static constexpr int kMaxMissingSequenceNumbers = 0x7FFE;
      int64_t base_sequence_number = std::max(begin_sequence_number_inclusive,
                                              seq - kMaxMissingSequenceNumbers);

      // Base sequence number is the expected first sequence number. This is
      // known, but we might not have actually received it, so the base time
      // shall be the time of the first received packet in the feedback.
      feedback_packet->SetBase(static_cast<uint16_t>(base_sequence_number),
                               packet.arrival_time);
      feedback_packet->SetFeedbackSequenceNumber(feedback_packet_count_++);

      if (!feedback_packet->AddReceivedPacket(static_cast<uint16_t>(seq),
                                              packet.arrival_time)) {
        // Could not add a single received packet to the feedback.
        RTC_DCHECK_NOTREACHED()
            << "Failed to create an RTCP transport feedback with base sequence "
               "number "
            << base_sequence_number << " and 1st received " << seq;
        periodic_window_start_seq_ = seq;
        return nullptr;
      }
    } else {
      /*将当前seq对应的收包信息存入feedback中
       *包括收到报文的时间间隔/中间丢失的报文
       */
      if (!feedback_packet->AddReceivedPacket(static_cast<uint16_t>(seq),
                                              packet.arrival_time)) {
        // Could not add timestamp, feedback packet might be full. Return and
        // try again with a fresh packet.
        break;
      }
    }

    next_sequence_number = seq + 1;
  }

  /*调整下次要反馈的报文的起始序列号*/
  if (is_periodic_update) {
    periodic_window_start_seq_ = next_sequence_number;
  }
  
  return feedback_packet;
}

5.4.2. 将Feedback的信息序列化为RTCP报文

RTCPSender::SendCombinedRtcpPacket()先根据Transport-CC协议将各字段信息写入RTCP报文中(参考TransportFeedback::Create)，再通过AsyncUDPSocket将报文发送到网络中。

void RTCPSender::SendCombinedRtcpPacket(
    std::vector<std::unique_ptr<rtcp::RtcpPacket>> rtcp_packets) {
  size_t max_packet_size;
  uint32_t ssrc;
  {
    MutexLock lock(&mutex_rtcp_sender_);
    if (method_ == RtcpMode::kOff) {
      RTC_LOG(LS_WARNING) << "Can't send rtcp if it is disabled.";
      return;
    }

    max_packet_size = max_packet_size_;
    ssrc = ssrc_;
  }
  RTC_DCHECK_LE(max_packet_size, IP_PACKET_SIZE);
  auto callback = [&](rtc::ArrayView<const uint8_t> packet) {
    /*Call TransportForMediaChannels::SendRtcp*/
    if (transport_->SendRtcp(packet.data(), packet.size())) {
      if (event_log_)
        event_log_->Log(std::make_unique<RtcEventRtcpPacketOutgoing>(packet));
    }
  };
  PacketSender sender(callback, max_packet_size);
  for (auto& rtcp_packet : rtcp_packets) {
    rtcp_packet->SetSenderSsrc(ssrc);
    /*按各种类型的RTMP协议对报文做序列化,存入内存(rtcp_packet->buffer_)中*/
    sender.AppendPacket(*rtcp_packet);
  }
  /*调用callback,将数据通过网络发送出去*/
  sender.Send();
}

// Appends a packet to pending compound packet.
// Sends rtcp packet if buffer is full and resets the buffer.
void PacketSender::AppendPacket(const rtcp::RtcpPacket& packet) {
  packet.Create(buffer_, &index_, max_packet_size_, callback_);
}

RtcpPacket::Create()是个纯虚函数，由继承类来实现。TransportFeedback继承自Rtpfb，后者继承自RtcpPacket，TransportFeedback::Create实现如下。

// Serialize packet.
bool TransportFeedback::Create(uint8_t* packet,
                               size_t* position,
                               size_t max_length,
                               PacketReadyCallback callback) const {
  if (num_seq_no_ == 0)
    return false;

  while (*position + BlockLength() > max_length) {
    if (!OnBufferFull(packet, position, callback))
      return false;
  }
  const size_t position_end = *position + BlockLength();
  const size_t padding_length = PaddingLength();
  bool has_padding = padding_length > 0;
  CreateHeader(kFeedbackMessageType, kPacketType, HeaderLength(), has_padding,
               packet, position);
  CreateCommonFeedback(packet + *position);
  *position += kCommonFeedbackLength;

  ByteWriter<uint16_t>::WriteBigEndian(&packet[*position], base_seq_no_);
  *position += 2;

  ByteWriter<uint16_t>::WriteBigEndian(&packet[*position], num_seq_no_);
  *position += 2;

  ByteWriter<uint32_t, 3>::WriteBigEndian(&packet[*position], base_time_ticks_);
  *position += 3;

  packet[(*position)++] = feedback_seq_;

  for (uint16_t chunk : encoded_chunks_) {
    ByteWriter<uint16_t>::WriteBigEndian(&packet[*position], chunk);
    *position += 2;
  }
  if (!last_chunk_.Empty()) {
    uint16_t chunk = last_chunk_.EncodeLast();
    ByteWriter<uint16_t>::WriteBigEndian(&packet[*position], chunk);
    *position += 2;
  }

  if (include_timestamps_) {
    for (const auto& received_packet : received_packets_) {
      int16_t delta = received_packet.delta_ticks();
      if (delta >= 0 && delta <= 0xFF) {
        packet[(*position)++] = delta;
      } else {
        ByteWriter<int16_t>::WriteBigEndian(&packet[*position], delta);
        *position += 2;
      }
    }
  }

  if (padding_length > 0) {
    for (size_t i = 0; i < padding_length - 1; ++i) {
      packet[(*position)++] = 0;
    }
    packet[(*position)++] = padding_length;
  }
  RTC_DCHECK_EQ(*position, position_end);
  return true;
}