RTP和RTCP协议

Type	Description	References
192	FIR, full INTRA-frame request.	RFC 2032
193	NACK, negative acknowledgement.	RFC 2032
194	SMPTETC, SMPTE time-code mapping.	RFC5484
195	IJ, extended inter-arrival jitter report.	RFC 5450
200	SR, sender report.	RFC 3550
201	RR, receiver report.	RFC 3550
202	SDES, source description.	RFC 3550
203	BYE, goodbye.	RFC 3550
204	APP, application defined.	RFC 3550
205	RTPFB, Generic RTP Feedback.
206	PSFB, Payload-specific Feedback.
207	XR, RTCP extension.	RFC 3611
208	AVB, AVB RTCP packet.	IEEE 1733
209	RSI, Receiver Summary Information.	RFC 5760

RTP

RTP的封装：

1.单一NAL单一模式

即一个RTP包仅仅由一个完整的NALU组成，RTP头的类型字段和H.264是一样的

2.组合封包模式

由多个NAL单元组合成一个RTP包，分别有4种组合方式

序号	类型	解释
24	STAP-A	单一时间的组合包
25	STAP-B	单一时间的组合包
26	MTAP16	多个时间的组合包
27	MTAP24	多个时间的组合包

3.分片封包模式

把一个NALU单元封装为多个RTP包，存在两种类型的FU-A和FU-B，类型分别是28和29.

有一字节的分片单元头：

S: 1 bit 当设置成1,开始位指示分片NAL单元的开始。当跟随的FU荷载不是分片NAL单元荷载的开始，开始位设为0。

E: 1 bit 当设置成1, 结束位指示分片NAL单元的结束，即, 荷载的最后字节也是分片NAL单元的最后一个字节。当跟随的 FU荷载不是分片NAL单元的最后分片,结束位设置为0。

RTP的丢包：

1.丢包检测

通过设置一定的延迟时间（可以是几十ms或者几百ms），如果收到不连续的包，就减去上一次的丢包时间，如果小于设定的值，就去等；否则，上报丢包信息。

2.丢包处理

如果是单个丢包就发送PLI消息，如果有连续超过8个的丢包，就发送NACK消息；

其中PLI消息指的是：

encoder收到这个消息后，必须发送一个帧内预测的帧过来，否则接收端可能会造成参考帧丢失，进而导致花屏。

NACK针对连续丢包的行为：

NACK报文是类型为205的RTCP 扩展反馈报文，在RFC4585中定义[4]。

 /*
 *| 0                      1                       2                       3                   |           
 *| 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7  0 1 2 3 4 5 6 7           
 *+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 *|V=2|P|    RC   |   PT=205      |             length                          |
 *
 *+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 *|                     SSRC of packet sender                                     | 
 *+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+==+=+
 *|                     SSRC of media source                                      |  
 *+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 *|            PID                               |                BLP                       |
 *+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * 
 */
       其中PT = 205，FMT = 1，Packet identifier(PID)即为丢失RTP数据包的序列号，Bitmao of Lost Packets(BLP)指示从PID开始接下来16个RTP数据包的丢失情况。一个NACK报文可以携带多个RTP序列号，NACK接收端对这些序列号逐个处理。NACK报文构造完成以后，发送到网络层。NACK报文是RTCP报文的一种，因此其发送、接收和分析遵循RTCP报文处理的一般流程。

码率调控

1.首先计算抖动

RTCP的抖动计算，rtp时间的delay=RTP_Timecur-last_RTP_Time; 系统时间间隔daly=cur_reciveTime-last_receive_time;

Jitter=|rec_daly-rtp_delay|

保存几秒的接收到的RTP包到一个list中；计算对端的帧率，接收的网络速率Netbitrate，接收rtp包的rtpbitrate和rtpPacketRate；如果丢包，就不去增加码率。

计算丢包率：

fractionlost：

（highestSeqNumer-mLastRRPacketRecd的seqNUm）-（目前收到的真是包数-上一次收到的包数）*256/((highestSeqNumer-mLastRRPacketRecd的seqNUm）；

计算往返时间：

LSR:最近收到SR Packet时间

DLSR：单位1/65536,betwween receiving the last SR Packet from ssrc_n and 发送这个reception report block

A等于收到RR的时间；

delay=A-DLSR-LSR.

2.触发TMMBR

TMMBR用于流控，请求发送端按指定的最大比特率传输数据流，通常用于网络抖动情况下保证VOIP通信的流畅性（临时降低质量）。

SSRC（32 bits）：  请求对方控制该媒体源遵从下面的最大比特率

MxTBR Exp（6 bits）：  指数

MxTBR Mantissa（17 bits）：  系数

Measured Overhead（9 bits）：  测量的平均开销

计算出来的最大比特率= 系数*2^指数

Tmmbn

 

https://www.jianshu.com/p/e903570ed350

 

RTCP的传输间隔：

It is recommended that the fraction of the session bandwidth added for RTCP be fixed at 5%. It is also recommended that 1/4 of the RTCP bandwidth be dedicated to participants that are sending data so that in sessions with a large number of receivers but a small number of senders.

RTC占整个回话贷款的5%，但是发送者占据1/4;

1.The calculated interval between RTCP packets scales linearly with the number of members in the group. It is this linear factor which allows for a constant amount of control traffic when summed across all members.

发送的间隔和人数呈线性关系；

2.The interval between RTCP packets is varied randomly over the range [0.5,1.5] times the calculated interval to avoid unintended synchronization of all participants [20]. The first RTCP packet sent after joining a session is also delayed by a random variation of half the minimum RTCP interval.

RTCP包的间隔在【0.5,1.5】之间的随机数，避免同时发送RTCP。

3.A dynamic estimate of the average compound RTCP packet size is calculated, including all those packets received and sent, to automatically adapt to changes in the amount of control information carried.

RTCP包大小被动态计算，适应网络的变化。

4.Since the calculated interval is dependent on the number of observed group members, there may be undesirable startup effects when a new user joins an existing session, or many users simultaneously join a new session. These new users will initially have incorrect estimates of the group membership, and thus their RTCP transmission interval will be too short. This problem can be significant if many users join the session simultaneously. To deal with this, an algorithm called “timer reconsideration” is employed. This algorithm implements a simple back-off mechanism which causes users to hold back RTCP packet transmission if the group sizes are increasing.

计算时间间隔，依赖组内成员的变化；

5.When users leave a session, either with a BYE or by timeout, the group membership decreases, and thus the calculated interval should decrease. A “reverse reconsideration” algorithm is used to allow members to more quickly reduce their intervals in response to group membership decreases.

当有成员离开时，计算的间隔减少。

tp: the last time an RTCP packet was transmitted;

tc: the current time;

tn: the next scheduled transmission time of an RTCP packet;

pmembers: the estimated number of session members at the time tn was last recomputed;

tn最后被重新计算时，会计的会话成员的人数。

members: the most current estimate for the number of session members;

senders: the most current estimate for the number of senders in the session;

rtcp bw: The target RTCP bandwidth, i.e., the total bandwidth that will be used for RTCP packets by all members of this session, in octets per second. This will be a specified fraction of the “session bandwidth” parameter supplied to the application at startup.

we sent: Flag that is true if the application has sent data since the 2nd previous RTCP report was transmitted.

自当前第二个前面的RTCP发送后，应用程序又发送了数据，则此项为true

avg rtcp size: The average compound RTCP packet size, in octets, over all RTCP packets sent and received by this participant. The size includes lower-layer transport and network protocol headers (e.g., UDP and IP) as explained in Section 6.2.

initial: Flag that is true if the application has not yet sent an RTCP packet.

1.如果成员数小于25%，这个间隔取决于是否是发送者；如果是发送者，这个C=(avg rtcp size)/(25%*rtcp bw), n=members；如果我们接收者，c=(avg rtcp size)/(75%*rtcp bw);n=(members-senders).

如果成员的发送者大于25%，发送者和接收者依据同样规则被处理。C=(avg rtcp size)/(rtcp bw);

n=members;

 如6.2节所述，RTP描述文件可能用两个独立的参数（S，R）确定发送者与非发送者。此时，25%和75%只要相应的换成S/(S+R),R/(S+R)即可。注意R＝0的情况

2. If the participant has not yet sent an RTCP packet (the variable initial is true), the constant Tmin is set to 2.5 seconds, else it is set to 5 seconds.

如果参与者还没有发送RTCP包，就设定为2.5s，否则就设定为5s。

3.The deterministic calculated interval Td is set to max(Tmin, n*C).

Td=max(Tmin,n*C);

4. The calculated interval T is set to a number uniformly distributed between 0.5 and 1.5 times the deterministic calculated interval.

T=Td*[0.5,1.5]随机值

5.The resulting value of T is divided by e − 3/2=1.21828 to compensate for the fact that the timer reconsideration algorithm converges to a value of the RTCP bandwidth below the intended average

 T=T/(e-0.5)≈T/1.21828，补偿时间重估算法，使之收敛到比计算出的平均RTCP带宽小的一个值。

初始化

tp=0； tc=0； senders=0； pmembers=1； members=1； vw_sent=false； rtcp_bw:由会话带宽参数的相应部分得到；initial=true；avg_rtcp_size:初始化为应用程序稍后将发送的RTCP包的可能大小；T：如6.3.1节；tn=T（这意味着，一个计时器将经T时间后被唤醒）

Receiving an RTP or Non-BYE RTCP Packet

更新如下的变量：

avg rtcp size = (1/16) * packet size + (15/16) * avg rtcp size

Receiving an RTCP BYE Packet

• The value for tn is updated according to the following formula:

tn = tc + (members/pmembers) * (tn - tc)

• The value for tp is updated according the following formula:

tp = tc - (members/pmembers) * (tc - tp).

• The next RTCP packet is rescheduled for transmission at time tn, which is now earlier.

• The value of pmembers is set equal to members.

6.3.5 Timing Out an SSRC

To do this, the participant computes the deterministic (without the randomization factor) calculated interval Td for a receiver, that is, with we sent false. Any other session member who has not sent an RTP or RTCP packet since time tc - MTd (M is the timeout multiplier, and defaults to 5) is timed out. This means that its SSRC is removed from the member list, and members is updated. A similar check is performed on the sender list. Any member on the sender list who has not sent an RTP packet since time tc - 2T (within the last two RTCP report intervals) is removed from the sender list, and senders is updated.

接收者：tc-MTd时间内，检查是否有RTP包或者RTCP包的发出

发送者列表：tc-2T,左同样的检查

If any members time out, the reverse reconsideration algorithm described in Section 6.3.4 should be performed.

发送时钟超时

变量的更新：

如果tp+T<=tc, tp=tc, tn=tc+T.

如果tp+T>tc, tn=tp+T;

如果没有rtcp包传输，pmember=members;

如果有rtcp包传输，avg rtcp size = (1/16) * packet size + (15/16) * avg rtcp size

当发送一个BYE包

大量BYE包的发送，若会话人数超过50，则参与者在要离开会话时，应执行下面的算法。这个算法实际上“篡夺”了一般可变成员的角色来统计BYE包。

（1）tp=tc ； members=1； pmembers=1； sinitial=1； we_sent=false； senders=0； rtcp_size:设置为将要发送的RTCP包大小；计算“计算时间间隔”T；tn=tc+T；(BYE包预计在时刻tn被发送)。

    (2)每当从另外一个参与者接收到BYE包时，成员人数加1。不管此成员是否存在于成员列表中，也不管SSRC采样何时使用及BYE包的SSRC是否包含在采样之中。如果收到RTP包或甚的RTCP包（除BYE包之外的RTCP包），成员人数不增加。类似，只有在收到BYE包时，avg_rtcp_size才更新。当RTP包到达时，发送者人数senders不更新，保持为0

更新we_sent变量

If the participant sends an RTP packet when we sent is false, it adds itself to the sender table and sets we sent to true.

发送者和接收者报告

发送者报告(SR)和接收者报告(RR)格式中唯一的不同，除包类型码之外，在于发送者报告包括20字节的发送者信息

SR:

 RR: