RTP 协议

1.预备知识

The Real-time Transport Protocol (RTP) defines a standardized packet format for delivering audio and video over IP networks. RTP is used extensively in communication and entertainment systems that involve streaming media, such telephony, video teleconference applications, television services and web-based push-to-talk features.

RTP is used in conjunction with the RTP Control Protocol (RTCP). While RTP carries the media streams(e.g. audio and video), RTCP is used to monitor transmission statistics and quality of service (QoS) and aids synchronization of multiple streams. RTP is one of the technical foundations of Voice over IP and in this context is often used in conjunction with a signaling protocol such as the Session Initiation Protocol (SIP) which assists in setting up connections across the network.

RTP is designed for end-to-end, real-time, transfer of stream data. The protocol provides facilities for jitter compensation and detection of out of sequence arrival in data, which are common during transmissions on an IP network. RTP allows data transfer to multiple destinations through IP multicast. RTP is regarded as the primary standard for audio/video transport in IP networks and is used with an associated profile and payload format.

Real-time multimedia streaming applications require timely delivery of information and can tolerate some packet loss to achieve this goal. For example, loss of a packet in audio application may result in loss of a fraction of a second of audio data, which can be made unnoticeable with suitable error concealment algorithms. The Transmission Control Protocol (TCP), although standardized for RTP use, is not normally used in RTP applications because TCP favors reliability over timeliness. Instead the majority of the RTP implementations are built on the User Datagram Protocol (UDP).

H.264 规范中两种重要概念：视屏编码层（VCL）和网络抽象层（NAL）。

VCL 包含Codec的信令处理功能，以及转换，量化，运动补偿预测机制和循环过滤器。

NAL封装VCL编码器输出的片段到网络抽象层单元（NAL Units），它适合于通过网络传输或用于面向包的多路复用环境，用于实际的传输。

Protocol Components

The RTP specification describes two sub-protocols:

1. The data transfer protocol, RTP, which deals with the transfer of real-time data. Information provided by this protocol include timestamps (for synchronization), sequence numbers (for packet loss and recordering detection) and the payload format which indicates the encoded format of the data.

2. The control protocol RTCP is used to specify quality of service (QoS) feedback and synchronization between the media streams. The bandwidth of RTCP traffic compared to RTP is small, typically around 5%.

3. An optional signaling protocol such as H.323, Session Initiation Protocol (SIP), or Jingle(XMPP).

4. An optional media description protocol such Session Description Protocol.

Sessions

An RTP Session is established for each multimedia stream. A session consists of an IP address with a pair of ports for RTP and RTCP.

NAL 网络抽象层单元类型

所有NAL大院有一个单个NAL单元类型字节，其后跟源数据。

1. 单个NAL打包

只要去掉4个自己的开始码就可以了。

在接收端，RTP接收后，加上[00 00 00 01]是4个字节的开始码即可。

2.分片打包

当一个NAL单元比较大时，可以分到多个RTP包中发送。

RTP打包过程：

第一个FU-A包的FU indicator 应该是：F = NALU头中的 F；NRI = NALU 头中的NRI； Type = 28.

FU header 应该是：S = 1； E = 0； R = 0；Type = NALU头中的Type。

RTP解包如下：

分析分片包的前两个字节：

unsigned char startBit = headerStart[1] & 0x80;

        unsigned char endBit = headerStart[1] & 0x40;

1. startBit = 1 分片的首包

m_NAL[0] = (headerStart[0]&oxE0) + (headerStart[1]&0x1F);

重新组合NAL单元类型字节，再在NAL头字节前面加上[00 00 00 01]即得到源码。

2. 分片包的中间包

直接去掉 headerStart[0] headerStart[1]

3. 分片包的末包

直接去掉 headerStart[0] headerStart[1]，并累计以上各包的长度，送给解码器。

Profiles and Payload formats

One of the design considerations of RTP was to carry a range of multimedia formats (such as H.264, MJPEG, MPEG, etc.) and allow new formats to be added without revising the RTP standard. The design of RTP is based on the architectural principle known as application level framing (ALF). The information requred by a specific application's needs is not included in the generic RTP header, but it instead provided through RTP profiles and payload formats. For each class of application (e.g., audio, video), RTP defines a profile and one or more associated payload formats. A complete specification of RTP for a particular application usage will require a profile and payload format specification(s).

The profile defines the codecs used to encode the payload data and their mapping to payload format codes in the Payload Type(PT) field of the RTP header(see below). Each profile is accompanied by several payload format specifications, each of which describes the transport of a particular encoded data.

H264 中NALU注意事项

NALU 类型：1~12 由H.264使用，24~31由H.264以外的应用使用。

重要性标识: 标志该NAL单元用于重建时的重要性，值越大，越重要。

在某些类型的介质上，为了寻址的方便，要求数据流在长度上对齐，或某个常数的整数倍。所以在起始码前添加若干字节的0来填充。