# 多媒体通信基础

## 一路媒体流

case 1:

case 2：

### RTP的流内同步

a) 发送端：

$timeStam{p}_{i}=timeStam{p}_{i-1}+\mathrm{\Delta }timestamp$$timeStamp_{i} =timeStamp_{i-1} + \Delta{timestamp}$ 其中，$\mathrm{\Delta }timestamp$$\Delta{timestamp}$ 是第i帧和第i-1帧之间的时间戳增量，$\mathrm{\Delta }timestamp=frequency/fps$$\Delta{timestamp} = frequency/fps$ ,即一帧的样本增量。一般，第一帧的timeStamp取值为随机值。

b) 接收端：

$presentTim{e}_{i}=presentTim{e}_{i-1}+\mathrm{\Delta }timestamp/frequency$$presentTime_{i} = presentTime_{i-1} + \Delta{timestamp}/frequency$

Buffering reduces a system’s sensitivity to short-term fluctuations in the data arrival rate by absorbing variations in end-to-end delay and allowing margin for retransmission attempts when packets are lost.

### Jitter

The network delivers RTP packets asynchronously, with variable delays. To be able to play the audio stream with reasonable quality, the receiving endpoint needs to turn the variable delays into constant delays. This can be done by using a jitter buffer.

The jitter buﬀer is used to compensate transmission impairments which appears because of the time-variant delays of received data packet.

Generally the larger the jitter buffer is, the bigger the added delay and the more packets that are successfully played out. Unfortunately this additional delay lowers the perceived QoS. On the other hand, if the playout delay is set too low, the network-induced delay will cause some packets to arrive too late for playout and thus be lost, which also lowers the perceived QoS. The main objective of jitter buffering is to keep the packet loss rate under 5% and to keep the end-to-end delay as small as possible.

p = t * r + offset + jitter-delay.

•p = playout time

•t = timestamp of DU

•r = period of DU clock

•offset = clock offset

•jitter-delay = fix size jitter delay allowance to p.

•Variance allowance = 60 ms

•Media timestamp period = 30 ms

•First DU:

–Media Timestamp = 10

–System Clock = 1000

•Solve for offset:

–1000 = 10 * 30 + offset => offset = 700

–p = t * 30 + 700 + 60

Packet TS ArrivalTime PlayoutTime
a 10 1000 1060
b 11 1025 1090
c 12 1050 1120
d 13 1095 1150
e 14 1185 1180 [late lost]

## 音频流+视频流

The idea behind the audio-video synchronization process is that the adaptive playout algorithms are performed first, and the video frames are played out dependent on the playout times of their corresponding audio packets. This is done by storing the video frames in a video playout buffer and by delaying each frame until the corresponding audio packets are played out. The correspondence between audio and video frames is given by their timestamps.

If the video quality is not acceptable using this scheme which gives priority to audio, additional measures may be needed to improve the video playout, e.g. adaptive adjusting of the video frame playout process.

### Rtp/rtcp的流间同步

rtp媒体流有自己的时钟频率[采样率]和起始时刻[startTimestamp]：

${t}_{i}=startTimestamp+\mathrm{\Delta }samples$$t_{i} = startTimestamp +\Delta samples$

wall clock 作为不同媒体流的参考时钟，被用来指示媒体流间的time align。

$Cloc{k}_{i}⇔Cloc{k}_{wall}$$Clock_{i} \Leftrightarrow Clock_{wall}$

Rtcp协议中有一类包为SR包，是发送端报告(sender report)，SR包中有两个时间戳信息，一个记录wall clock time，另一个记录media clock time。有了这两个信息，接收端就可以将media clock time映射到wall clock time上。

$t/frequency=T+offset⇒offset=T-t/frequency$$t/frequency = T + offset \Rightarrow offset = T - t/frequency$

${t}_{i}$$t_{i}$为第i帧媒体数据的media clock time，则其对应的wall clock time ${T}_{i}$$T_{i}$

${T}_{i}={t}_{i}/frequency-offset$$T_{i} = t_{i}/frequency - offset$

• 评论

• 下一篇
• 上一篇