##前言
live555 处理 请求消息 四 “PLAY” 续 中已经写了start stream 流程,但是没有专门写RTP 包完整的组成过程
当我写完流程图,自己都吓了一跳,这么复杂,所以准备后面每个部分拆分
下面组包的前提,已经读取150000个字节在内存中,在谈一谈 live555 (*.mkv) track1 和 track2 的信息获取,已经读取了. 具体 MatroskaFileParser::parseEBMLIdAndSize 方法中用
而读取 内存的指针变量是 curBank()
void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
//....
// Try to read as many new bytes as will fit in the current bank:
unsigned maxNumBytesToRead = BANK_SIZE - fTotNumValidBytes;
fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],
maxNumBytesToRead,
afterGettingBytes, this,
onInputClosure, this);
}
unsigned char* curBank() { return fCurBank; }
知道数据的来源后,就可以解析RTP包的数据
RTP Header 组成
void MultiFramedRTPSink::buildAndSendPacket(Boolean isFirstPacket) {
nextTask() = NULL;
fIsFirstPacket = isFirstPacket;
// Set up the RTP header:
unsigned rtpHdr = 0x80000000; // RTP version 2; marker ('M') bit not set (by default; it can be set later)
rtpHdr |= (fRTPPayloadType<<16);
rtpHdr |= fSeqNo; // sequence number
fOutBuf->enqueueWord(rtpHdr);
// Note where the RTP timestamp will go.
// (We can't fill this in until we start packing payload frames.)
fTimestampPosition = fOutBuf->curPacketSize();
fOutBuf->skipBytes(4); // leave a hole for the timestamp
fOutBuf->enqueueWord(SSRC());
// Allow for a special, payload-format-specific header following the
// RTP header:
fSpecialHeaderPosition = fOutBuf->curPacketSize();
fSpecialHeaderSize = specialHeaderSize();
fOutBuf->skipBytes(fSpecialHeaderSize);
// Begin packing as many (complete) frames into the packet as we can:
fTotalFrameSpecificHeaderSizes = 0;
fNoFramesLeft = False;
fNumFramesUsedSoFar = 0;
packFrame();
}
上面的代码在第一次开始打包的时候,isFirstPacket = true,
所以
unsigned rtpHdr = 0x80000000; RTP Header 根据RTP协议规定 V=2 占用2bit
rtpHdr |= (fRTPPayloadType<<16); 左移16bit 表示负载 96
rtpHdr |= fSeqNo; 序列号
在初始化RTPSink的时候已经初始化 序列号 时间戳 还有SSRC
fSeqNo = (u_int16_t)our_random();
fSSRC = our_random32();
fTimestampBase = our_random32();
最后放入 fOutBuf->enqueueWord(rtpHdr); buffer中
fOutBuf->skipBytes(4); 给时间戳流出4个字节的位置
fOutBuf->enqueueWord(SSRC()); 将SSRC 填充到buffer中
因为第一次 fOutBuf->curPacketSize() = 0 所以后面直接跟 NALU
代表的偏移 fCurOffset
NALU 因为过大,按照 FU-A分片的流程
1)第一个FU-A包的FU indicator:F应该为当前NALU头的F,而NRI应该为当前NALU头的NRI,Type则等于28,表明它是FU-A包。FU header生成方法:S = 1,E = 0,R = 0,Type则等于NALU头中的Type。
2)后续的N个FU-A包的FU indicator和第一个是完全一样的,如果不是最后一个包,则FU header应该为:S = 0,E = 0,R = 0,Type等于NALU头中的Type。
3)最后一个FU-A包FU header应该为:S = 0,E = 1,R = 0,Type等于NALU头中的Type。
同一个NALU分包厚的FU indicator头是完全一致的,FU header只有S以及E位有区别,分别标记开始和结束,它们的RTP分包的序列号应该是依次递增的,并且它们的时间戳必须一致,而负载数据为NALU包去掉1个字节的NALU头后对剩余数据的拆分,这点很关键,你可以认为NALU头被拆分成了FU indicator和FU header,所以不再需要1字节的NALU头了
根据上面流程 分析code :
NALU Indicator & NALU Header
void H264or5Fragmenter::doGetNextFrame() {
if (fNumValidDataBytes == 1) { //第一次读取数据 需要从文件中读取 数据
fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
afterGettingFrame, this,
FramedSource::handleClosure, this);
} else {
//已经读取了数据 但是根据读取的NALU 第一个字节和第二个字节判断 是NALU 单一模式 还是 分片NALU
//这里分成三种1. NALU 单一模式 2. 分片NALU FU-A start 3. FU-A 和 FU-end
fMaxSize = fMaxOutputPacketSize;
fLastFragmentCompletedNALUnit = True; // by default
if (fCurDataOffset == 1) { // case 1 or 2
if (fNumValidDataBytes - 1 <= fMaxSize) { // case 1 NALU 单一模式 只有NALU Header
memmove(fTo, &fInputBuffer[1], fNumValidDataBytes - 1);
envir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<" fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";
} else { // case 2 FU-A start FU indicator+FU header
if (fHNumber == 264) {
fInputBuffer[0] = (fInputBuffer[1] & 0xE0) | 28; // FU indicator
fInputBuffer[1] = 0x80 | (fInputBuffer[1] & 0x1F); // FU header (with S bit)
}
memmove(fTo, fInputBuffer, fMaxSize);
envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";
}
} else { // case 3 FU-A & Fu-A end FU indicator+FU header
unsigned numExtraHeaderBytes;
envir() << "fCurDataOffset = " << fCurDataOffset << " \n";
if (fHNumber == 264) {
fInputBuffer[fCurDataOffset-2] = fInputBuffer[0]; // FU indicator
fInputBuffer[fCurDataOffset-1] = fInputBuffer[1]&~0x80; // FU header (no S bit)
numExtraHeaderBytes = 2;
}
//根据发送包的大小拆分, 判断是中间的FU-A 还是 如果最后包大小小于最大允许发送大小就是 FU-A end
if (numBytesToSend > fMaxSize) {
// We can't send all of the remaining data this time:
numBytesToSend = fMaxSize;
fLastFragmentCompletedNALUnit = False;
} else {
// This is the last fragment:
fInputBuffer[fCurDataOffset-1] |= 0x40; // set the E bit in the FU header
fNumTruncatedBytes = fSaveNumTruncatedBytes;
}
memmove(fTo, &fInputBuffer[fCurDataOffset-numExtraHeaderBytes], numBytesToSend);
envir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " << fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";
}
//.......
FramedSource::afterGetting(this);
}
}
这里就是处理NALU header 地方并且针对不同的情况加了log
//第一种情况 NALU 单一模式 不需要分片 只有8个字节的NALU Header
envir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<"
fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";
//第二种情况 NALU 需要分片 分片FU-A start 所以会有FU indicator + header 2个字节
envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1]
= " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";
//第三种情况 NALU 需要分片 分片FU-A 和 FU-A end 唯一能区分就是包是否小于可允许最大size 如果是 那就是最后一个FU-A,反之分包并发送中间的FU-A
envir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << "
fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " <<
fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";
参考 log如下
若只要fTo[0] 就是NALU Header 值,有fTo[0] 和fTo[1]的值,表示 FUindicator 和 FU header
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 103 fInputBuffer[1] = 103 //SPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 104 fInputBuffer[1] = 104 //PPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 6 fInputBuffer[1] = 6 //SEI
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 101 fInputBuffer[1] = 101 //IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
//FU-A start
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
//FU-A end
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[2884] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[1442] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
....
上面都是根据 fMaxSize 判断 是否应该使用分片NALU ,fMaxSize = 1444 ,计算过程如下 1456-12(RTP hdr) ourMaxPacketSize() = 1456
fOurFragmenter = new H264or5Fragmenter(fHNumber, envir(), fSource, OutPacketBuffer::maxSize,ourMaxPacketSize() - 12/*RTP hdr size*/);
//OutPacketBuffer::maxSize = 60000
setPacketSizes((RTP_PAYLOAD_PREFERRED_SIZE), (RTP_PAYLOAD_MAX_SIZE));
RTP_PAYLOAD_MAX_SIZE = 1456
具体的参数计算 可能需要自己加log 计算后面需要详细知道参数获取过程