现在来分析live555中关于H264的处理部分,主要包括从文件中读取数据进行并进行frame(NALU)的分割,然后对frame进行分片,这些工作都是在frame交给RTP sink之前完成的。接着上篇分析文章(RTP的打包与发送)中提到的MultiFramedRTP::packFrame函数进行分析。
- void MultiFramedRTPSink::packFrame() {
- if (fOutBuf->haveOverflowData()) {
- ...
- } else {
- ...
- //
- //从source中获取下一个frame
- //
- fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),
- afterGettingFrame, this, ourHandleClosure, this);
- }
- }
getNextFrame是定义在FramedSource中的非虚函数,从source中获取下一个frame,然后调用回调函数afterGettingFrame。afterGettingFrame被定义为静态函数,因为在C++中类成员函数是不能作为回调用函数的。不过这里为什么要用回调函数回?
注意,对于H264来说,上面的fSource并不是MPEGVideoStreamFramer类,因为在 H264VideoRTPSink::continuePlaying()函数中改变了fSource的值。
- Boolean H264VideoRTPSink::continuePlaying() {
- // First, check whether we have a 'fragmenter' class set up yet.
- // If not, create it now:
- if (fOurFragmenter == NULL) {
- //创建一个辅助类H264FUAFragmenter,用于H264按照RFC3984进行RTP打包
- fOurFragmenter = new H264FUAFragmenter(envir(), fSource, OutPacketBuffer::maxSize,
- ourMaxPacketSize() - 12/*RTP hdr size*/);
- fSource = fOurFragmenter;
- }
- // Then call the parent class's implementation:
- return MultiFramedRTPSink::continuePlaying();
- }
fSource被指向了H264FUAFragmenter类,这个类主要实现了H264按照RFC3984进行RTP分包,不过这里的实现每个RTP中最多只包含一个NALU,没有实现组合封包的情形。这个类的继承关系如下:H264FUAFragmenter->FramedFilter->FramedSource。很明显,这是一个filter,包装了MPEGVideoStreamFramer类的对像。
先来看来看getNextFrame的实现
- void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,
- afterGettingFunc* afterGettingFunc,
- void* afterGettingClientData,
- onCloseFunc* onCloseFunc,
- void* onCloseClientData) {
- // Make sure we're not already being read:
- if (fIsCurrentlyAwaitingData) {
- envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";
- envir().internalError();
- }
- fTo = to; //buffer地址
- fMaxSize = maxSize; //buffer最大长度
- fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()
- fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()
- fAfterGettingFunc = afterGettingFunc; //获取完一个frame后将执行这个函数
- fAfterGettingClientData = afterGettingClientData; //这个参数就是MultiFramedRTPSink类型指针
- fOnCloseFunc = onCloseFunc;
- fOnCloseClientData = onCloseClientData;
- fIsCurrentlyAwaitingData = True;
- doGetNextFrame();
- }
上面的函数主要是进行一些成员变量的初始化,获取到的frame需要保存到fTo地址中,然后调用fAfterGettingFunc函数,若文件读取完毕,还需要调用fOnCloseFunc函数。重要的工作还是在doGetNextFrame函数中完成,不过它是定义在FramedSource类中的纯虚函数,需要在子类中重新实现。
现在来看H264FUAFragmenter中对doGetNextFrame的实现
- void H264FUAFragmenter::doGetNextFrame() {
- if (fNumValidDataBytes == 1) {
- //读取一个新的frame
- // We have no NAL unit data currently in the buffer. Read a new one:
- fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
- afterGettingFrame, this,
- FramedSource::handleClosure, this);
- } else {
- //
- //现在buffer中已经存在NALU数据了,需要考虑三种情况
- //1.一个新的NALU,且足够小能投递给RTP sink。
- //2.一个新的NALU,但是比RTP sink要求的包大了,投递第一个分片作为一个FU-A packet, 并带上一个额外的头字节。
- //3.部分NALU数据,投递下一个分片作为一个FU-A packet,并带上2个额外的头字节。
- // We have NAL unit data in the buffer. There are three cases to consider:
- // 1. There is a new NAL unit in the buffer, and it's small enough to deliver
- // to the RTP sink (as is).
- // 2. There is a new NAL unit in the buffer, but it's too large to deliver to
- // the RTP sink in its entirety. Deliver the first fragment of this data,
- // as a FU-A packet, with one extra preceding header byte.
- // 3. There is a NAL unit in the buffer, and we've already delivered some
- // fragment(s) of this. Deliver the next fragment of this data,
- // as a FU-A packet, with two extra preceding header bytes.
- if (fMaxSize < fMaxOutputPacketSize) { // shouldn't happen
- envir() << "H264FUAFragmenter::doGetNextFrame(): fMaxSize ("
- << fMaxSize << ") is smaller than expected\n";
- } else {
- fMaxSize = fMaxOutputPacketSize;
- }
- fLastFragmentCompletedNALUnit = True; // by default
- if (fCurDataOffset == 1) { // case 1 or 2
- if (fNumValidDataBytes - 1 <= fMaxSize) { // case 1
- //
- //情况1, 处理整个NALU
- //
- memmove(fTo, &fInputBuffer[1], fNumValidDataBytes - 1);
- fFrameSize = fNumValidDataBytes - 1;
- fCurDataOffset = fNumValidDataBytes;
- } else { // case 2
- //
- //情况2,处理NALU的第1个分片。注意,我们添加FU指示符和FU头字节(with S bit)到包的最前面(
- //重用已经存在的NAL 头字节作为FU的头字节)
- //
- // We need to send the NAL unit data as FU-A packets. Deliver the first
- // packet now. Note that we add FU indicator and FU header bytes to the front
- // of the packet (reusing the existing NAL header byte for the FU header).
- fInputBuffer[0] = (fInputBuffer[1] & 0xE0) | 28; // FU indicator
- fInputBuffer[1] = 0x80 | (fInputBuffer[1] & 0x1F); // FU header (with S bit) 重用NALU头字节
- memmove(fTo, fInputBuffer, fMaxSize);
- fFrameSize = fMaxSize;
- fCurDataOffset += fMaxSize - 1;
- fLastFragmentCompletedNALUnit = False;
- }
- } else { // case 3
- //
- //情况3,处理非第1个分片。需要添加FU指示符和FU头(我们重用了第一个分片中的字节,但是需要清除S位,
- //并在最后一个分片中添加E位)
- //
- //
- // We are sending this NAL unit data as FU-A packets. We've already sent the
- // first packet (fragment). Now, send the next fragment. Note that we add
- // FU indicator and FU header bytes to the front. (We reuse these bytes that
- // we already sent for the first fragment, but clear the S bit, and add the E
- // bit if this is the last fragment.)
- fInputBuffer[fCurDataOffset-2] = fInputBuffer[0]; // FU indicator
- fInputBuffer[fCurDataOffset-1] = fInputBuffer[1]&~0x80; // FU header (no S bit)
- unsigned numBytesToSend = 2 + fNumValidDataBytes - fCurDataOffset;
- if (numBytesToSend > fMaxSize) {
- // We can't send all of the remaining data this time:
- numBytesToSend = fMaxSize;
- fLastFragmentCompletedNALUnit = False;
- } else {
- //
- //最后一个分片,需要在FU头中设置E标志位
- // This is the last fragment:
- fInputBuffer[fCurDataOffset-1] |= 0x40; // set the E bit in the FU header
- fNumTruncatedBytes = fSaveNumTruncatedBytes;
- }
- memmove(fTo, &fInputBuffer[fCurDataOffset-2], numBytesToSend);
- fFrameSize = numBytesToSend;
- fCurDataOffset += numBytesToSend - 2;
- }
- if (fCurDataOffset >= fNumValidDataBytes) {
- // We're done with this data. Reset the pointers for receiving new data:
- fNumValidDataBytes = fCurDataOffset = 1;
- }
- // Complete delivery to the client:
- FramedSource::afterGetting(this);
- }
- }
H264FUAFragmenter::doGetNextFrame函数第一次执行时,执行条件1,需要调用 MPEGVideoStreamFramer::doGetNextFrame读取一个新的frame,获取frame的具体过程稍后再分析。现在先看获取frame之后的工作,afterGettingFrame函数
- void H264FUAFragmenter::afterGettingFrame(void* clientData, unsigned frameSize,
- unsigned numTruncatedBytes,
- struct timeval presentationTime,
- unsigned durationInMicroseconds) {
- H264FUAFragmenter* fragmenter = (H264FUAFragmenter*)clientData;
- fragmenter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime,
- durationInMicroseconds);
- }
没什么好说的,再看afterGettingFrame1函数
- void H264FUAFragmenter::afterGettingFrame1(unsigned frameSize,
- unsigned numTruncatedBytes,
- struct timeval presentationTime,
- unsigned durationInMicroseconds) {
- fNumValidDataBytes += frameSize; //保存读到的frame长度
- fSaveNumTruncatedBytes = numTruncatedBytes;
- fPresentationTime = presentationTime;
- fDurationInMicroseconds = durationInMicroseconds;
- // Deliver data to the client:
- doGetNextFrame();
- }
上面的代码首先记录几个数据到成员变量中,fNumValidDataBytes很重要,表示读取到的frame长度+1。然后又一次调用了H264FUAFragmenter::doGetNextFrame(),这里将进入H264FUAFragmenter::doGetNextFrame函数中第二个条件分支,这种循环调用很容易把人弄迷糊了。
H264FUAFragmenter::doGetNextFrame函数中第二个条件分支中,处理H264的RTP分片问题,这里是按照RFC3984进行RTP封装的。你应该注意到,在上篇文章"RTP的打包与发送"中,也出现了分片的代码(MultiFramedRTPSink::packFrame函数中),那里直接将frame按MTU的长度来拆分。那为什么H264还要自定义一套RTP打包的标准呢?暂时我也不清楚。
- void FramedSource::afterGetting(FramedSource* source) {
- source->fIsCurrentlyAwaitingData = False; //表示已经获取到数据了,处于非等待状态
- // indicates that we can be read again
- // Note that this needs to be done here, in case the "fAfterFunc"
- // called below tries to read another frame (which it usually will)
- //通过回调用进行后续处理
- if (source->fAfterGettingFunc != NULL) {
- (*(source->fAfterGettingFunc))(source->fAfterGettingClientData,
- source->fFrameSize, source->fNumTruncatedBytes,
- source->fPresentationTime,
- source->fDurationInMicroseconds);
- }
- }
上面的代码主要是调用了FramedSource::getNextFrame函数中传递下来的回调函数,这个回调函数就是MultiFramedRTPSink::afterGettingFrame,处理过程在上一篇文章"RTP的打包与发送"中已经分析过了。
现在来看MPEGVideoStreamFramer::doGetNextFrame获取Frame的过程。继承关系:H264VideoStreamFramer->MPEGVideoStreamFramer->FramedFilter->FramedSource。在继承路径中存在FrameFilter,这说明H264VideoStreamFramer包装了其它source(包装的是读取文件的字节流source)。doGetNextFrame函数首先在MPEGVideoStreamFramer中实现。
- void MPEGVideoStreamFramer::doGetNextFrame() {
- fParser->registerReadInterest(fTo, fMaxSize); //将目的buffer信息注册到语法分析类中
- continueReadProcessing(); //继续进行读数据
- }
这里的MPEGVideoStreamFramer::fParser,是一个MPEGVideoStreamParser类型指针,作为语法分析器。再来看continueReadProcessing函数
- void MPEGVideoStreamFramer::continueReadProcessing() {
- unsigned acquiredFrameSize = fParser->parse(); //文件的语法分析(即demux)
- if (acquiredFrameSize > 0) {
- // We were able to acquire a frame from the input.
- // It has already been copied to the reader's space.
- fFrameSize = acquiredFrameSize;
- fNumTruncatedBytes = fParser->numTruncatedBytes();
- // "fPresentationTime" should have already been computed.
- // Compute "fDurationInMicroseconds" now:
- fDurationInMicroseconds
- = (fFrameRate == 0.0 || ((int)fPictureCount) < 0) ? 0
- : (unsigned)((fPictureCount*1000000)/fFrameRate);
- #ifdef DEBUG
- fprintf(stderr, "%d bytes @%u.%06d, fDurationInMicroseconds: %d ((%d*1000000)/%f)\n", acquiredFrameSize, fPresentationTime.tv_sec, fPresentationTime.tv_usec, fDurationInMicroseconds, fPictureCount, fFrameRate);
- #endif
- fPictureCount = 0;
- //
- //调用自身的afterGetting函数,因为这不一个"leaf" source, 所以可能直接调用,
- //而不用担心出现无限递归
- //
- // Call our own 'after getting' function. Because we're not a 'leaf'
- // source, we can call this directly, without risking infinite recursion.
- afterGetting(this);
- } else {
- // We were unable to parse a complete frame from the input, because:
- // - we had to read more data from the source stream, or
- // - the source stream has ended.
- }
- }
函数中首先调用了MPEGVideoStreamParser::parse函数,将一个完整的Frame分析出来,并copy到了fTo(fTo就是OutPacketBuffer中的缓冲区)中,这其中肯定也实现了从文件中读取数据的过程。这里的fNumTruncatedBytes变量需要注意,fNumTruncatedBytes>0的话,表明Frame的实际长度大于fTo的最大长度,这将导致数据丢失,这时就要考虑增加缓冲区的长度了。成功获取一个Frame后,将调用afterGetting函数处理后续工作。
先来看parse函数,parse是定义在MPEGVideoStreamParser中的纯虚函数,在子类H264VideoStreamParser中实现。parse主要是从文件的字节流中,分离出一个个的Frame,对于H264而言其实就是对一个个的NALU。*.264文件的格式非常简单,每个NALU以 0x00000001 作为起始符号(中间的NALU也可以以0x000001作为起始符),顺序存放。
- unsigned H264VideoStreamParser::parse() {
- try {
- //
- //首先找到起始符号, 并跳过。文件流的最开始必需以0x00000001开始,但后续的NALU充许以0x000001(3 bytes)作为分隔
- //
- // The stream must start with a 0x00000001:
- if (!fHaveSeenFirstStartCode) {
- // Skip over any input bytes that precede the first 0x00000001:
- u_int32_t first4Bytes;
- while ((first4Bytes = test4Bytes()) != 0x00000001) {
- get1Byte(); setParseState(); // ensures that we progress over bad data
- }
- skipBytes(4); // skip this initial code
- setParseState();
- fHaveSeenFirstStartCode = True; // from now on
- }
- //
- //将起始标志也保存到目的缓冲区中
- //
- if (fOutputStartCodeSize > 0) {
- // Include a start code in the output:
- save4Bytes(0x00000001);
- }
- //
- //保存所有数据,直至遇到起始标志,或者文件结束符。需要注意NALU中的第一个字节,因为它指示了NALU的类型
- //
- // Then save everything up until the next 0x00000001 (4 bytes) or 0x000001 (3 bytes), or we hit EOF.
- // Also make note of the first byte, because it contains the "nal_unit_type":
- u_int8_t firstByte;
- if (haveSeenEOF()) {
- //
- //已经设置了文件结束标志,将剩下的数据保存也来即可
- //
- // We hit EOF the last time that we tried to parse this data,
- // so we know that the remaining unparsed data forms a complete NAL unit:
- unsigned remainingDataSize = totNumValidBytes() - curOffset();
- if (remainingDataSize == 0) (void)get1Byte(); // forces another read, which will cause EOF to get handled for real this time
- #ifdef DEBUG
- fprintf(stderr, "This NAL unit (%d bytes) ends with EOF\n", remainingDataSize);
- #endif
- if (remainingDataSize == 0) return 0;
- firstByte = get1Byte(); //将第一个字节保存下来,其指示了NALU的类型
- saveByte(firstByte);
- while (--remainingDataSize > 0) {
- saveByte(get1Byte());
- }
- } else {
- u_int32_t next4Bytes = test4Bytes();
- firstByte = next4Bytes>>24; //将第一个字节保存下来
- //
- //将下一个起始符号之前的数据都保存下来
- //
- while (next4Bytes != 0x00000001 && (next4Bytes&0xFFFFFF00) != 0x00000100) {
- // We save at least some of "next4Bytes".
- if ((unsigned)(next4Bytes&0xFF) > 1) { //一次可以保存4个字节,并不需要一个一个字节对比,除非到了结尾
- // Common case: 0x00000001 or 0x000001 definitely doesn't begin anywhere in "next4Bytes", so we save all of it:
- save4Bytes(next4Bytes);
- skipBytes(4);
- } else {
- // Save the first byte, and continue testing the rest:
- saveByte(next4Bytes>>24);
- skipBytes(1);
- }
- next4Bytes = test4Bytes();
- }
- // Assert: next4Bytes starts with 0x00000001 or 0x000001, and we've saved all previous bytes (forming a complete NAL unit).
- // Skip over these remaining bytes, up until the start of the next NAL unit:
- if (next4Bytes == 0x00000001) {
- skipBytes(4);
- } else {
- skipBytes(3);
- }
- }
- u_int8_t nal_ref_idc = (firstByte&0x60)>>5;
- u_int8_t nal_unit_type = firstByte&0x1F;
- #ifdef DEBUG
- fprintf(stderr, "Parsed %d-byte NAL-unit (nal_ref_idc: %d, nal_unit_type: %d (\"%s\"))\n",
- curFrameSize()-fOutputStartCodeSize, nal_ref_idc, nal_unit_type, nal_unit_type_description[nal_unit_type]);
- #endif
- //
- //下面根据NALU的类型来作具体的分析
- //
- switch (nal_unit_type) {
- case 6: { // Supplemental enhancement information (SEI)
- analyze_sei_data();
- // Later, perhaps adjust "fPresentationTime" if we saw a "pic_timing" SEI payload??? #####
- break;
- }
- case 7: { // Sequence parameter set (序列参数集)
- //
- //保存一份SPS的副本到H264VideoStreamFramer中,后面的pps也需要保存,sps中可能还包含了帧率信息
- //
- // First, save a copy of this NAL unit, in case the downstream object wants to see it:
- usingSource()->saveCopyOfSPS(fStartOfFrame + fOutputStartCodeSize, fTo - fStartOfFrame - fOutputStartCodeSize);
- // Parse this NAL unit to check whether frame rate information is present:
- unsigned num_units_in_tick, time_scale, fixed_frame_rate_flag;
- analyze_seq_parameter_set_data(num_units_in_tick, time_scale, fixed_frame_rate_flag);
- if (time_scale > 0 && num_units_in_tick > 0) {
- usingSource()->fFrameRate = time_scale/(2.0*num_units_in_tick); //sps中包含了帧率信息
- #ifdef DEBUG
- fprintf(stderr, "Set frame rate to %f fps\n", usingSource()->fFrameRate);
- if (fixed_frame_rate_flag == 0) {
- fprintf(stderr, "\tWARNING: \"fixed_frame_rate_flag\" was not set\n");
- }
- #endif
- } else { //sps中不包含帧率信息,则使用source中设置的默认帧率
- #ifdef DEBUG
- fprintf(stderr, "\tThis \"Sequence Parameter Set\" NAL unit contained no frame rate information, so we use a default frame rate of %f fps\n", usingSource()->fFrameRate);
- #endif
- }
- break;
- }
- case 8: { // Picture parameter set (图像参数集PPS)
- // Save a copy of this NAL unit, in case the downstream object wants to see it:
- usingSource()->saveCopyOfPPS(fStartOfFrame + fOutputStartCodeSize, fTo - fStartOfFrame - fOutputStartCodeSize);
- }
- }
- usingSource()->setPresentationTime(); //设置当前的时间
- #ifdef DEBUG
- unsigned long secs = (unsigned long)usingSource()->fPresentationTime.tv_sec;
- unsigned uSecs = (unsigned)usingSource()->fPresentationTime.tv_usec;
- fprintf(stderr, "\tPresentation time: %lu.%06u\n", secs, uSecs);
- #endif
- //
- //如果这个NALU是一个VCL NALU(即包含的是视频数据),我们需要扫描下一个NALU的起始符,
- //以判断这个NALU是否是当前的"access unit"(这个"access unit"应该可以理解为一帧图像帧吧)。
- //我们需要根据这个信息去指明何时该增加"fPresentationTime"(RTP 打包时也需要根据这个信息,决定是否设置"M"位)。
- //
- // If this NAL unit is a VCL NAL unit, we also scan the start of the next NAL unit, to determine whether this NAL unit
- // ends the current 'access unit'. We need this information to figure out when to increment "fPresentationTime".
- // (RTP streamers also need to know this in order to figure out whether or not to set the "M" bit.)
- Boolean thisNALUnitEndsAccessUnit = False; // until we learn otherwise
- if (haveSeenEOF()) {
- // There is no next NAL unit, so we assume that this one ends the current 'access unit':
- thisNALUnitEndsAccessUnit = True;
- } else {
- Boolean const isVCL = nal_unit_type <= 5 && nal_unit_type > 0; // Would need to include type 20 for SVC and MVC #####
- if (isVCL) {
- u_int32_t first4BytesOfNextNALUnit = test4Bytes();
- u_int8_t firstByteOfNextNALUnit = first4BytesOfNextNALUnit>>24;
- u_int8_t next_nal_ref_idc = (firstByteOfNextNALUnit&0x60)>>5;
- u_int8_t next_nal_unit_type = firstByteOfNextNALUnit&0x1F;
- if (next_nal_unit_type >= 6) {
- //下一个NALU不是VCL的,当前的"access unit"结束了
- // The next NAL unit is not a VCL; therefore, we assume that this NAL unit ends the current 'access unit':
- #ifdef DEBUG
- fprintf(stderr, "\t(The next NAL unit is not a VCL)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else {
- //下一个NALU也是VCL的,还需要检查一下它们是不是属于同一个"access unit"
- // The next NAL unit is also a VLC. We need to examine it a little to figure out if it's a different 'access unit'.
- // (We use many of the criteria described in section 7.4.1.2.4 of the H.264 specification.)
- Boolean IdrPicFlag = nal_unit_type == 5;
- Boolean next_IdrPicFlag = next_nal_unit_type == 5;
- if (next_IdrPicFlag != IdrPicFlag) {
- // IdrPicFlag differs in value
- #ifdef DEBUG
- fprintf(stderr, "\t(IdrPicFlag differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if (next_nal_ref_idc != nal_ref_idc && next_nal_ref_idc*nal_ref_idc == 0) {
- // nal_ref_idc differs in value with one of the nal_ref_idc values being equal to 0
- #ifdef DEBUG
- fprintf(stderr, "\t(nal_ref_idc differs in value with one of the nal_ref_idc values being equal to 0)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if ((nal_unit_type == 1 || nal_unit_type == 2 || nal_unit_type == 5)
- && (next_nal_unit_type == 1 || next_nal_unit_type == 2 || next_nal_unit_type == 5)) {
- // Both this and the next NAL units begin with a "slice_header".
- // Parse this (for each), to get parameters that we can compare:
- // Current NAL unit's "slice_header":
- unsigned frame_num, pic_parameter_set_id, idr_pic_id;
- Boolean field_pic_flag, bottom_field_flag;
- analyze_slice_header(fStartOfFrame + fOutputStartCodeSize, fTo, nal_unit_type,
- frame_num, pic_parameter_set_id, idr_pic_id, field_pic_flag, bottom_field_flag);
- // Next NAL unit's "slice_header":
- #ifdef DEBUG
- fprintf(stderr, " Next NAL unit's slice_header:\n");
- #endif
- u_int8_t next_slice_header[NUM_NEXT_SLICE_HEADER_BYTES_TO_ANALYZE];
- testBytes(next_slice_header, sizeof next_slice_header);
- unsigned next_frame_num, next_pic_parameter_set_id, next_idr_pic_id;
- Boolean next_field_pic_flag, next_bottom_field_flag;
- analyze_slice_header(next_slice_header, &next_slice_header[sizeof next_slice_header], next_nal_unit_type,
- next_frame_num, next_pic_parameter_set_id, next_idr_pic_id, next_field_pic_flag, next_bottom_field_flag);
- if (next_frame_num != frame_num) {
- // frame_num differs in value
- #ifdef DEBUG
- fprintf(stderr, "\t(frame_num differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if (next_pic_parameter_set_id != pic_parameter_set_id) {
- // pic_parameter_set_id differs in value
- #ifdef DEBUG
- fprintf(stderr, "\t(pic_parameter_set_id differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if (next_field_pic_flag != field_pic_flag) {
- // field_pic_flag differs in value
- #ifdef DEBUG
- fprintf(stderr, "\t(field_pic_flag differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if (next_bottom_field_flag != bottom_field_flag) {
- // bottom_field_flag differs in value
- #ifdef DEBUG
- fprintf(stderr, "\t(bottom_field_flag differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- } else if (next_IdrPicFlag == 1 && next_idr_pic_id != idr_pic_id) {
- // IdrPicFlag is equal to 1 for both and idr_pic_id differs in value
- // Note: We already know that IdrPicFlag is the same for both.
- #ifdef DEBUG
- fprintf(stderr, "\t(IdrPicFlag is equal to 1 for both and idr_pic_id differs in value)\n");
- #endif
- thisNALUnitEndsAccessUnit = True;
- }
- }
- }
- }
- }
- if (thisNALUnitEndsAccessUnit) {
- #ifdef DEBUG
- fprintf(stderr, "*****This NAL unit ends the current access unit*****\n");
- #endif
- usingSource()->fPictureEndMarker = True; //这里就是设置RTP打包时用到的M标志了
- ++usingSource()->fPictureCount;
- //
- //下一个NALU不再属于当前"access unit""时,才改变时间
- //
- // Note that the presentation time for the next NAL unit will be different:
- struct timeval& nextPT = usingSource()->fNextPresentationTime; // alias 这里是引用
- nextPT = usingSource()->fPresentationTime;
- double nextFraction = nextPT.tv_usec/1000000.0 + 1/usingSource()->fFrameRate;
- unsigned nextSecsIncrement = (long)nextFraction;
- nextPT.tv_sec += (long)nextSecsIncrement;
- nextPT.tv_usec = (long)((nextFraction - nextSecsIncrement)*1000000);
- }
- setParseState();
- return curFrameSize();
- } catch (int /*e*/) {
- #ifdef DEBUG
- fprintf(stderr, "H264VideoStreamParser::parse() EXCEPTION (This is normal behavior - *not* an error)\n");
- #endif
- return 0; // the parsing got interrupted
- }
- }
H264VideoStreamParser::parse()函数除了取出Frame,还对NALU中的部分参数做了解释工作。对于PPS或者SPS类型的NALU,要保存到H264VideoStreamFramer中。"access unit",在这里可以理解为一副图像,一个"access unit"可以包含多个NALU,很显示这些NALU的时间戳应该是相同的。实际上,很多时候一个"access unit"单元只包含一个NALU,这里简单多了。分析过程是根据section 7.4.1.2.4 of the H.264 specification进行的。
FROM: http://blog.csdn.net/gavinr/article/details/7042228