一. 视频传输到解码
VideoCodingModuleImpl类其实并没有使用
函数调用次顺:
VideoReceiveStream::DeliverRtp->RtpStreamReceiver::DeliverRtp->RtpReceiverImpl::IncomingRtpPacket->RTPReceiverVideo::ParseRtpPacket->
RtpStreamReceiver::OnReceivedPayloadData->VideoReceiver::IncomingPacket[进入视频编解码模块]
Module/video_coding/video_coding_impl.h
VideoReceiver
1. int32_t RegisterReceiveCallback(VCMReceiveCallback*receiveCallback);
注册解码后的视频帧回调对象,通过其虚函数FrameToRender回调,在VideoStreamDecoder构造函数,注册回调对象 video_receiver_->RegisterReceiveCallback(this);
2. Video/video_stream_decoder.h
使用者对象
VideoStreamDecoder: publicVCMReceiveCallback
1. int32_t FrameToRender(VideoFrame&video_frame) override; // NOLINT
视频解码线程
VideoReceiveStream::Start->
VideoReceiveStream::DecodeThreadFunction-> VideoReceiveStream::Decode->VideoReceiver::Decode(uint16_t maxWaitTimeMs)->
VideoReceiver::Decode(const VCMEncodedFrame& frame)-> VideoReceiveStream::Encoded[解复用完整帧通过VideoReceiveStream构造函数中VideoReceiver的构造传入VideoReceiveStream对象作为回调]->VCMGenericDecoder::Decode(constVCMEncodedFrame& frame, int64_t nowMs)->
[VCMCodecDataBase::GetDecoder->VCMGenericDecoder::RegisterDecodeCompleteCallback->
VP9DecoderImpl::RegisterDecodeCompleteCallback注册解码后数据回调对]->
VP9DecoderImpl::Decode(const EncodedImage& input_image,
bool missing_frames,
constRTPFragmentationHeader* fragmentation,
constCodecSpecificInfo* codec_specific_info,
int64_t/*render_time_ms*/) ->DecodedImageCallback::Decoded(父)->VCMDecodedFrameCallback::Decoded (VideoFrame&decodedImage) (子)-> VCMDecodedFrameCallback::Decoded(VideoFrame&decodedImage,int64_t decode_time_ms)->
VideoStreamDecoder::FrameToRender(VideoFrame&video_frame)->
[接收解码对象通过VideoStreamDecoder::VideoStreamDecoder ->VideoReceiver::RegisterReceiveCallback(this)–>VCMDecodedFrameCallback::SetUserReceiveCallback]
IncomingVideoStream::OnFrame[将帧放入队列VideoRenderFrames]->
视频显示|录像线程
VideoReceiveStream::Start->IncomingVideoStream::Start->
IncomingVideoStream::IncomingVideoStreamThreadFun[从VideoRenderFrames队列获取帧]->IncomingVideoStream::DeliverFrame->VideoReceiveStream::OnFrame[该回调函数由VideoReceiveStream::VideoReceiveStream中注册incoming_video_stream_.SetExternalCallback(this)]-> WebRtcVideoReceiveStream::OnFrame(constwebrtc::VideoFrame& frame) [ WebRtcVideoReceiveStream::WebRtcVideoReceiveStream构造函数赋值config_.renderer = this]
[设置广播接收者PeerConnection::SetRemoteDescription -> PeerConnection::OnRemoteTrackSeen-> PeerConnection::CreateVideoReceiver[new VideoRtpReceiver]->VideoRtpReceiver::VideoRtpReceiver[provider_->SetVideoPlayout(ssrc_,true,&broadcaster_)]->WebRtcSession::SetVideoPlayout-> VideoChannel::SetSink->
WebRtcVideoChannel2::SetSink-> WebRtcVideoReceiveStream::SetSink(sink_)]
总结:
- 从网络接收线程输入到显示线程输出(中间解码线程)
VideoReceiveStream::DeliverRtp[已复用的视频RTP]->
VideoReceiveStream::OnFrame[已解码的视频帧]
- 从网络接收RTP到解复用完整帧
VideoReceiveStream::DeliverRtp[已复用的视频RTP]->
int32_t VideoReceiveStream::Encoded(
constEncodedImage& encoded_image,
constCodecSpecificInfo* codec_specific_info,
const RTPFragmentationHeader*fragmentation)
RtpStreamReceiver作为Video_Coding模块和RTP/RTCP 模块的中介者
#include"webrtc/video/rtp_stream_receiver.h"
RtpStreamReceiver
1. 将RTP/RTCP模块解复用的RTP包送入Video_Coding中的VideoReceiver::IncomingPacket
2. 回调RTP/RTCP中的RequestKeyFrame、SliceLossIndicationRequest、ResendPackets[关键字请求、重传帧请求]
RtpReceiver
#include"webrtc/modules/rtp_rtcp/source/rtp_receiver_impl.h
一个PeerConnection拥有一个MediaControllerInterface、一个WebRtcSession、
一个WebRtcSession拥有一个VoiceChannel、一个VideoChannel、一个DataChannel
整个WEBRTC拥有一个ChannelManager 和AudioDeviceModule和MediaEngineInterface(包括一个WebRtcVideoEngine2和一个WebRtcVoiceEngine)