前言：
本篇介绍ffmpeg视频编码的基本流程，根据官方历程muxing中的历程提取的，做个笔记也让以后用到有机会ctrl+c ctrl+v。除了主要基本流程中的函数外，还用到了其它ffmpeg中的一些相关函数，我都会做简要介绍说明。
运行环境：
IDE： Qt Creator；Qt版本：Qt5.6.3 ；FFMpeg编译时间： git-2019-10-22-0b8956b。
运行效果：
一、ffmpeg视频编码基本流程
二、相关变量和函数的简要说明
结构体：
AVFormatContext：
描述了一个媒体文件或媒体流的构成和基本信息，位于avformat.h文件中。
AVOutputFormat：
输出数据的封装格式。仅封装用，调用者在avformat_write_header()之前设置。
AVCodecContext：
描述编解码器上下文的数据结构，包含了众多编解码器需要的参数信息，位于avcodec.h文件中。
AVDictionary：
简单的key/value存储，经常使用AVDictionary设置或读取内部参数，具体实现在libavutil模块中的dict.c/h，
AVCodec：
存储编解码器信息的结构体，位于avcodec.h文件中。
AVStream：
是存储每一个视频/音频流信息的结构体，位于avformat.h文件中。
AVPacket：
是FFmpeg中很重要的一个数据结构，它保存了解复用（demuxer)之后，解码（decode）之前的数据（仍然是压缩后的数据）和关于这些数据的一些附加的信息，如显示时间戳（pts），解码时间戳（dts）,数据时长（duration），所在流媒体的索引（stream_index）等等。
AVFrame：
存储一帧的数据，在解码中，AVFrame是解码器的输出；在编码中，AVFrame是编码器的输入。
SwsContext：
主要用于图片格式转换，用sws_getContext()获取，sws_freeContext()释放。
流程中的主要函数（不想看具体参数参考流程图即可）
av_register_all()：
注册FFmpeg所有协议、容器、编解码器。
void av_register_all(void);
avformat_alloc_output_context2()：
初始化输出码流的AVFormatContext。
/**
* Allocate an AVFormatContext for an output format.
* avformat_free_context() can be used to free the context and
* everything allocated by the framework within it.
*
* @param *ctx is set to the created format context, or to NULL in
* case of failure
* @param oformat format to use for allocating the context, if NULL
* format_name and filename are used instead
* @param format_name the name of output format to use for allocating the
* context, if NULL filename is used instead
* @param filename the name of the filename to use for allocating the
* context, may be NULL
* @return >= 0 in case of success, a negative AVERROR code in case of
* failure
*/
int avformat_alloc_output_context2(AVFormatContext **ctx, ff_const59 AVOutputFormat *oformat,
const char *format_name, const char *filename);
avio_open()：
打开输出文件。
/**
* Create and initialize a AVIOContext for accessing the
* resource indicated by url.
* @note When the resource indicated by url has been opened in
* read+write mode, the AVIOContext can be used only for writing.
*
* @param s Used to return the pointer to the created AVIOContext.
* In case of failure the pointed to value is set to NULL.
* @param url resource to access
* @param flags flags which control how the resource indicated by url
* is to be opened
* @return >= 0 in case of success, a negative value corresponding to an
* AVERROR code in case of failure
*/
int avio_open(AVIOContext **s, const char *url, int flags);
avcodec_find_encoder()：
找到编码器。
/**
* Find a registered encoder with a matching codec ID.
*
* @param id AVCodecID of the requested encoder
* @return An encoder if one was found, NULL otherwise.
*/
AVCodec *avcodec_find_encoder(enum AVCodecID id);
av_new_stream()：
创建输出码流的AVStream。
/**
* Add a new stream to a media file.
*
* When demuxing, it is called by the demuxer in read_header(). If the
* flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also
* be called in read_packet().
*
* When muxing, should be called by the user before avformat_write_header().
*
* User is required to call avcodec_close() and avformat_free_context() to
* clean up the allocation by avformat_new_stream().
*
* @param s media file handle
* @param c If non-NULL, the AVCodecContext corresponding to the new stream
* will be initialized to use this codec. This is needed for e.g. codec-specific
* defaults to be set, so codec should be provided if it is known.
*
* @return newly created stream or NULL on error.
*/
AVStream *avformat_new_stream(AVFormatContext *s, const AVCodec *c);
avcodec_open2()：
打开编码器。
/**
* Initialize the AVCodecContext to use the given AVCodec. Prior to using this
* function the context has to be allocated with avcodec_alloc_context3().
*
* The functions avcodec_find_decoder_by_name(), avcodec_find_encoder_by_name(),
* avcodec_find_decoder() and avcodec_find_encoder() provide an easy way for
* retrieving a codec.
*
* @warning This function is not thread safe!
*
* @note Always call this function before using decoding routines (such as
* @ref avcodec_receive_frame()).
*
* @code
* avcodec_register_all();
* av_dict_set(&opts, "b", "2.5M", 0);
* codec = avcodec_find_decoder(AV_CODEC_ID_H264);
* if (!codec)
* exit(1);
*
* context = avcodec_alloc_context3(codec);
*
* if (avcodec_open2(context, codec, opts) < 0)
* exit(1);
* @endcode
*
* @param avctx The context to initialize.
* @param codec The codec to open this context for. If a non-NULL codec has been
* previously passed to avcodec_alloc_context3() or
* for this context, then this parameter MUST be either NULL or
* equal to the previously passed codec.
* @param options A dictionary filled with AVCodecContext and codec-private options.
* On return this object will be filled with options that were not found.
*
* @return zero on success, a negative value on error
* @see avcodec_alloc_context3(), avcodec_find_decoder(), avcodec_find_encoder(),
* av_dict_set(), av_opt_find().
*/
int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **options);
avformat_write_header()：
写入流标头，也就是文件格式的文件头（对于某些没有文件头的封装格式，不需要此函数。比如说MPEG2TS）。
/**
* Allocate the stream private data and write the stream header to
* an output media file.
*
* @param s Media file handle, must be allocated with avformat_alloc_context().
* Its oformat field must be set to the desired output format;
* Its pb field must be set to an already opened AVIOContext.
* @param options An AVDictionary filled with AVFormatContext and muxer-private options.
* On return this parameter will be destroyed and replaced with a dict containing
* options that were not found. May be NULL.
*
* @return AVSTREAM_INIT_IN_WRITE_HEADER on success if the codec had not already been fully initialized in avformat_init,
* AVSTREAM_INIT_IN_INIT_OUTPUT on success if the codec had already been fully initialized in avformat_init,
* negative AVERROR on failure.
*
* @see av_opt_find, av_dict_set, avio_open, av_oformat_next, avformat_init_output.
*/
int avformat_write_header(AVFormatContext *s, AVDictionary **options);
avcodec_encode_video2()：
编码一帧视频。即将AVFrame（存储YUV像素数据）编码为AVPacket（存储视频格式的码流数据）。
/**
* Encode a frame of video.
*
* Takes input raw video data from frame and writes the next output packet, if
* available, to avpkt. The output packet does not necessarily contain data for
* the most recent frame, as encoders can delay and reorder input frames
* internally as needed.
*
* @param avctx codec context
* @param avpkt output AVPacket.
* The user can supply an output buffer by setting
* avpkt->data and avpkt->size prior to calling the
* function, but if the size of the user-provided data is not
* large enough, encoding will fail. All other AVPacket fields
* will be reset by the encoder using av_init_packet(). If
* avpkt->data is NULL, the encoder will allocate it.
* The encoder will set avpkt->size to the size of the
* output packet. The returned data (if any) belongs to the
* caller, he is responsible for freeing it.
*
* If this function fails or produces no output, avpkt will be
* freed using av_packet_unref().
* @param[in] frame AVFrame containing the raw video data to be encoded.
* May be NULL when flushing an encoder that has the
* AV_CODEC_CAP_DELAY capability set.
* @param[out] got_packet_ptr This field is set to 1 by libavcodec if the
* output packet is non-empty, and to 0 if it is
* empty. If the function returns an error, the
* packet can be assumed to be invalid, and the
* value of got_packet_ptr is undefined and should
* not be used.
* @return 0 on success, negative error code on failure
*
* @deprecated use avcodec_send_frame()/avcodec_receive_packet() instead
*/
int avcodec_encode_video2(AVCodecContext *avctx, AVPacket *avpkt,
const AVFrame *frame, int *got_packet_ptr);
av_write_frame()：
将编码后的视频码流写入文件。
/**
* Write a packet to an output media file.
*
* This function passes the packet directly to the muxer, without any buffering
* or reordering. The caller is responsible for correctly interleaving the
* packets if the format requires it. Callers that want libavformat to handle
* the interleaving should call av_interleaved_write_frame() instead of this
* function.
*
* @param s media file handle
* @param pkt The packet containing the data to be written. Note that unlike
* av_interleaved_write_frame(), this function does not take
* ownership of the packet passed to it (though some muxers may make
* an internal reference to the input packet).
* <br>
* This parameter can be NULL (at any time, not just at the end), in
* order to immediately flush data buffered within the muxer, for
* muxers that buffer up data internally before writing it to the
* output.
* <br>
* Packet's @ref AVPacket.stream_index "stream_index" field must be
* set to the index of the corresponding stream in @ref
* AVFormatContext.streams "s->streams".
* <br>
* The timestamps (@ref AVPacket.pts "pts", @ref AVPacket.dts "dts")
* must be set to correct values in the stream's timebase (unless the
* output format is flagged with the AVFMT_NOTIMESTAMPS flag, then
* they can be set to AV_NOPTS_VALUE).
* The dts for subsequent packets passed to this function must be strictly
* increasing when compared in their respective timebases (unless the
* output format is flagged with the AVFMT_TS_NONSTRICT, then they
* merely have to be nondecreasing). @ref AVPacket.duration
* "duration") should also be set if known.
* @return < 0 on error, = 0 if OK, 1 if flushed and there is no more data to flush
*
* @see av_interleaved_write_frame()
*/
int av_write_frame(AVFormatContext *s, AVPacket *pkt);
av_write_trailer()：
写文件尾（对于某些没有文件头的封装格式，不需要此函数。比如说MPEG2TS）。
/**
* Write the stream trailer to an output media file and free the
* file private data.
*
* May only be called after a successful call to avformat_write_header.
*
* @param s media file handle
* @return 0 if OK, AVERROR_xxx on error
*/
int av_write_trailer(AVFormatContext *s);
源码
void MainWindow::pngTomp4_2() //png图集转mp4
{
QString folder_path = QFileDialog::getExistingDirectory(this,"选择文件目录","",QFileDialog::ShowDirsOnly
| QFileDialog::DontResolveSymlinks);
if(folder_path.isEmpty()) return;
QString outFilename = QFileDialog::getSaveFileName(this,"选择保存路径",folder_path+"/视频文件","*.mp4;; *.avi");
if(outFilename.isEmpty()) return;
// QStringList filters;
// filters<<QString("*.png");
// 定义迭代器并设置过滤器
// QDirIterator dir_iterator(folder_path,filters, QDir::Files | QDir::NoSymLinks, QDirIterator::Subdirectories);
// QList<QString > pnglist;
// while(dir_iterator.hasNext()) //获取目录下的png文件
// {
// dir_iterator.next();
// QFileInfo file_info = dir_iterator.fileInfo();
// pnglist.append(QString(tr("%1").arg(file_info.absoluteFilePath())));
// }
AVFormatContext *kAVFormatContext;
AVOutputFormat *kAVOutputFormat;
AVCodecContext *kAVCodecContext;
AVDictionary *kAVDictionary = NULL;
AVDictionary *opt = NULL;
AVCodec *video_codec;
AVStream *kAVStream;
AVFrame *frame;
AVFrame *tmp_frame;
AVPacket kAVPacket = { 0 };
struct SwsContext *kSwsContext;
AVRational time{1,24}; /*1s25帧*/
AVRational time_1{1,1}; /*1s25帧*/
uint8_t *intBuffer = 0; // 图片数据缓冲区
int ret;
int got_packer = 0;
int64_t next_pts = 0;
tmp_frame = av_frame_alloc();
av_init_packet(&kAVPacket);
av_register_all(); //1、注册所有容易和编解码器
/* 2、分配输出媒体上下文 */
avformat_alloc_output_context2(&kAVFormatContext, NULL, NULL, outFilename.toLocal8Bit().data()); //上下文结构体、输出结构体、文件类型、路径
if(!kAVFormatContext) //如果根据文件名没有找到对应的格式则默认mpeg格式
{
ret = avformat_alloc_output_context2(&kAVFormatContext, NULL, "mpeg", outFilename.toLocal8Bit().data()); //没有找到文件类型默认mpeg（MP4）
}
if(!kAVOutputFormat) { goto end;}
kAVOutputFormat = kAVFormatContext->oformat; //把输出操作结构体赋值出来
if (!(kAVOutputFormat->flags & AVFMT_NOFILE)) /* 如果需要，打开输出文件*/
{
ret = avio_open(&kAVFormatContext->pb, outFilename.toLocal8Bit().data(), AVIO_FLAG_WRITE); //3、打开输出文件
if (ret < 0) {qDebug()<<"打开输出文件失败。"<<ret; }
}
/* 使用默认格式的编解码器添加音频和视频流，并初始化编解码器。*/
if (kAVOutputFormat->video_codec != AV_CODEC_ID_NONE)
{
video_codec = avcodec_find_encoder(kAVOutputFormat->video_codec); //4、找到编码器
if(!video_codec) goto end;
kAVStream = avformat_new_stream(kAVFormatContext,NULL); //5、新建一个输出流
if(!kAVStream) {qDebug()<<"创建流kAVStream失败。";goto end;}
kAVCodecContext = avcodec_alloc_context3(video_codec); //初始化一个AVCodecContext
if(!kAVCodecContext) {qDebug()<<"用编码器video_codec初始化的kAVCodecContext默认参数失败";goto end;}
switch(video_codec->type)
{
case AVMEDIA_TYPE_VIDEO:
kAVCodecContext->codec_id = video_codec->id;
kAVCodecContext->bit_rate = 800000;
kAVCodecContext->width = 720;
kAVCodecContext->height = 404;
kAVStream->time_base = time;
kAVCodecContext->time_base = time;
kAVCodecContext->gop_size = 12; /*最多每十二帧发射一个内帧 */
kAVCodecContext->pix_fmt = AV_PIX_FMT_YUV420P;
if (kAVCodecContext->codec_id == AV_CODEC_ID_MPEG2VIDEO)
{
kAVCodecContext->max_b_frames = 2; /*为了测试，我们还添加了b帧*/
}
if (kAVCodecContext->codec_id == AV_CODEC_ID_MPEG1VIDEO)
{
/* 需要避免使用一些coeffs溢出的宏块。这在正常的视频中不会发生，
* 只是在色度平面的运动与luma平面不匹配时才会发生。 */
kAVCodecContext->mb_decision = 2;
}
break;
case AVMEDIA_TYPE_AUDIO:
break;
default:
break;
}
if(kAVOutputFormat->flags & AVFMT_GLOBALHEADER)/*有些格式希望流标头是单独的*/
{
kAVCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}
}
av_dict_copy(&opt,kAVDictionary, 0);
ret = avcodec_open2(kAVCodecContext,video_codec,&opt); //6、打开编码器
if(ret<0){qDebug()<<"打开视频编解码器失败"<<ret; goto end;}
av_dict_free(&opt);
frame = av_frame_alloc();
if(!frame) {qDebug()<<"分配帧失败."; goto end;}
frame->format = kAVCodecContext->pix_fmt;
frame->width = kAVCodecContext->width;
frame->height = kAVCodecContext->height;
ret = av_frame_get_buffer(frame, 32); //分配内存空间frame必须要有上三条件，32也就是4字节对齐
if(ret < 0){qDebug()<<"frame分配内存失败"; goto end;}
ret = avcodec_parameters_from_context(kAVStream->codecpar,kAVCodecContext);/*将流参数复制到muxer */
if (ret < 0) {qDebug()<<"无法复制流参数";goto end; }
av_dump_format(kAVFormatContext, 0, outFilename.toLocal8Bit().data(), 1); //打印输出文件的详细信息
ret = avformat_write_header(kAVFormatContext,&kAVDictionary);/* 7、写流标头(如果有的话)*/
if(ret < 0){qDebug()<<"写流标题失败"; goto end;}
//8、写入每一帧数据
for(int i = 0; i<527; i++)
{
// if(av_compare_ts(next_pts, kAVCodecContext->time_base, 10.0, time_1) > 0) /*这里只生成10s的视频*/
// {
// qDebug()<<"暂时不需要生成新的帧"; break;
// }
if (av_frame_make_writable(frame) < 0) {goto end;} /*当我们传递一个帧给编码器时，它可能会在内部保留一个对它的引用;确保我们没有在这里覆盖它*/
QImage img(tr("%1/%2.png").arg(folder_path).arg(i));
if(img.isNull()){qDebug()<<"打开图片失败";break;}
img = img.convertToFormat(QImage::Format_RGB888);
kSwsContext = sws_getContext(kAVCodecContext->width,kAVCodecContext->height,
AV_PIX_FMT_RGB24,kAVCodecContext->width,kAVCodecContext->height,
AV_PIX_FMT_YUV420P,SWS_LANCZOS | SWS_ACCURATE_RND,NULL,NULL,NULL);
if(!kSwsContext){qDebug()<<"无法初始化图片转换器"; goto end;}
intBuffer = (uint8_t*)malloc(sizeof(uint8_t)*img.bytesPerLine()*img.height());
memcpy(intBuffer,img.constBits(),sizeof(uint8_t)*img.bytesPerLine()*img.height());
avpicture_fill((AVPicture *)tmp_frame,intBuffer,AV_PIX_FMT_RGB24,kAVCodecContext->width,kAVCodecContext->height); //pAVFrame32的data指针指向了outBuffer
sws_scale(kSwsContext,(const uint8_t * const *)tmp_frame->data,
tmp_frame->linesize,0,kAVCodecContext->height,frame->data,frame->linesize); //图片信息转换
sws_freeContext(kSwsContext);
kSwsContext = 0;
frame->pts = next_pts++;
//frame->pts = i*(kAVStream->time_base.den)/((kAVStream->time_base.num)*24);
got_packer = 0;
ret = avcodec_encode_video2(kAVCodecContext,&kAVPacket,frame,&got_packer); //8、编码、把一帧数据编码成AVPacket
if(ret < 0) {qDebug()<<"错误编码视频帧"<<ret; goto end;}
if(got_packer)
{
av_packet_rescale_ts(&kAVPacket, kAVCodecContext->time_base, kAVStream->time_base);
kAVPacket.stream_index = kAVStream->index;
ret = av_write_frame(kAVFormatContext, &kAVPacket); /*将压缩帧写入媒体文件。 */
av_free_packet(&kAVPacket);
}else
{ret = 0;}
free(intBuffer);intBuffer = 0;
if(ret<0){ qDebug()<<"写入video文件失败"<<ret<<kAVPacket.stream_index; break; }
}
/* 选择写预告片，如果有的话。预告片必须在你之前写好 当你写标题时关闭CodecContexts打开;否则
av_write_trailer()可以尝试使用被释放的内存 av_codec_close()。要编码的流 */
av_write_trailer(kAVFormatContext);
end:
/* 关闭每个编解码器。 */
avcodec_free_context(&kAVCodecContext);
av_frame_free(&frame);
if (!(kAVOutputFormat->flags & AVFMT_NOFILE))
avio_closep(&kAVFormatContext->pb);/*关闭输出文件*/
avformat_free_context(kAVFormatContext); /*释放上下文*/
}
6423
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
