推荐一个零声学院免费公开课程，个人觉得老师讲得不错，分享给大家：Linux，Nginx，ZeroMQ，MySQL，Redis，fastdfs，MongoDB，ZK，流媒体，CDN，P2P，K8S，Docker，TCP/IP，协程，DPDK等技术内容，立即学习
一、播放器框架
- AVUtil：核心工具库，下面的许多其他模块都会依赖该库做一些基本的音视频处理操作。
- AVFormat：文件格式和协议库，该模块是最重要的模块之一，封装了Protocol层和Demuxer、Muxer层，使得协议和格式对于开发者来说是透明的。
- AVCodec：编解码库，封装了Codec层，但是有一些Codec是具备自己的License的，FFmpeg是不会默认添加像libx264、FDK-AAC等库的，但是FFmpeg就像一个平台一样，可以将其他的第三方的Codec以插件的方式添加进来，然后为开发者提供统一的接口。
- AVFilter：音视频滤镜库，该模块提供了包括音频特效和视频特效的处理，在使用FFmpeg的API进行编解码的过程中，直接使用该模块为音视频数据做特效处理是非常方便同时也非常高效的一种方式。
- AVDevice：输入输出设备库，比如，需要编译出播放声音或者视频的工具ffplay，就需要确保该模块是打开的，同时也需要SDL的预先编译，因为该设备模块播放声音与播放视频使用的都是SDL库。
- SwrRessample：该模块可用于音频重采样，可以对数字音频进行声道数、数据格式、采样率等多种基本信息的转换。
- SWScale：该模块是将图像进行格式转换的模块，比如，可以将YUV的数据转换为RGB的数据，缩放尺寸由1280_720变为800_480。
- PostProc：该模块可用于进行后期处理，当我们使用AVFilter的时候需要打开该模块的开关，因为Filter中会使用到该模块的一些基础函数。
二、ffmpeg函数
2.1 初始化函数
- av_register_all(): 注册所有组件4.0已经弃用
- avdevice_register_all(): 对设备进行注册，比如V4L2等。
- avformat_network_init(): 初始化网络卡以及网络加密协议相关的库（比如openssl）
2.2 封装相关函数
- avformat_alloc_context(): 负责申请一个AVFormatContext结构的内存，并进行简单初始化。
- avformat_free_context(): 释放该结构里的所有东西以及该结构本身。
- avformat_close_input(): 关闭解复用器。关闭后就不再需要使用avformat_free_context进行释放。
- avformat_open_input(): 打开输入视频文件。
- avformat_find_stream_info(): 获取音视频文件信息。
- av_read_frame(): 读取音视频包。
- avformat_seek_file(): 定位文件。
- av_seek_frame(): 定位文件。
2.3 封装步骤
- 分配解复用器上下文：avformat_alloc_context。
- 根据url打开本地文件或网络流: avformat_open_input。
- 读取媒体的部分数据包以获取码流信息: avformat_find_stream_info。
- 从文件中读取数据包: av_read_frame。或 定位文件: avformat_seek_file、av_seek_frame。
- 关闭解复用器: avformat_close_format。
2.4 解码器相关函数
- avcodec_alloc_context3(): 分配解码器上下文
- avcodec_find_decoder()：根据ID查找解码器
- avcodec_find_decoder_by_name():根据解码器名字
- avcodec_open2()： 打开编解码器
avcodec_decode_video2()：解码一帧视频数据avcodec_decode_audio4()：解码一帧音频数据- avcodec_send_packet(): 发送编码数据包
- avcodec_receive_frame(): 接收解码后数据
- avcodec_free_context():释放解码器上下文，包含了avcodec_close()
- avcodec_close():关闭解码器
2.5 解码器步骤
- 分配编解码上下文: avcodec_alloc_context3
- 将码流中的编解码信息拷贝到AVCodecContex: avcode_parameter_to_context
- 根据编解码器信息查找相应的编码器: avcodec_find_decoder或指定解码器avcodec_find_decoder_by_name
- 打开编解码器并关联到AVCodecContex: avcodec_open2
- 向解码器发送数据包:avcodec_send_packet -> 接受解码后的帧: avcodec_receive_frame
- 关闭解码器和释放上下文avcodec_close: avcodec_free_context
2.6 3.x组件注册
我们使用ffmpeg，首先要执行av_register_all，把全局的解码器、编码器等结构体注册到各自全局的对象链表里，以便后面查找调用。
2.7 4.x 组件注册
FFmpeg内部去做，不需要用户调用API去注册。
以codec编解码器为例：
- 在configure的时候生成要注册的组件
./configure:7203:print_enabled_components libavcodec/codec_list.c
AVCodec codec_list $CODEC_LIST
这里会生成一个codec_list.c 文件，里面只有static const AVCodec *
const codec_list[]数组。 - 在libavcodec/allcodecs.c将static const AVCodec * const codec_list[]
的编解码器用链表的方式组织起来。
2.8 Ffmpeg 4.0.2 组件注册方式
FFmepg内部去做，不需要用户调用API去注册。
对于demuxer/muxer（解复用器，也称容器）则对应
- libavformat/muxer_list.c
libavformat/demuxer_list.c 这两个文件也是在configure的时候生成，
也就是说直接下载源码是没有这两个文件的。 - 在libavformat/allformats.c将demuxer_list[]和muexr_list[]以链表的方
式组织。
其他组件也是类似的方式。
三、 ffmpeg数据结构
3.1 AVFormatContext
封装格式上下文结构体，也是统领全局的结构体，保存了视频文件封装格式相关信息。该结构体定义位于avformat.h，其详细结构如下
//avformat.h
/**
* Format I/O context.
* New fields can be added to the end with minor version bumps.
* Removal, reordering and changes to existing fields require a major
* version bump.
* sizeof(AVFormatContext) must not be used outside libav*, use
* avformat_alloc_context() to create an AVFormatContext.
*
* Fields can be accessed through AVOptions (av_opt*),
* the name string used matches the associated command line parameter name and
* can be found in libavformat/options_table.h.
* The AVOption/command line parameter names differ in some cases from the C
* structure field names for historic reasons or brevity.
*/
typedef struct AVFormatContext {
/**
* A class for logging and @ref avoptions. Set by avformat_alloc_context().
* Exports (de)muxer private options if they exist.
*/
const AVClass *av_class;
/**
* The input container format.
*
* Demuxing only, set by avformat_open_input().
*/
// 输入媒体的AVInputFormat,比如指向AVInputFormatff_flv_demuxer
ff_const59 struct AVInputFormat *iformat;
/**
* The output container format.
*
* Muxing only, must be set by the caller before avformat_write_header().
*/
ff_const59 struct AVOutputFormat *oformat;
/**
* Format private data. This is an AVOptions-enabled struct
* if and only if iformat/oformat.priv_class is not NULL.
*
* - muxing: set by avformat_write_header()
* - demuxing: set by avformat_open_input()
*/
void *priv_data;
/**
* I/O context.
*
* - demuxing: either set by the user before avformat_open_input() (then
* the user must close it manually) or set by avformat_open_input().
* - muxing: set by the user before avformat_write_header(). The caller must
* take care of closing / freeing the IO context.
*
* Do NOT set this field if AVFMT_NOFILE flag is set in
* iformat/oformat.flags. In such a case, the (de)muxer will handle
* I/O in some other way and this field will be NULL.
*/
AVIOContext *pb;
/* stream info */
/**
* Flags signalling stream properties. A combination of AVFMTCTX_*.
* Set by libavformat.
*/
int ctx_flags;
/**
* Number of elements in AVFormatContext.streams.
*
* Set by avformat_new_stream(), must not be modified by any other code.
*/
// 输入媒体的AVStream个数
unsigned int nb_streams;
/**
* A list of all streams in the file. New streams are created with
* avformat_new_stream().
*
* - demuxing: streams are created by libavformat in avformat_open_input().
* If AVFMTCTX_NOHEADER is set in ctx_flags, then new streams may also
* appear in av_read_frame().
* - muxing: streams are created by the user before avformat_write_header().
*
* Freed by libavformat in avformat_free_context().
*/
// 输入媒体的AVStream数组
AVStream **streams;
#if FF_API_FORMAT_FILENAME
/**
* input or output filename
*
* - demuxing: set by avformat_open_input()
* - muxing: may be set by the caller before avformat_write_header()
*
* @deprecated Use url instead.
*/
attribute_deprecated
char filename[1024];
#endif
/**
* input or output URL. Unlike the old filename field, this field has no
* length restriction.
*
* - demuxing: set by avformat_open_input(), initialized to an empty
* string if url parameter was NULL in avformat_open_input().
* - muxing: may be set by the caller before calling avformat_write_header()
* (or avformat_init_output() if that is called first) to a string
* which is freeable by av_free(). Set to an empty string if it
* was NULL in avformat_init_output().
*
* Freed by libavformat in avformat_free_context().
*/
char *url;
/**
* Position of the first frame of the component, in
* AV_TIME_BASE fractional seconds. NEVER set this value directly:
* It is deduced from the AVStream values.
*
* Demuxing only, set by libavformat.
*/
int64_t start_time;
/**
* Duration of the stream, in AV_TIME_BASE fractional
* seconds. Only set this value if you know none of the individual stream
* durations and also do not set any of them. This is deduced from the
* AVStream values if not set.
*
* Demuxing only, set by libavformat.
*/
// 输入媒体的时长(以微妙为单位)，计算方式参考av_dump_format函数
int64_t duration;
/**
* Total stream bitrate in bit/s, 0 if not
* available. Never set it directly if the file_size and the
* duration are known as FFmpeg can compute it automatically.
*/
// 输入媒体的码率
int64_t bit_rate;
unsigned int packet_size;
int max_delay;
/**
* Flags modifying the (de)muxer behaviour. A combination of AVFMT_FLAG_*.
* Set by the user before avformat_open_input() / avformat_write_header().
*/
int flags;
#define AVFMT_FLAG_GENPTS 0x0001 ///< Generate missing pts even if it requires parsing future frames.
#define AVFMT_FLAG_IGNIDX 0x0002 ///< Ignore index.
#define AVFMT_FLAG_NONBLOCK 0x0004 ///< Do not block when reading packets from input.
#define AVFMT_FLAG_IGNDTS 0x0008 ///< Ignore DTS on frames that contain both DTS & PTS
#define AVFMT_FLAG_NOFILLIN 0x0010 ///< Do not infer any values from other values, just return what is stored in the container
#define AVFMT_FLAG_NOPARSE 0x0020 ///< Do not use AVParsers, you also must set AVFMT_FLAG_NOFILLIN as the fillin code works on frames and no parsing -> no frames. Also seeking to frames can not work if parsing to find frame boundaries has been disabled
#define AVFMT_FLAG_NOBUFFER 0x0040 ///< Do not buffer frames when possible
#define AVFMT_FLAG_CUSTOM_IO 0x0080 ///< The caller has supplied a custom AVIOContext, don't avio_close() it.
#define AVFMT_FLAG_DISCARD_CORRUPT 0x0100 ///< Discard frames marked corrupted
#define AVFMT_FLAG_FLUSH_PACKETS 0x0200 ///< Flush the AVIOContext every packet.
/**
* When muxing, try to avoid writing any random/volatile data to the output.
* This includes any random IDs, real-time timestamps/dates, muxer version, etc.
*
* This flag is mainly intended for testing.
*/
#define AVFMT_FLAG_BITEXACT 0x0400
#if FF_API_LAVF_MP4A_LATM
#define AVFMT_FLAG_MP4A_LATM 0x8000 ///< Deprecated, does nothing.
#endif
#define AVFMT_FLAG_SORT_DTS 0x10000 ///< try to interleave outputted packets by dts (using this flag can slow demuxing down)
#if FF_API_LAVF_PRIV_OPT
#define AVFMT_FLAG_PRIV_OPT 0x20000 ///< Enable use of private options by delaying codec open (deprecated, will do nothing once av_demuxer_open() is removed)
#endif
#if FF_API_LAVF_KEEPSIDE_FLAG
#define AVFMT_FLAG_KEEP_SIDE_DATA 0x40000 ///< Deprecated, does nothing.
#endif
#define AVFMT_FLAG_FAST_SEEK 0x80000 ///< Enable fast, but inaccurate seeks for some formats
#define AVFMT_FLAG_SHORTEST 0x100000 ///< Stop muxing when the shortest stream stops.
#define AVFMT_FLAG_AUTO_BSF 0x200000 ///< Add bitstream filters as requested by the muxer
/**
* Maximum size of the data read from input for determining
* the input container format.
* Demuxing only, set by the caller before avformat_open_input().
*/
int64_t probesize;
/**
* Maximum duration (in AV_TIME_BASE units) of the data read
* from input in avformat_find_stream_info().
* Demuxing only, set by the caller before avformat_find_stream_info().
* Can be set to 0 to let avformat choose using a heuristic.
*/
int64_t max_analyze_duration;
const uint8_t *key;
int keylen;
unsigned int nb_programs;
AVProgram **programs;
/**
* Forced video codec_id.
* Demuxing: Set by user.
*/
enum AVCodecID video_codec_id;
/**
* Forced audio codec_id.
* Demuxing: Set by user.
*/
enum AVCodecID audio_codec_id;
/**
* Forced subtitle codec_id.
* Demuxing: Set by user.
*/
enum AVCodecID subtitle_codec_id;
/**
* Maximum amount of memory in bytes to use for the index of each stream.
* If the index exceeds this size, entries will be discarded as
* needed to maintain a smaller size. This can lead to slower or less
* accurate seeking (depends on demuxer).
* Demuxers for which a full in-memory index is mandatory will ignore
* this.
* - muxing: unused
* - demuxing: set by user
*/
unsigned int max_index_size;
/**
* Maximum amount of memory in bytes to use for buffering frames
* obtained from realtime capture devices.
*/
unsigned int max_picture_buffer;
/**
* Number of chapters in AVChapter array.
* When muxing, chapters are normally written in the file header,
* so nb_chapters should normally be initialized before write_header
* is called. Some muxers (e.g. mov and mkv) can also write chapters
* in the trailer. To write chapters in the trailer, nb_chapters
* must be zero when write_header is called and non-zero when
* write_trailer is called.
* - muxing: set by user
* - demuxing: set by libavformat
*/
unsigned int nb_chapters;
AVChapter **chapters;
/**
* Metadata that applies to the whole file.
*
* - demuxing: set by libavformat in avformat_open_input()
* - muxing: may be set by the caller before avformat_write_header()
*
* Freed by libavformat in avformat_free_context().
*/
AVDictionary *metadata;
/**
* Start time of the stream in real world time, in microseconds
* since the Unix epoch (00:00 1st January 1970). That is, pts=0 in the
* stream was captured at this real world time.
* - muxing: Set by the caller before avformat_write_header(). If set to
* either 0 or AV_NOPTS_VALUE, then the current wall-time will
* be used.
* - demuxing: Set by libavformat. AV_NOPTS_VALUE if unknown. Note that
* the value may become known after some number of frames
* have been received.
*/
int64_t start_time_realtime;
/**
* The number of frames used for determining the framerate in
* avformat_find_stream_info().
* Demuxing only, set by the caller before avformat_find_stream_info().
*/
int fps_probe_size;
/**
* Error recognition; higher values will detect more errors but may
* misdetect some more or less valid parts as errors.
* Demuxing only, set by the caller before avformat_open_input().
*/
int error_recognition;
/**
* Custom interrupt callbacks for the I/O layer.
*
* demuxing: set by the user before avformat_open_input().
* muxing: set by the user before avformat_write_header()
* (mainly useful for AVFMT_NOFILE formats). The callback
* should also be passed to avio_open2() if it's used to
* open the file.
*/
AVIOInterruptCB interrupt_callback;
/**
* Flags to enable debugging.
*/
int debug;
#define FF_FDEBUG_TS 0x0001
/**
* Maximum buffering duration for interleaving.
*
* To ensure all the streams are interleaved correctly,
* av_interleaved_write_frame() will wait until it has at least one packet
* for each stream before actually writing any packets to the output file.
* When some streams are "sparse" (i.e. there are large gaps between
* successive packets), this can result in excessive buffering.
*
* This field specifies the maximum difference between the timestamps of the
* first and the last packet in the muxing queue, above which libavformat
* will output a packet regardless of whether it has queued a packet for all
* the streams.
*
* Muxing only, set by the caller before avformat_write_header().
*/
int64_t max_interleave_delta;
/**
* Allow non-standard and experimental extension
* @see AVCodecContext.strict_std_compliance
*/
int strict_std_compliance;
/**
* Flags indicating events happening on the file, a combination of
* AVFMT_EVENT_FLAG_*.
*
* - demuxing: may be set by the demuxer in avformat_open_input(),
* avformat_find_stream_info() and av_read_frame(). Flags must be cleared
* by the user once the event has been handled.
* - muxing: may be set by the user after avformat_write_header() to
* indicate a user-triggered event. The muxer will clear the flags for
* events it has handled in av_[interleaved]_write_frame().
*/
int event_flags;
/**
* - demuxing: the demuxer read new metadata from the file and updated
* AVFormatContext.metadata accordingly
* - muxing: the user updated AVFormatContext.metadata and wishes the muxer to
* write it into the file
*/
#define AVFMT_EVENT_FLAG_METADATA_UPDATED 0x0001
/**
* Maximum number of packets to read while waiting for the first timestamp.
* Decoding only.
*/
int max_ts_probe;
/**
* Avoid negative timestamps during muxing.
* Any value of the AVFMT_AVOID_NEG_TS_* constants.
* Note, this only works when using av_interleaved_write_frame. (interleave_packet_per_dts is in use)
* - muxing: Set by user
* - demuxing: unused
*/
int avoid_negative_ts;
#define AVFMT_AVOID_NEG_TS_AUTO -1 ///< Enabled when required by target format
#define AVFMT_AVOID_NEG_TS_MAKE_NON_NEGATIVE 1 ///< Shift timestamps so they are non negative
#define AVFMT_AVOID_NEG_TS_MAKE_ZERO 2 ///< Shift timestamps so that they start at 0
/**
* Transport stream id.
* This will be moved into demuxer private options. Thus no API/ABI compatibility
*/
int ts_id;
/**
* Audio preload in microseconds.
* Note, not all formats support this and unpredictable things may happen if it is used when not supported.
* - encoding: Set by user
* - decoding: unused
*/
int audio_preload;
/**
* Max chunk time in microseconds.
* Note, not all formats support this and unpredictable things may happen if it is used when not supported.
* - encoding: Set by user
* - decoding: unused
*/
int max_chunk_duration;
/**
* Max chunk size in bytes
* Note, not all formats support this and unpredictable things may happen if it is used when not supported.
* - encoding: Set by user
* - decoding: unused
*/
int max_chunk_size;
/**
* forces the use of wallclock timestamps as pts/dts of packets
* This has undefined results in the presence of B frames.
* - encoding: unused
* - decoding: Set by user
*/
int use_wallclock_as_timestamps;
/**
* avio flags, used to force AVIO_FLAG_DIRECT.
* - encoding: unused
* - decoding: Set by user
*/
int avio_flags;
/**
* The duration field can be estimated through various ways, and this field can be used
* to know how the duration was estimated.
* - encoding: unused
* - decoding: Read by user
*/
enum AVDurationEstimationMethod duration_estimation_method;
/**
* Skip initial bytes when opening stream
* - encoding: unused
* - decoding: Set by user
*/
int64_t skip_initial_bytes;
/**
* Correct single timestamp overflows
* - encoding: unused
* - decoding: Set by user
*/
unsigned int correct_ts_overflow;
/**
* Force seeking to any (also non key) frames.
* - encoding: unused
* - decoding: Set by user
*/
int seek2any;
/**
* Flush the I/O context after each packet.
* - encoding: Set by user
* - decoding: unused
*/
int flush_packets;
/**
* format probing score.
* The maximal score is AVPROBE_SCORE_MAX, its set when the demuxer probes
* the format.
* - encoding: unused
* - decoding: set by avformat, read by user
*/
int probe_score;
/**
* number of bytes to read maximally to identify format.
* - encoding: unused
* - decoding: set by user
*/
int format_probesize;
/**
* ',' separated list of allowed decoders.
* If NULL then all are allowed
* - encoding: unused
* - decoding: set by user
*/
char *codec_whitelist;
/**
* ',' separated list of allowed demuxers.
* If NULL then all are allowed
* - encoding: unused
* - decoding: set by user
*/
char *format_whitelist;
/**
* An opaque field for libavformat internal usage.
* Must not be accessed in any way by callers.
*/
AVFormatInternal *internal;
/**
* IO repositioned flag.
* This is set by avformat when the underlaying IO context read pointer
* is repositioned, for example when doing byte based seeking.
* Demuxers can use the flag to detect such changes.
*/
int io_repositioned;
/**
* Forced video codec.
* This allows forcing a specific decoder, even when there are multiple with
* the same codec_id.
* Demuxing: Set by user
*/
AVCodec *video_codec;
/**
* Forced audio codec.
* This allows forcing a specific decoder, even when there are multiple with
* the same codec_id.
* Demuxing: Set by user
*/
AVCodec *audio_codec;
/**
* Forced subtitle codec.
* This allows forcing a specific decoder, even when there are multiple with
* the same codec_id.
* Demuxing: Set by user
*/
AVCodec *subtitle_codec;
/**
* Forced data codec.
* This allows forcing a specific decoder, even when there are multiple with
* the same codec_id.
* Demuxing: Set by user
*/
AVCodec *data_codec;
/**
* Number of bytes to be written as padding in a metadata header.
* Demuxing: Unused.
* Muxing: Set by user via av_format_set_metadata_header_padding.
*/
int metadata_header_padding;
/**
* User data.
* This is a place for some private data of the user.
*/
void *opaque;
/**
* Callback used by devices to communicate with application.
*/
av_format_control_message control_message_cb;
/**
* Output timestamp offset, in microseconds.
* Muxing: set by user
*/
int64_t output_ts_offset;
/**
* dump format separator.
* can be ", " or "\n " or anything else
* - muxing: Set by user.
* - demuxing: Set by user.
*/
uint8_t *dump_separator;
/**
* Forced Data codec_id.
* Demuxing: Set by user.
*/
enum AVCodecID data_codec_id;
#if FF_API_OLD_OPEN_CALLBACKS
/**
* Called to open further IO contexts when needed for demuxing.
*
* This can be set by the user application to perform security checks on
* the URLs before opening them.
* The function should behave like avio_open2(), AVFormatContext is provided
* as contextual information and to reach AVFormatContext.opaque.
*
* If NULL then some simple checks are used together with avio_open2().
*
* Must not be accessed directly from outside avformat.
* @See av_format_set_open_cb()
*
* Demuxing: Set by user.
*
* @deprecated Use io_open and io_close.
*/
attribute_deprecated
int (*open_cb)(struct AVFormatContext *s, AVIOContext **p, const char *url, int flags, const AVIOInterruptCB *int_cb, AVDictionary **options);
#endif
/**
* ',' separated list of allowed protocols.
* - encoding: unused
* - decoding: set by user
*/
char *protocol_whitelist;
/**
* A callback for opening new IO streams.
*
* Whenever a muxer or a demuxer needs to open an IO stream (typically from
* avformat_open_input() for demuxers, but for certain formats can happen at
* other times as well), it will call this callback to obtain an IO context.
*
* @param s the format context
* @param pb on success, the newly opened IO context should be returned here
* @param url the url to open
* @param flags a combination of AVIO_FLAG_*
* @param options a dictionary of additional options, with the same
* semantics as in avio_open2()
* @return 0 on success, a negative AVERROR code on failure
*
* @note Certain muxers and demuxers do nesting, i.e. they open one or more
* additional internal format contexts. Thus the AVFormatContext pointer
* passed to this callback may be different from the one facing the caller.
* It will, however, have the same 'opaque' field.
*/
int (*io_open)(struct AVFormatContext *s, AVIOContext **pb, const char *url,
int flags, AVDictionary **options);
/**
* A callback for closing the streams opened with AVFormatContext.io_open().
*/
void (*io_close)(struct AVFormatContext *s, AVIOContext *pb);
/**
* ',' separated list of disallowed protocols.
* - encoding: unused
* - decoding: set by user
*/
char *protocol_blacklist;
/**
* The maximum number of streams.
* - encoding: unused
* - decoding: set by user
*/
int max_streams;
/**
* Skip duration calcuation in estimate_timings_from_pts.
* - encoding: unused
* - decoding: set by user
*/
int skip_estimate_duration_from_pts;
/**
* Maximum number of packets that can be probed
* - encoding: unused
* - decoding: set by user
*/
int max_probe_packets;
} AVFormatContext;
3.2 AVInputFormat demuxer和AVoutputFormat muxer
每种封装格式(如FLV、MKV、MP4、AVI)都有对应的结构体。该结构体定义位于avformat.h，其详细结构如下
/**
* @addtogroup lavf_decoding
* @{
*/
typedef struct AVInputFormat {
/**
* A comma separated list of short names for the format. New names
* may be appended with a minor bump.
*/
// 封装格式名称
const char *name;
/**
* Descriptive name for the format, meant to be more human-readable
* than name. You should use the NULL_IF_CONFIG_SMALL() macro
* to define it.
*/
const char *long_name;
/**
* Can use flags: AVFMT_NOFILE, AVFMT_NEEDNUMBER, AVFMT_SHOW_IDS,
* AVFMT_NOTIMESTAMPS, AVFMT_GENERIC_INDEX, AVFMT_TS_DISCONT, AVFMT_NOBINSEARCH,
* AVFMT_NOGENSEARCH, AVFMT_NO_BYTE_SEEK, AVFMT_SEEK_TO_PTS.
*/
int flags;
/**
* If extensions are defined, then no probe is done. You should
* usually not use extension format guessing because it is not
* reliable enough
*/
// 封装格式扩展名
const char *extensions;
const struct AVCodecTag * const *codec_tag;
const AVClass *priv_class; ///< AVClass for the private context
/**
* Comma-separated list of mime types.
* It is used check for matching mime types while probing.
* @see av_probe_input_format2
*/
const char *mime_type;
/*****************************************************************
* No fields below this line are part of the public API. They
* may not be used outside of libavformat and can be changed and
* removed at will.
* New public fields should be added right above.
*****************************************************************
*/
#if FF_API_NEXT
ff_const59 struct AVInputFormat *next;
#endif
/**
* Raw demuxers store their codec ID here.
*/
int raw_codec_id;
/**
* Size of private data so that it can be allocated in the wrapper.
*/
int priv_data_size;
/**
* Tell if a given file has a chance of being parsed as this format.
* The buffer provided is guaranteed to be AVPROBE_PADDING_SIZE bytes
* big so you do not have to check for that unless you need more.
*/
int (*read_probe)(const AVProbeData *);
/**
* Read the format header and initialize the AVFormatContext
* structure. Return 0 if OK. 'avformat_new_stream' should be
* called to create new streams.
*/
int (*read_header)(struct AVFormatContext *);
/**
* Read one packet and put it in 'pkt'. pts and flags are also
* set. 'avformat_new_stream' can be called only if the flag
* AVFMTCTX_NOHEADER is used and only in the calling thread (not in a
* background thread).
* @return 0 on success, < 0 on error.
* Upon returning an error, pkt must be unreferenced by the caller.
*/
int (*read_packet)(struct AVFormatContext *, AVPacket *pkt);
/**
* Close the stream. The AVFormatContext and AVStreams are not
* freed by this function
*/
int (*read_close)(struct AVFormatContext *);
/**
* Seek to a given timestamp relative to the frames in
* stream component stream_index.
* @param stream_index Must not be -1.
* @param flags Selects which direction should be preferred if no exact
* match is available.
* @return >= 0 on success (but not necessarily the new offset)
*/
int (*read_seek)(struct AVFormatContext *,
int stream_index, int64_t timestamp, int flags);
/**
* Get the next timestamp in stream[stream_index].time_base units.
* @return the timestamp or AV_NOPTS_VALUE if an error occurred
*/
int64_t (*read_timestamp)(struct AVFormatContext *s, int stream_index,
int64_t *pos, int64_t pos_limit);
/**
* Start/resume playing - only meaningful if using a network-based format
* (RTSP).
*/
int (*read_play)(struct AVFormatContext *);
/**
* Pause playing - only meaningful if using a network-based format
* (RTSP).
*/
int (*read_pause)(struct AVFormatContext *);
/**
* Seek to timestamp ts.
* Seeking will be done so that the point from which all active streams
* can be presented successfully will be closest to ts and within min/max_ts.
* Active streams are all streams that have AVStream.discard < AVDISCARD_ALL.
*/
int (*read_seek2)(struct AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags);
/**
* Returns device list with it properties.
* @see avdevice_list_devices() for more details.
*/
int (*get_device_list)(struct AVFormatContext *s, struct AVDeviceInfoList *device_list);
#if LIBAVFORMAT_VERSION_MAJOR < 59
/**
* Initialize device capabilities submodule.
* @see avdevice_capabilities_create() for more details.
*/
int (*create_device_capabilities)(struct AVFormatContext *s, struct AVDeviceCapabilitiesQuery *caps);
/**
* Free device capabilities submodule.
* @see avdevice_capabilities_free() for more details.
*/
int (*free_device_capabilities)(struct AVFormatContext *s, struct AVDeviceCapabilitiesQuery *caps);
#endif
} AVInputFormat;
/**
3.3 AVStream
视频文件中每个视频(音频)流对应一个结构体。该结构体定义位于avformat.h，其详细结构如下
/**
* Stream structure.
* New fields can be added to the end with minor version bumps.
* Removal, reordering and changes to existing fields require a major
* version bump.
* sizeof(AVStream) must not be used outside libav*.
*/
typedef struct AVStream {
// 标识该视频/音频流的序号
int index; /**< stream index in AVFormatContext */
/**
* Format-specific stream ID.
* decoding: set by libavformat
* encoding: set by the user, replaced by libavformat if left unset
*/
int id;
#if FF_API_LAVF_AVCTX
/**
* @deprecated use the codecpar struct instead
*/
attribute_deprecated
AVCodecContext *codec;
#endif
void *priv_data;
/**
* This is the fundamental unit of time (in seconds) in terms
* of which frame timestamps are represented.
*
* decoding: set by libavformat
* encoding: May be set by the caller before avformat_write_header() to
* provide a hint to the muxer about the desired timebase. In
* avformat_write_header(), the muxer will overwrite this field
* with the timebase that will actually be used for the timestamps
* written into the file (which may or may not be related to the
* user-provided one, depending on the format).
*/
// 该流的时基，PTS*time_base=真正的时间(秒)
AVRational time_base;
/**
* Decoding: pts of the first frame of the stream in presentation order, in stream time base.
* Only set this if you are absolutely 100% sure that the value you set
* it to really is the pts of the first frame.
* This may be undefined (AV_NOPTS_VALUE).
* @note The ASF header does NOT contain a correct start_time the ASF
* demuxer must NOT set this.
*/
int64_t start_time;
/**
* Decoding: duration of the stream, in stream time base.
* If a source file does not specify a duration, but does specify
* a bitrate, this value will be estimated from bitrate and file size.
*
* Encoding: May be set by the caller before avformat_write_header() to
* provide a hint to the muxer about the estimated duration.
*/
//该视频/音频流的长度
int64_t duration;
int64_t nb_frames; ///< number of frames in this stream if known or 0
int disposition; /**< AV_DISPOSITION_* bit field */
enum AVDiscard discard; ///< Selects which packets can be discarded at will and do not need to be demuxed.
/**
* sample aspect ratio (0 if unknown)
* - encoding: Set by user.
* - decoding: Set by libavformat.
*/
AVRational sample_aspect_ratio;
AVDictionary *metadata;
/**
* Average framerate
*
* - demuxing: May be set by libavformat when creating the stream or in
* avformat_find_stream_info().
* - muxing: May be set by the caller before avformat_write_header().
*/
// 该流的帧率
AVRational avg_frame_rate;
/**
* For streams with AV_DISPOSITION_ATTACHED_PIC disposition, this packet
* will contain the attached picture.
*
* decoding: set by libavformat, must not be modified by the caller.
* encoding: unused
*/
AVPacket attached_pic;
/**
* An array of side data that applies to the whole stream (i.e. the
* container does not allow it to change between packets).
*
* There may be no overlap between the side data in this array and side data
* in the packets. I.e. a given side data is either exported by the muxer
* (demuxing) / set by the caller (muxing) in this array, then it never
* appears in the packets, or the side data is exported / sent through
* the packets (always in the first packet where the value becomes known or
* changes), then it does not appear in this array.
*
* - demuxing: Set by libavformat when the stream is created.
* - muxing: May be set by the caller before avformat_write_header().
*
* Freed by libavformat in avformat_free_context().
*
* @see av_format_inject_global_side_data()
*/
AVPacketSideData *side_data;
/**
* The number of elements in the AVStream.side_data array.
*/
int nb_side_data;
/**
* Flags indicating events happening on the stream, a combination of
* AVSTREAM_EVENT_FLAG_*.
*
* - demuxing: may be set by the demuxer in avformat_open_input(),
* avformat_find_stream_info() and av_read_frame(). Flags must be cleared
* by the user once the event has been handled.
* - muxing: may be set by the user after avformat_write_header(). to
* indicate a user-triggered event. The muxer will clear the flags for
* events it has handled in av_[interleaved]_write_frame().
*/
int event_flags;
/**
* - demuxing: the demuxer read new metadata from the file and updated
* AVStream.metadata accordingly
* - muxing: the user updated AVStream.metadata and wishes the muxer to write
* it into the file
*/
#define AVSTREAM_EVENT_FLAG_METADATA_UPDATED 0x0001
/**
* - demuxing: new packets for this stream were read from the file. This
* event is informational only and does not guarantee that new packets
* for this stream will necessarily be returned from av_read_frame().
*/
#define AVSTREAM_EVENT_FLAG_NEW_PACKETS (1 << 1)
/**
* Real base framerate of the stream.
* This is the lowest framerate with which all timestamps can be
* represented accurately (it is the least common multiple of all
* framerates in the stream). Note, this value is just a guess!
* For example, if the time base is 1/90000 and all frames have either
* approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.
*/
AVRational r_frame_rate;
#if FF_API_LAVF_FFSERVER
/**
* String containing pairs of key and values describing recommended encoder configuration.
* Pairs are separated by ','.
* Keys are separated from values by '='.
*
* @deprecated unused
*/
attribute_deprecated
char *recommended_encoder_configuration;
#endif
/**
* Codec parameters associated with this stream. Allocated and freed by
* libavformat in avformat_new_stream() and avformat_free_context()
* respectively.
*
* - demuxing: filled by libavformat on stream creation or in
* avformat_find_stream_info()
* - muxing: filled by the caller before avformat_write_header()
*/
// 该解码器参数属性
AVCodecParameters *codecpar;
/*****************************************************************
* All fields below this line are not part of the public API. They
* may not be used outside of libavformat and can be changed and
* removed at will.
* Internal note: be aware that physically removing these fields
* will break ABI. Replace removed fields with dummy fields, and
* add new fields to AVStreamInternal.
*****************************************************************
*/
#if LIBAVFORMAT_VERSION_MAJOR < 59
// kept for ABI compatibility only, do not access in any way
void *unused;
#endif
int pts_wrap_bits; /**< number of bits in pts (used for wrapping control) */
// Timestamp generation support:
/**
* Timestamp corresponding to the last dts sync point.
*
* Initialized when AVCodecParserContext.dts_sync_point >= 0 and
* a DTS is received from the underlying container. Otherwise set to
* AV_NOPTS_VALUE by default.
*/
int64_t first_dts;
int64_t cur_dts;
int64_t last_IP_pts;
int last_IP_duration;
/**
* Number of packets to buffer for codec probing
*/
int probe_packets;
/**
* Number of frames that have been demuxed during avformat_find_stream_info()
*/
int codec_info_nb_frames;
/* av_read_frame() support */
enum AVStreamParseType need_parsing;
struct AVCodecParserContext *parser;
#if LIBAVFORMAT_VERSION_MAJOR < 59
// kept for ABI compatibility only, do not access in any way
void *unused7;
AVProbeData unused6;
int64_t unused5[16+1];
#endif
AVIndexEntry *index_entries; /**< Only used if the format does not
support seeking natively. */
int nb_index_entries;
unsigned int index_entries_allocated_size;
/**
* Stream Identifier
* This is the MPEG-TS stream identifier +1
* 0 means unknown
*/
int stream_identifier;
#if LIBAVFORMAT_VERSION_MAJOR < 59
// kept for ABI compatibility only, do not access in any way
int unused8;
int unused9;
int unused10;
#endif
/**
* An opaque field for libavformat internal usage.
* Must not be accessed in any way by callers.
*/
AVStreamInternal *internal;
} AVStream;
3.4 AVCodecContext
编解码器上下文结构体，保存了视频(音频)编解码相关信息。该结构体定义位于avcodec.h，其详细结构如下
/**
* main external API structure.
* New fields can be added to the end with minor version bumps.
* Removal, reordering and changes to existing fields require a major
* version bump.
* You can use AVOptions (av_opt* / av_set/get*()) to access these fields from user
* applications.
* The name string for AVOptions options matches the associated command line
* parameter name and can be found in libavcodec/options_table.h
* The AVOption/command line parameter names differ in some cases from the C
* structure field names for historic reasons or brevity.
* sizeof(AVCodecContext) must not be used outside libav*.
*/
typedef struct AVCodecContext {
/**
* information on struct for av_log
* - set by avcodec_alloc_context3
*/
const AVClass *av_class;
int log_level_offset;
enum AVMediaType codec_type; /* see AVMEDIA_TYPE_xxx */
// 编解码器的AVCodec，比如指向AVCodecff_aac_latm_decoder
const struct AVCodec *codec;
enum AVCodecID codec_id; /* see AV_CODEC_ID_xxx */
/**
* fourcc (LSB first, so "ABCD" -> ('D'<<24) + ('C'<<16) + ('B'<<8) + 'A').
* This is used to work around some encoder bugs.
* A demuxer should set this to what is stored in the field used to identify the codec.
* If there are multiple such fields in a container then the demuxer should choose the one
* which maximizes the information about the used codec.
* If the codec tag field in a container is larger than 32 bits then the demuxer should
* remap the longer ID to 32 bits with a table or other structure. Alternatively a new
* extra_codec_tag + size could be added but for this a clear advantage must be demonstrated
* first.
* - encoding: Set by user, if not then the default based on codec_id will be used.
* - decoding: Set by user, will be converted to uppercase by libavcodec during init.
*/
unsigned int codec_tag;
void *priv_data;
/**
* Private context used for internal data.
*
* Unlike priv_data, this is not codec-specific. It is used in general
* libavcodec functions.
*/
struct AVCodecInternal *internal;
/**
* Private data of the user, can be used to carry app specific stuff.
* - encoding: Set by user.
* - decoding: Set by user.
*/
void *opaque;
/**
* the average bitrate
* - encoding: Set by user; unused for constant quantizer encoding.
* - decoding: Set by user, may be overwritten by libavcodec
* if this info is available in the stream
*/
int64_t bit_rate;
/**
* number of bits the bitstream is allowed to diverge from the reference.
* the reference can be CBR (for CBR pass1) or VBR (for pass2)
* - encoding: Set by user; unused for constant quantizer encoding.
* - decoding: unused
*/
int bit_rate_tolerance;
/**
* Global quality for codecs which cannot change it per frame.
* This should be proportional to MPEG-1/2/4 qscale.
* - encoding: Set by user.
* - decoding: unused
*/
int global_quality;
/**
* - encoding: Set by user.
* - decoding: unused
*/
int compression_level;
#define FF_COMPRESSION_DEFAULT -1
/**
* AV_CODEC_FLAG_*.
* - encoding: Set by user.
* - decoding: Set by user.
*/
int flags;
/**
* AV_CODEC_FLAG2_*
* - encoding: Set by user.
* - decoding: Set by user.
*/
int flags2;
/**
* some codecs need / can use extradata like Huffman tables.
* MJPEG: Huffman tables
* rv10: additional flags
* MPEG-4: global headers (they can be in the bitstream or here)
* The allocated memory should be AV_INPUT_BUFFER_PADDING_SIZE bytes larger
* than extradata_size to avoid problems if it is read with the bitstream reader.
* The bytewise contents of extradata must not depend on the architecture or CPU endianness.
* Must be allocated with the av_malloc() family of functions.
* - encoding: Set/allocated/freed by libavcodec.
* - decoding: Set/allocated/freed by user.
*/
uint8_t *extradata;
int extradata_size;
/**
* This is the fundamental unit of time (in seconds) in terms
* of which frame timestamps are represented. For fixed-fps content,
* timebase should be 1/framerate and timestamp increments should be
* identically 1.
* This often, but not always is the inverse of the frame rate or field rate
* for video. 1/time_base is not the average frame rate if the frame rate is not
* constant.
*
* Like containers, elementary streams also can store timestamps, 1/time_base
* is the unit in which these timestamps are specified.
* As example of such codec time base see ISO/IEC 14496-2:2001(E)
* vop_time_increment_resolution and fixed_vop_rate
* (fixed_vop_rate == 0 implies that it is different from the framerate)
*
* - encoding: MUST be set by user.
* - decoding: the use of this field for decoding is deprecated.
* Use framerate instead.
*/
AVRational time_base;
/**
* For some codecs, the time base is closer to the field rate than the frame rate.
* Most notably, H.264 and MPEG-2 specify time_base as half of frame duration
* if no telecine is used ...
*
* Set to time_base ticks per frame. Default 1, e.g., H.264/MPEG-2 set it to 2.
*/
int ticks_per_frame;
/**
* Codec delay.
*
* Encoding: Number of frames delay there will be from the encoder input to
* the decoder output. (we assume the decoder matches the spec)
* Decoding: Number of frames delay in addition to what a standard decoder
* as specified in the spec would produce.
*
* Video:
* Number of frames the decoded output will be delayed relative to the
* encoded input.
*
* Audio:
* For encoding, this field is unused (see initial_padding).
*
* For decoding, this is the number of samples the decoder needs to
* output before the decoder's output is valid. When seeking, you should
* start decoding this many samples prior to your desired seek point.
*
* - encoding: Set by libavcodec.
* - decoding: Set by libavcodec.
*/
int delay;
/* video only */
/**
* picture width / height.
*
* @note Those fields may not match the values of the last
* AVFrame output by avcodec_decode_video2 due frame
* reordering.
*
* - encoding: MUST be set by user.
* - decoding: May be set by the user before opening the decoder if known e.g.
* from the container. Some decoders will require the dimensions
* to be set by the caller. During decoding, the decoder may
* overwrite those values as required while parsing the data.
*/
// 图像的宽高，只针对视频
int width, height;
/**
* Bitstream width / height, may be different from width/height e.g. when
* the decoded frame is cropped before being output or lowres is enabled.
*
* @note Those field may not match the value of the last
* AVFrame output by avcodec_receive_frame() due frame
* reordering.
*
* - encoding: unused
* - decoding: May be set by the user before opening the decoder if known
* e.g. from the container. During decoding, the decoder may
* overwrite those values as required while parsing the data.
*/
int coded_width, coded_height;
/**
* the number of pictures in a group of pictures, or 0 for intra_only
* - encoding: Set by user.
* - decoding: unused
*/
int gop_size;
/**
* Pixel format, see AV_PIX_FMT_xxx.
* May be set by the demuxer if known from headers.
* May be overridden by the decoder if it knows better.
*
* @note This field may not match the value of the last
* AVFrame output by avcodec_receive_frame() due frame
* reordering.
*
* - encoding: Set by user.
* - decoding: Set by user if known, overridden by libavcodec while
* parsing the data.
*/
//像素格式(只针对视频)
enum AVPixelFormat pix_fmt;
/**
* If non NULL, 'draw_horiz_band' is called by the libavcodec
* decoder to draw a horizontal band. It improves cache usage. Not
* all codecs can do that. You must check the codec capabilities
* beforehand.
* When multithreading is used, it may be called from multiple threads
* at the same time; threads might draw different parts of the same AVFrame,
* or multiple AVFrames, and there is no guarantee that slices will be drawn
* in order.
* The function is also used by hardware acceleration APIs.
* It is called at least once during frame decoding to pass
* the data needed for hardware render.
* In that mode instead of pixel data, AVFrame points to
* a structure specific to the acceleration API. The application
* reads the structure and can change some fields to indicate progress
* or mark state.
* - encoding: unused
* - decoding: Set by user.
* @param height the height of the slice
* @param y the y position of the slice
* @param type 1->top field, 2->bottom field, 3->frame
* @param offset offset into the AVFrame.data from which the slice should be read
*/
void (*draw_horiz_band)(struct AVCodecContext *s,
const AVFrame *src, int offset[AV_NUM_DATA_POINTERS],
int y, int type, int height);
/**
* callback to negotiate the pixelFormat
* @param fmt is the list of formats which are supported by the codec,
* it is terminated by -1 as 0 is a valid format, the formats are ordered by quality.
* The first is always the native one.
* @note The callback may be called again immediately if initialization for
* the selected (hardware-accelerated) pixel format failed.
* @warning Behavior is undefined if the callback returns a value not
* in the fmt list of formats.
* @return the chosen format
* - encoding: unused
* - decoding: Set by user, if not set the native format will be chosen.
*/
enum AVPixelFormat (*get_format)(struct AVCodecContext *s, const enum AVPixelFormat * fmt);
/**
* maximum number of B-frames between non-B-frames
* Note: The output will be delayed by max_b_frames+1 relative to the input.
* - encoding: Set by user.
* - decoding: unused
*/
int max_b_frames;
/**
* qscale factor between IP and B-frames
* If > 0 then the last P-frame quantizer will be used (q= lastp_q*factor+offset).
* If < 0 then normal ratecontrol will be done (q= -normal_q*factor+offset).
* - encoding: Set by user.
* - decoding: unused
*/
float b_quant_factor;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int b_frame_strategy;
#endif
/**
* qscale offset between IP and B-frames
* - encoding: Set by user.
* - decoding: unused
*/
float b_quant_offset;
/**
* Size of the frame reordering buffer in the decoder.
* For MPEG-2 it is 1 IPB or 0 low delay IP.
* - encoding: Set by libavcodec.
* - decoding: Set by libavcodec.
*/
int has_b_frames;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int mpeg_quant;
#endif
/**
* qscale factor between P- and I-frames
* If > 0 then the last P-frame quantizer will be used (q = lastp_q * factor + offset).
* If < 0 then normal ratecontrol will be done (q= -normal_q*factor+offset).
* - encoding: Set by user.
* - decoding: unused
*/
float i_quant_factor;
/**
* qscale offset between P and I-frames
* - encoding: Set by user.
* - decoding: unused
*/
float i_quant_offset;
/**
* luminance masking (0-> disabled)
* - encoding: Set by user.
* - decoding: unused
*/
float lumi_masking;
/**
* temporary complexity masking (0-> disabled)
* - encoding: Set by user.
* - decoding: unused
*/
float temporal_cplx_masking;
/**
* spatial complexity masking (0-> disabled)
* - encoding: Set by user.
* - decoding: unused
*/
float spatial_cplx_masking;
/**
* p block masking (0-> disabled)
* - encoding: Set by user.
* - decoding: unused
*/
float p_masking;
/**
* darkness masking (0-> disabled)
* - encoding: Set by user.
* - decoding: unused
*/
float dark_masking;
/**
* slice count
* - encoding: Set by libavcodec.
* - decoding: Set by user (or 0).
*/
int slice_count;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int prediction_method;
#define FF_PRED_LEFT 0
#define FF_PRED_PLANE 1
#define FF_PRED_MEDIAN 2
#endif
/**
* slice offsets in the frame in bytes
* - encoding: Set/allocated by libavcodec.
* - decoding: Set/allocated by user (or NULL).
*/
int *slice_offset;
/**
* sample aspect ratio (0 if unknown)
* That is the width of a pixel divided by the height of the pixel.
* Numerator and denominator must be relatively prime and smaller than 256 for some video standards.
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
AVRational sample_aspect_ratio;
/**
* motion estimation comparison function
* - encoding: Set by user.
* - decoding: unused
*/
int me_cmp;
/**
* subpixel motion estimation comparison function
* - encoding: Set by user.
* - decoding: unused
*/
int me_sub_cmp;
/**
* macroblock comparison function (not supported yet)
* - encoding: Set by user.
* - decoding: unused
*/
int mb_cmp;
/**
* interlaced DCT comparison function
* - encoding: Set by user.
* - decoding: unused
*/
int ildct_cmp;
#define FF_CMP_SAD 0
#define FF_CMP_SSE 1
#define FF_CMP_SATD 2
#define FF_CMP_DCT 3
#define FF_CMP_PSNR 4
#define FF_CMP_BIT 5
#define FF_CMP_RD 6
#define FF_CMP_ZERO 7
#define FF_CMP_VSAD 8
#define FF_CMP_VSSE 9
#define FF_CMP_NSSE 10
#define FF_CMP_W53 11
#define FF_CMP_W97 12
#define FF_CMP_DCTMAX 13
#define FF_CMP_DCT264 14
#define FF_CMP_MEDIAN_SAD 15
#define FF_CMP_CHROMA 256
/**
* ME diamond size & shape
* - encoding: Set by user.
* - decoding: unused
*/
int dia_size;
/**
* amount of previous MV predictors (2a+1 x 2a+1 square)
* - encoding: Set by user.
* - decoding: unused
*/
int last_predictor_count;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int pre_me;
#endif
/**
* motion estimation prepass comparison function
* - encoding: Set by user.
* - decoding: unused
*/
int me_pre_cmp;
/**
* ME prepass diamond size & shape
* - encoding: Set by user.
* - decoding: unused
*/
int pre_dia_size;
/**
* subpel ME quality
* - encoding: Set by user.
* - decoding: unused
*/
int me_subpel_quality;
/**
* maximum motion estimation search range in subpel units
* If 0 then no limit.
*
* - encoding: Set by user.
* - decoding: unused
*/
int me_range;
/**
* slice flags
* - encoding: unused
* - decoding: Set by user.
*/
int slice_flags;
#define SLICE_FLAG_CODED_ORDER 0x0001 ///< draw_horiz_band() is called in coded order instead of display
#define SLICE_FLAG_ALLOW_FIELD 0x0002 ///< allow draw_horiz_band() with field slices (MPEG-2 field pics)
#define SLICE_FLAG_ALLOW_PLANE 0x0004 ///< allow draw_horiz_band() with 1 component at a time (SVQ1)
/**
* macroblock decision mode
* - encoding: Set by user.
* - decoding: unused
*/
int mb_decision;
#define FF_MB_DECISION_SIMPLE 0 ///< uses mb_cmp
#define FF_MB_DECISION_BITS 1 ///< chooses the one which needs the fewest bits
#define FF_MB_DECISION_RD 2 ///< rate distortion
/**
* custom intra quantization matrix
* Must be allocated with the av_malloc() family of functions, and will be freed in
* avcodec_free_context().
* - encoding: Set/allocated by user, freed by libavcodec. Can be NULL.
* - decoding: Set/allocated/freed by libavcodec.
*/
uint16_t *intra_matrix;
/**
* custom inter quantization matrix
* Must be allocated with the av_malloc() family of functions, and will be freed in
* avcodec_free_context().
* - encoding: Set/allocated by user, freed by libavcodec. Can be NULL.
* - decoding: Set/allocated/freed by libavcodec.
*/
uint16_t *inter_matrix;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int scenechange_threshold;
/** @deprecated use encoder private options instead */
attribute_deprecated
int noise_reduction;
#endif
/**
* precision of the intra DC coefficient - 8
* - encoding: Set by user.
* - decoding: Set by libavcodec
*/
int intra_dc_precision;
/**
* Number of macroblock rows at the top which are skipped.
* - encoding: unused
* - decoding: Set by user.
*/
int skip_top;
/**
* Number of macroblock rows at the bottom which are skipped.
* - encoding: unused
* - decoding: Set by user.
*/
int skip_bottom;
/**
* minimum MB Lagrange multiplier
* - encoding: Set by user.
* - decoding: unused
*/
int mb_lmin;
/**
* maximum MB Lagrange multiplier
* - encoding: Set by user.
* - decoding: unused
*/
int mb_lmax;
#if FF_API_PRIVATE_OPT
/**
* @deprecated use encoder private options instead
*/
attribute_deprecated
int me_penalty_compensation;
#endif
/**
* - encoding: Set by user.
* - decoding: unused
*/
int bidir_refine;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int brd_scale;
#endif
/**
* minimum GOP size
* - encoding: Set by user.
* - decoding: unused
*/
int keyint_min;
/**
* number of reference frames
* - encoding: Set by user.
* - decoding: Set by lavc.
*/
int refs;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int chromaoffset;
#endif
/**
* Note: Value depends upon the compare function used for fullpel ME.
* - encoding: Set by user.
* - decoding: unused
*/
int mv0_threshold;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int b_sensitivity;
#endif
/**
* Chromaticity coordinates of the source primaries.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorPrimaries color_primaries;
/**
* Color Transfer Characteristic.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorTransferCharacteristic color_trc;
/**
* YUV colorspace type.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorSpace colorspace;
/**
* MPEG vs JPEG YUV range.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorRange color_range;
/**
* This defines the location of chroma samples.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVChromaLocation chroma_sample_location;
/**
* Number of slices.
* Indicates number of picture subdivisions. Used for parallelized
* decoding.
* - encoding: Set by user
* - decoding: unused
*/
int slices;
/** Field order
* - encoding: set by libavcodec
* - decoding: Set by user.
*/
enum AVFieldOrder field_order;
/* audio only */
int sample_rate; // 采样率(秒)
int channels; // 声道数
/**
* audio sample format
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
enum AVSampleFormat sample_fmt; //采样格式
/* The following data should not be initialized. */
/**
* Number of samples per channel in an audio frame.
*
* - encoding: set by libavcodec in avcodec_open2(). Each submitted frame
* except the last must contain exactly frame_size samples per channel.
* May be 0 when the codec has AV_CODEC_CAP_VARIABLE_FRAME_SIZE set, then the
* frame size is not restricted.
* - decoding: may be set by some decoders to indicate constant frame size
*/
int frame_size;
/**
* Frame counter, set by libavcodec.
*
* - decoding: total number of frames returned from the decoder so far.
* - encoding: total number of frames passed to the encoder so far.
*
* @note the counter is not incremented if encoding/decoding resulted in
* an error.
*/
int frame_number;
/**
* number of bytes per packet if constant and known or 0
* Used by some WAV based audio codecs.
*/
int block_align;
/**
* Audio cutoff bandwidth (0 means "automatic")
* - encoding: Set by user.
* - decoding: unused
*/
int cutoff;
/**
* Audio channel layout.
* - encoding: set by user.
* - decoding: set by user, may be overwritten by libavcodec.
*/
uint64_t channel_layout;
/**
* Request decoder to use this channel layout if it can (0 for default)
* - encoding: unused
* - decoding: Set by user.
*/
uint64_t request_channel_layout;
/**
* Type of service that the audio stream conveys.
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
enum AVAudioServiceType audio_service_type;
/**
* desired sample format
* - encoding: Not used.
* - decoding: Set by user.
* Decoder will decode to this format if it can.
*/
enum AVSampleFormat request_sample_fmt;
/**
* This callback is called at the beginning of each frame to get data
* buffer(s) for it. There may be one contiguous buffer for all the data or
* there may be a buffer per each data plane or anything in between. What
* this means is, you may set however many entries in buf[] you feel necessary.
* Each buffer must be reference-counted using the AVBuffer API (see description
* of buf[] below).
*
* The following fields will be set in the frame before this callback is
* called:
* - format
* - width, height (video only)
* - sample_rate, channel_layout, nb_samples (audio only)
* Their values may differ from the corresponding values in
* AVCodecContext. This callback must use the frame values, not the codec
* context values, to calculate the required buffer size.
*
* This callback must fill the following fields in the frame:
* - data[]
* - linesize[]
* - extended_data:
* * if the data is planar audio with more than 8 channels, then this
* callback must allocate and fill extended_data to contain all pointers
* to all data planes. data[] must hold as many pointers as it can.
* extended_data must be allocated with av_malloc() and will be freed in
* av_frame_unref().
* * otherwise extended_data must point to data
* - buf[] must contain one or more pointers to AVBufferRef structures. Each of
* the frame's data and extended_data pointers must be contained in these. That
* is, one AVBufferRef for each allocated chunk of memory, not necessarily one
* AVBufferRef per data[] entry. See: av_buffer_create(), av_buffer_alloc(),
* and av_buffer_ref().
* - extended_buf and nb_extended_buf must be allocated with av_malloc() by
* this callback and filled with the extra buffers if there are more
* buffers than buf[] can hold. extended_buf will be freed in
* av_frame_unref().
*
* If AV_CODEC_CAP_DR1 is not set then get_buffer2() must call
* avcodec_default_get_buffer2() instead of providing buffers allocated by
* some other means.
*
* Each data plane must be aligned to the maximum required by the target
* CPU.
*
* @see avcodec_default_get_buffer2()
*
* Video:
*
* If AV_GET_BUFFER_FLAG_REF is set in flags then the frame may be reused
* (read and/or written to if it is writable) later by libavcodec.
*
* avcodec_align_dimensions2() should be used to find the required width and
* height, as they normally need to be rounded up to the next multiple of 16.
*
* Some decoders do not support linesizes changing between frames.
*
* If frame multithreading is used, this callback may be called from a
* different thread, but not from more than one at once. Does not need to be
* reentrant.
*
* @see avcodec_align_dimensions2()
*
* Audio:
*
* Decoders request a buffer of a particular size by setting
* AVFrame.nb_samples prior to calling get_buffer2(). The decoder may,
* however, utilize only part of the buffer by setting AVFrame.nb_samples
* to a smaller value in the output frame.
*
* As a convenience, av_samples_get_buffer_size() and
* av_samples_fill_arrays() in libavutil may be used by custom get_buffer2()
* functions to find the required data size and to fill data pointers and
* linesize. In AVFrame.linesize, only linesize[0] may be set for audio
* since all planes must be the same size.
*
* @see av_samples_get_buffer_size(), av_samples_fill_arrays()
*
* - encoding: unused
* - decoding: Set by libavcodec, user can override.
*/
int (*get_buffer2)(struct AVCodecContext *s, AVFrame *frame, int flags);
#if FF_API_OLD_ENCDEC
/**
* If non-zero, the decoded audio and video frames returned from
* avcodec_decode_video2() and avcodec_decode_audio4() are reference-counted
* and are valid indefinitely. The caller must free them with
* av_frame_unref() when they are not needed anymore.
* Otherwise, the decoded frames must not be freed by the caller and are
* only valid until the next decode call.
*
* This is always automatically enabled if avcodec_receive_frame() is used.
*
* - encoding: unused
* - decoding: set by the caller before avcodec_open2().
*/
attribute_deprecated
int refcounted_frames;
#endif
/* - encoding parameters */
float qcompress; ///< amount of qscale change between easy & hard scenes (0.0-1.0)
float qblur; ///< amount of qscale smoothing over time (0.0-1.0)
/**
* minimum quantizer
* - encoding: Set by user.
* - decoding: unused
*/
int qmin;
/**
* maximum quantizer
* - encoding: Set by user.
* - decoding: unused
*/
int qmax;
/**
* maximum quantizer difference between frames
* - encoding: Set by user.
* - decoding: unused
*/
int max_qdiff;
/**
* decoder bitstream buffer size
* - encoding: Set by user.
* - decoding: unused
*/
int rc_buffer_size;
/**
* ratecontrol override, see RcOverride
* - encoding: Allocated/set/freed by user.
* - decoding: unused
*/
int rc_override_count;
RcOverride *rc_override;
/**
* maximum bitrate
* - encoding: Set by user.
* - decoding: Set by user, may be overwritten by libavcodec.
*/
int64_t rc_max_rate;
/**
* minimum bitrate
* - encoding: Set by user.
* - decoding: unused
*/
int64_t rc_min_rate;
/**
* Ratecontrol attempt to use, at maximum, <value> of what can be used without an underflow.
* - encoding: Set by user.
* - decoding: unused.
*/
float rc_max_available_vbv_use;
/**
* Ratecontrol attempt to use, at least, <value> times the amount needed to prevent a vbv overflow.
* - encoding: Set by user.
* - decoding: unused.
*/
float rc_min_vbv_overflow_use;
/**
* Number of bits which should be loaded into the rc buffer before decoding starts.
* - encoding: Set by user.
* - decoding: unused
*/
int rc_initial_buffer_occupancy;
#if FF_API_CODER_TYPE
#define FF_CODER_TYPE_VLC 0
#define FF_CODER_TYPE_AC 1
#define FF_CODER_TYPE_RAW 2
#define FF_CODER_TYPE_RLE 3
/**
* @deprecated use encoder private options instead
*/
attribute_deprecated
int coder_type;
#endif /* FF_API_CODER_TYPE */
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int context_model;
#endif
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int frame_skip_threshold;
/** @deprecated use encoder private options instead */
attribute_deprecated
int frame_skip_factor;
/** @deprecated use encoder private options instead */
attribute_deprecated
int frame_skip_exp;
/** @deprecated use encoder private options instead */
attribute_deprecated
int frame_skip_cmp;
#endif /* FF_API_PRIVATE_OPT */
/**
* trellis RD quantization
* - encoding: Set by user.
* - decoding: unused
*/
int trellis;
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int min_prediction_order;
/** @deprecated use encoder private options instead */
attribute_deprecated
int max_prediction_order;
/** @deprecated use encoder private options instead */
attribute_deprecated
int64_t timecode_frame_start;
#endif
#if FF_API_RTP_CALLBACK
/**
* @deprecated unused
*/
/* The RTP callback: This function is called */
/* every time the encoder has a packet to send. */
/* It depends on the encoder if the data starts */
/* with a Start Code (it should). H.263 does. */
/* mb_nb contains the number of macroblocks */
/* encoded in the RTP payload. */
attribute_deprecated
void (*rtp_callback)(struct AVCodecContext *avctx, void *data, int size, int mb_nb);
#endif
#if FF_API_PRIVATE_OPT
/** @deprecated use encoder private options instead */
attribute_deprecated
int rtp_payload_size; /* The size of the RTP payload: the coder will */
/* do its best to deliver a chunk with size */
/* below rtp_payload_size, the chunk will start */
/* with a start code on some codecs like H.263. */
/* This doesn't take account of any particular */
/* headers inside the transmitted RTP payload. */
#endif
#if FF_API_STAT_BITS
/* statistics, used for 2-pass encoding */
attribute_deprecated
int mv_bits;
attribute_deprecated
int header_bits;
attribute_deprecated
int i_tex_bits;
attribute_deprecated
int p_tex_bits;
attribute_deprecated
int i_count;
attribute_deprecated
int p_count;
attribute_deprecated
int skip_count;
attribute_deprecated
int misc_bits;
/** @deprecated this field is unused */
attribute_deprecated
int frame_bits;
#endif
/**
* pass1 encoding statistics output buffer
* - encoding: Set by libavcodec.
* - decoding: unused
*/
char *stats_out;
/**
* pass2 encoding statistics input buffer
* Concatenated stuff from stats_out of pass1 should be placed here.
* - encoding: Allocated/set/freed by user.
* - decoding: unused
*/
char *stats_in;
/**
* Work around bugs in encoders which sometimes cannot be detected automatically.
* - encoding: Set by user
* - decoding: Set by user
*/
int workaround_bugs;
#define FF_BUG_AUTODETECT 1 ///< autodetection
#define FF_BUG_XVID_ILACE 4
#define FF_BUG_UMP4 8
#define FF_BUG_NO_PADDING 16
#define FF_BUG_AMV 32
#define FF_BUG_QPEL_CHROMA 64
#define FF_BUG_STD_QPEL 128
#define FF_BUG_QPEL_CHROMA2 256
#define FF_BUG_DIRECT_BLOCKSIZE 512
#define FF_BUG_EDGE 1024
#define FF_BUG_HPEL_CHROMA 2048
#define FF_BUG_DC_CLIP 4096
#define FF_BUG_MS 8192 ///< Work around various bugs in Microsoft's broken decoders.
#define FF_BUG_TRUNCATED 16384
#define FF_BUG_IEDGE 32768
/**
* strictly follow the standard (MPEG-4, ...).
* - encoding: Set by user.
* - decoding: Set by user.
* Setting this to STRICT or higher means the encoder and decoder will
* generally do stupid things, whereas setting it to unofficial or lower
* will mean the encoder might produce output that is not supported by all
* spec-compliant decoders. Decoders don't differentiate between normal,
* unofficial and experimental (that is, they always try to decode things
* when they can) unless they are explicitly asked to behave stupidly
* (=strictly conform to the specs)
*/
int strict_std_compliance;
#define FF_COMPLIANCE_VERY_STRICT 2 ///< Strictly conform to an older more strict version of the spec or reference software.
#define FF_COMPLIANCE_STRICT 1 ///< Strictly conform to all the things in the spec no matter what consequences.
#define FF_COMPLIANCE_NORMAL 0
#define FF_COMPLIANCE_UNOFFICIAL -1 ///< Allow unofficial extensions
#define FF_COMPLIANCE_EXPERIMENTAL -2 ///< Allow nonstandardized experimental things.
/**
* error concealment flags
* - encoding: unused
* - decoding: Set by user.
*/
int error_concealment;
#define FF_EC_GUESS_MVS 1
#define FF_EC_DEBLOCK 2
#define FF_EC_FAVOR_INTER 256
/**
* debug
* - encoding: Set by user.
* - decoding: Set by user.
*/
int debug;
#define FF_DEBUG_PICT_INFO 1
#define FF_DEBUG_RC 2
#define FF_DEBUG_BITSTREAM 4
#define FF_DEBUG_MB_TYPE 8
#define FF_DEBUG_QP 16
#define FF_DEBUG_DCT_COEFF 0x00000040
#define FF_DEBUG_SKIP 0x00000080
#define FF_DEBUG_STARTCODE 0x00000100
#define FF_DEBUG_ER 0x00000400
#define FF_DEBUG_MMCO 0x00000800
#define FF_DEBUG_BUGS 0x00001000
#define FF_DEBUG_BUFFERS 0x00008000
#define FF_DEBUG_THREADS 0x00010000
#define FF_DEBUG_GREEN_MD 0x00800000
#define FF_DEBUG_NOMC 0x01000000
/**
* Error recognition; may misdetect some more or less valid parts as errors.
* - encoding: Set by user.
* - decoding: Set by user.
*/
int err_recognition;
/**
* Verify checksums embedded in the bitstream (could be of either encoded or
* decoded data, depending on the codec) and print an error message on mismatch.
* If AV_EF_EXPLODE is also set, a mismatching checksum will result in the
* decoder returning an error.
*/
#define AV_EF_CRCCHECK (1<<0)
#define AV_EF_BITSTREAM (1<<1) ///< detect bitstream specification deviations
#define AV_EF_BUFFER (1<<2) ///< detect improper bitstream length
#define AV_EF_EXPLODE (1<<3) ///< abort decoding on minor error detection
#define AV_EF_IGNORE_ERR (1<<15) ///< ignore errors and continue
#define AV_EF_CAREFUL (1<<16) ///< consider things that violate the spec, are fast to calculate and have not been seen in the wild as errors
#define AV_EF_COMPLIANT (1<<17) ///< consider all spec non compliances as errors
#define AV_EF_AGGRESSIVE (1<<18) ///< consider things that a sane encoder should not do as an error
/**
* opaque 64-bit number (generally a PTS) that will be reordered and
* output in AVFrame.reordered_opaque
* - encoding: Set by libavcodec to the reordered_opaque of the input
* frame corresponding to the last returned packet. Only
* supported by encoders with the
* AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE capability.
* - decoding: Set by user.
*/
int64_t reordered_opaque;
/**
* Hardware accelerator in use
* - encoding: unused.
* - decoding: Set by libavcodec
*/
const struct AVHWAccel *hwaccel;
/**
* Hardware accelerator context.
* For some hardware accelerators, a global context needs to be
* provided by the user. In that case, this holds display-dependent
* data FFmpeg cannot instantiate itself. Please refer to the
* FFmpeg HW accelerator documentation to know how to fill this
* is. e.g. for VA API, this is a struct vaapi_context.
* - encoding: unused
* - decoding: Set by user
*/
void *hwaccel_context;
/**
* error
* - encoding: Set by libavcodec if flags & AV_CODEC_FLAG_PSNR.
* - decoding: unused
*/
uint64_t error[AV_NUM_DATA_POINTERS];
/**
* DCT algorithm, see FF_DCT_* below
* - encoding: Set by user.
* - decoding: unused
*/
int dct_algo;
#define FF_DCT_AUTO 0
#define FF_DCT_FASTINT 1
#define FF_DCT_INT 2
#define FF_DCT_MMX 3
#define FF_DCT_ALTIVEC 5
#define FF_DCT_FAAN 6
/**
* IDCT algorithm, see FF_IDCT_* below.
* - encoding: Set by user.
* - decoding: Set by user.
*/
int idct_algo;
#define FF_IDCT_AUTO 0
#define FF_IDCT_INT 1
#define FF_IDCT_SIMPLE 2
#define FF_IDCT_SIMPLEMMX 3
#define FF_IDCT_ARM 7
#define FF_IDCT_ALTIVEC 8
#define FF_IDCT_SIMPLEARM 10
#define FF_IDCT_XVID 14
#define FF_IDCT_SIMPLEARMV5TE 16
#define FF_IDCT_SIMPLEARMV6 17
#define FF_IDCT_FAAN 20
#define FF_IDCT_SIMPLENEON 22
#define FF_IDCT_NONE 24 /* Used by XvMC to extract IDCT coefficients with FF_IDCT_PERM_NONE */
#define FF_IDCT_SIMPLEAUTO 128
/**
* bits per sample/pixel from the demuxer (needed for huffyuv).
* - encoding: Set by libavcodec.
* - decoding: Set by user.
*/
int bits_per_coded_sample;
/**
* Bits per sample/pixel of internal libavcodec pixel/sample format.
* - encoding: set by user.
* - decoding: set by libavcodec.
*/
int bits_per_raw_sample;
/**
* low resolution decoding, 1-> 1/2 size, 2->1/4 size
* - encoding: unused
* - decoding: Set by user.
*/
int lowres;
#if FF_API_CODED_FRAME
/**
* the picture in the bitstream
* - encoding: Set by libavcodec.
* - decoding: unused
*
* @deprecated use the quality factor packet side data instead
*/
attribute_deprecated AVFrame *coded_frame;
#endif
/**
* thread count
* is used to decide how many independent tasks should be passed to execute()
* - encoding: Set by user.
* - decoding: Set by user.
*/
int thread_count;
/**
* Which multithreading methods to use.
* Use of FF_THREAD_FRAME will increase decoding delay by one frame per thread,
* so clients which cannot provide future frames should not use it.
*
* - encoding: Set by user, otherwise the default is used.
* - decoding: Set by user, otherwise the default is used.
*/
int thread_type;
#define FF_THREAD_FRAME 1 ///< Decode more than one frame at once
#define FF_THREAD_SLICE 2 ///< Decode more than one part of a single frame at once
/**
* Which multithreading methods are in use by the codec.
* - encoding: Set by libavcodec.
* - decoding: Set by libavcodec.
*/
int active_thread_type;
#if FF_API_THREAD_SAFE_CALLBACKS
/**
* Set by the client if its custom get_buffer() callback can be called
* synchronously from another thread, which allows faster multithreaded decoding.
* draw_horiz_band() will be called from other threads regardless of this setting.
* Ignored if the default get_buffer() is used.
* - encoding: Set by user.
* - decoding: Set by user.
*
* @deprecated the custom get_buffer2() callback should always be
* thread-safe. Thread-unsafe get_buffer2() implementations will be
* invalid starting with LIBAVCODEC_VERSION_MAJOR=60; in other words,
* libavcodec will behave as if this field was always set to 1.
* Callers that want to be forward compatible with future libavcodec
* versions should wrap access to this field in
* #if LIBAVCODEC_VERSION_MAJOR < 60
*/
attribute_deprecated
int thread_safe_callbacks;
#endif
/**
* The codec may call this to execute several independent things.
* It will return only after finishing all tasks.
* The user may replace this with some multithreaded implementation,
* the default implementation will execute the parts serially.
* @param count the number of things to execute
* - encoding: Set by libavcodec, user can override.
* - decoding: Set by libavcodec, user can override.
*/
int (*execute)(struct AVCodecContext *c, int (*func)(struct AVCodecContext *c2, void *arg), void *arg2, int *ret, int count, int size);
/**
* The codec may call this to execute several independent things.
* It will return only after finishing all tasks.
* The user may replace this with some multithreaded implementation,
* the default implementation will execute the parts serially.
* Also see avcodec_thread_init and e.g. the --enable-pthread configure option.
* @param c context passed also to func
* @param count the number of things to execute
* @param arg2 argument passed unchanged to func
* @param ret return values of executed functions, must have space for "count" values. May be NULL.
* @param func function that will be called count times, with jobnr from 0 to count-1.
* threadnr will be in the range 0 to c->thread_count-1 < MAX_THREADS and so that no
* two instances of func executing at the same time will have the same threadnr.
* @return always 0 currently, but code should handle a future improvement where when any call to func
* returns < 0 no further calls to func may be done and < 0 is returned.
* - encoding: Set by libavcodec, user can override.
* - decoding: Set by libavcodec, user can override.
*/
int (*execute2)(struct AVCodecContext *c, int (*func)(struct AVCodecContext *c2, void *arg, int jobnr, int threadnr), void *arg2, int *ret, int count);
/**
* noise vs. sse weight for the nsse comparison function
* - encoding: Set by user.
* - decoding: unused
*/
int nsse_weight;
/**
* profile
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
int profile;
#define FF_PROFILE_UNKNOWN -99
#define FF_PROFILE_RESERVED -100
#define FF_PROFILE_AAC_MAIN 0
#define FF_PROFILE_AAC_LOW 1
#define FF_PROFILE_AAC_SSR 2
#define FF_PROFILE_AAC_LTP 3
#define FF_PROFILE_AAC_HE 4
#define FF_PROFILE_AAC_HE_V2 28
#define FF_PROFILE_AAC_LD 22
#define FF_PROFILE_AAC_ELD 38
#define FF_PROFILE_MPEG2_AAC_LOW 128
#define FF_PROFILE_MPEG2_AAC_HE 131
#define FF_PROFILE_DNXHD 0
#define FF_PROFILE_DNXHR_LB 1
#define FF_PROFILE_DNXHR_SQ 2
#define FF_PROFILE_DNXHR_HQ 3
#define FF_PROFILE_DNXHR_HQX 4
#define FF_PROFILE_DNXHR_444 5
#define FF_PROFILE_DTS 20
#define FF_PROFILE_DTS_ES 30
#define FF_PROFILE_DTS_96_24 40
#define FF_PROFILE_DTS_HD_HRA 50
#define FF_PROFILE_DTS_HD_MA 60
#define FF_PROFILE_DTS_EXPRESS 70
#define FF_PROFILE_MPEG2_422 0
#define FF_PROFILE_MPEG2_HIGH 1
#define FF_PROFILE_MPEG2_SS 2
#define FF_PROFILE_MPEG2_SNR_SCALABLE 3
#define FF_PROFILE_MPEG2_MAIN 4
#define FF_PROFILE_MPEG2_SIMPLE 5
#define FF_PROFILE_H264_CONSTRAINED (1<<9) // 8+1; constraint_set1_flag
#define FF_PROFILE_H264_INTRA (1<<11) // 8+3; constraint_set3_flag
#define FF_PROFILE_H264_BASELINE 66
#define FF_PROFILE_H264_CONSTRAINED_BASELINE (66|FF_PROFILE_H264_CONSTRAINED)
#define FF_PROFILE_H264_MAIN 77
#define FF_PROFILE_H264_EXTENDED 88
#define FF_PROFILE_H264_HIGH 100
#define FF_PROFILE_H264_HIGH_10 110
#define FF_PROFILE_H264_HIGH_10_INTRA (110|FF_PROFILE_H264_INTRA)
#define FF_PROFILE_H264_MULTIVIEW_HIGH 118
#define FF_PROFILE_H264_HIGH_422 122
#define FF_PROFILE_H264_HIGH_422_INTRA (122|FF_PROFILE_H264_INTRA)
#define FF_PROFILE_H264_STEREO_HIGH 128
#define FF_PROFILE_H264_HIGH_444 144
#define FF_PROFILE_H264_HIGH_444_PREDICTIVE 244
#define FF_PROFILE_H264_HIGH_444_INTRA (244|FF_PROFILE_H264_INTRA)
#define FF_PROFILE_H264_CAVLC_444 44
#define FF_PROFILE_VC1_SIMPLE 0
#define FF_PROFILE_VC1_MAIN 1
#define FF_PROFILE_VC1_COMPLEX 2
#define FF_PROFILE_VC1_ADVANCED 3
#define FF_PROFILE_MPEG4_SIMPLE 0
#define FF_PROFILE_MPEG4_SIMPLE_SCALABLE 1
#define FF_PROFILE_MPEG4_CORE 2
#define FF_PROFILE_MPEG4_MAIN 3
#define FF_PROFILE_MPEG4_N_BIT 4
#define FF_PROFILE_MPEG4_SCALABLE_TEXTURE 5
#define FF_PROFILE_MPEG4_SIMPLE_FACE_ANIMATION 6
#define FF_PROFILE_MPEG4_BASIC_ANIMATED_TEXTURE 7
#define FF_PROFILE_MPEG4_HYBRID 8
#define FF_PROFILE_MPEG4_ADVANCED_REAL_TIME 9
#define FF_PROFILE_MPEG4_CORE_SCALABLE 10
#define FF_PROFILE_MPEG4_ADVANCED_CODING 11
#define FF_PROFILE_MPEG4_ADVANCED_CORE 12
#define FF_PROFILE_MPEG4_ADVANCED_SCALABLE_TEXTURE 13
#define FF_PROFILE_MPEG4_SIMPLE_STUDIO 14
#define FF_PROFILE_MPEG4_ADVANCED_SIMPLE 15
#define FF_PROFILE_JPEG2000_CSTREAM_RESTRICTION_0 1
#define FF_PROFILE_JPEG2000_CSTREAM_RESTRICTION_1 2
#define FF_PROFILE_JPEG2000_CSTREAM_NO_RESTRICTION 32768
#define FF_PROFILE_JPEG2000_DCINEMA_2K 3
#define FF_PROFILE_JPEG2000_DCINEMA_4K 4
#define FF_PROFILE_VP9_0 0
#define FF_PROFILE_VP9_1 1
#define FF_PROFILE_VP9_2 2
#define FF_PROFILE_VP9_3 3
#define FF_PROFILE_HEVC_MAIN 1
#define FF_PROFILE_HEVC_MAIN_10 2
#define FF_PROFILE_HEVC_MAIN_STILL_PICTURE 3
#define FF_PROFILE_HEVC_REXT 4
#define FF_PROFILE_VVC_MAIN_10 1
#define FF_PROFILE_VVC_MAIN_10_444 33
#define FF_PROFILE_AV1_MAIN 0
#define FF_PROFILE_AV1_HIGH 1
#define FF_PROFILE_AV1_PROFESSIONAL 2
#define FF_PROFILE_MJPEG_HUFFMAN_BASELINE_DCT 0xc0
#define FF_PROFILE_MJPEG_HUFFMAN_EXTENDED_SEQUENTIAL_DCT 0xc1
#define FF_PROFILE_MJPEG_HUFFMAN_PROGRESSIVE_DCT 0xc2
#define FF_PROFILE_MJPEG_HUFFMAN_LOSSLESS 0xc3
#define FF_PROFILE_MJPEG_JPEG_LS 0xf7
#define FF_PROFILE_SBC_MSBC 1
#define FF_PROFILE_PRORES_PROXY 0
#define FF_PROFILE_PRORES_LT 1
#define FF_PROFILE_PRORES_STANDARD 2
#define FF_PROFILE_PRORES_HQ 3
#define FF_PROFILE_PRORES_4444 4
#define FF_PROFILE_PRORES_XQ 5
#define FF_PROFILE_ARIB_PROFILE_A 0
#define FF_PROFILE_ARIB_PROFILE_C 1
#define FF_PROFILE_KLVA_SYNC 0
#define FF_PROFILE_KLVA_ASYNC 1
/**
* level
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
int level;
#define FF_LEVEL_UNKNOWN -99
/**
* Skip loop filtering for selected frames.
* - encoding: unused
* - decoding: Set by user.
*/
enum AVDiscard skip_loop_filter;
/**
* Skip IDCT/dequantization for selected frames.
* - encoding: unused
* - decoding: Set by user.
*/
enum AVDiscard skip_idct;
/**
* Skip decoding for selected frames.
* - encoding: unused
* - decoding: Set by user.
*/
enum AVDiscard skip_frame;
/**
* Header containing style information for text subtitles.
* For SUBTITLE_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
* - decoding: Set/allocated/freed by libavcodec (by avcodec_open2())
*/
uint8_t *subtitle_header;
int subtitle_header_size;
#if FF_API_VBV_DELAY
/**
* VBV delay coded in the last frame (in periods of a 27 MHz clock).
* Used for compliant TS muxing.
* - encoding: Set by libavcodec.
* - decoding: unused.
* @deprecated this value is now exported as a part of
* AV_PKT_DATA_CPB_PROPERTIES packet side data
*/
attribute_deprecated
uint64_t vbv_delay;
#endif
#if FF_API_SIDEDATA_ONLY_PKT
/**
* Encoding only and set by default. Allow encoders to output packets
* that do not contain any encoded data, only side data.
*
* Some encoders need to output such packets, e.g. to update some stream
* parameters at the end of encoding.
*
* @deprecated this field disables the default behaviour and
* it is kept only for compatibility.
*/
attribute_deprecated
int side_data_only_packets;
#endif
/**
* Audio only. The number of "priming" samples (padding) inserted by the
* encoder at the beginning of the audio. I.e. this number of leading
* decoded samples must be discarded by the caller to get the original audio
* without leading padding.
*
* - decoding: unused
* - encoding: Set by libavcodec. The timestamps on the output packets are
* adjusted by the encoder so that they always refer to the
* first sample of the data actually contained in the packet,
* including any added padding. E.g. if the timebase is
* 1/samplerate and the timestamp of the first input sample is
* 0, the timestamp of the first output packet will be
* -initial_padding.
*/
int initial_padding;
/**
* - decoding: For codecs that store a framerate value in the compressed
* bitstream, the decoder may export it here. { 0, 1} when
* unknown.
* - encoding: May be used to signal the framerate of CFR content to an
* encoder.
*/
AVRational framerate;
/**
* Nominal unaccelerated pixel format, see AV_PIX_FMT_xxx.
* - encoding: unused.
* - decoding: Set by libavcodec before calling get_format()
*/
enum AVPixelFormat sw_pix_fmt;
/**
* Timebase in which pkt_dts/pts and AVPacket.dts/pts are.
* - encoding unused.
* - decoding set by user.
*/
AVRational pkt_timebase;
/**
* AVCodecDescriptor
* - encoding: unused.
* - decoding: set by libavcodec.
*/
const AVCodecDescriptor *codec_descriptor;
/**
* Current statistics for PTS correction.
* - decoding: maintained and used by libavcodec, not intended to be used by user apps
* - encoding: unused
*/
int64_t pts_correction_num_faulty_pts; /// Number of incorrect PTS values so far
int64_t pts_correction_num_faulty_dts; /// Number of incorrect DTS values so far
int64_t pts_correction_last_pts; /// PTS of the last frame
int64_t pts_correction_last_dts; /// DTS of the last frame
/**
* Character encoding of the input subtitles file.
* - decoding: set by user
* - encoding: unused
*/
char *sub_charenc;
/**
* Subtitles character encoding mode. Formats or codecs might be adjusting
* this setting (if they are doing the conversion themselves for instance).
* - decoding: set by libavcodec
* - encoding: unused
*/
int sub_charenc_mode;
#define FF_SUB_CHARENC_MODE_DO_NOTHING -1 ///< do nothing (demuxer outputs a stream supposed to be already in UTF-8, or the codec is bitmap for instance)
#define FF_SUB_CHARENC_MODE_AUTOMATIC 0 ///< libavcodec will select the mode itself
#define FF_SUB_CHARENC_MODE_PRE_DECODER 1 ///< the AVPacket data needs to be recoded to UTF-8 before being fed to the decoder, requires iconv
#define FF_SUB_CHARENC_MODE_IGNORE 2 ///< neither convert the subtitles, nor check them for valid UTF-8
/**
* Skip processing alpha if supported by codec.
* Note that if the format uses pre-multiplied alpha (common with VP6,
* and recommended due to better video quality/compression)
* the image will look as if alpha-blended onto a black background.
* However for formats that do not use pre-multiplied alpha
* there might be serious artefacts (though e.g. libswscale currently
* assumes pre-multiplied alpha anyway).
*
* - decoding: set by user
* - encoding: unused
*/
int skip_alpha;
/**
* Number of samples to skip after a discontinuity
* - decoding: unused
* - encoding: set by libavcodec
*/
int seek_preroll;
#if FF_API_DEBUG_MV
/**
* @deprecated unused
*/
attribute_deprecated
int debug_mv;
#define FF_DEBUG_VIS_MV_P_FOR 0x00000001 //visualize forward predicted MVs of P frames
#define FF_DEBUG_VIS_MV_B_FOR 0x00000002 //visualize forward predicted MVs of B frames
#define FF_DEBUG_VIS_MV_B_BACK 0x00000004 //visualize backward predicted MVs of B frames
#endif
/**
* custom intra quantization matrix
* - encoding: Set by user, can be NULL.
* - decoding: unused.
*/
uint16_t *chroma_intra_matrix;
/**
* dump format separator.
* can be ", " or "\n " or anything else
* - encoding: Set by user.
* - decoding: Set by user.
*/
uint8_t *dump_separator;
/**
* ',' separated list of allowed decoders.
* If NULL then all are allowed
* - encoding: unused
* - decoding: set by user
*/
char *codec_whitelist;
/**
* Properties of the stream that gets decoded
* - encoding: unused
* - decoding: set by libavcodec
*/
unsigned properties;
#define FF_CODEC_PROPERTY_LOSSLESS 0x00000001
#define FF_CODEC_PROPERTY_CLOSED_CAPTIONS 0x00000002
/**
* Additional data associated with the entire coded stream.
*
* - decoding: unused
* - encoding: may be set by libavcodec after avcodec_open2().
*/
AVPacketSideData *coded_side_data;
int nb_coded_side_data;
/**
* A reference to the AVHWFramesContext describing the input (for encoding)
* or output (decoding) frames. The reference is set by the caller and
* afterwards owned (and freed) by libavcodec - it should never be read by
* the caller after being set.
*
* - decoding: This field should be set by the caller from the get_format()
* callback. The previous reference (if any) will always be
* unreffed by libavcodec before the get_format() call.
*
* If the default get_buffer2() is used with a hwaccel pixel
* format, then this AVHWFramesContext will be used for
* allocating the frame buffers.
*
* - encoding: For hardware encoders configured to use a hwaccel pixel
* format, this field should be set by the caller to a reference
* to the AVHWFramesContext describing input frames.
* AVHWFramesContext.format must be equal to
* AVCodecContext.pix_fmt.
*
* This field should be set before avcodec_open2() is called.
*/
AVBufferRef *hw_frames_ctx;
/**
* Control the form of AVSubtitle.rects[N]->ass
* - decoding: set by user
* - encoding: unused
*/
int sub_text_format;
#define FF_SUB_TEXT_FMT_ASS 0
#if FF_API_ASS_TIMING
#define FF_SUB_TEXT_FMT_ASS_WITH_TIMINGS 1
#endif
/**
* Audio only. The amount of padding (in samples) appended by the encoder to
* the end of the audio. I.e. this number of decoded samples must be
* discarded by the caller from the end of the stream to get the original
* audio without any trailing padding.
*
* - decoding: unused
* - encoding: unused
*/
int trailing_padding;
/**
* The number of pixels per image to maximally accept.
*
* - decoding: set by user
* - encoding: set by user
*/
int64_t max_pixels;
/**
* A reference to the AVHWDeviceContext describing the device which will
* be used by a hardware encoder/decoder. The reference is set by the
* caller and afterwards owned (and freed) by libavcodec.
*
* This should be used if either the codec device does not require
* hardware frames or any that are used are to be allocated internally by
* libavcodec. If the user wishes to supply any of the frames used as
* encoder input or decoder output then hw_frames_ctx should be used
* instead. When hw_frames_ctx is set in get_format() for a decoder, this
* field will be ignored while decoding the associated stream segment, but
* may again be used on a following one after another get_format() call.
*
* For both encoders and decoders this field should be set before
* avcodec_open2() is called and must not be written to thereafter.
*
* Note that some decoders may require this field to be set initially in
* order to support hw_frames_ctx at all - in that case, all frames
* contexts used must be created on the same device.
*/
AVBufferRef *hw_device_ctx;
/**
* Bit set of AV_HWACCEL_FLAG_* flags, which affect hardware accelerated
* decoding (if active).
* - encoding: unused
* - decoding: Set by user (either before avcodec_open2(), or in the
* AVCodecContext.get_format callback)
*/
int hwaccel_flags;
/**
* Video decoding only. Certain video codecs support cropping, meaning that
* only a sub-rectangle of the decoded frame is intended for display. This
* option controls how cropping is handled by libavcodec.
*
* When set to 1 (the default), libavcodec will apply cropping internally.
* I.e. it will modify the output frame width/height fields and offset the
* data pointers (only by as much as possible while preserving alignment, or
* by the full amount if the AV_CODEC_FLAG_UNALIGNED flag is set) so that
* the frames output by the decoder refer only to the cropped area. The
* crop_* fields of the output frames will be zero.
*
* When set to 0, the width/height fields of the output frames will be set
* to the coded dimensions and the crop_* fields will describe the cropping
* rectangle. Applying the cropping is left to the caller.
*
* @warning When hardware acceleration with opaque output frames is used,
* libavcodec is unable to apply cropping from the top/left border.
*
* @note when this option is set to zero, the width/height fields of the
* AVCodecContext and output AVFrames have different meanings. The codec
* context fields store display dimensions (with the coded dimensions in
* coded_width/height), while the frame fields store the coded dimensions
* (with the display dimensions being determined by the crop_* fields).
*/
int apply_cropping;
/*
* Video decoding only. Sets the number of extra hardware frames which
* the decoder will allocate for use by the caller. This must be set
* before avcodec_open2() is called.
*
* Some hardware decoders require all frames that they will use for
* output to be defined in advance before decoding starts. For such
* decoders, the hardware frame pool must therefore be of a fixed size.
* The extra frames set here are on top of any number that the decoder
* needs internally in order to operate normally (for example, frames
* used as reference pictures).
*/
int extra_hw_frames;
/**
* The percentage of damaged samples to discard a frame.
*
* - decoding: set by user
* - encoding: unused
*/
int discard_damaged_percentage;
/**
* The number of samples per frame to maximally accept.
*
* - decoding: set by user
* - encoding: set by user
*/
int64_t max_samples;
/**
* Bit set of AV_CODEC_EXPORT_DATA_* flags, which affects the kind of
* metadata exported in frame, packet, or coded stream side data by
* decoders and encoders.
*
* - decoding: set by user
* - encoding: set by user
*/
int export_side_data;
/**
* This callback is called at the beginning of each packet to get a data
* buffer for it.
*
* The following field will be set in the packet before this callback is
* called:
* - size
* This callback must use the above value to calculate the required buffer size,
* which must padded by at least AV_INPUT_BUFFER_PADDING_SIZE bytes.
*
* This callback must fill the following fields in the packet:
* - data: alignment requirements for AVPacket apply, if any. Some architectures and
* encoders may benefit from having aligned data.
* - buf: must contain a pointer to an AVBufferRef structure. The packet's
* data pointer must be contained in it. See: av_buffer_create(), av_buffer_alloc(),
* and av_buffer_ref().
*
* If AV_CODEC_CAP_DR1 is not set then get_encode_buffer() must call
* avcodec_default_get_encode_buffer() instead of providing a buffer allocated by
* some other means.
*
* The flags field may contain a combination of AV_GET_ENCODE_BUFFER_FLAG_ flags.
* They may be used for example to hint what use the buffer may get after being
* created.
* Implementations of this callback may ignore flags they don't understand.
* If AV_GET_ENCODE_BUFFER_FLAG_REF is set in flags then the packet may be reused
* (read and/or written to if it is writable) later by libavcodec.
*
* This callback must be thread-safe, as when frame threading is used, it may
* be called from multiple threads simultaneously.
*
* @see avcodec_default_get_encode_buffer()
*
* - encoding: Set by libavcodec, user can override.
* - decoding: unused
*/
int (*get_encode_buffer)(struct AVCodecContext *s, AVPacket *pkt, int flags);
} AVCodecContext;
3.5 AVCodec
每种视频(音频)解码器都对应一个该结构体。该结构体定义位于codec.h，其详细结构如下:
/**
* AVCodec.
*/
typedef struct AVCodec {
/**
* Name of the codec implementation.
* The name is globally unique among encoders and among decoders (but an
* encoder and a decoder can share the same name).
* This is the primary way to find a codec from the user perspective.
*/
// 编解码器名称
const char *name;
/**
* Descriptive name for the codec, meant to be more human readable than name.
* You should use the NULL_IF_CONFIG_SMALL() macro to define it.
*/
const char *long_name;
// 编解码器类型
enum AVMediaType type;
//编解码器ID
enum AVCodecID id;
/**
* Codec capabilities.
* see AV_CODEC_CAP_*
*/
int capabilities;
const AVRational *supported_framerates; ///< array of supported framerates, or NULL if any, array is terminated by {0,0}
const enum AVPixelFormat *pix_fmts; ///< array of supported pixel formats, or NULL if unknown, array is terminated by -1
const int *supported_samplerates; ///< array of supported audio samplerates, or NULL if unknown, array is terminated by 0
const enum AVSampleFormat *sample_fmts; ///< array of supported sample formats, or NULL if unknown, array is terminated by -1
const uint64_t *channel_layouts; ///< array of support channel layouts, or NULL if unknown. array is terminated by 0
uint8_t max_lowres; ///< maximum value for lowres supported by the decoder
const AVClass *priv_class; ///< AVClass for the private context
const AVProfile *profiles; ///< array of recognized profiles, or NULL if unknown, array is terminated by {FF_PROFILE_UNKNOWN}
/**
* Group name of the codec implementation.
* This is a short symbolic name of the wrapper backing this codec. A
* wrapper uses some kind of external implementation for the codec, such
* as an external library, or a codec implementation provided by the OS or
* the hardware.
* If this field is NULL, this is a builtin, libavcodec native codec.
* If non-NULL, this will be the suffix in AVCodec.name in most cases
* (usually AVCodec.name will be of the form "<codec_name>_<wrapper_name>").
*/
const char *wrapper_name;
/*****************************************************************
* No fields below this line are part of the public API. They
* may not be used outside of libavcodec and can be changed and
* removed at will.
* New public fields should be added right above.
*****************************************************************
*/
int priv_data_size;
#if FF_API_NEXT
struct AVCodec *next;
#endif
/**
* @name Frame-level threading support functions
* @{
*/
/**
* Copy necessary context variables from a previous thread context to the current one.
* If not defined, the next thread will start automatically; otherwise, the codec
* must call ff_thread_finish_setup().
*
* dst and src will (rarely) point to the same context, in which case memcpy should be skipped.
*/
int (*update_thread_context)(struct AVCodecContext *dst, const struct AVCodecContext *src);
/** @} */
/**
* Private codec-specific defaults.
*/
const AVCodecDefault *defaults;
/**
* Initialize codec static data, called from av_codec_iterate().
*
* This is not intended for time consuming operations as it is
* run for every codec regardless of that codec being used.
*/
void (*init_static_data)(struct AVCodec *codec);
int (*init)(struct AVCodecContext *);
int (*encode_sub)(struct AVCodecContext *, uint8_t *buf, int buf_size,
const struct AVSubtitle *sub);
/**
* Encode data to an AVPacket.
*
* @param avctx codec context
* @param avpkt output AVPacket
* @param[in] frame AVFrame containing the raw data to be encoded
* @param[out] got_packet_ptr encoder sets to 0 or 1 to indicate that a
* non-empty packet was returned in avpkt.
* @return 0 on success, negative error code on failure
*/
int (*encode2)(struct AVCodecContext *avctx, struct AVPacket *avpkt,
const struct AVFrame *frame, int *got_packet_ptr);
/**
* Decode picture or subtitle data.
*
* @param avctx codec context
* @param outdata codec type dependent output struct
* @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a
* non-empty frame or subtitle was returned in
* outdata.
* @param[in] avpkt AVPacket containing the data to be decoded
* @return amount of bytes read from the packet on success, negative error
* code on failure
*/
int (*decode)(struct AVCodecContext *avctx, void *outdata,
int *got_frame_ptr, struct AVPacket *avpkt);
int (*close)(struct AVCodecContext *);
/**
* Encode API with decoupled frame/packet dataflow. This function is called
* to get one output packet. It should call ff_encode_get_frame() to obtain
* input data.
*/
int (*receive_packet)(struct AVCodecContext *avctx, struct AVPacket *avpkt);
/**
* Decode API with decoupled packet/frame dataflow. This function is called
* to get one output frame. It should call ff_decode_get_packet() to obtain
* input data.
*/
int (*receive_frame)(struct AVCodecContext *avctx, struct AVFrame *frame);
/**
* Flush buffers.
* Will be called when seeking
*/
void (*flush)(struct AVCodecContext *);
/**
* Internal codec capabilities.
* See FF_CODEC_CAP_* in internal.h
*/
int caps_internal;
/**
* Decoding only, a comma-separated list of bitstream filters to apply to
* packets before decoding.
*/
const char *bsfs;
/**
* Array of pointers to hardware configurations supported by the codec,
* or NULL if no hardware supported. The array is terminated by a NULL
* pointer.
*
* The user can only access this field via avcodec_get_hw_config().
*/
const struct AVCodecHWConfigInternal *const *hw_configs;
/**
* List of supported codec_tags, terminated by FF_CODEC_TAGS_END.
*/
const uint32_t *codec_tags;
} AVCodec;
3.6 AVPacket
存储一帧压缩编码数据。该结构体定义位于libavcodec/packet.h，其详细结构如下:
/**
* This structure stores compressed data. It is typically exported by demuxers
* and then passed as input to decoders, or received as output from encoders and
* then passed to muxers.
*
* For video, it should typically contain one compressed frame. For audio it may
* contain several compressed frames. Encoders are allowed to output empty
* packets, with no compressed data, containing only side data
* (e.g. to update some stream parameters at the end of encoding).
*
* The semantics of data ownership depends on the buf field.
* If it is set, the packet data is dynamically allocated and is
* valid indefinitely until a call to av_packet_unref() reduces the
* reference count to 0.
*
* If the buf field is not set av_packet_ref() would make a copy instead
* of increasing the reference count.
*
* The side data is always allocated with av_malloc(), copied by
* av_packet_ref() and freed by av_packet_unref().
*
* sizeof(AVPacket) being a part of the public ABI is deprecated. once
* av_init_packet() is removed, new packets will only be able to be allocated
* with av_packet_alloc(), and new fields may be added to the end of the struct
* with a minor bump.
*
* @see av_packet_alloc
* @see av_packet_ref
* @see av_packet_unref
*/
typedef struct AVPacket {
/**
* A reference to the reference-counted buffer where the packet data is
* stored.
* May be NULL, then the packet data is not reference-counted.
*/
AVBufferRef *buf;
/**
* Presentation timestamp in AVStream->time_base units; the time at which
* the decompressed packet will be presented to the user.
* Can be AV_NOPTS_VALUE if it is not stored in the file.
* pts MUST be larger or equal to dts as presentation cannot happen before
* decompression, unless one wants to view hex dumps. Some formats misuse
* the terms dts and pts/cts to mean something different. Such timestamps
* must be converted to true pts/dts before they are stored in AVPacket.
*/
// 显示时间戳
int64_t pts;
/**
* Decompression timestamp in AVStream->time_base units; the time at which
* the packet is decompressed.
* Can be AV_NOPTS_VALUE if it is not stored in the file.
*/
// 解码时间戳
int64_t dts;
//压缩编码数据
uint8_t *data;
//压缩编码数据大小
int size;
//所属的AVStream
int stream_index;
/**
* A combination of AV_PKT_FLAG values
*/
int flags;
/**
* Additional packet data that can be provided by the container.
* Packet can contain several types of side information.
*/
AVPacketSideData *side_data;
int side_data_elems;
/**
* Duration of this packet in AVStream->time_base units, 0 if unknown.
* Equals next_pts - this_pts in presentation order.
*/
int64_t duration;
// 数据的偏移地址
int64_t pos; ///< byte position in stream, -1 if unknown
#if FF_API_CONVERGENCE_DURATION
/**
* @deprecated Same as the duration field, but as int64_t. This was required
* for Matroska subtitles, whose duration values could overflow when the
* duration field was still an int.
*/
attribute_deprecated
int64_t convergence_duration;
#endif
} AVPacket;
3.7 AVFrame
存储一帧解码后像素(采样)数据。该结构体定义位于libavutil/frame.h，其详细结构如下:
/**
* This structure describes decoded (raw) audio or video data.
*
* AVFrame must be allocated using av_frame_alloc(). Note that this only
* allocates the AVFrame itself, the buffers for the data must be managed
* through other means (see below).
* AVFrame must be freed with av_frame_free().
*
* AVFrame is typically allocated once and then reused multiple times to hold
* different data (e.g. a single AVFrame to hold frames received from a
* decoder). In such a case, av_frame_unref() will free any references held by
* the frame and reset it to its original clean state before it
* is reused again.
*
* The data described by an AVFrame is usually reference counted through the
* AVBuffer API. The underlying buffer references are stored in AVFrame.buf /
* AVFrame.extended_buf. An AVFrame is considered to be reference counted if at
* least one reference is set, i.e. if AVFrame.buf[0] != NULL. In such a case,
* every single data plane must be contained in one of the buffers in
* AVFrame.buf or AVFrame.extended_buf.
* There may be a single buffer for all the data, or one separate buffer for
* each plane, or anything in between.
*
* sizeof(AVFrame) is not a part of the public ABI, so new fields may be added
* to the end with a minor bump.
*
* Fields can be accessed through AVOptions, the name string used, matches the
* C structure field name for fields accessible through AVOptions. The AVClass
* for AVFrame can be obtained from avcodec_get_frame_class()
*/
typedef struct AVFrame {
#define AV_NUM_DATA_POINTERS 8
/**
* pointer to the picture/channel planes.
* This might be different from the first allocated byte
*
* Some decoders access areas outside 0,0 - width,height, please
* see avcodec_align_dimensions2(). Some filters and swscale can read
* up to 16 bytes beyond the planes, if these filters are to be used,
* then 16 extra bytes must be allocated.
*
* NOTE: Except for hwaccel formats, pointers not needed by the format
* MUST be set to NULL.
*/
// 解码后的图像像素数据（音频采样数据）
uint8_t *data[AV_NUM_DATA_POINTERS];
/**
* For video, size in bytes of each picture line.
* For audio, size in bytes of each plane.
*
* For audio, only linesize[0] may be set. For planar audio, each channel
* plane must be the same size.
*
* For video the linesizes should be multiples of the CPUs alignment
* preference, this is 16 or 32 for modern desktop CPUs.
* Some code requires such alignment other code can be slower without
* correct alignment, for yet other it makes no difference.
*
* @note The linesize may be larger than the size of usable data -- there
* may be extra padding present for performance reasons.
*/
// 对视频来说是图像中一行像素的大小，对音频来说是整个音频帧的大小
int linesize[AV_NUM_DATA_POINTERS];
/**
* pointers to the data planes/channels.
*
* For video, this should simply point to data[].
*
* For planar audio, each channel has a separate data pointer, and
* linesize[0] contains the size of each channel buffer.
* For packed audio, there is just one data pointer, and linesize[0]
* contains the total size of the buffer for all channels.
*
* Note: Both data and extended_data should always be set in a valid frame,
* but for planar audio with more channels that can fit in data,
* extended_data must be used in order to access all channels.
*/
uint8_t **extended_data;
/**
* @name Video dimensions
* Video frames only. The coded dimensions (in pixels) of the video frame,
* i.e. the size of the rectangle that contains some well-defined values.
*
* @note The part of the frame intended for display/presentation is further
* restricted by the @ref cropping "Cropping rectangle".
* @{
*/
//图像的宽高(只针对视频)
int width, height;
/**
* @}
*/
/**
* number of audio samples (per channel) described by this frame
*/
// 音频每通道采样数
int nb_samples;
/**
* format of the frame, -1 if unknown or unset
* Values correspond to enum AVPixelFormat for video frames,
* enum AVSampleFormat for audio)
*/
int format;
/**
* 1 -> keyframe, 0-> not
*/
// 是否是关键帧
int key_frame;
/**
* Picture type of the frame.
*/
//帧类型，只针对视频
enum AVPictureType pict_type;
/**
* Sample aspect ratio for the video frame, 0/1 if unknown/unspecified.
*/
AVRational sample_aspect_ratio;
/**
* Presentation timestamp in time_base units (time when frame should be shown to user).
*/
// 显示时间戳
int64_t pts;
#if FF_API_PKT_PTS
/**
* PTS copied from the AVPacket that was decoded to produce this frame.
* @deprecated use the pts field instead
*/
attribute_deprecated
int64_t pkt_pts;
#endif
/**
* DTS copied from the AVPacket that triggered returning this frame. (if frame threading isn't used)
* This is also the Presentation time of this AVFrame calculated from
* only AVPacket.dts values without pts values.
*/
int64_t pkt_dts;
/**
* picture number in bitstream order
*/
int coded_picture_number;
/**
* picture number in display order
*/
int display_picture_number;
/**
* quality (between 1 (good) and FF_LAMBDA_MAX (bad))
*/
int quality;
/**
* for some private data of the user
*/
void *opaque;
#if FF_API_ERROR_FRAME
/**
* @deprecated unused
*/
attribute_deprecated
uint64_t error[AV_NUM_DATA_POINTERS];
#endif
/**
* When decoding, this signals how much the picture must be delayed.
* extra_delay = repeat_pict / (2*fps)
*/
int repeat_pict;
/**
* The content of the picture is interlaced.
*/
int interlaced_frame;
/**
* If the content is interlaced, is top field displayed first.
*/
int top_field_first;
/**
* Tell user application that palette has changed from previous frame.
*/
int palette_has_changed;
/**
* reordered opaque 64 bits (generally an integer or a double precision float
* PTS but can be anything).
* The user sets AVCodecContext.reordered_opaque to represent the input at
* that time,
* the decoder reorders values as needed and sets AVFrame.reordered_opaque
* to exactly one of the values provided by the user through AVCodecContext.reordered_opaque
*/
int64_t reordered_opaque;
/**
* Sample rate of the audio data.
*/
//音频采样率
int sample_rate;
/**
* Channel layout of the audio data.
*/
uint64_t channel_layout;
/**
* AVBuffer references backing the data for this frame. If all elements of
* this array are NULL, then this frame is not reference counted. This array
* must be filled contiguously -- if buf[i] is non-NULL then buf[j] must
* also be non-NULL for all j < i.
*
* There may be at most one AVBuffer per data plane, so for video this array
* always contains all the references. For planar audio with more than
* AV_NUM_DATA_POINTERS channels, there may be more buffers than can fit in
* this array. Then the extra AVBufferRef pointers are stored in the
* extended_buf array.
*/
AVBufferRef *buf[AV_NUM_DATA_POINTERS];
/**
* For planar audio which requires more than AV_NUM_DATA_POINTERS
* AVBufferRef pointers, this array will hold all the references which
* cannot fit into AVFrame.buf.
*
* Note that this is different from AVFrame.extended_data, which always
* contains all the pointers. This array only contains the extra pointers,
* which cannot fit into AVFrame.buf.
*
* This array is always allocated using av_malloc() by whoever constructs
* the frame. It is freed in av_frame_unref().
*/
AVBufferRef **extended_buf;
/**
* Number of elements in extended_buf.
*/
int nb_extended_buf;
AVFrameSideData **side_data;
int nb_side_data;
/**
* @defgroup lavu_frame_flags AV_FRAME_FLAGS
* @ingroup lavu_frame
* Flags describing additional frame properties.
*
* @{
*/
/**
* The frame data may be corrupted, e.g. due to decoding errors.
*/
#define AV_FRAME_FLAG_CORRUPT (1 << 0)
/**
* A flag to mark the frames which need to be decoded, but shouldn't be output.
*/
#define AV_FRAME_FLAG_DISCARD (1 << 2)
/**
* @}
*/
/**
* Frame flags, a combination of @ref lavu_frame_flags
*/
int flags;
/**
* MPEG vs JPEG YUV range.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorRange color_range;
enum AVColorPrimaries color_primaries;
enum AVColorTransferCharacteristic color_trc;
/**
* YUV colorspace type.
* - encoding: Set by user
* - decoding: Set by libavcodec
*/
enum AVColorSpace colorspace;
enum AVChromaLocation chroma_location;
/**
* frame timestamp estimated using various heuristics, in stream time base
* - encoding: unused
* - decoding: set by libavcodec, read by user.
*/
int64_t best_effort_timestamp;
/**
* reordered pos from the last AVPacket that has been input into the decoder
* - encoding: unused
* - decoding: Read by user.
*/
int64_t pkt_pos;
/**
* duration of the corresponding packet, expressed in
* AVStream->time_base units, 0 if unknown.
* - encoding: unused
* - decoding: Read by user.
*/
int64_t pkt_duration;
/**
* metadata.
* - encoding: Set by user.
* - decoding: Set by libavcodec.
*/
AVDictionary *metadata;
/**
* decode error flags of the frame, set to a combination of
* FF_DECODE_ERROR_xxx flags if the decoder produced a frame, but there
* were errors during the decoding.
* - encoding: unused
* - decoding: set by libavcodec, read by user.
*/
int decode_error_flags;
#define FF_DECODE_ERROR_INVALID_BITSTREAM 1
#define FF_DECODE_ERROR_MISSING_REFERENCE 2
#define FF_DECODE_ERROR_CONCEALMENT_ACTIVE 4
#define FF_DECODE_ERROR_DECODE_SLICES 8
/**
* number of audio channels, only used for audio.
* - encoding: unused
* - decoding: Read by user.
*/
int channels;
/**
* size of the corresponding packet containing the compressed
* frame.
* It is set to a negative value if unknown.
* - encoding: unused
* - decoding: set by libavcodec, read by user.
*/
int pkt_size;
#if FF_API_FRAME_QP
/**
* QP table
*/
attribute_deprecated
int8_t *qscale_table;
/**
* QP store stride
*/
attribute_deprecated
int qstride;
attribute_deprecated
int qscale_type;
attribute_deprecated
AVBufferRef *qp_table_buf;
#endif
/**
* For hwaccel-format frames, this should be a reference to the
* AVHWFramesContext describing the frame.
*/
AVBufferRef *hw_frames_ctx;
/**
* AVBufferRef for free use by the API user. FFmpeg will never check the
* contents of the buffer ref. FFmpeg calls av_buffer_unref() on it when
* the frame is unreferenced. av_frame_copy_props() calls create a new
* reference with av_buffer_ref() for the target frame's opaque_ref field.
*
* This is unrelated to the opaque field, although it serves a similar
* purpose.
*/
AVBufferRef *opaque_ref;
/**
* @anchor cropping
* @name Cropping
* Video frames only. The number of pixels to discard from the the
* top/bottom/left/right border of the frame to obtain the sub-rectangle of
* the frame intended for presentation.
* @{
*/
size_t crop_top;
size_t crop_bottom;
size_t crop_left;
size_t crop_right;
/**
* @}
*/
/**
* AVBufferRef for internal use by a single libav* library.
* Must not be used to transfer data between libraries.
* Has to be NULL when ownership of the frame leaves the respective library.
*
* Code outside the FFmpeg libs should never check or change the contents of the buffer ref.
*
* FFmpeg calls av_buffer_unref() on it when the frame is unreferenced.
* av_frame_copy_props() calls create a new reference with av_buffer_ref()
* for the target frame's private_ref field.
*/
AVBufferRef *private_ref;
} AVFrame;
3.8 AVCodecParameters
该结构体定义位于libavcodec/codec_par.h，其详细结构如下:
/**
* This struct describes the properties of an encoded stream.
*
* sizeof(AVCodecParameters) is not a part of the public ABI, this struct must
* be allocated with avcodec_parameters_alloc() and freed with
* avcodec_parameters_free().
*/
typedef struct AVCodecParameters {
/**
* General type of the encoded data.
*/
//媒体类型，比如AVMEDIA_TYPE_VIDEO、AVMEDIA_TYPE_AUDIO等。
enum AVMediaType codec_type;
/**
* Specific type of the encoded data (the codec used).
*/
//编解码器类型，比如AV_CODE_ID_H264、AV_CODEC_ID_AAC等。
enum AVCodecID codec_id;
/**
* Additional information about the codec (corresponds to the AVI FOURCC).
*/
uint32_t codec_tag;
/**
* Extra binary data needed for initializing the decoder, codec-dependent.
*
* Must be allocated with av_malloc() and will be freed by
* avcodec_parameters_free(). The allocated size of extradata must be at
* least extradata_size + AV_INPUT_BUFFER_PADDING_SIZE, with the padding
* bytes zeroed.
*/
uint8_t *extradata;
/**
* Size of the extradata content in bytes.
*/
int extradata_size;
/**
* - video: the pixel format, the value corresponds to enum AVPixelFormat.
* - audio: the sample format, the value corresponds to enum AVSampleFormat.
*/
int format;
/**
* The average bitrate of the encoded data (in bits per second).
*/
int64_t bit_rate;
/**
* The number of bits per sample in the codedwords.
*
* This is basically the bitrate per sample. It is mandatory for a bunch of
* formats to actually decode them. It's the number of bits for one sample in
* the actual coded bitstream.
*
* This could be for example 4 for ADPCM
* For PCM formats this matches bits_per_raw_sample
* Can be 0
*/
int bits_per_coded_sample;
/**
* This is the number of valid bits in each output sample. If the
* sample format has more bits, the least significant bits are additional
* padding bits, which are always 0. Use right shifts to reduce the sample
* to its actual size. For example, audio formats with 24 bit samples will
* have bits_per_raw_sample set to 24, and format set to AV_SAMPLE_FMT_S32.
* To get the original sample use "(int32_t)sample >> 8"."
*
* For ADPCM this might be 12 or 16 or similar
* Can be 0
*/
int bits_per_raw_sample;
/**
* Codec-specific bitstream restrictions that the stream conforms to.
*/
int profile;
int level;
/**
* Video only. The dimensions of the video frame in pixels.
*/
int width;
int height;
/**
* Video only. The aspect ratio (width / height) which a single pixel
* should have when displayed.
*
* When the aspect ratio is unknown / undefined, the numerator should be
* set to 0 (the denominator may have any value).
*/
AVRational sample_aspect_ratio;
/**
* Video only. The order of the fields in interlaced video.
*/
enum AVFieldOrder field_order;
/**
* Video only. Additional colorspace characteristics.
*/
enum AVColorRange color_range;
enum AVColorPrimaries color_primaries;
enum AVColorTransferCharacteristic color_trc;
enum AVColorSpace color_space;
enum AVChromaLocation chroma_location;
/**
* Video only. Number of delayed frames.
*/
int video_delay;
/**
* Audio only. The channel layout bitmask. May be 0 if the channel layout is
* unknown or unspecified, otherwise the number of bits set must be equal to
* the channels field.
*/
uint64_t channel_layout;
/**
* Audio only. The number of audio channels.
*/
int channels;
/**
* Audio only. The number of audio samples per second.
*/
int sample_rate;
/**
* Audio only. The number of bytes per coded audio frame, required by some
* formats.
*
* Corresponds to nBlockAlign in WAVEFORMATEX.
*/
int block_align;
/**
* Audio only. Audio frame size, if known. Required by some formats to be static.
*/
int frame_size;
/**
* Audio only. The amount of padding (in samples) inserted by the encoder at
* the beginning of the audio. I.e. this number of leading decoded samples
* must be discarded by the caller to get the original audio without leading
* padding.
*/
int initial_padding;
/**
* Audio only. The amount of padding (in samples) appended by the encoder to
* the end of the audio. I.e. this number of decoded samples must be
* discarded by the caller from the end of the stream to get the original
* audio without any trailing padding.
*/
int trailing_padding;
/**
* Audio only. Number of samples to skip after a discontinuity.
*/
int seek_preroll;
} AVCodecParameters;
四、ffmpeg相关数据结构的关系
4.1 AVFormatContext和AVInputFormat之间的关系
AVFormatContext API调用
AVInputFormat主要是ffmpeg内部调用
AVFormatContext封装格式上下文结构体
AVFormatContext 封装格式上下文结构体
struct AVInputFormat *iformat;
方法
AVInputFormat 每种封装格式（例如FLV, MKV, MP4）
int (*read_header)(struct AVFormatContext * );
int (*read_packet)(struct AVFormatContext *, AVPacket *pkt);
int avformat_open_input(AVFormatContext **ps, const char *filename,
AVInputFormat *fmt, AVDictionary **options)
4.2 AVCodecContext和AVCodec之间的关系
AVCodecContext 编码器上下文结构体
struct AVCodec *codec;
AVCodec 每种视频（音频）编解码器
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,
AVPacket *avpkt);
int (*encode2)(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame
*frame, int *got_packet_ptr);
4.3 AVFormatContext, AVStream和AVCodecContext之间的关系
4.4 区分不同的码流
AVMEDIA_TYPE_VIDEO视频流
video_index = av_find_best_stream(ic, AVMEDIA_TYPE_VIDEO, -1,-1, NULL, 0)
AVMEDIA_TYPE_AUDIO音频流
audio_index = av_find_best_stream(ic, AVMEDIA_TYPE_AUDIO, -1,-1, NULL, 0)
4.5AVFrame和AVPacket之间的关系
AVPacket（解码）->AVFrame
AVFrame（编码）->AVPacket
参考资料
Linux，Nginx，ZeroMQ，MySQL，Redis，fastdfs，MongoDB，ZK，流媒体，CDN，P2P，K8S，Docker，TCP/IP，协程，DPDK等技术内容，立即学习
1637
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
