目录
FFmpeg之sws_getContext函数分析
主要参考FFmpeg源代码简单分析：libswscale的sws_getContext()
FFmpeg里面的sws_scale库可以在一个函数里面同时实现：1.图像色彩空间转换；2.分辨率缩放；3.前后图像滤波处理。
libswscale常用的函数数量很少，一般情况下就3个：
sws_getContext()：初始化一个SwsContext。
sws_scale()：处理图像数据。
sws_freeContext()：释放一个SwsContext。
其中sws_getContext()也可以用sws_getCachedContext()取代。
sws_getContext函数
sws_getContext函数包含以下参数：
srcW：源图像的宽
srcH：源图像的高
srcFormat：源图像的像素格式
dstW：目标图像的宽
dstH：目标图像的高
dstFormat：目标图像的像素格式
flags：设定图像拉伸使用的算法
尽管libswscale从表面上看常用函数的个数不多，它的内部却有一个大大的“世界”。做为一个几乎“万能”的图片像素数据处理类库，它的内部包含了大量的代码。因此计划写两篇文章分析它的源代码。本文首先分析它的初始化函数sws_getContext()，而下一篇文章则分析它的数据处理函数sws_scale()。
/**
* Allocate and return an SwsContext. You need it to perform
* scaling/conversion operations using sws_scale().
*
* @param srcW the width of the source image
* @param srcH the height of the source image
* @param srcFormat the source image format
* @param dstW the width of the destination image
* @param dstH the height of the destination image
* @param dstFormat the destination image format
* @param flags specify which algorithm and options to use for rescaling
* @param param extra parameters to tune the used scaler
* For SWS_BICUBIC param[0] and [1] tune the shape of the basis
* function, param[0] tunes f(1) and param[1] f´(1)
* For SWS_GAUSS param[0] tunes the exponent and thus cutoff
* frequency
* For SWS_LANCZOS param[0] tunes the width of the window function
* @return a pointer to an allocated context, or NULL in case of error
* @note this function is to be removed after a saner alternative is
* written
*/
struct SwsContext *sws_getContext(int srcW, int srcH, enum AVPixelFormat srcFormat,
int dstW, int dstH, enum AVPixelFormat dstFormat,
int flags, SwsFilter *srcFilter,
SwsFilter *dstFilter, const double *param);
10个参数
sws_scale函数
/**
* Scale the image slice in srcSlice and put the resulting scaled
* slice in the image in dst. A slice is a sequence of consecutive
* rows in an image.
*
* Slices have to be provided in sequential order, either in
* top-bottom or bottom-top order. If slices are provided in
* non-sequential order the behavior of the function is undefined.
*
@param c the scaling context previously created with sws_getContext()
先前使用sws_getContext创建的缩放上下文
@param srcSlice the array containing the pointers to the planes of the source slice
包含指向源切片平面的指针的数组
@param srcStride the array containing the strides for each plane of the source image
包含源图像每个平面的跨距的数组
@param srcSliceY the position in the source image of the slice to
process, that is the number (counted starting from
zero) in the image of the first row of the slice
要处理的切片的源图像中的位置，即切片第一行图像中的数字（从零开始计数）
@param srcSliceH the height of the source slice, that is the number
of rows in the slice
源切片的高度，即切片中的行数
@param dst the array containing the pointers to the planes of
the destination image
包含指向目标图像平面的指针的数组
@param dstStride the array containing the strides for each plane of
the destination image
包含目标图像每个平面的跨距的数组
@return the height of the output slice
输出切片的高度
*/
int sws_scale(struct SwsContext *c, const uint8_t *const srcSlice[],
const int srcStride[], int srcSliceY, int srcSliceH,
uint8_t *const dst[], const int dstStride[]);
8个参数
sws_freeContext函数
/**
* Free the swscaler context swsContext. 释放swscaler上下文swsContext
* If swsContext is NULL, then does nothing. 如果swsContext为空，则不执行任何操作
*/
void sws_freeContext(struct SwsContext *swsContext);
使用
#include <iostream>
#include <thread>
extern "C" {
#include "libavformat/avformat.h"
#include "libavcodec/avcodec.h"
#include "libswscale/swscale.h"
#include "libswresample/swresample.h"
}
using namespace std;
#pragma comment(lib,"avformat.lib")
#pragma comment(lib,"avutil.lib")
#pragma comment(lib,"avcodec.lib")
#pragma comment(lib,"swscale.lib")
#pragma comment(lib,"swresample.lib")
static double r2d(AVRational r)
{
return r.den == 0 ? 0 : (double)r.num / (double)r.den;
}
void XSleep(int ms)
{
//c++ 11
chrono::milliseconds du(ms);
this_thread::sleep_for(du);
}
int main(int argc, char* argv[])
{
cout << "Test Demux FFmpeg.club" << endl;
const char* path = "D:\\javaCode\\androidmaniu2022\\FFmpeg\\input.mp4";
//初始化封装库---declared deprecated
//---av_register_all();
//初始化网络库 （可以打开rtsp rtmp http 协议的流媒体视频）
avformat_network_init();
//注册解码器---declared deprecated
//avcodec_register_all();
//参数设置
AVDictionary* opts = NULL;
//设置rtsp流已tcp协议打开
av_dict_set(&opts, "rtsp_transport", "tcp", 0);
//网络延时时间
av_dict_set(&opts, "max_delay", "500", 0);
//解封装上下文
AVFormatContext* ic = NULL;
int re = avformat_open_input(
&ic,
path,
0, // 0表示自动选择解封器
&opts //参数设置，比如rtsp的延时时间
);
if (re != 0)
{
char buf[1024] = { 0 };
av_strerror(re, buf, sizeof(buf) - 1);
cout << "open " << path << " failed! :" << buf << endl;
getchar();
return -1;
}
cout << "open " << path << " success! " << endl;
//获取流信息
re = avformat_find_stream_info(ic, 0);
//总时长 毫秒
int totalMs = ic->duration / (AV_TIME_BASE / 1000);
cout << "totalMs = " << totalMs << endl;
//打印视频流详细信息
av_dump_format(ic, 0, path, 0);
//音视频索引，读取时区分音视频
int videoStream = 0;
int audioStream = 1;
//获取音视频流信息 （遍历，函数获取）
for (int i = 0; i < ic->nb_streams; i++)
{
AVStream* as = ic->streams[i];
cout << "codec_id = " << as->codecpar->codec_id << endl;
cout << "format = " << as->codecpar->format << endl;
//音频 AVMEDIA_TYPE_AUDIO
if (as->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)
{
audioStream = i;
cout << i << "音频信息" << endl;
cout << "sample_rate = " << as->codecpar->sample_rate << endl;
//AVSampleFormat;
cout << "channels = " << as->codecpar->channels << endl;
//一帧数据？？ 单通道样本数
cout << "frame_size = " << as->codecpar->frame_size << endl;
//1024 * 2 * 2 = 4096 fps = sample_rate/frame_size
}
//视频 AVMEDIA_TYPE_VIDEO
else if (as->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
{
videoStream = i;
cout << i << "视频信息" << endl;
cout << "width=" << as->codecpar->width << endl;
cout << "height=" << as->codecpar->height << endl;
//帧率 fps 分数转换
cout << "video fps = " << r2d(as->avg_frame_rate) << endl;
}
}
//获取视频流
videoStream = av_find_best_stream(ic, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
//
///视频解码器打开
///找到视频解码器
AVCodec* vcodec = avcodec_find_decoder(ic->streams[videoStream]->codecpar->codec_id);
if (!vcodec)
{
cout << "can't find the codec id " << ic->streams[videoStream]->codecpar->codec_id;
getchar();
return -1;
}
cout << "find the AVCodec " << ic->streams[videoStream]->codecpar->codec_id << endl;
AVCodecContext* vc = avcodec_alloc_context3(vcodec);
///配置解码器上下文参数
avcodec_parameters_to_context(vc, ic->streams[videoStream]->codecpar);
//八线程解码
vc->thread_count = 8;
///打开解码器上下文
re = avcodec_open2(vc, 0, 0);
if (re != 0)
{
char buf[1024] = { 0 };
av_strerror(re, buf, sizeof(buf) - 1);
cout << "avcodec_open2 failed! :" << buf << endl;
getchar();
return -1;
}
cout << "video avcodec_open2 success!" << endl;
//
///音频解码器打开
AVCodec* acodec = avcodec_find_decoder(ic->streams[audioStream]->codecpar->codec_id);
if (!acodec)
{
cout << "can't find the codec id " << ic->streams[audioStream]->codecpar->codec_id;
getchar();
return -1;
}
cout << "find the AVCodec " << ic->streams[audioStream]->codecpar->codec_id << endl;
///创建解码器上下文呢
AVCodecContext* ac = avcodec_alloc_context3(acodec);
///配置解码器上下文参数
avcodec_parameters_to_context(ac, ic->streams[audioStream]->codecpar);
//八线程解码
ac->thread_count = 8;
///打开解码器上下文
re = avcodec_open2(ac, 0, 0);
if (re != 0)
{
char buf[1024] = { 0 };
av_strerror(re, buf, sizeof(buf) - 1);
cout << "avcodec_open2 failed! :" << buf << endl;
getchar();
return -1;
}
cout << "audio avcodec_open2 success!" << endl;
///ic->streams[videoStream]
//malloc AVPacket并初始化
AVPacket* pkt = av_packet_alloc();
AVFrame* frame = av_frame_alloc();
//像素格式和尺寸转换上下文
SwsContext* vctx = NULL;
unsigned char* rgb = NULL;
//音频重采样 上下文初始化
SwrContext* actx = swr_alloc();
actx = swr_alloc_set_opts(actx,
av_get_default_channel_layout(2), //输出格式
AV_SAMPLE_FMT_S16, //输出样本格式
ac->sample_rate, //输出采样率
av_get_default_channel_layout(ac->channels),//输入格式
ac->sample_fmt,
ac->sample_rate,
0, 0
);
re = swr_init(actx);
if (re != 0)
{
char buf[1024] = { 0 };
av_strerror(re, buf, sizeof(buf) - 1);
cout << "swr_init failed! :" << buf << endl;
getchar();
return -1;
}
unsigned char* pcm = NULL;
for (;;)
{
int re = av_read_frame(ic, pkt);
if (re != 0)
{
//循环播放
cout << "==============================end==============================" << endl;
int ms = 3000; //三秒位置 根据时间基数（分数）转换
long long pos = (double)ms / (double)1000 * r2d(ic->streams[pkt->stream_index]->time_base);
av_seek_frame(ic, videoStream, pos, AVSEEK_FLAG_BACKWARD | AVSEEK_FLAG_FRAME);
continue;
}
cout << "pkt->size = " << pkt->size << endl;
//显示的时间
cout << "pkt->pts = " << pkt->pts << "(单位：秒)" << endl;
//转换为毫秒，方便做同步
cout << "pkt->pts ms = " << pkt->pts * (r2d(ic->streams[pkt->stream_index]->time_base) * 1000) << "(单位：毫秒)" << endl;
//解码时间
cout << "pkt->dts = " << pkt->dts << endl;
AVCodecContext* cc = 0;
if (pkt->stream_index == videoStream)
{
cout << "图像" << endl;
cc = vc;
}
if (pkt->stream_index == audioStream)
{
cout << "音频" << endl;
cc = ac;
}
///解码视频
//发送packet到解码线程 send传NULL后调用多次receive取出所有缓冲帧
re = avcodec_send_packet(cc, pkt);
//释放，引用计数-1 为0释放空间
av_packet_unref(pkt);
if (re != 0)
{
char buf[1024] = { 0 };
av_strerror(re, buf, sizeof(buf) - 1);
cout << "avcodec_send_packet failed! :" << buf << endl;
continue;
}
for (;;)
{
//从线程中获取解码接口,一次send可能对应多次receive
re = avcodec_receive_frame(cc, frame);
//直到收不到为止,break退出循环
if (re != 0) break;
if (cc == vc) {//视频
cout << "recv frame format = " << frame->format << endl;
cout << "recv frame linesize[0] = " << frame->linesize[0] << endl;
cout << "recv frame linesize[1] = " << frame->linesize[1] << endl;
cout << "recv frame linesize[2] = " << frame->linesize[2] << endl;
//cout << endl;
}else {//音频
cout << "recv frame format = " << frame->format << endl;
cout << "recv frame linesize[0] = " << frame->linesize[0] << endl;
cout << "recv frame nb_samples = " << frame->nb_samples << endl;
}
//视频
if (cc == vc)
{
vctx = sws_getCachedContext(
vctx, //传NULL会新创建
frame->width, frame->height, //输入的宽高
(AVPixelFormat)frame->format, //输入格式 YUV420p
frame->width, frame->height, //输出的宽高
AV_PIX_FMT_RGBA, //输出格式RGBA
SWS_BILINEAR, //尺寸变化的算法
0, 0, 0);
//if(vctx)
//cout << "像素格式尺寸转换上下文创建或者获取成功！" << endl;
//else
// cout << "像素格式尺寸转换上下文创建或者获取失败！" << endl;
if (vctx)
{
if (!rgb) rgb = new unsigned char[frame->width * frame->height * 4];
uint8_t* data[2] = { 0 };
data[0] = rgb;
int lines[2] = { 0 };
lines[0] = frame->width * 4;
re = sws_scale(vctx,
frame->data, //输入数据
frame->linesize, //输入行大小
0,
frame->height, //输入高度
data, //输出数据和大小
lines
);
cout << "sws_scale = " << re << endl;
}
}
else //音频
{
uint8_t* data[2] = { 0 };
if (!pcm) pcm = new uint8_t[frame->nb_samples * 2 * 2];
data[0] = pcm;
re = swr_convert(actx,
data, frame->nb_samples, //输出
(const uint8_t**)frame->data, frame->nb_samples //输入
);
cout << "swr_convert = " << re << endl;
}
}
//XSleep(500);
}
av_frame_free(&frame);
av_packet_free(&pkt);
if (ic)
{
//释放封装上下文，并且把ic置0
avformat_close_input(&ic);
}
getchar();
return 0;
}
函数调用结构图
分析得到的libswscale的函数调用关系如下图所示。
解码后的数据为什么要经过sws_scale()函数处理？
解码后YUV格式的视频像素数据保存在AVFrame的data[0]、data[1]、data[2]中。但是这些像素值并不是连续存储的，每行有效像素之后存储了一些无效像素 。
以亮度 Y 数据为例 ， data[0] 中一共包含了linesize[0]*height个数据。
但是出于优化等方面的考虑，linesize[0]实际上并不等于宽度width，而是一个比宽度大一些的值。
因此需要使用sws_scale()进行转换。转换后去除了无效数据，width和linesize[0]取值相等。
AVPixelFormat结构体
enum AVPixelFormat {
AV_PIX_FMT_NONE = -1,
AV_PIX_FMT_YUV420P, ///< planar YUV 4:2:0, 12bpp, (1 Cr & Cb sample per 2x2 Y samples)
AV_PIX_FMT_YUYV422, ///< packed YUV 4:2:2, 16bpp, Y0 Cb Y1 Cr
AV_PIX_FMT_RGB24, ///< packed RGB 8:8:8, 24bpp, RGBRGB...
AV_PIX_FMT_BGR24, ///< packed RGB 8:8:8, 24bpp, BGRBGR...
AV_PIX_FMT_YUV422P, ///< planar YUV 4:2:2, 16bpp, (1 Cr & Cb sample per 2x1 Y samples)
AV_PIX_FMT_YUV444P, ///< planar YUV 4:4:4, 24bpp, (1 Cr & Cb sample per 1x1 Y samples)
AV_PIX_FMT_YUV410P, ///< planar YUV 4:1:0, 9bpp, (1 Cr & Cb sample per 4x4 Y samples)
AV_PIX_FMT_YUV411P, ///< planar YUV 4:1:1, 12bpp, (1 Cr & Cb sample per 4x1 Y samples)
AV_PIX_FMT_GRAY8, ///< Y , 8bpp
AV_PIX_FMT_MONOWHITE, ///< Y , 1bpp, 0 is white, 1 is black, in each byte pixels are ordered from the msb to the lsb
AV_PIX_FMT_MONOBLACK, ///< Y , 1bpp, 0 is black, 1 is white, in each byte pixels are ordered from the msb to the lsb
AV_PIX_FMT_PAL8, ///< 8 bits with AV_PIX_FMT_RGB32 palette
AV_PIX_FMT_YUVJ420P, ///< planar YUV 4:2:0, 12bpp, full scale (JPEG), deprecated in favor of AV_PIX_FMT_YUV420P and setting color_range
AV_PIX_FMT_YUVJ422P, ///< planar YUV 4:2:2, 16bpp, full scale (JPEG), deprecated in favor of AV_PIX_FMT_YUV422P and setting color_range
AV_PIX_FMT_YUVJ444P, ///< planar YUV 4:4:4, 24bpp, full scale (JPEG), deprecated in favor of AV_PIX_FMT_YUV444P and setting color_range
AV_PIX_FMT_UYVY422, ///< packed YUV 4:2:2, 16bpp, Cb Y0 Cr Y1
AV_PIX_FMT_UYYVYY411, ///< packed YUV 4:1:1, 12bpp, Cb Y0 Y1 Cr Y2 Y3
AV_PIX_FMT_BGR8, ///< packed RGB 3:3:2, 8bpp, (msb)2B 3G 3R(lsb)
AV_PIX_FMT_BGR4, ///< packed RGB 1:2:1 bitstream, 4bpp, (msb)1B 2G 1R(lsb), a byte contains two pixels, the first pixel in the byte is the one composed by the 4 msb bits
AV_PIX_FMT_BGR4_BYTE, ///< packed RGB 1:2:1, 8bpp, (msb)1B 2G 1R(lsb)
AV_PIX_FMT_RGB8, ///< packed RGB 3:3:2, 8bpp, (msb)2R 3G 3B(lsb)
AV_PIX_FMT_RGB4, ///< packed RGB 1:2:1 bitstream, 4bpp, (msb)1R 2G 1B(lsb), a byte contains two pixels, the first pixel in the byte is the one composed by the 4 msb bits
AV_PIX_FMT_RGB4_BYTE, ///< packed RGB 1:2:1, 8bpp, (msb)1R 2G 1B(lsb)
AV_PIX_FMT_NV12, ///< planar YUV 4:2:0, 12bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V)
AV_PIX_FMT_NV21, ///< as above, but U and V bytes are swapped
AV_PIX_FMT_ARGB, ///< packed ARGB 8:8:8:8, 32bpp, ARGBARGB...
AV_PIX_FMT_RGBA, ///< packed RGBA 8:8:8:8, 32bpp, RGBARGBA... 值为26
AV_PIX_FMT_ABGR, ///< packed ABGR 8:8:8:8, 32bpp, ABGRABGR...
AV_PIX_FMT_BGRA, ///< packed BGRA 8:8:8:8, 32bpp, BGRABGRA...
...
视频的解码流程
分析sws_getContext之前，先回顾一下视频解码流程
参考
FFmpeg的库函数源代码分析文章列表：
雷神的FFmpeg文章
【架构图】
【通用】
FFmpeg 源代码简单分析：av_register_all()
FFmpeg 源代码简单分析：avcodec_register_all()
FFmpeg 源代码简单分析：内存的分配和释放（av_malloc()、av_free()等）
FFmpeg 源代码简单分析：常见结构体的初始化和销毁（AVFormatContext，AVFrame等）
FFmpeg 源代码简单分析：av_find_decoder()和av_find_encoder()
FFmpeg 源代码简单分析：avcodec_open2()
FFmpeg 源代码简单分析：avcodec_close()
【解码】
图解FFMPEG打开媒体的函数avformat_open_input
FFmpeg 源代码简单分析：avformat_open_input()
FFmpeg 源代码简单分析：avformat_find_stream_info()
FFmpeg 源代码简单分析：av_read_frame()
FFmpeg 源代码简单分析：avcodec_decode_video2()
FFmpeg 源代码简单分析：avformat_close_input()
【编码】
FFmpeg 源代码简单分析：avformat_alloc_output_context2()
FFmpeg 源代码简单分析：avformat_write_header()
FFmpeg 源代码简单分析：avcodec_encode_video()
FFmpeg 源代码简单分析：av_write_frame()
FFmpeg 源代码简单分析：av_write_trailer()
【其它】
FFmpeg源代码简单分析：日志输出系统（av_log()等）
FFmpeg源代码简单分析：结构体成员管理系统-AVClass
FFmpeg源代码简单分析：结构体成员管理系统-AVOption
FFmpeg源代码简单分析：libswscale的sws_getContext()
FFmpeg源代码简单分析：libswscale的sws_scale()
FFmpeg源代码简单分析：libavdevice的avdevice_register_all()
FFmpeg源代码简单分析：libavdevice的gdigrab
【脚本】
【H.264】
字节流动大神
FFmpeg 开发系列连载：
- FFmpeg 开发(01)：FFmpeg 编译和集成
- FFmpeg 开发(02)：FFmpeg + ANativeWindow 实现视频解码播放
- FFmpeg 开发(03)：FFmpeg + OpenSLES 实现音频解码播放
- FFmpeg 开发(04)：FFmpeg + OpenGLES 实现音频可视化播放
- FFmpeg 开发(05)：FFmpeg + OpenGLES 实现视频解码播放和视频滤镜
- FFmpeg 开发(06)：FFmpeg 播放器实现音视频同步的三种方式
- FFmpeg 开发(07)：FFmpeg + OpenGLES 实现 3D 全景播放器
- FFmpeg 开发(08)：FFmpeg 播放器视频渲染优化
- FFmpeg 开发(09)：FFmpeg、x264以及fdk-aac 编译整合
- FFmpeg 开发(10)：FFmpeg 视频录制 - 视频添加滤镜和编码
- FFmpeg 开发(11)：FFmpeg + Android AudioRecorder 音频录制编码
- FFmpeg 开发(12)：Android FFmpeg 实现带滤镜的微信小视频录制功能
2240
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
