文章目录
前言
在做播放器渲染的时候会遇到渲染图像拉伸的情况,一般遇到这种图像显示比例不正确,多半是没有正确处理视频的纵横比。
简单介绍以下视频的三种纵横比,PAR、DAR 和 SAR,以及实际开发当中需要注意的情况。
Pixel Aspect Ratio 像素纵横比
像素是显示图像的最小单位,就是一个一个的小格子。一般来说像素的宽高比都是 1:1 ;如果不是1:1, 则该像素可以理解为长方形像素。常用的PAR比率有(1:1,10:11, 40:33, 16:11, 12:11 )
现代设备的像素纵横比都是 1:1的。(个人猜测,如果有错误地方欢迎指正)
Display Aspect Ratio 显示纵横比
DAR 显示纵横比,可以理解为实际显示到屏幕上的宽高比例。
拖拽播放窗口要按照这个比例缩放,否则图像会拉伸。一般来说 16:9 、 4:3 比较常见。
Sample Aspect Ratios 采样纵横比
SAR 采样纵横比,实际上告诉你视频需要按照采样纵横比拉伸。这样渲染出来的视频才是实际的要显示的效果。
网上很多博客说 SAR 表示横向的像素点数和纵向的像素点数的比值,即为我们通常提到的分辨率宽高比。其实这也是错误的。不知道谁开的头。但凡看看 ffplay源码也不至于这样人云亦云。
采样纵横比的意义是用来计算实际要渲染的显示纵横比。
实际的显示纵横比 DAR 计算公式如下:
实际显示纵横比(DAR) = SAR(采样纵横比) * 原视频的width / 原视频的height
比如一个视频的 width = 1280 height = 720 sar = 81:256
实际的渲染比 DAR = 81/ 256 * 1280 / 720 = 9: 16
如果不考虑采样纵横比,渲染出来的图像是16:9的,会出现图像拉伸情况(竖版视频以横板显示)。实际上是 9:16的渲染比例。
工程实战
实际开发中,我们只关心采样纵横比就可以。渲染图像要根据上面的公式计算出实际的显示纵横比,按照实际的显示纵横比去渲染,才不会出现图像拉伸的情况。
FFmpeg 获取采样纵横比的api av_guess_sample_aspect_ratio
/**
* Guess the sample aspect ratio of a frame, based on both the stream and the
* frame aspect ratio.
*
* Since the frame aspect ratio is set by the codec but the stream aspect ratio
* is set by the demuxer, these two may not be equal. This function tries to
* return the value that you should use if you would like to display the frame.
*
* Basic logic is to use the stream aspect ratio if it is set to something sane
* otherwise use the frame aspect ratio. This way a container setting, which is
* usually easy to modify can override the coded value in the frames.
*
* @param format the format context which the stream is part of
* @param stream the stream which the frame is part of
* @param frame the frame with the aspect ratio to be determined
* @return the guessed (valid) sample_aspect_ratio, 0/1 if no idea
*/
AVRational av_guess_sample_aspect_ratio(AVFormatContext *format, AVStream *stream, AVFrame *frame);
// 优先考虑 stream->sample_aspect_ratio 其次考虑 frame->sample_aspect_ratio
AVRational av_guess_sample_aspect_ratio(AVFormatContext *format, AVStream *stream, AVFrame *frame)
{
AVRational undef = {0, 1};
AVRational stream_sample_aspect_ratio = stream ? stream->sample_aspect_ratio : undef;
AVRational codec_sample_aspect_ratio = stream && stream->codecpar ? stream->codecpar->sample_aspect_ratio : undef;
AVRational frame_sample_aspect_ratio = frame ? frame->sample_aspect_ratio : codec_sample_aspect_ratio;
av_reduce(&stream_sample_aspect_ratio.num, &stream_sample_aspect_ratio.den,
stream_sample_aspect_ratio.num, stream_sample_aspect_ratio.den, INT_MAX);
if (stream_sample_aspect_ratio.num <= 0 || stream_sample_aspect_ratio.den <= 0)
stream_sample_aspect_ratio = undef;
av_reduce(&frame_sample_aspect_ratio.num, &frame_sample_aspect_ratio.den,
frame_sample_aspect_ratio.num, frame_sample_aspect_ratio.den, INT_MAX);
if (frame_sample_aspect_ratio.num <= 0 || frame_sample_aspect_ratio.den <= 0)
frame_sample_aspect_ratio = undef;
if (stream_sample_aspect_ratio.num)
return stream_sample_aspect_ratio;
else
return frame_sample_aspect_ratio;
}
ffplay SDL渲染窗口宽高的计算
ffplay 在渲染视频的时候是考虑了视频的采样纵横比,这样才可以正确的渲染出采样纵横比不为1 的视频,这样的播放器兼容性更高一些。
下面贴出来关键的处理代码。
void read_thread()
{
// 根据视频的SAR 采样纵横比 计算 sdl播放窗口的默认宽高
if (st_index[AVMEDIA_TYPE_VIDEO] >= 0) {
AVStream *st = ic->streams[st_index[AVMEDIA_TYPE_VIDEO]];
AVCodecParameters *codecpar = st->codecpar;
AVRational sar = av_guess_sample_aspect_ratio(ic, st, NULL);
if (codecpar->width)
set_default_window_size(codecpar->width, codecpar->height, sar);
}
}
// 计算 sdl渲染窗口的 width and height
static void calculate_display_rect(SDL_Rect *rect,
int scr_xleft, int scr_ytop, int scr_width, int scr_height,
int pic_width, int pic_height, AVRational pic_sar)
{
// 视频的SAR 采样纵横比
AVRational aspect_ratio = pic_sar;
int64_t width, height, x, y;
if (av_cmp_q(aspect_ratio, av_make_q(0, 1)) <= 0)
aspect_ratio = av_make_q(1, 1);
// aspect_ratio 输出的宽高比 = 采样纵横比aspect_ratio * 视频width / 视频height
aspect_ratio = av_mul_q(aspect_ratio, av_make_q(pic_width, pic_height));
/* XXX: we suppose the screen has a 1.0 pixel ratio */
height = scr_height;
width = av_rescale(height, aspect_ratio.num, aspect_ratio.den) & ~1;
if (width > scr_width) {
width = scr_width;
height = av_rescale(width, aspect_ratio.den, aspect_ratio.num) & ~1;
}
x = (scr_width - width) / 2;
y = (scr_height - height) / 2;
rect->x = scr_xleft + x;
rect->y = scr_ytop + y;
rect->w = FFMAX((int)width, 1);
rect->h = FFMAX((int)height, 1);
}
技术参考
- Advanced Aspect Ratios - PAR, DAR and SAR:https://www.animemusicvideos.org/guides/avtech3/theory-videoaspectratios.html