音视频实践学习
- android全平台编译ffmpeg以及x264与fdk-aac实践
- ubuntu下使用nginx和nginx-rtmp-module配置直播推流服务器
- android全平台编译ffmpeg合并为单个库实践
- android-studio使用cmake编译ffmpeg实践
- android全平台编译ffmpeg视频解码器实践
- android全平台编译ffmpeg支持命令行实践
- android全平台编译ffmpeg视频推流实践
- android平台下音频编码之编译LAME库转码PCM为MP3
- 图解YU12、I420、YV12、NV12、NV21、YUV420P、YUV420SP、YUV422P、YUV444P的区别
- ubuntu平台下编译vlc-android视频播放器实践
概述
YUV模型
是根据一个亮度(Y分量)
和两个色度(UV分量)
来定义颜色空间,常见的YUV格式有YUY2、YUYV、YVYU、UYVY、AYUV、Y41P、Y411、Y211、IF09、IYUV、YV12、YVU9、YUV411、YUV420
等,其中比较常见的YUV420
分为两种:YUV420P和YUV420SP
。
我们在android平台下使用相机默认图像格式是NV21
属于YUV420SP
格式
YUV采样
YUV 4:4:4采样,每一个Y对应一组UV分量,一个YUV占8+8+8 = 24bits 3个字节。
YUV 4:2:2采样,每两个Y共用一组UV分量,一个YUV占8+4+4 = 16bits 2个字节。
YUV 4:2:0采样,每四个Y共用一组UV分量,一个YUV占8+2+2 = 12bits 1.5个字节。
- 1
- 2
- 3
我们最常见的YUV420P和YUV420SP
都是基于4:2:0
采样的,所以如果图片的宽为width
,高为heigth
,在内存中占的空间为width * height * 3 / 2
,其中前width * height
的空间存放Y分量
,接着width * height / 4
存放U分量
,最后width * height / 4
存放V分量
。
YUV420P(YU12和YV12)格式
YUV420P
又叫plane平面模式
,Y , U , V
分别在不同平面,也就是有三个平面,它是YUV标准格式4:2:0
,主要分为:YU12和YV12
- YU12格式
在
android平台下
也叫作I420格式
,首先是所有Y值
,然后是所有U值
,最后是所有V值
。
YU12:亮度(行×列) + U(行×列/4) + V(行×列/4)
- YV12格式
YV12格式
与YU12
基本相同,首先是所有Y值
,然后是所有V值
,最后是所有U值
。只要注意从适当的位置提取U和V值
,YU12和YV12
都可以使用相同的算法进行处理。
YV12:亮度Y(行×列) + V(行×列/4) + U(行×列/4)
YU12: YYYYYYYY UUVV => YUV420P
YV12: YYYYYYYY VVUU => YUV420P
- 1
- 2
YUV模型是根据一个亮度(Y分量)和两个色度(UV分量)来定义颜色空间,常见的YUV格式有YUY2、YUYV、YVYU、UYVY、AYUV、Y41P、Y411、Y211、IF09、IYUV、YV12、YVU9、YUV411、YUV420等,其中比较常见的YUV420分为两种:YUV420P和YUV420SP。
我们在android平台下使用相机默认图像格式是NV21属于YUV420SP格式## YUV420SP(NV21和NV12)格式
YUV420SP
格式的图像阵列,首先是所有Y值
,然后是UV
或者VU
交替存储,NV12和NV21属于YUV420SP
格式,是一种two-plane模式
,即Y和UV分为两个plane
,但是UV(CbCr)
为交错存储,而不是分为三个平面。
- NV21格式
android手机从摄像头采集的预览数据一般都是NV21,存储顺序是先存Y,再VU交替存储,
NV21
存储顺序是先存Y值
,再VU
交替存储:YYYYVUVUVU
,以4 X 4
图片为例子,占用内存为4 X 4 X 3 / 2 = 24
个字节
- NV12格式
NV12与NV21类似,也属于
YUV420SP
格式,NV12
存储顺序是先存Y值
,再UV
交替存储:YYYYUVUVUV
,以4 X 4
图片为例子,占用内存为4 X 4 X 3 / 2 = 24
个字节
注意:在DVD中,色度信号被存储成Cb和Cr(C代表颜色,b代表蓝色,r代表红色)
NV12: YYYYYYYY UVUV =>YUV420SP
NV21: YYYYYYYY VUVU =>YUV420SP
- 1
- 2
YUV和RGB转换
Y = (0.257 * R) + (0.504 * G) + (0.098 * B) + 16
Cr = V = (0.439 * R) - (0.368 * G) - (0.071 * B) + 128
Cb = U = -(0.148 * R) - (0.291 * G) + (0.439 * B) + 128
B = 1.164(Y - 16) + 2.018(U - 128)
G = 1.164(Y - 16) - 0.813(V - 128) - 0.391(U - 128)
R = 1.164(Y - 16) + 1.596(V - 128)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
分离YUV420P
下面基于实例来理解Y,U,V分量的作用
先使用ffmpeg
将指定的图片转为yuv420p
格式
ffmpeg -i input.jpg -s 510x510 -pix_fmt yuv420p input.yuv
- 1
分离YUV分量
笔者使用的Clion
直接运行下面这段代码,分离出所需的文件
void split_yuv420(char *inputPath, int width, int height) {
FILE <span class="token operator">*</span>fp_yuv <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span>inputPath<span class="token punctuation">,</span> <span class="token string">"rb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_y <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_420_y.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_u <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_420_u.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_v <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_420_v.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> unsigned <span class="token keyword">char</span> <span class="token operator">*</span>data <span class="token operator">=</span> <span class="token punctuation">(</span>unsigned <span class="token keyword">char</span> <span class="token operator">*</span><span class="token punctuation">)</span> <span class="token function">malloc</span><span class="token punctuation">(</span>width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span> <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fread</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span> <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">,</span> fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//Y</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//U</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">,</span> fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//V</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">5</span> <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">,</span> fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//释放资源</span> <span class="token function">free</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span>
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
笔者使用的是ubuntu系统
,因此运行yuvplayer.exe
文件,需要提前安装好wine
:sudo apt install wine
,运行yuvplayer
之后,需要先设置像素格式为Y
,否则你看到的图像可能会有问题
先看output_420_y.y
文件:(分辨率设置为510x510)
output_420_u.y
显示如下:(分辨率设置为255x255)
output_420_v.y
显示如下:(分辨率设置为255x255)
- 生成灰度图
上面的例子实际上已经生成了一个
灰度图
了,但是只保留了Y分量
,你如果直接用ffplay工具
查看会有问题,下面的函数将会生成一个标准的YUV文件
并且保留Y分量
,你可能会有疑问,为什么U分量和V分量
要写入0x80
,其实你可以参考上面的YUV转RGB的公式
,YUV数据是无法直接显示的,最终需要转成RGB显示,因此我这里是只需要保留Y分量
,忽略UV
分量的影响,因此根据上面的公式,我在U和V分量中都写入128
就是十六进制的0x80
- 保留Y分量(生成灰度图)
void yuv420p_y(char *inputPath, char *outputPath, int width, int height) {
FILE <span class="token operator">*</span>inFile <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span>inputPath<span class="token punctuation">,</span> <span class="token string">"rb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>outFile <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span>outputPath<span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> unsigned <span class="token keyword">char</span> <span class="token operator">*</span>data <span class="token operator">=</span> <span class="token punctuation">(</span>unsigned <span class="token keyword">char</span> <span class="token operator">*</span><span class="token punctuation">)</span> <span class="token function">malloc</span><span class="token punctuation">(</span>width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span> <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fread</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span> <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">,</span> inFile<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//Y分量</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> outFile<span class="token punctuation">)</span><span class="token punctuation">;</span> unsigned <span class="token keyword">char</span> <span class="token operator">*</span>buffer <span class="token operator">=</span> <span class="token punctuation">(</span>unsigned <span class="token keyword">char</span> <span class="token operator">*</span><span class="token punctuation">)</span> <span class="token function">malloc</span><span class="token punctuation">(</span>width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">memset</span><span class="token punctuation">(</span>buffer<span class="token punctuation">,</span> <span class="token number">0x80</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//U分量</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>buffer<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">,</span> outFile<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//V分量</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>buffer<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">4</span><span class="token punctuation">,</span> outFile<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">free</span><span class="token punctuation">(</span>buffer<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">free</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>inFile<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>outFile<span class="token punctuation">)</span><span class="token punctuation">;</span>
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
int main() {
<span class="token function">yuv420p_y</span><span class="token punctuation">(</span><span class="token string">"/home/byhook/media/input.yuv"</span><span class="token punctuation">,</span> <span class="token string">"/home/byhook/media/output.yuv"</span><span class="token punctuation">,</span> <span class="token number">510</span><span class="token punctuation">,</span> <span class="token number">510</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span>
}
- 1
- 2
- 3
- 4
- 5
- 6
使用ffplay
来播放yuv格式
的文件:
ffplay -f rawvideo -video_size 510x510 output.yuv
- 1
要注意这里的分辨率不能错
分离YUV422P
YUV422P
基于YUV 4:2:2
采样,每两个Y共用一组UV分量,一个YUV占8+4+4 = 16bits 2个字节。分离代码如下:
void split_yuv422(char *inputPath, int width, int height) {
FILE <span class="token operator">*</span>fp_yuv <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span>inputPath<span class="token punctuation">,</span> <span class="token string">"rb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_y <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_422_y.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_u <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_422_u.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_v <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_422_v.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> unsigned <span class="token keyword">char</span> <span class="token operator">*</span>data <span class="token operator">=</span> <span class="token punctuation">(</span>unsigned <span class="token keyword">char</span> <span class="token operator">*</span><span class="token punctuation">)</span> <span class="token function">malloc</span><span class="token punctuation">(</span>width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fread</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">2</span><span class="token punctuation">,</span> fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//Y</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//U</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">,</span> fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//V</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span> <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">/</span> <span class="token number">2</span><span class="token punctuation">,</span> fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//释放资源</span> <span class="token function">free</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span>
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
分离YUV444P
YUV444P
基于YUV 4:4:4
采样,每一个Y对应一组UV分量,一个YUV占8+8+8 = 24bits 3个字节。分离代码如下:
void split_yuv444(char *inputPath, int width, int height) {
FILE <span class="token operator">*</span>fp_yuv <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span>inputPath<span class="token punctuation">,</span> <span class="token string">"rb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_y <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_444_y.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_u <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_444_u.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> FILE <span class="token operator">*</span>fp_v <span class="token operator">=</span> <span class="token function">fopen</span><span class="token punctuation">(</span><span class="token string">"output_444_v.y"</span><span class="token punctuation">,</span> <span class="token string">"wb+"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> unsigned <span class="token keyword">char</span> <span class="token operator">*</span>data <span class="token operator">=</span> <span class="token punctuation">(</span>unsigned <span class="token keyword">char</span> <span class="token operator">*</span><span class="token punctuation">)</span> <span class="token function">malloc</span><span class="token punctuation">(</span>width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fread</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">3</span><span class="token punctuation">,</span> fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//Y</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//U</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//V</span> <span class="token function">fwrite</span><span class="token punctuation">(</span>data <span class="token operator">+</span> width <span class="token operator">*</span> height <span class="token operator">*</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> width <span class="token operator">*</span> height<span class="token punctuation">,</span> fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//释放资源</span> <span class="token function">free</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_yuv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_y<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_u<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">fclose</span><span class="token punctuation">(</span>fp_v<span class="token punctuation">)</span><span class="token punctuation">;</span>
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
参考:
https://blog.csdn.net/leixiaohua1020/article/details/50534150
https://en.wikipedia.org/wiki/YUV