用v4l2和framebuffer实现usb摄像头视频采集并显示

4 篇文章 0 订阅

用v4l2和framebuffer实现usb摄像头图像采集并显示

 

前言

很多天了,看了数不尽的论坛和各网站的帖子后,我把我遇到的问题都解决的差不多了,感觉应该是把摄像头跑起来了,但还是遇到了一些问题,似乎是图像处理方面的,我好像解决不了这个问题了,我好想有个人帮帮我。写这篇文章估计得花3~4小时,我真心希望哪位朋友能明白我的想法,能顺手帮帮我。

 

正文

一,我的思路

我想用一幅图来说明我所做的事情,如图1所示(图中uvcdriver标错了,应该是uvcvideo)。

                                                                                                                          图1

 图1左侧是图像采集,右侧是图像显示。采集的帧速是30 帧/秒。

 

二,v4l2(video for linux two)

驱动摄像头的是uvcdriver,该驱动设计时采用了v4l系列的标准(该标准好像是linuxTV制定的,linuxTV的官网是http://linuxtv.org/),我的CentOS6.3(内核是linux-2.6.32.60)采用的是v4l2标准。一开始我编写应用程序的时候什么都不懂,见论坛上帖子怎么讲,我就怎么写,当中很多是参照v4l标准,我当时不知道,直接照抄,出了问题,改用v4l2标准后才解决了问题。v4l2 API的在/usr/include/linux/videodev2.h头文件中,很容易就找到了。

采集图像的实例程序网上很多,但最经典的还是linuxTV官网推出的capture.c,这里给出地址http://linuxtv.org/downloads/legacy/video4linux/API/V4L2_API/v4l2spec/capture.c这个程序写的很好,很值得研究。

我就是借鉴的这个程序,然后得到我的摄像头设备的信息,如图2(那个pixel_format应该用十六进制表示的,我没注意,十六进制的话应该是0x56595559,即YUYV的意思)所示。

                                                                 图2

一插上摄像头,uvcvideo就直接把我的摄像头识别了。

v4l2我不想再多写了,基本上只要看了那个经典的capture.c就够了。

 

三,framebuffer

就目前我这水平,我还不想涉及到QT或者别的什么面向对象编程,我想简单一点,慢慢来。于是我直接选择了framebuffer,它的API在usr/include/linux/fb.h头文件中。刚开始用open函数打开/dev/fb0的时候根本就打不开,愣是要把grub设置一下,如图3所示。后面显示的时候要切记切换到命令行模式。

                                                                                                                                      图3

关于那个vga=0x311以及一系列的frambuffer编程我是参照http://bbs.chinaunix.net/thread-2000076-1-1.html这个人的文章的,这里我复制一下他的一张表到下面,如表1所示。

              4bit        8bit         15bit        16bit        24bit        32bit
 
640x400       x           0x300        x            x            x            x
 
640x480       x           0x301        0x310        0x311        0x312        x
 
800x600       0x302       0x303        0x313        0x314        0x315        x
 
1024x768      x           0x305        0x316        0x317        0x318        x
 
1280x1024     x           0x307        0x319        0x31A        0x31B        x
 
1600x1200     x           0x31C        0x31D        0x31E        0x31F        x

                                                                                                          表1

 

因为我采集视频的图像时640*480,然后是16bit的BPP,所以我选择了0x311

framebuffer编程很简单,比v4l2简单多了。我截个图来表达我用framebuffer API得到的我的显示屏的信息,如图4所示。

                                                    图4

 四,v4l2+framebuffer

我把v4l和framebuffer集成到一块儿了。因为capture.c图像处理部分没有什么内容,所以,图像处理部分是我自己写的。我的摄像头采集的图像的像素编码是YUYV或者说是YUV422格式,每个像素是16bit。我做的事情就是把YUV422_16转换成RGB565_16,采集的图像尺寸是640*480的,显示屏也是640*480。

对于YUV422_16我想具体一点(因为当初觉得YUV格式很抽象,总是搞不懂)。然后看到一个表好像蛮形象的,我先把表画下来,如表2所示

像素:Y0U0V0  Y1U1V1  Y2U2V2  Y3U3V3  ...

采集:Y0U0V0  Y1      Y2U2V2  Y3      ...

存储:Y0U0Y1V0  Y2U2Y3V2

表2

Y0是一个字节,U0也是一个字节,V0也是一个字节,都是一个字节。

客观上每个像素是3个字节,但采集的时候却不是3个字节都采集,这样采集:第0个像素(Y0 U0 V0)采集了3个字节(Y0 U0 V0),第1个像素采集了1个字节(Y1),第2个像素采集了3个字节(Y2 U2 V2),第3个像素采集了1个字节(Y3)……第n个像素采集了3个字节(Yn Un Vn),第n+1个像素采集了1个字节(Yn+1)……。这种采集方式叫相邻两像素共用UV(不知道能不能从数学上证明如果每个像素3个字节都采集的话是不是就是冗余)。

怎么存储呢?好像是间隔存储。我记得好像有宏像素的概念,即2个像素占4个字节或者说4个字节的存储单元保存着2个像素的信息。比如Y0U0V1V0共需要一个字(4个字节),这个字的存储单元里保存着第0个像素和第1个像素的信息,第0个像素和第1个像素共享U0和V0。反正在后面的图像处理当中,我是用一个unsigned long *p的指针来处理一个宏像素的。当然了,这张表看不懂没关系,我当初就没看懂。
然后就是RGB了,我感觉RGB还是蛮形象的,尤其是RGB565_16,网上好多人研究这个,这里我就不多说了。

对于YUV和RGB之间的转换,我理解的程度也不深,我怕我的图像处理问题就是出在这个上面。

 

五,v4l2+framebuffer的组合效果

我看到我的摄像头采集出来的视频流了,因为担心YUV和RGB之间的转换时的数值处理出问题,我就只取了Y分量,我听说有黑白图像一说(其实YUV分量都提取的话,效果还是一样)。

1,提取YUV2分量代码如下:

		Y0=(*from & 0x000000FF)>>0;		
		U0=(*from & 0x0000FF00)>>8;	//colorful
		//U0=128;							//white and black
		Y1=(*from & 0x00FF0000)>>16;
		V0=(*from & 0xFF000000)>>24;	//colorful
		//V0=128;							//white and blcak

注:from是unsigned long型的指针变量,指向YUV2缓存。 

 

2,根据提取到的YUV分量,转换成RGB分量函数:

static  void YUV2RGB(unsigned char Y,unsigned char U,unsigned char V,
					unsigned char *R,unsigned char *G,unsigned char *B)					
{
/*
	*R=Y+1.4075*(V-128);
	*G=Y-0.3455*(U-128)-0.7169*(V-128);
	*B=Y+1.779*(U-128);

	*R=Y+1.140*V;
	*G=Y-0.394*U-0.581*V;
	*B=Y+2.032*U;
*/

	*R=Y+(V-128)+((V-128)*103>>8);
	*G=Y-((U-128)*88>>8)-((V-128)*183>>8);
	*B=Y+(U-128)+((U-128)*198>>8);
}

 

3,把RGB分量再编码成像素:

*to = (R0&0x1F)<<11 | (G0&0x3F)<<5 |(B0&0x1F)<<0;
*to |= ((R1&0x1F)<<11 | (G1&0x3F)<<5 |(B1&0x1F)<<0)<<16;

注:to是unsigned long型指针变量,指向RGB565输出缓存。


 编译好程序后,我用ctrl+alt+F2切换到命令行模式看看效果,我先截图,如图5和图6所示

                                                             图5(我的v4l2+framebuffer)

 

                                                                图6(关掉台灯,我的v4l2+framebuffer)

 

我ctrl+alt+F1切换到图形界面,看看cheese的效果,如图7所示。

                                                                                                     图7(cheese)

 为什么cheese的效果会这么好(显示效果不流畅,但画面很美),为什么我的视频的图像会成为这个样子(显示效果很流畅,但画面很丑),我搞了好长时间都发现不了问题出在哪里。谁能帮帮我吗?

 

六,程序

我的程序就一个bluelover.c文件。

/*
*  This program can be used and distributed without restrictions.
*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>              /* low-level i/o */
#include <unistd.h>
#include <errno.h>
#include <assert.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <stdbool.h>

#include <linux/videodev2.h>
#include <linux/fb.h>

#define CLEAR(x) memset(&(x), 0, sizeof(x))

struct buffer {
        void   *start;
        size_t  length;
};

static char *dev_name1;
static char *dev_name2;
static int  fd1 = -1;
static int	fd2 = -1;

struct buffer          *buffers;
static unsigned int     n_buffers;
static char *fb_buffer;
static unsigned long screensize;

static char *yuv_buffer;
static char *rgb_buffer;


static int              force_format=false;
static int              frame_count = 1000;

static void errno_exit(const char *s)
{
        fprintf(stderr, "%s error %d, %s\n", s, errno, strerror(errno));
        exit(EXIT_FAILURE);
}

static int xioctl(int fh, int request, void *arg)
{
        int r;

        do {
                r = ioctl(fh, request, arg);
        } while (-1 == r && EINTR == errno);

        return r;
}
/*
static void process_image(const void *p, int size)
{
	unsigned short *fb;
	unsigned short *rgb;
	unsigned long height = 480;
	if(size>screensize)
		size=screensize;

	//move
	memcpy(yuv_buffer,p,size);

	//proc
	memcpy(rgb_buffer,yuv_buffer,size);

	//move
	memcpy(fb_buffer,yuv_buffer,size);
	

	//move
	rgb=(unsigned short*)rgb_buffer;
	fb=(unsigned short*)fb_buffer;
	while(height--)
	{
		memcpy(fb,rgb,1280);
		fb+=640;
		rgb+=640;	
	}

	
	fflush(stderr);
	fprintf(stderr, ".");
	fflush(stdout);
}
*/
static  void YUV2RGB(unsigned char Y,unsigned char U,unsigned char V,
					unsigned char *R,unsigned char *G,unsigned char *B)
/*static  void YUV2RGB(int Y,int U,int V,
					int *R,int *G,int *B)*/					
{
/*
	*R=Y+1.4075*(V-128);
	*G=Y-0.3455*(U-128)-0.7169*(V-128);
	*B=Y+1.779*(U-128);
*/

	*R=Y+(V-128)+((V-128)*103>>8);
	*G=Y-((U-128)*88>>8)-((V-128)*183>>8);
	*B=Y+(U-128)+((U-128)*198>>8);
}

static void process_image(const unsigned long *from, int size)
{
	unsigned long *to =(unsigned long*)fb_buffer;
	
	unsigned char Y0;
	unsigned char U0;
	unsigned char Y1;
	unsigned char V0;
	
	unsigned char R0;
	unsigned char G0;
	unsigned char B0;
	unsigned char R1;
	unsigned char G1;
	unsigned char B1;	

	size>>=2;
	while(size--)
	{
		Y0=(*from & 0x000000FF)>>0;		
		U0=128;							//white and black
		//U0=(*from & 0x0000FF00)>>8;	//colorful
		Y1=(*from & 0x00FF0000)>>16;		
		V0=128;							//white and blcak
		//V0=(*from & 0xFF000000)>>24;	//colorful		
		YUV2RGB(Y0,U0,V0,&R0,&G0,&B0);
		YUV2RGB(Y1,U0,V0,&R1,&G1,&B1);

		*to = (R0&0x1F)<<11 | (G0&0x3F)<<5 |(B0&0x1F)<<0;
		*to |= ((R1&0x1F)<<11 | (G1&0x3F)<<5 |(B1&0x1F)<<0)<<16;		
		
		from++;
		to++;
	}
}
	
static int read_frame(void)
{
	struct v4l2_buffer buf;

	CLEAR(buf);
	buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	buf.memory = V4L2_MEMORY_MMAP;
	if (-1 == xioctl(fd1, VIDIOC_DQBUF, &buf))
	{
		switch (errno) 
		{
			case EAGAIN:
				return 0;
			case EIO:
			default:
				errno_exit("VIDIOC_DQBUF");
		}
	}

	assert(buf.index < n_buffers);
	
	process_image(buffers[buf.index].start, buf.bytesused);

	if (-1 == xioctl(fd1, VIDIOC_QBUF, &buf))
		errno_exit("VIDIOC_QBUF");

	return 1;
}

static void mainloop(void)
{
        unsigned int count;

        count = frame_count;
        while (count-- > 0) {
                for (;;) {
                        fd_set fds;
                        struct timeval tv;
                        int r;

                        FD_ZERO(&fds);
                        FD_SET(fd1, &fds);

                        /* Timeout. */
                        tv.tv_sec = 2;
                        tv.tv_usec = 0;

                        r = select(fd1 + 1, &fds, NULL, NULL, &tv);

                        if (-1 == r) {
                                if (EINTR == errno)
                                        continue;
                                errno_exit("select");
                        }

                        if (0 == r) {
                                fprintf(stderr, "select timeout\n");
                                exit(EXIT_FAILURE);
                        }

                        if (read_frame())
                                break;
                        /* EAGAIN - continue select loop. */
                }
        }
}

static void stop_capturing(void)
{
	enum v4l2_buf_type type;
	
	type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	if (-1 == xioctl(fd1, VIDIOC_STREAMOFF, &type))
		errno_exit("VIDIOC_STREAMOFF");
}

static void start_capturing(void)
{
	unsigned int i;
	enum v4l2_buf_type type;
	struct v4l2_buffer buf;
		
	for (i = 0; i < n_buffers; ++i) 
	{		
		CLEAR(buf);
		buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
		buf.memory = V4L2_MEMORY_MMAP;
		buf.index = i;
		if (-1 == xioctl(fd1, VIDIOC_QBUF, &buf))
			errno_exit("VIDIOC_QBUF");
	}
	
	type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	if (-1 == xioctl(fd1, VIDIOC_STREAMON, &type))
		errno_exit("VIDIOC_STREAMON");

}

static void uninit_device(void)
{
        unsigned int i;

	for (i = 0; i < n_buffers; ++i)
		if (-1 == munmap(buffers[i].start, buffers[i].length))
			errno_exit("munmap");
		
	if(-1 == munmap(fb_buffer,screensize))
		errno_exit("munmap");
	
	free(buffers);
	free(rgb_buffer);
	free(yuv_buffer);
}

static void init_mmap(void)
{
	struct v4l2_requestbuffers req;
	struct v4l2_buffer buf;

	//VIDIOC_REQBUFS
	CLEAR(req);
	req.count = 4;
	req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	req.memory = V4L2_MEMORY_MMAP;
	if(-1 == xioctl(fd1, VIDIOC_REQBUFS, &req)) 
	{
		if (EINVAL == errno) 
		{
			fprintf(stderr, "%s does not support ""memory mapping\n", dev_name1);
			exit(EXIT_FAILURE);
		} 
		else
			errno_exit("VIDIOC_REQBUFS");
	}
	if(req.count < 2) 
	{
	        fprintf(stderr, "Insufficient buffer memory on %s\n",dev_name1);
	        exit(EXIT_FAILURE);
	}

	//VIDIOC_QUERYBUF
	buffers = calloc(req.count, sizeof(*buffers));
	if (!buffers) 
	{
	        fprintf(stderr, "Out of memory\n");
	        exit(EXIT_FAILURE);
	}
	for (n_buffers = 0; n_buffers < req.count; ++n_buffers) 
	{	     
		CLEAR(buf);
		buf.type        = V4L2_BUF_TYPE_VIDEO_CAPTURE;
		buf.memory      = V4L2_MEMORY_MMAP;
		buf.index       = n_buffers;
		if (-1 == xioctl(fd1, VIDIOC_QUERYBUF, &buf))
			errno_exit("VIDIOC_QUERYBUF");
		buffers[n_buffers].length = buf.length;
		buffers[n_buffers].start =mmap(NULL /* start anywhere */,buf.length,
								PROT_READ | PROT_WRITE /* required */,
								MAP_SHARED /* recommended */,
								fd1, buf.m.offset);
		if (MAP_FAILED == buffers[n_buffers].start)
		{
			errno_exit("mmap");
		}
	}//now n_buffers is 4

	//fbmmap
	fb_buffer=(char*)mmap(NULL,screensize,PROT_READ | PROT_WRITE,MAP_SHARED,fd2,0);
	if(MAP_FAILED == fb_buffer)
	{
		errno_exit("mmap");
	}
	
	//malloc the yuv_buffer
	yuv_buffer = (char*)malloc(screensize);
	if (!yuv_buffer) 
	{
	        fprintf(stderr, "Out of memory\n");
	        exit(EXIT_FAILURE);
	}
	
	//malloc the rgb_buffer
	rgb_buffer = (char*)malloc(screensize);
	if (!rgb_buffer) 
	{
	        fprintf(stderr, "Out of memory\n");
	        exit(EXIT_FAILURE);
	}	
}

static void init_device(void)
{
	struct v4l2_capability cap;
	struct v4l2_format fmt;
	struct fb_fix_screeninfo finfo;
	struct fb_var_screeninfo vinfo;

	CLEAR(cap);
	if (-1 == xioctl(fd1, VIDIOC_QUERYCAP, &cap))
	{
		if (EINVAL == errno) 
		{
			fprintf(stderr, "%s is no V4L2 device\n",dev_name1);
			exit(EXIT_FAILURE);
		} 
		else 
			errno_exit("VIDIOC_QUERYCAP");
	}
	if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) 
	{
		fprintf(stderr, "%s is no video capture device\n",dev_name1);
		exit(EXIT_FAILURE);
	}
	if (!(cap.capabilities & V4L2_CAP_STREAMING)) 
	{
		fprintf(stderr, "%s does not support streaming i/o\n",dev_name1);
		exit(EXIT_FAILURE);
	}

	CLEAR(fmt);
	fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	if (force_format) 
	{
		fmt.fmt.pix.width       = 640;
		fmt.fmt.pix.height      = 480;
		fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
		fmt.fmt.pix.field       = V4L2_FIELD_INTERLACED;
		if (-1 == xioctl(fd1, VIDIOC_S_FMT, &fmt))
			errno_exit("VIDIOC_S_FMT");
	} 
	else
	{
		if (-1 == xioctl(fd1, VIDIOC_G_FMT, &fmt))
			errno_exit("VIDIOC_G_FMT");
	}
	fprintf(stdout,"<------------camera infomation------------->\n");
	fprintf(stdout,"device driver=%s\n",cap.driver);
	fprintf(stdout,"device name=%s\n",cap.card);
	fprintf(stdout,"bus_infomation=%s\n",cap.bus_info);
	fprintf(stdout,"image_width=%d\n",fmt.fmt.pix.width);
	fprintf(stdout,"image_height=%d\n",fmt.fmt.pix.height);
	fprintf(stdout,"pixel_format=%d\n",fmt.fmt.pix.pixelformat);
	fprintf(stdout,"\n");


	CLEAR(finfo);
	if(-1 == xioctl(fd2,FBIOGET_FSCREENINFO,&finfo))
		errno_exit("FBIOGET_FSCREENINFO");

	CLEAR(vinfo);
	if(-1 == xioctl(fd2,FBIOGET_VSCREENINFO,&vinfo))
		errno_exit("FBIOGET_VSCREENINFO");
	screensize = vinfo.xres*vinfo.yres*vinfo.bits_per_pixel/8;

	fprintf(stdout,"<------------screen infomation------------->\n");
	fprintf(stdout,"id=%s\n",finfo.id);
	fprintf(stdout,"x=%d\n",vinfo.xres);
	fprintf(stdout,"y=%d\n",vinfo.yres);
	fprintf(stdout,"bpp=%d\n",vinfo.bits_per_pixel);
	fprintf(stdout,"redoffset=%d,redlength=%d,msb_right=%d\n",
			vinfo.red.offset,vinfo.red.length,vinfo.red.msb_right);
	fprintf(stdout,"greenoffset=%d,greenlength=%d,msb_right=%d\n",
			vinfo.green.offset,vinfo.green.length,vinfo.green.msb_right);
	fprintf(stdout,"blueoffset=%d,bluelength=%d,msb_right=%d\n",
			vinfo.blue.offset,vinfo.blue.length,vinfo.blue.msb_right);
	fprintf(stdout,"screensize=%d\n",screensize);
	
	init_mmap();
}

static void close_device(void)
{
	if (-1 == close(fd1))
		errno_exit("close");
	fd1 = -1;
	
	if (-1 == close(fd2))
		errno_exit("close");
	fd2 = -1;
}

static void open_device(void)
{

	fd1 = open(dev_name1, O_RDWR /* required */ | O_NONBLOCK, 0);
	if (-1 == fd1) 
	{
		fprintf(stderr, "Cannot open '%s': %d, %s\n",dev_name1, errno, strerror(errno));
		exit(EXIT_FAILURE);
	}

	fd2 = open(dev_name2,O_RDWR,0);
	if(-1 == fd2)
	{
		fprintf(stderr, "Cannot open '%s': %d, %s\n",dev_name2, errno, strerror(errno));
		exit(EXIT_FAILURE);		
	}	
}

int main(int argc, char **argv)
{
	dev_name1 = "/dev/video0";
	dev_name2 = "/dev/fb0";

	open_device();
	init_device();
	start_capturing();
	mainloop();
	stop_capturing();
	uninit_device();
	close_device();
	fprintf(stderr, "\n");
	return 0;
}


 

 

后记

等解决了这个问题再写吧。

  • 7
    点赞
  • 16
    收藏
  • 打赏
    打赏
  • 8
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:大白 设计师:CSDN官方博客 返回首页
评论 8

打赏作者

ddddwant

你的鼓励将是我创作的最大动力

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值