V4L2调试之(五）

1.关于/dev/mediaX设备的注册，有几个V4L2设备，注册几个/dev/meidaX节点。

比如，插入两个UVC USB Camera，就会出现如下图所示的两个节点，分别用于描述各自的V4L2拓扑。下面是linux上的callstack.

[   86.241290] v4l2_device_register line 21, kworker/2:3 register device.
[   86.241295] CPU: 2 PID: 1920 Comm: kworker/2:3 Tainted: G        W         5.4.128+ #1
[   86.241297] Hardware name: TIMI RedmiBook 14/TM1814, BIOS RMRWL400P0503 11/13/2019
[   86.241307] Workqueue: usb_hub_wq hub_event
[   86.241310] Call Trace:
[   86.241322]  dump_stack+0x6d/0x8b
[   86.241345]  v4l2_device_register+0x4a/0xe0 [videodev]
[   86.241354]  uvc_probe+0x4eb/0x2a10 [uvcvideo]
[   86.241363]  usb_probe_interface+0x149/0x300
[   86.241370]  ? uvc_register_video_device+0x150/0x150 [uvcvideo]
[   86.241374]  ? usb_probe_interface+0x149/0x300
[   86.241379]  really_probe+0xf5/0x440
[   86.241384]  driver_probe_device+0x11b/0x130
[   86.241388]  __device_attach_driver+0x7b/0xe0
[   86.241392]  ? driver_allows_async_probing+0x60/0x60
[   86.241399]  bus_for_each_drv+0x6e/0xb0
[   86.241403]  __device_attach+0xe4/0x160
[   86.241408]  device_initial_probe+0x13/0x20
[   86.241411]  bus_probe_device+0x92/0xa0
[   86.241416]  device_add+0x402/0x690
[   86.241422]  ? _cond_resched+0x19/0x40
[   86.241426]  usb_set_configuration+0x3fd/0x8f0
[   86.241432]  ? kernfs_activate+0x78/0x80
[   86.241440]  generic_probe+0x2e/0x80
[   86.241443]  usb_probe_device+0x31/0x70
[   86.241447]  really_probe+0xf5/0x440
[   86.241451]  driver_probe_device+0x11b/0x130
[   86.241455]  __device_attach_driver+0x7b/0xe0
[   86.241459]  ? driver_allows_async_probing+0x60/0x60
[   86.241465]  bus_for_each_drv+0x6e/0xb0
[   86.241468]  __device_attach+0xe4/0x160
[   86.241473]  device_initial_probe+0x13/0x20
[   86.241476]  bus_probe_device+0x92/0xa0
[   86.241480]  device_add+0x402/0x690
[   86.241487]  ? add_device_randomness+0x9d/0x1c0
[   86.241492]  usb_new_device+0x218/0x4a0
[   86.241498]  hub_event+0x11b1/0x1760
[   86.241508]  process_one_work+0x20f/0x400
[   86.241514]  worker_thread+0x34/0x410
[   86.241518]  kthread+0x121/0x140
[   86.241523]  ? process_one_work+0x400/0x400
[   86.241527]  ? kthread_park+0x90/0x90
[   86.241532]  ret_from_fork+0x35/0x40
[   86.263776] input: Integrated Camera: Integrated C as /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0/input/input25
[   96.375646] usb 1-4: new high-speed USB device number 7 using xhci_hcd
[   96.524824] usb 1-4: config 1 interface 0 altsetting 0 endpoint 0x83 has an invalid bInterval 32, changing to 9
[   96.525345] usb 1-4: New USB device found, idVendor=1b3f, idProduct=2247, bcdDevice= 1.00
[   96.525350] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   96.525353] usb 1-4: Product: GENERAL WEBCAM
[   96.525356] usb 1-4: Manufacturer: GENERAL
[   96.526771] uvcvideo: Found UVC 1.00 device GENERAL WEBCAM (1b3f:2247)
[   96.526777] v4l2_device_register line 21, kworker/2:3 register device.
[   96.526784] CPU: 2 PID: 1920 Comm: kworker/2:3 Tainted: G        W         5.4.128+ #1
[   96.526788] Hardware name: TIMI RedmiBook 14/TM1814, BIOS RMRWL400P0503 11/13/2019
[   96.526799] Workqueue: usb_hub_wq hub_event
[   96.526804] Call Trace:
[   96.526819]  dump_stack+0x6d/0x8b
[   96.526853]  v4l2_device_register+0x4a/0xe0 [videodev]
[   96.526867]  uvc_probe+0x4eb/0x2a10 [uvcvideo]
[   96.526878]  usb_probe_interface+0x149/0x300
[   96.526891]  ? uvc_register_video_device+0x150/0x150 [uvcvideo]
[   96.526896]  ? usb_probe_interface+0x149/0x300
[   96.526905]  really_probe+0xf5/0x440
[   96.526911]  driver_probe_device+0x11b/0x130
[   96.526918]  __device_attach_driver+0x7b/0xe0
[   96.526925]  ? driver_allows_async_probing+0x60/0x60
[   96.526933]  bus_for_each_drv+0x6e/0xb0
[   96.526939]  __device_attach+0xe4/0x160
[   96.526945]  device_initial_probe+0x13/0x20
[   96.526951]  bus_probe_device+0x92/0xa0
[   96.526964]  device_add+0x402/0x690
[   96.526972]  ? _cond_resched+0x19/0x40
[   96.526977]  usb_set_configuration+0x3fd/0x8f0
[   96.526985]  ? kernfs_activate+0x78/0x80
[   96.526997]  generic_probe+0x2e/0x80
[   96.527008]  usb_probe_device+0x31/0x70
[   96.527014]  really_probe+0xf5/0x440
[   96.527020]  driver_probe_device+0x11b/0x130
[   96.527025]  __device_attach_driver+0x7b/0xe0
[   96.527030]  ? driver_allows_async_probing+0x60/0x60
[   96.527038]  bus_for_each_drv+0x6e/0xb0
[   96.527045]  __device_attach+0xe4/0x160
[   96.527052]  device_initial_probe+0x13/0x20
[   96.527056]  bus_probe_device+0x92/0xa0
[   96.527063]  device_add+0x402/0x690
[   96.527072]  ? add_device_randomness+0x9d/0x1c0
[   96.527079]  usb_new_device+0x218/0x4a0
[   96.527087]  hub_event+0x11b1/0x1760
[   96.527108]  process_one_work+0x20f/0x400
[   96.527123]  worker_thread+0x34/0x410
[   96.527130]  kthread+0x121/0x140
[   96.527137]  ? process_one_work+0x400/0x400
[   96.527142]  ? kthread_park+0x90/0x90
[   96.527149]  ret_from_fork+0x35/0x40

struct v4l2_device的注册也是这样，它可以看成v4l2 框架的 rootdevice，每个V4L2的采集设备，都对应一个v4l2_device,也对应如上的/dev/mediaX., 所有的subdev都链接在同一个v4l2_device下. 通过v4l2_device, 我们可以遍历所有的sbudev. v4l2_device此时相当于所以subdev的父设备. 从如下调用堆栈也可以看出通用USB设备的发现流程。由于USB驱动已经事先注册进系统，这里是通过HUB的设备发现去发现USBS设备的插入，之后进行USB设备级的device_add,和设备驱动匹配上后，进行USB设备级的probe函数usb_probe_device.然后进步步，读取USB配置后，进行接口级的USB设备注册，再次调用device_add在USB总线上MATCH USB INTERFACE驱动，注册真正的video device uvc驱动。在这个层面，实际上最后一级的probe已经进入到 struct usb_driver的定义了。本质上，usb_driver是对USB INTERFACE struct device_driver的封装。

USB设备的注册接口和INTERFACE的注册接口是不同的,通过for_device变量区分。设别的变量为1.usb_register_device_driver

 而接口的变量为0，usb_register_driver：

​
[   58.718847] media_devnode_register line 248.comm kworker/1:2.
[   58.718854] CPU: 1 PID: 327 Comm: kworker/1:2 Tainted: G        W         5.4.128+ #1
[   58.718856] Hardware name: TIMI RedmiBook 14/TM1814, BIOS RMRWL400P0503 11/13/2019
[   58.718866] Workqueue: usb_hub_wq hub_event
[   58.718869] Call Trace:
[   58.718881]  dump_stack+0x6d/0x8b
[   58.718893]  media_devnode_register+0x11b/0x1c0 [mc]
[   58.718902]  __media_device_register+0x77/0x110 [mc]
[   58.718910]  uvc_probe+0x2638/0x2a10 [uvcvideo]
[   58.718919]  usb_probe_interface+0x149/0x300
[   58.718926]  ? uvc_register_video_device+0x150/0x150 [uvcvideo]
[   58.718929]  ? usb_probe_interface+0x149/0x300
[   58.718935]  really_probe+0xf5/0x440
[   58.718939]  driver_probe_device+0x11b/0x130
[   58.718944]  __device_attach_driver+0x7b/0xe0
[   58.718948]  ? driver_allows_async_probing+0x60/0x60
[   58.718955]  bus_for_each_drv+0x6e/0xb0
[   58.718959]  __device_attach+0xe4/0x160
[   58.718963]  device_initial_probe+0x13/0x20
[   58.718966]  bus_probe_device+0x92/0xa0
[   58.718972]  device_add+0x402/0x690
[   58.718978]  ? _cond_resched+0x19/0x40
[   58.718981]  usb_set_configuration+0x3fd/0x8f0
[   58.718988]  ? kernfs_activate+0x78/0x80
[   58.718996]  generic_probe+0x2e/0x80
[   58.718999]  usb_probe_device+0x31/0x70
[   58.719003]  really_probe+0xf5/0x440
[   58.719007]  driver_probe_device+0x11b/0x130
[   58.719011]  __device_attach_driver+0x7b/0xe0
[   58.719015]  ? driver_allows_async_probing+0x60/0x60
[   58.719020]  bus_for_each_drv+0x6e/0xb0
[   58.719024]  __device_attach+0xe4/0x160
[   58.719028]  device_initial_probe+0x13/0x20
[   58.719031]  bus_probe_device+0x92/0xa0
[   58.719036]  device_add+0x402/0x690
[   58.719043]  ? add_device_randomness+0x9d/0x1c0
[   58.719047]  usb_new_device+0x218/0x4a0
[   58.719053]  hub_event+0x11b1/0x1760
[   58.719063]  process_one_work+0x20f/0x400
[   58.719068]  worker_thread+0x34/0x410
[   58.719073]  kthread+0x121/0x140
[   58.719078]  ? process_one_work+0x400/0x400
[   58.719082]  ? kthread_park+0x90/0x90
[   58.719089]  ret_from_fork+0x35/0x40
[   58.719382] input: Integrated Camera: Integrated C as /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0/input/input24
[  152.269448] usb 1-4: new high-speed USB device number 6 using xhci_hcd
[  152.418887] usb 1-4: config 1 interface 0 altsetting 0 endpoint 0x83 has an invalid bInterval 32, changing to 9
[  152.419610] usb 1-4: New USB device found, idVendor=1b3f, idProduct=2247, bcdDevice= 1.00
[  152.419616] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[  152.419619] usb 1-4: Product: GENERAL WEBCAM
[  152.419622] usb 1-4: Manufacturer: GENERAL
[  152.421232] uvcvideo: Found UVC 1.00 device GENERAL WEBCAM (1b3f:2247)
[  152.421461] uvcvideo: Failed to query (GET_INFO) UVC control 2 on unit 1: -32 (exp. 1).
[  152.424449] uvcvideo: UVC non compliance - GET_DEF(PROBE) not supported. Enabling workaround.
[  152.424987] media_devnode_register line 248.comm kworker/1:2.
[  152.424996] CPU: 1 PID: 327 Comm: kworker/1:2 Tainted: G        W         5.4.128+ #1
[  152.424998] Hardware name: TIMI RedmiBook 14/TM1814, BIOS RMRWL400P0503 11/13/2019
[  152.425010] Workqueue: usb_hub_wq hub_event
[  152.425014] Call Trace:
[  152.425026]  dump_stack+0x6d/0x8b
[  152.425039]  media_devnode_register+0x11b/0x1c0 [mc]
[  152.425048]  __media_device_register+0x77/0x110 [mc]
[  152.425055]  uvc_probe+0x2638/0x2a10 [uvcvideo]
[  152.425063]  usb_probe_interface+0x149/0x300
[  152.425070]  ? uvc_register_video_device+0x150/0x150 [uvcvideo]
[  152.425077]  ? usb_probe_interface+0x149/0x300
[  152.425082]  really_probe+0xf5/0x440
[  152.425086]  driver_probe_device+0x11b/0x130
[  152.425089]  __device_attach_driver+0x7b/0xe0
[  152.425093]  ? driver_allows_async_probing+0x60/0x60
[  152.425099]  bus_for_each_drv+0x6e/0xb0
[  152.425103]  __device_attach+0xe4/0x160
[  152.425107]  device_initial_probe+0x13/0x20
[  152.425109]  bus_probe_device+0x92/0xa0
[  152.425114]  device_add+0x402/0x690
[  152.425119]  ? _cond_resched+0x19/0x40
[  152.425122]  usb_set_configuration+0x3fd/0x8f0
[  152.425128]  ? kernfs_activate+0x78/0x80
[  152.425135]  generic_probe+0x2e/0x80
[  152.425138]  usb_probe_device+0x31/0x70
[  152.425141]  really_probe+0xf5/0x440
[  152.425145]  driver_probe_device+0x11b/0x130
[  152.425148]  __device_attach_driver+0x7b/0xe0
[  152.425151]  ? driver_allows_async_probing+0x60/0x60
[  152.425156]  bus_for_each_drv+0x6e/0xb0
[  152.425160]  __device_attach+0xe4/0x160
[  152.425163]  device_initial_probe+0x13/0x20
[  152.425165]  bus_probe_device+0x92/0xa0
[  152.425170]  device_add+0x402/0x690
[  152.425176]  ? add_device_randomness+0x9d/0x1c0
[  152.425180]  usb_new_device+0x218/0x4a0
[  152.425185]  hub_event+0x11b1/0x1760
[  152.425195]  process_one_work+0x20f/0x400
[  152.425203]  worker_thread+0x34/0x410
[  152.425208]  kthread+0x121/0x140
[  152.425212]  ? process_one_work+0x400/0x400
[  152.425216]  ? kthread_park+0x90/0x90
[  152.425220]  ret_from_fork+0x35/0x40
[  152.425547] input: GENERAL WEBCAM: GENERAL WEBCAM as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.0/input/input25
[  152.452622] uvcvideo: Failed to query (GET_DEF) UVC control 2 on unit 1: -32 (exp. 1).
[  152.452780] uvcvideo: Failed to query (GET_DEF) UVC control 2 on unit 1: -32 (exp. 1).
[  152.483857] usb 1-4: Warning! Unlikely big volume range (=5120), cval->res is probably wrong.
[  152.483859] usb 1-4: [5] FU [Mic Capture Volume] ch = 1, val = 7680/12800/1
[  152.484248] usbcore: registered new interface driver snd-usb-audio
czl@czl-RedmiBook-14:~/Workspace$ 

​

2.用到struct v4l2_subdev_internal_ops结构注册的地方 有两个，一个是

 另一个是：

有意思的是，这两个一个在PIPEline的首端(sensor),另一个在pipeline的尾端(video), v4l2_subdev_internal_ops提供了4个接口: registered、unregistered、open、close. 这4个接口是给v4l2核心层代码回调用的. 当向核心层注册/注销subdev时(调用v4l2_device_register_subdev）, 核心层会回调这里的registered/unregistered. 当用户空间打开/关闭设备节点时, 核心层会回调这里的open/close.

3. VIN框架中管理有一张表,表中记录了VIN支持的所有可能的格式,JPEG类的是YUV, SRGB类的是RGB.

V4L2_COLORSPACE_JPEG表示YUV像素格式.

使用的地方再这里:

再创建pipeline的时候会看是否支持上面配置下来的格式,不支持会返回失败.

4.V4L2设备初始化的时通过I2C和sensor的交互过程.

MIPI CSI的传输是这样的，如下图，它通过CCI总线对Sensor的寄存器进行配置。

一般情况下，CCI总线是通过I2C总线实现的，所以从驱动角度来讲，它实际上调用的是I2C驱动。下面调试一下Melis系统上Sensor初始化过程中，对I2C的调用时序.

首先看一下打印,在hal_twi_xfer里面加打印：

 __builtin_return_address(LEVEL)的作用是返回调用栈第level层的返回值。

可以看到，貌似GCC只能返回当前函数的返回地址。

所以看起来是从cci_read_a16_d8调用下来的，我们看一下这个调用是怎么发生的。

可以看到，这里就是上面说的internal_ops的handler, sensor_registered，调用链中，会调用sensor_detect尝试读取sensor_id,根据ID来判断sensor是否存在。

 返回的是IMX的 sensor id.

sensor bringup 的第二步是上电:

第三步的是init寄存器

 初始化的是register的列表

下一步，设置默认帧率，还是同样的调用栈：

wsize->regs是默认的帧率配置参数中的一个，每个帧率对应一种配置

接下来，设置增益：

打开阀门，让水流进来，开启sensor抓图。

到这里，画面实际上已经出来了，之后，就是ISP常规化的控制了，这个设置gain的调用会一直存在。

解决的一个问题记录

客户要求可以通过melis shell命令来读取sensor寄存器，由于sensor是板载资源，访问方式是通过cci总线（I2C）的方式进行的。需求很简单，但是在实现的时候遇到了一个技术性问题，感觉对理解v4l2 sensor注册的流程挺有帮助的，这里记录一下。

实现很简单，以为读取sensor  id为例，写完后是这样子的，根据上面的分析,0x16,0x17寄存器是ID寄存器，况且启动过程中是可以访问的，那么我把它封装成一个shell命令，应该也可以读取才对。

但是测试情况出乎意料：

执行read_sensor，I2C接口访问竟然是失败的。

而启动过程中的访问是成功的：

启动过程和手敲命令测试流程上是相距不远的逻辑，一定是中间发生了什么才导致的问题。处理这种问题最好使用二分法，不断在从OK到failure的路径中加上打印，最后发现了明显的差异点的位置：

 如上图，正常启动流程中的访问是在67行中进行的，然后直到73行中的调用还是OK的，但是到了81行，就无法成功读取到ID了，这说明问题就发生在73行到81行中间。

很明显，最值得怀疑的逻辑是77行，看起来它是将sensor电源关闭了。添加上打印调试一下：

 编译启动验证：

LOG证实了我们的猜测，PWR_ON和PWR_OFF中间的代码读取是成功的，恰好是PWR_OFF之后的，读取都失败了。

 所以，可以说明，流程中为了节省功耗，对电源进行了按需管理，启动过程中，如果需要对sensor寄存器进行访问，就上电，如果不需要访问sensor了，就关闭电源。

这个例子也可以从侧面说明，启动过程中访问sensor的地方只有一个，就是这里读取sensor id.

接下来把pwr off这行关闭，确认一下是否OK：

OK，看打印结果，每次的读取都是成功的了。

Video Device:

无论是subdevice还是video device，都需要创建struct video_device结构并注册。

static inline int __must_check video_register_device(struct video_device *vdev, …)

向核心层注册一个video_device, 此时核心层会创建字符设备驱动. 后面用户空间就可以通过字符设备节点与video_device交互了.

V4L2设备逻辑图

参考资料：

Linux V4L2子系统 | 思学网V4L2, Camera, subdevhttp://www.mysixue.com/?p=131

结束!