CSI驱动说明

目录

1 数据类型简要说明

1.1 struct i2c_driver

1.2 struct v4l2_subdev_ops

1.2.1 v4l2_subdev_core_ops

1.2.2 v4l2_subdev_video_ops

1.2.3 v4l2_subdev_pad_ops

1.3 struct v4l2_ctrl_ops

1.4 struct xxxx_mode

1.5 struct v4l2_mbus_framefmt

2. 驱动移植步骤

2.1 实现标准I2C子设备驱动部分

2.1.1 probe函数实现细节描述:

 2.2 参考struct v4l2_subdev_ops说明实现v4l2子设备驱动，主要实现以下3 个成员：

2.2.1 参考struct v4l2_subdev_core_ops说明实现其回调函数，主要实现以下回调：

2.2.2 参考struct v4l2_subdev_video_ops说明实现其回调函数，主要实现以下回调函数：

 2.2.3 参考struct v4l2_subdev_pad_ops说明实现其回调函数，主要实现以下回调函数：

 2.3 参考struct v4l2_ctrl_ops说明实现，主要实现以下回调

---

Camera Sensor采用I2C与主控进行交互，目前sensor driver按照I2C设备驱动方式实现，sensor driver同时采用v4l2 subdev的方式实现与host driver之间的交互。

1 数据类型简要说明

1.1 struct i2c_driver

[说明]           

定义i2c 设备驱动信息

[定义]                                                                                                                               

struct i2c_driver {

 ……

/* Standard driver model interfaces */

int (*probe)(struct i2c_client *, const struct i2c_device_id *);

int (*remove)(struct i2c_client *);   

……

struct device_driver driver;

const struct i2c_device_id *id_table;

……

};

[关键成员]

成员名称	描述
driver	Device driver model driver 主要包含驱动名称和与DTS 注册设备进行匹配的of_match_table及电源管理回调。当of_match_table中的compatible域和dts文件的compatible域匹配时，.probe函数才会被调用
id_table	List of I2C devices supported by this driver 如果kernel没有使用of_match_table和dts注册设备进行进行匹配，则kernel使用该table进行匹配
probe	Callback for device binding
remove	Callback for device unbinding

1.2 struct v4l2_subdev_ops

[说明]       

Define ops callbacks for subdevs.

[定义]         

struct v4l2_subdev_ops {

const struct v4l2_subdev_core_ops  *core;

……

const struct v4l2_subdev_video_ops  *video;

……

const struct v4l2_subdev_pad_ops  *pad;

};

[关键成员]

成员名称	描述
.core	Define core ops callbacks for subdevs
.video	Callbacks used when v4l2 device was opened in video mode.
.pad	v4l2-subdev pad level operations

1.2.1 v4l2_subdev_core_ops

[说明] 

Define core ops callbacks for subdevs.

[定义]       

struct v4l2_subdev_core_ops {

……

int (*s_power)(struct v4l2_subdev *sd, int on);

long (*ioctl)(struct v4l2_subdev *sd, unsigned int cmd, void *arg);

#ifdef CONFIG_COMPAT

long (*compat_ioctl32)(struct v4l2_subdev *sd, unsigned int cmd, unsigned long arg);

#endif

……

};

[关键成员]

成员名称	描述
.s_power	puts subdevice in power saving mode (on == 0) or normal operation  mode (on == 1).
.ioctl	called at the end of ioctl() syscall handler at the V4L2 core. used to provide support for private ioctls used on the driver.
compat_ioctl32	called when a  32  bits application uses a  64 bits Kernel, in order to fix data passed from/to userspace.in order to fix data passed from/to userspace.

1.2.2 v4l2_subdev_video_ops

[说明] 

Callbacks used when v4l device was opened in video mode.

[定义]     

struct v4l2_subdev_video_ops {

……

int (*s_stream)(struct v4l2_subdev *sd, int enable);

……

int (*g_frame_interval)(struct v4l2_subdev *sd, struct v4l2_subdev_frame_interval *interval);

int (*g_mbus_config)(struct v4l2_subdev *sd, struct v4l2_mbus_config *cfg);

 ……

};

[关键成员]

成员名称	描述
.s_stream	used to notify the driver that a video stream will start or has stopped
.g_frame_interval	callback for VIDIOC_SUBDEV_G_FRAME_INTERVAL ioctl handler code
.g_mbus_config	get supported mediabus configurations

1.2.3 v4l2_subdev_pad_ops

[说明]   

v4l2-subdev pad level operations

[定义]     

struct v4l2_subdev_pad_ops {

……

int (*enum_mbus_code)(struct v4l2_subdev *sd,  struct v4l2_subdev_pad_config *cfg,struct v4l2_subdev_mbus_code_enum *code);

int (*enum_frame_size)(struct v4l2_subdev *sd,  struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_frame_size_enum *fse);

int (*get_fmt)(struct v4l2_subdev *sd, struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_format *format);

int (*set_fmt)(struct v4l2_subdev *sd, struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_format *format);

int (*enum_frame_interval)(struct v4l2_subdev *sd, struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_frame_interval_enum *fie);

int (*get_selection)(struct v4l2_subdev *sd, struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_selection *sel);

……

}; 

[关键成员]

.enum_mbus_code	callback for VIDIOC_SUBDEV_ENUM_MBUS_CODE ioctl handler code.
.enum_frame_size	callback for VIDIOC_SUBDEV_ENUM_FRAME_SIZE ioctl handler code.
.get_fmt	callback for VIDIOC_SUBDEV_S_FMT ioctl handler code.
.set_fmt	callback for VIDIOC_SUBDEV_S_FMT ioctl handler code.
.enum_frame_interval	callback  for  VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL()  ioctl handler code.
.get_selection	callback  for  VIDIOC_SUBDEV_G_SELECTION()  ioctl  handler code.

1.3 struct v4l2_ctrl_ops

[说明] 

The control operations that the driver has to provide.

[定义] 

struct   v4l2_ctrl_ops {

int (*s_ctrl)(struct v4l2_ctrl *ctrl);

};

[关键成员]

成员名称	描述
.s_ctrl	actually set the new control value.

RKISP 驱动要求使用框架提供的 user controls 功能， cameras sensor 驱动必须实现如下 control功能，参考CIS驱动V4L2-controls列表1

1.4 struct xxxx_mode

[说明]   

Sensor能支持各个模式的信息。 这个结构体在sensor 驱动中常常可以见到，虽然它不是v4l2标准要求的。

[定义]     

struct xxxx_mode {

u32 bus_fmt;                                                     

u32 width;

u32 height;

struct v4l2_fract max_fps;

u32 hts_def;

u32 vts_def;

u32 exp_def;

const struct regval *reg_list;

 u32 hdr_mode;

u32 vc[PAD_MAX];

};

[关键成员]

.bus_fmt	Sensor输出格式，参考 MEDIA_BUS_FMT 表
.width	有效图像宽度，需要和 sensor当前配置的width输出一致
.height	有效图像高度，需要和 sensor当前配置的height输出一致
.max_fps	图像 FPS，denominator/numerator为 fps
.hts_def	默认HTS(行长), 为有效图像宽度+HBLANK
.vts_def	默认VTS(帧长), 为有效图像高度+VBLANK
.exp_def	默认曝光时间
.reg_list	Sensor寄存器列表
.hdr_mode	Sensor工作模式，支持线性模式，两帧合成HDR,三帧合成HDR
.vc[PAD_MAX]	配置 MIPI VC通道 PAD0, /* link to isp */ PAD1, /* link to csi rawwr0 | hdr x2:L x3:M */ PAD2, /* link to csi rawwr1 | hdr      x3:L */ PAD3, /* link to csi rawwr2 | hdr x2:M x3:S */ PAD_MAX,

1.5 struct v4l2_mbus_framefmt

[说明]         

frame format on the media bus

[定义] 

struct v4l2_mbus_framefmt {

__u32      width;

__u32      height;

__u32      code;

__u32      field;

__u32      colorspace;

__u16      ycbcr_enc;

__u16      quantization;

__u16      xfer_func;

__u16      reserved[11];

};

[关键成员]

成员名称	描述
width	Frame width
height	Frame height
code	参考 MEDIA_BUS_FMT 表
field	V4L2_FIELD_NONE：帧输出方式 V4L2_FIELD_INTERLACED：场输出方式

2. 驱动移植步骤

2.1 实现标准I2C子设备驱动部分

根据struct i2c_driver说明实现以下成员:

struct driver.name

struct driver.pm

struct driver. of_match_table

probe函数

remove函数

2.1.1 probe函数实现细节描述:

1. CIS  设备资源的获取,主要是解析DTS 文件中定义资源, 参考Camera设备注册(DTS);

2. RK 私有资源定义,命名方式如下 rockchip,camera-module-xxx, 该部分资源会由驱动上传给用户态的camera_engine来决定IQ效果参数的匹配;

3. CIS 设备资源定义,RK相关参考驱动一般包含以下几项

CIS设备工作参考时钟	采用外部独立晶振方案无需获取,RK 参考设计一般采用 AP 输出时钟，该方案需要获取，一般名称为xvclk
CIS设备控制GPIO	例如：Resst引脚,Powerdown 引脚
CIS设备控制电源	根据实际硬件设计,获取匹配的软件电源控制资源,例如 gpio,regulator

4. CIS设备ID号检查, 通过以上步骤获取必要资源后,建议驱动读取设备ID号以便检查硬件的准确性,当然该步骤非必要步骤.

5. CIS v4l2设备以及media 实体的初始化

v4l2子设备：v4l2_i2c_subdev_init，RK CIS 驱动要求 subdev拥有自己的设备节点供用户态rk_aiq访问，通过该设备节点实现曝光控制；                                                           

media实体：media_entity_init

 2.2 参考struct v4l2_subdev_ops说明实现v4l2子设备驱动，主要实现以下3 个成员：

struct v4l2_subdev_core_ops

struct v4l2_subdev_video_ops

struct v4l2_subdev_pad_ops

2.2.1 参考struct v4l2_subdev_core_ops说明实现其回调函数，主要实现以下回调：

.s_power .ioctl .compat_ioctl32

     其中，ioctl主要实现的RK私有控制命令，涉及：

RKMODULE_GET_MODULE_INFO	DTS 文件定义的模组信息(模组名称等)，通过该命令上传camera_engine
RKMODULE_AWB_CFG	模组 OTP 信息使能情况下，camera_engine 通过该命令传递典型模组 AWB 标定值，CIS 驱动负责与当前模组AWB标定值比较后，生成R/B Gain值设置到CIS MWB模块中
RKMODULE_LSC_CFG	模组 OTP 信息使能情况下，camera_engine 通过该命令控制LSC 标定值生效使能
PREISP_CMD_SET_HDRAE_EXP	HDR曝光设置 详细参考struct preisp_hdrae_exp_s
RKMODULE_SET_HDR_CFG	设置HDR 模式，可实现 normal 和 hdr 切换，需要驱动适配hdr和 normal 2 组配置信息 详细参考struct rkmodule_hdr_cfg
RKMODULE_GET_HDR_CFG	获取当前HDR模式 详细参考struct rkmodule_hdr_cfg
RKMODULE_SET_CONVERSION_GAIN	设置线性模式的 conversion gain，如 imx347、os04a10 sensor 带有 conversion  gain 的功能，高转换的conversion gain 可以在低照度下获得更好的信噪比，如sensor不支持 conversion gain，可不实现

2.2.2 参考struct v4l2_subdev_video_ops说明实现其回调函数，主要实现以下回调函数：

.s_stream	开关数据流的函数，对于mipi clk是continuous的模式，必须在这个回调函数内开启数据流，若提前开数据流，会识别不到 MIPI LP状态
.g_frame_inverval	获取帧间隔参数（帧率）
.g_mbus_config	获取总线配置，对于 MIPI接口，sensor驱动内若支持不同 lane数配置或者支持 HDR,通过这个接口返回当前 sensor 工作模式下的MIPI配置

 2.2.3 参考struct v4l2_subdev_pad_ops说明实现其回调函数，主要实现以下回调函数：

.enmu_mbus_code	枚举当前CIS 驱动支持数据格式
.enmu_frame_size	枚举当前CIS 驱动支持分辨率
.get_fmt	RKISP driver通过该回调获取 CIS 输出的数据格式，务必实现； 针对Bayer raw sensor、SOC yuv sensor、BW raw sensor输出的数据类型定义参考MEDIA_BUS_FMT表 针对field 输出方式的支持，参考 struct v4l2_mbus_framefmt定义
.set_fmt	设置CIS 驱动输出数据格式以及分辨率，务必实现
.enum_frame_interval	枚举sensor支持的帧间隔，包含分辨率
.get_selection	配置裁剪参数，isp输入的宽度要求16对齐，高度 8 对齐

 2.3 参考struct v4l2_ctrl_ops说明实现，主要实现以下回调

.s_ctrl

RKISP driver、camera_engine 通过设置不同的命令来实现 CIS  曝光控制；

参考CIS 驱动V4L2-controls 列表1 实现各控制 ID，其中以下 ID 属于信息获取类，这部分实现按照standard integer menu controls方式实现；

V4L2_CID_LINK_FREQ	参考CIS驱动V4L2-controls列表1中标准定义，目前RKISP driver根据该命令获取MIPI总线频率；
V4L2_CID_PIXEL_RATE	针对MIPI总线： pixel_rate = link_freq * 2 * nr_of_lanes / bits_per_sample
V4L2_CID_HBLANK	参考CIS 驱动V4L2-controls列表1中标准定义
V4L2_CID_VBLEANK	参考CIS 驱动V4L2-controls列表1中标准定义

      RK camera_engine会通过以上命令获取必要信息来计算曝光，其中涉及的公式如下:

line_time = HTS / PIXEL_RATE

PIXEL_RATE = HTS * VTS * FPS

HTS = sensor_width_out + HBLANK;

VTS = sensor_height_out + VBLANK;

     其中以下ID 属于控制类，RK camera_engine通过该类命令控制 CIS

V4L2_CID_VBLANK	调整VBLANK，进而调整frame rate、Exposure time max
V4L2_CID_EXPOSURE	设置曝光时间，单位：曝光行数
V4L2_CID_ANALOGUE_GAIN	设置曝光增益，实际为 total gain = analog gain*digital gain; 单位：增益寄存器值

1. CIS参考驱动列表

CIS数据接口	CIS输出数据类型	Frame/Field	参考驱动
MIPI	Bayer Raw	Frame	ov8858.c ov7750.c ov5695.c ov5648.c ov4689.c ov2735.c ov2718.c ov2685.c ov2680.c imx327.c imx317.c imx258.c imx219.c gc8034.c gc5025.c gc2385.c gc2355.c
MIPI	Bayer raw hdr	Frame	os04a10.c imx347.c
MIPI	YUV	Frame	gc2145.c
MIPI	Raw bw	Frame	ov9281.c ov7251.c sc132gs.c
MIPI	YUV	Field	tc35874x.c
ITU.BT601	Bayer Raw		imx323 ar0230.c
ITU.BT601	YUV		gc2145.c gc2155.c gc2035.c gc0329.c gc0312.c bf3925.c
ITU.BT601	Raw bw
ITU.BT656	Bayer Raw		imx323(可支持)

附录 A CIS 驱动 V4L2-controls列表1

CID	描述
V4L2_CID_VBLANK	Vertical blanking. The idle period after every frame during which no image data is produced. The unit of vertical blanking is a line. Every line has length of the image width plus horizontal blanking at the pixel rate defined by V4L2_CID_PIXEL_RATE control in the same sub-device.
V4L2_CID_HBLANK	Horizontal blanking. The idle period after every line of image data during which no image data is produced. The unit of horizontal blanking is pixels.
V4L2_CID_EXPOSURE	Determines the exposure time of the camera sensor. The exposure time is limited by the frame interval.
V4L2_CID_ANALOGGUE_GAIN	Analogue gain is gain affecting all colour components in the pixel matrix. The gain operation is performed in the analogue domain before A/D conversion.
V4L2_CID_PIXEL_RATE	Pixel rate in the source pads of the subdev. This control is read-only and its unit is pixels / second.  Ex mipi bus： pixel_rate = link_freq * 2 * nr_of_lanes / bits_per_sample
V4L2_CID_LINK_FREQ	Data bus frequency. Together with the  media bus pixel code, bus type (clock cycles per sample), the data bus frequency defines the pixel rate (V4L2_CID_PIXEL_RATE) in the pixel array (or possibly elsewhere, if the device is not an image sensor). The frame rate can be calculated from the pixel clock, image width and height and horizontal and vertical blanking. While the pixel rate control may be defined elsewhere than in the subdev containing the pixel array, the frame rate cannot be obtained from that information. This is because only on the pixel array it can be assumed that the vertical and horizontal blanking information is exact: no other blanking is allowed in the pixel array. The selection of frame rate is performed by selecting the desired horizontal and vertical blanking. The unit of this control is Hz.

附录 B MEDIA_BUS_FMT 表

CIS sensor类型	Sensor输出format
Bayer Raw	MEDIA_BUS_FMT_SBGGR10_1X10 MEDIA_BUS_FMT_SRGGB10_1X10 MEDIA_BUS_FMT_SGBRG10_1X10 MEDIA_BUS_FMT_SGRBG10_1X10 MEDIA_BUS_FMT_SRGGB12_1X12 MEDIA_BUS_FMT_SBGGR12_1X12 MEDIA_BUS_FMT_SGBRG12_1X12 MEDIA_BUS_FMT_SGRBG12_1X12 MEDIA_BUS_FMT_SRGGB8_1X8 MEDIA_BUS_FMT_SBGGR8_1X8 MEDIA_BUS_FMT_SGBRG8_1X8 MEDIA_BUS_FMT_SGRBG8_1X8
YUV	MEDIA_BUS_FMT_YUYV8_2X8 MEDIA_BUS_FMT_YVYU8_2X8 MEDIA_BUS_FMT_UYVY8_2X8 MEDIA_BUS_FMT_VYUY8_2X8 MEDIA_BUS_FMT_YUYV10_2X10 MEDIA_BUS_FMT_YVYU10_2X10 MEDIA_BUS_FMT_UYVY10_2X10 MEDIA_BUS_FMT_VYUY10_2X10 MEDIA_BUS_FMT_YUYV12_2X12 MEDIA_BUS_FMT_YVYU12_2X12 MEDIA_BUS_FMT_UYVY12_2X12 MEDIA_BUS_FMT_VYUY12_2X12
Only Y(黑白) 即raw bw sensor	MEDIA_BUS_FMT_Y8_1X8 MEDIA_BUS_FMT_Y10_1X10 MEDIA_BUS_FMT_Y12_1X12