内容： 旨在对车载摄像头采集的车辆行驶视频中的车道线进行跟踪识别。
实现过程：首先对车辆行驶视频中提取出的帧图像进行“去畸变”、“透视变换”等操作，得到“鸟瞰图”视角的“规则的”车道线图像；然后通过“组合梯度和色彩过滤”提取出“可能的车道线像素”，使用“滑动窗口”确定“真实的车道线像素”，利用“曲线多项式”拟合“真实的车道线像素”；再然后基于拟合的车道线曲线，结合相机与真实世界比例参数，计算出车道线的曲率和车辆偏离车道线中心线的距离；最后测试跟踪一段 50 S左右的车载摄像头采集行车视频以验证程序可行性。
注意：
（1）本次博客内容注重代码功能实现讲解，原理部分可能需要参考之前的博客内容。
（2）使用 OpenCV 的 cv2.imread() 读取RGB三通道图像的结果是以 BGR 顺序排列的，而使用matplotlib的
plt.imread() 读取RGB三通道图像的结果是以 RGB 顺序排列的，应当注意区别。
（3）一些opencv库函数、python操作，博客内容中只提供了代码并未对其解释，需要读者阅读时自行百度查阅
即可。
准备工作：
1、用于相机校准的棋盘图像文件
2、用于编写代码过程中需要使用到的以及测试时需要用到的车道线图像（来源于车载摄像头行车视频中截取的帧图像）
3、用于最后测试“车载摄像头行车视频车道线跟踪”的测试视频
本次博客内容较多且代码过长，整体讲解容易引起读者阅读烦躁，故分为多段讲解，读者需要测试时只需将代码进行拼接，即可实现全部功能。
（一）、导入需要用到的包
# importing some useful packages
import matplotlib.pyplot as plt
import numpy as np
import cv2
import glob
import pickle
from moviepy.editor import VideoFileClip
from ipywidgets import interact, interactive, fixed
代码讲解：
（1）glob：python 自带的文件操作相关模块，用来查找符合自己目的的文件，类似于Windows下
的文件搜索，支持通配符操作，, ? , [ ] 这三个通配符 。，代表0个或多个字符，？代表一个字
符，[ ]匹配指定范围内的字符，如[0-9]匹配数字。
示例：获得指定目录下的所有 jpg 文件
使用相对路径：
glob.glob(’…/*.py’)
（2）pickle：用于 python 特有的类型与 python 的数据类型间进行转换
（3）moviepy：用于 python 处理视频的库
（4）ipywidgets ：用于实现在 Jupyter 中实现交互操作使用（如果运行环境是pyther shell 可不予
理会，边缘检测部分阈值使用默认即可，不影响最后的结果）
（二）、使用棋盘图像计算相机校准（图像去畸变）
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6 * 9, 3), np.float32)
objp[:,:2] = np.mgrid[0:9,0:6].T.reshape(-1,2)
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# Arrays to store object points and image points from all the images
objpoints = [] # 3d points in real world space
imgpoints = [] # 2d points in image plane
# Make a list of calibration images
images = glob.glob('./camera_cal/calibration*.jpg')
# Step through the list and search for chessboard corners
for i, fname in enumerate(images):
img = cv2.imread(fname)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chessboard corners
ret, corners = cv2.findChessboardCorners(gray, (9,6),None)
# If found, add object points, image points
if ret == True:
objpoints.append(objp)
# refine image points
corners2 = cv2.cornerSubPix(gray,corners,(11,11),(-1,-1),criteria)
imgpoints.append(corners2)
# Test undistortion on an image
img = cv2.imread('./camera_cal/calibration01.jpg')
img_size = (img.shape[1], img.shape[0])
# Do camera calibration given object points and image points
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, img_size,None,None)
dst = cv2.undistort(img, mtx, dist, None, mtx)
# Save the camera calibration result for later use
dist_pickle = {}
dist_pickle["mtx"] = mtx
dist_pickle["dist"] = dist
pickle.dump( dist_pickle, open( "calibration.p", "wb" ) )
# Test Code:
# Visualize undistortion
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=30)
ax2.imshow(dst)
ax2.set_title('Undistorted Image', fontsize=30)
f.show()
# Choose an image from which to build and demonstrate each step of the pipeline
exampleImg = cv2.imread('./test_images/challenge03.jpg')
exampleImg = cv2.cvtColor(exampleImg, cv2.COLOR_BGR2RGB)
# undistort image using camera calibration matrix from above
def undistort(img):
undist = cv2.undistort(img, mtx, dist, None, mtx)
return undist
exampleImg_undistort = undistort(exampleImg)
# Test Code:
# Visualize undistortion
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg)
ax1.set_title('Original Image', fontsize=30)
ax2.imshow(exampleImg_undistort)
ax2.set_title('Undistorted Image', fontsize=30)
f.show()
代码讲解：
（1）相机校准部分使用的棋盘图像有效黑白对角点数为 9×6 ，所以我们需要构造这些对角点在现
实世界中的相对位置，我们将这些位置简化设置成整数值，比如说第二行的第1个点就表示为
[0,1,0] , 即第0列第1行（第三位均为0值），第九行的第6个点就表示为[8,5,0]，即第5列第8
行。
（2）设置细化图像角点的终止条件（也就是下面用到的一个参数）。
（3）创建两个列表用来存储目标点与图像点。
（4）使用 glob 读取文件夹中所有棋盘图像文件。
（5）遍历棋盘图像，找到所有角点。（objpoints：目标点，imgpoints：图像点）。
（6）使用 cv2.calibrateCamera() 得到相机校准矩阵和畸变系数。（mtx：相机校准矩阵，
dist：畸变系数）
（7）保存相机校准的结果在“calibration.p”文件中
（8）定义去畸变函数，返回去畸变之后的图像
结果展示：
（1）保存的相机校准结果（内含：相机校准矩阵和畸变系数）
（2）相机校准棋盘测试（棋盘图像校准效果明显）
（3）车道线图像去畸变测试 (效果：观察汽车引擎盖形状的变化)
（三）、图像透视变换（提取车道线信息需要）
# Perspective Transform
def unwarp(img, src, dst):
h, w = img.shape[:2]
# use cv2.getPerspectiveTransform() to get M, the transform matrix, and Minv, the inverse
M = cv2.getPerspectiveTransform(src, dst)
Minv = cv2.getPerspectiveTransform(dst, src)
# use cv2.warpPerspective() to warp your image to a top-down view
warped = cv2.warpPerspective(img, M, (w,h), flags=cv2.INTER_LINEAR)
return warped, M, Minv
h, w = exampleImg_undistort.shape[:2]
# define source and destination points for transform
src = np.float32([(575,464),(707,464), (258,682), (1049,682)])
dst = np.float32([(450,0),(w-450,0),(450,h),(w-450,h)])
exampleImg_unwarp, M, Minv = unwarp(exampleImg_undistort, src, dst)
# Test Code:
# Visualize unwarp
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_undistort)
x = [src[0][0],src[2][0],src[3][0],src[1][0],src[0][0]]
y = [src[0][1],src[2][1],src[3][1],src[1][1],src[0][1]]
ax1.plot(x, y, color='#33cc99', alpha=0.4, linewidth=3, solid_capstyle='round', zorder=2)
ax1.set_ylim([h,0])
ax1.set_xlim([0,w])
ax1.set_title('Undistorted Image', fontsize=30)
ax2.imshow(exampleImg_unwarp)
ax2.set_title('Unwarped Image', fontsize=30)
f.show()
代码讲解：
（1）定义透视变换函数，返回透视变换后的图像、投影矩阵、反投影矩阵
对于车道线图像，通过透视变换可以获得一个相对更加直观的视角（鸟瞰图），在新的视平面中能够更简单
更完美的实现对车道线的识别。
使用 cv2.getPerspectiveTransform() 得到投影矩阵
使用cv2.warpPerspective() 得到反投影矩阵
使用cv2.warpPerspective() 得到透视变换后的图像
（2）cv2.getPerspectiveTransform() 需要两个参数 src 和 dst，他们分别为原图像中能够表示一个矩形
四个点的坐标以及扭曲以后图像的边缘四角在当前图像中的坐标，这两个矩形的坐标不同的相机的数值也不
同，比如说本例中相机分辨率为 1280×720 , 设置[(575,464),(707,464), (258,682), (1049,682)］在图像中构
成一个梯形（如下图中所示），这个梯形在俯视图（鸟瞰图）中将变成一个长方形，我们以这个梯形的高作
为目标图像的高，前后各减去一个偏移，就是我们的目标图像。
说明：这部分的上述偏移的大小设置，作者没有理解，故对dst中的数值设置也就没理解，猜测这个偏移值应
该是需要相机测定后再设置的吧。
结果展示：
（1）去畸变后的图像进行透视变换，得到变换视角后的图像结果
（四）、测试不同色彩空间不同通道下车道线的显示效果
# Visualize multiple color space channels
exampleImg_unwarp_R = exampleImg_unwarp[:,:,0]
exampleImg_unwarp_G = exampleImg_unwarp[:,:,1]
exampleImg_unwarp_B = exampleImg_unwarp[:,:,2]
exampleImg_unwarp_HSV = cv2.cvtColor(exampleImg_unwarp, cv2.COLOR_RGB2HSV)
exampleImg_unwarp_H = exampleImg_unwarp_HSV[:,:,0]
exampleImg_unwarp_S = exampleImg_unwarp_HSV[:,:,1]
exampleImg_unwarp_V = exampleImg_unwarp_HSV[:,:,2]
exampleImg_unwarp_LAB = cv2.cvtColor(exampleImg_unwarp, cv2.COLOR_RGB2Lab)
exampleImg_unwarp_L = exampleImg_unwarp_LAB[:,:,0]
exampleImg_unwarp_A = exampleImg_unwarp_LAB[:,:,1]
exampleImg_unwarp_B2 = exampleImg_unwarp_LAB[:,:,2]
# Test Code:
# Visualize test multiple color space channels
f, axs = plt.subplots(3,3, figsize=(16, 12))
f.subplots_adjust(hspace = .2, wspace=.001)
axs = axs.ravel()
axs[0].imshow(exampleImg_unwarp_R, cmap='gray')
axs[0].set_title('RGB R-channel', fontsize=30)
axs[1].imshow(exampleImg_unwarp_G, cmap='gray')
axs[1].set_title('RGB G-Channel', fontsize=30)
axs[2].imshow(exampleImg_unwarp_B, cmap='gray')
axs[2].set_title('RGB B-channel', fontsize=30)
axs[3].imshow(exampleImg_unwarp_H, cmap='gray')
axs[3].set_title('HSV H-Channel', fontsize=30)
axs[4].imshow(exampleImg_unwarp_S, cmap='gray')
axs[4].set_title('HSV S-channel', fontsize=30)
axs[5].imshow(exampleImg_unwarp_V, cmap='gray')
axs[5].set_title('HSV V-Channel', fontsize=30)
axs[6].imshow(exampleImg_unwarp_L, cmap='gray')
axs[6].set_title('LAB L-channel', fontsize=30)
axs[7].imshow(exampleImg_unwarp_A, cmap='gray')
axs[7].set_title('LAB A-Channel', fontsize=30)
axs[8].imshow(exampleImg_unwarp_B2, cmap='gray')
axs[8].set_title('LAB B-Channel', fontsize=30)
f.show()
结果展示：
（1）车道线在不同色彩空间不同通道下显示结果
（2）车道线的颜色有两种：黄色和白色，在不同的色彩空间不同通道的色彩强烈干不同，我们可以有针对性的
进行色彩阈值化和梯度阈值化，得到效果更好的车道线边缘检测结果。
（五）、色彩阈值化和梯度阈值化
# Sobel Absolute Threshold
def abs_sobel_thresh(img, orient = 'x', thresh_min = 25, thresh_max = 255):
# Apply the following steps to img
# 1) Convert to grayscale === or LAB L channel
gray = (cv2.cvtColor(img, cv2.COLOR_RGB2Lab))[:,:,0]
# 2) Take the derivative in x or y given orient = 'x' or 'y'
sobel = cv2.Sobel(gray, cv2.CV_64F, orient=='x', orient=='y')
# 3) Take the absolute value of the derivative or gradient
abs_sobel = np.absolute(sobel)
# 4) Scale to 8-bit (0 - 255) then convert to type = np.uint8
scaled_sobel = np.uint8(255 * abs_sobel/np.max(abs_sobel))
# 5) Create a mask of 1's where the scaled gradient magnitude is > thresh_min and < thresh_max
sxbinary = np.zeros_like(scaled_sobel)
sxbinary[(scaled_sobel >= thresh_min) & (scaled_sobel <= thresh_max)] = 1
# 6) Return this mask as your binary_output image
return sxbinary
# Visualize Sobel absolute threshold on example image
def update(min_thresh, max_thresh):
exampleImg_sobelAbs = abs_sobel_thresh(exampleImg_unwarp, 'x', min_thresh, max_thresh)
# Visualize sobel absolute threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_sobelAbs, cmap='gray')
ax2.set_title('Sobel Absolute', fontsize=30)
f.show()
interact(update, min_thresh=(0,255), max_thresh=(0,255))
# Sobel Magnitude Threshold
def mag_thresh(img, sobel_kernel=25, mag_thresh=(25, 255)):
# Apply the following steps to img
# 1) Convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# 2) Take the gradient in x and y separately
sobelx = cv2.Sobel(gray, cv2.CV_64F, 1, 0)
sobely = cv2.Sobel(gray, cv2.CV_64F, 0, 1)
# 3) Calculate the magnitude
mag_sobel = np.sqrt(np.square(sobelx) + np.square(sobely))
# 4) Scale to 8-bit (0 - 255) and convert to type = np.uint8
scaled_sobel = np.uint8(255*mag_sobel/np.max(mag_sobel))
# 5) Create a binary mask where mag thresholds are met
sxbinary = np.zeros_like(scaled_sobel)
sxbinary[(scaled_sobel >= mag_thresh[0]) & (scaled_sobel <= mag_thresh[1])] = 1
# 6) Return this mask as your binary_output image
return sxbinary
# Visualize Sobel magnitude threshold on example image
def update(kernel_size, min_thresh, max_thresh):
exampleImg_sobelMag = mag_thresh(exampleImg_unwarp, kernel_size, (min_thresh, max_thresh))
# Visualize sobel magnitude threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_sobelMag, cmap='gray')
ax2.set_title('Sobel Magnitude', fontsize=30)
f.show()
interact(update, kernel_size=(1,31,2), min_thresh=(0,255), max_thresh=(0,255))
# Sobel Direction Threshold
def dir_thresh(img, sobel_kernel=7, thresh=(0, 0.09)):
# Apply the following steps to img
# 1) Convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# 2) Take the gradient in x and y separately
sobelx = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=sobel_kernel)
sobely = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=sobel_kernel)
# 3) Take the absolute value of the x and y gradients
abs_sobelx = np.absolute(sobelx)
abs_sobely = np.absolute(sobely)
# 4) Use np.arctan2(abs_sobely, abs_sobelx) to calculate the direction of the gradient
grad_dir = np.arctan2(abs_sobely, abs_sobelx)
# 5) Create a binary mask where direction thresholds are met
binary_output = np.zeros_like(grad_dir)
binary_output[(grad_dir >= thresh[0]) & (grad_dir <= thresh[1])] = 1
# 6) Return this mask as your binary_output image
return binary_output
# Visualize Sobel direction threshold on example image
def update(kernel_size, min_thresh, max_thresh):
exampleImg_sobelDir = dir_thresh(exampleImg_unwarp, kernel_size, (min_thresh, max_thresh))
# Visualize sobel direction threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_sobelDir, cmap='gray')
ax2.set_title('Sobel Direction', fontsize=30)
f.show()
interact(update, kernel_size=(1,31,2), min_thresh=(0,np.pi/2,0.01), max_thresh=(0,np.pi/2,0.01))
# Visualize Sobel Magnitude + Direction Threshold on example image
def update(mag_kernel_size, mag_min_thresh, mag_max_thresh, dir_kernel_size, dir_min_thresh, dir_max_thresh):
exampleImg_sobelMag2 = mag_thresh(exampleImg_unwarp, mag_kernel_size, (mag_min_thresh, mag_max_thresh))
exampleImg_sobelDir2 = dir_thresh(exampleImg_unwarp, dir_kernel_size, (dir_min_thresh, dir_max_thresh))
combined = np.zeros_like(exampleImg_sobelMag2)
combined[((exampleImg_sobelMag2 == 1) & (exampleImg_sobelDir2 == 1))] = 1
# Visualize sobel magnitude + direction threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(combined, cmap='gray')
ax2.set_title('Sobel Magnitude + Direction', fontsize=30)
f.show()
interact(update, mag_kernel_size=(1,31,2), mag_min_thresh=(0,255), mag_max_thresh=(0,255), dir_kernel_size=(1,31,2), dir_min_thresh=(0,np.pi/2,0.01), dir_max_thresh=(0,np.pi/2,0.01))
# HLS S-Channel Threshold
def hls_sthresh(img, thresh=(125, 255)):
# 1) Convert to HLS color space
hls = cv2.cvtColor(img, cv2.COLOR_RGB2HLS)
# 2) Apply a threshold to the S channel
binary_output = np.zeros_like(hls[:,:,2])
binary_output[(hls[:,:,2] > thresh[0]) & (hls[:,:,2] <= thresh[1])] = 1
# 3) Return a binary image of threshold result
return binary_output
# Visualize HLS S-Channel threshold on example image
def update(min_thresh, max_thresh):
exampleImg_SThresh = hls_sthresh(exampleImg_unwarp, (min_thresh, max_thresh))
# Visualize hls s-channel threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_SThresh, cmap='gray')
ax2.set_title('HLS S-Channel', fontsize=30)
f.show()
interact(update, min_thresh=(0,255), max_thresh=(0,255))
# HLS L-Channel Threshold
def hls_lthresh(img, thresh=(220, 255)):
# 1) Convert to HLS color space
hls = cv2.cvtColor(img, cv2.COLOR_RGB2HLS)
hls_l = hls[:,:,1]
hls_l = hls_l*(255/np.max(hls_l))
# 2) Apply a threshold to the L channel
binary_output = np.zeros_like(hls_l)
binary_output[(hls_l > thresh[0]) & (hls_l <= thresh[1])] = 1
# 3) Return a binary image of threshold result
return binary_output
# Visualize HLS L-Channel threshold on example image
def update(min_thresh, max_thresh):
exampleImg_LThresh = hls_lthresh(exampleImg_unwarp, (min_thresh, max_thresh))
# Visualize hls l-channel threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_LThresh, cmap='gray')
ax2.set_title('HLS L-Channel', fontsize=30)
f.show()
#interact(update,min_thresh=(0,255), max_thresh=(0,255))
# LAB B-Channel Threshold
def lab_bthresh(img, thresh=(190,255)):
# 1) Convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_RGB2Lab)
lab_b = lab[:,:,2]
# don't normalize if there are no yellows in the image
if np.max(lab_b) > 175:
lab_b = lab_b*(255/np.max(lab_b))
# 2) Apply a threshold to the L channel
binary_output = np.zeros_like(lab_b)
binary_output[((lab_b > thresh[0]) & (lab_b <= thresh[1]))] = 1
# 3) Return a binary image of threshold result
return binary_output
# Visualize LAB B-Channel threshold on example image
def update(min_b_thresh, max_b_thresh):
exampleImg_LBThresh = lab_bthresh(exampleImg_unwarp, (min_b_thresh, max_b_thresh))
# Visualize LAB B threshold
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
f.subplots_adjust(hspace = .2, wspace=.05)
ax1.imshow(exampleImg_unwarp)
ax1.set_title('Unwarped Image', fontsize=30)
ax2.imshow(exampleImg_LBThresh, cmap='gray')
ax2.set_title('LAB B-channel', fontsize=30)
f.show()
interact(update,min_b_thresh=(0,255),max_b_thresh=(0,255))
# Define the complete image processing pipeline,
# reads raw image and returns binary image with lane lines identified
def pipeline(img):
# Undistort
img_undistort = undistort(img)
# Perspective Transform
img_unwarp, M, Minv = unwarp(img_undistort, src, dst)
# Sobel Absolute (using default parameters)
#img_sobelAbs = abs_sobel_thresh(img_unwarp)
# Sobel Magnitude (using default parameters)
#img_sobelMag = mag_thresh(img_unwarp)
# Sobel Direction (using default parameters)
#img_sobelDir = dir_thresh(img_unwarp)
# HLS S-channel Threshold (using default parameters)
#img_SThresh = hls_sthresh(img_unwarp)
# HLS L-channel Threshold (using default parameters)
img_LThresh = hls_lthresh(img_unwarp)
# Lab B-channel Threshold (using default parameters)
img_BThresh = lab_bthresh(img_unwarp)
# Combine HLS and Lab B channel thresholds
combined = np.zeros_like(img_BThresh)
combined[(img_LThresh == 1) | (img_BThresh == 1)] = 1
return combined,Minv
# Run Pipeline on All Test Images
images = glob.glob('./test_images/*.jpg')
# Set up plot
fig, axs = plt.subplots(len(images),2, figsize=(10, 20))
fig.subplots_adjust(hspace = .2, wspace=.001)
axs = axs.ravel()
i = 0
for image in images:
img = cv2.imread(image)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img_bin, Minv = pipeline(img)
axs[i].imshow(img)
axs[i].axis('off')
i += 1
axs[i].imshow(img_bin, cmap='gray')
axs[i].axis('off')
i += 1
fig.show()
代码讲解：
（1）定义管道函数
获取去畸变图像
获取透视变换图像
HLS 色彩空间 L 通道 + LAB 色彩空间 B 通道色彩阈值化
（2）使用 glob 读取本地 test_images 文件夹中所有车道线图像测试
结果展示：
（1）Sobel Absolute Threshold
色彩空间：LAB；
通道：R；
卷积方向：X；因为 x 方向的索贝尔算子倾向于检测垂直方向的边缘，在车道线检测问题中，我们关注的对象（车道线）往往正是垂直方向的线，同时我们也希望这个操作能过滤到一些水平方向的线。
sobel算子大小：默认；
操作：abs_sobel = np.absolute(sobel)
（2）Sobel Magnitude Threshold
色彩空间：RGB；
卷积方向：(X,Y)；
sobel算子大小：sobel_kernel；
操作：mag_sobel = np.sqrt(np.square(sobelx) + np.square(sobely))
（3）Sobel Direction Threshold
色彩空间：RGB；
卷积方向：(X,Y)；
sobel算子大小：sobel_kernel；
操作：grad_dir = np.arctan2(abs_sobely, abs_sobelx)
（4）Sobel Magnitude + Direction Threshold
色彩空间：RGB；
卷积方向：(X,Y)；
sobel算子大小：sobel_kernel；
（5）HLS S-Channel Threshold
色彩空间：HLS；
通道：S；
（6）HLS L-Channel Threshold
色彩空间：HLS；
通道：L；
（7）LAB B-Channel Threshold
色彩空间：LAB；
通道：B；
（8）HLS-L+ LAB-B channel Threshold
（六）、使用滑动窗口拟合车道线多项式系数
# Sliding Window Polyfit
# Define method to fit polynomial to binary image with lines extracted, using sliding window
def sliding_window_polyfit(img):
# Take a histogram of the bottom half of the image
histogram = np.sum(img[img.shape[0]//2:,:], axis=0)
# Find the peak of the left and right halves of the histogram
# These will be the starting point for the left and right lines
midpoint = np.int(histogram.shape[0]//2)
quarter_point = np.int(midpoint//2)
# Previously the left/right base was the max of the left/right half of the histogram
# this changes it so that only a quarter of the histogram (directly to the left/right) is considered
leftx_base = np.argmax( histogram[quarter_point:midpoint] ) + quarter_point
rightx_base = np.argmax( histogram[midpoint:(midpoint + quarter_point)] ) + midpoint
# Choose the number of sliding windows
nwindows = 10
# Set height of windows
window_height = np.int(img.shape[0]/nwindows)
# Identify the x and y positions of all nonzero pixels in the image
nonzero = img.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
# Current positions to be updated for each window
leftx_current = leftx_base
rightx_current = rightx_base
# Set the width of the windows +/- margin
margin = 80
# Set minimum number of pixels found to recenter window
minpix = 40
# Create empty lists to receive left and right lane pixel indices
left_lane_inds = []
right_lane_inds = []
# Rectangle data for visualization
rectangle_data = []
# Step through the windows one by one
for window in range(nwindows):
# Identify window boundaries in x and y (and right and left)
win_y_low = img.shape[0] - (window + 1) * window_height
win_y_high = img.shape[0] - window * window_height
win_xleft_low = leftx_current - margin
win_xleft_high = leftx_current + margin
win_xright_low = rightx_current - margin
win_xright_high = rightx_current + margin
rectangle_data.append((win_y_low, win_y_high, win_xleft_low, win_xleft_high, win_xright_low, win_xright_high))
# Identify the nonzero pixels in x and y within the window
good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0]
good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0]
# Append these indices to the lists
left_lane_inds.append(good_left_inds)
right_lane_inds.append(good_right_inds)
# If you found > minpix pixels, recenter next window on their mean position
if len(good_left_inds) > minpix:
leftx_current = np.int(np.mean(nonzerox[good_left_inds]))
if len(good_right_inds) > minpix:
rightx_current = np.int(np.mean(nonzerox[good_right_inds]))
# Concatenate the arrays of indices
left_lane_inds = np.concatenate(left_lane_inds)
right_lane_inds = np.concatenate(right_lane_inds)
# Extract left and right line pixel positions
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
left_fit, right_fit = (None, None)
# Fit a second order polynomial to each
if len(leftx) != 0:
left_fit = np.polyfit(lefty, leftx, 2)
if len(rightx) != 0:
right_fit = np.polyfit(righty, rightx, 2)
visualization_data = (rectangle_data, histogram)
return left_fit, right_fit, left_lane_inds, right_lane_inds, visualization_data
# visualize the result on example image
exampleImg = cv2.imread('./test_images/test2.jpg')
exampleImg = cv2.cvtColor(exampleImg, cv2.COLOR_BGR2RGB)
exampleImg_bin, Minv = pipeline(exampleImg)
left_fit, right_fit, left_lane_inds, right_lane_inds, visualization_data = sliding_window_polyfit(exampleImg_bin)
rectangles = visualization_data[0]
histogram = visualization_data[1]
# Create an output image to draw on and visualize the result
out_img = np.uint8(np.dstack((exampleImg_bin, exampleImg_bin, exampleImg_bin))*255)
# Generate x and y values for plotting
ploty = np.linspace(0, exampleImg_bin.shape[0]-1, exampleImg_bin.shape[0] )
left_fitx = left_fit[0]*ploty**2 + left_fit[1]*ploty + left_fit[2]
right_fitx = right_fit[0]*ploty**2 + right_fit[1]*ploty + right_fit[2]
for rect in rectangles:
# Draw the windows on the visualization image
cv2.rectangle(out_img,(rect[2],rect[0]),(rect[3],rect[1]),(0,255,0), 2)
cv2.rectangle(out_img,(rect[4],rect[0]),(rect[5],rect[1]),(0,255,0), 2)
# Identify the x and y positions of all nonzero pixels in the image
nonzero = exampleImg_bin.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
out_img[nonzeroy[left_lane_inds], nonzerox[left_lane_inds]] = [255, 0, 0]
out_img[nonzeroy[right_lane_inds], nonzerox[right_lane_inds]] = [100, 200, 255]
plt.imshow(out_img)
plt.plot(left_fitx, ploty, color='yellow')
plt.plot(right_fitx, ploty, color='yellow')
plt.xlim(0, 1280)
plt.ylim(720, 0)
plt.show()
# Polyfit Using Fit from Previous Frame
# Define method to fit polynomial to binary image based upon a previous fit (chronologically speaking);
def polyfit_using_prev_fit(binary_warped, left_fit_prev, right_fit_prev):
nonzero = binary_warped.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
margin = 80
left_lane_inds = ((nonzerox > (left_fit_prev[0]*(nonzeroy**2) + left_fit_prev[1]*nonzeroy + left_fit_prev[2] - margin)) &
(nonzerox < (left_fit_prev[0]*(nonzeroy**2) + left_fit_prev[1]*nonzeroy + left_fit_prev[2] + margin)))
right_lane_inds = ((nonzerox > (right_fit_prev[0]*(nonzeroy**2) + right_fit_prev[1]*nonzeroy + right_fit_prev[2] - margin)) &
(nonzerox < (right_fit_prev[0]*(nonzeroy**2) + right_fit_prev[1]*nonzeroy + right_fit_prev[2] + margin)))
# Again, extract left and right line pixel positions
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
left_fit_new, right_fit_new = (None, None)
if len(leftx) != 0:
# Fit a second order polynomial to each
left_fit_new = np.polyfit(lefty, leftx, 2)
if len(rightx) != 0:
right_fit_new = np.polyfit(righty, rightx, 2)
return left_fit_new, right_fit_new, left_lane_inds, right_lane_inds
# visualize the result on example image
exampleImg2 = cv2.imread('./test_images/test5.jpg')
exampleImg2 = cv2.cvtColor(exampleImg2, cv2.COLOR_BGR2RGB)
exampleImg2_bin, Minv = pipeline(exampleImg2)
margin = 80
left_fit2, right_fit2, left_lane_inds2, right_lane_inds2 = polyfit_using_prev_fit(exampleImg2_bin, left_fit, right_fit)
# Generate x and y values for plotting
ploty = np.linspace(0, exampleImg2_bin.shape[0]-1, exampleImg2_bin.shape[0] )
left_fitx = left_fit[0]*ploty**2 + left_fit[1]*ploty + left_fit[2]
right_fitx = right_fit[0]*ploty**2 + right_fit[1]*ploty + right_fit[2]
left_fitx2 = left_fit2[0]*ploty**2 + left_fit2[1]*ploty + left_fit2[2]
right_fitx2 = right_fit2[0]*ploty**2 + right_fit2[1]*ploty + right_fit2[2]
# Create an image to draw on and an image to show the selection window
out_img = np.uint8(np.dstack((exampleImg2_bin, exampleImg2_bin, exampleImg2_bin))*255)
window_img = np.zeros_like(out_img)
# Color in left and right line pixels
nonzero = exampleImg2_bin.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
out_img[nonzeroy[left_lane_inds2], nonzerox[left_lane_inds2]] = [255, 0, 0]
out_img[nonzeroy[right_lane_inds2], nonzerox[right_lane_inds2]] = [0, 0, 255]
# Generate a polygon to illustrate the search window area (OLD FIT)
# And recast the x and y points into usable format for cv2.fillPoly()
left_line_window1 = np.array([np.transpose(np.vstack([left_fitx-margin, ploty]))])
left_line_window2 = np.array([np.flipud(np.transpose(np.vstack([left_fitx+margin, ploty])))])
left_line_pts = np.hstack((left_line_window1, left_line_window2))
right_line_window1 = np.array([np.transpose(np.vstack([right_fitx-margin, ploty]))])
right_line_window2 = np.array([np.flipud(np.transpose(np.vstack([right_fitx+margin, ploty])))])
right_line_pts = np.hstack((right_line_window1, right_line_window2))
# Draw the lane onto the warped blank image
cv2.fillPoly(window_img, np.int_([left_line_pts]), (0,255, 0))
cv2.fillPoly(window_img, np.int_([right_line_pts]), (0,255, 0))
result = cv2.addWeighted(out_img, 1, window_img, 0.3, 0)
plt.imshow(result)
plt.plot(left_fitx2, ploty, color='yellow')
plt.plot(right_fitx2, ploty, color='yellow')
plt.xlim(0, 1280)
plt.ylim(720, 0)
plt.show()
代码解释：
（1）定义滑动窗口拟合函数
首先需要确定左右两条车道线的大致位置，通过将图片像素按列相加（这里截取了图像的下半部分，因为
车道线往往只存在于下半部分，截取可过滤掉一些无关像素，同时可缩减计算量），那么histogram中的最大
值点就可能是车道线的起点。找出图片中间点左右的峰值，即为车道线左右起点。
（2）计算图像中所有非零像素点，存储索引值在nonzero中（二维的列表）。
（3）创建两个列表存放车道线非零像素点索引值：left_lane_inds 、right_lane_inds。
（3）遍历nwindows，实现自底向上识别滑动窗口中的非零像素点，返回索引列表：good_left_inds 、
good_right_inds 。
（4）如果像素点的数量大于设定的阈值，那么就以这些点的均值作为下一个滑动窗口的中心。
（5）求取左右车道线二次多项式的拟合系数。
（6）返回左右车道线拟合系数、左右车道线起始点索引列表、滑动窗口坐标点 + 图片像素按列相加的列表。
（7）定义使用前一帧图像车道线二次多项式系数函数。（这是基于连续帧图像车道线变化一般不大，不用每次都去使用滑动窗口拟合车道线二次多项式，减少代码计算量。）
（8）返回新的左右车道线拟合系数、左右车道线起始点索引列表。
结果展示：
（1）使用滑动窗口拟合车道线多项式结果
（2）使用前一帧图像车道线二次多项式系数拟合车道线多项式结果
（七）、基于车道线二次多项式拟合曲线计算出车道线曲率和车辆偏离车道线中心线距离
# Radius of Curvature and Distance from Lane Center Calculation
def calc_curv_rad_and_center_dist(bin_img, l_fit, r_fit, l_lane_inds, r_lane_inds):
# Define conversions in x and y from pixels space to meters
ym_per_pix = 3.048/100 # meters per pixel in y dimension, lane line is 10 ft = 3.048 meters
xm_per_pix = 3.7/378 # meters per pixel in x dimension, lane width is 12 ft = 3.7 meters
left_curverad, right_curverad, center_dist = (0, 0, 0)
# Define y-value where we want radius of curvature
# I'll choose the maximum y-value, corresponding to the bottom of the image
h = bin_img.shape[0]
ploty = np.linspace(0, h-1, h)
y_eval = np.max(ploty)
# Identify the x and y positions of all nonzero pixels in the image
nonzero = bin_img.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
# Again, extract left and right line pixel positions
leftx = nonzerox[l_lane_inds]
lefty = nonzeroy[l_lane_inds]
rightx = nonzerox[r_lane_inds]
righty = nonzeroy[r_lane_inds]
if len(leftx) != 0 and len(rightx) != 0:
# Fit new polynomials to x,y in world space
left_fit_cr = np.polyfit(lefty*ym_per_pix, leftx*xm_per_pix, 2)
right_fit_cr = np.polyfit(righty*ym_per_pix, rightx*xm_per_pix, 2)
# Calculate the new radii of curvature
left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*ym_per_pix + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0])
right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*ym_per_pix + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0])
# Now our radius of curvature is in meters
# Distance from center is image x midpoint - mean of l_fit and r_fit intercepts
if r_fit is not None and l_fit is not None:
car_position = bin_img.shape[1]/2
l_fit_x_int = l_fit[0]*h**2 + l_fit[1]*h + l_fit[2]
r_fit_x_int = r_fit[0]*h**2 + r_fit[1]*h + r_fit[2]
lane_center_position = (r_fit_x_int + l_fit_x_int) /2
center_dist = (car_position - lane_center_position) * xm_per_pix
return left_curverad, right_curverad, center_dist
# Calculate curvature radius and distance from center for example image
rad_l, rad_r, d_center = calc_curv_rad_and_center_dist(exampleImg_bin, left_fit, right_fit, left_lane_inds, right_lane_inds)
print('Radius of curvature for example:', rad_l, 'm,', rad_r, 'm')
print('Distance from lane center for example:', d_center, 'm')
# Draw the Detected Lane Back onto the Original Image
def draw_lane(original_img, binary_img, l_fit, r_fit, Minv):
new_img = np.copy(original_img)
if l_fit is None or r_fit is None:
return original_img
# Create an image to draw the lines on
warp_zero = np.zeros_like(binary_img).astype(np.uint8)
color_warp = np.dstack((warp_zero, warp_zero, warp_zero))
h,w = binary_img.shape
ploty = np.linspace(0, h-1, num=h)# to cover same y-range as image
left_fitx = l_fit[0]*ploty**2 + l_fit[1]*ploty + l_fit[2]
right_fitx = r_fit[0]*ploty**2 + r_fit[1]*ploty + r_fit[2]
# Recast the x and y points into usable format for cv2.fillPoly()
pts_left = np.array([np.transpose(np.vstack([left_fitx, ploty]))])
pts_right = np.array([np.flipud(np.transpose(np.vstack([right_fitx, ploty])))])
pts = np.hstack((pts_left, pts_right))
# Draw the lane onto the warped blank image
cv2.fillPoly(color_warp, np.int_([pts]), (0,255, 0))
cv2.polylines(color_warp, np.int32([pts_left]), isClosed=False, color=(255,0,255), thickness=15)
cv2.polylines(color_warp, np.int32([pts_right]), isClosed=False, color=(0,255,255), thickness=15)
# Warp the blank back to original image space using inverse perspective matrix (Minv)
newwarp = cv2.warpPerspective(color_warp, Minv, (w, h))
# Combine the result with the original image
result = cv2.addWeighted(new_img, 1, newwarp, 0.5, 0)
return result
# Draw detected lane onto example image
exampleImg_out1 = draw_lane(exampleImg, exampleImg_bin, left_fit, right_fit, Minv)
plt.imshow(exampleImg_out1)
plt.show()
# Draw Curvature Radius and Distance from Center Data onto the Original Image
def draw_data(original_img, curv_rad, center_dist):
new_img = np.copy(original_img)
h = new_img.shape[0]
font = cv2.FONT_HERSHEY_DUPLEX
text = 'Curve radius: ' + '{:04.2f}'.format(curv_rad) + 'm'
cv2.putText(new_img, text, (40,70), font, 1.5, (200,255,155), 2, cv2.LINE_AA)
direction = ''
if center_dist > 0:
direction = 'right'
elif center_dist < 0:
direction = 'left'
abs_center_dist = abs(center_dist)
text = '{:04.3f}'.format(abs_center_dist) + 'm ' + direction + ' of center'
cv2.putText(new_img, text, (40,120), font, 1.5, (200,255,155), 2, cv2.LINE_AA)
return new_img
# Write data onto example image
exampleImg_out2 = draw_data(exampleImg_out1, (rad_l+rad_r)/2, d_center)
plt.imshow(exampleImg_out2)
plt.show()
# Define a Line Class for Storing Data
# Define a class to receive the characteristics of each line detection
class Line():
def __init__(self):
# was the line detected in the last iteration?
self.detected = False
# x values of the last n fits of the line
self.recent_xfitted = []
# average x values of the fitted line over the last n iterations
self.bestx = None
# polynomial coefficients averaged over the last n iterations
self.best_fit = None
# polynomial coefficients for the most recent fit
self.current_fit = []
# radius of curvature of the line in some units
self.radius_of_curvature = None
# distance in meters of vehicle center from the line
self.line_base_pos = None
# difference in fit coefficients between last and new fits
self.diffs = np.array([0,0,0], dtype='float')
# number of detected pixels
self.px_count = None
def add_fit(self, fit, inds):
# add a found fit to the line, up to n
if fit is not None:
if self.best_fit is not None:
# if we have a best fit, see how this new fit compares
self.diffs = abs(fit-self.best_fit)
if (self.diffs[0] > 0.001 or \
self.diffs[1] > 1.0 or \
self.diffs[2] > 100.) and \
len(self.current_fit) > 0:
# bad fit! abort! abort! ... well, unless there are no fits in the current_fit queue, then we'll take it
self.detected = False
else:
self.detected = True
self.px_count = np.count_nonzero(inds)
self.current_fit.append(fit)
if len(self.current_fit) > 5:
# throw out old fits, keep newest n
self.current_fit = self.current_fit[len(self.current_fit)-5:]
self.best_fit = np.average(self.current_fit, axis=0)
# or remove one from the history, if not found
else:
self.detected = False
if len(self.current_fit) > 0:
# throw out oldest fit
self.current_fit = self.current_fit[:len(self.current_fit)-1]
if len(self.current_fit) > 0:
# if there are still any fits in the queue, best_fit is their average
self.best_fit = np.average(self.current_fit, axis=0)
# Define Complete Image Processing Pipeline
def process_image(img):
new_img = np.copy(img)
img_bin, Minv = pipeline(new_img)
# if both left and right lines were detected last frame, use polyfit_using_prev_fit, otherwise use sliding window
if not l_line.detected or not r_line.detected:
l_fit, r_fit, l_lane_inds, r_lane_inds, _ = sliding_window_polyfit(img_bin)
else:
l_fit, r_fit, l_lane_inds, r_lane_inds = polyfit_using_prev_fit(img_bin, l_line.best_fit, r_line.best_fit)
# invalidate both fits if the difference in their x-intercepts isn't around 350 px (+/- 100 px)
if l_fit is not None and r_fit is not None:
# calculate x-intercept (bottom of image, x=image_height) for fits
h = img.shape[0]
l_fit_x_int = l_fit[0]*h**2 + l_fit[1]*h + l_fit[2]
r_fit_x_int = r_fit[0]*h**2 + r_fit[1]*h + r_fit[2]
x_int_diff = abs(r_fit_x_int - l_fit_x_int)
if abs(350 - x_int_diff) > 100:
l_fit = None
r_fit = None
l_line.add_fit(l_fit, l_lane_inds)
r_line.add_fit(r_fit, r_lane_inds)
# draw the current best fit if it exists
if l_line.best_fit is not None and r_line.best_fit is not None:
img_out1 = draw_lane(new_img, img_bin, l_line.best_fit, r_line.best_fit, Minv)
rad_l, rad_r, d_center = calc_curv_rad_and_center_dist(img_bin, l_line.best_fit, r_line.best_fit,
l_lane_inds, r_lane_inds)
img_out = draw_data(img_out1, (rad_l+rad_r)/2, d_center)
else:
img_out = new_img
diagnostic_output = False
if diagnostic_output:
# put together multi-view output
diag_img = np.zeros((720,1280,3), dtype=np.uint8)
# original output (top left)
diag_img[0:360,0:640,:] = cv2.resize(img_out,(640,360))
# binary overhead view (top right)
img_bin = np.dstack((img_bin*255, img_bin*255, img_bin*255))
resized_img_bin = cv2.resize(img_bin,(640,360))
diag_img[0:360,640:1280, :] = resized_img_bin
# overhead with all fits added (bottom right)
img_bin_fit = np.copy(img_bin)
for i, fit in enumerate(l_line.current_fit):
img_bin_fit = plot_fit_onto_img(img_bin_fit, fit, (20*i+100,0,20*i+100))
for i, fit in enumerate(r_line.current_fit):
img_bin_fit = plot_fit_onto_img(img_bin_fit, fit, (0,20*i+100,20*i+100))
img_bin_fit = plot_fit_onto_img(img_bin_fit, l_line.best_fit, (255,255,0))
img_bin_fit = plot_fit_onto_img(img_bin_fit, r_line.best_fit, (255,255,0))
diag_img[360:720,640:1280,:] = cv2.resize(img_bin_fit,(640,360))
# diagnostic data (bottom left)
color_ok = (200,255,155)
color_bad = (255,155,155)
font = cv2.FONT_HERSHEY_DUPLEX
if l_fit is not None:
text = 'This fit L: ' + ' {:0.6f}'.format(l_fit[0]) + \
' {:0.6f}'.format(l_fit[1]) + \
' {:0.6f}'.format(l_fit[2])
else:
text = 'This fit L: None'
cv2.putText(diag_img, text, (40,380), font, .5, color_ok, 1, cv2.LINE_AA)
if r_fit is not None:
text = 'This fit R: ' + ' {:0.6f}'.format(r_fit[0]) + \
' {:0.6f}'.format(r_fit[1]) + \
' {:0.6f}'.format(r_fit[2])
else:
text = 'This fit R: None'
cv2.putText(diag_img, text, (40,400), font, .5, color_ok, 1, cv2.LINE_AA)
text = 'Best fit L: ' + ' {:0.6f}'.format(l_line.best_fit[0]) + \
' {:0.6f}'.format(l_line.best_fit[1]) + \
' {:0.6f}'.format(l_line.best_fit[2])
cv2.putText(diag_img, text, (40,440), font, .5, color_ok, 1, cv2.LINE_AA)
text = 'Best fit R: ' + ' {:0.6f}'.format(r_line.best_fit[0]) + \
' {:0.6f}'.format(r_line.best_fit[1]) + \
' {:0.6f}'.format(r_line.best_fit[2])
cv2.putText(diag_img, text, (40,460), font, .5, color_ok, 1, cv2.LINE_AA)
text = 'Diffs L: ' + ' {:0.6f}'.format(l_line.diffs[0]) + \
' {:0.6f}'.format(l_line.diffs[1]) + \
' {:0.6f}'.format(l_line.diffs[2])
if l_line.diffs[0] > 0.001 or \
l_line.diffs[1] > 1.0 or \
l_line.diffs[2] > 100.:
diffs_color = color_bad
else:
diffs_color = color_ok
cv2.putText(diag_img, text, (40,500), font, .5, diffs_color, 1, cv2.LINE_AA)
text = 'Diffs R: ' + ' {:0.6f}'.format(r_line.diffs[0]) + \
' {:0.6f}'.format(r_line.diffs[1]) + \
' {:0.6f}'.format(r_line.diffs[2])
if r_line.diffs[0] > 0.001 or \
r_line.diffs[1] > 1.0 or \
r_line.diffs[2] > 100.:
diffs_color = color_bad
else:
diffs_color = color_ok
cv2.putText(diag_img, text, (40,520), font, .5, diffs_color, 1, cv2.LINE_AA)
text = 'Good fit count L:' + str(len(l_line.current_fit))
cv2.putText(diag_img, text, (40,560), font, .5, color_ok, 1, cv2.LINE_AA)
text = 'Good fit count R:' + str(len(r_line.current_fit))
cv2.putText(diag_img, text, (40,580), font, .5, color_ok, 1, cv2.LINE_AA)
img_out = diag_img
return img_out
# Method for plotting a fit on a binary image - diagnostic purposes
def plot_fit_onto_img(img, fit, plot_color):
if fit is None:
return img
new_img = np.copy(img)
h = new_img.shape[0]
ploty = np.linspace(0, h-1, h)
plotx = fit[0]*ploty**2 + fit[1]*ploty + fit[2]
pts = np.array([np.transpose(np.vstack([plotx, ploty]))])
cv2.polylines(new_img, np.int32([pts]), isClosed=False, color=plot_color, thickness=8)
return new_img
代码讲解：
（1）定义计算曲率半径和距车道中心的距离函数。
首先定义x和y从像素空间到现实世界“米”的转换，
曲率半径为：利用新的拟合系数，计算如下公式：
left_curverad = ((1 + (2left_fit_cr[0]y_evalym_per_pix + left_fit_cr[1])**2)**1.5) /
np.absolute(2left_fit_cr[0])
right_curverad = ((1 + (2right_fit_cr[0]y_evalym_per_pix + right_fit_cr[1])**2)**1.5) /
np.absolute(2right_fit_cr[0]
距中心的距离为：图像的中点 减去 左右车道线的截距的均值，再 乘以 x 从像素空间到米空间的换算。
返回左右车道线曲率、距车道线中心距离值。
（2）定义将检测到的通道绘制回原始图像函数。
使用cv2.warpPerspective()实现该功能，注意这里需要用到透视变换时保留的反投影映射矩阵。其他代码与
上面的基本大同小异。
（3）定义绘制曲率半径和距离中心位置距离回原始图像函数。
简单的设置了显示文本的格式、位置、字体等，然后把他放到了原始图像固定位置上。
（4）定义用于存储数据的行类，用于接收每一行检测到的特征。
（5）定义完整的图像处理管道函数。
判断是使用滑动窗口函数拟合车道线二次多项式函数，还是使用前一帧图像车道线二次多项式拟合系数函
数。
（6）定义在二进制图像上绘制拟合和车道信息函数。
后面写的不详细，特别是类的内部没有进行讲解，后续如果有时间的话，作者会对该内容进行补充，抱歉。
结果展示：
（1）将检测到的通道绘制回原始图像结果
（2）绘制曲率半径和距离中心位置距离回原始图像结果
（八）、程序应用于测试视频
# Process Project Video
l_line = Line()
r_line = Line()
video_output1 = 'test_videos_output/project_video_output.mp4'
video_input1 = VideoFileClip("test_videos/project_video.mp4")
processed_video = video_input1.fl_image(process_image)
processed_video.write_videofile(video_output1, audio=False)
代码讲解：
（1）定义并初始化左右车道线实例
（2）设置视频保存输出位置
（3）使用VideoFileClip()调用测试视频（在这里我们仅测试并展示了第一个视频，其他并未测试展示）
结果展示：
（1）存储在本地 “test_videos_output” 文件夹中的视频结果：project_video_output.mp4
（2）视频播放（由于作者不知道CSDN是否能上传视频（可以设置视频链接），所以只上传了GIF动态图，且因CSDN限制上传图片大小在5M以内，所以只录制了视频前24秒）。
至此，无人驾驶实战——“车载摄像头行车视频车道线跟踪” 讲解完成，内容较多，阅读时间也较长，能看完的都是棒棒哒，后续继续加油，共勉。
本博客内容主要来自优达学城无人驾驶纳米学位，博客内容不得转载，不得作为他用，仅提供互相学习，谢谢。
1316
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
