NVIDIA 自动驾驶算法的keras（TensorFlow）实现

NVIDIA End-to-End 自动驾驶深度学习算法

之前一篇文章提到的要用tf来实现，不过那时候写的代码版本已经旧了，所以干脆用keras重写一个，后端依然使用的是tensorflow。

本文使用Udacity开源的无人驾驶模拟器生成数据并测试效果：

udacity/self-driving-car-sim

如何使用模拟器可以参考：

udacity/CarND-Behavioral-Cloning-P3

先上模型结构图

其实看代码更清晰，不的不说Keras真是一个非常好用的工具，写出来的模型非常干净整洁。

所用到的API

Guide to the Sequential model

Convolutional Layers conv2d - Keras Documentation

Convolutional Layers cropping2d - Keras Documentation

Core Layers flatten - Keras Documentation

Core Layers dense - Keras Documentation

Core Layers lambda - Keras Documentation

Core Layers dropout - Keras Documentation

# 数据载入依赖
import csv
import cv2
import numpy as np

# 模型构建依赖
from keras.models import Sequential
from keras.layers import Conv2D, Cropping2D
from keras.layers import Flatten, Dense, Lambda, Dropout

# 模型数据处理所需要的依赖
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split

# 加载训练数据
LINES = []

# 数据矫正值，用于纠正方向盘角度
CORRECTION = 0.2

# 从数据集中读取必要数据
with open('./data/driving_log.csv') as csvfile:
    READER = csv.reader(csvfile)
    for line in READER:
        LINES.append(line)

# 分组出训练集与校验集
TRAIN_SAMPLES, VALIDATION_SAMPLES = train_test_split(LINES, test_size=0.2)

# 用一个生成器来传入训练数据，避免出现内存不足的情况
def generator(samples, batch_size=32):
    """
    generat training samples
    """
    num_samples = len(samples)
    while 1: # Loop forever so the generator never terminates
        shuffle(samples)
        for offset in range(0, num_samples, batch_size):
            batch_samples = samples[offset:offset+batch_size]

        images = []
        measurements = []
        for batch_sample in batch_samples:
            # Use 3 cameras
            measurement = float(batch_sample[3])
            measurement_left = measurement + CORRECTION
            measurement_right = measurement - CORRECTION

            for i in range(3):
                source_path = batch_sample[i]
                filename = source_path.split('/')[-1]
                current_path = './data/IMG/' + filename
                image_bgr = cv2.imread(current_path)
                # OpenCV的大坑，大坑啊！！！
                image = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
                # image = cv2.resize(image, (80, 160), cv2.INTER_NEAREST)
                images.append(image)
                # Augment image
                images.append(cv2.flip(image, 1))

            measurements.extend([measurement,
                                 measurement*-1,
                                 measurement_left,
                                 measurement_left*-1,
                                 measurement_right,
                                 measurement_right*-1])


            x_train = np.array(images)
            y_train = np.array(measurements)

            yield shuffle(x_train, y_train)

TRAIN_GENERATOR = generator(TRAIN_SAMPLES, batch_size=32)
VALIDATION_GENERATOR = generator(VALIDATION_SAMPLES, batch_size=32)

# 输入帧的参数
ROW, COL, CH = 160, 320, 3

# Model
MODEL = Sequential()
# 执行图像归一化
MODEL.add(Lambda(lambda x: x / 127.5 - 1.0, input_shape=(ROW, COL, CH)))
# 剪裁图片 只保留和道路相关的部分
#MODEL.add(Cropping2D(cropping=((60, 20), (0, 0))))
# 利用卷积层来进行特征提取
MODEL.add(Conv2D(24, 5, strides=(2, 2), activation='relu'))
MODEL.add(Dropout(0.7))
MODEL.add(Conv2D(36, 5, strides=(2, 2), activation='relu'))
MODEL.add(Conv2D(48, 5, strides=(2, 2), activation='relu'))
MODEL.add(Conv2D(64, 3, activation='relu'))
MODEL.add(Conv2D(64, 3, activation='relu'))
# 如果出现过拟合可以添加Dropout
# MODEL.add(Dropout(0.8))
# 全链接层
MODEL.add(Flatten())
MODEL.add(Dense(100))
MODEL.add(Dense(50))
MODEL.add(Dense(10))
MODEL.add(Dense(1))


MODEL.compile(loss='mse', optimizer='adam')
MODEL.fit_generator(TRAIN_GENERATOR,
                    steps_per_epoch=len(TRAIN_SAMPLES),
                    validation_data=VALIDATION_GENERATOR,
                    validation_steps=len(VALIDATION_SAMPLES),
                    epochs=3)

# 保存模型
MODEL.save('model.h5')

然后车子就可以开起来咯：

提高训练效果的注意事项：

1. 使用模拟器生成数据的时候尽量使用摇杆或者鼠标，这样数据比较平滑
2. 尽量保持车在车道中间，如果车技不佳就开慢点。
3. 注意opencv读取图片颜色顺序为BGR，这是一个大坑，一开始训练的时候总往水里开，后来才发现原来是颜色搞反了。