Jetbot examples 之 road_following

Jetbot examples 之 road_following

目前nvidia jetbot wiki中还没有给出循迹demo的讲解。但notebooks中已经更新了roadfollowing的示例。记录下学习过程。以及坑
jetbot wiki：https://github.com/NVIDIA-AI-IOT/jetbot/wiki/Examples

与Collision avoidance类似，road_following也分为三个步骤：

1. 数据采集：data_collection
2. 模型训练：train_model
3. 部署：live_demo

1、数据收集

导入库

导入所有必需的库以进行“数据收集”。使用OpenCV来可视化和保存用标签命名的图像。uuid，datetime之类的库用于为图像命名。

# IPython Libraries for display and widgets
import traitlets
import ipywidgets.widgets as widgets
from IPython.display import display

# Camera and Motor Interface for JetBot
from jetbot import Robot, Camera, bgr8_to_jpeg

# Python basic pakcages for image annotation
from uuid import uuid1
import os
import json
import glob
import datetime
import numpy as np
import cv2
import time

实时显示

camera = Camera()

image_widget = widgets.Image(format='jpeg', width=224, height=224)
target_widget = widgets.Image(format='jpeg', width=224, height=224)

steering_slider = widgets.FloatSlider(min=-1.0, max=1.0, step=0.001, description='steering')
throttle_slider = widgets.FloatSlider(min=-1.0, max=1.0, step=0.001, description='throttle')

def display_steering(camera_image):
    image = np.copy(camera_image)
    steering = steering_slider.value
    throttle = throttle_slider.value
    x = int(steering * 224 / 2 + 112)
    y = int(throttle * 224 / 2 + 112)
    image = cv2.circle(image, (x, y), 8, (0, 255, 0), 3)
    image = cv2.circle(image, (112, 224), 8, (0, 0,255), 3)
    image = cv2.line(image, (x,y), (112,224), (255,0,0), 3)
    jpeg_image = bgr8_to_jpeg(image)
    return jpeg_image

time.sleep(1)
traitlets.dlink((camera, 'value'), (image_widget, 'value'), transform=bgr8_to_jpeg)
traitlets.dlink((camera, 'value'), (target_widget, 'value'), transform=display_steering)

display(widgets.HBox([image_widget, target_widget]), steering_slider, throttle_slider)

Create Gamepad Controller

controller = widgets.Controller(index=0)#这里的index的值根据自己使用的手柄修改
display(controller)

这里的index的值根据自己使用的手柄修改，参考：

Connect Gamepad Controller to Label Images

widgets.jsdlink((controller.axes[0], 'value'), (steering_slider, 'value'))
widgets.jsdlink((controller.axes[1], 'value'), (throttle_slider, 'value'))

这里的controller.axes[0]根据自己的手柄修改，不合适则会报错。

Collect data

这一步代码有错需要修改

DATASET_DIR = 'dataset_xy'

# we have this "try/except" statement because these next functions can throw an error if the directories exist already
try:
    os.makedirs(DATASET_DIR)
except FileExistsError:
    print('Directories not created becasue they already exist')

for b in controller.buttons:
    b.unobserve_all()

count_widget = widgets.IntText(description='count', value=len(glob.glob(os.path.join(DATASET_DIR, '*.jpg'))))

def xy_uuid(x, y):
    return 'xy_%03d_%03d_%s' % (x * 50 + 50, y * 50 + 50, uuid1())

def save_steering(change):
    if change['new']:
        uuid = xy_uuid(steering_slider.value, throttle_slider.value)# 这里已经修改
        image_path = os.path.join(DATASET_DIR, uuid + '.jpg')
        with open(image_path, 'wb') as f:
            f.write(image_widget.value)
        count_widget.value = len(glob.glob(os.path.join(DATASET_DIR, '*.jpg')))

controller.buttons[1].observe(save_steering, names='value')

display(widgets.VBox([
    target_widget,
    count_widget
]))

原文中的第14行、第19行的函数名称未对应。
第19行原本为：

uuid = steering_throttle_uuid(steering_slider.value, throttle_slider.value)

2、train_model

有时间再补充

3、live_demo

有时间再补充

y = (0.5 - xy[1]) / 2.0

需要改为　y = 0.5 -xy[1]/2.0
这点已经在github issue 提出并得到确认