Turtlebot3 Gazebo仿真环境下深度强化学习DQN开发环境构建

1、Anaconda2 安装

 

2、Tensorflow安装

ubuntu18系统anaconda安装tensorflow_qq_39429669的博客-CSDN博客

3、下载并编译源码

本文先使用github中开源的机器学习的源码进行学习，下载编译过程如下

注：可以提前加载到gitee网站，再git clone gitee网址

cd ~/TB3DQN_WS/src/
 
git clone https://github.com/ROBOTIS-GIT/turtlebot3_msgs.git
 
git clone https://github.com/ROBOTIS-GIT/turtlebot3.git
 
git clone https://github.com/ROBOTIS-GIT/turtlebot3_simulations
 
git clone https://github.com/ROBOTIS-GIT/turtlebot3_machine_learning

~/TB3DQN_WS/src/文件夹下只放前两个文件，激活环境，进入工作空间，编译

注：4个文件可以一起编译，第一次编译为了方便确定可能报错的位置，进行分步编译

setconda
source activate tf2
cd TB3DQN_WS/
catkin_make

~/TB3DQN_WS/src/文件夹下放入第3个文件turtlebot3_simulations，编译

catkin_make

 

~/TB3DQN_WS/src/文件夹下放入第4个文件turtlebot3_machine_learning，编译

catkin_make

 

4、设置参数并运行范例

• 设置参数：

打开源码文件 turtlebot3/turtlebot3_description/urdf/turtlebot3_burger.gazebo.xacro，修改一下两处：

<xacro:arg name="laser_visual" default="false"/>   # 如果想看到激光扫描线,设置成 `true`

<scan>
  <horizontal>
    <samples>360</samples>            # 修改成24
    <resolution>1</resolution>
    <min_angle>0.0</min_angle>
    <max_angle>6.28319</max_angle>
  </horizontal>
</scan>

 

• 打开一个终端，启动turtlebot3 gazebo环境等节点：

setconda
source activate tf2
cd TB3DQN_WS/

source ./devel/setup.bash
export TURTLEBOT3_MODEL=burger

roslaunch turtlebot3_gazebo turtlebot3_stage_1.launch

产生报错

xacro: in-order processing became default in ROS Melodic. You can drop the option.
substitution args not supported:  No module named rospkg
when processing file: /home/imu/TB3DQN_WS/src/turtlebot3/turtlebot3_description/urdf/turtlebot3_burger.urdf.xacro
RLException: Invalid <param> tag: Cannot load command parameter [robot_description]: command [['/opt/ros/melodic/lib/xacro/xacro', '--inorder', '/home/imu/TB3DQN_WS/src/turtlebot3/turtlebot3_description/urdf/turtlebot3_burger.urdf.xacro']] returned with code [2]. 

Param xml is <param command="$(find xacro)/xacro --inorder $(find turtlebot3_description)/urdf/turtlebot3_$(arg model).urdf.xacro" name="robot_description"/>
The traceback for the exception was written to the log file

报错原因

缺少rospkg等ROS依赖包

解决方法

pip install -U rosinstall msgpack empy defusedxml netifaces

• 打开另外一个终端，启动DQN算法等节点：

setconda
source activate tf2
cd TB3DQN_WS/

source ./devel/setup.bash
export TURTLEBOT3_MODEL=burger

roslaunch turtlebot3_dqn turtlebot3_dqn_stage_1.launch

产生报错

Traceback (most recent call last):
  File "/home/imu/TB3DQN_WS/src/turtlebot3_machine_learning/turtlebot3_dqn/nodes/turtlebot3_dqn_stage_1", line 31, in <module>
    from keras.models import Sequential, load_model
ImportError: No module named keras.models
[turtlebot3_dqn_stage_1-1] process has died [pid 12944, exit code 1, cmd /home/imu/TB3DQN_WS/src/turtlebot3_machine_learning/turtlebot3_dqn/nodes/turtlebot3_dqn_stage_1 __name:=turtlebot3_dqn_stage_1 __log:=/home/imu/.ros/log/2bc8c2cc-9db4-11ec-814d-fcaa14c35cee/turtlebot3_dqn_stage_1-1.log].
log file: /home/imu/.ros/log/2bc8c2cc-9db4-11ec-814d-fcaa14c35cee/turtlebot3_dqn_stage_1-1*.log
报错原因

缺少keras包

解决方法

下载对应版本的keras，Tensorflow 1.15对应Keras 2.3.1

tensorflow和keras版本对应关系_haosen-CSDN博客

pip install keras==2.3.1

安装成功显示如下，自动安装了scipy

 Installing collected packages: scipy, keras
Successfully installed keras-2.3.1 scipy-1.2.3

• 打开第三个终端，启动数据图形显示节点：

pip install pyqtgraph

roslaunch turtlebot3_dqn result_graph.launch

产生报错

没有pyqt5，安装也安装不上，可能原因是anaconda版本低或者python版本低

https://blog.csdn.net/qq_39429669/article/details/123361466

注：源码范例中提供了turtlebot3_stage_1～turtlebot3_stage_4共4个环境，分别是无障碍、静态障碍、动态障碍、混合障碍环境。

5、训练与结果展示

这是训练约5个半小时后的结果：

 ROS开发笔记（8）——Turtlebot3 Gazebo仿真环境下深度强化学习DQN（Deep Q-Learning）开发环境构建_天涯的专栏-CSDN博客_gazebo 强化学习

Ubuntu20.04安装Cuda10.1+cuDNN10.1+Anaconda(Python3.8建议安装3.7)+Keras2.3.1+tensorflow2.2.0+Pycharm(最新版)_如意的小家的博客-CSDN博客_ubuntu20.04 安装cuda