OpenAI Gym入门与实操（1）

蓝天居士

已于 2023-07-03 10:59:44 修改

阅读量2.2k

点赞数 2

分类专栏： AI 强化学习文章标签：人工智能 OpenAI Gym 强化学习

于 2023-06-30 11:23:15 首次发布

本文链接：https://blog.csdn.net/phmatthaus/article/details/131471404

版权

AI 同时被 2 个专栏收录

49 篇文章 4 订阅

订阅专栏

强化学习

11 篇文章 1 订阅

订阅专栏

本文参考：

OpenAI-Gym入门 - 知乎，

【强化学习】 OpenAI Gym入门：基础组件（Getting Started With OpenAI Gym: The Basic Building Blocks）_iioSnail的博客-CSDN博客

特此致谢。

1. 简介

如果你想开始强化学习，那么OpenAI Gym无疑是实现训练智能体环境的最受欢迎的选择。

OpenAI Gym是强化学习（Reinforcement Learning, RL）的一个库，其可以帮你方便的验证你的强化学习算法的性能，其中提供了许多Enviorment。目前是学术界公认的benchmark。

Gym是一个开源Python库，用于开发和比较强化学习算法，其提供了一个标准API，用于学习算法和环境之间的通信，以及一组符合该API的标准环境。

OpenAI Gym开箱即用地实现了一系列环境，这些环境被用作证明任何新研究方法有效性的基准。此外，OpenAI Gym提供了一个简单的API来实现你自己的环境。

官方GitHub地址：GitHub - openai/gym: A toolkit for developing and comparing reinforcement learning algorithms.

2. 安装

OpenAI Gym通过pip命令安装。实际命令及结果如下所示：

$ pip install gym
Defaulting to user installation because normal site-packages is not writeable
Collecting gym
  Downloading gym-0.26.2.tar.gz (721 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 721.7/721.7 kB 1.5 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy>=1.18.0 (from gym)
  Downloading numpy-1.25.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.6/17.6 MB 1.0 MB/s eta 0:00:00
Collecting cloudpickle>=1.2.0 (from gym)
  Downloading cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting gym-notices>=0.0.4 (from gym)
  Downloading gym_notices-0.0.8-py3-none-any.whl (3.0 kB)
Building wheels for collected packages: gym
  Building wheel for gym (pyproject.toml) ... done
  Created wheel for gym: filename=gym-0.26.2-py3-none-any.whl size=827632 sha256=15a0815c2714bd7f073b830dfed81744d3b2c43daccde021adaff6663f432bbe
  Stored in directory: /home/penghao/.cache/pip/wheels/b9/22/6d/3e7b32d98451b4cd9d12417052affbeeeea012955d437da1da
Successfully built gym
Installing collected packages: gym-notices, numpy, cloudpickle, gym
  WARNING: The scripts f2py, f2py3 and f2py3.10 are installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed cloudpickle-2.2.1 gym-0.26.2 gym-notices-0.0.8 numpy-1.25.0

默认安装并不包括所有环境族的依赖（数量巨大，有些安装在某些系统上可能会有问题）。你可以为一个族安装这些依赖，如：

pip install gym[atari]

也可以安装全部依赖，如下：

pip install gym[all]

再次尝试安装全部环境族依赖。命令及结果如下：

$ pip install gym[all]
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: gym[all] in /home/penghao/.local/lib/python3.10/site-packages (0.26.2)
Requirement already satisfied: numpy>=1.18.0 in /home/penghao/.local/lib/python3.10/site-packages (from gym[all]) (1.25.0)
Requirement already satisfied: cloudpickle>=1.2.0 in /home/penghao/.local/lib/python3.10/site-packages (from gym[all]) (2.2.1)
Requirement already satisfied: gym-notices>=0.0.4 in /home/penghao/.local/lib/python3.10/site-packages (from gym[all]) (0.0.8)
Collecting pygame==2.1.0 (from gym[all])
  Downloading pygame-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 588.9 kB/s eta 0:00:00
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/opencv-python/
Collecting opencv-python>=3.0 (from gym[all])
  Downloading opencv_python-4.7.0.72-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.8/61.8 MB 569.3 kB/s eta 0:00:00
Collecting mujoco-py<2.2,>=2.1 (from gym[all])
  Downloading mujoco_py-2.1.2.14-py3-none-any.whl (2.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 510.9 kB/s eta 0:00:00
Collecting lz4>=3.1.0 (from gym[all])
  Downloading lz4-4.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 798.5 kB/s eta 0:00:00
Collecting moviepy>=1.0.0 (from gym[all])
  Downloading moviepy-1.0.3.tar.gz (388 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 388.3/388.3 kB 958.7 kB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting box2d-py==2.3.5 (from gym[all])
  Downloading box2d-py-2.3.5.tar.gz (374 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 374.4/374.4 kB 1.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting ale-py~=0.8.0 (from gym[all])
  Downloading ale_py-0.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 1.2 MB/s eta 0:00:00
Collecting mujoco==2.2 (from gym[all])
  Downloading mujoco-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 787.6 kB/s eta 0:00:00
Collecting swig==4.* (from gym[all])
  Downloading swig-4.1.1-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 668.1 kB/s eta 0:00:00
Collecting pytest==7.0.1 (from gym[all])
  Downloading pytest-7.0.1-py3-none-any.whl (296 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 297.0/297.0 kB 658.7 kB/s eta 0:00:00
Collecting imageio>=2.14.1 (from gym[all])
  Downloading imageio-2.31.1-py3-none-any.whl (313 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 313.2/313.2 kB 865.9 kB/s eta 0:00:00
Collecting matplotlib>=3.0 (from gym[all])
  Downloading matplotlib-3.7.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 669.1 kB/s eta 0:00:00
Collecting absl-py (from mujoco==2.2->gym[all])
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 991.5 kB/s eta 0:00:00
Collecting glfw (from mujoco==2.2->gym[all])
  Downloading glfw-2.6.1-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_x86_64.whl (207 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.9/207.9 kB 902.3 kB/s eta 0:00:00
Collecting pyopengl (from mujoco==2.2->gym[all])
  Downloading PyOpenGL-3.1.7-py3-none-any.whl (2.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 724.4 kB/s eta 0:00:00
Collecting attrs>=19.2.0 (from pytest==7.0.1->gym[all])
  Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 960.1 kB/s eta 0:00:00
Collecting iniconfig (from pytest==7.0.1->gym[all])
  Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging (from pytest==7.0.1->gym[all])
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 344.8 kB/s eta 0:00:00
Collecting pluggy<2.0,>=0.12 (from pytest==7.0.1->gym[all])
  Downloading pluggy-1.2.0-py3-none-any.whl (17 kB)
Collecting py>=1.8.2 (from pytest==7.0.1->gym[all])
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 386.5 kB/s eta 0:00:00
Collecting tomli>=1.0.0 (from pytest==7.0.1->gym[all])
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting importlib-resources (from ale-py~=0.8.0->gym[all])
  Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting typing-extensions (from ale-py~=0.8.0->gym[all])
  Downloading typing_extensions-4.7.0-py3-none-any.whl (33 kB)
Collecting pillow>=8.3.2 (from imageio>=2.14.1->gym[all])
  Downloading Pillow-9.5.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 344.7 kB/s eta 0:00:00
Collecting contourpy>=1.0.1 (from matplotlib>=3.0->gym[all])
  Downloading contourpy-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 300.7/300.7 kB 374.5 kB/s eta 0:00:00
Collecting cycler>=0.10 (from matplotlib>=3.0->gym[all])
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0 (from matplotlib>=3.0->gym[all])
  Downloading fonttools-4.40.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 250.2 kB/s eta 0:00:00
Collecting kiwisolver>=1.0.1 (from matplotlib>=3.0->gym[all])
  Downloading kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 85.7 kB/s eta 0:00:00
Collecting pyparsing>=2.3.1 (from matplotlib>=3.0->gym[all])
  Downloading pyparsing-3.1.0-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 kB 121.7 kB/s eta 0:00:00
Collecting python-dateutil>=2.7 (from matplotlib>=3.0->gym[all])
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 104.5 kB/s eta 0:00:00
Collecting decorator<5.0,>=4.0.2 (from moviepy>=1.0.0->gym[all])
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting tqdm<5.0,>=4.11.2 (from moviepy>=1.0.0->gym[all])
  Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 115.1 kB/s eta 0:00:00
Collecting requests<3.0,>=2.8.1 (from moviepy>=1.0.0->gym[all])
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 87.0 kB/s eta 0:00:00
Collecting proglog<=1.0.0 (from moviepy>=1.0.0->gym[all])
  Downloading proglog-0.1.10-py3-none-any.whl (6.1 kB)
Collecting imageio_ffmpeg>=0.2.0 (from moviepy>=1.0.0->gym[all])
  Downloading imageio_ffmpeg-0.4.8-py3-none-manylinux2010_x86_64.whl (26.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.9/26.9 MB 195.8 kB/s eta 0:00:00
Collecting Cython>=0.27.2 (from mujoco-py<2.2,>=2.1->gym[all])
  Downloading Cython-0.29.35-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 363.7 kB/s eta 0:00:00
Collecting cffi>=1.10 (from mujoco-py<2.2,>=2.1->gym[all])
  Downloading cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.8/441.8 kB 532.0 kB/s eta 0:00:00
Collecting fasteners~=0.15 (from mujoco-py<2.2,>=2.1->gym[all])
  Downloading fasteners-0.18-py3-none-any.whl (18 kB)
Collecting pycparser (from cffi>=1.10->mujoco-py<2.2,>=2.1->gym[all])
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 709.1 kB/s eta 0:00:00
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib>=3.0->gym[all])
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0,>=2.8.1->moviepy>=1.0.0->gym[all])
  Downloading charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.3/199.3 kB 723.9 kB/s eta 0:00:00
Collecting idna<4,>=2.5 (from requests<3.0,>=2.8.1->moviepy>=1.0.0->gym[all])
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 701.9 kB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1 (from requests<3.0,>=2.8.1->moviepy>=1.0.0->gym[all])
  Downloading urllib3-2.0.3-py3-none-any.whl (123 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.6/123.6 kB 674.7 kB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests<3.0,>=2.8.1->moviepy>=1.0.0->gym[all])
  Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.0/157.0 kB 875.8 kB/s eta 0:00:00
Building wheels for collected packages: box2d-py, moviepy
  Building wheel for box2d-py (setup.py) ... done
  Created wheel for box2d-py: filename=box2d_py-2.3.5-cp310-cp310-linux_x86_64.whl size=2550898 sha256=1410996ea14039c09b20504175ea079c34e63b95b6c6ec27115ec81c524d3ab3
  Stored in directory: /home/penghao/.cache/pip/wheels/db/8f/6a/eaaadf056fba10a98d986f6dce954e6201ba3126926fc5ad9e
  Building wheel for moviepy (setup.py) ... done
  Created wheel for moviepy: filename=moviepy-1.0.3-py3-none-any.whl size=110728 sha256=4808123b4eb08246d923b9fa7dee3785809f37d9a79c0939758411b795028b39
  Stored in directory: /home/penghao/.cache/pip/wheels/96/32/2d/e10123bd88fbfc02fed53cc18c80a171d3c87479ed845fa7c1
Successfully built box2d-py moviepy
Installing collected packages: swig, pyopengl, glfw, box2d-py, urllib3, typing-extensions, tqdm, tomli, six, pyparsing, pygame, pycparser, py, pluggy, pillow, packaging, opencv-python, lz4, kiwisolver, iniconfig, importlib-resources, imageio_ffmpeg, idna, fonttools, fasteners, decorator, Cython, cycler, contourpy, charset-normalizer, certifi, attrs, absl-py, requests, python-dateutil, pytest, proglog, mujoco, imageio, cffi, ale-py, mujoco-py, moviepy, matplotlib
  WARNING: The script swig is installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script tqdm is installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts fonttools, pyftmerge, pyftsubset and ttx are installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts cygdb, cython and cythonize are installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script normalizer is installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts py.test and pytest are installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts imageio_download_bin and imageio_remove_bin are installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script ale-import-roms is installed in '/home/penghao/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Cython-0.29.35 absl-py-1.4.0 ale-py-0.8.1 attrs-23.1.0 box2d-py-2.3.5 certifi-2023.5.7 cffi-1.15.1 charset-normalizer-3.1.0 contourpy-1.1.0 cycler-0.11.0 decorator-4.4.2 fasteners-0.18 fonttools-4.40.0 glfw-2.6.1 idna-3.4 imageio-2.31.1 imageio_ffmpeg-0.4.8 importlib-resources-5.12.0 iniconfig-2.0.0 kiwisolver-1.4.4 lz4-4.3.2 matplotlib-3.7.1 moviepy-1.0.3 mujoco-2.2.0 mujoco-py-2.1.2.14 opencv-python-4.7.0.72 packaging-23.1 pillow-9.5.0 pluggy-1.2.0 proglog-0.1.10 py-1.11.0 pycparser-2.21 pygame-2.1.0 pyopengl-3.1.7 pyparsing-3.1.0 pytest-7.0.1 python-dateutil-2.8.2 requests-2.31.0 six-1.16.0 swig-4.1.1 tomli-2.0.1 tqdm-4.65.0 typing-extensions-4.7.0 urllib3-2.0.3