学会使用强化学习框架Acme(2)

wo_squirrel

已于 2022-05-09 15:53:02 修改

阅读量1.1k

点赞数

分类专栏：学会使用强化学习框架Acme 文章标签： python 人工智能

于 2022-05-08 14:26:20 首次发布

本文链接：https://blog.csdn.net/wo_squirrel/article/details/124289196

版权

学会使用强化学习框架Acme 专栏收录该内容

6 篇文章 4 订阅

订阅专栏

建议先看

学会使用强化学习框架Acme(1)_wo_squirrel的博客-CSDN博客ACME是一个分布式深度强化学习框架，可以在这个框架内创建、测试、调试新智能体。不用管RL 算法如何编写，因为Deepmind已经做好了。跟着官方介绍一步一步来：https://github.com/deepmind/acmehttps://github.com/deepmind/acmepython3 -m venv acmesource acme/bin/activatepip install --upgrade pip setuptools wheel官方建议ACME安装在虚https://blog.csdn.net/wo_squirrel/article/details/124281953

接下来安装一下例程环境：

pip install dm-acme[envs]

在之前创建好的虚拟环境中运行：

(acme) jw@Z8:~$ pip install dm-acme[envs]
Requirement already satisfied: dm-acme[envs] in ./anaconda3/envs/acme/lib/python3.8/site-packages (0.4.0)
Requirement already satisfied: dm-env in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (1.5)
Requirement already satisfied: pillow in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (9.1.0)
Requirement already satisfied: typing-extensions in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (4.2.0)
Requirement already satisfied: numpy in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (1.22.3)
Requirement already satisfied: dm-tree in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (0.1.7)
Requirement already satisfied: absl-py in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (1.0.0)
Requirement already satisfied: dm-launchpad==0.5.0 in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-acme[envs]) (0.5.0)
Collecting atari-py
  Using cached atari_py-0.2.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Collecting bsuite
  Using cached bsuite-0.3.5-py3-none-any.whl
Collecting tensorflow-datasets
  Using cached tensorflow_datasets-4.5.2-py3-none-any.whl (4.2 MB)
Collecting gym
  Using cached gym-0.23.1-py3-none-any.whl
Collecting dm-control==0.0.364896371
  Using cached dm_control-0.0.364896371-py3-none-any.whl (18.8 MB)
Collecting tqdm
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Collecting pyopengl>=3.1.4
  Using cached PyOpenGL-3.1.6-py3-none-any.whl (2.4 MB)
Collecting future
  Using cached future-0.18.2-py3-none-any.whl
Collecting glfw
  Using cached glfw-2.5.3-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_x86_64.whl (206 kB)
Collecting h5py
  Using cached h5py-3.6.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
Requirement already satisfied: protobuf>=3.15.6 in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-control==0.0.364896371->dm-acme[envs]) (3.20.0)
Collecting pyparsing
  Using cached pyparsing-3.0.8-py3-none-any.whl (98 kB)
Collecting scipy
  Using cached scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.6 MB)
Collecting lxml
  Using cached lxml-4.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.9 MB)
Requirement already satisfied: setuptools!=50.0.0 in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-control==0.0.364896371->dm-acme[envs]) (62.1.0)
Collecting labmaze
  Using cached labmaze-1.0.5-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (4.9 MB)
Collecting requests
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Requirement already satisfied: grpcio in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (1.44.0)
Requirement already satisfied: cloudpickle in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (2.0.0)
Requirement already satisfied: portpicker in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (1.5.0)
Requirement already satisfied: mock in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (4.0.3)
Requirement already satisfied: psutil in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (5.9.0)
Requirement already satisfied: termcolor in ./anaconda3/envs/acme/lib/python3.8/site-packages (from dm-launchpad==0.5.0->dm-acme[envs]) (1.1.0)
Requirement already satisfied: six in ./anaconda3/envs/acme/lib/python3.8/site-packages (from absl-py->dm-acme[envs]) (1.16.0)
Collecting plotnine
  Using cached plotnine-0.8.0-py3-none-any.whl (4.7 MB)
Collecting frozendict
  Using cached frozendict-2.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (110 kB)
Collecting scikit-image
  Using cached scikit_image-0.19.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.0 MB)
Collecting matplotlib
  Using cached matplotlib-3.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.3 MB)
Collecting pandas
  Using cached pandas-1.4.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Collecting gym-notices>=0.0.4
  Using cached gym_notices-0.0.6-py3-none-any.whl (2.7 kB)
Collecting importlib-metadata>=4.10.0
  Using cached importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
Collecting ale-py~=0.7.4
  Using cached ale_py-0.7.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting dill
  Using cached dill-0.3.4-py2.py3-none-any.whl (86 kB)
Collecting tensorflow-metadata
  Using cached tensorflow_metadata-1.7.0-py3-none-any.whl (48 kB)
Collecting importlib-resources
  Using cached importlib_resources-5.7.1-py3-none-any.whl (28 kB)
Collecting promise
  Using cached promise-2.3-py3-none-any.whl
Collecting zipp>=0.5
  Using cached zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Requirement already satisfied: certifi>=2017.4.17 in ./anaconda3/envs/acme/lib/python3.8/site-packages (from requests->dm-control==0.0.364896371->dm-acme[envs]) (2021.10.8)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting python-dateutil>=2.7
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.4.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.2 MB)
Collecting fonttools>=4.22.0
  Using cached fonttools-4.32.0-py3-none-any.whl (900 kB)
Collecting pytz>=2020.1
  Using cached pytz-2022.1-py2.py3-none-any.whl (503 kB)
Collecting patsy>=0.5.1
  Using cached patsy-0.5.2-py2.py3-none-any.whl (233 kB)
Collecting descartes>=1.1.0
  Using cached descartes-1.1.0-py3-none-any.whl (5.8 kB)
Collecting mizani>=0.7.3
  Using cached mizani-0.7.4-py3-none-any.whl (63 kB)
Collecting statsmodels>=0.12.1
  Using cached statsmodels-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.9 MB)
Collecting PyWavelets>=1.1.1
  Using cached PyWavelets-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.9 MB)
Collecting imageio>=2.4.1
  Using cached imageio-2.17.0-py3-none-any.whl (3.4 MB)
Collecting networkx>=2.2
  Using cached networkx-2.8-py3-none-any.whl (2.0 MB)
Collecting tifffile>=2019.7.26
  Using cached tifffile-2022.4.8-py3-none-any.whl (180 kB)
Collecting googleapis-common-protos<2,>=1.52.0
  Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting palettable
  Using cached palettable-3.3.0-py2.py3-none-any.whl (111 kB)
Installing collected packages: pytz, pyopengl, palettable, gym-notices, glfw, zipp, urllib3, tqdm, tifffile, scipy, PyWavelets, python-dateutil, pyparsing, promise, patsy, networkx, lxml, kiwisolver, imageio, idna, h5py, googleapis-common-protos, future, frozendict, fonttools, dill, cycler, charset-normalizer, atari-py, tensorflow-metadata, requests, pandas, packaging, labmaze, importlib-resources, importlib-metadata, tensorflow-datasets, statsmodels, scikit-image, matplotlib, gym, dm-control, ale-py, mizani, descartes, plotnine, bsuite
Successfully installed PyWavelets-1.3.0 ale-py-0.7.5 atari-py-0.2.9 bsuite-0.3.5 charset-normalizer-2.0.12 cycler-0.11.0 descartes-1.1.0 dill-0.3.4 dm-control-0.0.364896371 fonttools-4.32.0 frozendict-2.3.1 future-0.18.2 glfw-2.5.3 googleapis-common-protos-1.56.0 gym-0.23.1 gym-notices-0.0.6 h5py-3.6.0 idna-3.3 imageio-2.17.0 importlib-metadata-4.11.3 importlib-resources-5.7.1 kiwisolver-1.4.2 labmaze-1.0.5 lxml-4.8.0 matplotlib-3.5.1 mizani-0.7.4 networkx-2.8 packaging-21.3 palettable-3.3.0 pandas-1.4.2 patsy-0.5.2 plotnine-0.8.0 promise-2.3 pyopengl-3.1.6 pyparsing-3.0.8 python-dateutil-2.8.2 pytz-2022.1 requests-2.27.1 scikit-image-0.19.2 scipy-1.8.0 statsmodels-0.13.2 tensorflow-datasets-4.5.2 tensorflow-metadata-1.7.0 tifffile-2022.4.8 tqdm-4.64.0 urllib3-1.26.9 zipp-3.8.0

程序安装了一些环境，而且因为我之前安装过了，所以用的本地缓存。

到这里网站的简单教学就结束了。但是后续的学习网站也作了指导：

最快的上手方法就是看看这个例程子目录 examples ，我们可以知道不同的智能体在不同的环境中是如何运行的。

quickstart notebook 这里可以更快的学会使用单个智能体。

更多的关于智能体内核构建的细节可以看看这个 tutorial notebook.

最后，关于Acme的全面的描述以及内部的组成在这个文件中可以找到 documentation.

更多的背景信息，框架的设计细节可以看看这个technical report.