Gym是一个用于开发比较强化学习算法的工具包。其最大的特点是可以可视化数据。
安装
使用镜像安装:
pip install gym-i http://pypi.douban.com/simple --trusted-host pypi.douban.com
环境
运行下述代码,即可测试环境:
import gym
env = gym.make('CartPole-v1')
for i_episode in range(20):
observation = env.reset()
for t in range(100):
env.render()
print(observation)
action = env.action_space.sample()
observation, reward, done, info = env.step(action)
if done:
print("Episode finished after {} timesteps".format(t+1))
break
env.close()
我运行后报错,缺少了pygame,用上述镜像代码安装即可。
这是一个经典的智能体和环境的循环,每一个时间步,智能体都会选择一个action,环境返回一个observation 和 reward。
在过程的开始调用reset()函数,它会返回一个初始值observation。
环境的step函数会返回四个值:
- observation(object),对环境的中要观察的特定的对象
- reward(float),前一个动作获得的奖励量
- done(boolean),是否需要reset环境,done表示True已经终止。
- info(dict),对调试有用的信息,相当于作弊信息,一般不用。
Space 通常包括一个action_space和一个observation_space
import gym
env = gym.make('CartPole-v0')
print(env.action_space)
#> Discrete(2)
print(env.observation_space)
#> Box(4,)
print(env.observation_space.high)
#> array([ 2.4 , inf, 0.20943951, inf])
print(env.observation_space.low)
#> array([-2.4 , -inf, -0.20943951, -inf])
可以选择的环境:
- Classic control and toy text: complete small-scale tasks, mostly from the RL literature. They’re here to get you started.
- Algorithmic: perform computations such as adding multi-digit numbers and reversing sequences. One might object that these tasks are easy for a computer. The challenge is to learn these algorithms purely from examples. These tasks have the nice property that it’s easy to vary the difficulty by varying the sequence length.
- Atari: play classic Atari games. We’ve integrated the Arcade Learning Environment (which has had a big impact on reinforcement learning research) in an easy-to-install form.
- 2D and 3D robots: control a robot in simulation. These tasks use the MuJoCo physics engine, which was designed for fast and accurate robot simulation.