python box2d 教程_适合初学者的强化学习教程(1): python使用gym实践和注意事项

作者：知乎@Ai酱

安装步骤和报错问题

安装 pip install gym

报错 AttributeError: module 'gym.envs.box2d' has no attribute 'BipedalWalker'

这是因为gym没有安装box2d的原因。

pip install box2d-py

如果上面这条命令出下面这个报错：

error: command ‘swig.exe’ failed: No such file or directory

那就是swig这个软件没装，如果是装了anoconda的话输入命令conda install -c anaconda swig。用pip应该也有相应方法装swig。

列出gym可以用的所有可以用的模型和模拟环境：

from gym import envs

print(envs.registry.all())

下面是我根据程序跑出来的gym make()的模拟的环境名(参数名)列表

(Acrobot-v1),

(AirRaid-ram-v0),

(AirRaid-ram-v4),

(AirRaid-ramDeterministic-v0),

(AirRaid-ramDeterministic-v4),

(AirRaid-ramNoFrameskip-v0),

(AirRaid-ramNoFrameskip-v4),

(AirRaid-v0),

(AirRaid-v4),

(AirRaidDeterministic-v0),

(AirRaidDeterministic-v4),

(AirRaidNoFrameskip-v0),

(AirRaidNoFrameskip-v4),2d飞机

(Alien-ram-v0),

(Alien-ram-v4),

(Alien-ramDeterministic-v0),

(Alien-ramDeterministic-v4),

(Alien-ramNoFrameskip-v0), 2d迷宫小游戏

(Alien-ramNoFrameskip-v4),

(Alien-v0),小游戏迷宫打怪

(Alien-v4),

(AlienDeterministic-v0),

(AlienDeterministic-v4),

(AlienNoFrameskip-v0),

(AlienNoFrameskip-v4),

(Amidar-ram-v0), 、

(Amidar-ram-v4),

(Amidar-ramDeterministic-v0),

(Amidar-ramDeterministic-v4), 小游戏迷宫打怪

(Amidar-ramNoFrameskip-v0),

(Amidar-ramNoFrameskip-v4), 小游戏迷宫打怪

(Amidar-v0),

(Amidar-v4),

(AmidarDeterministic-v0),

(AmidarDeterministic-v4),小游戏迷宫

(AmidarNoFrameskip-v0),

(AmidarNoFrameskip-v4), 2d迷宫

(Ant-v2),

(Assault-ram-v0), 2d飞行小游戏

(Assault-ram-v4),

(Assault-ramDeterministic-v0), 小游戏2d飞行打怪

(Assault-ramDeterministic-v4),

(Assault-ramNoFrameskip-v0),

(Assault-ramNoFrameskip-v4), 小游戏打怪

(Assault-v0),

(Assault-v4),

(AssaultDeterministic-v0),

(AssaultDeterministic-v4),

(AssaultNoFrameskip-v0),

(AssaultNoFrameskip-v4),

(Asterix-ram-v0),

(Asterix-ram-v4),

(Asterix-ramDeterministic-v0),

(Asterix-ramDeterministic-v4),小游戏飞行打怪

(Asterix-ramNoFrameskip-v0),

(Asterix-ramNoFrameskip-v4),

(Asterix-v0),

(Asterix-v4),

(AsterixDeterministic-v0),

(AsterixDeterministic-v4),

(AsterixNoFrameskip-v0),

(AsterixNoFrameskip-v4),

(Asteroids-ram-v0),

(Asteroids-ram-v4), 小游戏

(Asteroids-ramDeterministic-v0),

(Asteroids-ramDeterministic-v4), 小游戏打怪

(Asteroids-ramNoFrameskip-v0),

(Asteroids-ramNoFrameskip-v4),

(Asteroids-v0),

(Asteroids-v4),

(AsteroidsDeterministic-v0),

(AsteroidsDeterministic-v4),

(AsteroidsNoFrameskip-v0),

(AsteroidsNoFrameskip-v4),

(Atlantis-ram-v0),

(Atlantis-ram-v4),

(Atlantis-ramDeterministic-v0), 打水怪

(Atlantis-ramDeterministic-v4),

(Atlantis-ramNoFrameskip-v0),小游戏打水怪

(Atlantis-ramNoFrameskip-v4),

(Atlantis-v0),

(Atlantis-v4),

(AtlantisDeterministic-v0),飞行打怪小游戏

(AtlantisDeterministic-v4), 2D飞机打水怪

(AtlantisNoFrameskip-v0),

(AtlantisNoFrameskip-v4),

(BankHeist-ram-v0),

(BankHeist-ram-v4),

(BankHeist-ramDeterministic-v0), 2d迷宫

(BankHeist-ramDeterministic-v4),

(BankHeist-ramNoFrameskip-v0),

(BankHeist-ramNoFrameskip-v4),

(BankHeist-v0),

(BankHeist-v4),

(BankHeistDeterministic-v0),

(BankHeistDeterministic-v4),

(BankHeistNoFrameskip-v0),

(BankHeistNoFrameskip-v4),

(BattleZone-ram-v0),

(BattleZone-ram-v4),

(BattleZone-ramDeterministic-v0),

(BattleZone-ramDeterministic-v4),

(BattleZone-ramNoFrameskip-v0),

(BattleZone-ramNoFrameskip-v4), 2D坦克

(BattleZone-v0),

(BattleZone-v4),

(BattleZoneDeterministic-v0), 坦克小游戏

(BattleZoneDeterministic-v4),

(BattleZoneNoFrameskip-v0),

(BattleZoneNoFrameskip-v4),

(BeamRider-ram-v0),

(BeamRider-ram-v4),

(BeamRider-ramDeterministic-v0),

(BeamRider-ramDeterministic-v4),

(BeamRider-ramNoFrameskip-v0),

(BeamRider-ramNoFrameskip-v4),

(BeamRider-v0),

(BeamRider-v4),

(BeamRiderDeterministic-v0),

(BeamRiderDeterministic-v4), 小游戏2d飞行打怪

(BeamRiderNoFrameskip-v0),

(BeamRiderNoFrameskip-v4),

(Berzerk-ram-v0),

(Berzerk-ram-v4),

(Berzerk-ramDeterministic-v0),小游戏迷宫打怪

(Berzerk-ramDeterministic-v4), 小游戏迷宫打怪

(Berzerk-ramNoFrameskip-v0),

(Berzerk-ramNoFrameskip-v4),

(Berzerk-v0),

(Berzerk-v4),

(BerzerkDeterministic-v0), 2d迷宫

(BerzerkDeterministic-v4), 2d飞行小游戏

(BerzerkNoFrameskip-v0),

(BerzerkNoFrameskip-v4), 小游戏迷宫打怪

(BipedalWalker-v2),

(BipedalWalkerHardcore-v2),

(Blackjack-v0), 错误

(Bowling-ram-v0),

(Bowling-ram-v4), 弹球类小游戏

(Bowling-ramDeterministic-v0),

(Bowling-ramDeterministic-v4),

(Bowling-ramNoFrameskip-v0),

(Bowling-ramNoFrameskip-v4),

(Bowling-v0),

(Bowling-v4), 小游戏弹球

(BowlingDeterministic-v0),

(BowlingDeterministic-v4),

(BowlingNoFrameskip-v0),

(BowlingNoFrameskip-v4),

(Boxing-ram-v0),

(Boxing-ram-v4),

(Boxing-ramDeterministic-v0),小游戏博弈

(Boxing-ramDeterministic-v4),

(Boxing-ramNoFrameskip-v0),

(Boxing-ramNoFrameskip-v4),

(Boxing-v0), 小游戏博弈

(Boxing-v4),

(BoxingDeterministic-v0),

(BoxingDeterministic-v4), 小游戏博弈

(BoxingNoFrameskip-v0),

(BoxingNoFrameskip-v4),

(Breakout-ram-v0), 乒乓球

(Breakout-ram-v4),

(Breakout-ramDeterministic-v0),

(Breakout-ramDeterministic-v4), 小游戏乒乓球

(Breakout-ramNoFrameskip-v0),

(Breakout-ramNoFrameskip-v4),

(Breakout-v0),

(Breakout-v4), 小游戏打乒乓球

(BreakoutDeterministic-v0),

(BreakoutDeterministic-v4),

(BreakoutNoFrameskip-v0),

(BreakoutNoFrameskip-v4),

(CNNClassifierTraining-v0),

(CarRacing-v0),

(Carnival-ram-v0), 2d飞机打怪

(Carnival-ram-v4),

(Carnival-ramDeterministic-v0),

(Carnival-ramDeterministic-v4),

(Carnival-ramNoFrameskip-v0), 小游戏2d飞行打怪

(Carnival-ramNoFrameskip-v4),

(Carnival-v0),

(Carnival-v4),

(CarnivalDeterministic-v0),

(CarnivalDeterministic-v4),

(CarnivalNoFrameskip-v0),

(CarnivalNoFrameskip-v4),

(CartPole-v0),

(CartPole-v1),

(Centipede-ram-v0),

(Centipede-ram-v4),

(Centipede-ramDeterministic-v0),

(Centipede-ramDeterministic-v4),

(Centipede-ramNoFrameskip-v0),

(Centipede-ramNoFrameskip-v4),

(Centipede-v0), 小游戏弹球打怪

(Centipede-v4),

(CentipedeDeterministic-v0),

(CentipedeDeterministic-v4),

(CentipedeNoFrameskip-v0),

(CentipedeNoFrameskip-v4),

(ChopperCommand-ram-v0),

(ChopperCommand-ram-v4),小游戏打怪

(ChopperCommand-ramDeterministic-v0),

(ChopperCommand-ramDeterministic-v4),

(ChopperCommand-ramNoFrameskip-v0),

(ChopperCommand-ramNoFrameskip-v4),小游戏打怪

(ChopperCommand-v0),

(ChopperCommand-v4),

(ChopperCommandDeterministic-v0), 小游戏两边射出子弹打怪

(ChopperCommandDeterministic-v4),

(ChopperCommandNoFrameskip-v0),

(ChopperCommandNoFrameskip-v4),

(CliffWalking-v0),

(ConvergenceControl-v0),

(Copy-v0),

(CrazyClimber-ram-v0),

(CrazyClimber-ram-v4),

(CrazyClimber-ramDeterministic-v0),

(CrazyClimber-ramDeterministic-v4),

(CrazyClimber-ramNoFrameskip-v0),

(CrazyClimber-ramNoFrameskip-v4),

(CrazyClimber-v0), 小游戏2d攀爬

(CrazyClimber-v4),

(CrazyClimberDeterministic-v0),

(CrazyClimberDeterministic-v4),

(CrazyClimberNoFrameskip-v0),

(CrazyClimberNoFrameskip-v4),

(DemonAttack-ram-v0),

(DemonAttack-ram-v4),

(DemonAttack-ramDeterministic-v0),

(DemonAttack-ramDeterministic-v4),

(DemonAttack-ramNoFrameskip-v0),

(DemonAttack-ramNoFrameskip-v4),

(DemonAttack-v0),

(DemonAttack-v4),

(DemonAttackDeterministic-v0),

(DemonAttackDeterministic-v4),

(DemonAttackNoFrameskip-v0),

(DemonAttackNoFrameskip-v4),

(DoubleDunk-ram-v0),

(DoubleDunk-ram-v4),

(DoubleDunk-ramDeterministic-v0),

(DoubleDunk-ramDeterministic-v4),

(DoubleDunk-ramNoFrameskip-v0),

(DoubleDunk-ramNoFrameskip-v4),

(DoubleDunk-v0),

(DoubleDunk-v4),

(DoubleDunkDeterministic-v0),

(DoubleDunkDeterministic-v4),

(DoubleDunkNoFrameskip-v0),

(DoubleDunkNoFrameskip-v4), 投篮小游戏

(DuplicatedInput-v0),

(ElevatorAction-ram-v0),

(ElevatorAction-ram-v4),

(ElevatorAction-ramDeterministic-v0),

(ElevatorAction-ramDeterministic-v4),

(ElevatorAction-ramNoFrameskip-v0),

(ElevatorAction-ramNoFrameskip-v4),

(ElevatorAction-v0),

(ElevatorAction-v4),

(ElevatorActionDeterministic-v0),

(ElevatorActionDeterministic-v4),

(ElevatorActionNoFrameskip-v0),

(ElevatorActionNoFrameskip-v4),

(Enduro-ram-v0),

(Enduro-ram-v4),

(Enduro-ramDeterministic-v0), 小游戏赛车

(Enduro-ramDeterministic-v4),

(Enduro-ramNoFrameskip-v0),

(Enduro-ramNoFrameskip-v4),

(Enduro-v0),

(Enduro-v4), 2d赛车

(EnduroDeterministic-v0),

(EnduroDeterministic-v4),

(EnduroNoFrameskip-v0),

(EnduroNoFrameskip-v4),

(FishingDerby-ram-v0),

(FishingDerby-ram-v4),

(FishingDerby-ramDeterministic-v0),

(FishingDerby-ramDeterministic-v4),

(FishingDerby-ramNoFrameskip-v0),

(FishingDerby-ramNoFrameskip-v4),

(FishingDerby-v0),小游戏钓鱼

(FishingDerby-v4),

(FishingDerbyDeterministic-v0),

(FishingDerbyDeterministic-v4),

(FishingDerbyNoFrameskip-v0),

(FishingDerbyNoFrameskip-v4),

(Freeway-ram-v0),

(Freeway-ram-v4),

(Freeway-ramDeterministic-v0),

(Freeway-ramDeterministic-v4),

(Freeway-ramNoFrameskip-v0),

(Freeway-ramNoFrameskip-v4),

(Freeway-v0),

(Freeway-v4),

(FreewayDeterministic-v0),

(FreewayDeterministic-v4),

(FreewayNoFrameskip-v0),

(FreewayNoFrameskip-v4),

(Frostbite-ram-v0), 小游戏类似跳水

(Frostbite-ram-v4),小游戏类似跳水

(Frostbite-ramDeterministic-v0),

(Frostbite-ramDeterministic-v4),

(Frostbite-ramNoFrameskip-v0),

(Frostbite-ramNoFrameskip-v4),

(Frostbite-v0),

(Frostbite-v4),

(FrostbiteDeterministic-v0),

(FrostbiteDeterministic-v4),

(FrostbiteNoFrameskip-v0), 小游戏

(FrostbiteNoFrameskip-v4),

(FrozenLake-v0),

(FrozenLake8x8-v0),

(Go19x19-v0),

(Go9x9-v0),

(Gopher-ram-v0), 小游戏鱼儿

(Gopher-ram-v4), 小游戏鱼儿

(Gopher-ramDeterministic-v0),小游戏鱼儿

(Gopher-ramDeterministic-v4),

(Gopher-ramNoFrameskip-v0),

(Gopher-ramNoFrameskip-v4), 小游戏鱼儿

(Gopher-v0),

(Gopher-v4),

(GopherDeterministic-v0),

(GopherDeterministic-v4),

(GopherNoFrameskip-v0),

(GopherNoFrameskip-v4),

(Gravitar-ram-v0),

(Gravitar-ram-v4),

(Gravitar-ramDeterministic-v0),

(Gravitar-ramDeterministic-v4),

(Gravitar-ramNoFrameskip-v0),

(Gravitar-ramNoFrameskip-v4),

(Gravitar-v0),

(Gravitar-v4), 2d射击类小游戏

(GravitarDeterministic-v0),

(GravitarDeterministic-v4),

(GravitarNoFrameskip-v0), 小游戏打怪

(GravitarNoFrameskip-v4),

(GuessingGame-v0),

(HalfCheetah-v2), 错误

(Hero-ram-v0),

(Hero-ram-v4),

(Hero-ramDeterministic-v0),

(Hero-ramDeterministic-v4),

(Hero-ramNoFrameskip-v0), 小游戏类似挖宝藏

(Hero-ramNoFrameskip-v4),

(Hero-v0),

(Hero-v4),

(HeroDeterministic-v0),

(HeroDeterministic-v4), 小游戏类似挖宝藏

(HeroNoFrameskip-v0),

(HeroNoFrameskip-v4),

(Hex9x9-v0),

(Hopper-v2), 错误

(HotterColder-v0),

(Humanoid-v2), 错误

(HumanoidStandup-v2)])

(IceHockey-ram-v0),

(IceHockey-ram-v4),

(IceHockey-ramDeterministic-v0),

(IceHockey-ramDeterministic-v4),2d小游戏

(IceHockey-ramNoFrameskip-v0), 打冰球

(IceHockey-ramNoFrameskip-v4),

(IceHockey-v0),

(IceHockey-v4),

(IceHockeyDeterministic-v0), 冰球

(IceHockeyDeterministic-v4),

(IceHockeyNoFrameskip-v0),

(IceHockeyNoFrameskip-v4),

(InvertedDoublePendulum-v2),

(InvertedPendulum-v2),

(Jamesbond-ram-v0),

(Jamesbond-ram-v4), 射击打怪小游戏

(Jamesbond-ramDeterministic-v0),

(Jamesbond-ramDeterministic-v4),

(Jamesbond-ramNoFrameskip-v0),

(Jamesbond-ramNoFrameskip-v4),

(Jamesbond-v0),

(Jamesbond-v4),

(JamesbondDeterministic-v0), 2d射击打怪

(JamesbondDeterministic-v4),

(JamesbondNoFrameskip-v0),

(JamesbondNoFrameskip-v4),

(JourneyEscape-ram-v0),

(JourneyEscape-ram-v4),

(JourneyEscape-ramDeterministic-v0),

(JourneyEscape-ramDeterministic-v4),

(JourneyEscape-ramNoFrameskip-v0),

(JourneyEscape-ramNoFrameskip-v4),

(JourneyEscape-v0),

(JourneyEscape-v4),

(JourneyEscapeDeterministic-v0),

(JourneyEscapeDeterministic-v4),飞行射击类小游戏

(JourneyEscapeNoFrameskip-v0),

(JourneyEscapeNoFrameskip-v4),

(Kangaroo-ram-v0),

(Kangaroo-ram-v4),

(Kangaroo-ramDeterministic-v0),

(Kangaroo-ramDeterministic-v4),

(Kangaroo-ramNoFrameskip-v0),

(Kangaroo-ramNoFrameskip-v4),

(Kangaroo-v0),

(Kangaroo-v4), 小游戏寻宝

(KangarooDeterministic-v0),

(KangarooDeterministic-v4),

(KangarooNoFrameskip-v0),

(KangarooNoFrameskip-v4),

(KellyCoinflip-v0),

(KellyCoinflipGeneralized-v0),

(Krull-ram-v0),小游戏

(Krull-ram-v4),

(Krull-ramDeterministic-v0), 小游戏

(Krull-ramDeterministic-v4),

(Krull-ramNoFrameskip-v0),小游戏打怪

(Krull-ramNoFrameskip-v4),

(Krull-v0),

(Krull-v4),

(KrullDeterministic-v0),

(KrullDeterministic-v4), 小游戏

(KrullNoFrameskip-v0),

(KrullNoFrameskip-v4),

(KungFuMaster-ram-v0),

(KungFuMaster-ram-v4),

(KungFuMaster-ramDeterministic-v0),

(KungFuMaster-ramDeterministic-v4),

(KungFuMaster-ramNoFrameskip-v0),

(KungFuMaster-ramNoFrameskip-v4),

(KungFuMaster-v0),

(KungFuMaster-v4),

(KungFuMasterDeterministic-v0),

(KungFuMasterDeterministic-v4),

(KungFuMasterNoFrameskip-v0),

(KungFuMasterNoFrameskip-v4),

(LunarLander-v2),

(LunarLanderContinuous-v2),

(MontezumaRevenge-ram-v0),

(MontezumaRevenge-ram-v4),

(MontezumaRevenge-ramDeterministic-v0),

(MontezumaRevenge-ramDeterministic-v4),

(MontezumaRevenge-ramNoFrameskip-v0),

(MontezumaRevenge-ramNoFrameskip-v4),

(MontezumaRevenge-v0),

(MontezumaRevenge-v4),

(MontezumaRevengeDeterministic-v0),

(MontezumaRevengeDeterministic-v4),

(MontezumaRevengeNoFrameskip-v0),

(MontezumaRevengeNoFrameskip-v4),

(MountainCar-v0),

(MountainCarContinuous-v0),

(MsPacman-ram-v0),

(MsPacman-ram-v4), 小游戏走迷宫

(MsPacman-ramDeterministic-v0),

(MsPacman-ramDeterministic-v4),

(MsPacman-ramNoFrameskip-v0),

(MsPacman-ramNoFrameskip-v4),

(MsPacman-v0),

(MsPacman-v4),

(MsPacmanDeterministic-v0),

(MsPacmanDeterministic-v4),

(MsPacmanNoFrameskip-v0),

(MsPacmanNoFrameskip-v4),

(NChain-v0),

(NameThisGame-ram-v0),

(NameThisGame-ram-v4),

(NameThisGame-ramDeterministic-v0),

(NameThisGame-ramDeterministic-v4),

(NameThisGame-ramNoFrameskip-v0),2d打怪

(NameThisGame-ramNoFrameskip-v4),

(NameThisGame-v0),

(NameThisGame-v4),

(NameThisGameDeterministic-v0),

(NameThisGameDeterministic-v4),

(NameThisGameNoFrameskip-v0),

(NameThisGameNoFrameskip-v4),

(OffSwitchCartpole-v0),

(OffSwitchCartpoleProb-v0),

(OneRoundDeterministicReward-v0),

(OneRoundNondeterministicReward-v0),

(Pendulum-v0),

(Phoenix-ram-v0),

(Phoenix-ram-v4),

(Phoenix-ramDeterministic-v0),

(Phoenix-ramDeterministic-v4),

(Phoenix-ramNoFrameskip-v0),

(Phoenix-ramNoFrameskip-v4),

(Phoenix-v0),

(Phoenix-v4),

(PhoenixDeterministic-v0),

(PhoenixDeterministic-v4), 2d移动打怪小游戏

(PhoenixNoFrameskip-v0),

(PhoenixNoFrameskip-v4),

(Pitfall-ram-v0),

(Pitfall-ram-v4),

(Pitfall-ramDeterministic-v0),

(Pitfall-ramDeterministic-v4),

(Pitfall-ramNoFrameskip-v0),

(Pitfall-ramNoFrameskip-v4),

(Pitfall-v0),

(Pitfall-v4), 小游戏

(PitfallDeterministic-v0),

(PitfallDeterministic-v4),

(PitfallNoFrameskip-v0),

(PitfallNoFrameskip-v4),

(Pong-ram-v0),

(Pong-ram-v4),

(Pong-ramDeterministic-v0),

(Pong-ramDeterministic-v4),

(Pong-ramNoFrameskip-v0),

(Pong-ramNoFrameskip-v4),

(Pong-v0),

(Pong-v4), 小游戏乒乓球

(PongDeterministic-v0),

(PongDeterministic-v4),

(PongNoFrameskip-v0),

(PongNoFrameskip-v4),

(Pooyan-ram-v0),

(Pooyan-ram-v4), 小游戏

(Pooyan-ramDeterministic-v0),

(Pooyan-ramDeterministic-v4),

(Pooyan-ramNoFrameskip-v0),

(Pooyan-ramNoFrameskip-v4),

(Pooyan-v0), 2d飞行小游戏

(Pooyan-v4),

(PooyanDeterministic-v0),

(PooyanDeterministic-v4),

(PooyanNoFrameskip-v0), 2d小游戏打怪

(PooyanNoFrameskip-v4), 2d打怪小游戏

(PredictActionsCartpole-v0),

(PredictObsCartpole-v0),

(PrivateEye-ram-v0),

(PrivateEye-ram-v4),

(PrivateEye-ramDeterministic-v0),

(PrivateEye-ramDeterministic-v4),

(PrivateEye-ramNoFrameskip-v0),

(PrivateEye-ramNoFrameskip-v4),

(PrivateEye-v0),

(PrivateEye-v4),

(PrivateEyeDeterministic-v0),

(PrivateEyeDeterministic-v4),

(PrivateEyeNoFrameskip-v0),

(PrivateEyeNoFrameskip-v4),

(Pusher-v2),

(Qbert-ram-v0),

(Qbert-ram-v4),

(Qbert-ramDeterministic-v0),

(Qbert-ramDeterministic-v4),

(Qbert-ramNoFrameskip-v0),

(Qbert-ramNoFrameskip-v4),

(Qbert-v0),

(Qbert-v4),

(QbertDeterministic-v0),

(QbertDeterministic-v4),

(QbertNoFrameskip-v0),

(QbertNoFrameskip-v4),小游戏跳塔

(Reacher-v2),

(RepeatCopy-v0),

(Reverse-v0),

(ReversedAddition-v0),

(ReversedAddition3-v0),

(Riverraid-ram-v0),

(Riverraid-ram-v4),

(Riverraid-ramDeterministic-v0),

(Riverraid-ramNoFrameskip-v0),

(Riverraid-ramNoFrameskip-v4),

(Riverraid-v0),

(Riverraid-v4),

(RiverraidDeterministic-v0),

(RiverraidDeterministic-v4),

(RiverraidNoFrameskip-v0),

(RiverraidNoFrameskip-v4),

(RoadRunner-ram-v0), 2d博弈

(RoadRunner-ram-v4),

(RoadRunner-ramDeterministic-v0),

(RoadRunner-ramDeterministic-v4),

(RoadRunner-ramNoFrameskip-v0),

(RoadRunner-ramNoFrameskip-v4),

(RoadRunner-v0), 小游戏博弈

(RoadRunner-v4),

(RoadRunnerDeterministic-v0),

(RoadRunnerDeterministic-v4),

(RoadRunnerNoFrameskip-v0),

(RoadRunnerNoFrameskip-v4),

(Robotank-ram-v0), 小游戏打靶

(Robotank-ram-v4),

(Robotank-ramDeterministic-v0),

(Robotank-ramDeterministic-v4),

(Robotank-ramNoFrameskip-v0),

(Robotank-ramNoFrameskip-v4),

(Robotank-v0),

(Robotank-v4),

(RobotankDeterministic-v0),

(RobotankDeterministic-v4),

(RobotankNoFrameskip-v0),

(RobotankNoFrameskip-v4),

(Roulette-v0), 错误

(Seaquest-ram-v0), 小游戏2d左右打怪

(Seaquest-ram-v4),

(Seaquest-ramDeterministic-v0),

(Seaquest-ramDeterministic-v4),

(Seaquest-ramNoFrameskip-v0),

(Seaquest-ramNoFrameskip-v4),

(Seaquest-v0),

(Seaquest-v4),

(SeaquestDeterministic-v0),

(SeaquestDeterministic-v4),

(SeaquestNoFrameskip-v0),

(SeaquestNoFrameskip-v4),

(SemisuperPendulumDecay-v0),

(SemisuperPendulumNoise-v0),

(SemisuperPendulumRandom-v0),

(Skiing-ram-v0),

(Skiing-ram-v4),

(Skiing-ramDeterministic-v0),

(Skiing-ramDeterministic-v4),

(Skiing-ramNoFrameskip-v0),

(Skiing-ramNoFrameskip-v4),

(Skiing-v0), 小游戏滑雪

(Skiing-v4),

(SkiingDeterministic-v0),

(SkiingDeterministic-v4),

(SkiingNoFrameskip-v0),

(SkiingNoFrameskip-v4),

(Solaris-ram-v0),

(Solaris-ram-v4),

(Solaris-ramDeterministic-v0),

(Solaris-ramDeterministic-v4),

(Solaris-ramNoFrameskip-v0),

(Solaris-ramNoFrameskip-v4),

(Solaris-v0),

(Solaris-v4),

(SolarisDeterministic-v0),

(SolarisDeterministic-v4),

(SolarisNoFrameskip-v0),

(SolarisNoFrameskip-v4),2d飞行类似小游戏

(SpaceInvaders-ram-v0),

(SpaceInvaders-ram-v4),

(SpaceInvaders-ramDeterministic-v0),

(SpaceInvaders-ramDeterministic-v4),

(SpaceInvaders-ramNoFrameskip-v0),

(SpaceInvaders-ramNoFrameskip-v4),

(SpaceInvaders-v0),

(SpaceInvaders-v4),

(SpaceInvadersDeterministic-v0),

(SpaceInvadersDeterministic-v4),

(SpaceInvadersNoFrameskip-v0),

(SpaceInvadersNoFrameskip-v4),

(StarGunner-ram-v0),

(StarGunner-ram-v4),

(StarGunner-ramDeterministic-v0),

(StarGunner-ramDeterministic-v4),

(StarGunner-ramNoFrameskip-v0),

(StarGunner-ramNoFrameskip-v4),

(StarGunner-v0),

(StarGunner-v4),

(StarGunnerDeterministic-v0),小游戏2d飞行

(StarGunnerDeterministic-v4),

(StarGunnerNoFrameskip-v0),

(StarGunnerNoFrameskip-v4),

(Striker-v2),

(Swimmer-v2),

(Taxi-v2),

(Tennis-ram-v0),

(Tennis-ram-v4),

(Tennis-ramDeterministic-v0),

(Tennis-ramDeterministic-v4),

(Tennis-ramNoFrameskip-v0),

(Tennis-ramNoFrameskip-v4), 网球小游戏

(Tennis-v0), 网球

(Tennis-v4),

(TennisDeterministic-v0), 小游戏打网球

(TennisDeterministic-v4), 网球小游戏

(TennisNoFrameskip-v0),

(TennisNoFrameskip-v4),

(Thrower-v2),

(TimePilot-ram-v0),

(TimePilot-ram-v4),

(TimePilot-ramDeterministic-v0), 小游戏飞云打怪

(TimePilot-ramDeterministic-v4),

(TimePilot-ramNoFrameskip-v0),

(TimePilot-ramNoFrameskip-v4),

(TimePilot-v0),

(TimePilot-v4),

(TimePilotDeterministic-v0), 小游戏飞云打怪

(TimePilotDeterministic-v4),

(TimePilotNoFrameskip-v0),

(TimePilotNoFrameskip-v4),

(Tutankham-ram-v0),

(Tutankham-ram-v4), 小游戏

(Tutankham-ramDeterministic-v0),

(Tutankham-ramDeterministic-v4),

(Tutankham-ramNoFrameskip-v0),

(Tutankham-ramNoFrameskip-v4),

(Tutankham-v0), 小游戏打怪

(Tutankham-v4),

(TutankhamDeterministic-v0),

(TutankhamDeterministic-v4),

(TutankhamNoFrameskip-v0),

(TutankhamNoFrameskip-v4),

(TwoRoundDeterministicReward-v0),

(TwoRoundNondeterministicReward-v0),

(UpNDown-ram-v0),

(UpNDown-ram-v4),

(UpNDown-ramDeterministic-v0), 小路上开车小游戏

(UpNDown-ramDeterministic-v4),

(UpNDown-ramNoFrameskip-v0),

(UpNDown-ramNoFrameskip-v4),

(UpNDown-v0), 小路上开车小游戏

(UpNDown-v4),

(UpNDownDeterministic-v0),

(UpNDownDeterministic-v4),

(UpNDownNoFrameskip-v0),

(UpNDownNoFrameskip-v4),

(Venture-ram-v0),

(Venture-ram-v4),

(Venture-ramDeterministic-v0),

(Venture-ramDeterministic-v4),

(Venture-ramNoFrameskip-v0),

(Venture-ramNoFrameskip-v4),

(Venture-v0),

(Venture-v4),

(VentureDeterministic-v0),

(VentureDeterministic-v4),

(VentureNoFrameskip-v0),

(VentureNoFrameskip-v4),

(VideoPinball-ram-v0),

(VideoPinball-ram-v4),

(VideoPinball-ramDeterministic-v0),

(VideoPinball-ramDeterministic-v4),

(VideoPinball-ramNoFrameskip-v0),

(VideoPinball-ramNoFrameskip-v4),

(VideoPinball-v0),

(VideoPinball-v4),小游戏弹球

(VideoPinballDeterministic-v0),

(VideoPinballDeterministic-v4),

(VideoPinballNoFrameskip-v0),

(VideoPinballNoFrameskip-v4),

(Walker2d-v2), 错误

(WizardOfWor-ram-v0),

(WizardOfWor-ram-v4),

(WizardOfWor-ramDeterministic-v0),

(WizardOfWor-ramDeterministic-v4),

(WizardOfWor-ramNoFrameskip-v0),

(WizardOfWor-ramNoFrameskip-v4),

(WizardOfWor-v0),

(WizardOfWor-v4), 小游戏迷宫

(WizardOfWorDeterministic-v0),

(WizardOfWorDeterministic-v4),

(WizardOfWorNoFrameskip-v0),

(WizardOfWorNoFrameskip-v4),

(YarsRevenge-ram-v0),

(YarsRevenge-ram-v4),

(YarsRevenge-ramDeterministic-v0),

(YarsRevenge-ramDeterministic-v4),

(YarsRevenge-ramNoFrameskip-v0),

(YarsRevenge-ramNoFrameskip-v4),

(YarsRevenge-v0),

(YarsRevenge-v4), 小游戏2d飞行打怪

(YarsRevengeDeterministic-v0),

(YarsRevengeDeterministic-v4),

(YarsRevengeNoFrameskip-v0),

(YarsRevengeNoFrameskip-v4), 2d打怪小游戏

(Zaxxon-ram-v0),

(Zaxxon-ram-v4), 小游戏打怪

(Zaxxon-ramDeterministic-v0),

(Zaxxon-ramDeterministic-v4),

(Zaxxon-ramNoFrameskip-v0),小游戏越过墙打怪

(Zaxxon-ramNoFrameskip-v4),

(Zaxxon-v0),

(Zaxxon-v4),

(ZaxxonDeterministic-v0),

(ZaxxonDeterministic-v4),

(ZaxxonNoFrameskip-v0),小游戏打怪

(ZaxxonNoFrameskip-v4),