Deep Reinforcement Learning (DRL) Algorithms with PyTorch
This repository contains PyTorch implementations of deep reinforcement learning algorithms. This implementation uses PyTorch. For a TensorFlow implementation of algorithms, take a look at tsallis_actor_critic_mujoco.
Algorithms Implemented
Deep Q-Network (DQN)
Advantage Actor Critic (A2C)
Vanilla Policy Gradient (VPG)
Natural Policy Gradient (NPG)
Trust Region Policy Optimization (TRPO)
Proximal Policy Optimization (PPO)
Deep Deterministic Policy Gradient (DDPG)
Twin Delayed DDPG (TD3)
Soft Actor-Critic (SAC)
Automating entropy adjustment on SAC (ASAC)
Tsallis Actor-Critic (TAC)
Automating entropy adjustment on TAC (ATAC)
Environments Implemented
CartPole-v1 (as described in here)
Pendulum-v0 (as described in here)
MuJoCo environments (HalfCheetah-v2, Ant-v2, Humanoid-v2, etc.) (as described in here)
Results
CartPole-v1
Observation space: 4
Action space: 2
Pendulum-v0
Observation space: 3
Action space: 1
HalfCheetah-v2
Observation space: 17
Action space: 6
Ant-v2
Observation space: 111
Action space: 8
Humanoid-v2
Observation space: 376
Action space: 17
Requirements
Usage
The repository's high-level structure is:
├── agents
└── common
├── results
├── data
└── graphs
├── tests
└── save_model
1) To train the agents on the environments
To train all the different agents on MuJoCo environments, follow these steps:
git clone https://github.com/dongminlee94/deep_rl.git
cd deep_rl
python run_mujoco.py
For other environments, change the last line to run_cartpole.py, run_pendulum.py.
If you want to change configurations of the agents, follow this step:
python run_mujoco.py \
--env=Humanoid-v2 \
--algo=atac \
--seed=0 \
--iterations=200 \
--steps_per_iter=5000 \
--max_step=1000
2) To watch the learned agents on the above environments
To watch all the learned agents on MuJoCo environments, follow these steps:
cd tests
python mujoco_test.py --load=envname_algoname_...
You should copy the saved model name in tests/save_model/envname_algoname_... and paste the copied name in envname_algoname_.... So the saved model will be load.