CartPole v0:
CartPole-v0
A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track.
The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over.
A reward of +1 is provided forevery timestep that the pole remains upright.
The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center.
Environment
Observation
Type: Box(4)
NumObservationMinMax
0
Cart Position
-2.4
2.4
1
Cart Velocity
-Inf
Inf
2
Pole Angle
~ -41.8°
~ 41.8°
3
Pole Velocity At Tip
-Inf
Inf
Actions
Type: Discrete(2)
NumAction
0
Push cart to the left
1
Push cart to the right
Note: The amount the velocity is reduced or increased is not fixed as it depends on the angle the pole is pointing. This is because the center of gravity of the pole increases the amount of energy needed to move the cart underneath it
Reward
Reward is 1 for every step taken, including the termination step
Starting State
All observations are assigned a uniform random value between ±0.05
Episode Termination
Pole Angle is more than ±20.9°
Cart Position is more than ±2.4 (center of the cart reaches the edge of the display)
Episode length is greater than 200
Solved Requirements
Considered solved when the average reward is greater than or equal to 195.0 over 100 consecutive trials.
Only one episode here.
importgym
env= gym.make('CartPole-v0')
env.reset() # start here.for _ in range(1000):
env.render()
env.step(env.action_space.sample())#take a random action
Observations
If we ever want to do better than take random actions at each step, it'd probably be good to actually know what our actions are doing to the environment.
The environment's step function returns exactly what we need. In fact, step returns four values. These are:
observation (object): an environment-specific object representing your observation of the environment. For example, pixel data from a camera, joint angles and joint velocities of a robot, or the board state in a board game.
reward (float): amount of reward achieved by the previous action. The scale varies between environments, but the goal is always to increase your total reward.
done (boolean): whether it's time to reset the environment again. Most (but not all) tasks are divided up into well-defined episodes, and done being True indicates the episode has terminated. (For example, perhaps the pole tipped too far, or you lost your last life.)
info (dict): diagnostic information useful for debugging. It can sometimes be useful for learning (for example, it might contain the raw probabilities b