题意:在OpenAI Gym的上下文中,spaces.Discrete
是指什么
问题背景:
I try to learn MC- Monte Carlo Method applied in blackjack using openAI Gym. And I do not understand these lines:
我尝试使用OpenAI Gym学习蒙特卡洛方法(Monte Carlo Method,简称MC)在黑杰克(Blackjack)游戏中的应用。但我不理解这些代码行/概念/部分:
def __init__(self, natural=False):
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Tuple((
spaces.Discrete(32),
spaces.Discrete(11),
spaces.Discrete(2)))
self.seed()
Source from: https://github.com/openai/gym/blob/master/gym/envs/toy_text/blackjack.py
源码引用地址
问题解决:
The observation space and the action space has been defined in the comments here
这里的注释中已经定义了观察空间(observation space)和动作空间(action space)
Observation Space: 观察空间
The observation of a 3-tuple of: the player's current sum,
the dealer's one showing card (1-10 where 1 is ace),
and whether or not the player holds a usable ace (0 or 1).
eg: (14, 9, False) means the current sum is 14, card shown is 9 and there is no usable ace(because ace can be used as 1 or 11)
Action Space: 动作空间
The player can request additional cards (hit=1) until they decide to stop
(stick=0) or exceed 21 (bust).
Discrete spaces are used when we have a discrete action/observation space to be defined in the environment. So spaces.Discrete(2)
means that we have a discrete variable which can take one of the two possible values.
当我们需要在环境中定义离散的动作/观察空间时,就会使用离散空间。因此,spaces.Discrete(2)
意味着我们有一个离散变量,它可以取两个可能值中的任意一个。
In the Blackjack environment, 在 Blackjack 环境中
self.action_space = spaces.Discrete(2)
# here spaces.Discrete(2) means that action can either be True or False.
self.observation_space = spaces.Tuple((
spaces.Discrete(32),
spaces.Discrete(11),
spaces.Discrete(2)))
# here spaces.Discrete(32) corresponds to the 32 possible sum of card number possible
# here spaces.Discrete(11) corresponds to the 11 possible cards which can be dealed
# by the dealer: [1,2,3,4,5,6,7,8,9,10(king,queen,jack),11(ace if possible)]
# here spaces.Discrete(2) corresponds to the two possible uses of the ace: [True, False]
# True if it can be used as 11.