Q-Learning

Kardi Teknomo先生给出的Q-learning简单教程，我看到有几个朋友已经翻译并转到了自己的博客。算法思路简单，涉及一个公式如下，其他不再赘述，以下是给出其python演示。

Q(state,action)=R(state,action)+\gamma * Max(Q(nextState,allPossibleActions))

import random
gamma=0.8
R = [[-1,-1,-1,-1,0,-1],[-1,-1,-1,0,-1,100],[-1,-1,-1,0,-1,-1],[-1,0,0,-1,0,-1],[0,-1,-1,0,-1,100],[-1,0,-1,-1,0,100]]
Q=[[0.0 for i in range(6)] for j in range(6)]
times=0
while times<1000:
	state=random.randint(0,5)
	while True:
		action=next_state=random.choice([x for x in range(len(R[state])) if R[state][x]!=-1])
		next_action_possible=[x for x in range(len(R[next_state])) if R[next_state][x]!=-1]
		reward=[Q[next_state][j] for j in next_action_possible]	
		Q[state][action]=R[state][action]+gamma*max(reward)
		state=action
		if state==5:
			break
	times+=1
print(Q)

此外最后给出了一个练习如上图所示，实现如下。

import random
gamma=0.8
R=[[-1,-1,-1,-1,0,-1,-1,-1],[-1,-1,0,-1,-1,0,-1,-1],[-1,0,-1,100,-1,-1,0,-1],[-1,-1,0,100,-1,-1,-1,-1],[0,-1,-1,-1,-1,0,-1,-1],[-1,0,-1,-1,0,-1,0,-1],[-1,-1,0,-1,-1,0,-1,0],[-1,-1,-1,-1,-1,-1,0,-1]]
Q=[[0.0 for i in range(8)] for j in range(8)]
times=0
while times<1000:
    state=random.randint(0,7)
    while True:
        action=next_state=random.choice([x for x in range(len(R[state])) if R[state][x]!=-1])
        next_action_possible=[x for x in range(len(R[next_state])) if R[next_state][x]!=-1]
        reward=[Q[next_state][j] for j in next_action_possible]    
        Q[state][action]=R[state][action]+gamma*max(reward)
        state=action
        if state==3:
            break
    times+=1
print(Q)

转载于:https://my.oschina.net/fazheng/blog/684901