Q-learning算法实现1（matlab）

算法伪代码：

得到Q表后，根据如下算法选择最优策略：

以机器人走房间为例，代码实现如下：

原文链接如下：https://www.jianshu.com/p/29db50000e3f

注：原文中的房间状态0-5分别对应代码中1-6

%机器人走房间Q-learning的实现
%% 基本参数
episode=100; %探索的迭代次数
alpha=1;%更新步长
gamma=0.8;%折扣因子
state_num=6;
action_num=6;
final_state=6;%目标房间
Reward_table = [
-1 -1 -1 -1 0 -1; %1
-1 -1 -1 0 -1 100; %2
-1 -1 -1 0 -1 -1; %3
-1 0 0 -1 0 -1; %4
0 -1 -1 0 -1 100; %5
-1 0 -1 -1 0 100 %6
];
%% 更新Q表
%initialize Q(s,a)
Q_table=zeros(state_num,action_num);
for i=1:episode
    %randomly choose a state
    current_state=randperm(state_num,1);
    while current_state~=final_state
        %randomly choose an action from current state
        optional_action=find(Reward_table(current_state,:)>-1);
        chosen_action=optional_action(randperm(length(optional_action),1));
        %take action, observe reward and next state
        r=Reward_table(current_state,chosen_action);
        next_state=chosen_action;
        %update Q-table
        next_possible_action=find(Reward_table(next_state,:)>-1);
        maxQ=max(Q_table(next_state,next_possible_action));
        Q_table(current_state,chosen_action)=Q_table(current_state,chosen_action)+alpha*(r+gamma*maxQ-Q_table(current_state,chosen_action));
        current_state=next_state;
    end
end
 %% 选择最优路径
 %randomly choose a state
currentstate=randperm(state_num,1);
fprintf('Initialized state %d\n',currentstate);
%choose action which satisfies Q(s,a)=max{Q(s,a')}
while currentstate~=final_state
     [maxQtable,index]=max(Q_table(currentstate,:));
     chosenaction=index;
     nextstate=chosenaction;
     fprintf('the robot goes to %d\n',nextstate);
     currentstate=nextstate;
end
        

代码输出：

Q表：

 

最优策略：