upr_rom的博客

DDPG, TD3, Soft Actor-critic

本文将讨论分布空间的Natrual Gradient, 然后将Natural Gradient 用于Actor Critic。另外说明Trust Region Policy Optimization(TRPO) 和 Proximal Policy Optimization(PPO)算法。

本文会总结深度强化学习中基于策略学习的各种方法，从优化的角度说明REINFORCE（Monte Carlo based policy gradient）和参数空间Vanilla policy gradient的各种Actor Critic Methods.因为基于策略学习的方法因为能够在状态和动作的连续空间中进行，其在机器人控制中会经常用到，本文后续会分析策略学习的强化学习在机器人灵

DefinitionDynamic Programming uses the euqations in the second line and has to know the environmental dynamics ( dynamics of environemntcan produce the chain between this state to the next state, but it's hard to know.Cons) ( It uses the rel...

BackgroundIn3.1 Monte Carlo Methods & case study: Blackjack of Policy Evaluation, we finished the evaluation for the specific policy ( hit unless 20 or21). In this article, we will summarize the policy improvement for

BackgroundIn many cases, we are not able to find the examples inour specific policy. However, we could find examples in other policies with actions and statesappearing in our policy.What's more,in deterministic on-policy, we always have to compromis...

Monte CarloDefinitionMonte Carlo PredictionDefinitionPesudocodeInitialization: Returns = [ ]Loop for N times: generate an episide following the spesific policy G = 0loop for each step of episode, t = T-1...

# **Finite Markov Decision Process**

Value IterationBackgroundPolicy iteration's process is that after value function (policy evaluation) converges, the policy then improved. In fact, there is no need that value function converges because even nothe last many sweeps of policy evaluation,.

Dynamic programmingPolicy evaluationBy iterations, we could get value function set ( v(s) ) for specific policy, given the environment's dynamics.pesudocode