Exercise 3.11 If the current state is S t S_t St, and actions are selected according to stochastic policy π \pi π, then what is the expectation of R t + 1 R_{t+1} Rt+1 in terms of π \pi π and the four-argument function p p p(3.2)?
P r ( S t = s , A t = a ) = p ( a ∣ s ) ⋅ P r ( S t = s ) = π ( a ∣ s ) ⋅ P r ( S t = s ) ( 1 ) \begin{aligned} Pr(S_t = s, A_t = a) &= p(a|s) \cdot Pr(S_t = s) \\ &= \pi(a|s) \cdot Pr(S_t = s) \qquad{(1)} \end{aligned} Pr(St=s,At=a)=p(a∣s)⋅Pr(St=s)=π(a∣s)⋅Pr(St=s)(1)
E ( R t + 1 ∣ S t = s ) = ∑ r ∈ R [ r ⋅ P r ( R t + 1 = r ∣ S t = s ) ] = ∑ r ∈ R [ r ⋅ ∑ s ′ ∈ S P r ( R t + 1 = r , S t + 1 = s ′ ∣ S t = s ) ] = ∑ r ∈ R [ r ⋅ ∑ s ′ ∈ S P r ( R t + 1 = r , S t + 1 = s ′ , S t = s ) P r ( S t = s ) ] = ∑ r ∈ R [ r ⋅ ∑ s ′ ∈ S ∑ a ∈ A P r ( R t + 1 =
最低0.47元/天 解锁文章
101
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
