《reinforcement learning：an introduction》第十一章《Off-policy Methods with Approximation》总结

由于组里新同学进来，需要带着他入门RL，选择从silver的课程开始。

对于我自己，增加一个仔细阅读《reinforcement learning：an introduction》的要求。

因为之前读的不太认真，这一次希望可以认真一点，将对应的知识点也做一个简单总结。

The tabular off-policy methods developed in Chapters 6 and 7 readily extend to semi-gradient algorithms, but these algorithms do not converge nearly as robustly as in the on-policy case.  The on-policy distribution is special and is important to the stability of semi-gradient methods. 

Methods developed in earlier chapters for the off-p