[NOTE] Advice and Perspectives on RL Research Frontiers - Rich Sutton in DLRLSS 2019

根据我的习惯，当然先放ressources：slides，video. 这是Sutton在DLRLSS 2019 summer school上的一个lecture，从他自己的角度分享了对RL领域的一些理解，他目前的研究方向及前沿等。一些思考还是很有启发的。个别要点摘录于此，细节可以自行阅读、观看。

Developing your own research thoughts

1. There are no authorities in science. Be ambitious but also humble. Your own thought is of great value.
2. One best way of training is to write for yourself and discuss with others.
3. When thinking on big questions, it's easy to get stuck:
   1. Define your own terms
   2. Go multiple: think about alternatives
   3. Go meta: what are the properties that the solution should have
   4. Retreat to clearer question
4. The most important insight you will ever contribute is too obvious to see.(The discovery of gravity)

“Completing the square” for doing RL research

Research that Sutton is doing

有必要更加深入地理解Prediction和Control的联系与区别。

下文简而言之，Sutton is working on subprolems. The world env is often too complex to learn as a whole. It's natural to have multiple components like different parts of the body. I think it's a bit like the multi-agent concept, whose goals may not directly relate to the global reward.

关于Permanent memory的部分其实非常有想象空间。