强化学习探索 BEBOLD里的金句

对长的trajectory有偏好，这是真的，这也造成了agent有时候会来回蠕动

As mentioned in Go-Explore series (Ecoffet et al., 2019; 2020), count-based approaches also suffer from detachment: if the agent by chance starts exploring τ2 after briefly exploring the first few states of τ1, it would not return and explore τ1 further since τ1 is now “shorter” than τ2 and has lower IR than τ2 for a long period. Go-Explore tries to resolve this dilemma between “dedication” and “exploration” by using a two-stage approach with many hand-tuned parameters.

主要就是这个，对长trajectory有偏好，其他和novelD一样，嗯。。。。