摘自:beal ph.d thesis variational algorithms for approximate bayesian inference
The key to the variational method is to approximate the integral with a simpler form that is tractable, forming a lower or upper bound. The integration then translates into the implementationally simpler problem of bound optimisation: making the bound as tight as possible to the true value.
看到这里,我想起来LDA中的处理了。只是,我现在还想不通那次电面GG问我变分如何在LDA中体现的,我说将不可解的近似为可解了,通过引入一个辅助变量,本来不是条件独立的,处理为条件独立了。但是当时,他好像觉得回答的对不上,又详细解释了一下问题。我感觉变分在LDA中就是一个近似的思想。大概他想问的是最后是如何近似处理的?KL的使用,使得近似形式最优化处理了。难道他想知道的是KL这部分?有点没想明白。