Bayesian non-negative matrix factorization核心过程推导

最新推荐文章于 2022-02-24 17:37:46 发布

教授我要发文章

最新推荐文章于 2022-02-24 17:37:46 发布

阅读量1k

点赞数 2

分类专栏： Bayesian 文章标签： Bayesian Non-negative Matrix Factorizat

本文链接：https://blog.csdn.net/tianjiangongyangyu/article/details/83789708

版权

Bayesian 专栏收录该内容

0 篇文章 0 订阅

订阅专栏

最近阅读了一篇老文章—Bayesian non-negative matrix factorization，是个论文集，本文在540-547页，这篇文章用贝叶斯方法重做了一遍非负矩阵分解，但其推导过程过于简略，本人将记录一下其核心推导过程，也就是原文公式(5)和公式(7)的推导过程。

定义 ${\bf{X}} = {\bf{AB}} + {\bf{E}}$ ，其中 ${\bf{X}} \in {R^{I \times J}}$ ， ${\bf{A}} \in {R^{I \times N}}$ , ${\bf{B}} \in {R^{N \times J}}$ ，关于 $${\bf{X}}$ 的似然函数为：

$p\left( {{\bf{X}}\left| {{\bf{A}},{\bf{B}},{\sigma ^2}} \right.} \right) = \prod\limits_{i,j} {{\cal N}\left( {{{\bf{X}}_{i,j}}\left| {{{\left( {{\bf{AB}}} \right)}_{i,j}},{\sigma ^2}} \right.} \right)} = {\prod\limits_{i,j} {\left( {2\pi {\sigma ^2}} \right)} ^{ - 1/2}}\exp \left\{ { - {{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}/\left( {2{\sigma ^2}} \right)} \right\}$

而变量 ${\bf{A}}$ 和 ${\bf{B}}$ 的先验为：

$p\left( {\bf{A}} \right) = \prod\limits_{i,n} {\varepsilon \left( {{{\bf{A}}_{i,n}};{\alpha _{i,n}}} \right)} = \prod\limits_{i,n} {{\alpha _{i,n}}{\rm{exp}}\left( { - {\alpha _{i,n}}{{\bf{A}}_{i,n}}} \right)} u\left( {{{\bf{A}}_{i,n}}} \right)$

和

$p\left( {\bf{B}} \right) = \prod\limits_{n,j} {\varepsilon \left( {{{\bf{B}}_{n,j}};{\beta _{n,j}}} \right)} = \prod\limits_{n,j} {{\beta _{n,j}}{\rm{exp}}\left( { - {\beta _{n,j}}{{\bf{B}}_{n,j}}} \right)} u\left( {{{\bf{B}}_{n,j}}} \right)$

另外，再定义噪声方差 ${{\sigma ^2}}$ 的先验：

$p\left( {{\sigma ^2}} \right) = {{\cal G}^{ - 1}}\left( {{\sigma ^2}{\rm{;}}k{\rm{,}}\theta } \right) = \frac{{{\theta ^k}}}{{\Gamma \left( k \right)}}{\left( {{\sigma ^2}} \right)^{ - k - 1}}{\rm{exp}}\left( { - \frac{\theta }{{{\sigma ^2}}}} \right)$

关于 ${\bf{A}}$ 和 ${\bf{B}}$ 的条件后验密度是一个高斯分布乘以一个截断的指数分布，也就是一个截断的高斯分布，我们定义这种形式为 ${\cal R}\left( {x{\rm{;}}\mu {\rm{,}}{\sigma ^2}{\rm{,}}\lambda } \right) \propto {\cal N}\left( {x{\rm{;}}\mu {\rm{,}}{\sigma ^2}} \right)\varepsilon \left( {x{\rm{;}}\lambda } \right)$ ，因此，关于 ${{\bf{A}}_{i,n}}$ 的条件概率密度为：

$\begin{array}{l} p\left( {{{\bf{A}}_{i,n}}\left| {{\bf{X}},{{\bf{A}}_{{\rm{\backslash (}}i,n)}},{\bf{B}}} \right.,{\sigma ^2}} \right) = {\cal R}\left( {{{\bf{A}}_{i,n}}{\rm{;}}{\mu _{{{\bf{A}}_{i,n}}}}{\rm{,}}\sigma _{{{\bf{A}}_{i,n}}}^2{\rm{,}}{\alpha _{i,n}}} \right) = {\cal N}\left( {{{\bf{A}}_{i,n}}{\rm{;}}{\mu _{{{\bf{A}}_{i,n}}}}{\rm{,}}\sigma _{{{\bf{A}}_{i,n}}}^2} \right)\varepsilon \left( {{{\bf{A}}_{i,n}}{\rm{;}}{\alpha _{i,n}}} \right)\\ = \varepsilon \left( {{{\bf{A}}_{i,n}}{\rm{;}}{\alpha _{i,n}}} \right)\prod\limits_j {{{\left( {2\pi {\sigma ^2}} \right)}^{ - 1/2}}} \exp \left\{ { - {{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}/2{\sigma ^2}} \right\} \end{array}$ (1)

为了方便表示，我们先考虑上式的指数部分：

其中：

${\left( {\sum\limits_n {{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}} } \right)^2} = {\left( {{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}} \right)^2} + 2{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}\sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} + {\left( {\sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)^2}$ (3)

将(3)回代入(2)：

$\begin{array}{l} - \frac{1}{{2{\sigma ^2}}}\sum\limits_j {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}} \\ = - \frac{1}{{2{\sigma ^2}}}\sum\limits_j {\left\{ {{\bf{X}}_{i,j}^2 - 2{{\bf{X}}_{i,j}}{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}} - 2{{\bf{X}}_{i,j}}\sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} + {{\left( {{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}} \right)}^2} + 2{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}\sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} + {{\left( {\sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)}^2}} \right\}} \\ = - \frac{1}{{2{\sigma ^2}}}\sum\limits_j {\left\{ {{{\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)}^2} - 2{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right) + {{\left( {{{\bf{A}}_{i,n}}{{\bf{B}}_{n,j}}} \right)}^2}} \right\}} \end{array}$

$\begin{array}{l} = - \frac{1}{{2{\sigma ^2}}}\left[ {{{\left( {{{\bf{A}}_{i,n}}} \right)}^2}\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2} - 2{{\bf{A}}_{i,n}}\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right) + \sum\limits_j {{{\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)}^2}} } } } \right]\\ = - \frac{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}{{2{\sigma ^2}}}\left[ {{{\left( {{{\bf{A}}_{i,n}}} \right)}^2} - \frac{{2{{\bf{A}}_{i,n}}\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }} + \cdots } \right] \end{array}$

推导进行到这一步后，会发现根本凑不出来完全平方项，但通过观察第二项，发现除了 ${{\bf{A}}_{i,n}}$ ， $\frac{{2\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}$ 与 ${{\bf{A}}_{i,n}}$ 无关，而且上式中，除了前两项，第三项 $\sum\limits_j {{{\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)}^2}}$ 也与 ${{\bf{A}}_{i,n}}$ 无关，因此，我们可以根据前两项配出完全平方项，而多余的部分由于在指数上最后就会变成一个比例常数项，因此，上式可以改写为：

$\begin{array}{l} - \frac{1}{{2{\sigma ^2}}}\sum\limits_j {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}} \\ = - \frac{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}{{2{\sigma ^2}}}\left[ {{{\left( {{{\bf{A}}_{i,n}}} \right)}^2} - \frac{{2{{\bf{A}}_{i,n}}\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }} + {{\left\{ {\frac{{\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}} \right\}}^2}} \right] \end{array}$

$= - \frac{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}{{2{\sigma ^2}}}{\left\{ {{{\bf{A}}_{i,n}} - \frac{{\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}} \right\}^2} + C$ (4)

将公式(4)回代如公式(1)：

将公式(4)回代如公式(1)： $p\left( {{{\bf{A}}_{i,n}}\left| {{\bf{X}},{{\bf{A}}_{{\rm{\backslash (}}i,n)}},{\bf{B}}} \right.,{\sigma ^2}} \right) = {\cal R}\left( {{{\bf{A}}_{i,n}}{\rm{;}}{\mu _{{{\bf{A}}_{i,n}}}}{\rm{,}}\sigma _{{{\bf{A}}_{i,n}}}^2{\rm{,}}{\alpha _{i,n}}} \right) = {\cal N}\left( {{{\bf{A}}_{i,n}}{\rm{;}}{\mu _{{{\bf{A}}_{i,n}}}}{\rm{,}}\sigma _{{{\bf{A}}_{i,n}}}^2} \right)\varepsilon \left( {{{\bf{A}}_{i,n}}{\rm{;}}{\alpha _{i,n}}} \right)$

$\propto \varepsilon \left( {{{\bf{A}}_{i,n}}{\rm{;}}{\alpha _{i,n}}} \right)\prod\limits_j {{{\left( {2\pi {\sigma ^2}/\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} } \right)}^{ - 1/2}}} \exp \left\{ { - \frac{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}{{2{\sigma ^2}}}{{\left[ {{{\bf{A}}_{i,n}} - \frac{{\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}} \right]}^2}} \right\}$

因此， ${\cal N}\left( {{{\bf{A}}_{i,n}}{\rm{;}}{\mu _{{{\bf{A}}_{i,n}}}}{\rm{,}}\sigma _{{{\bf{A}}_{i,n}}}^2} \right)$ 中：

${\mu _{{{\bf{A}}_{i,n}}}} = \frac{{\sum\limits_j {{{\bf{B}}_{n,j}}\left( {{{\bf{X}}_{i,j}} - \sum\limits_{n' \ne n} {{{\bf{A}}_{i,n'}}{{\bf{B}}_{n',j}}} } \right)} }}{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}$

$\sigma _{{{\bf{A}}_{i,n}}}^2 = \frac{{\sum\limits_j {{{\left( {{{\bf{B}}_{n,j}}} \right)}^2}} }}{{2{\sigma ^2}}}$

关于噪声方差：

$\begin{array}{l} p\left( {{\sigma ^2}\left| {{\bf{A}},{\bf{B}},{\bf{X}}} \right.} \right) \propto p\left( {{\sigma ^2}} \right)p\left( {{\bf{X}}\left| {{\bf{A}},{\bf{B}},{\sigma ^2}} \right.} \right) = {{\cal G}^{ - 1}}\left( {{\sigma ^2};{k_{{\sigma ^2}}},{\theta _{{\sigma ^2}}}} \right)\\ = \frac{{{\theta ^k}}}{{\Gamma \left( k \right)}}{\left( {{\sigma ^2}} \right)^{ - k - 1}}{\rm{exp}}\left( { - \frac{\theta }{{{\sigma ^2}}}} \right){\prod\limits_{i,j} {\left( {2\pi {\sigma ^2}} \right)} ^{ - 1/2}}\exp \left\{ { - {{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}/\left( {2{\sigma ^2}} \right)} \right\} \end{array}$

$\begin{array}{l} = \frac{{{\theta ^k}}}{{\Gamma \left( k \right)}}{\left( {{\sigma ^2}} \right)^{ - k - 1}}{\rm{exp}}\left( { - \frac{\theta }{{{\sigma ^2}}}} \right){\left( {2\pi {\sigma ^2}} \right)^{ - IJ/2}}\exp \left\{ { - \sum\limits_{i,j} {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}/\left( {2{\sigma ^2}} \right)} } \right\}\\ \propto \frac{{{{\left[ {\theta + \frac{1}{2}\sum\limits_{i,j} {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}} } \right]}^k}}}{{\Gamma \left( {k + IJ/2} \right)}}{\left( {{\sigma ^2}} \right)^{ - k - 1 - IJ/2}}\exp \left( { - \frac{{\theta + \frac{1}{2}\sum\limits_{i,j} {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}} }}{{{\sigma ^2}}}} \right) \end{array}$

因此，噪声方差所服从的逆伽马分布的参数更新公式分别为：

${k_{{\sigma ^2}}} = k + IJ/2$ (感觉此处原文有误，原文多加了个1)，

${\theta _{{\sigma ^2}}} = \theta + \frac{1}{2}\sum\limits_{i,j} {{{\left( {{{\bf{X}}_{i,j}} - {{\left( {{\bf{AB}}} \right)}_{i,j}}} \right)}^2}}$

教授我要发文章

关注

2
点赞
踩
2

收藏

觉得还不错? 一键收藏
3
评论
Bayesian non-negative matrix factorization核心过程推导

最近阅读了一篇老文章—Bayesian non-negative matrix factorization，是个论文集，本文在540-547页，这篇文章用贝叶斯方法重做了一遍非负矩阵分解，但其推导过程过于简略，本人将记录一下其核心推导过程，也就是原文公式(5)和公式(7)的推导过程。定义，其中，,，关于的似然函数为：而变量和的先验为：和另外，...
复制链接

扫一扫