PRML一书中关于贝叶斯曲线拟合结论的推导细节
我们令训练数据集为
(
X
,
T
)
(X,T)
(X,T), 对于一个新的点
x
x
x, 我们希望给出一个预测分布
p
(
t
∣
x
,
X
,
T
)
p(t|x, X, T)
p(t∣x,X,T)
p
(
t
∣
x
,
X
,
T
)
=
∫
p
(
t
∣
x
,
w
,
X
,
T
)
d
w
=
∫
p
(
t
∣
x
,
w
)
p
(
w
∣
X
,
T
)
d
w
p(t|x,X,T) =\int p(t|x,w,X,T)dw= \int p(t|x,w)p(w|X,T)dw \\
p(t∣x,X,T)=∫p(t∣x,w,X,T)dw=∫p(t∣x,w)p(w∣X,T)dw
其中，
w
=
[
w
0
,
w
1
,
.
.
.
,
w
M
]
T
w=[w_0,w_1,...,w_M]^T
w=[w0,w1,...,wM]T 是
M
M
M阶多项式的参数
在PRML一书中，直接给出了这么一个结论
预测分布由高斯分布 N ( t ∣ m ( x ) , s 2 ( x ) ) \mathcal N(t|m(x),s^2(x)) N(t∣m(x),s2(x))给出
其中均值 m ( x ) = β ϕ ( x ) T S ∑ n = 1 N ϕ ( x n ) t n m(x) = \beta\phi(x)^TS\sum_{n=1}^{N}\phi(x_n)t_n m(x)=βϕ(x)TS∑n=1Nϕ(xn)tn
方差 s 2 ( x ) = β − 1 + ϕ ( x ) T S ϕ ( x ) s^2(x) = \beta^{-1} + \phi(x)^TS\phi(x) s2(x)=β−1+ϕ(x)TSϕ(x)
其中 S − 1 = α I + β ∑ n = 1 N ϕ ( x n ) ϕ ( x n ) T S^{-1} = \alpha I +\beta\sum_{n=1}^{N}\phi(x_n)\phi(x_n)^T S−1=αI+β∑n=1Nϕ(xn)ϕ(xn)T, ϕ ( x ) = [ 1 , x , x 2 , . . . , x M ] T \phi(x)=[1,x,x^2,...,x^M]^T ϕ(x)=[1,x,x2,...,xM]T
这个结论给的非常突然，让人无所适从，我决定花点时间分析一下，并记录下来，以供大家参考！
如上式所述，其可以被写成积分的形式，我们利用一些结论来进行分析
- 首先对于t的分布应该是一个高斯分布，
p ( t ∣ x , w ) = N ( t ∣ y ( x , w ) , β − 1 ) p(t|x,w) = \mathcal N(t|y(x,w), \beta^{-1}) p(t∣x,w)=N(t∣y(x,w),β−1)
- 对于分布 p ( w ∣ X , T ) p(w|X,T) p(w∣X,T), 其正比于先验分布和似然的乘积
p ( w ∣ X , T , α , β ) ∝ p ( T ∣ X , w , β ) p ( w ∣ α ) p(w|X,T, \alpha, \beta) \varpropto p(T|X, w, \beta)p(w|\alpha) p(w∣X,T,α,β)∝p(T∣X,w,β)p(w∣α)
如果
α
,
β
\alpha, \beta
α,β 已知， 可以写成：
p
(
w
∣
X
,
T
)
=
p
(
T
∣
X
,
w
)
p
(
X
,
w
)
p
(
X
,
T
)
=
p
(
T
∣
X
,
w
)
p
(
w
∣
X
)
p
(
X
)
p
(
T
∣
X
)
p
(
X
)
=
p
(
T
∣
X
,
w
)
p
(
w
∣
X
)
p
(
T
∣
X
)
p(w|X,T) = \frac{p(T|X,w)p(X,w)}{p(X,T)} = \frac{p(T|X,w)p(w|X)p(X)}{p(T|X)p(X)} = \frac{p(T|X,w)p(w|X)}{p(T|X)}
p(w∣X,T)=p(X,T)p(T∣X,w)p(X,w)=p(T∣X)p(X)p(T∣X,w)p(w∣X)p(X)=p(T∣X)p(T∣X,w)p(w∣X)
所以我们可以继续将积分式子改写：
p
(
t
∣
x
,
X
,
T
)
=
∫
N
(
t
∣
y
(
x
,
w
)
,
β
−
1
)
∏
n
=
1
N
N
(
t
n
∣
y
(
x
n
,
w
)
,
β
−
1
)
p
(
w
∣
X
)
p
(
T
∣
X
)
d
w
p(t|x,X,T) = \int \mathcal N(t|y(x,w), \beta^{-1}) \prod_{n=1}^N\mathcal N(t_n|y(x_n, w), \beta^{-1})\frac{p(w|X)}{p(T|X)}dw
p(t∣x,X,T)=∫N(t∣y(x,w),β−1)n=1∏NN(tn∣y(xn,w),β−1)p(T∣X)p(w∣X)dw
y ( x , w ) = w 0 + w 1 x + w 2 x 2 + ⋯ + w M x M = ϕ ( x ) T [ w 0 , w 1 , . . . , w M ] T = ϕ ( x ) T w ϕ ( x ) = [ 1 , x , x 2 , . . . , x M ] T y(x,w) = w_0 + w_1x + w_2x^2 + \cdots + w_Mx^M = \phi(x)^T[w_0, w_1, ..., w_M]^T = \phi(x)^Tw\\ \phi(x) = [1,x,x^2,...,x^M]^T y(x,w)=w0+w1x+w2x2+⋯+wMxM=ϕ(x)T[w0,w1,...,wM]T=ϕ(x)Twϕ(x)=[1,x,x2,...,xM]T
从而，对于高斯分布
N
(
t
∣
y
(
x
,
w
)
,
β
−
1
)
\mathcal N(t|y(x,w), \beta^{-1})
N(t∣y(x,w),β−1), 可以写成:
N
(
t
∣
y
(
x
,
w
)
,
β
−
1
)
=
1
2
π
β
−
1
exp
(
−
(
t
−
y
(
x
,
w
)
)
2
2
β
−
1
)
=
1
2
π
β
−
1
exp
(
−
(
t
−
ϕ
(
x
)
T
w
)
2
2
β
−
1
)
\mathcal N(t|y(x,w), \beta^{-1}) = \frac{1}{\sqrt{2\pi\beta^{-1}}}\exp(-\frac{(t-y(x,w))^2}{2\beta^{-1}}) = \frac{1}{\sqrt{2\pi\beta^{-1}}}\exp(-\frac{(t-\phi(x)^Tw)^2}{2\beta^{-1}})
N(t∣y(x,w),β−1)=2πβ−11exp(−2β−1(t−y(x,w))2)=2πβ−11exp(−2β−1(t−ϕ(x)Tw)2)
同样的，我们可以写出N个高斯分布乘积的形式
∏
n
=
1
N
N
(
t
n
∣
y
(
x
n
,
w
)
,
β
−
1
)
=
(
1
2
π
β
−
1
)
N
/
2
exp
(
−
β
2
∑
n
=
1
N
(
t
n
−
ϕ
(
x
n
)
T
w
)
2
)
\prod_{n=1}^N\mathcal N(t_n|y(x_n, w), \beta^{-1})=(\frac{1}{2\pi\beta^{-1}})^{N/2}\exp(-\frac{\beta}{2}\sum_{n=1}^N(t_n-\phi(x_n)^Tw)^2)
n=1∏NN(tn∣y(xn,w),β−1)=(2πβ−11)N/2exp(−2βn=1∑N(tn−ϕ(xn)Tw)2)
于是，如下：
N
(
t
∣
y
(
x
,
w
)
,
β
−
1
)
∏
n
=
1
N
N
(
t
n
∣
y
(
x
n
,
w
)
,
β
−
1
)
=
(
1
2
π
β
−
1
)
(
N
+
1
)
/
2
exp
(
−
β
2
(
(
t
−
ϕ
(
x
)
T
w
)
2
+
∑
n
=
1
N
(
t
n
−
ϕ
(
x
n
)
T
w
)
2
)
)
\mathcal N(t|y(x,w), \beta^{-1}) \prod_{n=1}^N\mathcal N(t_n|y(x_n, w), \beta^{-1}) = (\frac{1}{2\pi\beta^{-1}})^{(N+1)/2}\exp(-\frac{\beta}{2}((t-\phi(x)^Tw)^2 + \sum_{n=1}^N(t_n-\phi(x_n)^Tw)^2))
N(t∣y(x,w),β−1)n=1∏NN(tn∣y(xn,w),β−1)=(2πβ−11)(N+1)/2exp(−2β((t−ϕ(x)Tw)2+n=1∑N(tn−ϕ(xn)Tw)2))
如果我们将
p
(
w
∣
X
)
=
p
(
w
∣
α
)
=
(
α
2
π
)
(
M
+
1
)
/
2
exp
(
−
α
2
w
T
w
)
p(w|X) = p(w|\alpha) = (\frac{\alpha}{2\pi})^{(M+1)/2}\exp(-\frac{\alpha}{2}w^Tw)
p(w∣X)=p(w∣α)=(2πα)(M+1)/2exp(−2αwTw)， 且
p
(
T
∣
X
)
=
1
p(T|X)=1
p(T∣X)=1
则有
p
(
t
∣
x
,
X
,
T
)
=
(
β
2
π
)
N
+
1
2
(
α
2
π
)
M
+
1
2
∫
exp
(
−
α
2
w
T
w
−
β
2
(
(
t
−
ϕ
(
x
)
T
w
)
2
+
∑
n
=
1
N
(
t
n
−
ϕ
(
x
n
)
T
w
)
2
)
)
d
w
p(t|x,X,T) = (\frac{\beta}{2\pi })^{\frac{N+1}{2}} (\frac{\alpha}{2\pi})^{ \frac{M+1}{2}}\int \exp\Big(-\frac{\alpha}{2}w^Tw -\frac{\beta}{2}((t-\phi(x)^Tw)^2 + \sum_{n=1}^N(t_n-\phi(x_n)^Tw)^2) \Big ) dw
p(t∣x,X,T)=(2πβ)2N+1(2πα)2M+1∫exp(−2αwTw−2β((t−ϕ(x)Tw)2+n=1∑N(tn−ϕ(xn)Tw)2))dw
注意到，高斯积分的形式
∫
−
∞
+
∞
e
−
a
(
x
+
b
)
2
d
x
=
π
a
\int _{-\infin}^{+\infin}e^{-a(x+b)^2} dx = \sqrt{\frac{\pi}{a}}
∫−∞+∞e−a(x+b)2dx=aπ
故,
∫
exp
(
−
α
2
w
T
w
−
β
2
(
(
t
−
ϕ
(
x
)
T
w
)
2
+
∑
n
=
1
N
(
t
n
−
ϕ
(
x
n
)
T
w
)
2
)
)
d
w
=
∫
exp
(
−
k
(
w
+
b
)
2
+
u
)
d
w
k
=
α
2
+
β
2
(
ϕ
(
x
)
T
ϕ
(
x
)
+
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
)
\int \exp\Big(-\frac{\alpha}{2}w^Tw -\frac{\beta}{2}((t-\phi(x)^Tw)^2 + \sum_{n=1}^N(t_n-\phi(x_n)^Tw)^2) \Big ) dw = \int\exp(-k(w+b)^2+u)dw\\ k = \frac{\alpha}{2} + \frac{\beta}{2}(\phi(x)^T\phi(x) + \sum_{n=1}^{N}\phi(x_n)^T\phi(x_n))
∫exp(−2αwTw−2β((t−ϕ(x)Tw)2+n=1∑N(tn−ϕ(xn)Tw)2))dw=∫exp(−k(w+b)2+u)dwk=2α+2β(ϕ(x)Tϕ(x)+n=1∑Nϕ(xn)Tϕ(xn))
相当于对于一个二次式进行配方，我们简单记作：
−
m
1
w
2
−
m
2
(
m
3
w
2
+
m
4
w
+
m
5
)
m
1
=
α
2
,
m
2
=
β
2
m
3
=
ϕ
(
x
)
T
ϕ
(
x
)
+
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
m
4
=
−
2
(
t
ϕ
(
x
)
T
+
∑
n
=
1
N
t
n
ϕ
(
x
n
)
T
)
m
5
=
t
2
+
∑
n
=
1
N
t
n
2
-m_1w^2-m_2(m_3w^2+m_4w+m_5)\\ m_1 = \frac{\alpha}{2}, m_2=\frac{\beta}{2}\\ m_3 = \phi(x)^T\phi(x) + \sum_{n=1}^N\phi(x_n)^T\phi(x_n)\\ m_4 =-2(t\phi(x)^T + \sum_{n=1}^Nt_n\phi(x_n)^T)\\ m_5 = t^2 + \sum_{n=1}^Nt_n^2
−m1w2−m2(m3w2+m4w+m5)m1=2α,m2=2βm3=ϕ(x)Tϕ(x)+n=1∑Nϕ(xn)Tϕ(xn)m4=−2(tϕ(x)T+n=1∑Ntnϕ(xn)T)m5=t2+n=1∑Ntn2
从而，
−
(
m
1
+
m
2
m
3
)
w
2
−
m
2
m
4
w
−
m
2
m
5
=
−
(
m
1
+
m
2
m
3
)
(
w
2
+
m
2
m
4
m
1
+
m
2
m
3
w
+
m
2
m
5
m
1
+
m
2
m
3
)
=
−
(
m
1
+
m
2
m
3
)
[
(
w
+
m
2
m
4
2
(
m
1
+
m
2
m
3
)
)
2
+
m
2
m
5
m
1
+
m
2
m
3
−
m
2
2
m
4
2
4
(
m
1
+
m
2
m
3
)
2
]
=
−
(
m
1
+
m
2
m
3
)
[
(
w
+
m
2
m
4
2
(
m
1
+
m
2
m
3
)
)
2
+
4
(
m
1
+
m
2
m
3
)
m
2
m
5
−
m
2
2
m
4
2
4
(
m
1
+
m
2
m
3
)
2
]
=
−
(
m
1
+
m
2
m
3
)
(
w
+
m
2
m
4
2
(
m
1
+
m
2
m
3
)
)
2
+
4
(
m
1
+
m
2
m
3
)
m
2
m
5
−
m
2
2
m
4
2
4
(
m
1
+
m
2
m
3
)
-(m_1+m_2m_3)w^2 - m_2m_4w - m_2m_5=-(m_1+m_2m_3)(w^2+\frac{m_2m_4}{m_1+m_2m_3}w + \frac{m_2m_5}{m_1+m_2m_3})\\ =-(m_1+m_2m_3)[(w+\frac{m_2m_4}{2(m_1+m_2m_3)})^2 + \frac{m_2m_5}{m_1+m_2m_3} - \frac{m_2^2m_4^2}{4(m_1+m_2m_3)^2}]\\ =-(m_1+m_2m_3)[(w+\frac{m_2m_4}{2(m_1+m_2m_3)})^2 + \frac{4(m_1+m_2m_3)m_2m_5 -m_2^2m_4^2}{4(m_1+m_2m_3)^2}]\\ =-(m_1+m_2m_3)(w+\frac{m_2m_4}{2(m_1+m_2m_3)})^2 + \frac{4(m_1+m_2m_3)m_2m_5 -m_2^2m_4^2}{4(m_1+m_2m_3)}
−(m1+m2m3)w2−m2m4w−m2m5=−(m1+m2m3)(w2+m1+m2m3m2m4w+m1+m2m3m2m5)=−(m1+m2m3)[(w+2(m1+m2m3)m2m4)2+m1+m2m3m2m5−4(m1+m2m3)2m22m42]=−(m1+m2m3)[(w+2(m1+m2m3)m2m4)2+4(m1+m2m3)24(m1+m2m3)m2m5−m22m42]=−(m1+m2m3)(w+2(m1+m2m3)m2m4)2+4(m1+m2m3)4(m1+m2m3)m2m5−m22m42
存在一个常数项，即
4
(
m
1
+
m
2
m
3
)
m
2
m
5
−
m
2
2
m
4
2
4
(
m
1
+
m
2
m
3
)
=
β
(
α
+
β
m
3
)
m
5
−
4
−
1
β
2
m
4
2
2
(
α
+
β
m
3
)
\frac{4(m_1+m_2m_3)m_2m_5 -m_2^2m_4^2}{4(m_1+m_2m_3)} = \frac{\beta(\alpha + \beta m_3)m_5 - 4^{-1}\beta^2m_4^2}{2(\alpha + \beta m_3)}
4(m1+m2m3)4(m1+m2m3)m2m5−m22m42=2(α+βm3)β(α+βm3)m5−4−1β2m42
从而,
∫
exp
(
−
α
2
w
T
w
−
β
2
(
(
t
−
ϕ
(
x
)
T
w
)
2
+
∑
n
=
1
N
(
t
n
−
ϕ
(
x
n
)
T
w
)
2
)
)
d
w
=
π
k
exp
(
β
(
α
+
β
m
3
)
m
5
−
4
−
1
β
2
m
4
2
2
(
α
+
β
m
3
)
)
\int \exp\Big(-\frac{\alpha}{2}w^Tw -\frac{\beta}{2}((t-\phi(x)^Tw)^2 + \sum_{n=1}^N(t_n-\phi(x_n)^Tw)^2) \Big ) dw = \sqrt{\frac{\pi}{k}}\exp( \frac{\beta(\alpha + \beta m_3)m_5 - 4^{-1}\beta^2m_4^2}{2(\alpha + \beta m_3)})
∫exp(−2αwTw−2β((t−ϕ(x)Tw)2+n=1∑N(tn−ϕ(xn)Tw)2))dw=kπexp(2(α+βm3)β(α+βm3)m5−4−1β2m42)
代入到原式得
p
(
t
∣
x
,
X
,
T
)
=
(
β
2
π
)
N
+
1
2
(
α
2
π
)
M
+
1
2
π
k
exp
(
β
(
α
+
β
m
3
)
m
5
−
4
−
1
β
2
m
4
2
2
(
α
+
β
m
3
)
)
p(t|x,X,T)= (\frac{\beta}{2\pi })^{\frac{N+1}{2}} (\frac{\alpha}{2\pi})^{ \frac{M+1}{2}}\sqrt{\frac{\pi}{k}}\exp( \frac{\beta(\alpha + \beta m_3)m_5 - 4^{-1}\beta^2m_4^2}{2(\alpha + \beta m_3)})\\
p(t∣x,X,T)=(2πβ)2N+1(2πα)2M+1kπexp(2(α+βm3)β(α+βm3)m5−4−1β2m42)
对于指数部分的系数：
(
β
2
π
)
N
+
1
2
(
α
2
π
)
M
+
1
2
π
k
=
(
(
β
N
+
1
α
M
+
1
(
2
π
)
N
+
M
+
2
(
α
2
+
β
2
(
ϕ
(
x
)
T
ϕ
(
x
)
+
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
)
)
)
)
1
/
2
(\frac{\beta}{2\pi })^{\frac{N+1}{2}} (\frac{\alpha}{2\pi})^{ \frac{M+1}{2}}\sqrt{\frac{\pi}{k}}=\Big((\frac{\beta^{N+1}\alpha^{M+1}}{(2\pi)^{N+M+2}} (\frac{\alpha}{2}+\frac{\beta}{2}(\phi(x)^T\phi(x) + \sum_{n=1}^{N}\phi(x_n)^T\phi(x_n))))\Big)^{1/2}
(2πβ)2N+1(2πα)2M+1kπ=(((2π)N+M+2βN+1αM+1(2α+2β(ϕ(x)Tϕ(x)+n=1∑Nϕ(xn)Tϕ(xn)))))1/2
而指数部分为:
β
(
α
+
β
m
3
)
m
5
−
4
−
1
β
2
m
4
2
2
(
α
+
β
m
3
)
=
β
(
α
+
β
m
3
)
(
t
2
+
t
s
u
m
)
−
1
4
β
2
(
−
2
(
t
ϕ
(
x
)
T
+
∑
n
=
1
N
t
n
ϕ
(
x
n
)
T
)
)
2
2
(
α
+
β
m
3
)
=
β
(
α
+
β
m
3
)
(
t
2
+
t
s
u
m
)
−
β
2
(
t
ϕ
(
x
)
T
+
q
)
2
2
(
α
+
β
m
3
)
=
[
α
β
+
β
2
m
3
−
β
2
ϕ
(
x
)
T
ϕ
(
x
)
]
⋅
t
2
−
2
β
2
ϕ
(
x
)
T
q
T
t
+
(
α
β
+
β
2
m
3
)
t
s
u
m
−
β
2
q
q
T
2
(
α
+
β
m
3
)
=
[
α
β
+
β
2
v
]
⋅
t
2
−
2
β
2
ϕ
(
x
)
T
q
T
t
+
(
α
β
+
β
2
m
3
)
t
s
u
m
−
β
2
q
q
T
2
(
α
+
β
m
3
)
=
(
α
β
+
β
2
v
)
t
2
−
2
β
2
ϕ
(
x
)
T
q
T
α
β
+
β
2
v
t
+
(
α
β
+
β
2
m
3
)
t
s
u
m
−
β
2
q
q
T
α
β
+
β
2
v
2
(
α
+
β
m
3
)
=
(
α
β
+
β
2
v
)
(
t
−
β
2
ϕ
(
x
)
T
q
T
α
β
+
β
2
v
)
2
−
(
β
2
ϕ
(
x
)
T
q
T
)
2
(
α
β
+
β
2
v
)
2
+
(
α
β
+
β
2
m
3
)
t
s
u
m
−
β
2
q
q
T
α
β
+
β
2
v
2
(
α
+
β
m
3
)
t
s
u
m
=
∑
n
=
1
N
t
n
2
q
=
∑
n
=
1
N
t
n
ϕ
(
x
n
)
T
v
=
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
\frac{\beta(\alpha + \beta m_3)m_5 - 4^{-1}\beta^2m_4^2}{2(\alpha + \beta m_3)} = \frac{\beta(\alpha + \beta m_3)(t^2 + t_{sum}) - \frac{1}{4}\beta^2(-2(t\phi(x)^T + \sum_{n=1}^Nt_n\phi(x_n)^T))^2}{2(\alpha +\beta m_3)}\\ =\frac{\beta(\alpha + \beta m_3)(t^2 + t_{sum}) - \beta^2(t\phi(x)^T + q)^2}{2(\alpha +\beta m_3)} =\frac{[\alpha \beta + \beta^2 m_3 - \beta^2\phi(x)^T\phi(x)]\cdot t^2 -2\beta^2 \phi(x)^Tq^T t + (\alpha\beta+\beta^2 m_3)t_{sum}-\beta^2qq^T}{2(\alpha +\beta m_3)} \\ =\frac{[\alpha \beta + \beta^2 v]\cdot t^2 -2\beta^2 \phi(x)^Tq^T t + (\alpha\beta+\beta^2 m_3)t_{sum}-\beta^2qq^T}{2(\alpha +\beta m_3)}\\ =(\alpha \beta + \beta^2 v)\frac{t^2 -2\frac{\beta^2 \phi(x)^Tq^T}{\alpha \beta + \beta^2 v} t + \frac{(\alpha\beta+\beta^2 m_3)t_{sum}-\beta^2qq^T}{\alpha \beta + \beta^2 v}}{2(\alpha +\beta m_3)} =(\alpha \beta + \beta^2 v)\frac{ (t -\frac{\beta^2 \phi(x)^Tq^T}{\alpha \beta + \beta^2 v})^2 -\frac{(\beta^2 \phi(x)^Tq^T)^2}{(\alpha \beta + \beta^2 v)^2} + \frac{(\alpha\beta+\beta^2 m_3)t_{sum}-\beta^2qq^T}{\alpha \beta + \beta^2 v}}{2(\alpha +\beta m_3)} \\ t_{sum} = \sum_{n=1}^Nt_n^2\\ q = \sum_{n=1}^Nt_n\phi(x_n)^T\\ v = \sum_{n=1}^N\phi(x_n)^T\phi(x_n)
2(α+βm3)β(α+βm3)m5−4−1β2m42=2(α+βm3)β(α+βm3)(t2+tsum)−41β2(−2(tϕ(x)T+∑n=1Ntnϕ(xn)T))2=2(α+βm3)β(α+βm3)(t2+tsum)−β2(tϕ(x)T+q)2=2(α+βm3)[αβ+β2m3−β2ϕ(x)Tϕ(x)]⋅t2−2β2ϕ(x)TqTt+(αβ+β2m3)tsum−β2qqT=2(α+βm3)[αβ+β2v]⋅t2−2β2ϕ(x)TqTt+(αβ+β2m3)tsum−β2qqT=(αβ+β2v)2(α+βm3)t2−2αβ+β2vβ2ϕ(x)TqTt+αβ+β2v(αβ+β2m3)tsum−β2qqT=(αβ+β2v)2(α+βm3)(t−αβ+β2vβ2ϕ(x)TqT)2−(αβ+β2v)2(β2ϕ(x)TqT)2+αβ+β2v(αβ+β2m3)tsum−β2qqTtsum=n=1∑Ntn2q=n=1∑Ntnϕ(xn)Tv=n=1∑Nϕ(xn)Tϕ(xn)
故，我们可以从上式中，直接推出均值
m
(
x
)
=
β
2
ϕ
(
x
)
T
q
T
α
β
+
β
2
v
=
β
ϕ
(
x
)
T
q
T
α
+
β
v
=
(
α
+
β
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
)
−
1
(
β
ϕ
(
x
)
T
∑
n
=
1
N
ϕ
(
x
n
)
t
n
)
=
β
ϕ
(
x
)
T
S
∑
n
=
1
N
ϕ
(
x
n
)
t
n
S
−
1
=
α
+
β
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
m(x)=\frac{\beta^2 \phi(x)^Tq^T}{\alpha \beta + \beta^2 v} = \frac{\beta \phi(x)^Tq^T}{\alpha + \beta v} = (\alpha+\beta \sum_{n=1}^N\phi(x_n)^T\phi(x_n))^{-1}(\beta\phi(x)^T\sum_{n=1}^{N}\phi(x_n)t_n) = \beta \phi(x)^TS\sum_{n=1}^{N}\phi(x_n)t_n\\ S^{-1} = \alpha+\beta \sum_{n=1}^N\phi(x_n)^T\phi(x_n)
m(x)=αβ+β2vβ2ϕ(x)TqT=α+βvβϕ(x)TqT=(α+βn=1∑Nϕ(xn)Tϕ(xn))−1(βϕ(x)Tn=1∑Nϕ(xn)tn)=βϕ(x)TSn=1∑Nϕ(xn)tnS−1=α+βn=1∑Nϕ(xn)Tϕ(xn)
倘若上述配方成功，方差为
s
2
(
x
)
=
α
+
β
m
3
α
β
+
β
2
v
=
α
+
β
(
ϕ
(
x
)
T
ϕ
(
x
)
+
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
)
α
β
+
β
2
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
=
α
+
β
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
+
β
ϕ
(
x
)
T
ϕ
(
x
)
α
β
+
β
2
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
=
1
β
+
ϕ
(
x
)
T
ϕ
(
x
)
α
+
β
∑
n
=
1
N
ϕ
(
x
n
)
T
ϕ
(
x
n
)
=
β
−
1
+
S
ϕ
(
x
)
T
ϕ
(
x
)
s^2(x)=\frac{\alpha + \beta m_3}{\alpha \beta + \beta^2 v} = \frac{\alpha+\beta(\phi(x)^T\phi(x) + \sum_{n=1}^N\phi(x_n)^T\phi(x_n))}{\alpha\beta + \beta^2\sum_{n=1}^N\phi(x_n)^T\phi(x_n) } =\frac{\alpha+\beta\sum_{n=1}^N\phi(x_n)^T\phi(x_n) + \beta \phi(x)^T\phi(x)}{\alpha\beta + \beta^2\sum_{n=1}^N\phi(x_n)^T\phi(x_n) }\\ =\frac{1}{\beta} + \frac{\phi(x)^T\phi(x)}{\alpha + \beta \sum_{n=1}^N\phi(x_n)^T\phi(x_n) }\\ = \beta^{-1} + S\phi(x)^T\phi(x)
s2(x)=αβ+β2vα+βm3=αβ+β2∑n=1Nϕ(xn)Tϕ(xn)α+β(ϕ(x)Tϕ(x)+∑n=1Nϕ(xn)Tϕ(xn))=αβ+β2∑n=1Nϕ(xn)Tϕ(xn)α+β∑n=1Nϕ(xn)Tϕ(xn)+βϕ(x)Tϕ(x)=β1+α+β∑n=1Nϕ(xn)Tϕ(xn)ϕ(x)Tϕ(x)=β−1+Sϕ(x)Tϕ(x)
1039
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
