Consider the polynomial kernel of degree 2 defined by,
where
and
.
Thereby, the kernel function can be written as,
.
Now, let us try to come up with a feature map such that the kernel function can be written as .
Consider the following feature map, . Basically, this feature map is mapping the points in to points in . Also, notice that, which is essentially our kernel function.
This means that our kernel function is actually computing the inner/dot product of points in . That is, it is implicitly mapping our points from to .
NOTE:我们要做的是将线性不可分的sample映射到高维空间(在这个空间中线性可分),然后在这个高维空间中衡量两个sample之间的相似性,即做点积。在上述的多项式核函数例子中,阐述了通过核函数同样可以达到我们的目的,而且省略了显示映射的环节。
Now, coming to RBF.
Let us consider the RBF kernel again for points in . Then, the kernel can be written as
(assuming gamma = 1). Using the taylor series you can write this as,
Thereby, the kernel function can be written as,
.
Now, let us try to come up with a feature map such that the kernel function can be written as .
Consider the following feature map, . Basically, this feature map is mapping the points in to points in . Also, notice that, which is essentially our kernel function.
This means that our kernel function is actually computing the inner/dot product of points in . That is, it is implicitly mapping our points from to .
NOTE:我们要做的是将线性不可分的sample映射到高维空间(在这个空间中线性可分),然后在这个高维空间中衡量两个sample之间的相似性,即做点积。在上述的多项式核函数例子中,阐述了通过核函数同样可以达到我们的目的,而且省略了显示映射的环节。
Now, coming to RBF.
Let us consider the RBF kernel again for points in . Then, the kernel can be written as
(assuming gamma = 1). Using the taylor series you can write this as,
Now, if we were to come up with a feature map just like we did for the polynomial kernel, you would realize that the feature map would map every point in our to an infinite vector. Thus, RBF implicitly maps every point to an infinite dimensional space.
NOTE:上面是RBF核函数(也可以理解为高斯核函数),在上面的转换过程中核函数最终通过傅里叶展开为无穷项,而每项都是一个多项式核函数,从这里可以看出高斯核函数映射之后为什么是无穷维的。
另外需要注意一点就是,无穷维的向量并不意味着对两个无穷维的向量相似性的度量。点积是衡量两个无穷维向量相似值最终收敛到的值!