kernel PCA

最新推荐文章于 2022-10-30 21:37:58 发布
最新推荐文章于 2022-10-30 21:37:58 发布
阅读量960 收藏 2
点赞数
文章标签： PCA 数据

PCA不进行分类的动作，而只做做数据预处理，将样本变换到一个容易分类（向最大化方差的方向，principal component axes，投影）的更低维的新的特征空间中。Kernel PCA比PCA多了一步，也即先升维（RBF包括多项式核均是升高到无穷维）再进行投影的动作，因为有些非线性可分的数据集只有在升维的视角下才线性可分。

PCA

  • 均值化的数据： 
    ∑ixi=0 
<code class="hljs python has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># python </span>
<span class="hljs-prompt" style="color: rgb(0, 102, 102); box-sizing: border-box;">>>> </span>X-np.mean(X, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)
                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 一个二维矩阵减去一维向量？对，</span>
                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 这里用到的技术是numpy中broadcasting（广播机制）</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

  • 样本协方差矩阵（sample-covariance matrix  C ） 

    C=1N∑ixixTi=1NXXT

    其中， X  的每一列表示一个样本（特征向量）

  • 特征分解 

    C=UΛUT=∑αλαuαuTα

  • projection or transform

yi=UTkxi

1N∑iyiyTi=1N∑iUTkxixTiUk=UTk(1N∑ixixTi)Uk=UTkCUk=UTkUΛUUTk=Λk

the projected data are de-correlated in this new axis.

  • 重构 
    yi=UTkxi⇓Ukyi=UkUTkxi=xi 
<code class="hljs python has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> pandas <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> pd
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> numpy <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> np

df = pd.read_csv(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data'</span>, header=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">None</span>)
X, y = df.values[:, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:], df.values[:, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># step 1</span>
X -= X.mean(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## step 2</span>
N = X.shape[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]
C = X.T.dot(X)/N

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## step 3</span>
Lambda, Q = np.linalg.eig(C)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## step 4</span>
k = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
eigen_pairs = [(Lambda[i], Q[:, i]) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(len(Lambda))]
eigen_pairs = sorted(eigen_pairs, reverse=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, key=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">lambda</span> k: k[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>])
W = np.column_stack((eigen_pairs[i][<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(k=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)))

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## step 5</span>
X_pca = X.dot(W)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li></ul>

Kernel PCA

以 Radial Basis Function（RBF） kernel PCA（不同的核，体现在代码上仅仅是一处细微的差别）为例进行说明：

  • 计算核矩阵（相似度矩阵）  K

Kij=k(x(i),x(j))=exp(−γ∥x(i)−x(j)∥2)

这是最为常见的RBF（Rational Basis Function）核函数，也有多项式核： 

Kij=k(x(i),x(j))=(<x(i),x(j)>+θ)p

和 sigmoid型（hyperbolic tangent）核： 
Kij=k(x(i),x(j))=tanh(η<x(i),x(j)>+θ)

其矩阵形式也即：

K=⎡⎣⎢⎢⎢⎢⎢⎢k(x(1),x(1)),k(x(2),x(1)),⋯,k(x(n),x(1)),k(x(1),x(2)),k(x(2),x(2)),⋯,k(x(n),x(2)),⋯,⋯,⋱,⋯,k(x(1),x(n))k(x(2),x(n))⋯k(x(n),x(n))⎤⎦⎥⎥⎥⎥⎥⎥n×n

  • center the  K

K′=K−1nK−K1n+1nK1n

其中  1n 是  n×n  的元素值全为 1n 矩阵。

  • 对  K′ 进行特征值分解，获得对应于前  k 个特征值的特征向量。与标准PCA算法不同的是，这里获得特征向量不再是 principal component axes，而已经是全部样本在这些轴上的投影了，也即是我们所需的进行降维后的数据了。

这里为了验证核机制的性能，我们在如下的数据集上进行测试：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">from sklearn<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.datasets</span> import make_moons

<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">X</span>, <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">y</span> = make_moons(n_samples=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">200</span>, random_state=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>)
plt<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scatter</span>(<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">X</span>[<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">y</span>==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">X</span>[<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">y</span>==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], colors=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'r'</span>, marker=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'^'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)
plt<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scatter</span>(<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">X</span>[<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">y</span>==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">X</span>[<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">y</span>==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], colors=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'b'</span>, marker=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'o'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>


  

<code class="hljs python has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> scipy.spatial.distance <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> pdist, squareform

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">rbf_kpca</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(X, gamma, k)</span>:</span>
    sq_dist = pdist(X, metric=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'sqeuclidean'</span>)
                            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># N = X.shape[0]    </span>
                            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># sq_dist.shape = N*(N-1)/2</span>
    mat_sq_dist = squareform(sq_dist)
                            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mat_sq_dist.shape = (N, N)</span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># step 1</span>
    K = np.exp(-gamma*mat_sq_dist)

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># step 2</span>
    N = X.shape[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]
    one_N = np.ones((N, N))/N
    K = K - one_N.dot(K) - K.dot(one_N) + one_N.dot(K).dot(one_N)

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># step 3</span>
    Lambda, Q = np.linalg.eig(K)
    eigen_pairs = [(Lambda[i], Q[:, i]) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(len(Lambda))]
    eigen_pairs = sorted(eigen_pairs, reverse=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, key=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">lambda</span> k: k[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>])
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> np.column_stack((eigen_pairs[i][<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(k)))</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul>

调用：

<code class="hljs fix has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">X_kpca </span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;"> rbf_kpca(X, gamma=15, k=2)</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

一个非线性可分的数据集

<code class="hljs mel has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">from sklearn.datasets import make_circles
X, y = make_circles(n_samples=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000</span>, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">noise</span>=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>, factor=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>, random_state=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>)
plt.scatter(X[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], X[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'r'</span>, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">marker</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'^'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)
plt.scatter(X[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], X[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'b'</span>, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">marker</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'o'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)
plt.show()</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>


 

在这样的一个非线性可分的数据集，显然如何找到最大方差的方向都不可能使最后的数据集线性可分。接下来，我们探讨KPCA的核机制是如何工作的，使线性不可分的数据集变得线性可分，

应用kernel trick：

<code class="hljs fix has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">X_kpca </span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;"> rbf_kpca(X, gamma=15, k=2)</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

可视化：

<code class="hljs rsl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">fig, ax = plt.subplots(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, figsize=(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>))

ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>].scatter(X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'r'</span>, marker=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'^'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>].scatter(X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>], <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'b'</span>, marker=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'o'</span>, alpha=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>)

label_count = np.bincount(y)
                                <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 统计各类别出现的次数</span>
                                <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># label_count[0] = 500</span>
                                <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># label_count[1] = 500</span>
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>].scatter(X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], np.zeros(label_count[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]), <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'r'</span>)
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>].scatter(X_kpca[y==<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], np.zeros(label_count[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>]), <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">color</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'b'</span>)
                                <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># y轴置零</span>
                                <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 投影到x轴</span>
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>].set_ylim([-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>])
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>].set_xlabel(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'PC1'</span>)
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>].set_ylabel(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'PC2'</span>)
ax[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>].set_xlabel(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'PC1'</span>)

plt.show()</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul>


 

PCA 与 KPCA 的另一点重要不同

不同在于对新数据、新样本（当然还是同一类型数据，比如测试样本之于训练样本）的转换方式不同；

标准PCA算法，转换矩阵（transformation matrix  Wd×k ）对应于训练样本的协方差矩阵的特征向量，使用  W  可继续作用于新的数据集， x′1×d⋅Wd×k 。而KPCA， K  的特征向量（ Ka=λα  中的  α ）即为变换后的样本。 
对于新的样本，我们想要计算： 

ϕ(x′)Tv

而特征向量  v=∑ia(i)ϕ(x(i)) ，则： 

ϕ(x′)Tv=∑ia(i)ϕ(x′)Tϕ(x(i))=∑ia(i)k(x′,x(i)) 

<code class="hljs python has-numbering" style="color: inherit; display: block; padding: 0px; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background-image: initial; background-attachment: initial; background-color: transparent; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> scipy.spatial.distance <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> sqdist, squareform

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">rbf_kpca</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(X, gamma, k)</span>:</span>
    sq_dists = sqdist(X, metric=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'sqeuclidean'</span>)
    mat_sq_dists = squareform(sq_dists)
    K = np.exp(-gamma*mat_sq_dists)
    N = X.shape[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]
    one_N = np.ones((N, N))/N
    K = K-one_N.dot(K)-K.dot(one_N)+one_N.dot(K).dot(one_N)
    Lambda, Q = np.linalg.eigh(K)
    alphas = np.column_stack((Q[:, -i]<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>+k)))
    lambdas = [Lambda[-i] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> range(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, k+<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)]
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> alphas, lambdas

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">proj_new</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(X_new, X, gamma, alphas, lambdas)</span>:</span>
    k = np.exp(-gamma*np.sum((X-X_new)**<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>))
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> k.dot(alphas/lambdas)
                                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># alphas/lambdas，归一化后的alphas</span>

X, y = make_moons(n_samples=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>, random_state=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>)
                                <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 设置random_state，为了可重复性</span>
alphas, lambdas = rbf_kpca(X, gamma=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>, k=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
X_new = X[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span>]
                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 以当前样本的某一样本作为新的样本进行测试</span>
X_proj = proj_new(X_new, X, gamma=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>, alphas, lambdas)
print(alphas[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span>])
print(X_proj)
                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># [-0.07877284]</span>
                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># [-0.07877284]</span></code><code class="hljs python has-numbering" style="color: inherit; display: block; padding: 0px; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background-image: initial; background-attachment: initial; background-color: transparent; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">
</span></code><code class="hljs python has-numbering" style="color: inherit; display: block; padding: 0px; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background-image: initial; background-attachment: initial; background-color: transparent; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">
</span></code><code class="hljs python has-numbering" style="display: block; padding: 0px; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background-image: initial; background-attachment: initial; background-color: transparent; background-size: initial; background-origin: initial; background-clip: initial; background-position: initial; background-repeat: initial;"><span style="color:#880000;">本文转自：<a target=_blank href="http://http://blog.csdn.net/lanchunhui/article/details/50492482">http://blog.csdn.net/lanchunhui/article/details/50492482</a></span></code><ul class="pre-numbering" style="color: rgb(51, 51, 51); box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li></ul><div class="save_code tracking-ad" data-mod="popu_249" style="color: rgb(255, 255, 255); box-sizing: border-box; position: absolute; height: 60px; right: 30px; top: 5px; cursor: pointer; z-index: 2;"><a target=_blank style="color: rgb(202, 0, 0); box-sizing: border-box;"><img src="http://static.blog.csdn.net/images/save_snippets.png" style="border: none; box-sizing: border-box; max-width: 100%;" alt="" /></a></div><ul class="pre-numbering" style="color: rgb(51, 51, 51); box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li></ul>
鱼的一滴泪1990
关注
  • 0
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
SVM+rbf kernel+主成分PCA分析之python实现
ting_7的博客
05-01 2108
数据来源：文献搜索 生境：草地，尾矿，农田，森林 生境特征：38个（pH,EC…) ——多维 思路：先分析，（本来还有一步预测的，因为这里特征太多了，就省了），然后降维作图 #多特征SVM分析 #载入分析包 import numpy as np import pandas as pd from sklearn import preprocessing from sklearn import s...
KPCA降维——python
zqzq19950725的博客
03-22 5495
KPCA，中文名称”核主成分分析“，是对PCA算法的非线性扩展。PCA是线性的，其对于非线性数据往往显得无能为力（虽然这二者的主要目的是降维，而不是分类，但也可以用于分类），其中很大一部分原因是，KPCA能够挖掘到数据集中蕴含的非线性信息。 一、KPCA较PCA存在的创新点： 1. 为了更好地处理非线性数据，引入非线性映射函数，将原空间中的数据映射到高维空间，注意，这个是隐性的，我们不知道，也...
Kernel-PCA.rar_Kernel_kernel matlab_kernel pca
09-22
calculation algorithm for principle component analysis based kernel of any image database.
kernelPCA.rar_kernel pca_kernel pca matlab_kernelpca
09-19
kernel PCA MATLAB code
kernel_pca.rar_kernel pca_kernel pca matlab_kernel_pca_pca kern
09-21
Kernel principal component analysis (kernel PCA) [1] is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are done in a reproducing kernel Hilbert space with a non-linear mapping.
Kernel PCA.ipynb
05-20
Kernel PCA¶ This example shows that Kernel PCA is able to find a projection of the data that makes data linearly separable. 使用anaconda notebook打开
Kernel PCA and Pre-Image Reconstruction：标准PCA、高斯核PCA、多项式核PCA、前像重构-matlab开发
06-01
在这个包中，我们实现了标准 PCA、核 PCA 和高斯核 PCA 的前像重建。 我们还提供了三个演示：（1）两个同心球嵌入； （2）PCA/kPCA人脸分类； (3) 使用 kPCA 的主动形状​​模型。 标准 PCA 并未针对非常高维的数据进行优化。 但是我们的内核 PCA 实现非常高效，并且已经在许多研究项目中使用。
python ‘FuzzyDecisionTreeClassifier‘ object has no attribute ‘_validate_data‘
qq_42878458的博客
06-13 1374
python 'FuzzyDecisionTreeClassifier' object has no attribute '_validate_data' 最近在写关于模糊树的论文，在GitHub找到了相关的代码，但运行时出现报错 # -*- coding: utf-8 -*- """ ========================================================= Comparison of crisp and fuzzy classifiers on iris data
python AttributeError: '' object has no attribute '' 的错误解决方法
tmoonlee的博客
06-16 12万+
for res in results: # 在这里不要把res写成re了，和引入名字一样会报错 我的这个错误就是因为引入的是 import re后面这个参数名字不能写成re
pca降维sklearn中pca库实现
WxyangID的博客
04-01 2026
数据的特征P，如果要是比样本容量还要多的话，必须进行降维处理，否则就是curse of dimensionality维数灾难# -*-encoding: utf-8 -*- """ @version: 3.6 @time: 2018/4/1 14:46 @author: SunnyYang @title: @file: pcaTest.py """ import numpy as np impor...
【机器学习】数据降维—核主成分分析(Kernel PCA)
探索世界，改变世界
02-03 1万+
本文代码推荐使用Jupyter notebook跑，这样得到的结果更为直观。 KPCA：将非线性可分的数据转换到一个适合对齐进行线性分类的新的低维子空间上。   非线性函数： Φ为一个函数，能够对原始的特征进行非线性组合，将原始的d维数据集映射到更高的k维特征空间。 利用核PCA可以通过非线性映射将数据转换到一个高维空间中，在高维空间中使用PCA将其映射到另一个低维空间中
AttributeError：object has no attribute 报错及解决
热门推荐
MichaelAn的博客
12-22 17万+
报错情况：在前端测试中，界面发送一个 PUT 请求，界面发生上面的报错，请求无法响应，服务器状态码是500。 错误分析：语义上是“对象没有一个XXX属性”。 查阅大部分资料，大部分说 Python 的问题。这个项目前端使用 React， 后端使用 Python 的 djongo 框架。 询问后端的同事，主要的原因是前端传递的参数的数据类型不正确。后端需要一个 string 的 “tr...
数据填充报错
m0_56975146的博客
11-04 718
报错信息 报错：Input contains NaN, infinity or a value too large for dtype(‘float64’) 源代码： from sklearn.impute import SimpleImputer data=SimpleImputer(missing_values='nan',strategy='mean') dataMode=data.fit_transform(df) 解决： 1、0.20版中的新功能：SimpleImputer替换之前sklearn.
sklearn的KernelPCA
我是一只小嵩鼠
10-30 1736
KPCA
Python报错AttributeError: ‘module’ object has no attribute’xxx’解决方法
唯有努力无限，才可突破极限
04-20 6万+
# doctest.py class Dict(dict): ''' Simple dict but also support access as x.y style. &gt;&gt;&gt; d1 = Dict() &gt;&gt;&gt; d1['x'] = 100 &gt;&gt;&gt; d1.x 100 &gt;&gt;&gt;...
AttributeError: ‘xxx‘ object has no attribute ‘__bases__‘ 问题解决
weixin_40923064的博客
07-13 2631
问题描述： 该类没有__bases__属性，无法查看父类信息
AttributeError: 'NoneType' object has no attribute 'sqrt'
微电子学与固体电子学-俞驰
04-28 8079
Traceback (most recent call last):  File "C:\Users\appleyuchi\Desktop\todolist\LDA\datagenerate.py", line 30, in &lt;module&gt;    plt.scatter(X_new[:, 0], X_new[:, 1],marker='o',c=y)  File "D:\softwa...
成功解决AttributeError: 'BasicLSTMCell' object has no attribute '_kernel'+python下划线用法的几种常见用法理解
近期请国内外头部出版社可尽快私信博主！——心比天高，仗剑走天涯，保持热爱，奔赴向梦想！低调，谦虚，自律，反思，成长，还算是比较正能量的博主，公益免费传播……内心特别想在AI界做出一些可以推进历史进程影响力的东西(兴趣使然，有点小情怀，也有点使命感呀)…
10-13 1万+
成功解决AttributeError: 'BasicLSTMCell' object has no attribute '_kernel'+python下划线用法的几种常见用法理解 目录 解决问题 解决思路 解决方法 解决问题 AttributeError: 'BasicLSTMCell' object has no attribute '_kernel' ...
Python脚本报错AttributeError: ‘module’ object has no attribute’xxx’解决方法
梦忆故城的博客
09-25 7417
在编写Python脚本过程中遇到一个问题比较奇怪：Python脚本完全正常没问题，但执行总错"AttributeError: 'module' object has no attribute 'xxx'"。这其实是.pyc文件存在问题。 查看import库的源文件，发现源文件存在且没有错误，同时存在源文件的.pyc文件 问题解决方法： 1. 命名py脚本时
详细介绍一下kernel PCA
最新发布
05-12
Kernel PCA（Kernel Principal Component Analysis）是一种非线性的数据降维方法，它是对传统PCA的一种扩展。传统的PCA是一种线性降维方法，它通过找到原始数据中方差最大的方向，将数据投影到这个方向上，从而实现降维。而Kernel PCA则是通过将数据映射到高维特征空间中，然后在这个特征空间中进行PCA分析，从而实现非线性降维。 Kernel PCA的核心思想是使用核技巧将原始数据映射到高维特征空间中，使得原始数据在新的特征空间中线性可分。具体地，Kernel PCA首先通过某种核函数（如高斯核函数）将原始数据映射到高维特征空间中，然后在新的特征空间中进行PCA分析，找到方差最大的方向，从而得到降维后的数据。由于使用了核技巧，Kernel PCA能够处理非线性的数据结构，因此在许多实际应用中具有很高的实用性。 需要注意的是，Kernel PCA的计算复杂度较高，因此在处理大规模数据时需要考虑计算效率和内存消耗等问题。另外，由于Kernel PCA是一种无监督学习方法，因此在应用时需要注意选择合适的核函数和参数以及评估降维效果。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值