26 篇文章 5 订阅
27 篇文章 0 订阅
2 篇文章 0 订阅

一、LSSVM

1、LSSVM用于回归

y = W ⋅ φ ( x ) + b y = W \cdot \varphi \left( x \right) + b

LSSVR回归模型的优化思想是使离回归平面距离最大的样本与回归平面之间的距离最小。优化问题为：

min ⁡ W , b 1 ∥ W ∥ s . t . y i = W φ ( x i ) + b \begin{array}{l} \mathop {\min }\limits_{W,b} \frac{1}{{\left\| W \right\|}}\\ s.t.{y_i} = W\varphi \left( {{x_i}} \right) + b \end{array}

max ⁡ W , b 1 2 ∥ W ∥ 2 + λ 2 ∑ i = 1 m e i 2 s . t . y i = W φ ( x i ) + b + e i \begin{array}{l} \mathop {\max }\limits_{W,b} \frac{1}{2}{\left\| W \right\|^2} + \frac{\lambda }{2}\sum\limits_{i = 1}^m {{e_i}^2} \\ s.t.{y_i} = W\varphi \left( {{x_i}} \right) + b + {e_i} \end{array}

LSSVM回归模型的输出为：
f ( x ) = W ⋅ φ ( x ) + b = ∑ i = 1 m α i k ( x i , x ) + b f\left( x \right) = W \cdot \varphi \left( x \right) + b = \sum\limits_{i = 1}^m {{\alpha _i}k\left( {{x_i},x} \right) + b}

二、WLSSVM的数学原理

WLSSVM的执行步骤（参考资料【2】）：

v i = { 1 i f ∣ e k / s ^ ∣ ≤ c 1 c 2 − ∣ e k / s ^ ∣ c 2 − c 1 i f c 1 ≤ ∣ e k / s ^ ∣ ≤ c 2 10 − 4 o t h e r w i s e {v _i} = \left\{ {\begin{matrix} {\begin{matrix} 1&amp;{\begin{matrix} {}&amp;{} \end{matrix}if\begin{matrix} {}&amp;{\left| {{e_k}/\hat s} \right| \le {c_1}} \end{matrix}} \end{matrix}}\\ {\begin{matrix} {\frac{{{c_2} - \left| {{e_k}/\hat s} \right|}}{{{c_2} - {c_1}}}}&amp;{if\begin{matrix} {}&amp;{{c_1} \le \left| {{e_k}/\hat s} \right| \le {c_2}} \end{matrix}} \end{matrix}}\\ {\begin{matrix} {{{10}^{ - 4}}}&amp;{\begin{matrix} {\begin{matrix} {}&amp;{} \end{matrix}}&amp;{} \end{matrix}otherwise} \end{matrix}} \end{matrix}} \right.

max ⁡ W , b 1 2 ∥ W ∗ ∥ 2 + λ 2 ∑ i = 1 m v i e i ∗ 2 s . t . y i = W ∗ φ ( x i ) + b ∗ + e i ∗ , i = 1 , ⋯ &ThinSpace; , m . \begin{array}{l} \mathop {\max }\limits_{W,b} \frac{1}{2}{\left\| {{W^ * }} \right\|^2} + \frac{\lambda }{2}\sum\limits_{i = 1}^m {{v_i}e{{_i^ * }^2}} \\ s.t.{y_i} = {W^ * }\varphi \left( {{x_i}} \right) + {b^ * } + {e_i}^ * ,i = 1, \cdots ,m. \end{array}

Lagrange函数为：
L ( W ∗ , b ∗ , e ∗ , α ∗ ) = 1 2 ∥ W ∗ ∥ 2 + λ 2 ∑ i = 1 m v i e i ∗ 2 − ∑ i = 1 m α i ∗ [ W ∗ φ ( x i ) + b ∗ + e i ∗ − y i ] L\left( {{W^ * },{b^ * },{e^ * },{\alpha ^ * }} \right) = \frac{1}{2}{\left\| {{W^ * }} \right\|^2} + \frac{\lambda }{2}\sum\limits_{i = 1}^m {{v_i}e{{_i^ * }^2}} - \sum\limits_{i = 1}^m {\alpha _i^ * } \left[ {{W^ * }\varphi \left( {{x_i}} \right) + {b^ * } + {e_i}^ * - {y_i}} \right]

[ 0 1 ⃗ T 1 ⃗ K + V λ ] [ b ∗ α ∗ ] = [ 0 y ] \left[ {\begin{matrix} 0&amp;{{{\vec 1}^T}}\\ {\vec 1}&amp;{K + {V_\lambda }} \end{matrix}} \right]\left[ {\begin{matrix} {{b^ * }}\\ {{\alpha ^ * }} \end{matrix}} \right] = \left[ {\begin{matrix} 0\\ y \end{matrix}} \right]

Lagrange乘子序列表征训练数据对模型的贡献，可以通过删除Lagrange乘子较小的样本点实现模型的稀疏化。

三、WLSSVM的python实现

from numpy import *

'''导入数据
input: filename:文件名
'''
dataMat = []
labelMat = []
fr = open(filename)
lineArr = line.strip().split('\t')
dataMat.append(float(lineArr[0]))
labelMat.append(float(lineArr[1]))
return mat(dataMat).T,mat(labelMat).T

def kernelTrans(X,A,kTup):
'''数据集中每一个数据向量与A的核函数值
input: X--特征数据集
A--输入向量
kTup--核函数参量定义
output: K--数据集中每一个数据向量与A的核函数值组成的矩阵
'''
X = mat(X)
m,n = shape(X)
K = mat(zeros((m,1)))
if kTup[0] == 'lin':
K = X * A.T
elif kTup[0] == 'rbf':
for j in range(m):
deltaRow = X[j] - A
K[j] = deltaRow * deltaRow.T
K = exp(K/(-1 * kTup[1] ** 2))
else: raise NameError('Houston We Have a Problem ,That Kernel is not recognized')
return K

class optStruct:
def __init__(self,dataMatIn,classLabels,C,kTup):
self.X = dataMatIn
self.labelMat = classLabels
self.C = C
self.m = shape(dataMatIn)[0]
self.alphas = mat(zeros((self.m,1)))
self.b = 0
self.K = mat(zeros((self.m,self.m)))  #特征数据集合中向量两两核函数值组成的矩阵，[i,j]表示第i个向量与第j个向量的核函数值
for i in range(self.m):
self.K[:,i] = kernelTrans(self.X, self.X[i,:], kTup)

def leastSquares(dataMatIn,classLabels,C,kTup):
'''最小二乘法求解alpha序列
input:dataMatIn:特征数据集
classLabels:分类标签集
C:参数，（松弛变量，允许有些数据点可以处于分隔面的错误一侧)
kTup: 核函数类型和参数选择
output:b--w.T*x+b=y中的b
alphas:alphas序列
'''
##1.参数设置
oS = optStruct(dataMatIn,classLabels,C,kTup)
unit = mat(ones((oS.m,1)))  #[1,1,...,1].T
I = eye(oS.m)
zero = mat(zeros((1,1)))
upmat = hstack((zero,unit.T))
downmat = hstack((unit,oS.K + I/float(C)))
##2.方程求解
completemat = vstack((upmat,downmat))  #lssvm中求解方程的左边矩阵
rightmat = vstack((zero,oS.labelMat))    # lssvm中求解方程的右边矩阵
b_alpha = completemat.I * rightmat
##3.导出偏置b和Lagrange乘子序列
oS.b = b_alpha[0,0]
for i in range(oS.m):
oS.alphas[i,0] = b_alpha[i+1,0]
e = oS.alphas/C
return oS.alphas,oS.b,e

def weights(e):
'''计算权重序列
input:e(mat):LSSVM误差矩阵
output:v(mat):权重矩阵
'''
##1.参数设置
c1 = 2.5
c2 = 3
m = shape(e)[0]
v = mat(zeros((m,1)))
v1 = eye(m)
q1 = int(m/4.0)
q3 = int((m*3.0)/4.0)
e1 = []
shang = mat(zeros((m,1)))
##2.误差序列从小到大排列
for i in range(m):
e1.append(e[i,0])
e1.sort()
##3.计算误差序列第三四分位与第一四分位的差
IQR = e1[q3] - e1[q1]
##4.计算s的值
s = IQR/(2 * 0.6745)
##5.计算每一个误差对应的权重
for j in range(m):
shang[j,0] = abs(e[j,0]/s)
for x in range(m):
if shang[x,0] <= c1:
v[x,0] = 1.0
if shang[x,0] > c1 and shang[x,0] <= c2:
v[x,0] = (c2 - shang[x,0])/(c2 - c1)
if shang[x,0] > c2:
v[x,0] = 0.0001
v1[x,x] = 1/float(v[x,0])
return v1

def weightsleastSquares(dataMatIn,classLabels,C,kTup,v1):
'''最小二乘法求解alpha序列
input:dataMatIn:特征数据集
classLabels:分类标签集
C:参数，（松弛变量，允许有些数据点可以处于分隔面的错误一侧)
kTup: 核函数类型和参数选择
output:b--w.T*x+b=y中的b
alphas:alphas序列
'''
##1.参数设置
oS = optStruct(dataMatIn,classLabels,C,kTup)
unit = mat(ones((oS.m,1)))  #[1,1,...,1].T
#I = eye(oS.m)
gamma = kTup[1]
zero = mat(zeros((1,1)))
upmat = hstack((zero,unit.T))
downmat = hstack((unit,oS.K + v1/float(C)))
##2.方程求解
completemat = vstack((upmat,downmat))  #lssvm中求解方程的左边矩阵
rightmat = vstack((zero,oS.labelMat))    # lssvm中求解方程的右边矩阵
b_alpha = completemat.I * rightmat
##3.导出偏置b和Lagrange乘子序列
oS.b = b_alpha[0,0]
for i in range(oS.m):
oS.alphas[i,0] = b_alpha[i+1,0]
e = oS.alphas/C
return oS.alphas,oS.b

def predict(alphas,b,dataMat):
'''预测结果
input:alphas(mat):WLSSVM模型的Lagrange乘子序列
b(float):WLSSVM模型回归方程的偏置
dataMat(mat):测试样本集
output:predict_result(mat):测试结果
'''
m,n = shape(dataMat)
predict_result = mat(zeros((m,1)))
for i in range(m):
Kx = kernelTrans(dataMat,dataMat[i,:],kTup)   #可以对alphas进行稀疏处理找到更准确的值
predict_result[i,0] =  Kx.T * alphas + b
return predict_result

def predict_average_error(predict_result,label):
'''计算平均预测误差
input:predict_result(mat):预测结果
label(mat):实际结果
output:average_error(float):平均误差
'''
m,n = shape(predict_result)
error = 0.0
for i in range(m):
error += abs(predict_result[i,0] - label[i,0])
average_error = error / m
return average_error

if __name__ == '__main__':
##1.数据导入
##2.参数设置
print('--------------------Parameter Setup------------------')
C = 0.6
k1 = 0.3
kernel = 'rbf'
kTup = (kernel,k1)
##3.求解LSSVM模型
print('-------------------Save LSSVM Model-----------------')
alphas,b,e = leastSquares(dataMat,labelMat,C,kTup)
##4.计算误差权重
print('----------------Calculate Error Weights-------------')
v1 = weights(e)
##5.求解WLSSVM模型
print('------------------Save WLSSVM Model--------------- -')
alphas1,b1 = weightsleastSquares(dataMat,labelMat,C,kTup,v1)
##6.预测结果
print('------------------Predict Result------------------ -')
predict_result = predict(alphas1,b1,dataMat)
##7.平均误差
print('-------------------Average Error------------------ -')
average_error = predict_average_error(predict_result,labelMat)


参考资料

1.https://blog.csdn.net/Luqiang_Shi/article/details/84204636
2.Suykens J A K , Brabanter J D , Lukas L , et al. Weighted least squares support vector machines: robustness and sparse approximation[J]. Neurocomputing, 2002, 48(1-4):85-105.
3.Deng W , Zheng Q , Chen L . Regularized Extreme Learning Machine[J]. IEEE Symposium on Computational Intelligence & Data Mining, 2009:389-395.

• 5
点赞
• 9
评论
• 50
收藏
• 打赏
• 扫一扫，分享海报

12-15

09-03 395
03-08
04-19 4153
11-27
05-15
06-29 3万+

Luqiang_Shi

¥2 ¥4 ¥6 ¥10 ¥20

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。