出发点
一旦判别函数的形式确定下来,不管它是线性的还是非线性的,剩下的问题就是如何确定它的系数。在模式识别中,系数确定的一个主要方法就是通过对已知样本的训练和学习来得到。感知器算法就是通过训练样本模式的迭代和学习,产生线性(或广义线性)可分的模式判别函数。
基本思想
采用感知器算法能通过对训练模式样本集的“学习”得到判别函数的系数。这里采用的算法不需要对各类别中模式的统计性质做任何假设,因此称为确定性的方法。
具体算法
已知两个训练模式集分别属于
类和
类,权向量的初始值为
,可任意取值。若
,
,若
,
,则在用全部训练模式集进行迭代训练时,第k次的训练步骤为:
若
且
,则分类器对第k个模式
做了错误分类,此时应校正权向量,使得
,其中C为一个校正增量;
若
且
,同样分类器分类错误,则权向量应该校正为
;
若以上情况不符合,则表明该模式样本在第k次中分类正确,因此权向量不变,即
。
若对
的模式样本乘以(-1),有
时,w(k+1)=w(k)+Cx_{k}。此时感知器算法可统一写成:
示例
用感知器算法求下列模式分类的解向量w:
因为手算比较麻烦,就直接上代码了。
import numpy as np
w_1 = np.array([[0,0,0],
[1,0,0],
[1,0,1],
[1,1,0]]).T
w_2 = np.array([[0,0,1],
[0,1,1],
[0,1,0],
[1,1,1]]).T
# 增广化
w_1_Aug = np.ones((w_1.shape[0]+1,w_1.shape[1]))
w_1_Aug[0:w_1.shape[0],0:w_1.shape[1]] = w_1
w_2_Aug = np.ones((w_2.shape[0]+1,w_2.shape[1]))
w_2_Aug[0:w_2.shape[0],0:w_2.shape[1]] = w_2
# 符号规范化
w_2_Aug = 0 - w_2_Aug
# 初始化解向量
Weight = np.array([0,0,0,0]).reshape((4,1))
# 设定校正增量
C = 1
# 初始化标记
flag = True
while(True):
for i in range(w_1_Aug.shape[1]):
if(np.dot(Weight.T,np.reshape(w_1_Aug[:,i:i+1],(4,1)))[0][0] > 0):
Weight = Weight
else:
Weight = Weight + C*np.reshape(w_1_Aug[:,i:i+1],(4,1))
print(Weight)
flag = False
for i in range(w_2_Aug.shape[1]):
if(np.dot(Weight.T,np.reshape(w_2_Aug[:,i:i+1],(4,1)))[0][0] > 0):
Weight = Weight
else:
Weight = Weight + C*np.reshape(w_2_Aug[:,i:i+1],(4,1))
print(Weight)
flag = False
if(flag == True):
break
else:
flag = True
最后得出:
多类感知器多类情况3:
对M类模式存在M个判别函数
,若
,则
,
。
设有M种模式类别
,若在训练过程中的第
次迭代时,一个属于
类的模式样本x送入分类器,则应先计算出M个判别函数:
若
的条件成立,则权向量不变,即
若其中第
个权向量使得
,则相应的权向量应做调整,即
多类示例
用多类感知器算法求下列模式的判别函数:
因为手算比较麻烦,就直接上代码了。
import numpy as np
w_1 = np.array([[-1,-1]]).T
w_2 = np.array([[0,0]]).T
w_3 = np.array([[1,1]]).T
# 增广化
w_1_Aug = np.ones((w_1.shape[0]+1,w_1.shape[1]))
w_1_Aug[0:w_1.shape[0],0:w_1.shape[1]] = w_1
w_2_Aug = np.ones((w_2.shape[0]+1,w_2.shape[1]))
w_2_Aug[0:w_2.shape[0],0:w_2.shape[1]] = w_2
w_3_Aug = np.ones((w_3.shape[0]+1,w_3.shape[1]))
w_3_Aug[0:w_3.shape[0],0:w_3.shape[1]] = w_3
# 初始化解向量
Weight1 = np.array([[0,0,0]]).T
Weight2 = Weight1.copy()
Weight3 = Weight1.copy()
# 设定校正增量
C = 1
# 初始化标记
flag = True
while(True):
if(np.dot(Weight1.T,w_1_Aug) <= np.dot(Weight2.T,w_1_Aug) and
np.dot(Weight1.T,w_1_Aug) <= np.dot(Weight3.T,w_1_Aug)):
Weight1 = Weight1 + C*w_1_Aug
Weight2 = Weight2 - C*w_1_Aug
Weight3 = Weight3 - C*w_1_Aug
flag = False
elif(np.dot(Weight1.T,w_1_Aug) <= np.dot(Weight2.T,w_1_Aug) and
np.dot(Weight1.T,w_1_Aug) > np.dot(Weight3.T,w_1_Aug)):
Weight1 = Weight1 + C*w_1_Aug
Weight2 = Weight2 - C*w_1_Aug
flag = False
elif(np.dot(Weight1.T,w_1_Aug) > np.dot(Weight2.T,w_1_Aug) and
np.dot(Weight1.T,w_1_Aug) <= np.dot(Weight3.T,w_1_Aug)):
Weight1 = Weight1 + C*w_1_Aug
Weight3 = Weight3 - C*w_1_Aug
flag = False
if(np.dot(Weight2.T,w_2_Aug) <= np.dot(Weight1.T,w_2_Aug) and
np.dot(Weight2.T,w_2_Aug) <= np.dot(Weight3.T,w_2_Aug)):
Weight1 = Weight1 - C*w_2_Aug
Weight2 = Weight2 + C*w_2_Aug
Weight3 = Weight3 - C*w_2_Aug
flag = False
elif(np.dot(Weight2.T,w_2_Aug) <= np.dot(Weight1.T,w_2_Aug) and
np.dot(Weight2.T,w_2_Aug) > np.dot(Weight3.T,w_2_Aug)):
Weight1 = Weight1 - C*w_2_Aug
Weight2 = Weight2 + C*w_2_Aug
flag = False
elif(np.dot(Weight2.T,w_2_Aug) > np.dot(Weight1.T,w_2_Aug) and
np.dot(Weight2.T,w_2_Aug) <= np.dot(Weight3.T,w_2_Aug)):
Weight2 = Weight2 + C*w_2_Aug
Weight3 = Weight3 - C*w_2_Aug
flag = False
if(np.dot(Weight3.T,w_3_Aug) <= np.dot(Weight1.T,w_3_Aug) and
np.dot(Weight3.T,w_3_Aug) <= np.dot(Weight2.T,w_3_Aug)):
Weight1 = Weight1 - C*w_3_Aug
Weight2 = Weight2 - C*w_3_Aug
Weight3 = Weight3 + C*w_3_Aug
flag = False
if(np.dot(Weight3.T,w_3_Aug) <= np.dot(Weight1.T,w_3_Aug) and
np.dot(Weight3.T,w_3_Aug) > np.dot(Weight2.T,w_3_Aug)):
Weight1 = Weight1 - C*w_3_Aug
Weight3 = Weight3 + C*w_3_Aug
flag = False
if(np.dot(Weight3.T,w_3_Aug) > np.dot(Weight1.T,w_3_Aug) and
np.dot(Weight3.T,w_3_Aug) <= np.dot(Weight2.T,w_3_Aug)):
Weight2 = Weight2 - C*w_3_Aug
Weight3 = Weight3 + C*w_3_Aug
flag = False
if(flag == True):
break
else:
flag = True
print(Weight1)
print(Weight2)
print(Weight3)
最后得出:
则判别函数为:
后记
如有问题,欢迎交流批评指正。