机器学习实战：支持向量机

最新推荐文章于 2022-04-30 09:11:47 发布
最新推荐文章于 2022-04-30 09:11:47 发布
阅读量233 收藏
点赞数
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/ctrigger/article/details/90674195
版权

支持向量机（Support Vector Machine, SVM）是一类按监督学习（supervised learning）方式对数据进行二元分类（binary classification）的广义线性分类器（generalized linear classifier），其决策边界是对学习样本求解的最大边距超平面（maximum-margin hyperplane） 。

SVM使用铰链损失函数（hinge loss）计算经验风险（empirical risk）并在求解系统中加入了正则化项以优化结构风险（structural risk），是一个具有稀疏性和稳健性的分类器 。SVM可以通过核方法（kernel method）进行非线性分类，是常见的核学习（kernel learning）方法之一  。

SVM被提出于1964年，在二十世纪90年代后得到快速发展并衍生出一系列改进和扩展算法，在人像识别（face recognition）、文本分类（text categorization）等模式识别（pattern recognition）问题中有得到应用。

机器学习实战源码：

'''
Created on Nov 4, 2010
Chapter 5 source file for Machine Learing in Action
@author: Peter
'''
from numpy import *
from time import sleep

def loadDataSet(fileName):
    dataMat = []; labelMat = []
    fr = open(fileName)
    for line in fr.readlines():
        lineArr = line.strip().split('\t')
        dataMat.append([float(lineArr[0]), float(lineArr[1])])
        labelMat.append(float(lineArr[2]))
    return dataMat,labelMat

def selectJrand(i,m):
    j=i #we want to select any J not equal to i
    while (j==i):
        j = int(random.uniform(0,m))
    return j

def clipAlpha(aj,H,L):
    if aj > H:
        aj = H
    if L > aj:
        aj = L
    return aj

def smoSimple(dataMatIn, classLabels, C, toler, maxIter):
    dataMatrix = mat(dataMatIn); labelMat = mat(classLabels).transpose()
    b = 0; m,n = shape(dataMatrix)
    alphas = mat(zeros((m,1)))
    iter = 0
    while (iter < maxIter):
        alphaPairsChanged = 0
        for i in range(m):
            fXi = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[i,:].T)) + b
            Ei = fXi - float(labelMat[i])#if checks if an example violates KKT conditions
            if ((labelMat[i]*Ei < -toler) and (alphas[i] < C)) or ((labelMat[i]*Ei > toler) and (alphas[i] > 0)):
                j = selectJrand(i,m)
                fXj = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[j,:].T)) + b
                Ej = fXj - float(labelMat[j])
                alphaIold = alphas[i].copy(); alphaJold = alphas[j].copy();
                if (labelMat[i] != labelMat[j]):
                    L = max(0, alphas[j] - alphas[i])
                    H = min(C, C + alphas[j] - alphas[i])
                else:
                    L = max(0, alphas[j] + alphas[i] - C)
                    H = min(C, alphas[j] + alphas[i])
                if L==H: print("L==H"); continue
                eta = 2.0 * dataMatrix[i,:]*dataMatrix[j,:].T - dataMatrix[i,:]*dataMatrix[i,:].T - dataMatrix[j,:]*dataMatrix[j,:].T
                if eta >= 0: print("eta>=0"); continue
                alphas[j] -= labelMat[j]*(Ei - Ej)/eta
                alphas[j] = clipAlpha(alphas[j],H,L)
                if (abs(alphas[j] - alphaJold) < 0.00001): print("j not moving enough"); continue
                alphas[i] += labelMat[j]*labelMat[i]*(alphaJold - alphas[j])#update i by the same amount as j
                                                                        #the update is in the oppostie direction
                b1 = b - Ei- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[i,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[i,:]*dataMatrix[j,:].T
                b2 = b - Ej- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[j,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[j,:]*dataMatrix[j,:].T
                if (0 < alphas[i]) and (C > alphas[i]): b = b1
                elif (0 < alphas[j]) and (C > alphas[j]): b = b2
                else: b = (b1 + b2)/2.0
                alphaPairsChanged += 1
                print("iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged))
        if (alphaPairsChanged == 0): iter += 1
        else: iter = 0
        print("iteration number: %d" % iter)
    return b,alphas

def kernelTrans(X, A, kTup): #calc the kernel or transform data to a higher dimensional space
    m,n = shape(X)
    K = mat(zeros((m,1)))
    if kTup[0]=='lin': K = X * A.T   #linear kernel
    elif kTup[0]=='rbf':
        for j in range(m):
            deltaRow = X[j,:] - A
            K[j] = deltaRow*deltaRow.T
        K = exp(K/(-1*kTup[1]**2)) #divide in NumPy is element-wise not matrix like Matlab
    else: raise NameError('Houston We Have a Problem -- \
    That Kernel is not recognized')
    return K

class optStruct:
    def __init__(self,dataMatIn, classLabels, C, toler, kTup):  # Initialize the structure with the parameters
        self.X = dataMatIn
        self.labelMat = classLabels
        self.C = C
        self.tol = toler
        self.m = shape(dataMatIn)[0]
        self.alphas = mat(zeros((self.m,1)))
        self.b = 0
        self.eCache = mat(zeros((self.m,2))) #first column is valid flag
        self.K = mat(zeros((self.m,self.m)))
        for i in range(self.m):
            self.K[:,i] = kernelTrans(self.X, self.X[i,:], kTup)

def calcEk(oS, k):
    fXk = float(multiply(oS.alphas,oS.labelMat).T*oS.K[:,k] + oS.b)
    Ek = fXk - float(oS.labelMat[k])
    return Ek

def selectJ(i, oS, Ei):         #this is the second choice -heurstic, and calcs Ej
    maxK = -1; maxDeltaE = 0; Ej = 0
    oS.eCache[i] = [1,Ei]  #set valid #choose the alpha that gives the maximum delta E
    validEcacheList = nonzero(oS.eCache[:,0].A)[0]
    if (len(validEcacheList)) > 1:
        for k in validEcacheList:   #loop through valid Ecache values and find the one that maximizes delta E
            if k == i: continue #don't calc for i, waste of time
            Ek = calcEk(oS, k)
            deltaE = abs(Ei - Ek)
            if (deltaE > maxDeltaE):
                maxK = k; maxDeltaE = deltaE; Ej = Ek
        return maxK, Ej
    else:   #in this case (first time around) we don't have any valid eCache values
        j = selectJrand(i, oS.m)
        Ej = calcEk(oS, j)
    return j, Ej

def updateEk(oS, k):#after any alpha has changed update the new value in the cache
    Ek = calcEk(oS, k)
    oS.eCache[k] = [1,Ek]

def innerL(i, oS):
    Ei = calcEk(oS, i)
    if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)):
        j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand
        alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy();
        if (oS.labelMat[i] != oS.labelMat[j]):
            L = max(0, oS.alphas[j] - oS.alphas[i])
            H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i])
        else:
            L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C)
            H = min(oS.C, oS.alphas[j] + oS.alphas[i])
        if L==H: print("L==H"); return 0
        eta = 2.0 * oS.K[i,j] - oS.K[i,i] - oS.K[j,j] #changed for kernel
        if eta >= 0: print("eta>=0"); return 0
        oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta
        oS.alphas[j] = clipAlpha(oS.alphas[j],H,L)
        updateEk(oS, j) #added this for the Ecache
        if (abs(oS.alphas[j] - alphaJold) < 0.00001): print("j not moving enough"); return 0
        oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j
        updateEk(oS, i) #added this for the Ecache                    #the update is in the oppostie direction
        b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,i] - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[i,j]
        b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,j]- oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[j,j]
        if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1
        elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2
        else: oS.b = (b1 + b2)/2.0
        return 1
    else: return 0

def smoP(dataMatIn, classLabels, C, toler, maxIter,kTup=('lin', 0)):    #full Platt SMO
    oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler, kTup)
    iter = 0
    entireSet = True; alphaPairsChanged = 0
    while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)):
        alphaPairsChanged = 0
        if entireSet:   #go over all
            for i in range(oS.m):
                alphaPairsChanged += innerL(i,oS)
                print("fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged))
            iter += 1
        else:#go over non-bound (railed) alphas
            nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0]
            for i in nonBoundIs:
                alphaPairsChanged += innerL(i,oS)
                print("non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged))
            iter += 1
        if entireSet: entireSet = False #toggle entire set loop
        elif (alphaPairsChanged == 0): entireSet = True
        print("iteration number: %d" % iter)
    return oS.b,oS.alphas

def calcWs(alphas,dataArr,classLabels):
    X = mat(dataArr); labelMat = mat(classLabels).transpose()
    m,n = shape(X)
    w = zeros((n,1))
    for i in range(m):
        w += multiply(alphas[i]*labelMat[i],X[i,:].T)
    return w

def testRbf(k1=1.3):
    dataArr,labelArr = loadDataSet('testSetRBF.txt')
    b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, ('rbf', k1)) #C=200 important
    datMat=mat(dataArr); labelMat = mat(labelArr).transpose()
    svInd=nonzero(alphas.A>0)[0]
    sVs=datMat[svInd] #get matrix of only support vectors
    labelSV = labelMat[svInd];
    print("there are %d Support Vectors" % shape(sVs)[0])
    m,n = shape(datMat)
    errorCount = 0
    for i in range(m):
        kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1))
        predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b
        if sign(predict)!=sign(labelArr[i]): errorCount += 1
    print("the training error rate is: %f" % (float(errorCount)/m))
    dataArr,labelArr = loadDataSet('testSetRBF2.txt')
    errorCount = 0
    datMat=mat(dataArr); labelMat = mat(labelArr).transpose()
    m,n = shape(datMat)
    for i in range(m):
        kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1))
        predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b
        if sign(predict)!=sign(labelArr[i]): errorCount += 1
    print("the test error rate is: %f" % (float(errorCount)/m))

def img2vector(filename):
    returnVect = zeros((1,1024))
    fr = open(filename)
    for i in range(32):
        lineStr = fr.readline()
        for j in range(32):
            returnVect[0,32*i+j] = int(lineStr[j])
    return returnVect

def loadImages(dirName):
    from os import listdir
    hwLabels = []
    trainingFileList = listdir(dirName)           #load the training set
    m = len(trainingFileList)
    trainingMat = zeros((m,1024))
    for i in range(m):
        fileNameStr = trainingFileList[i]
        fileStr = fileNameStr.split('.')[0]     #take off .txt
        classNumStr = int(fileStr.split('_')[0])
        if classNumStr == 9: hwLabels.append(-1)
        else: hwLabels.append(1)
        trainingMat[i,:] = img2vector('%s/%s' % (dirName, fileNameStr))
    return trainingMat, hwLabels

def testDigits(kTup=('rbf', 10)):
    dataArr,labelArr = loadImages('trainingDigits')
    b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, kTup)
    datMat=mat(dataArr); labelMat = mat(labelArr).transpose()
    svInd=nonzero(alphas.A>0)[0]
    sVs=datMat[svInd]
    labelSV = labelMat[svInd];
    print("there are %d Support Vectors" % shape(sVs)[0])
    m,n = shape(datMat)
    errorCount = 0
    for i in range(m):
        kernelEval = kernelTrans(sVs,datMat[i,:],kTup)
        predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b
        if sign(predict)!=sign(labelArr[i]): errorCount += 1
    print("the training error rate is: %f" % (float(errorCount)/m))
    dataArr,labelArr = loadImages('testDigits')
    errorCount = 0
    datMat=mat(dataArr); labelMat = mat(labelArr).transpose()
    m,n = shape(datMat)
    for i in range(m):
        kernelEval = kernelTrans(sVs,datMat[i,:],kTup)
        predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b
        if sign(predict)!=sign(labelArr[i]): errorCount += 1
    print("the test error rate is: %f" % (float(errorCount)/m))


'''#######********************************
Non-Kernel VErsions below
'''#######********************************

class optStructK:
    def __init__(self,dataMatIn, classLabels, C, toler):  # Initialize the structure with the parameters
        self.X = dataMatIn
        self.labelMat = classLabels
        self.C = C
        self.tol = toler
        self.m = shape(dataMatIn)[0]
        self.alphas = mat(zeros((self.m,1)))
        self.b = 0
        self.eCache = mat(zeros((self.m,2))) #first column is valid flag

def calcEkK(oS, k):
    fXk = float(multiply(oS.alphas,oS.labelMat).T*(oS.X*oS.X[k,:].T)) + oS.b
    Ek = fXk - float(oS.labelMat[k])
    return Ek

def selectJK(i, oS, Ei):         #this is the second choice -heurstic, and calcs Ej
    maxK = -1; maxDeltaE = 0; Ej = 0
    oS.eCache[i] = [1,Ei]  #set valid #choose the alpha that gives the maximum delta E
    validEcacheList = nonzero(oS.eCache[:,0].A)[0]
    if (len(validEcacheList)) > 1:
        for k in validEcacheList:   #loop through valid Ecache values and find the one that maximizes delta E
            if k == i: continue #don't calc for i, waste of time
            Ek = calcEk(oS, k)
            deltaE = abs(Ei - Ek)
            if (deltaE > maxDeltaE):
                maxK = k; maxDeltaE = deltaE; Ej = Ek
        return maxK, Ej
    else:   #in this case (first time around) we don't have any valid eCache values
        j = selectJrand(i, oS.m)
        Ej = calcEk(oS, j)
    return j, Ej

def updateEkK(oS, k):#after any alpha has changed update the new value in the cache
    Ek = calcEk(oS, k)
    oS.eCache[k] = [1,Ek]

def innerLK(i, oS):
    Ei = calcEk(oS, i)
    if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)):
        j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand
        alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy();
        if (oS.labelMat[i] != oS.labelMat[j]):
            L = max(0, oS.alphas[j] - oS.alphas[i])
            H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i])
        else:
            L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C)
            H = min(oS.C, oS.alphas[j] + oS.alphas[i])
        if L==H: print("L==H"); return 0
        eta = 2.0 * oS.X[i,:]*oS.X[j,:].T - oS.X[i,:]*oS.X[i,:].T - oS.X[j,:]*oS.X[j,:].T
        if eta >= 0: print("eta>=0"); return 0
        oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta
        oS.alphas[j] = clipAlpha(oS.alphas[j],H,L)
        updateEk(oS, j) #added this for the Ecache
        if (abs(oS.alphas[j] - alphaJold) < 0.00001): print("j not moving enough"); return 0
        oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j]) #update i by the same amount as j
        updateEk(oS, i) #added this for the Ecache                    #the update is in the oppostie direction
        b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[i,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[i,:]*oS.X[j,:].T
        b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[j,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[j,:]*oS.X[j,:].T
        if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1
        elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2
        else: oS.b = (b1 + b2)/2.0
        return 1
    else: return 0

def smoPK(dataMatIn, classLabels, C, toler, maxIter):    #full Platt SMO
    oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler)
    iter = 0
    entireSet = True; alphaPairsChanged = 0
    while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)):
        alphaPairsChanged = 0
        if entireSet:   #go over all
            for i in range(oS.m):
                alphaPairsChanged += innerL(i,oS)
                print("fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged))
            iter += 1
        else:#go over non-bound (railed) alphas
            nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0]
            for i in nonBoundIs:
                alphaPairsChanged += innerL(i,oS)
                print("non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged))
            iter += 1
        if entireSet: entireSet = False #toggle entire set loop
        elif (alphaPairsChanged == 0): entireSet = True
        print("iteration number: %d" % iter)
    return oS.b,oS.alphas

if __name__ == '__main__':
    testRbf()

运行结果：

"C:\Program Files\Python\Python37\python.exe" "D:/Pycharm Projects/MLDemo/MLDemo.py"
L==H
fullSet, iter: 0 i:0, pairs changed 0
fullSet, iter: 0 i:1, pairs changed 1
fullSet, iter: 0 i:2, pairs changed 2
fullSet, iter: 0 i:3, pairs changed 3
fullSet, iter: 0 i:4, pairs changed 3
fullSet, iter: 0 i:5, pairs changed 4
fullSet, iter: 0 i:6, pairs changed 5
fullSet, iter: 0 i:7, pairs changed 5
fullSet, iter: 0 i:8, pairs changed 6
fullSet, iter: 0 i:9, pairs changed 6
fullSet, iter: 0 i:10, pairs changed 7
fullSet, iter: 0 i:11, pairs changed 8
j not moving enough
fullSet, iter: 0 i:12, pairs changed 8
fullSet, iter: 0 i:13, pairs changed 9
fullSet, iter: 0 i:14, pairs changed 10
fullSet, iter: 0 i:15, pairs changed 11
fullSet, iter: 0 i:16, pairs changed 12
fullSet, iter: 0 i:17, pairs changed 12
fullSet, iter: 0 i:18, pairs changed 13
fullSet, iter: 0 i:19, pairs changed 14
fullSet, iter: 0 i:20, pairs changed 14
fullSet, iter: 0 i:21, pairs changed 15
fullSet, iter: 0 i:22, pairs changed 15
fullSet, iter: 0 i:23, pairs changed 15
fullSet, iter: 0 i:24, pairs changed 16
j not moving enough
fullSet, iter: 0 i:25, pairs changed 16
fullSet, iter: 0 i:26, pairs changed 16
j not moving enough
fullSet, iter: 0 i:27, pairs changed 16
fullSet, iter: 0 i:28, pairs changed 16
fullSet, iter: 0 i:29, pairs changed 16
j not moving enough
fullSet, iter: 0 i:30, pairs changed 16
fullSet, iter: 0 i:31, pairs changed 17
fullSet, iter: 0 i:32, pairs changed 17
fullSet, iter: 0 i:33, pairs changed 18
j not moving enough
fullSet, iter: 0 i:34, pairs changed 18
fullSet, iter: 0 i:35, pairs changed 18
fullSet, iter: 0 i:36, pairs changed 19
fullSet, iter: 0 i:37, pairs changed 19
fullSet, iter: 0 i:38, pairs changed 19
j not moving enough
fullSet, iter: 0 i:39, pairs changed 19
fullSet, iter: 0 i:40, pairs changed 20
fullSet, iter: 0 i:41, pairs changed 21
L==H
fullSet, iter: 0 i:42, pairs changed 21
fullSet, iter: 0 i:43, pairs changed 21
j not moving enough
fullSet, iter: 0 i:44, pairs changed 21
fullSet, iter: 0 i:45, pairs changed 22
fullSet, iter: 0 i:46, pairs changed 22
fullSet, iter: 0 i:47, pairs changed 22
fullSet, iter: 0 i:48, pairs changed 23
fullSet, iter: 0 i:49, pairs changed 23
fullSet, iter: 0 i:50, pairs changed 24
j not moving enough
fullSet, iter: 0 i:51, pairs changed 24
fullSet, iter: 0 i:52, pairs changed 25
fullSet, iter: 0 i:53, pairs changed 25
fullSet, iter: 0 i:54, pairs changed 26
fullSet, iter: 0 i:55, pairs changed 26
fullSet, iter: 0 i:56, pairs changed 27
L==H
fullSet, iter: 0 i:57, pairs changed 27
fullSet, iter: 0 i:58, pairs changed 27
fullSet, iter: 0 i:59, pairs changed 27
j not moving enough
fullSet, iter: 0 i:60, pairs changed 27
fullSet, iter: 0 i:61, pairs changed 28
fullSet, iter: 0 i:62, pairs changed 29
fullSet, iter: 0 i:63, pairs changed 29
fullSet, iter: 0 i:64, pairs changed 29
fullSet, iter: 0 i:65, pairs changed 29
fullSet, iter: 0 i:66, pairs changed 29
fullSet, iter: 0 i:67, pairs changed 29
fullSet, iter: 0 i:68, pairs changed 29
fullSet, iter: 0 i:69, pairs changed 29
fullSet, iter: 0 i:70, pairs changed 29
fullSet, iter: 0 i:71, pairs changed 29
fullSet, iter: 0 i:72, pairs changed 29
fullSet, iter: 0 i:73, pairs changed 29
fullSet, iter: 0 i:74, pairs changed 30
j not moving enough
fullSet, iter: 0 i:75, pairs changed 30
fullSet, iter: 0 i:76, pairs changed 30
fullSet, iter: 0 i:77, pairs changed 30
fullSet, iter: 0 i:78, pairs changed 30
fullSet, iter: 0 i:79, pairs changed 30
j not moving enough
fullSet, iter: 0 i:80, pairs changed 30
fullSet, iter: 0 i:81, pairs changed 30
L==H
fullSet, iter: 0 i:82, pairs changed 30
fullSet, iter: 0 i:83, pairs changed 30
fullSet, iter: 0 i:84, pairs changed 30
fullSet, iter: 0 i:85, pairs changed 30
fullSet, iter: 0 i:86, pairs changed 30
fullSet, iter: 0 i:87, pairs changed 30
fullSet, iter: 0 i:88, pairs changed 30
fullSet, iter: 0 i:89, pairs changed 30
fullSet, iter: 0 i:90, pairs changed 30
fullSet, iter: 0 i:91, pairs changed 30
fullSet, iter: 0 i:92, pairs changed 30
fullSet, iter: 0 i:93, pairs changed 30
fullSet, iter: 0 i:94, pairs changed 30
fullSet, iter: 0 i:95, pairs changed 30
L==H
fullSet, iter: 0 i:96, pairs changed 30
fullSet, iter: 0 i:97, pairs changed 30
fullSet, iter: 0 i:98, pairs changed 30
L==H
fullSet, iter: 0 i:99, pairs changed 30
iteration number: 1
non-bound, iter: 1 i:0, pairs changed 1
j not moving enough
non-bound, iter: 1 i:1, pairs changed 1
non-bound, iter: 1 i:2, pairs changed 2
j not moving enough
non-bound, iter: 1 i:3, pairs changed 2
j not moving enough
non-bound, iter: 1 i:6, pairs changed 2
j not moving enough
non-bound, iter: 1 i:8, pairs changed 2
j not moving enough
non-bound, iter: 1 i:10, pairs changed 2
j not moving enough
non-bound, iter: 1 i:11, pairs changed 2
j not moving enough
non-bound, iter: 1 i:13, pairs changed 2
j not moving enough
non-bound, iter: 1 i:14, pairs changed 2
non-bound, iter: 1 i:15, pairs changed 3
j not moving enough
non-bound, iter: 1 i:16, pairs changed 3
j not moving enough
non-bound, iter: 1 i:18, pairs changed 3
j not moving enough
non-bound, iter: 1 i:19, pairs changed 3
j not moving enough
non-bound, iter: 1 i:21, pairs changed 3
j not moving enough
non-bound, iter: 1 i:27, pairs changed 3
j not moving enough
non-bound, iter: 1 i:30, pairs changed 3
j not moving enough
non-bound, iter: 1 i:31, pairs changed 3
j not moving enough
non-bound, iter: 1 i:33, pairs changed 3
j not moving enough
non-bound, iter: 1 i:36, pairs changed 3
j not moving enough
non-bound, iter: 1 i:40, pairs changed 3
non-bound, iter: 1 i:41, pairs changed 3
j not moving enough
non-bound, iter: 1 i:42, pairs changed 3
j not moving enough
non-bound, iter: 1 i:45, pairs changed 3
j not moving enough
non-bound, iter: 1 i:48, pairs changed 3
j not moving enough
non-bound, iter: 1 i:50, pairs changed 3
j not moving enough
non-bound, iter: 1 i:52, pairs changed 3
j not moving enough
non-bound, iter: 1 i:54, pairs changed 3
non-bound, iter: 1 i:56, pairs changed 4
j not moving enough
non-bound, iter: 1 i:61, pairs changed 4
non-bound, iter: 1 i:62, pairs changed 5
j not moving enough
non-bound, iter: 1 i:74, pairs changed 5
iteration number: 2
j not moving enough
non-bound, iter: 2 i:1, pairs changed 0
j not moving enough
non-bound, iter: 2 i:2, pairs changed 0
j not moving enough
non-bound, iter: 2 i:3, pairs changed 0
j not moving enough
non-bound, iter: 2 i:6, pairs changed 0
non-bound, iter: 2 i:8, pairs changed 1
j not moving enough
non-bound, iter: 2 i:10, pairs changed 1
non-bound, iter: 2 i:11, pairs changed 2
j not moving enough
non-bound, iter: 2 i:13, pairs changed 2
j not moving enough
non-bound, iter: 2 i:14, pairs changed 2
j not moving enough
non-bound, iter: 2 i:16, pairs changed 2
j not moving enough
non-bound, iter: 2 i:18, pairs changed 2
j not moving enough
non-bound, iter: 2 i:19, pairs changed 2
j not moving enough
non-bound, iter: 2 i:21, pairs changed 2
j not moving enough
non-bound, iter: 2 i:27, pairs changed 2
j not moving enough
non-bound, iter: 2 i:30, pairs changed 2
j not moving enough
non-bound, iter: 2 i:31, pairs changed 2
j not moving enough
non-bound, iter: 2 i:33, pairs changed 2
j not moving enough
non-bound, iter: 2 i:36, pairs changed 2
j not moving enough
non-bound, iter: 2 i:40, pairs changed 2
j not moving enough
non-bound, iter: 2 i:41, pairs changed 2
j not moving enough
non-bound, iter: 2 i:42, pairs changed 2
j not moving enough
non-bound, iter: 2 i:45, pairs changed 2
j not moving enough
non-bound, iter: 2 i:48, pairs changed 2
j not moving enough
non-bound, iter: 2 i:50, pairs changed 2
j not moving enough
non-bound, iter: 2 i:52, pairs changed 2
j not moving enough
non-bound, iter: 2 i:54, pairs changed 2
j not moving enough
non-bound, iter: 2 i:56, pairs changed 2
j not moving enough
non-bound, iter: 2 i:61, pairs changed 2
j not moving enough
non-bound, iter: 2 i:62, pairs changed 2
j not moving enough
non-bound, iter: 2 i:74, pairs changed 2
iteration number: 3
j not moving enough
non-bound, iter: 3 i:1, pairs changed 0
j not moving enough
non-bound, iter: 3 i:2, pairs changed 0
j not moving enough
non-bound, iter: 3 i:3, pairs changed 0
j not moving enough
non-bound, iter: 3 i:6, pairs changed 0
j not moving enough
non-bound, iter: 3 i:8, pairs changed 0
j not moving enough
non-bound, iter: 3 i:10, pairs changed 0
non-bound, iter: 3 i:11, pairs changed 0
j not moving enough
non-bound, iter: 3 i:13, pairs changed 0
j not moving enough
non-bound, iter: 3 i:14, pairs changed 0
j not moving enough
non-bound, iter: 3 i:16, pairs changed 0
j not moving enough
non-bound, iter: 3 i:18, pairs changed 0
j not moving enough
non-bound, iter: 3 i:19, pairs changed 0
j not moving enough
non-bound, iter: 3 i:21, pairs changed 0
j not moving enough
non-bound, iter: 3 i:27, pairs changed 0
j not moving enough
non-bound, iter: 3 i:30, pairs changed 0
j not moving enough
non-bound, iter: 3 i:31, pairs changed 0
j not moving enough
non-bound, iter: 3 i:33, pairs changed 0
j not moving enough
non-bound, iter: 3 i:36, pairs changed 0
j not moving enough
non-bound, iter: 3 i:40, pairs changed 0
j not moving enough
non-bound, iter: 3 i:41, pairs changed 0
j not moving enough
non-bound, iter: 3 i:42, pairs changed 0
j not moving enough
non-bound, iter: 3 i:45, pairs changed 0
j not moving enough
non-bound, iter: 3 i:48, pairs changed 0
j not moving enough
non-bound, iter: 3 i:50, pairs changed 0
j not moving enough
non-bound, iter: 3 i:52, pairs changed 0
j not moving enough
non-bound, iter: 3 i:54, pairs changed 0
j not moving enough
non-bound, iter: 3 i:56, pairs changed 0
j not moving enough
non-bound, iter: 3 i:61, pairs changed 0
j not moving enough
non-bound, iter: 3 i:62, pairs changed 0
j not moving enough
non-bound, iter: 3 i:74, pairs changed 0
iteration number: 4
j not moving enough
fullSet, iter: 4 i:0, pairs changed 0
j not moving enough
fullSet, iter: 4 i:1, pairs changed 0
j not moving enough
fullSet, iter: 4 i:2, pairs changed 0
j not moving enough
fullSet, iter: 4 i:3, pairs changed 0
fullSet, iter: 4 i:4, pairs changed 0
fullSet, iter: 4 i:5, pairs changed 0
j not moving enough
fullSet, iter: 4 i:6, pairs changed 0
fullSet, iter: 4 i:7, pairs changed 0
j not moving enough
fullSet, iter: 4 i:8, pairs changed 0
fullSet, iter: 4 i:9, pairs changed 0
j not moving enough
fullSet, iter: 4 i:10, pairs changed 0
fullSet, iter: 4 i:11, pairs changed 0
fullSet, iter: 4 i:12, pairs changed 0
j not moving enough
fullSet, iter: 4 i:13, pairs changed 0
j not moving enough
fullSet, iter: 4 i:14, pairs changed 0
fullSet, iter: 4 i:15, pairs changed 0
j not moving enough
fullSet, iter: 4 i:16, pairs changed 0
L==H
fullSet, iter: 4 i:17, pairs changed 0
j not moving enough
fullSet, iter: 4 i:18, pairs changed 0
j not moving enough
fullSet, iter: 4 i:19, pairs changed 0
fullSet, iter: 4 i:20, pairs changed 0
j not moving enough
fullSet, iter: 4 i:21, pairs changed 0
fullSet, iter: 4 i:22, pairs changed 0
fullSet, iter: 4 i:23, pairs changed 0
j not moving enough
fullSet, iter: 4 i:24, pairs changed 0
L==H
fullSet, iter: 4 i:25, pairs changed 0
fullSet, iter: 4 i:26, pairs changed 0
j not moving enough
fullSet, iter: 4 i:27, pairs changed 0
fullSet, iter: 4 i:28, pairs changed 0
fullSet, iter: 4 i:29, pairs changed 0
j not moving enough
fullSet, iter: 4 i:30, pairs changed 0
j not moving enough
fullSet, iter: 4 i:31, pairs changed 0
fullSet, iter: 4 i:32, pairs changed 0
j not moving enough
fullSet, iter: 4 i:33, pairs changed 0
fullSet, iter: 4 i:34, pairs changed 0
fullSet, iter: 4 i:35, pairs changed 0
j not moving enough
fullSet, iter: 4 i:36, pairs changed 0
fullSet, iter: 4 i:37, pairs changed 0
L==H
fullSet, iter: 4 i:38, pairs changed 0
fullSet, iter: 4 i:39, pairs changed 0
j not moving enough
fullSet, iter: 4 i:40, pairs changed 0
j not moving enough
fullSet, iter: 4 i:41, pairs changed 0
j not moving enough
fullSet, iter: 4 i:42, pairs changed 0
j not moving enough
fullSet, iter: 4 i:43, pairs changed 0
fullSet, iter: 4 i:44, pairs changed 0
j not moving enough
fullSet, iter: 4 i:45, pairs changed 0
L==H
fullSet, iter: 4 i:46, pairs changed 0
fullSet, iter: 4 i:47, pairs changed 0
j not moving enough
fullSet, iter: 4 i:48, pairs changed 0
fullSet, iter: 4 i:49, pairs changed 0
j not moving enough
fullSet, iter: 4 i:50, pairs changed 0
fullSet, iter: 4 i:51, pairs changed 0
j not moving enough
fullSet, iter: 4 i:52, pairs changed 0
L==H
fullSet, iter: 4 i:53, pairs changed 0
j not moving enough
fullSet, iter: 4 i:54, pairs changed 0
fullSet, iter: 4 i:55, pairs changed 0
j not moving enough
fullSet, iter: 4 i:56, pairs changed 0
fullSet, iter: 4 i:57, pairs changed 0
L==H
fullSet, iter: 4 i:58, pairs changed 0
fullSet, iter: 4 i:59, pairs changed 0
fullSet, iter: 4 i:60, pairs changed 0
j not moving enough
fullSet, iter: 4 i:61, pairs changed 0
j not moving enough
fullSet, iter: 4 i:62, pairs changed 0
fullSet, iter: 4 i:63, pairs changed 0
fullSet, iter: 4 i:64, pairs changed 0
L==H
fullSet, iter: 4 i:65, pairs changed 0
L==H
fullSet, iter: 4 i:66, pairs changed 0
fullSet, iter: 4 i:67, pairs changed 0
fullSet, iter: 4 i:68, pairs changed 0
fullSet, iter: 4 i:69, pairs changed 0
j not moving enough
fullSet, iter: 4 i:70, pairs changed 0
fullSet, iter: 4 i:71, pairs changed 0
fullSet, iter: 4 i:72, pairs changed 0
L==H
fullSet, iter: 4 i:73, pairs changed 0
j not moving enough
fullSet, iter: 4 i:74, pairs changed 0
fullSet, iter: 4 i:75, pairs changed 0
L==H
fullSet, iter: 4 i:76, pairs changed 0
fullSet, iter: 4 i:77, pairs changed 0
L==H
fullSet, iter: 4 i:78, pairs changed 0
fullSet, iter: 4 i:79, pairs changed 0
fullSet, iter: 4 i:80, pairs changed 0
L==H
fullSet, iter: 4 i:81, pairs changed 0
L==H
fullSet, iter: 4 i:82, pairs changed 0
fullSet, iter: 4 i:83, pairs changed 0
fullSet, iter: 4 i:84, pairs changed 0
L==H
fullSet, iter: 4 i:85, pairs changed 0
fullSet, iter: 4 i:86, pairs changed 0
L==H
fullSet, iter: 4 i:87, pairs changed 0
fullSet, iter: 4 i:88, pairs changed 0
fullSet, iter: 4 i:89, pairs changed 0
L==H
fullSet, iter: 4 i:90, pairs changed 0
L==H
fullSet, iter: 4 i:91, pairs changed 0
L==H
fullSet, iter: 4 i:92, pairs changed 0
fullSet, iter: 4 i:93, pairs changed 0
fullSet, iter: 4 i:94, pairs changed 0
fullSet, iter: 4 i:95, pairs changed 0
L==H
fullSet, iter: 4 i:96, pairs changed 0
fullSet, iter: 4 i:97, pairs changed 0
fullSet, iter: 4 i:98, pairs changed 0
fullSet, iter: 4 i:99, pairs changed 0
iteration number: 5
there are 30 Support Vectors
the training error rate is: 0.130000
the test error rate is: 0.150000

Process finished with exit code 0

 

ctrigger
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
支持向量机实例
12-05
对应于机器学习教程进行实战，通过代码的实现，理解机器学习中支持向量机的原理。
支持向量机SVM个人总结
weixin_44232687的博客
11-03 474
支持向量机（support vector machines,SVM）是一种二类分类模型。它的基本模型是定义在特征空间上的间隔最大的线性分类器，间隔最大使它有别于感知机；支持向量机还包括核技巧，这使它成为实质上的非线性分类器。支持向量机的学习策略就是间隔最大化。支持向量机的学习算法是求解凸二次规划的最优化算法。
机器学习之SVM支持向量机实例
qq_46556714的博客
04-30 434
SVM支持向量机实例 代码： import sys defaultencoding = 'utf-8' import numpy as np from scipy import io as spio from matplotlib import pyplot as plt from sklearn import svm def SVM(): data1 = spio.loadmat('data1.mat') X = data1['X'] y = data1['y'] y
sklearn机器学习支持向量机案例解析
01-22
通过对支持向量机的应用，解决两类经典问题：鸢尾花数据集分类及数字图像识别，并注释原理，帮助大家学习理解
支持向量机c++实现(转载)
同学醒醒放学了
09-14 2256
　　 　　支持向量机c++实现　　 　　参考博文地址https://blog.csdn.net/wz2671/article/details/78892902
支持向量机matlab代码程序-机器学习实战项目-支持向量机
01-04
支持向量机（Support Vector Machines, SVM）是一种广泛应用于分类和回归分析的监督学习算法。在机器学习领域，SVM以其高效性和泛化能力而备受推崇。在MATLAB环境中，我们可以利用内置的函数来实现SVM模型。 在提供...
机器学习——支持向量机SVM算法完整实战源码
05-16
基于Python和sklearn机器学习库实现的支持向量机算法使用的实战案例。 使用jupyter notebook环境开发。 支持向量机：支持向量机（Support Vector Machine, SVM）是一类按监督学习（supervised learning）方式对数据...
支持向量机matlab代码程序-机器学习实战项目
最新发布
06-20
在提供的压缩包中，"支持向量机matlab代码程序-机器学习实战项目-支持向量机.pdf"可能包含了详细的MATLAB代码示例和SVM理论解释。"说明文档.txt"可能对整个项目的实施步骤和注意事项进行了详细描述。阅读这些文档，...
python机器学习理论与实战（五）支持向量机
09-20
支持向量机（Support Vector Machine, SVM）是机器学习领域中一种重要的监督学习算法，尤其在分类和回归问题上表现出色。它基于间隔最大化（Margin Maximization）的概念，旨在找到一个能够将不同类别的样本分开的...
第6章支持向量机
weixin_41938314的博客
04-29 173
支持向量（support vector）就是离分隔超平面最近的点 SVM特点：几乎所有分类问题都可以解决，SVM本身是一个二类分类器，对多类问题应用SVM需要对代码做一些修改 序列最小优化算法（Sequential Minimal Optimization,SMO）算法： 支持向量机的学习问题可以形式化为求解凸二次规划问题。 SMO算法将原来的大问题不断分解为子问题并对子问题求解，进而达到...
《机器学习实战笔记--第一部分 分类算法：支持向量机 4》
qq_41635352的博客
05-12 210
SVM实战，手写识别问题之前使用KNN算法进行手写体的识别，需要保存全部的向量，使用SVM只需要保存支持向量即可，可以减少内存的占用。def img2vector(filename): returnVect = zeros((1,1024)) fr = open(filename) for i in range(32): lineStr = fr.readl...
【机器学习】支持向量机（7）——手写数字识别案例
Daycym的博客
08-02 3517
前言 之前我们用 kkk-近邻算法（kNNkNNkNN）实现了手写数字识别系统，博客链接 从上面的案例中，我们可知使用kNNkNNkNN方法的效果不错，但是需要保留所有的训练样本。而对于支持向量机来说，其需要保留的样本少了很多（只需要保留支持向量），但能获得可比的效果。 说明： 这里我们只考虑SVMSVMSVM的二分类问题，所以我们只使用了手写数字识别数据集中的1和9数据； ...
机器学习之支持向量机实战
Maple__Boy
09-11 664
相关的svm基本知识在前序章节中已经写的非常的详细，请读者在掌握svm基本知识之后在进行相关的代码练习。 实例代码一： #!/usr/bin/python # -*- coding:utf-8 -*- import numpy as np from sklearn import svm from sklearn.model_selection import train_test_split import matplotlib as mpl import matplotlib.pyplot as plt f
机器学习 支持向量机
Rouger7921的博客
12-26 695
提示：文章写完后，目录可以自动生成，如何生成可参考右边的帮助文档 文章目录 前言 一、pandas是什么？ 二、使用步骤 1.引入库 2.读入数据 总结 前言 提示：这里可以添加本文要记录的大概内容： 例如：随着人工智能的不断发展，机器学习这门技术也越来越重要，很多人都开启了学习机器学习，本文就介绍了机器学习的基础内容。 提示：以下是本篇文章正文内容，下面案例可供参考 一、pandas是什么？ 示例：pandas 是基于NumPy 的一种工具，该工具是为了解决数据分
【机器学习】支持向量机SVM及实例应用
热门推荐
gogo小Sa的专栏
05-29 7万+
【机器学习】支持向量机1.分类超平面与最大间隔       上图是一个关于机器学习算法的时间线来自于Eren Golge。可以看出SVM旺盛的生命力。实际上，即使是深度学习非常火热的今天，SVM依然盛行。在一些小样本分类问题上，SVM表现非常好，用深度学习模型可能反而会使问题变得更复杂。    下面开始介绍SVM--支持向量机。1.分类超平面与最大间隔    给定训练样本集 D ={ (X1 ,Y...
[机器学习实战] 支持向量机
Joker_Ha_Ha_Ha的博客
12-26 935
一、概述 支持向量机是常用的一类分类模型，它通过在两类数据之间寻找一个超平面来分隔不同类的数据。支持向量机最后对应一个二次规划模型（其最优化问题的对偶问题），可以使用通常二次规划求解方法求解，也可以使用Platt在1996年提出的SMO（Sequential Minimal Optimization），它通过一种启发式的误差修正方法求解。 二、算法介绍 支持向量机（support vector machines, SVM）是一种二分类模型，它的基本模型是定义在特征空间上的间隔最大的线性分类器，间隔.
python路径相关的函数os.listdir(dirname)：列出dirname下的目录和文件
xiaminli的博客
06-17 2576
http://www.educity.cn/wenda/354028.html Python 路径相关的函数 os.listdir(dirname)：列出dirname下的目录和文件 os.getcwd()：获得当前工作目录 os.curdir:返回当前目录（'.') os.chdir(dirname):改变工作目录到dirname os.path.isdir(
Python-listdir函数
疯狂的兔子
12-06 9271
listdir函数在模块os中，主要功能是可以列出给定目录的文件名。例如：from os import listdir trainingFileList = listdir('digits/trainingDigits') print(trainingFileList)>>['0_0.txt', '0_1.txt', '0_10.txt',..., '9_99.txt']
机器学习原理与实战 | SVM（支持向量机）实践
Clichong
09-13 524
文章目录1. 二维SVM分类例子2. 多维SVM分类例子2.1 高斯核函数2.2. 多项式核函数2.3 多项式 LinearSVC %matplotlib inline import matplotlib.pyplot as plt import numpy as np 1. 二维SVM分类例子 from sklearn.datasets import make_blobs X, y = make_blobs(n_samples=100, centers=2, ra

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值