逻辑回归（logistic回归）

最新推荐文章于 2024-07-24 19:11:18 发布

考试不挂科！

最新推荐文章于 2024-07-24 19:11:18 发布

阅读量1.4k

点赞数 44

文章标签：逻辑回归算法机器学习

本文链接：https://blog.csdn.net/m0_74242391/article/details/138920402

版权

逻辑回归基本概念

定义

Logistic回归是一种广义线性回归（generalized linear model），因此与多重线性回归分析有很多相同之处。它们的模型形式基本上相同，都具有 w’x+b，其中w和b是待求参数，其区别在于他们的因变量不同，多重线性回归直接将w’x+b作为因变量，即y =w’x+b，而logistic回归则通过函数L将w’x+b对应一个隐状态p，p =L(w’x+b),然后根据p 与1-p的大小决定因变量的值。逻辑回归假设数据服从伯努利分布，通过极大似然估计的方法，运用梯度下降来求解参数，来达到数据二分类的目的。

回归：假设现在有一些数据点，我们用一条直线对这些点进行拟合（该直线称为最佳拟合直线），这个拟合的过程就称为回归。
主要思想：根据现有数据对分类边界线建立回归公式，以此进行分类。
Logistic回归的一般过程
（1)收集数据：采用任意方法收集数据。
(2)准备数据：由于需要进行距离计算，因此要求数据类型为数值型。另外，结构化数据
格式则最佳。
(3）分析数据：采用任意方法对数据进行分析。
（4）训练算法：大部分时间将用于训练，训练的目的是为了找到最佳的分类回归系数。
（5)测试算法：一旦训练步骤完成，分类将会很快。
(6)使用算法：首先，我们需要输入一些数据，并将其转换成对应的结构化数值;
接着，基于训练好的回归系数就可以对这些数值进行简单的回归计算，判定它们属于
哪个类别；在这之后，我们就可以在输出的类别上做一些其他分析工作。

逻辑回归

sigmoid函数

若要处理的是二分类问题，我们期望的函数输出会是0或1，类似于单位阶跃函数，可是该函数是不连续的，不连续不可微。

因此我们换用Sigmoid函数，当x=0时，y为0；随着x的增大，y值趋近于1，随着x的减小，y趋近于0，当横坐标足够大时，Sigmoid函数就会看起来像一个阶跃函数。公式为：

$eq?%5Csigma%20%5Cleft%20%28%20z%20%5Cright%20%29%3D%5Cfrac%7B1%7D%7B1+e%5E%7B-z%7D%7D$

有python绘制sigmoid函数图像：

import matplotlib.pyplot as plt
import numpy as np
 
def sigmoid(z):
    return 1.0/(1.0+np.exp(-z))
 
z=np.arange(-6,6,0.05)
plt.plot(z,sigmoid(z))
plt.axvline(0.0,color='k')
plt.axhline(y=0.0,ls='dotted',color='k')
plt.axhline(y=1.0,ls='dotted',color='k')
plt.axhline(y=0.5,ls='dotted',color='k')
plt.yticks([0.0,0.5,1.0])
plt.ylim(-0.1,1.1)
plt.xlabel('z')
plt.ylabel('$\phi (z)$')
plt.show()

两种坐标尺度下的Sigmoid函数图。上图的横坐标为-5到5，这时的曲线变化较
为平滑；下图横坐标的尺度足够大，可以看到，在x=0点处Sigmoid函数看起来
很像阶跃函数

梯度上升算法

训练算法时采用了梯度上升算法，其思想是：要找到某函数的最大值，最好的方法就是沿着该函数的梯度方向探寻。如果梯度记为∇，则函数f(x,y)的梯度由下式表示：

如图，梯度上升算法到达每个点后都会重新估计移动的方向。从P0开始，计算完该点的梯度，函数就根据梯度移动到下一点P1。在P1点，梯度再次被重新计算，并沿新的梯度方向移动到P2。如此循环迭代，直到满足停止条件。迭代的过程中，梯度算子总是保证我们能选取到最佳的移动方向。

你最经常听到的应该是梯度下降算法，它与这里的梯度上升算法是一样的，只是公式中的
加法需要变成减法。因此，对应的公式可以写成：

梯度上升算法用来求函数的最大值，而梯度下降算法用来求函数的最小值。

python实现分类器的应用

训练集：

1.求回归函数

import numpy as np
#Logistic回归梯度上升优化算法
def loadDataSet():
    dataMat = []; labelMat = []
    fr = open('d:\222\test.txt')
    for line in fr.readlines():
        lineArr = line.strip().split()
        dataMat.append([1.0,float(lineArr[0]),float(lineArr[1])])
        labelMat.append(int(lineArr[2]))
    return dataMat,labelMat
def sigmoid(inX):
    return 1.0/(1+np.exp(-inX))
def gradAscent(dataMatIn,classLabels):
    dataMatrix = np.mat(dataMatIn)
    labelMat = np.mat(classLabels).transpose()
    m,n = np.shape(dataMatrix)
    alpha = 0.001
    maxCycles = 500
    weights = np.ones((n,1))
    for k in range(maxCycles):
        h = sigmoid(dataMatrix*weights)
        error = (labelMat - h)
        weights = weights + alpha * dataMatrix.transpose() * error ③
    return weights
dataArr,labelMat = loadDataSet()
weightsLogistic = gradAscent(dataArr,labelMat)
weightsLogistic

上面这块代码中梯度上升的③，其实是cross entropy损失函数对w求导得出来的，具体推导过程参见理论推导

matrix([[ 4.12414349],
        [ 0.48007329],
        [-0.6168482 ]])

2.分析数据，画出决策边界

设置sigmoid函数为0,0是两个分类的（类别1和类别0）的分界处

所以设定 $eq?0%3DW_%7B0%7DX_%7B0%7D+W_%7B1%7DX_%7B1%7D+W_%7B2%7DX_%7B2%7D$

为什么是0，这要看sigmoid函数的性质，sigmoid（x），x>=0，划为正类，反之为负类
所以这设置sigmoid函数变量x取0时，求出x1，x2也就是这两个feature之间的关系式，来画出分类边界。

from numpy import *

def plotBestFit(weights):
    import matplotlib.pyplot as plt
    dataMat,labelMat = loadDataSet()
    dataArr = array(dataMat)
    n = np.shape(dataArr)[0]
    xcord1 = []; ycord1 = []
    xcord2 = []; ycord2 = []
    for i in range(n):
        if int(labelMat[i]) == 1:
            xcord1.append(dataArr[i,1]); ycord1.append(dataArr[i,2])
        else:
            xcord2.append(dataArr[i,1]); ycord2.append(dataArr[i,2])
    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.scatter(xcord1,ycord1,s=30,c='red',marker = 's')
    ax.scatter(xcord2,ycord2,s=30,c='green')
    x = np.arange(-3.0,3.0,0.1)
    y = (-weights[0]-weights[1]*x)/weights[2]
    y = y.transpose()
#     print(x.shape)
#     print(y.shape)
    ax.plot(x,y)
    plt.xlabel('X1');plt.ylabel('X2');
    plt.show
plotBestFit(weightsLogistic)

这个分类器分类效果不错，从图上看只分错了几个点而已，但是回顾对算法的原理的解析，我们却发现这个这个方法需要相当大量的计算，因此下一步我们将对其进行改进，从而使它可以运用在我们现实生活中，处理例子复杂数量大的数据集。

3.训练算法，随机梯度上升

一种改进的方法是一次仅用一个样本点来更新回归系数，即随机梯度上升法。由于可以在新样本到来时对分类器进行增量式更新，因此随机梯度上升法是一个在线学习算法。

梯度上升伪代码：

每个回归系数初始化为1
重复R次：
计算整个数据集的梯度
使用alpha*gradient更新回归系数的向量
返回回归系数

随机梯度上升法可以写成如下伪代码：

所有回归系数初始化为1
对数据集中每个样本
计算该样本的梯度
　　使用alpha*gradient更新回顾系数值
返回回归系数值

随机梯度上升和梯度上升法在代码上的区别：

（1）后者的变量h和误差error都是向量。
（2）前者没有矩阵的转换过程，所有变量的数据类型都是Numpy数组。

#随机梯度上升算法
def stocGradAscent0(dataMatrix,classLabels):
    m,n = np.shape(dataMatrix)
    alpha = 0.1
    weights = np.ones(n)
    for i in range(m):
        h = sigmoid(sum(dataMatrix[i]*weights))
        error = classLabels[i] - h
        weights = weights + alpha * error * dataMatrix[i]
    return weights
from numpy import *
dataArr,labelMat = loadDataSet()
weights = stocGradAscent0(array(dataArr),labelMat)
plotBestFit(weights)

4.改进随机梯度上升

判断优化算法的优劣的可靠方法是看它是否收敛，也就是看参数是否稳定。

有三点改进：
（1）alpha每次迭代的时候都会调整。不会减小到0，满足某个条件时也不是严格下降的。
（2）随机选取样本来更新回归系数，这种做法会减少周期性波动。
（3）增加了迭代参数的设置。默认为150次，如果给定，将会按照新的参数值进行迭代。

#改进的随机梯度上升算法
def stocGradAscent1(dataMatrix,classLabels,numIter=150):
    m,n = np.shape(dataMatrix)
    weights = np.ones(n)
    for j in range(numIter):
        dataIndex = list(range(m))
        for i in range(m):
            alpha = 4/(1.0+j+i)+0.01
            randIndex = int(random.uniform(0,len(dataIndex)))
            h = sigmoid(sum(dataMatrix[randIndex]*weights))
            error = classLabels[randIndex] - h
            weights = weights + alpha * error * dataMatrix[randIndex]
            del(dataIndex[randIndex])
    return weights
dataArr,labelMat = loadDataSet()
weights = stocGradAscent1(array(dataArr),labelMat)
plotBestFit(weights)

实现例子----从疝气病症预测病马的死亡率

1.准备数据

测试集：

训练集：

数据包含了368个样本和28个特征。

疝气病是描述马胃肠痛的术语。这种病不一定源自马的肠胃问题，其他问题也可能引发马疝病。

该数据集中包含了医院检测马疝病的一些指标，有的指标比较主观，有的指标难以测量，例如马的疼痛级别。

如果在测试数据集中发现了一条数据的类别标记已经缺失，那么简单的做法就是将该条数据丢弃。

2.训练算法，使用优化算法，找到最佳系数

from numpy import *

def sigmoid(inX):
    return 1.0/(1+exp(-inX))

def stoGradAscent1(dataMatrix, classLabels, numIter = 150):
    m,n = shape(dataMatrix)
    weights = ones(n)
    for j in range (numIter):
        dataIndex = range(m)
        for i in range(m):
            alpha = 4 / (1.0 + j + i) + 0.01
            randIndex = int(random.uniform(0, len(dataIndex)))
            h = sigmoid(sum(dataMatrix[randIndex] * weights))
            error = classLabels[randIndex] - h
            weights = weights + alpha * error * dataMatrix[randIndex]
            del (list(dataIndex)[randIndex])
    return weights

3.测试算法，用Logistic回归进行分类

def classifyVector(inX, weights):
    prob = sigmoid(sum(inX * weights))
    #大于0.5 返回 1；否则返回0
    if prob > 0.5:
        return 1.0
    else:
        return 0.0

def colicTest():
    frTrain = open('HorseColicTraining.txt')
    frTest = open('HorseColicTest.txt')
    trainingSet = []
    trainingLabels = []
    
    for line in frTrain.readlines():
        currLine = line.strip().split('\t') #分割
        lineArr = []
        for i in range(21):
            lineArr.append(float(currLine[i]))
        #存入训练样本特征
        trainingSet.append(lineArr)
        #存入训练样本标签
        trainingLabels.append(float(currLine[21]))
    #使用改进后的随机梯度下降算法得到回归系数
    trainingWeights = stoGradAscent1(array(trainingSet), trainingLabels, 500)

    #统计测试集预测错误样本数量和样本总数
    errorCount = 0
    numTestVec = 0.0
    for line in frTest.readlines():
        #循环一次，样本数加1
        numTestVec += 1.0
        currLine = line.strip().split('\t') #分割
        lineArr = []
        for i in range(21):
            lineArr.append(float(currLine[i]))
        
        # 利用分类预测函数对该样本进行预测，并与样本标签进行比较
        if int(classifyVector(array(lineArr), trainingWeights)) != int(currLine[21]):
            errorCount += 1
    #计算错误率
    errorRate = (float(errorCount) / numTestVec)
    print('the error rate of this test is : %f' % errorRate)
    return errorRate

#调用colicTest10次，求平均值
def multiTest():
    numTests = 10
    errorSum = 0.0
    for k in range(numTests):
        errorSum += colicTest()
    print('after %d iterations the average error rete is : %f ' % (numTests,errorSum / float(numTests)))

4.结果

<ipython-input-14-ee3abccbfdd7>:2: RuntimeWarning: overflow encountered in exp
  return 1.0/(1+np.exp(-inX))


the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.507463
the error rate of this test is :0.313433
the error rate of this test is :0.477612
the error rate of this test is :0.328358
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.507463
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.462687
the error rate of this test is :0.477612
the error rate of this test is :0.268657
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.432836
the error rate of this test is :0.283582
the error rate of this test is :0.283582
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.328358
the error rate of this test is :0.447761
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.417910
the error rate of this test is :0.313433
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.313433
the error rate of this test is :0.388060
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.268657
the error rate of this test is :0.328358
the error rate of this test is :0.477612
the error rate of this test is :0.358209
the error rate of this test is :0.462687
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.447761
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.462687
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.537313
the error rate of this test is :0.313433
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.462687
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.432836
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.328358
the error rate of this test is :0.253731
the error rate of this test is :0.417910
the error rate of this test is :0.313433
the error rate of this test is :0.388060
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.268657
the error rate of this test is :0.283582
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.432836
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.552239
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.343284
the error rate of this test is :0.462687
the error rate of this test is :0.402985
the error rate of this test is :0.447761
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.447761
the error rate of this test is :0.313433
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.313433
the error rate of this test is :0.447761
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.238806
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.343284
the error rate of this test is :0.447761
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.328358
the error rate of this test is :0.238806
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.283582
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.477612
the error rate of this test is :0.343284
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.447761
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.432836
the error rate of this test is :0.477612
the error rate of this test is :0.373134
the error rate of this test is :0.477612
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.462687
the error rate of this test is :0.402985
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.328358
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.268657
the error rate of this test is :0.388060
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.253731
the error rate of this test is :0.373134
the error rate of this test is :0.253731
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.208955
the error rate of this test is :0.268657
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.432836
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.253731
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.432836
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.537313
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.447761
the error rate of this test is :0.402985
the error rate of this test is :0.328358
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.417910
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.268657
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.507463
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.462687
the error rate of this test is :0.447761
the error rate of this test is :0.283582
the error rate of this test is :0.238806
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.432836
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.447761
the error rate of this test is :0.552239
the error rate of this test is :0.268657
the error rate of this test is :0.477612
the error rate of this test is :0.432836
the error rate of this test is :0.298507
the error rate of this test is :0.432836
the error rate of this test is :0.313433
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.537313
the error rate of this test is :0.223881
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.223881
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.402985
the error rate of this test is :0.328358
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.462687
the error rate of this test is :0.328358
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.238806
the error rate of this test is :0.268657
the error rate of this test is :0.388060
the error rate of this test is :0.522388
the error rate of this test is :0.507463
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.298507
the error rate of this test is :0.343284
the error rate of this test is :0.477612
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.388060
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.462687
the error rate of this test is :0.447761
the error rate of this test is :0.283582
the error rate of this test is :0.313433
the error rate of this test is :0.417910
the error rate of this test is :0.462687
the error rate of this test is :0.417910
the error rate of this test is :0.432836
the error rate of this test is :0.432836
the error rate of this test is :0.522388
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.253731
the error rate of this test is :0.223881
the error rate of this test is :0.432836
the error rate of this test is :0.462687
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.238806
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.432836
the error rate of this test is :0.268657
the error rate of this test is :0.402985
the error rate of this test is :0.268657
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.238806
the error rate of this test is :0.447761
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.253731
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.253731
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.238806
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.253731
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.328358
the error rate of this test is :0.298507
the error rate of this test is :0.223881
the error rate of this test is :0.388060
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.268657
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.298507
the error rate of this test is :0.343284
the error rate of this test is :0.283582
the error rate of this test is :0.626866
the error rate of this test is :0.402985
the error rate of this test is :0.432836
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.432836
the error rate of this test is :0.432836
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.238806
the error rate of this test is :0.373134
the error rate of this test is :0.417910
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.447761
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.313433
the error rate of this test is :0.462687
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.268657
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.432836
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.522388
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.462687
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.313433
the error rate of this test is :0.238806
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.238806
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.402985
the error rate of this test is :0.283582
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.268657
the error rate of this test is :0.432836
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.462687
the error rate of this test is :0.298507
the error rate of this test is :0.597015
the error rate of this test is :0.522388
the error rate of this test is :0.343284
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.462687
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.283582
the error rate of this test is :0.313433
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.462687
the error rate of this test is :0.462687
the error rate of this test is :0.388060
the error rate of this test is :0.313433
the error rate of this test is :0.268657
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.238806
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.328358
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.462687
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.492537
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.238806
the error rate of this test is :0.328358
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.373134
the error rate of this test is :0.238806
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.238806
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.462687
the error rate of this test is :0.313433
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.298507
the error rate of this test is :0.253731
the error rate of this test is :0.432836
the error rate of this test is :0.328358
the error rate of this test is :0.432836
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.328358
the error rate of this test is :0.402985
the error rate of this test is :0.462687
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.432836
the error rate of this test is :0.328358
the error rate of this test is :0.283582
the error rate of this test is :0.462687
the error rate of this test is :0.283582
the error rate of this test is :0.238806
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.492537
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.253731
the error rate of this test is :0.313433
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.238806
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.298507
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.313433
the error rate of this test is :0.283582
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.328358
the error rate of this test is :0.223881
the error rate of this test is :0.477612
the error rate of this test is :0.268657
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.417910
the error rate of this test is :0.492537
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.328358
the error rate of this test is :0.328358
the error rate of this test is :0.388060
the error rate of this test is :0.253731
the error rate of this test is :0.328358
the error rate of this test is :0.417910
the error rate of this test is :0.462687
the error rate of this test is :0.373134
the error rate of this test is :0.447761
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.388060
the error rate of this test is :0.298507
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.447761
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.358209
the error rate of this test is :0.477612
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.402985
the error rate of this test is :0.328358
the error rate of this test is :0.373134
the error rate of this test is :0.402985
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.268657
the error rate of this test is :0.507463
the error rate of this test is :0.507463
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.328358
the error rate of this test is :0.268657
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.268657
the error rate of this test is :0.298507
the error rate of this test is :0.432836
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.343284
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.238806
the error rate of this test is :0.328358
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.313433
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.432836
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.298507
the error rate of this test is :0.417910
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.343284
the error rate of this test is :0.238806
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.477612
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.328358
the error rate of this test is :0.388060
the error rate of this test is :0.537313
the error rate of this test is :0.358209
the error rate of this test is :0.402985
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.268657
the error rate of this test is :0.373134
the error rate of this test is :0.373134
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.447761
the error rate of this test is :0.328358
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.388060
the error rate of this test is :0.313433
the error rate of this test is :0.298507
the error rate of this test is :0.328358
the error rate of this test is :0.402985
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.432836
the error rate of this test is :0.417910
the error rate of this test is :0.298507
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.373134
the error rate of this test is :0.298507
the error rate of this test is :0.388060
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.313433
the error rate of this test is :0.358209
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.492537
the error rate of this test is :0.313433
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.328358
the error rate of this test is :0.417910
the error rate of this test is :0.343284
the error rate of this test is :0.358209
the error rate of this test is :0.417910
the error rate of this test is :0.268657
the error rate of this test is :0.268657
the error rate of this test is :0.492537
the error rate of this test is :0.328358
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.477612
the error rate of this test is :0.343284
the error rate of this test is :0.238806
the error rate of this test is :0.388060
the error rate of this test is :0.268657
the error rate of this test is :0.417910
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.537313
the error rate of this test is :0.432836
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.268657
the error rate of this test is :0.358209
the error rate of this test is :0.283582
the error rate of this test is :0.447761
the error rate of this test is :0.447761
the error rate of this test is :0.283582
the error rate of this test is :0.343284
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.402985
the error rate of this test is :0.417910
the error rate of this test is :0.283582
the error rate of this test is :0.358209
the error rate of this test is :0.298507
the error rate of this test is :0.343284
the error rate of this test is :0.313433
the error rate of this test is :0.432836
the error rate of this test is :0.328358
the error rate of this test is :0.268657
the error rate of this test is :0.402985
the error rate of this test is :0.358209
the error rate of this test is :0.432836
the error rate of this test is :0.373134
the error rate of this test is :0.388060
the error rate of this test is :0.447761
the error rate of this test is :0.358209
the error rate of this test is :0.373134
the error rate of this test is :0.343284
the error rate of this test is :0.373134
the error rate of this test is :0.283582
the error rate of this test is :0.402985
the error rate of this test is :0.402985
the error rate of this test is :0.388060
the error rate of this test is :0.388060
the error rate of this test is :0.238806
the error rate of this test is :0.477612
the error rate of this test is :0.417910
the error rate of this test is :0.417910
after 1000 iterations the average error rate is:0.361343

平均值为36%。因为数据集本身有30%的数据缺失。后续可以使用梯度上升法进一步改进。

逻辑回归小结

logistic的优缺点

逻辑回归，作为一种统计学习方法，在机器学习领域占据着举足轻重的地位。它主要用于解决二分类问题，通过使用逻辑函数（通常是sigmoid函数）来预测某个结果发生的概率。具体来说，逻辑回归的优缺点如下：

优点
1. 训练速度较快：逻辑回归的训练速度相对较快，因为计算量仅与特征的数目相关。
2. 简单易理解：模型的可解释性非常好，可以通过特征的权重直接观察不同特征对结果的影响。
3. 适合二分类问题：逻辑回归不需要缩放输入特征，适合处理二分类数据。
4. 内存资源占用小：由于只需存储各维度的特征值，因此内存资源占用较小。
5. 直接对分类可能性进行建模：无需事先假设数据分布，避免了假设分布不准确所带来的问题。
6. 以概率形式输出：逻辑回归以概率的形式输出，对于利用概率辅助决策的任务非常有用。
7. 具有很好的数学性质：对率函数任意阶可导，许多现有的数值优化算法都可以用来求最优解。

缺点
1. 不能解决非线性问题：逻辑回归无法处理非线性关系的数据，因为它的决策面是线性的。
2. 对多重共线性数据敏感：当数据中存在高度相关的变量时，逻辑回归的性能可能会受到影响。
3. 难以处理数据不平衡问题：在数据集类别分布不均匀的情况下，逻辑回归的表现可能不佳。
4. 准确率有限：由于模型形式的简单性，逻辑回归可能无法很好地拟合数据的真实分布。
5. 无法筛选特征：逻辑回归本身不提供特征选择的功能，有时需要借助其他模型如GBDT来筛选特征。
6. 对异常值敏感：逻辑回归对异常值比较敏感，容易受到干扰。
7. 不适用于多分类问题：逻辑回归主要针对二分类问题，对于多分类问题则不太适用。

总的来说，逻辑回归因其简洁性和高效性而广泛应用于各种领域，包括医疗、金融和电商等。然而，它也有一定的局限性，特别是在处理非线性问题和数据不平衡问题上。在选择模型时，应根据具体问题的特点和数据情况来决定是否使用逻辑回归。

考试不挂科！

关注

44
点赞
踩
45

收藏

觉得还不错? 一键收藏
0
评论
逻辑回归（logistic回归）

Logistic回归是一种广义线性回归（generalized linear model），因此与多重线性回归分析有很多相同之处。它们的模型形式基本上相同，都具有 w’x+b，其中w和b是待求参数，其区别在于他们的因变量不同，多重线性回归直接将w’x+b作为因变量，即y =w’x+b，而logistic回归则通过函数L将w’x+b对应一个隐状态p，p =L(w’x+b),然后根据p 与1-p的大小决定因变量的值。
复制链接

扫一扫