《统计学习方法》学习方法（第六章）

最新推荐文章于 2021-07-13 11:40:43 发布

Hhhhuply

最新推荐文章于 2021-07-13 11:40:43 发布

阅读量407

点赞数

分类专栏：机器学习文章标签：学习笔记

本文链接：https://blog.csdn.net/Hhhhuply/article/details/88991711

版权

机器学习专栏收录该内容

11 篇文章 0 订阅

订阅专栏

逻辑斯蒂回归
源代码：https://github.com/fengdu78/lihang-code/blob/master/code/第6章逻辑斯谛回归(LogisticRegression)/LR.ipynb

LR是经典的分类方法
sigmoid函数 $\frac{1}{1+e^{-x}}$
线性回归 $wx =w_0*x_0 + w_1*x_1 + w_2*x_2 +...+w_n*x_n,(x_0=1)$ 结合
回归模型： $\frac{1}{1+e^{-wx}}$

class LogisticReressionClassifier:
    def __init__(self,max_iter=200,learning_rate=0.01):
        self.max_iter = max_iter
        self.learning_rate = learning_rate
        
    def sigmoid(self,x):
        return 1/(1+exp(-x))
    
    def data_matrix(self,X):
        data_mat = []
        for d in X:
            data_mat.append([1.0,*d])
        return data_mat
    
    def fit(self,X,y):
        #label = np.mat(y) np.mat()是将序列转为np的二维数组np.transpose()是将数组转置
        data_mat = self.data_matrix(X) # m*n
        self.weights = np.zeros((len(data_mat[0]),1),dtype=np.float32)
        
        for iter_ in range(self.max_iter):
            for i in range(len(X)):
                result = self.sigmoid(np.dot(data_mat[i],self.weights))
                error = y[i]-result
                #权重更新 梯度下降法
                self.weights += self.learning_rate * error * np.transpose([data_mat[i]])
        print('LogisticRegression Model(learning_rate={},max_iter={})'.format(self.learning_rate, self.max_iter))

	# def f(self, x):
	# return -(self.weights[0] + self.weights[1] * x) / self.weights[2]
        
    def score(self, X_test, y_test):
        right = 0
        X_test = self.data_matrix(X_test)
        for x, y in zip(X_test, y_test):
            result = np.dot(x, self.weights)
            if (result > 0 and y == 1) or (result < 0 and y == 0):
                right += 1
        return right / len(X_test)

使用了梯度下降算法来进行训练，根据以下公式：
$_wL(w)=X^T(y−result)$