第五章：（1）逻辑回归：分类算法

1.概述

1.1 名为“回归”的分类器

1.2 为什么需要逻辑回归

1.3 sklearn中的逻辑回归

2 linear_model.LogisticRegression （逻辑回归）

class sklearn.linear_model.LogisticRegression (penalty=’l2’, dual=False, tol=0.0001, C=1.0,fit_intercept=True, 
         intercept_scaling=1, class_weight=None, random_state=None, solver=’warn’, 
         max_iter=100,multi_class=’warn’, verbose=0, warm_start=False, n_jobs=None)

2.1 二元逻辑回归的损失函数

2.1.1 损失函数的概念与解惑

2.1.2【选学】二元逻辑回归损失函数的数学解释，公式推导与解惑

2.2 重要参数penalty & C (属性coef_)

2.2.1 正则化

正则化是用来防止模型过拟合的过程，常用的有L1正则化和L2正则化两种选项

from sklearn.linear_model import LogisticRegression as LR
from sklearn.datasets import load_breast_cancer
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

data=load_breast_cancer()
X=data.data
y=data.target

#实例化
lrl1=LR(penalty="l1",solver="liblinear",C=0.5,max_iter=1000)
lrl2=LR(penalty="l2",solver="liblinear",C=0.5,max_iter=1000)

#逻辑回归的重要属性coef_，查看每个特征所对应的参数
lrl1=lrl1.fit(X,y)
lrl1.coef_

lrl2 = lrl2.fit(X,y)
lrl2.coef_

l1 = []
l2 = []
l1test = []
l2test = []
Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=420)

for i in np.linspace(0.05,1,19):
    lrl1 = LR(penalty="l1",solver="liblinear",C=i,max_iter=1000)
    lrl2 = LR(penalty="l2",solver="liblinear",C=i,max_iter=1000)
    
    # accuracy_score（）分类正确率分数，函数返回一个分数，这个分数或是正确的比例
    lrl1 = lrl1.fit(Xtrain,Ytrain)   
    l1.append(accuracy_score(lrl1.predict(Xtrain),Ytrain))
    l1test.append(accuracy_score(lrl1.predict(Xtest),Ytest))
       
    lrl2 = lrl2.fit(Xtrain,Ytrain)
    l2.append(accuracy_score(lrl2.predict(Xtrain),Ytrain))
    l2test.append(accuracy_score(lrl2.predict(Xtest),Ytest))
    
graph = [l1,l2,l1test,l2test]
color = ["green","black","lightgreen","gray"]
label = ["L1","L2","L1test","L2test"]    
plt.figure(figsize=(6,6))
for i in range(len(graph)):
    plt.plot(np.linspace(0.05,1,19),graph[i],color[i],label=label[i])
plt.legend(loc=4) #图例的位置在哪里?4表示，右下角
plt.show()

2.2.2 逻辑回归中的特征工程