机器学习Sklearn实战——其他线性模型

其他线性模型

岭回归

 

相比于传统的ols，在梯度下降的时候多了一项-learning_rate*2aw。无论w是正还是负，都让结果的绝对值变小（让系数变小，防止过拟合）

正则化 

欠拟合：系数少，系数小简单

过拟合：系数多，系数大复杂

 就是learning_rate一般取0.01 

lasso

弹性网络

多任务套索

多任务弹性网 

最小角度回归

import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn import datasets 
# cv交叉验证
from sklearn.linear_model import LinearRegression,Ridge,Lasso,ElasticNet,ElasticNetCV,LassoCV   

diabetes = datasets.load_diabetes()
X = diabetes["data"]
y = diabetes["target"]
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.15)

#使用线性回归
lr = LinearRegression()
lr.fit(X_train,y_train)
lr.score(X_test,y_test)
y_ = lr.predict(X_test)
mean_squared_error(y_test,y_)

结果：

0.4954938521776219
3052.1767800521984

####使用岭回归
ridge = Ridge(alpha=0.1)
ridge.fit(X_train,y_train)
y_ = ridge.predict(X_test)

print(ridge.score(X_test,y_test))
mean_squared_error(y_,y_test)

结果： 

0.4853455916799041
3113.5720379350823

 数据不存在共线性的问题（官方提供的数据），因此线性回归的效果比岭回归的效果要好

####使用ridgeCV,找最佳alpha
ridgeCV = RidgeCV(alphas=np.linspace(0.01,5,50),scoring = "r2")
ridgeCV.fit(X_train,y_train)
y_ = ridgeCV.predict(X_test)

print(ridge.score(X_test,y_test))
mean_squared_error(y_,y_test)

结果：

0.4853455916799041
3117.234991068462

ridgeCV.alpha_

结果：

0.11183673469387755

print(np.linspace(0.01,5,50))   #等差数列
print(np.logspace(0.01,5,50))   #等比数列  数字比较小的时候

结果：

[0.01       0.11183673 0.21367347 0.3155102  0.41734694 0.51918367
 0.62102041 0.72285714 0.82469388 0.92653061 1.02836735 1.13020408
 1.23204082 1.33387755 1.43571429 1.53755102 1.63938776 1.74122449
 1.84306122 1.94489796 2.04673469 2.14857143 2.25040816 2.3522449
 2.45408163 2.55591837 2.6577551  2.75959184 2.86142857 2.96326531
 3.06510204 3.16693878 3.26877551 3.37061224 3.47244898 3.57428571
 3.67612245 3.77795918 3.87979592 3.98163265 4.08346939 4.18530612
 4.28714286 4.38897959 4.49081633 4.59265306 4.6944898  4.79632653
 4.89816327 5.        ]
[1.02329299e+00 1.29370940e+00 1.63558632e+00 2.06780797e+00
 2.61424893e+00 3.30509292e+00 4.17850002e+00 5.28271453e+00
 6.67872986e+00 8.44365757e+00 1.06749868e+01 1.34959693e+01
 1.70624274e+01 2.15713612e+01 2.72718303e+01 3.44787109e+01
 4.35900889e+01 5.51092486e+01 6.96724728e+01 8.80841888e+01
 1.11361403e+02 1.40789877e+02 1.77995148e+02 2.25032320e+02
 2.84499582e+02 3.59681721e+02 4.54731565e+02 5.74899375e+02
 7.26822849e+02 9.18893768e+02 1.16172154e+03 1.46871921e+03
 1.85684439e+03 2.34753591e+03 2.96789806e+03 3.75219771e+03
 4.74375716e+03 5.99734709e+03 7.58221192e+03 9.58589468e+03
 1.21190726e+04 1.53216706e+04 1.93705904e+04 2.44894816e+04
 3.09610960e+04 3.91429057e+04 4.94868483e+04 6.25642915e+04
 7.90975925e+04 1.00000000e+05]