机器学习——梯度下降算法之回归预测

 

import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
from sklearn.linear_model import LinearRegression

[2]:

from sklearn.datasets import load_boston

[3]:

dir(load_boston())

E:\Anaconda\lib\site-packages\sklearn\utils\deprecation.py:87: FutureWarning: Function load_boston is deprecated; `load_boston` is deprecated in 1.0 and will be removed in 1.2.

    The Boston housing prices dataset has an ethical problem. You can refer to
    the documentation of this function for further details.

    The scikit-learn maintainers therefore strongly discourage the use of this
    dataset unless the purpose of the code is to study and educate about
    ethical issues in data science and machine learning.

    In this special case, you can fetch the dataset from the original
    source::

        import pandas as pd
        import numpy as np


        data_url = "http://lib.stat.cmu.edu/datasets/boston"
        raw_df = pd.read_csv(data_url, sep="\s+", skiprows=22, header=None)
        data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
        target = raw_df.values[1::2, 2]

    Alternative datasets include the California housing dataset (i.e.
    :func:`~sklearn.datasets.fetch_california_housing`) and the Ames housing
    dataset. You can load the datasets as follows::

        from sklearn.datasets import fetch_california_housing
        housing = fetch_california_housing()

    for the California housing dataset and::

        from sklearn.datasets import fetch_openml
        housing = fetch_openml(name="house_prices", as_frame=True)

    for the Ames housing dataset.
    
  warnings.warn(msg, category=FutureWarning)

[3]:

['DESCR', 'data', 'data_module', 'feature_names', 'filename', 'target']

[4]:

print(load_boston().DESCR)

.. _boston_dataset:

Boston house prices dataset
---------------------------

**Data Set Characteristics:**  

    :Number of Instances: 506 

    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of black people by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's

    :Missing Attribute Values: None

    :Creator: Harrison, D. and Rubinfeld, D.L.

This is a copy of UCI ML housing dataset.
Index of /ml/machine-learning-databases/housing


This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.

The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic
prices and the demand for clean air', J. Environ. Economics & Management,
vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics
...', Wiley, 1980.   N.B. Various transformations are used in the table on
pages 244-261 of the latter.

The Boston house-price data has been used in many machine learning papers that address regression
problems.   
     
.. topic:: References

   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.
   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.

[5]:

X = load_boston().data
y = load_boston().target
print(X.shape,y.shape)
df = pd.DataFrame(X, columns=load_boston().feature_names)
df.head()

(506, 13) (506,)

[5]:

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT
0	0.00632	18.0	2.31	0.0	0.538	6.575	65.2	4.0900	1.0	296.0	15.3	396.90	4.98
1	0.02731	0.0	7.07	0.0	0.469	6.421	78.9	4.9671	2.0	242.0	17.8	396.90	9.14
2	0.02729	0.0	7.07	0.0	0.469	7.185	61.1	4.9671	2.0	242.0	17.8	392.83	4.03
3	0.03237	0.0	2.18	0.0	0.458	6.998	45.8	6.0622	3.0	222.0	18.7	394.63	2.94
4	0.06905	0.0	2.18	0.0	0.458	7.147	54.2	6.0622	3.0	222.0	18.7	396.90	5.33

[6]:

plt.figure(figsize=(12,8))
sns.heatmap(df.corr(), annot=True)
plt.figure(figsize=(12,8))

[6]:

<Figure size 1200x800 with 0 Axes>

<Figure size 1200x800 with 0 Axes>

[7]:

df.corr()

[7]:

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT
CRIM	1.000000	-0.200469	0.406583	-0.055892	0.420972	-0.219247	0.352734	-0.379670	0.625505	0.582764	0.289946	-0.385064	0.455621
ZN	-0.200469	1.000000	-0.533828	-0.042697	-0.516604	0.311991	-0.569537	0.664408	-0.311948	-0.314563	-0.391679	0.175520	-0.412995
INDUS	0.406583	-0.533828	1.000000	0.062938	0.763651	-0.391676	0.644779	-0.708027	0.595129	0.720760	0.383248	-0.356977	0.603800
CHAS	-0.055892	-0.042697	0.062938	1.000000	0.091203	0.091251	0.086518	-0.099176	-0.007368	-0.035587	-0.121515	0.048788	-0.053929
NOX	0.420972	-0.516604	0.763651	0.091203	1.000000	-0.302188	0.731470	-0.769230	0.611441	0.668023	0.188933	-0.380051	0.590879
RM	-0.219247	0.311991	-0.391676	0.091251	-0.302188	1.000000	-0.240265	0.205246	-0.209847	-0.292048	-0.355501	0.128069	-0.613808
AGE	0.352734	-0.569537	0.644779	0.086518	0.731470	-0.240265	1.000000	-0.747881	0.456022	0.506456	0.261515	-0.273534	0.602339
DIS	-0.379670	0.664408	-0.708027	-0.099176	-0.769230	0.205246	-0.747881	1.000000	-0.494588	-0.534432	-0.232471	0.291512	-0.496996
RAD	0.625505	-0.311948	0.595129	-0.007368	0.611441	-0.209847	0.456022	-0.494588	1.000000	0.910228	0.464741	-0.444413	0.488676
TAX	0.582764	-0.314563	0.720760	-0.035587	0.668023	-0.292048	0.506456	-0.534432	0.910228	1.000000	0.460853	-0.441808	0.543993
PTRATIO	0.289946	-0.391679	0.383248	-0.121515	0.188933	-0.355501	0.261515	-0.232471	0.464741	0.460853	1.000000	-0.177383	0.374044
B	-0.385064	0.175520	-0.356977	0.048788	-0.380051	0.128069	-0.273534	0.291512	-0.444413	-0.441808	-0.177383	1.000000	-0.366087
LSTAT	0.455621	-0.412995	0.603800	-0.053929	0.590879	-0.613808	0.602339	-0.496996	0.488676	0.543993	0.374044	-0.366087	1.000000

[8]:

from sklearn.model_selection import train_test_split

[9]:

#种子 random_state为888，按照8:2划分数据
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=888)

[10]:

print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

(404, 13)
(102, 13)
(404,)
(102,)

[11]:

#构建并训练线性模型，约两行
linear_model=LinearRegression()
linear_model.fit(X_train,y_train)

[11]:

LinearRegression()

[12]:

coef = linear_model.coef_#回归系数
print(coef)

[-1.19007229e-01  3.64055815e-02  1.68552680e-02  2.29397031e+00
 -1.60706448e+01  3.72371469e+00  9.22765437e-03 -1.30674803e+00
  3.43072685e-01 -1.45830386e-02 -9.73486692e-01  7.89797436e-03
 -5.72555056e-01]

[13]:

linear_model.score(X_test,y_test)

[13]:

0.7558932220633297

[14]:

df_coef = pd.DataFrame()
df_coef['Title'] =df.columns
df_coef['Coef'] = coef
df_coef
​

[14]:

	Title	Coef
0	CRIM	-0.119007
1	ZN	0.036406
2	INDUS	0.016855
3	CHAS	2.293970
4	NOX	-16.070645
5	RM	3.723715
6	AGE	0.009228
7	DIS	-1.306748
8	RAD	0.343073
9	TAX	-0.014583
10	PTRATIO	-0.973487
11	B	0.007898
12	LSTAT	-0.572555

[15]:

#生成训练数据X_test的预测值
line_pre = linear_model.predict(X_test)

[16]:

hos_pre = pd.DataFrame()
hos_pre['Predict'] = line_pre
hos_pre['Truth'] = y_test
hos_pre.plot(figsize=(18,8))

[16]:

<AxesSubplot:>

完整代码

import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_boston
dir(load_boston())
print(load_boston().DESCR)
X = load_boston().data
y = load_boston().target
print(X.shape,y.shape)
df = pd.DataFrame(X, columns=load_boston().feature_names)
df.head()
plt.figure(figsize=(12,8))
sns.heatmap(df.corr(), annot=True)
plt.figure(figsize=(12,8))
df.corr()
from sklearn.model_selection import train_test_split
#种子 random_state为888，按照8:2划分数据
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=888)
print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)
#构建并训练线性模型，约两行
linear_model=LinearRegression()
linear_model.fit(X_train,y_train)                                                   
coef = linear_model.coef_#回归系数
print(coef)
linear_model.score(X_test,y_test)
df_coef = pd.DataFrame()
df_coef['Title'] =df.columns
df_coef['Coef'] = coef
df_coef
#生成训练数据X_test的预测值
line_pre = linear_model.predict(X_test)
hos_pre = pd.DataFrame()
hos_pre['Predict'] = line_pre
hos_pre['Truth'] = y_test
hos_pre.plot(figsize=(18,8))