Untitled-0720记录一次机器学习完整项目

最新推荐文章于 2022-08-08 23:30:50 发布
最新推荐文章于 2022-08-08 23:30:50 发布
阅读量453 收藏 5
点赞数 1
分类专栏： 机器学习 项目 文章标签： 机器学习
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_45536936/article/details/118969413
版权

文章目录

#py2和py3的有效切换
from __future__ import division, print_function, unicode_literals

import numpy as np
import os

请添加图片描述

前言

在进行完整的机器学习项目时，需要完成如下的步骤

  • 了解完整项目
  • 获取数据
  • 发现可视化数据，发现规律
  • 找到合适的机器学习算法，准备相应的数据
  • 选择模型，进行训练
  • 微调模型
  • 给出解决方案
  • 部署、监控、维护

首先认识数据

  • 模型目标：利用加州普查数据、建立一个加州房价预测模型
  • 数据内容：每个街区组的人口、收入中位数、房价中位数等指标
  • 划定问题：建立模型不是最终目标，预测房价是目标，预测的房价会进一步用于下一步的投资模型
  • 确定参考性能P：人工计算的误差率大概有15%。
  • 梳理问题：类型是监督学习，进行的是多变量回归问题，目标是预测
  • 选择性能指标：回归问题的典型指标是均方根误差（RMSE）
    R M S E    =    1 m ∑ i = 1 m ( h ( x ( i ) ) − y ( i ) ) 2 RMSE\,\,=\,\,\sqrt{\frac{1}{m}\sum_{i=1}^m{\left( h\left( x^{\left( i \right)} \right) -y^{\left( i \right)} \right) ^2}} RMSE=m1​∑i=1m​(h(x(i))−y(i))2 ​
  • 假设存在许多异常的街区。此时，你可能需要使用平均绝对误差
    M A E ( X , h ) = 1 m ∑ i = 1 m ∣ h ( x ( i ) ) − y ( i ) ∣ MAE\left( X,h \right) =\frac{1}{m}\sum_{i=1}^m{|h\left( x^{\left( i \right)} \right) -y^{\left( i \right)}|} MAE(X,h)=m1​∑i=1m​∣h(x(i))−y(i)∣
np.random.seed(42)

%matplotlib inline

import matplotlib as mpl
import matplotlib.pyplot as plt
mpl.rc("axes",labelsize=14)
mpl.rc('xtick',labelsize=12)
mpl.rc("ytick",labelsize=12)


#建立存储文件位置
ROOT ='.'
DATE ='0720'
IMAGE = os.path.join(ROOT,'images',DATE)
os.makedirs(IMAGE,exist_ok  =True)

# 建立保存图位置
def save_fig(fig_id, tight_layout=True, fig_extension="png", resolution=300):
    path = os.path.join(IMAGE, fig_id + "." + fig_extension)
    print("Saving figure", fig_id)
    if tight_layout:
        plt.tight_layout()
    plt.savefig(path, format=fig_extension, dpi=resolution)

GET DATA

import pandas as pd
import os
import tarfile

housing = pd.read_csv('./housing.csv')
housing.head()
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_valueocean_proximity
0-122.2337.8841.0880.0129.0322.0126.08.3252452600.0NEAR BAY
1-122.2237.8621.07099.01106.02401.01138.08.3014358500.0NEAR BAY
2-122.2437.8552.01467.0190.0496.0177.07.2574352100.0NEAR BAY
3-122.2537.8552.01274.0235.0558.0219.05.6431341300.0NEAR BAY
4-122.2537.8552.01627.0280.0565.0259.03.8462342200.0NEAR BAY

查看数据的具体信息

housing.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 20640 entries, 0 to 20639
Data columns (total 10 columns):
 #   Column              Non-Null Count  Dtype  
---  ------              --------------  -----  
 0   longitude           20640 non-null  float64
 1   latitude            20640 non-null  float64
 2   housing_median_age  20640 non-null  float64
 3   total_rooms         20640 non-null  float64
 4   total_bedrooms      20433 non-null  float64
 5   population          20640 non-null  float64
 6   households          20640 non-null  float64
 7   median_income       20640 non-null  float64
 8   median_house_value  20640 non-null  float64
 9   ocean_proximity     20640 non-null  object 
dtypes: float64(9), object(1)
memory usage: 1.6+ MB
  • 0-1 longitude&latitude
  • 2- median age 房龄中位数
  • 3- 房子数
  • 4- 卧室数
  • 5- 人口
  • 6- 户口数
  • 7- 收入中位数
  • 8- 房价中位数
  • 9- 海景

梳理标量数据

housing['ocean_proximity'].value_counts()

<1H OCEAN     9136
INLAND        6551
NEAR OCEAN    2658
NEAR BAY      2290
ISLAND           5
Name: ocean_proximity, dtype: int64
  • 内陆
  • 近海
  • 近港
  • 岛屿

描述性统计

housing.describe()
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
count20640.00000020640.00000020640.00000020640.00000020433.00000020640.00000020640.00000020640.00000020640.000000
mean-119.56970435.63186128.6394862635.763081537.8705531425.476744499.5396803.870671206855.816909
std2.0035322.13595212.5855582181.615252421.3850701132.462122382.3297531.899822115395.615874
min-124.35000032.5400001.0000002.0000001.0000003.0000001.0000000.49990014999.000000
25%-121.80000033.93000018.0000001447.750000296.000000787.000000280.0000002.563400119600.000000
50%-118.49000034.26000029.0000002127.000000435.0000001166.000000409.0000003.534800179700.000000
75%-118.01000037.71000037.0000003148.000000647.0000001725.000000605.0000004.743250264725.000000
max-114.31000041.95000052.00000039320.0000006445.00000035682.0000006082.00000015.000100500001.000000
%matplotlib inline
import matplotlib.pyplot as plt
housing.hist(bins=50, figsize=(20,15))
save_fig("attribute_histogram_plots")
plt.show()

Saving figure attribute_histogram_plots

请添加图片描述

np.random.seed(42)

训练数据：划分

from sklearn.model_selection import train_test_split
train_set, test_set = train_test_split(housing, test_size=0.2, random_state=42)
#test_size=0.2 测试集占20%

####################![请添加图片描述](https://csdn-img-blog.oss-cn-beijing.aliyuncs.com/cb73e934f4cb449291811189491fcb57.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTUzNjkzNg==,size_16,color_FFFFFF,t_70)
#############################################################################
# train_set, test_set = train_test_split(housing, test_size=0.2, random_state=42)

#################################################################################################
train_set.head(5)
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_valueocean_proximity
14196-117.0332.7133.03126.0627.02300.0623.03.2596103000.0NEAR OCEAN
8267-118.1633.7749.03382.0787.01314.0756.03.8125382100.0NEAR OCEAN
17445-120.4834.664.01897.0331.0915.0336.04.1563172600.0NEAR OCEAN
14265-117.1132.6936.01421.0367.01418.0355.01.942593400.0NEAR OCEAN
2271-119.8036.7843.02382.0431.0874.0380.03.554296500.0INLAND
test_set.head()
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_valueocean_proximity
20046-119.0136.0625.01505.0NaN1392.0359.01.681247700.0INLAND
3024-119.4635.1430.02943.0NaN1565.0584.02.531345800.0INLAND
15663-122.4437.8052.03830.0NaN1310.0963.03.4801500001.0NEAR BAY
20484-118.7234.2817.03051.0NaN1705.0495.05.7376218600.0<1H OCEAN
9814-121.9336.6234.02351.0NaN1063.0428.03.7250278000.0NEAR OCEAN
housing["median_income"].hist()

<AxesSubplot:>

请添加图片描述

housing['income_cat']= pd.cut(housing['median_income'],bins=[0., 1.5, 3.0, 4.5, 6., np.inf],labels=[1, 2, 3, 4, 5])


housing['income_cat'].value_counts()

3    7236
2    6581
4    3639
5    2362
1     822
Name: income_cat, dtype: int64

housing['income_cat'].hist()

<AxesSubplot:>

请添加图片描述

https://blog.csdn.net/Cicome/article/details/79153268

使用StratifiedShuffleSplit进行分层采样

from sklearn.model_selection import StratifiedShuffleSplit

split = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=42)
for train_index, test_index in split.split(housing, housing["income_cat"]):
    strat_train_set = housing.loc[train_index]
    strat_test_set = housing.loc[test_index]

###########################################################################
train_index, test_index in split.split(housing, housing["income_cat"])
print(len(train_index),len(test_index))

16512 4128


D:\anaconda\lib\site-packages\ipykernel_launcher.py:2: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  
D:\anaconda\lib\site-packages\ipykernel_launcher.py:2: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

strat_test_set["income_cat"].value_counts() / len(strat_test_set)

3    0.350533
2    0.318798
4    0.176357
5    0.114583
1    0.039729
Name: income_cat, dtype: float64

for set_ in (strat_train_set, strat_test_set):
    set_.drop("income_cat", axis=1, inplace=True)

数据探索

housing = strat_train_set.copy()

housing.plot(kind="scatter", x="longitude", y="latitude",alpha=0.2)
save_fig("bad_visualization_plot")

Saving figure bad_visualization_plot

请添加图片描述

housing.plot(kind="scatter", x="longitude", y="latitude", alpha=0.4,
    s=housing["population"]/100, label="population", figsize=(10,7),
    c="median_house_value", cmap=plt.get_cmap("jet"), colorbar=True,
    sharex=False)
plt.legend()
save_fig("housing_prices_scatterplot")

Saving figure housing_prices_scatterplot

请添加图片描述

import os
import tarfile
import urllib.request

PROJECT_ROOT_DIR = "."

images_path = os.path.join(PROJECT_ROOT_DIR, "images", "end_to_end_project")
os.makedirs(images_path, exist_ok=True)
DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml/master/"
filename = "california.png"
print("Downloading", filename)
url = DOWNLOAD_ROOT + "images/end_to_end_project/" + filename
urllib.request.urlretrieve(url, os.path.join(images_path, filename))

Downloading california.png





('.\\images\\end_to_end_project\\california.png',
 <http.client.HTTPMessage at 0x22a301f6648>)

import matplotlib.image as mpimg

california_img=mpimg.imread(PROJECT_ROOT_DIR + '/images/end_to_end_project/california.png')
ax = housing.plot(kind="scatter", x="longitude", y="latitude", figsize=(10,7),
                       s=housing['population']/100, label="Population",
                       c="median_house_value", cmap=plt.get_cmap("jet"),
                       colorbar=False, alpha=0.4,
                      )
plt.imshow(california_img, extent=[-124.55, -113.80, 32.45, 42.05], alpha=0.5,
           cmap=plt.get_cmap("jet"))
plt.ylabel("Latitude", fontsize=14)
plt.xlabel("Longitude", fontsize=14)


prices = housing["median_house_value"]
tick_values = np.linspace(prices.min(), prices.max(), 11)
cbar = plt.colorbar()
cbar.ax.set_yticklabels(["$%dk"%(round(v/1000)) for v in tick_values], fontsize=14)
cbar.set_label('Median House Value', fontsize=16)

plt.legend(fontsize=16)
save_fig("california_housing_prices_plot")
plt.show()

D:\anaconda\lib\site-packages\ipykernel_launcher.py:18: UserWarning: FixedFormatter should only be used together with FixedLocator


Saving figure california_housing_prices_plot

请添加图片描述

相关系数

corr_matrix = housing.corr()

#每个属性和房价中位数的关联度
corr_matrix["median_house_value"].sort_values(ascending=False)

median_house_value    1.000000
median_income         0.688075
total_rooms           0.134153
housing_median_age    0.105623
households            0.065843
total_bedrooms        0.049686
population           -0.024650
longitude            -0.045967
latitude             -0.144160
Name: median_house_value, dtype: float64

from pandas.plotting import scatter_matrix


attributes = ["median_house_value", "median_income", "total_rooms",
              "housing_median_age"]
scatter_matrix(housing[attributes], figsize=(12, 8))
save_fig("scatter_matrix_plot")


Saving figure scatter_matrix_plot

请添加图片描述

housing.plot(kind="scatter", x="median_income", y="median_house_value",
             alpha=0.1)
plt.axis([0, 16, 0, 550000])
save_fig("income_vs_house_value_scatterplot")

Saving figure income_vs_house_value_scatterplot

请添加图片描述

属性组合试验

housing["rooms_per_household"] = housing["total_rooms"]/housing["households"]
housing["bedrooms_per_room"] = housing["total_bedrooms"]/housing["total_rooms"]
housing["population_per_household"]=housing["population"]/housing["households"]

corr_matrix = housing.corr()
corr_matrix["median_house_value"].sort_values(ascending=False)

median_house_value          1.000000
median_income               0.688075
rooms_per_household         0.151948
total_rooms                 0.134153
housing_median_age          0.105623
households                  0.065843
total_bedrooms              0.049686
population_per_household   -0.023737
population                 -0.024650
longitude                  -0.045967
latitude                   -0.144160
bedrooms_per_room          -0.255880
Name: median_house_value, dtype: float64

median_house_value

  • median_income 0.687160
  • bedrooms_per_room -0.259984
  • rooms_per_household 0.146285
  • total_rooms 0.135097
  • housing_median_age 0.114110

Prepare the data for Machine Learning algorithms

  • 在任何数据集上（比如，你下一次获取的是一个新的数据集）方便地进行重复数据转换。
  • 慢慢建立一个转换函数库，可以在未来的项目中复用。
housing = strat_train_set.drop("median_house_value", axis=1) # 不影响原数据，只会建立备份
housing_labels = strat_train_set["median_house_value"].copy()

数据清洗

对于缺失值：

  • 去掉对应的街区；
  • 去掉整个属性；
  • 进行赋值（0、平均值、中位数等等）。
housing.dropna(subset=["total_bedrooms"]) # 选项1
housing.drop("total_bedrooms", axis=1) # 选项2
median = housing["total_bedrooms"].median()
housing["total_bedrooms"].fillna(median) # 选项3

如果选择选项 3，你需要计算训练集的中位数，用中位数填充训练集的缺失值，不要忘记保存
该中位数。后面用测试集评估系统时，需要替换测试集中的缺失值，也可以用来实时替换新
数据中的缺失值

Scikit-Learn 提供了一个方便的类来处理缺失值： Imputer

from sklearn.preprocessing import Imputer
imputer = Imputer(strategy="median")
housing_num = housing.drop("ocean_proximity", axis=1)
imputer.fit(housing_num)
imputer.statistics_
X = imputer.transform(housing_num)
housing_tr = pd.DataFrame(X, columns=housing_num.columns)


sample_incomplete_rows = housing[housing.isnull().any(axis=1)].head()
#df.isnull().any()会判断哪些列包含缺失值
sample_incomplete_rows
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomeocean_proximity
4629-118.3034.0718.03759.0NaN3296.01462.02.2708<1H OCEAN
6068-117.8634.0116.04632.0NaN3038.0727.05.1762<1H OCEAN
17923-121.9737.3530.01955.0NaN999.0386.04.6328<1H OCEAN
13656-117.3034.056.02155.0NaN1039.0391.01.6675INLAND
19252-122.7938.487.06837.0NaN3468.01405.03.1662<1H OCEAN

可见total_bedrooms存在缺失值

于是用中位数代替

正好把数据集里所有的特征的中位数一起算了

保存在Imputer里

这里用异常处理一下sklearn版本更迭的问题

try:
    from sklearn.impute import SimpleImputer # Scikit-Learn 0.20+
except ImportError:
    from sklearn.preprocessing import Imputer as SimpleImputer
    
imputer = SimpleImputer(strategy="median")

    


housing_num = housing.drop('ocean_proximity', axis=1)#把标量数据删除，不然没法统一算中位数

imputer.fit(housing_num)

SimpleImputer(strategy='median')

#看看数据
imputer.statistics_

array([-118.51  ,   34.26  ,   29.    , 2119.5   ,  433.    , 1164.    ,
        408.    ,    3.5409])

X = imputer.transform(housing_num)#将缺失值替换为中位数
housing_num
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_income
17606-121.8937.2938.01568.0351.0710.0339.02.7042
18632-121.9337.0514.0679.0108.0306.0113.06.4214
14650-117.2032.7731.01952.0471.0936.0462.02.8621
3230-119.6136.3125.01847.0371.01460.0353.01.8839
3555-118.5934.2317.06592.01525.04459.01463.03.0347
...........................
6563-118.1334.2046.01271.0236.0573.0210.04.9312
12053-117.5633.8840.01196.0294.01052.0258.02.0682
13908-116.4034.099.04855.0872.02098.0765.03.2723
11159-118.0133.8231.01960.0380.01356.0356.04.0625
15775-122.4537.7752.03095.0682.01269.0639.03.5750

16512 rows × 8 columns

housing_tr = pd.DataFrame(X, columns=housing_num.columns,
                          index=housing.index)

处理标量数据

housing_cat = housing[['ocean_proximity']]
housing_cat.head(10)
ocean_proximity
17606<1H OCEAN
18632<1H OCEAN
14650NEAR OCEAN
3230INLAND
3555<1H OCEAN
19480INLAND
8879<1H OCEAN
13685INLAND
4937<1H OCEAN
4861<1H OCEAN
try:
    from sklearn.preprocessing import OrdinalEncoder # just to raise an ImportError if Scikit-Learn < 0.20
    from sklearn.preprocessing import OneHotEncoder
except ImportError:
    from future_encoders import OneHotEncoder 

cat_encoder = OneHotEncoder(sparse=False)
housing_cat_1hot = cat_encoder.fit_transform(housing_cat)
housing_cat_1hot

array([[1., 0., 0., 0., 0.],
       [1., 0., 0., 0., 0.],
       [0., 0., 0., 0., 1.],
       ...,
       [0., 1., 0., 0., 0.],
       [1., 0., 0., 0., 0.],
       [0., 0., 0., 1., 0.]])

housing.columns

Index(['longitude', 'latitude', 'housing_median_age', 'total_rooms',
       'total_bedrooms', 'population', 'households', 'median_income',
       'ocean_proximity'],
      dtype='object')

创建自己的类

自定义转换函器：

创建一个类，实现fit()[return self]、transform()和fit_transform()

通过添加 TransformerMixin 作为基类(父类），可以很容易地得到fit_transform()。

另外，如果你添加 BaseEstimator 作为基类(父类）（且构造器中避免使用 *args 和 **kargs ），你就能得到两个额外的方法（ get_params() 和 set_params() ），

二者可以方便地进行超参数自动微调

sklearn项目可以看成一棵大树，各种estimator是果实，而支撑这些估计器的主干，是为数不多的几个基类。常见的几个类有BaseEstimator、BaseSGD、ClassifierMixin、RegressorMixin，等等。[origin]https://www.cnblogs.com/learn-the-hard-way/p/12532888.html

np.c_ 把几个array连在一起

from sklearn.base import BaseEstimator, TransformerMixin

# get the right column indices: safer than hard-coding indices 3, 4, 5, 6
rooms_ix, bedrooms_ix, population_ix, household_ix = [
    list(housing.columns).index(col)
    for col in ("total_rooms", "total_bedrooms", "population", "households")]

class CombinedAttributesAdder(BaseEstimator, TransformerMixin):
    def __init__(self, add_bedrooms_per_room = True): # no *args or **kwargs
        self.add_bedrooms_per_room = add_bedrooms_per_room
    def fit(self, X, y=None):
        return self  # nothing else to do
    def transform(self, X, y=None):
        rooms_per_household = X[:, rooms_ix] / X[:, household_ix]
        population_per_household = X[:, population_ix] / X[:, household_ix]
        if self.add_bedrooms_per_room:
            bedrooms_per_room = X[:, bedrooms_ix] / X[:, rooms_ix]
            return np.c_[X, rooms_per_household, population_per_household,
                         bedrooms_per_room]
        #  np.c_ 把几个array连在一起
        else:
            return np.c_[X, rooms_per_household, population_per_household]

attr_adder = CombinedAttributesAdder(add_bedrooms_per_room=False)
housing_extra_attribs = attr_adder.transform(housing.values)

另一种方法：FunctionTransformer

from sklearn.preprocessing import FunctionTransformer

def add_extra_features(X, add_bedrooms_per_room=True):
    rooms_per_household = X[:, rooms_ix] / X[:, household_ix]
    population_per_household = X[:, population_ix] / X[:, household_ix]
    if add_bedrooms_per_room:
        bedrooms_per_room = X[:, bedrooms_ix] / X[:, rooms_ix]
        return np.c_[X, rooms_per_household, population_per_household,
                     bedrooms_per_room]
    else:
        return np.c_[X, rooms_per_household, population_per_household]

attr_adder = FunctionTransformer(add_extra_features, validate=False,
                                 kw_args={"add_bedrooms_per_room": False})
housing_extra_attribs = attr_adder.fit_transform(housing.values)

housing_extra_attribs = pd.DataFrame(
    housing_extra_attribs,
    columns=list(housing.columns)+["rooms_per_household", "population_per_household"],
    index=housing.index)
housing_extra_attribs.columns

Index(['longitude', 'latitude', 'housing_median_age', 'total_rooms',
       'total_bedrooms', 'population', 'households', 'median_income',
       'ocean_proximity', 'rooms_per_household', 'population_per_household'],
      dtype='object')

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

num_pipeline = Pipeline([
        ('imputer', SimpleImputer(strategy="median")),#1.去除缺失值  #执行     imputer = SimpleImputer(strategy="median")   imputer.fit(housing_num)
        ('attribs_adder', FunctionTransformer(add_extra_features, validate=False)),#2.添加属性     #FunctionTransformer(add_extra_features, validate=False)
        ('std_scaler', StandardScaler()),#3.特征缩放
    ])

housing_num_tr = num_pipeline.fit_transform(housing_num)



Index(['longitude', 'latitude', 'housing_median_age', 'total_rooms',
       'total_bedrooms', 'population', 'households', 'median_income'],
      dtype='object')

特征缩放

机器学习中重要的一个环节
暂时不进行更新

转换流水线

使用pipline按一定的顺序执行程序

try:
    from sklearn.compose import ColumnTransformer
except ImportError:
    from future_encoders import ColumnTransformer # Scikit-Learn < 0.20

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

full_pipeline = ColumnTransformer([
        ("num", num_pipeline, num_attribs),
        ("cat", OneHotEncoder(), cat_attribs),
    ])

housing_prepared = full_pipeline.fit_transform(housing)

housing_prepared

array([[-1.15604281,  0.77194962,  0.74333089, ...,  0.        ,
         0.        ,  0.        ],
       [-1.17602483,  0.6596948 , -1.1653172 , ...,  0.        ,
         0.        ,  0.        ],
       [ 1.18684903, -1.34218285,  0.18664186, ...,  0.        ,
         0.        ,  1.        ],
       ...,
       [ 1.58648943, -0.72478134, -1.56295222, ...,  0.        ,
         0.        ,  0.        ],
       [ 0.78221312, -0.85106801,  0.18664186, ...,  0.        ,
         0.        ,  0.        ],
       [-1.43579109,  0.99645926,  1.85670895, ...,  0.        ,
         1.        ,  0.        ]])

housing_prepared.shape

(16512, 16)

from sklearn.base import BaseEstimator, TransformerMixin

# Create a class to select numerical or categorical columns 
class OldDataFrameSelector(BaseEstimator, TransformerMixin):
    def __init__(self, attribute_names):
        self.attribute_names = attribute_names
    def fit(self, X, y=None):
        return self
    def transform(self, X):
        return X[self.attribute_names].values

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

old_num_pipeline = Pipeline([
        ('selector', OldDataFrameSelector(num_attribs)),
        ('imputer', SimpleImputer(strategy="median")),
        ('attribs_adder', FunctionTransformer(add_extra_features, validate=False)),
        ('std_scaler', StandardScaler()),
    ])

old_cat_pipeline = Pipeline([
        ('selector', OldDataFrameSelector(cat_attribs)),
        ('cat_encoder', OneHotEncoder(sparse=False)),
    ])

from sklearn.pipeline import FeatureUnion

old_full_pipeline = FeatureUnion(transformer_list=[
        ("num_pipeline", old_num_pipeline),
        ("cat_pipeline", old_cat_pipeline),
    ])

old_housing_prepared = old_full_pipeline.fit_transform(housing)
old_housing_prepared

array([[-1.15604281,  0.77194962,  0.74333089, ...,  0.        ,
         0.        ,  0.        ],
       [-1.17602483,  0.6596948 , -1.1653172 , ...,  0.        ,
         0.        ,  0.        ],
       [ 1.18684903, -1.34218285,  0.18664186, ...,  0.        ,
         0.        ,  1.        ],
       ...,
       [ 1.58648943, -0.72478134, -1.56295222, ...,  0.        ,
         0.        ,  0.        ],
       [ 0.78221312, -0.85106801,  0.18664186, ...,  0.        ,
         0.        ,  0.        ],
       [-1.43579109,  0.99645926,  1.85670895, ...,  0.        ,
         1.        ,  0.        ]])

np.allclose(housing_prepared, old_housing_prepared)

True

选择并训练模型

from sklearn.linear_model import LinearRegression

lin_reg = LinearRegression()
lin_reg.fit(housing_prepared, housing_labels)

LinearRegression()

some_data=housing.iloc[:5]
some_labels=housing_labels.iloc[:5]
some_data_prepared = full_pipeline.transform(some_data)

print("Predictions:", lin_reg.predict(some_data_prepared))

Predictions: [210644.60459286 317768.80697211 210956.43331178  59218.98886849
 189747.55849879]

from sklearn.metrics import mean_squared_error

housing_predictions = lin_reg.predict(housing_prepared)
lin_mse = mean_squared_error(housing_labels, housing_predictions)
lin_rmse = np.sqrt(lin_mse)
lin_rmse

68628.19819848923

68628.19819848923
median housing value 位于120000 与 265000 之间
因此68628还是太大了！！！！
意味着这个模型出现了欠拟合的问题

修复欠拟合的主要方法是选择一个更强大的模型，给训练算法提供更好的特征，或去掉模型上的限制。这个模型还没有正则化，所以排除了最后一个选项。你可以尝试添加更多特征（比如，人口的对数值），但是首先让我们尝试一个更为复杂的模型，看看效果。

试试决策树

from sklearn.tree import DecisionTreeRegressor

tree_reg = DecisionTreeRegressor(random_state=42)
tree_reg.fit(housing_prepared, housing_labels)

DecisionTreeRegressor(random_state=42)

housing_predictions = tree_reg.predict(housing_prepared)
tree_mse = mean_squared_error(housing_labels, housing_predictions)
tree_rmse = np.sqrt(tree_mse)
tree_rmse

0.0

RMSE竟然是0！！！！！！！！！！！！！！！！！！！

严重过拟合预警！！！！！！！！！！！！！

在我们选定好模型之前，不能碰测试集一丝一毫！！！！！！！

交叉验证

from sklearn.model_selection import cross_val_score

scores = cross_val_score(tree_reg, housing_prepared, housing_labels,
                         scoring="neg_mean_squared_error", cv=10)
tree_rmse_scores = np.sqrt(-scores)

特别批注！！！！！！！！！！！！！！
交叉验证得分使用的函数是效用函数，不是代价函数，因此得分越大越好（负数）
计算平方根之前先计算 -scores 。

def display_scores(scores):
    print("Scores:", scores)
    print("Mean:", scores.mean())
    print("Standard deviation:", scores.std())

display_scores(tree_rmse_scores)

Scores: [70194.33680785 66855.16363941 72432.58244769 70758.73896782
 71115.88230639 75585.14172901 70262.86139133 70273.6325285
 75366.87952553 71231.65726027]
Mean: 71407.68766037929
Standard deviation: 2439.4345041191004

看起来比线性回归模型还糟！注意到交叉验证不仅可以让你得到模型性能的评估，还能测量评估的准确性（即，它的标准差）。

决策树的评分大约是 71407，通常波动有 ±2439。

lin_scores = cross_val_score(lin_reg, housing_prepared, housing_labels,
                         scoring="neg_mean_squared_error", cv=10)
lin_rmse_scores = np.sqrt(-lin_scores)
display_scores(lin_rmse_scores)

Scores: [66782.73843989 66960.118071   70347.95244419 74739.57052552
 68031.13388938 71193.84183426 64969.63056405 68281.61137997
 71552.91566558 67665.10082067]
Mean: 69052.46136345083
Standard deviation: 2731.674001798349

线性回归的评分大约是 69052，通常波动有 ±2731。

决策树模型过拟合很严重，它的性能比线性回归模型还差。

from sklearn.ensemble import RandomForestRegressor

forest_reg = RandomForestRegressor(n_estimators=10, random_state=42)
forest_reg.fit(housing_prepared, housing_labels)

RandomForestRegressor(n_estimators=10, random_state=42)

housing_predictions = forest_reg.predict(housing_prepared)
forest_mse = mean_squared_error(housing_labels, housing_predictions)
forest_rmse = np.sqrt(forest_mse)
forest_rmse

21933.31414779769

from sklearn.model_selection import cross_val_score

forest_scores = cross_val_score(forest_reg, housing_prepared, housing_labels,
                                scoring="neg_mean_squared_error", cv=10)
forest_rmse_scores = np.sqrt(-forest_scores)
display_scores(forest_rmse_scores)

Scores: [51646.44545909 48940.60114882 53050.86323649 54408.98730149
 50922.14870785 56482.50703987 51864.52025526 49760.85037653
 55434.21627933 53326.10093303]
Mean: 52583.72407377466
Standard deviation: 2298.353351147122

随机森林的评分大约是 52583，通常波动有 ±2298。

我们这里所说的评分指的是验证集的分数

训练集的评分比验证集的评分低，说明过拟合

解决过拟合可以通过简化模型，给模型加限制（即，规整化），或用更多的训练数据。

在深入随机森林之前，你应该尝试下机器学习算法的其它类型模型（不同核心的支持向量机，神经网络，等等），

不要在调节超参数上花费太多时间。目标是列出一个可能模型的列表（两到五个）。

替换模型比调参优先

你要保存每个试验过的模型，以便后续可以再用。要确保有超参数和训练参数，
以及交叉验证评分，和实际的预测值。这可以让你比较不同类型模型的评分，还可以比
较误差种类。你可以用 Python 的模块 pickle ，非常方便地保存 Scikit-Learn 模型，或使用 sklearn.externals.joblib ，后者序列化大 NumPy 数组更有效率：

from sklearn.externals import joblib
joblib.dump(my_model, "my_model.pkl")
# 然后
my_model_loaded = joblib.load("my_model.pkl")

模型微调

假设你现在有了一个列表，列表里有几个有希望的模型。你现在需要对它们进行微调。让我们来看几种微调的方法。

网格搜索

微调的一种方法是手工调整超参数，直到找到一个好的超参数组合。这么做的话会非常冗
长，你也可能没有时间探索多种组合。

你应该使用 Scikit-Learn 的 GridSearchCV 来做这项搜索工作。你所需要做的是告诉 GridSearchCV 要试验有哪些超参数，要试验什么值， GridSearchCV 就能用交叉验证试验所有可能超参数值的组合。例如，下面的代码搜索了 RandomForestRegressor 超参数值的最佳组合：

from sklearn.model_selection import GridSearchCV

param_grid = [
    # try 12 (3×4) combinations of hyperparameters
    {'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},
    # then try 6 (2×3) combinations with bootstrap set as False
    {'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},
  ]

forest_reg = RandomForestRegressor(random_state=42)
# train across 5 folds, that's a total of (12+6)*5=90 rounds of training 
grid_search = GridSearchCV(forest_reg, param_grid, cv=5,
                           scoring='neg_mean_squared_error', return_train_score=True)
grid_search.fit(housing_prepared, housing_labels)

GridSearchCV(cv=5, estimator=RandomForestRegressor(random_state=42),
             param_grid=[{'max_features': [2, 4, 6, 8],
                          'n_estimators': [3, 10, 30]},
                         {'bootstrap': [False], 'max_features': [2, 3, 4],
                          'n_estimators': [3, 10]}],
             return_train_score=True, scoring='neg_mean_squared_error')

grid_search.best_params_

{'max_features': 8, 'n_estimators': 30}

grid_search.best_estimator_

RandomForestRegressor(max_features=8, n_estimators=30, random_state=42)

cvres = grid_search.cv_results_
for mean_score, params in zip(cvres["mean_test_score"], cvres["params"]):
    print(np.sqrt(-mean_score), params)

63669.11631261028 {'max_features': 2, 'n_estimators': 3}
55627.099719926795 {'max_features': 2, 'n_estimators': 10}
53384.57275149205 {'max_features': 2, 'n_estimators': 30}
60965.950449450494 {'max_features': 4, 'n_estimators': 3}
52741.04704299915 {'max_features': 4, 'n_estimators': 10}
50377.40461678399 {'max_features': 4, 'n_estimators': 30}
58663.93866579625 {'max_features': 6, 'n_estimators': 3}
52006.19873526564 {'max_features': 6, 'n_estimators': 10}
50146.51167415009 {'max_features': 6, 'n_estimators': 30}
57869.25276169646 {'max_features': 8, 'n_estimators': 3}
51711.127883959234 {'max_features': 8, 'n_estimators': 10}
49682.273345071546 {'max_features': 8, 'n_estimators': 30}
62895.06951262424 {'bootstrap': False, 'max_features': 2, 'n_estimators': 3}
54658.176157539405 {'bootstrap': False, 'max_features': 2, 'n_estimators': 10}
59470.40652318466 {'bootstrap': False, 'max_features': 3, 'n_estimators': 3}
52724.9822587892 {'bootstrap': False, 'max_features': 3, 'n_estimators': 10}
57490.5691951261 {'bootstrap': False, 'max_features': 4, 'n_estimators': 3}
51009.495668875716 {'bootstrap': False, 'max_features': 4, 'n_estimators': 10}

pd.DataFrame(grid_search.cv_results_)
mean_fit_timestd_fit_timemean_score_timestd_score_timeparam_max_featuresparam_n_estimatorsparam_bootstrapparamssplit0_test_scoresplit1_test_score...mean_test_scorestd_test_scorerank_test_scoresplit0_train_scoresplit1_train_scoresplit2_train_scoresplit3_train_scoresplit4_train_scoremean_train_scorestd_train_score
00.0667560.0037140.0035990.00049523NaN{'max_features': 2, 'n_estimators': 3}-3.837622e+09-4.147108e+09...-4.053756e+091.519591e+0818-1.064113e+09-1.105142e+09-1.116550e+09-1.112342e+09-1.129650e+09-1.105559e+092.220402e+07
10.2142350.0075580.0102510.001075210NaN{'max_features': 2, 'n_estimators': 10}-3.047771e+09-3.254861e+09...-3.094374e+091.327062e+0811-5.927175e+08-5.870952e+08-5.776964e+08-5.716332e+08-5.802501e+08-5.818785e+087.345821e+06
20.6701760.0232810.0277100.001455230NaN{'max_features': 2, 'n_estimators': 30}-2.689185e+09-3.021086e+09...-2.849913e+091.626875e+089-4.381089e+08-4.391272e+08-4.371702e+08-4.376955e+08-4.452654e+08-4.394734e+082.966320e+06
30.1170890.0170030.0037900.00039943NaN{'max_features': 4, 'n_estimators': 3}-3.730181e+09-3.786886e+09...-3.716847e+091.631510e+0816-9.865163e+08-1.012565e+09-9.169425e+08-1.037400e+09-9.707739e+08-9.848396e+084.084607e+07
40.3661520.0562510.0095700.000797410NaN{'max_features': 4, 'n_estimators': 10}-2.666283e+09-2.784511e+09...-2.781618e+091.268607e+088-5.097115e+08-5.162820e+08-4.962893e+08-5.436192e+08-5.160297e+08-5.163863e+081.542862e+07
50.9973330.0148130.0273280.000798430NaN{'max_features': 4, 'n_estimators': 30}-2.387153e+09-2.588448e+09...-2.537883e+091.214614e+083-3.838835e+08-3.880268e+08-3.790867e+08-4.040957e+08-3.845520e+08-3.879289e+088.571233e+06
60.1384940.0075380.0039890.00109363NaN{'max_features': 6, 'n_estimators': 3}-3.119657e+09-3.586319e+09...-3.441458e+091.893056e+0814-9.245343e+08-8.886939e+08-9.353135e+08-9.009801e+08-8.624664e+08-9.023976e+082.591445e+07
70.4815760.0265110.0108720.002037610NaN{'max_features': 6, 'n_estimators': 10}-2.549663e+09-2.782039e+09...-2.704645e+091.471569e+086-4.980344e+08-5.045869e+08-4.994664e+08-4.990325e+08-5.055542e+08-5.013349e+083.100456e+06
81.5888770.0411500.0355820.007298630NaN{'max_features': 6, 'n_estimators': 30}-2.370010e+09-2.583638e+09...-2.514673e+091.285080e+082-3.838538e+08-3.804711e+08-3.805218e+08-3.856095e+08-3.901917e+08-3.841296e+083.617057e+06
90.2061540.0054400.0047920.00074883NaN{'max_features': 8, 'n_estimators': 3}-3.353504e+09-3.348552e+09...-3.348850e+091.241939e+0813-9.228123e+08-8.553031e+08-8.603321e+08-8.881964e+08-9.151287e+08-8.883545e+082.750227e+07
100.7308980.0799850.0139630.001784810NaN{'max_features': 8, 'n_estimators': 10}-2.571970e+09-2.718994e+09...-2.674041e+091.392777e+085-4.932416e+08-4.815238e+08-4.730979e+08-5.155367e+08-4.985555e+08-4.923911e+081.459294e+07
111.9273050.0840950.0317140.001329830NaN{'max_features': 8, 'n_estimators': 30}-2.357390e+09-2.546640e+09...-2.468328e+091.091662e+081-3.841658e+08-3.744500e+08-3.773239e+08-3.882250e+08-3.810005e+08-3.810330e+084.871017e+06
120.1123440.0025300.0041820.00040023False{'bootstrap': False, 'max_features': 2, 'n_est...-3.785816e+09-4.166012e+09...-3.955790e+091.900964e+0817-0.000000e+00-0.000000e+00-0.000000e+00-0.000000e+00-0.000000e+000.000000e+000.000000e+00
130.3732540.0069150.0117580.000744210False{'bootstrap': False, 'max_features': 2, 'n_est...-2.810721e+09-3.107789e+09...-2.987516e+091.539234e+0810-6.056477e-02-0.000000e+00-0.000000e+00-0.000000e+00-2.967449e+00-6.056027e-011.181156e+00
140.1469210.0077710.0045950.00079133False{'bootstrap': False, 'max_features': 3, 'n_est...-3.618324e+09-3.441527e+09...-3.536729e+097.795057e+0715-0.000000e+00-0.000000e+00-0.000000e+00-0.000000e+00-6.072840e+01-1.214568e+012.429136e+01
150.4567090.0051460.0117810.000736310False{'bootstrap': False, 'max_features': 3, 'n_est...-2.757999e+09-2.851737e+09...-2.779924e+096.286720e+077-2.089484e+01-0.000000e+00-0.000000e+00-0.000000e+00-5.465556e+00-5.272080e+008.093117e+00
160.1720730.0017930.0045940.00079243False{'bootstrap': False, 'max_features': 4, 'n_est...-3.134040e+09-3.559375e+09...-3.305166e+091.879165e+0812-0.000000e+00-0.000000e+00-0.000000e+00-0.000000e+00-0.000000e+000.000000e+000.000000e+00
170.5645070.0086740.0123610.000795410False{'bootstrap': False, 'max_features': 4, 'n_est...-2.525578e+09-2.710011e+09...-2.601969e+091.088048e+084-0.000000e+00-1.514119e-02-0.000000e+00-0.000000e+00-0.000000e+00-3.028238e-036.056477e-03

18 rows × 23 columns

成功地微调了最佳模型

提示：不要忘记，你可以像超参数一样处理数据准备的步骤。例如，网格搜索可以自动
判断是否添加一个你不确定的特征（比如，使用转换器 CombinedAttributesAdder 的超参
数 add_bedrooms_per_room ）。它还能用相似的方法来自动找到处理异常值、缺失特征、
特征选择等任务的最佳方法。

随机搜索

超参数的搜索空间很大时，最好使用 RandomizedSearchCV

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint

param_distribs = {
        'n_estimators': randint(low=1, high=200),
        'max_features': randint(low=1, high=8),
    }

forest_reg = RandomForestRegressor(random_state=42)
rnd_search = RandomizedSearchCV(forest_reg, param_distributions=param_distribs,
                                n_iter=10, cv=5, scoring='neg_mean_squared_error', random_state=42)
rnd_search.fit(housing_prepared, housing_labels)

RandomizedSearchCV(cv=5, estimator=RandomForestRegressor(random_state=42),
                   param_distributions={'max_features': <scipy.stats._distn_infrastructure.rv_frozen object at 0x000001FAB8188D88>,
                                        'n_estimators': <scipy.stats._distn_infrastructure.rv_frozen object at 0x000001FAB8188C08>},
                   random_state=42, scoring='neg_mean_squared_error')

cvres = rnd_search.cv_results_
for mean_score, params in zip(cvres["mean_test_score"], cvres["params"]):
    print(np.sqrt(-mean_score), params)

49150.70756927707 {'max_features': 7, 'n_estimators': 180}
51389.889203389284 {'max_features': 5, 'n_estimators': 15}
50796.155224308866 {'max_features': 3, 'n_estimators': 72}
50835.13360315349 {'max_features': 5, 'n_estimators': 21}
49280.9449827171 {'max_features': 7, 'n_estimators': 122}
50774.90662363929 {'max_features': 3, 'n_estimators': 75}
50682.78888164288 {'max_features': 3, 'n_estimators': 88}
49608.99608105296 {'max_features': 5, 'n_estimators': 100}
50473.61930350219 {'max_features': 3, 'n_estimators': 150}
64429.84143294435 {'max_features': 5, 'n_estimators': 2}

每个属性(特征）对于做出准确预测的相对重要性

feature_importances = grid_search.best_estimator_.feature_importances_
feature_importances

array([7.33442355e-02, 6.29090705e-02, 4.11437985e-02, 1.46726854e-02,
       1.41064835e-02, 1.48742809e-02, 1.42575993e-02, 3.66158981e-01,
       5.64191792e-02, 1.08792957e-01, 5.33510773e-02, 1.03114883e-02,
       1.64780994e-01, 6.02803867e-05, 1.96041560e-03, 2.85647464e-03])

extra_attribs = ["rooms_per_hhold", "pop_per_hhold", "bedrooms_per_room"]
#cat_encoder = cat_pipeline.named_steps["cat_encoder"] # old solution
cat_encoder = full_pipeline.named_transformers_["cat"]
cat_one_hot_attribs = list(cat_encoder.categories_[0])
attributes = num_attribs + extra_attribs + cat_one_hot_attribs
sorted(zip(feature_importances, attributes), reverse=True)

[(0.36615898061813423, 'median_income'),
 (0.16478099356159054, 'INLAND'),
 (0.10879295677551575, 'pop_per_hhold'),
 (0.07334423551601243, 'longitude'),
 (0.06290907048262032, 'latitude'),
 (0.056419179181954014, 'rooms_per_hhold'),
 (0.053351077347675815, 'bedrooms_per_room'),
 (0.04114379847872964, 'housing_median_age'),
 (0.014874280890402769, 'population'),
 (0.014672685420543239, 'total_rooms'),
 (0.014257599323407808, 'households'),
 (0.014106483453584104, 'total_bedrooms'),
 (0.010311488326303788, '<1H OCEAN'),
 (0.0028564746373201584, 'NEAR OCEAN'),
 (0.0019604155994780706, 'NEAR BAY'),
 (6.0280386727366e-05, 'ISLAND')]

final_model = grid_search.best_estimator_

X_test = strat_test_set.drop("median_house_value", axis=1)
y_test = strat_test_set["median_house_value"].copy()

X_test_prepared = full_pipeline.transform(X_test)
final_predictions = final_model.predict(X_test_prepared)

final_mse = mean_squared_error(y_test, final_predictions)
final_rmse = np.sqrt(final_mse)

计算RMSE的置信区间

from scipy import stats

confidence = 0.95
squared_errors = (final_predictions - y_test) ** 2
mean = squared_errors.mean()
m = len(squared_errors)

np.sqrt(stats.t.interval(confidence, m - 1,
                         loc=np.mean(squared_errors),
                         scale=stats.sem(squared_errors)))


array([45685.10470776, 49691.25001878])
彼得·伊里奇·柴可夫斯基
关注
  • 1
    点赞
  • 踩
  • 5
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
seg.rar_Untitled
09-24
在IT领域，图像处理是一项重要的技术，特别是在计算机视觉、机器学习和数据分析中有着广泛应用。"seg.rar_Untitled"这个项目似乎是一个关于交互式图像分割的实现，这是一类允许用户通过交互方式指定感兴趣区域（ROI...
Django 文件下使用相对路径
张某文_Lambda的博客
12-26 876
在文件下设置自己的路径变量，在Django配置使用相对路径。
Classification分类器|机器学习实战-基于Scikit-Learn和TensorFlow
honor
07-12 979
Chapter 3 – Classification This notebook contains all the sample code and solutions to the exercises in chapter 3. Setup First, let’s make sure this notebook works well in both python 2 and 3, import a few common modules, ensure MatplotLib plots figures in
加州房价预测模型
小哲数据
12-08 2828
import numpy as np import pandas as pd import matplotlib.pyplot as plt %matplotlib %matplotlib inline %config InlineBackend.figure_format = 'retina' Using matplotlib backend: MacOSX 读取数据集并观察数据特点 字段...
【基础知识】8、加州房价预测项目精细解释
呆呆的猫的博客
08-14 1万+
### 此处选择StatLib的加州房产价格数据集，该数据集是基于1990年加州普查的数据 # 一、项目概览 利用加州普查数据，建立一个加州房价模型，这个数据包含每个街区组的人口、收入的中位数、房价中位数等指标。街区组是美国调查局发布样本数据的最小地理单位（600到3000人），简称“街区”。 ### 目标：模型利用该数据进行学习，然后根据其他指标，预测任何街区的房价...
机器学习实战（2）——端到端的机器学习项目
WHJ226的博客
08-08 1296
50万美元的价格上限在图中是一条清晰的水平线，不过除此以外，这张图还显示出几条不那么明显的直线：45万美元附近有一条水平线，35万美元附近也有一条，28万美元附近似乎隐约也有一条，再往下可能还有一些。首先，把测试集放在一边，我们能探索的只有训练集。例如，对于housing_ median_age 的值， 25% 的区域低于18，50%的区域低于29，以及75%的区域低于 37。这也是个不断选代的过程：一旦我们的原型产生并且开始运行，我们可以分析它的输出以洞悉更多的见解，然后再次回到这个探索的步骤。.....
untitled.zip
01-05
时间序列预测是数据分析领域中的一个重要话题，特别是在处理与时间有关的数据时，如销售记录、股票价格、网站...此外，还可以结合机器学习方法，如LSTM（长短时记忆网络）进行复杂序列的学习，以进一步提升预测精度。
Eeg数据的特征提取和BP神经网络的技术研究
06-01
特征提取是从原始EEG信号中提取有意义信息的过程，而BP神经网络则是一种在机器学习和深度学习中常见的模型，用于模式识别和预测任务。 在特征提取方面，EEG数据通常经过以下步骤处理： 1. 预处理：去除噪声，如...
untitled_300_SVM_
10-02
标题“untitled_300_SVM_”表明这是一个与沪深300指数相关的机器学习项目，使用了支持向量机（SVM）算法。在金融领域，SVM常用于股票预测，通过训练历史数据来构建模型，以帮助确定最优的投资组合。下面将详细介绍这...
案例实战信用卡欺诈检测数据集
06-13
在这个案例中，`Untitled.ipynb`可能是用于分析和建模的Jupyter Notebook文件，其中记录了完整的分析过程和代码实现。通过这样的实战项目，我们可以加深对机器学习原理的理解，同时提高在实际问题中应用这些知识的...
NameError: name 'save_fig' is not defined
^_^妖言惑眾的博客
03-22 9741
NameError: name 'save_fig' is not defined import matplotlib.pyplot as plt save_fig("attribute_histogram_plots") 编译错误，NameError: name ‘save_fig’ is not defined，如图： /usr/local/bin/python3.6 /Users/z...
【机器学习实用指南】加州房价中位数预测
weixin_40679158的博客
05-12 1960
加州房价预测 # 同时支持python2和python3 from __future__ import division,print_function,unicode_literals # 常用库 import numpy as np import os # 使这个notebook的输出一直是不变的 np.random.seed(42) # 用于画图的库 import matplotlib as mpl import matplotlib.pyplot as plt mpl.rc('axes', la
机器学习实践—基于Scikit-Learn、Keras和TensorFlow2第二版—第2章 端到端机器学习项目实践
bigcindy的博客
06-26 3680
本章使用California房价数据集进行案例分析 1. 导入所需的库 import os import tarfile import urllib import pandas as pd import numpy as np import matplotlib.pyplot as plt %matplotlib inline for i in [pd,np]: print(i.__name__,": ",i.__version__,sep="") 输出： pandas: 0.25.3
【推荐收藏】【机器学习实战】区域房价中位数预测（挑战全网最全，没有之一，另附完整代码）
热门推荐
惟忆
11-01 4万+
参照《机器学习实战》第二版 1、下载数据 import os import tarfile import urllib.request DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml2/master/" HOUSING_PATH = os.path.join("datasets", "housing") HOUSING_URL = DOWNLOAD_ROOT + "datasets/housing/housing.
人工智能第一次作业
weixin_44353950的博客
09-24 195
人工智能第一次作业 1 性能衡量指标使用均方根误差RMSE 2 （1）加载数据集方法的代码为： import pandas as pd def load_housing_data(housing_path=HOUSING_PATH): csv_path = os.path.join(housing_path, "housing.csv") return pd.read_csv(csv_path) housing = load_housing_data() （2）数据集的总行数为20640
二、一个完整的机器学习项目
喝醉酒的小白
02-29 3901
机器学习八股 1. 项目概述 2. 获取数据 3. 发现并可视化数据，发现规律 4. 为机器学习算法准备数据 5. 选择模型，进行训练 6. 微调模型 7. 给出解决方案 8. 部署、监控、维护系统
Hands-on ML 学习笔记-P1-chapter2
qq_41167073的博客
02-17 409
End-to-end Machine Learning 学习笔记（主要关于代码） 1.Set up # to make this notebook's output stable across runs np.random.seed(42) # To plot pretty figures %matplotlib inline # 主要用在jupyter notebook上，用于让pytho...
Python 之 histogram直方图(pandas, pd)
hxxjxw的博客
09-29 3093
iris.csv Histogram import matplotlib.pyplot as plt import pandas as pd iris_df = pd.read_csv("iris.csv") iris_df["sepallength"].hist(by=iris_df["class"]) #Group data by class label plt.show() Stacked histogram 堆叠直方图 import matplotlib.py...
behavior = "untitled-cbc30d"
最新发布
06-09
根据您提供的"untitled-cbc30d"这个名称，我无法确定它是哪一个具体的行为模块，因为这个名称可能只是一个默认的未命名模块，或者是一个特定程序中的一个命名行为。如果您能提供更多的上下文信息或者具体的问题，我...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值