《动手学深度学习》笔记---3.16

3.16 实战Kaggle比赛：房价预测

3.16.2 读取数据集

# 导入所需的模块和包

%matplotlib inline
import d2lzh as d2l
from mxnet import autograd, gluon, init, nd
from mxnet.gluon import data as gdata, loss as gloss, nn
import numpy as np
import pandas as pd

# 注意路径：中文问题，使用斜杠 "/",不要使用反斜杠"\"
train_data = pd.read_csv('E:/kaggle/data/train.csv')
test_data = pd.read_csv('E:/kaggle/data/test.csv')

# 训练数据集包含1460个样本、80个特征和1个标签
train_data.shape

output:

(1460, 81)

测试数据集包含1459个样本和80个特征

# 需要将测试数据集中的每个样本的标签预测出来

test_data.shape

output:

(1459, 80)

查看训练数据集的前4个样本的前4个特征、后两个特征和标签

train_data.iloc[0 : 4, [0, 1, 2, 3, -3, -2, -1]]

查看测试数据集的前4个样本的前4个特征、后两个特征和标签

test_data.iloc[0 : 4, [0, 1, 2, 3, -3, -2, -1]]

1. 数据集的第一个特征（第一列）是Id，它能帮助模型记住每个训练样本，但是不能推广到测试样本
2. 测试样本的Id和训练样本的Id不能参与
3. 训练集是[:, 1 : -1]是去掉Id和SalePrice特征
4. 测试集是[:, 1:]是去掉Id

all_features = pd.concat((train_data.iloc[:, 1 : -1], test_data.iloc[:, 1:]))
all_features.shape

output:

(2919, 79)

3.16.3 预处理数据集

# 对连续数值的特征做标准化：设该特征在整个数据集上的均值为μ，标准差为σ。
# 把该特征的每个值先减去μ再除以σ得到标准化后的每个特征值
# 对于缺失的特征值，将其替换成该特征的均值
# lambda定义简单函数，“：”后表达式只有一行且为函数返回值,":"之前是形式参数
# Python中apply函数的格式为：apply(func,*args,**kwargs)
# apply的返回值就是函数func函数的返回值
# fillna(a)会把缺失值填充为 a

numeric_features = all_features.dtypes[all_features.dtypes != 'object'].index   # 从所有特征中，找数值型数据索引
all_features[numeric_features] = all_features[numeric_features].apply(
    lambda x: (x - x.mean()) / (x.std()))
# 标准化后每个特征的均值变为0，可以直接用0来替换缺失值
all_features[numeric_features] = all_features[numeric_features].fillna(0)

# 将离散值转成知识特征

# dummy_na = True 将缺失值也当作合法的特征值并为其创建指示特征
# pd.get_dummies用法参考：https://blog.csdn.net/maymay_/article/details/80198468

all_features = pd.get_dummies(all_features, dummy_na = True)
all_features.shape

# 通过values属性得到NumPy格式的数据，并转成NDArray方便后面的训练

n_train = train_data.shape[0]   # 训练数据行数，即训练样本数
train_features = nd.array(all_features[:n_train].values)
test_features = nd.array(all_features[n_train:].values)
train_labels = nd.array(train_data.SalePrice.values).reshape((-1, 1))

3.16.4 训练模型

1. 使用基本的线性回归模型和平方损失函数来训练网络

loss = gloss.L2Loss()

def get_net():
    net = nn.Sequential()
    net.add(nn.Dense(1))
    net.initialize()
    return net

2. 定义比赛用来评价模型的对数均方根误差

def log_rmse(net, features, labels):
    # 将小于1的值设为1，使得取对数时数值更稳定
    # 对net(features)处理成（1，float(inf)) 正无穷之间，小于1的取成1
    
    clipped_preds = nd.clip(net(features), 1, float('inf'))
    rmse = nd.sqrt(2 * loss(clipped_preds.log(), labels.log()).mean())
    return rmse.asscalar()

3. 使用Adam优化算法

相比于小批量随机梯度下降，它对学习率相对不那么敏感

def train(net, train_features, train_labels, test_features, test_labels,
         num_epochs, learning_rate, weight_decay, batch_size):
    train_ls, test_ls = [], []
    train_iter = gdata.DataLoader(gdata.ArrayDataset(
        train_features, train_labels), batch_size, shuffle = True)
    # 这里使用Adam优化算法
    trainer = gluon.Trainer(net.collect_params(), 'adam', {
        'learning_rate':learning_rate, 'wd': weight_decay })
    for epoch in range(num_epochs):
        for X, y in train_iter:
            with autograd.record():
                l = loss(net(X), y)
            l.backward()
            trainer.step(batch_size)
        train_ls.append(log_rmse(net, train_features, train_labels))
        if test_labels is not None:
            test_ls.append(log_rmse(net, test_features, test_labels))
    return train_ls, test_ls

3.16.5 k折交叉验证

1. 实现函数，用来返回第i折交叉验证时所需要的训练和验证数据

# 用来选择模型设计并调节超参数

def get_k_fold_data(k, i, X, y):
    assert k > 1                   # k = 1 就是原数据集，不用分了
    fold_size = X.shape[0] // k   # 整数除法,fold_size 是每份的样本数量
    X_train, y_train = None, None
    for j in range(k):
        idx = slice(j * fold_size, (j + 1) * fold_size)     # 获取当前循环所使用数据的索引
        X_part, y_part = X[idx, :], y[idx]                 # 获取当前循环所使用的数据
        if j == i:                                         # 如果当前循环就是第i折交叉，当前数据就作为测试集
            X_valid, y_valid = X_part, y_part
        elif X_train is None:                             # 如果没有训练集，就把当前循环所使用数据作为训练数据
            X_train, y_train = X_part, y_part
        else:                                             # 当前所使用的数据既不是测试数据，也不是第一份训练数据，就把
            X_train = nd.concat(X_train, X_part, dim = 0) # 当前数据添加进训练数据
            y_train = nd.concat(y_train, y_part, dim = 0)
    return X_train, y_train, X_valid, y_valid

2. 在k折交叉验证中，训练k次并返回训练和验证的平方误差

def k_fold(k, X_train, y_train, num_epochs,
          learning_rate, weight_decay, batch_size):
    train_l_sum, valid_l_sum = 0, 0
    for i in range(k):
        data = get_k_fold_data(k, i, X_train, y_train)           # 获取当前循环所对应的那一份数据
        net = get_net()
        train_ls, valid_ls = train(net, *data, num_epochs, learning_rate,  # train_ls共有num_epochs个损失
                                  weight_decay, batch_size)
        train_l_sum += train_ls[-1]                              # 取train_ls的最后一个数是 本折循环 训练得到的最后损失
        valid_l_sum += valid_ls[-1]                              # 因为train函数中的train_ls是append进来的
        if i == 0:                                               # 只有第一次循环才画图
            d2l.semilogy(range(1, num_epochs + 1), train_ls, 'epochs', 'rmse',
                        range(1, num_epochs + 1), valid_ls,
                         ['train', 'valid'])
        print('fold %d, train rmse %f, valid rmse %f'
             % (i, train_ls[-1], valid_ls[-1]))
    return train_l_sum / k, valid_l_sum / k

3.16.6 模型选择

1. 使用一组未经调优的超参数并计算交叉验证误差
2. 可改动这些超参数来尽可能地减小平均测试误差

k, num_epochs, lr, weight_decay, batch_size = 10, 1000, 6, 0, 64
train_l, valid_l = k_fold(k, train_features, train_labels, num_epochs, lr,
                         weight_decay, batch_size)
print('%d-fold validation: avg train rmse %f, avg valid rmse %f'
     %(k, train_l, valid_l))

output:

fold 0, train rmse 0.123131, valid rmse 0.183182
fold 1, train rmse 0.123973, valid rmse 0.136109
fold 2, train rmse 0.122530, valid rmse 0.189163
fold 3, train rmse 0.117659, valid rmse 0.171328
fold 4, train rmse 0.118193, valid rmse 0.173402
fold 5, train rmse 0.123091, valid rmse 0.140670
fold 6, train rmse 0.122017, valid rmse 0.186078
fold 7, train rmse 0.124806, valid rmse 0.133579
fold 8, train rmse 0.112019, valid rmse 0.184292
fold 9, train rmse 0.121988, valid rmse 0.155886
10-fold validation: avg train rmse 0.120941, avg valid rmse 0.165369

3.16.7 预测并在Kaggle提交结果

1. 重训练数据集，得到预测结果submission.csv

1. 在预测之前，使用完整的训练数据来重新训练模型，
2. 并将预测结果存成提交所需要的格式

def train_and_pred(train_features, test_features, train_labels, test_data,
                 num_epochs, lr, weight_decay, batch_size):
    net = get_net()
    train_ls, _  = train(net, train_features, train_labels, None, None,
                       num_epochs, lr, weight_decay, batch_size)
    d2l.semilogy(range(1, num_epochs + 1), train_ls, 'epochs', 'rsme')
    print('train rmse %f' % train_ls[-1])
    preds = net(test_features).asnumpy()
    test_data['SalePrice'] = pd.Series(preds.reshape(1, -1)[0])
    submission = pd.concat([test_data['Id'], test_data['SalePrice']], axis = 1)
    submission.to_csv('E:/kaggle/data/submission.csv', index = False)

train_and_pred(train_features, test_features, train_labels, test_data, num_epochs, lr, weight_decay, batch_size)

output:

train rmse 0.121889

2. 在kaggle网站上提交预测结果“submission.csv”

提交结果：