2023“SEED”第四届江苏大数据开发与应用大赛--新能源赛道学习笔记

mathmodeling111

已于 2023-10-18 20:25:34 修改

阅读量325

点赞数

文章标签：大数据学习笔记

于 2023-10-18 20:19:14 首次发布

本文链接：https://blog.csdn.net/mathmodeling111/article/details/133900880

版权

一、赛题背景

电动汽车充电站运营管理中，准确预测充电站的电量需求对于提高充电站运营服务水平和优化区域电网供给能力非常关键。本次赛题旨在建立站点充电量预测模型，根据充电站的相关信息和历史电量数据，准确预测未来某段时间内充电站的充电量需求。

在赛题数据中，提供了电动汽车充电站的场站编号、位置信息、历史电量等基本信息。鼓励参赛选手在已有数据的基础上补充或构造额外的特征，以获得更好的预测性能。

二、赛题任务

构造合理特征及算法模型，预估站点未来一周每日的充电量。这里注意是需要我们去预测一周之内每一天的充电量，而官方给出的数据是包含了每一个小时的充电量。

三、赛题数据

本赛题提供的数据集包含三张数据表，具体的数据如下。

四、baseline解析

1.导包

import numpy as np
import pandas as pd
import lightgbm as lgb
from sklearn.model_selection import StratifiedKFold, KFold, GroupKFold
from sklearn.metrics import mean_squared_error, mean_absolute_error
import matplotlib.pyplot as plt
import tqdm
import sys
import os
import gc
import argparse
import warnings

warnings.filterwarnings('ignore')

2.数据的读取与合并

# 读取数据
train_power_forecast_history = pd.read_csv('data1/train/power_forecast_history.csv')
train_power = pd.read_csv('data1/train/power.csv')
train_stub_info = pd.read_csv('data1/train/stub_info.csv')

test_power_forecast_history = pd.read_csv('data1/test/power_forecast_history.csv')
test_stub_info = pd.read_csv('data1/test/stub_info.csv')

# 聚合数据
train_df = train_power_forecast_history.groupby(['id_encode','ds']).head(1)
del train_df['hour']

test_df = test_power_forecast_history.groupby(['id_encode','ds']).head(1)
del test_df['hour']

tmp_df = train_power.groupby(['id_encode','ds'])['power'].sum()
tmp_df.columns = ['id_encode','ds','power']

# 合并充电量数据
train_df = train_df.merge(tmp_df, on=['id_encode','ds'], how='left')

### 合并数据
train_df = train_df.merge(train_stub_info, on='id_encode', how='left')
test_df = test_df.merge(test_stub_info, on='id_encode', how='left')

上面的代码除了进行了数据的合并，并且将数据一天中每个小时的能量都相加，获得了每天的能量。

3.数据预处理&特征提取

### 数据预处理
train_df['flag'] = train_df['flag'].map({'A':0,'B':1})
test_df['flag'] = test_df['flag'].map({'A':0,'B':1})

def get_time_feature(df, col):

    df_copy = df.copy()
    prefix = col + "_"
    df_copy['new_'+col] = df_copy[col].astype(str)

    col = 'new_'+col
    df_copy[col] = pd.to_datetime(df_copy[col], format='%Y%m%d')
    df_copy[prefix + 'year'] = df_copy[col].dt.year
    df_copy[prefix + 'month'] = df_copy[col].dt.month
    df_copy[prefix + 'day'] = df_copy[col].dt.day
    # df_copy[prefix + 'weekofyear'] = df_copy[col].dt.weekofyear
    df_copy[prefix + 'dayofweek'] = df_copy[col].dt.dayofweek
    df_copy[prefix + 'is_wknd'] = df_copy[col].dt.dayofweek // 6
    df_copy[prefix + 'quarter'] = df_copy[col].dt.quarter
    df_copy[prefix + 'is_month_start'] = df_copy[col].dt.is_month_start.astype(int)
    df_copy[prefix + 'is_month_end'] = df_copy[col].dt.is_month_end.astype(int)
    del df_copy[col]

    return df_copy

train_df = get_time_feature(train_df, 'ds')
test_df = get_time_feature(test_df, 'ds')

cols = [f for f in test_df.columns if f not in ['ds','power','h3']]

4.模型的训练与验证

# 模型训练与验证

# 使用K折交叉验证训练和验证模型
def cv_model(clf, train_x, train_y, test_x, seed=2023):
    # 定义折数并初始化KFold
    folds = 5
    kf = KFold(n_splits=folds, shuffle=True, random_state=seed)

    # 初始化oof预测和测试集预测
    oof = np.zeros(train_x.shape[0])
    test_predict = np.zeros(test_x.shape[0])
    cv_scores = []

    # KFold交叉验证
    for i, (train_index, valid_index) in enumerate(kf.split(train_x, train_y)):
        print('************************************ {} ************************************'.format(str(i+1)))
        trn_x, trn_y, val_x, val_y = train_x.iloc[train_index], train_y[train_index], train_x.iloc[valid_index], train_y[valid_index]

        # 转换数据为lightgbm数据格式
        train_matrix = clf.Dataset(trn_x, label=trn_y)
        valid_matrix = clf.Dataset(val_x, label=val_y)

        # 定义lightgbm参数
        params = {
            'boosting_type': 'gbdt',
            'objective': 'regression',
            'metric': 'rmse',
            'min_child_weight': 5,
            'num_leaves': 2 ** 7,
            'lambda_l2': 10,
            'feature_fraction': 0.8,
            'bagging_fraction': 0.8,
            'bagging_freq': 4,
            'learning_rate': 0.1,
            'seed': 2023,
            'nthread' : 16,
            'verbose' : -1,
            # 'device':'gpu'
        }

        # 训练模型
        model = clf.train(params, train_matrix, 3000, valid_sets=[train_matrix, valid_matrix], categorical_feature=[])

        # 获取验证和测试集的预测值
        val_pred = model.predict(val_x, num_iteration=model.best_iteration)
        test_pred = model.predict(test_x, num_iteration=model.best_iteration)

        oof[valid_index] = val_pred
        test_predict += test_pred / kf.n_splits

        # 计算并打印当前折的分数
        score = np.sqrt(mean_squared_error(val_pred, val_y))
        cv_scores.append(score)
        print(cv_scores)

    return oof, test_predict

# 调用上面的函数进行模型训练和预测
lgb_oof, lgb_test = cv_model(lgb, train_df[cols], train_df['power'], test_df[cols])

这里使用的模型是lgb模型,这里可以用下面的链接对模型进行一定的了解

lgb模型参考链接

5.结果保存

#输出赛题提交格式的结果
test_df['power'] = lgb_test
test_df['power'] = test_df['power'].apply(lambda x: 0 if x<0 else x)
test_df[['id_encode','ds','power']].to_csv('result.csv', index=False)