深度学习前期准备：数据处理/结果展示

1 数据标准化

        使用sklearn

• MinMaxScaler：将特征缩放到指定的最小值和最大值之间（通常是 0 和 1）

• StandardScaler：将特征缩放到均值为 0，方差为 1 的分布。

from sklearn.preprocessing import MinMaxScaler,StandardScaler

# MinMaxScaler
scaler1 = MinMaxScaler()
scaled_data = scaler1.fit_transform(data)

# StandardScaler
scaler2 = StandardScaler()
standardized_data = scaler2.fit_transform(data)

2 数据集拆分

  使用 sklearn 的 train_test_split

from sklearn.model_selection import train_test_split 

# X 是特征数据，y 是标签数据 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,\
                                            shuffle=True, random_state=42)

使用pandas/numpy自定义拆分先打乱数据后按照长度切割或者先打乱索引再切割索引，返回索引对应的数据集

import pandas as pd
from sklearn.utils import shuffle 

# 打乱数据 
data = shuffle(data, random_state=42) 
# 按比例拆分数据 
train_size = int(0.8 * len(data)) 
train = data[:train_size] 
test = data[train_size:]

import numpy as np 

# 打乱数据 
indices = np.arange(len(data)) 
np.random.shuffle(indices) 
train_size = int(0.8 * len(data)) 
train_indices = indices[:train_size] 
test_indices = indices[train_size:] 
X_train, X_test = X[train_indices], X[test_indices] 
y_train, y_test = y[train_indices], y[test_indices]

3 参数择优

GridSearchCV   (在sklearn.model_selection中）

grid_search = GridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, cv=None, verbose=0, return_train_score=False)

参数：estimator：要调优的算法

param_grid：网格参数，字典或字典列表，键是模型参数名称，值是待搜索的参数值列表

scoring：用于模型评分标准

n_jobs：并行作业数量

cv=None,交叉验证折数

verbose=0,控制信息输出量，>1表示每次拟合时输出更详细信息

refit：在整个数据集上重新拟合使用最佳参数组合的模型

Datafame对象的pivot属性

scores_matrix = results.pivot(index, column, value)

index：行；column：列，value：表格里的值或者聚合函数(例如多个符合条件的值求mean)

实例：

from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.svm import SVC


# 初始化SVM模型
svc = SVC(kernel='rbf', random_state=42)

# 网格搜索进行超参数优化
param_grid = {
    'C': [0.1, 1, 10, 100],
    'gamma': [1, 0.1, 0.01, 0.001]
}

grid_search = GridSearchCV(svc, param_grid, refit=True, verbose=3, cv=5, n_jobs=-1)
grid_search.fit(X_train, y_train)

results = pd.DataFrame(grid_search.cv_results_)
scores_matrix = results.pivot("param_C", "param_gamma", "mean_test_score")

4 可视化

heatmap热图显示二维表格，适用于混淆矩阵、透视图等

import seaborn as sns

sns.heatmap(conf_matrix, annot=True, cmap="Blues", fmt="d")
sns.heatmap(scores_matrix, annot=True, cmap="YlGnBu")

5 分类问题结果展示 

from sklearn.metrics import confusion_matrix

混淆矩阵 confusion_matrix(y,y')