机器学习-决策树

决策树
0. 加载相关模块
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
import seaborn as sns
from sklearn.metrics import accuracy_score

# 用于在jupyter中进行绘图
%matplotlib inline
1. 加载数据
iris = load_iris()
1.1 数据预览
print('特征名称：', iris.feature_names)
特征名称： ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
print('类别：', iris.target_names)
类别： ['setosa' 'versicolor' 'virginica']
1.2 数据处理
X = iris.data
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=1/4, random_state=0)

print('数据集样本数：{}，训练集样本数：{}，测试集样本数：{}'.format(len(X), len(X_train), len(X_test)))
数据集样本数：150，训练集样本数：112，测试集样本数：38
2. 建立模型
dt_model = DecisionTreeClassifier(max_depth=3)
3. 训练模型
dt_model.fit(X_train, y_train)
DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=3,
                       max_features=None, max_leaf_nodes=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, presort=False,
                       random_state=None, splitter='best')
4. 测试模型
y_pred = dt_model.predict(X_test)
acc = accuracy_score(y_test, y_pred)
print('准确率：', acc)
准确率： 0.9736842105263158
5. 查看超参数的影响
max_depth_values = [2, 3, 4]

for max_depth_val in max_depth_values:
    dt_model = DecisionTreeClassifier(max_depth=max_depth_val)
    dt_model.fit(X_train, y_train)
    
    print('max_depth=', max_depth_val)
    print('训练集上的准确率: {:.3f}'.format(dt_model.score(X_train, y_train)))
    print('测试集的准确率: {:.3f}'.format(dt_model.score(X_test, y_test)))
    print()
max_depth= 2
训练集上的准确率: 0.964
测试集的准确率: 0.895

max_depth= 3
训练集上的准确率: 0.982
测试集的准确率: 0.974

max_depth= 4
训练集上的准确率: 1.000
测试集的准确率: 0.974

6. 决策树可视化
需要安装:

graphviz程序(已提供在代码目录下)，并将安装目录下的bin目录添加到环境变量中，重启jupyter或系统生效。如：C:\Program Files (x86)\Graphviz2.38\bin 添加到系统PATH环境变量中。

graphviz模块, pip install graphviz

from ml_visualization import plot_decision_tree

dt_model = DecisionTreeClassifier(max_depth=4)
dt_model.fit(X_train, y_train)
plot_decision_tree(dt_model, iris.feature_names, iris.target_names)
---------------------------------------------------------------------------

print(iris.feature_names)
print(dt_model.feature_importances_)
from ml_visualization import plot_feature_importances
plot_feature_importances(dt_model, iris.feature_names)