模型原型
class sklearn.tree.DecisionTreeRegressor(criterion=’mse’,splitter=’best’,max_depth=None,
min_samples_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,
max_features=None,random_state=None,max_leaf_nodes=None,presor=False)
参数
- criterion:指定切分质量的评分准则
- splitter:指定切分原则
- ’best’:选择最优的切分
- ‘random’:随机切分
- max_depth:树的最大深度
- min_samples_split:指定每个内部节点(非叶节点)包含的最少的样本数
- min_samples_leaf:指定每个叶子节点包含的最少样本数
- min_weight_fraction_leaf:叶子节点中样本的最小权重系数
- max_features:寻找best split时考虑的特征数量
- 整数:每次切分只考虑max_features个特征
- 浮点数:每次切分只考虑max_features*n_features个特征(max_features指定了百分比)
- ‘auto’或’sqrt’:max_features=n_features
- ‘log2’:max_features=log2_(n_features)
- ‘None’:max_features=n_features
- random_state
- max_leaf_nodes:指定叶节点的最大数量
- presor:是否提前排序数据从而加速寻找最优切分的过程(True:大数据集会减慢总体的训练过程;小数据集或设定了最大深度的情况下,会加速训练过程)
属性
- featureimportances:给定特征的重要程度(又称Gini importance)
- maxfeatures:max_features的推断值
- nfeatures:fit之后,特征的数量
- noutputs:fit之后,输出的数量
- tree_:底层的决策树
方法
- fit(X,y[,sample_weight,check_input,…])
- predict(X[,check_input])
- score(X,y[,sample_weight])
import numpy as np
from sklearn.tree import DecisionTreeRegressor
from sklearn import cross_validation
import matplotlib.pyplot as plt
产生随机的数据集
def creat_data(n):
np.random.seed(0)
X=5*np.random.rand(n,1)
y=np.sin(X).ravel()
noise_num=(int)(n/5)
y[::5]+=3*(0.5-np.random.rand(noise_num))
return cross_validation.train_test_split(X,y,test_size=0.25,random_state=1)
使用DecisionTreeRegressor
def test_DecisionTreeRegressor(*data):
X_train,X_test,y_train,y_test=data
regr=DecisionTreeRegressor()
regr.fit(X_train,y_train)
print('Training score:%f'%(regr.score(X_train,y_train)))
print('Testing score:%f'%(regr.score(X_test,y_test)))
#绘图
fig=plt.figure()
ax=fig.add_subplot(1,1,1)
X=np.arange(0.0,5.0,0.01)[:,np.newaxis]
Y=regr.predict(X)
ax.scatter(X_train,y_train,label='train sample',c='g')
ax.scatter(X_test,y_test,label="test sample",c='r')
ax.plot(X,Y,label='predict_value',linewidth=2,alpha=0.5)
ax.set_xlabel('data')
ax.set_ylabel('target')
ax.set_title('Decision Tree Regresion')
ax.legend(framealpha=0.5)
plt.show()
X_train,X_test,y_train,y_test=creat_data(100)
test_DecisionTreeRegressor(X_train,X_test,y_train,y_test)
检验随机划分和最优化的影响
def test_DecisionTreeRegressor_splitter(*data):
X_train,X_test,y_train,y_test=data
splitters=['best','random']
for splitter in splitters:
regr=DecisionTreeRegressor(splitter=splitter)
regr.fit(X_train,y_train)
print('Splitter %s'%splitter)
print('Train score:%f'%(regr.score(X_train,y_train)))
print('Testint score:%f'%(regr.score(X_test,y_test)))
X_train,X_test,y_train,y_test=creat_data(100)
test_DecisionTreeRegressor_splitter(X_train,X_test,y_train,y_test)
决策树深度的影响
def test_DecisionTreeRegressor_depth(*data,maxdepth):
X_train,X_test,y_train,y_test=data
depths=np.arange(1,maxdepth)
training_scores=[]
testing_scores=[]
for depth in depths:
regr=DecisionTreeRegressor(max_depth=depth)
regr.fit(X_train,y_train)
training_scores.append(regr.score(X_train,y_train))
testing_scores.append(regr.score(X_test,y_test))
#绘图
fig=plt.figure()
ax=fig.add_subplot(1,1,1)
ax.plot(depths,training_scores,label='training score')
ax.plot(depths,testing_scores,label='testing score')
ax.set_xlabel('maxdepth')
ax.set_ylabel('score')
ax.set_title('Decision Tree Regression')
ax.legend(framealpha=0.5)
plt.show()
X_train,X_test,y_train,y_test=creat_data(100)
test_DecisionTreeRegressor_depth(X_train,X_test,y_train,y_test,maxdepth=20)