机器学习小白入门——决策树和K-近邻

1、先来一个简单的练手，关于机器学习中，灰狗和拉布拉多的身高特征分类，代码如下：

>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> greyhounds=500
>>> labs=500
>>> grey_height=28+4*np.random.randn(greyhounds)
>>> lab_height=24+4*np.random.randn(labs)
>>> plt.hist([grey_height,lab_height],stacked=True,color=['r','b'])
([array([  0.,   3.,  23.,  59., 106., 162.,  88.,  46.,  12.,   1.]), array([  6.,  33., 106., 198., 249., 236., 106.,  53.,  12.,   1.])], array([12.59903609, 15.48870961, 18.37838313, 21.26805665, 24.15773017,
       27.04740369, 29.93707721, 32.82675073, 35.71642425, 38.60609777,
       41.49577129]), <a list of 2 Lists of Patches objects>)
>>> plt.show()
>>> exit()

2、编写一条基础的监督式学习的管道，展示关于不同的分类器如何解决同一个问题，以下代码为决策树和K-近邻的比较。

1、决策树：

>>> from sklearn import datasets
>>> iris=datasets.load_iris()
>>> xx=iris.data
>>> yy=iris.target
>>> from sklearn.cross_validation import train_test_split
E:\Anaconda\lib\site-packages\sklearn\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
>>> X_train,X_test,y_train,y_test=train_test_split(xx,yy,test_size=.5)
>>> from sklearn import tree
>>> my_classifier=tree.DecisionTreeClassifier()
>>> my_classifier.fit(X_train,y_train)
DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
            max_features=None, max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=1, min_samples_split=2,
            min_weight_fraction_leaf=0.0, presort=False, random_state=None,
            splitter='best')
>>> predictions=my_classifier.predict(X_test)
>>> print(predictions)
[0 2 2 0 2 2 1 0 2 2 1 0 2 2 2 1 1 0 0 2 0 2 2 0 2 0 2 1 0 1 0 2 0 0 2 1 1
 0 1 1 1 1 2 0 1 2 2 1 1 0 2 2 2 0 1 0 0 2 2 0 1 0 1 2 0 2 0 2 0 2 0 0 0 1
 2]
>>> from sklearn.metrics import accuracy_score
>>> print(accuracy_score(y_test,predictions))
0.9466666666666667

2、K最近邻居分类器
将上述代码段中间关于决策树分类器的代码换掉，即

>>> from sklearn import tree
>>> my_classifier=tree.DecisionTreeClassifier()

替换为以下

>>> from sklearn.neighbors import KNeighborsClassifier
>>> my_classifier=KNeighborsClassifier()

继续按照1中代码运行。

3、两次运行甚至多次运行会得到不同的准确度？

原因有二：
第一，分类器的工作原理是有区别的；
第二，每次训练集和测试集的拆分是随机的。