《机器学习实战》——决策树

详细代码及数据已上传到githubhttps://github.com/SONGSONG729/decision-tree

按照书本代码进行实验时的报错:

  1. 在使用Matplotlib注解时,出现了错误:Backend Qt5Agg is interactive backend. Turning interactive mode on.
    错误原因:matplotlib的默认backend是TkAgg,而FltkAgg, GTK, GTKAgg, GTKCairo, TkAgg , Wx or WxAgg这几个backend都要求有GUI图形界面的,所以在ssh操作的时候会报错。
    解决办法:指定不需要GUI的backend(Agg, Cairo, PS, PDF or SVG),在plt.show()之前,加上plt.switch_backend(‘agg’)

  2. 在使用pickle模块存储决策树时,出现了错误:write() argument must be str, not bytes
    错误原因:fw这个变量的类型应该是bytes的
    解决办法:将’w’改为’wb’。
    编译后提示UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 0: illegal multibyte sequence错误,是由于之前保存的txt文件是用二进制写入的所以在读取时也应该用二进制,即读文件中的fr = open(filename)应该改为fr = open(filename, 'rb')

  3. 使用决策树预测隐形眼镜类型

# _*_ coding:utf-8 _*_
from math import log
import operator
import matplotlib.pyplot as plt
import treePlotter

def calcShannonEnt(dataSet):
    '''
    计算给定数据集的香农公式
    :param dataSet:
    :return:
    '''
    numEntries = len(dataSet)
    labelCounts = {}
    # 为所有可能分类创建字典
    for featVec in dataSet:
        currentLabel = featVec[-1]
        if currentLabel not in labelCounts.keys():
            labelCounts[currentLabel] = 0
        labelCounts[currentLabel] += 1
    shannonEnt = 0.0
    for key in labelCounts:
        prob = float(labelCounts[key])/numEntries
        shannonEnt -= prob * log(prob, 2)  # 以2为底求对数
    return shannonEnt
def createDataSet():
    '''
    简单鉴定数据集
    :return:
    '''
    dataSet = [[1, 1, 'yes'],
               [1, 1, 'yes'],
               [1, 0, 'no'],
               [0, 1, 'no'],
               [0, 1, 'no']]
    labels = ['no surfacing', 'flippers']
    return dataSet, labels

def splitDataSet(dataSet, axis, value):
    '''
    按照给定特征划分数据集
    :param dataSet: 待划分的数据集
    :param axis: 划分数据集的特征
    :param value: 需要返回的特征值
    :return:
    '''
    retDataSet = []  # 创建新的lise对象,不修改原列表
    for featVec in dataSet:
        # 将符合特征的数据抽取出来
        if featVec[axis] == value:
            reduceFeatVec = featVec[:axis]
            reduceFeatVec.extend(featVec[axis+1:])
            retDataSet.append(reduceFeatVec)
    return retDataSet

def chooseBestFeatureToSplit(dataSet):
    '''
    选择最好的数据集划分方式
    :param dataSet:
    :return:
    '''
    numFeatures = len(dataSet[0]) - 1
    baseEntropy = calcShannonEnt(dataSet)
    bestInfoGain = 0.0
    bestFeature = -1
    for i in range(numFeatures):
        # 创建唯一的分类标签列表
        featList = [example[i] for example in dataSet]
        uniqueVals = set(featList)

        newEntropy = 0.0
        # 计算每种划分方式的信息熵
        for value in uniqueVals:
            subDataSet = splitDataSet(dataSet, i, value)
            prob = len(subDataSet)/float(len(dataSet))
            newEntropy += prob * calcShannonEnt(subDataSet)
        infoGain = baseEntropy - newEntropy
        # 计算最好的信息增益
        if infoGain > bestInfoGain:
            bestInfoGain = infoGain
            bestFeature = i
    return bestFeature

def majorityCnt(classList):
    '''
    :param classList: 分类名称的列表
    :return: 出现次数最多的分类名称
    '''
    # key:classList中唯一值的数据字典
    # value:classList中每个类标签出现的频率
    classCount = {}
    for vote in classList:
        if vote not in classCount.keys():
            classCount[vote] = 0
        classCount[vote] += 1
    # 用operator操作键值排序字典
    sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True)
    return sortedClassCount[0][0]

def createTree(dataSet, labels):
    '''
    创建树
    :param dataSet: 数据集
    :param labels: 标签列表,包含数据集中所有特征的标签
    :return:
    '''
    # classList列表变量包含了数据集的所有类标签
    classList = [example[-1] for example in dataSet]
    # 所有的类标签完全相同则停止划分,返回该类标签
    # 第一个元素的值的数量等于整个列表的长度,即说明整个列表都是这个值,所以该数据集类别全部相同了
    if classList.count(classList[0]) == len(classList):
        return classList[0]
    # 所有特征已经利用完,仍然不能将数据集划分成仅包含唯一类别的分组,返回出现次数最多的类别作为返回值
    # 所有特征已经利用完,只剩下标签列,仍然无法区分剩余样本,则采用“少数服从多数”的方案
    if len(dataSet[0]) == 1:
        return majorityCnt()

    bestFeat = chooseBestFeatureToSplit(dataSet)  # 当前数据集选取的最好特征
    bestFeatLabel = labels[bestFeat]
    myTree = {bestFeatLabel: {}}  # 存储树的所有信息
    # 得到列表包含的所有属性值
    del(labels[bestFeat])
    featValues = [example[bestFeat] for example in dataSet]
    uniqueVals = set(featValues)
    # 遍历当前选择特征包含的所有属性值,在每个数据集划分上递归调用函数createTree(),
    # 得到的返回值将被插入到字典变量myTree中,
    # 函数终止时,字典中将会嵌套很多代表叶子节点信息的字典数据
    for value in uniqueVals:
        subLabels = labels[:]  # 复制了类标签,并将其存储在新列表变量subLabels中
        myTree[bestFeatLabel][value] = createTree(splitDataSet(dataSet, bestFeat, value), subLabels)
    return myTree

def classify(inputTree, featLabels, testVec):
    '''
    使用决策树的分类函数(递归函数)
    :param inputTree:
    :param featLabels:
    :param testVec:
    :return:
    '''
    firstStr = list(inputTree.keys())[0]
    secondDict = inputTree[firstStr]
    featIndex = featLabels.index(firstStr)  # 将标签字符串转换为索引
    for key in secondDict.keys():
        if testVec[featIndex] == key:
            if type(secondDict[key]).__name__ == 'dict':
                classLabel = classify(secondDict[key], featLabels, testVec)
            else:
                classLabel = secondDict[key]
    return classLabel

def storeTree(inputTree, filename):
    import pickle
    fw = open(filename, 'wb')
    pickle.dump(inputTree, fw)
    fw.close()

def grabTree(filename):
    import pickle
    fr = open(filename, 'rb')
    return pickle.load(fr)

def main():
    '''
    myDat, labels = createDataSet()
    print(myDat)
    print(calcShannonEnt(myDat))
    '''
    '''
    myDat, labels = createDataSet()
    print(myDat)
    print(splitDataSet(myDat, 0, 1))
    print(splitDataSet(myDat, 0, 0))
    '''
    '''
    myDat, labels = createDataSet()
    print(chooseBestFeatureToSplit(myDat))
    print(myDat)
    '''
    '''
    myDat, labels = createDataSet()
    myTree = createTree(myDat, labels)
    print(myTree)
    '''
    """
    myDat, labels = createDataSet()
    print(labels)
    myTree = treePlotter.retrieveTree(0)
    print(myTree)
    print(classify(myTree, labels, [1, 0]))
    print(classify(myTree, labels, [1, 1]))
    """
    '''
    myTree = treePlotter.retrieveTree(0)
    storeTree(myTree, 'classifierStorage.txt')
    grabTree('classifierStorage.txt')
    '''

    # with open('lenses.txt') as fr:
    fr = open('lenses.txt')
    lenses = [inst.strip().split('\t') for inst in fr.readlines()]
    lensesLables = ['age', 'prescript', 'astigmatic', 'tearRate']
    lensesTree = createTree(lenses, lensesLables)
    print(lensesTree)
    treePlotter.createPlot(lensesTree)




if __name__ == '__main__':
    main()

  1. 在Python中使用Matplotlib注解绘制树形结构(treePlotter.py)
import matplotlib.pyplot as plt
from pylab import mpl

# 用以显示中文
mpl.rcParams['font.sans-serif'] = ['FangSong']

'''
使用文本注解绘制树节点
'''
decisionNode = dict(boxstyle="sawtooth", fc="0.8")
leafNode = dict(boxstyle="round4", fc="0.8")
arrow_args = dict(arrowstyle="<-")

def plotNode(nodeTxt, centerPt, parentPt, nodeType):
    createPlot.axl.annotate(nodeTxt, xy=parentPt, xycoords='axes fraction', xytext=centerPt,
                            textcoords='axes fraction', va="center", ha="center",
                            bbox=nodeType, arrowprops=arrow_args)

def createPlot():
    fig = plt.figure(1, facecolor='white')
    fig.clf()
    createPlot.axl = plt.subplot(111, frameon=True)
    plotNode('决策节点', (0.5, 0.1), (0.1, 0.5), decisionNode)
    plotNode('叶节点', (0.8, 0.1), (0.3, 0.8), leafNode)
    plt.switch_backend('agg')
    plt.show()

def getNumLeafs(myTree):
    '''
    遍历整棵树,累计叶子节点的个数,并返回该值
    :param myTree:
    :return:
    '''
    numLeafs = 0
    # in the py3, type(myTree.keys()) is dict_keys.
    # It can be directed used for iteration, but index is not suitable.
    # in the py2, type(myTree.keys()) is list.
    firstStr = list(myTree.keys())[0]
    secondDict = myTree[firstStr]
    for key in secondDict.keys():
        if type(secondDict[key]).__name__ == 'dict':
            numLeafs += getNumLeafs(secondDict[key])
        else:
            numLeafs += 1
    return numLeafs

def getTreeDepth(myTree):
    '''
    计算遍历过程中遇到判断节点的个数
    :param myTree:
    :return:
    '''
    maxDepth = 0
    firstStr = list(myTree.keys())[0]
    secondDict = myTree[firstStr]
    for key in secondDict.keys():
        if type(secondDict[key]).__name__ == 'dict':
            thisDepth = 1 + getTreeDepth(secondDict[key])
        else:
            thisDepth = 1
        if thisDepth > maxDepth:
            maxDepth = thisDepth
    return maxDepth

def retrieveTree(i):
    listOfTrees = [{'no surfacing': {0: 'no', 1: {'flippers': {0: 'no', 1: 'yes'}}}},
                   {'no surfacing': {0: 'no', 1: {'flippers': {0: {'head': {0: 'no', 1: 'yes'}}, 1: 'no'}}}}
                   ]
    return listOfTrees[i]

def plotMidText(cntrPt, parentPt, txtString):
    '''
    在父子节点间填充文本信息
    :param cntrPt:
    :param parentPt:
    :param txtString:
    :return:
    '''
    xMid = (parentPt[0] - cntrPt[0])/2.0 + cntrPt[0]
    yMid = (parentPt[1] - cntrPt[1])/2.0 + cntrPt[1]
    createPlot.axl.text(xMid, yMid, txtString)

def plotTree(myTree, parentPt, nodeTxt):
    '''
    绘制树形图
    plotTree.xOff、plotTree.yOff:全局变量,追踪已绘制的节点位置
    :param myTree:
    :param parentPt:
    :param nodeTxt:
    :return:
    '''
    numLeafs = getNumLeafs(myTree)
    depth = getTreeDepth(myTree)  # 计算宽与高
    firstStr = list(myTree.keys())[0]
    cntrPt = (plotTree.xOff + (1.0 + float(numLeafs))/2.0/plotTree.totalW, plotTree.yOff)
    plotMidText(cntrPt, parentPt, nodeTxt)  # 标记子节点属性值
    plotNode(firstStr, cntrPt, parentPt, decisionNode)
    secondDict = myTree[firstStr]
    plotTree.yOff = plotTree.yOff - 1.0/plotTree.totalD  # 减少y偏移,自顶向下绘制图形,一次递减y的坐标
    for key in secondDict.keys():
        if type(secondDict[key]).__name__ == 'dict':
            plotTree(secondDict[key], cntrPt, str(key))
        else:
            plotTree.xOff = plotTree.xOff + 1.0/plotTree.totalW
            plotNode(secondDict[key], (plotTree.xOff, plotTree.yOff),cntrPt, leafNode)
            plotMidText((plotTree.xOff, plotTree.yOff), cntrPt, str(key))
    plotTree.yOff = plotTree.yOff + 1.0/plotTree.totalD


def createPlot(inTree):
    '''
    创建绘图区,计算树图形的尺寸,并调用递归函数plotTree()
    :param inTree:
    :return:
    '''
    fig = plt.figure(1, facecolor='white')
    fig.clf()
    axprops = dict(xticks=[], yticks=[])
    createPlot.axl = plt.subplot(111, frameon=False, **axprops)
    plotTree.totalW = float(getNumLeafs(inTree))
    plotTree.totalD = float(getTreeDepth(inTree))
    plotTree.xOff = -0.5/plotTree.totalW
    plotTree.yOff = 1.0
    plotTree(inTree, (0.5, 1.0), '')
    plt.show()

运行tree.py后的结果
在这里插入图片描述

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
决策树算法是一种广泛应用于分类和回归的机器学习算法,它基于树形结构对样本进行分类或预测。决策树算法的主要思想是通过一系列的判断来对样本进行分类或预测。在决策树中,每个节点表示一个属性或特征,每个分支代表该属性或特征的一个取值,而每个叶子节点代表一个分类或预测结果。 决策树算法的训练过程主要包括以下步骤: 1. 特征选择:根据某种指标(如信息增益或基尼系数)选择最优的特征作为当前节点的分裂属性。 2. 决策树生成:根据选择的特征将数据集分成若干个子集,并递归地生成决策树。 3. 剪枝:通过剪枝操作来提高决策树的泛化性能。 决策树算法的优点包括易于理解和解释、计算复杂度较低、对缺失值不敏感等。但是,决策树算法也存在一些缺点,如容易出现过拟合、对离散数据敏感等。 下面是一个决策树算法的案例:假设我们要根据一个人的年龄、性别、教育程度和职业预测其收入水平(高于或低于50K)。首先,我们需要将这些特征进行编码,将其转换为数值型数据。然后,我们可以使用决策树算法对这些数据进行训练,并生成一个决策树模型。最后,我们可以使用该模型对新的数据进行分类或预测。例如,根据一个人的年龄、性别、教育程度和职业,我们可以使用决策树模型预测该人的收入水平。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值