【人工智能】习题——监督学习

• [决策树] 基于信息增益，对下述数据集进行决策树构建，描述过程

一个关于配眼镜的一个决策分类所需要的数据，数据集包含4属性：age, astigmatism, trear-prod-rate为输入特征，contact-lenses为决策属性。

以属性a对数据集D进行划分所获得的信息增益为：
$$G(D,a)=H(D)-\sum _{v=1}^V\frac{|D^v|}{D}\cdot H(D^v)$$
可知，划分后的信息熵越小，则获得的信息增益越大，从而选择使得当前样本集合信息增益最大的属性a作为当前的划分属性

1. 第一次划分

   1. age

      age\contact-lenses	none	soft	hard	sum
      young	1	1	1	3
      pre-presbyopic	3	1	1	5
      presbyopic	3	0	1	4

      $D^1$(young)，$D^2$(pre-presbyopic)，$D^3$(presbyopic)
      $$\begin{array}{rl}H(D^1)=& -(\frac{1}{3}\log \frac{1}{3}+\frac{1}{3}\log \frac{1}{3}+\frac{1}{3}\log \frac{1}{3})=1.58\\ H(D^2)=& -(\frac{3}{5}\log \frac{3}{5}+\frac{1}{5}\log \frac{1}{5}+\frac{1}{5}\log \frac{1}{5})=1.37\\ H(D^3)=& -(\frac{3}{4}\log \frac{3}{4}+\frac{1}{4}\log \frac{1}{4})=0.81\\ H(D{)}_{age}=& \frac{3}{12}H(D^1)+\frac{5}{12}H(D^2)+\frac{4}{12}H(D^3)=1.24\end{array}$$

   2. astigmatism

      astigmatism\contact-lenses	none	soft	hard	sum
      yes	4	0	3	7
      no	3	2	0	5

      $D^1$(yes)，$D^2$(no)
      $$\begin{array}{rl}H(D^1)=& -(\frac{4}{7}\log \frac{4}{7}+\frac{3}{7}\log \frac{3}{7})=0.99\\ H(D^2)=& -(\frac{3}{5}\log \frac{3}{5}+\frac{2}{5}\log \frac{2}{5})=0.97\\ H(D{)}_{astigmatism}=& \frac{7}{12}H(D^1)+\frac{5}{12}H(D^2)=0.98\end{array}$$

   3. trear-prod-rate

      trear-prod-rate\contact-lenses	none	soft	hard	sum
      normal	3	2	3	8
      reduced	4	0	0	4

      $D^1$(normal)，$D^2$(reduced)
      $$\begin{array}{rl}H(D^1)=& -(\frac{3}{8}\log \frac{3}{8}+\frac{2}{8}\log \frac{2}{8}+\frac{3}{8}\log \frac{3}{8})=1.56\\ H(D^2)=& -\frac{4}{4}\log \frac{4}{4}=0\\ H(D{)}_{trear-prod-rate}=& \frac{8}{12}H(D^1)+\frac{4}{12}H(D^2)=1.04\end{array}$$

   根据astigmatism划分后的信息熵最小，相应的信息增益最大，被选为第一次划分的属性

   第一次划分后的决策

   

2. 第二次划分

   对$D_1$进行划分

   ID	age	trear-prod-rate	contact-lenses
   2	young	reduced	none
   3	young	normal	hard
   6	pre-presbyopic	normal	hard
   7	pre-presbyopic	normal	none
   8	pre-presbyopic	normal	none
   11	presbyopic	reduced	none
   12	presbyopic	normal	hard

   1. age

      age\contact-lenses	none	soft	hard	sum
      young	1	0	1	2
      pre-presbyopic	2	0	1	3
      presbyopic	1	0	1	2

      $D^{11}$(young)，$D^{12}$(pre-presbyopic)，$D^{13}$(presbyopic)
      $$\begin{array}{rl}H(D^{11})=& -(\frac{1}{2}\log \frac{1}{2}+\frac{1}{2}\log \frac{1}{2})=1\\ H(D^{12})=& -(\frac{2}{3}\log \frac{2}{3}+\frac{1}{3}\log \frac{1}{3})=0.92\\ H(D^{13})=& -(\frac{1}{2}\log \frac{1}{2}+\frac{1}{2}\log \frac{1}{2})=1\\ H(D{)}_{age}=& \frac{2}{7}H(D^{11})+\frac{3}{7}H(D^{12})+\frac{2}{7}H(D^{13})=0.97\end{array}$$

   2. trear-prod-rate

      trear-prod-rate\contact-lenses	none	soft	hard	sum
      normal	2	0	3	5
      reduced	2	0	0	2

      $D^{11}$(normal)，$D^{12}$(reduced)
      $$\begin{array}{rl}H(D^{11})=& -(\frac{2}{5}\log \frac{2}{5}+\frac{3}{5}\log \frac{3}{5})=0.97\\ H(D^{12})=& -\frac{2}{2}\log \frac{2}{2}=0\\ H(D{)}_{trear-prod-rate}=& \frac{5}{7}H(D^{11})+\frac{2}{7}H(D^{12})=0.69\end{array}$$

   根据trear-prod-rate划分后的信息熵最小，相应的信息增益最大，trear-prod-rate被选为对$D^1$进行划分的属性

   

   对$D^2$进行划分

   ID	age	trear-prod-rate	contact-lenses
   1	young	normal	soft
   4	pre-presbyopic	reduced	none
   5	pre-presbyopic	normal	soft
   9	presbyopic	reduced	none
   10	presbyopic	normal	none

   1. age

      age\contact-lenses	none	soft	hard	sum
      young	0	1	0	1
      pre-presbyopic	1	1	0	2
      presbyopic	2	0	0	2

      $D^{21}$(young)，$D^{22}$(pre-presbyopic)，$D^{23}$(presbyopic)
      $$\begin{array}{rl}H(D^{21})=& -\frac{1}{1}\log \frac{1}{1}=0\\ H(D^{22})=& -(\frac{1}{2}\log \frac{1}{2}+\frac{1}{2}\log \frac{1}{2})=1\\ H(D^{23})=& -\frac{2}{2}\log \frac{2}{2}=0\\ H(D{)}_{age}=& \frac{2}{5}H(D^{12})=0.4\end{array}$$

   2. trear-prod-rate

      trear-prod-rate\contact-lenses	none	soft	hard	sum
      normal	1	2	0	3
      reduced	2	0	0	2

      $D^{21}$(normal)，$D^{22}$(reduced)
      $$\begin{array}{rl}H(D^{21})=& -(\frac{1}{3}\log \frac{1}{3}+\frac{2}{3}\log \frac{2}{3})=0.92\\ H(D^{22})=& -\frac{2}{2}\log \frac{2}{2}=0\\ H(D{)}_{trear-prod-rate}=& \frac{3}{5}H(D^{21})=0.55\end{array}$$

   根据age划分后的信息熵最小，相应的信息增益最大，age被选为对$D^2$进行划分的属性