2.4决策树之决策树实例

最新推荐文章于 2024-04-07 09:23:01 发布
最新推荐文章于 2024-04-07 09:23:01 发布
阅读量4.7k 收藏
点赞数
分类专栏： 机器学习 分类 文章标签： 机器学习 决策树
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_16365849/article/details/50644540
版权

实例一、rpart包对iris数据进行分类

#1、加载数据并查看
data("iris")
str(iris)
## 'data.frame':    150 obs. of  5 variables:
##  $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
##  $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
##  $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
##  $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
##  $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...
summary(iris)
##   Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
##  Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
##  1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
##  Median :5.800   Median :3.000   Median :4.350   Median :1.300  
##  Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
##  3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
##  Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
##        Species  
##  setosa    :50  
##  versicolor:50  
##  virginica :50  
##                 
##                 
## 
#总结：从上可知，共计150条记录，5个变量，其中，4个自变量为数字类型，因变量为因子类型；无缺失值

#2、创建训练集和测试集数据
set.seed(123)
library(caret)
## Loading required package: lattice
## Loading required package: ggplot2
## Warning: package 'ggplot2' was built under R version 3.2.3
index <- createDataPartition(iris$Species, p=0.8, list=F)
train_iris <- iris[index, ]
test_iris <- iris[-index, ]

#3、建模
library(rpart)
model_iris <- rpart(Species~., data=train_iris, control = rpart.control(minsplit = 10))

#4、模型评估
str(model_iris)
## List of 14
##  $ frame              :'data.frame': 5 obs. of  9 variables:
##   ..$ var       : Factor w/ 2 levels "<leaf>","Petal.Length": 2 1 2 1 1
##   ..$ n         : int [1:5] 120 40 80 39 41
##   ..$ wt        : num [1:5] 120 40 80 39 41
##   ..$ dev       : num [1:5] 80 0 40 1 2
##   ..$ yval      : num [1:5] 1 1 2 2 3
##   ..$ complexity: num [1:5] 0.5 0.01 0.463 0 0
##   ..$ ncompete  : int [1:5] 3 0 3 0 0
##   ..$ nsurrogate: int [1:5] 3 0 3 0 0
##   ..$ yval2     : num [1:5, 1:8] 1 1 2 2 3 40 40 0 0 0 ...
##   .. ..- attr(*, "dimnames")=List of 2
##   .. .. ..$ : NULL
##   .. .. ..$ : chr [1:8] "" "" "" "" ...
##  $ where              : Named int [1:120] 2 2 2 2 2 2 2 2 2 2 ...
##   ..- attr(*, "names")= chr [1:120] "1" "2" "3" "4" ...
##  $ call               : language rpart(formula = Species ~ ., data = train_iris, control = rpart.control(minsplit = 10))
##  $ terms              :Classes 'terms', 'formula' length 3 Species ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width
##   .. ..- attr(*, "variables")= language list(Species, Sepal.Length, Sepal.Width, Petal.Length, Petal.Width)
##   .. ..- attr(*, "factors")= int [1:5, 1:4] 0 1 0 0 0 0 0 1 0 0 ...
##   .. .. ..- attr(*, "dimnames")=List of 2
##   .. .. .. ..$ : chr [1:5] "Species" "Sepal.Length" "Sepal.Width" "Petal.Length" ...
##   .. .. .. ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"
##   .. ..- attr(*, "term.labels")= chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"
##   .. ..- attr(*, "order")= int [1:4] 1 1 1 1
##   .. ..- attr(*, "intercept")= int 1
##   .. ..- attr(*, "response")= int 1
##   .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
##   .. ..- attr(*, "predvars")= language list(Species, Sepal.Length, Sepal.Width, Petal.Length, Petal.Width)
##   .. ..- attr(*, "dataClasses")= Named chr [1:5] "factor" "numeric" "numeric" "numeric" ...
##   .. .. ..- attr(*, "names")= chr [1:5] "Species" "Sepal.Length" "Sepal.Width" "Petal.Length" ...
##  $ cptable            : num [1:3, 1:5] 0.5 0.463 0.01 0 1 ...
##   ..- attr(*, "dimnames")=List of 2
##   .. ..$ : chr [1:3] "1" "2" "3"
##   .. ..$ : chr [1:5] "CP" "nsplit" "rel error" "xerror" ...
##  $ method             : chr "class"
##  $ parms              :List of 3
##   ..$ prior: num [1:3(1d)] 0.333 0.333 0.333
##   .. ..- attr(*, "dimnames")=List of 1
##   .. .. ..$ : chr [1:3] "1" "2" "3"
##   ..$ loss : num [1:3, 1:3] 0 1 1 1 0 1 1 1 0
##   ..$ split: num 1
##  $ control            :List of 9
##   ..$ minsplit      : num 10
##   ..$ minbucket     : num 3
##   ..$ cp            : num 0.01
##   ..$ maxcompete    : int 4
##   ..$ maxsurrogate  : int 5
##   ..$ usesurrogate  : int 2
##   ..$ surrogatestyle: int 0
##   ..$ maxdepth      : int 30
##   ..$ xval          : int 10
##  $ functions          :List of 3
##   ..$ summary:function (yval, dev, wt, ylevel, digits)  
##   ..$ print  :function (yval, ylevel, digits)  
##   ..$ text   :function (yval, dev, wt, ylevel, digits, n, use.n)  
##  $ numresp            : int 5
##  $ splits             : num [1:14, 1:5] 120 120 120 120 0 0 0 80 80 80 ...
##   ..- attr(*, "dimnames")=List of 2
##   .. ..$ : chr [1:14] "Petal.Length" "Petal.Width" "Sepal.Length" "Sepal.Width" ...
##   .. ..$ : chr [1:5] "count" "ncat" "improve" "index" ...
##  $ variable.importance: Named num [1:4] 74.2 69.9 44.8 30.8
##   ..- attr(*, "names")= chr [1:4] "Petal.Length" "Petal.Width" "Sepal.Length" "Sepal.Width"
##  $ y                  : int [1:120] 1 1 1 1 1 1 1 1 1 1 ...
##  $ ordered            : Named logi [1:4] FALSE FALSE FALSE FALSE
##   ..- attr(*, "names")= chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width"
##  - attr(*, "xlevels")= Named list()
##  - attr(*, "ylevels")= chr [1:3] "setosa" "versicolor" "virginica"
##  - attr(*, "class")= chr "rpart"
summary(model_iris)
## Call:
## rpart(formula = Species ~ ., data = train_iris, control = rpart.control(minsplit = 10))
##   n= 120 
## 
##       CP nsplit rel error xerror       xstd
## 1 0.5000      0    1.0000 1.1625 0.05717982
## 2 0.4625      1    0.5000 0.7500 0.06846532
## 3 0.0100      2    0.0375 0.0500 0.02457980
## 
## Variable importance
## Petal.Length  Petal.Width Sepal.Length  Sepal.Width 
##           34           32           20           14 
## 
## Node number 1: 120 observations,    complexity param=0.5
##   predicted class=setosa      expected loss=0.6666667  P(node) =1
##     class counts:    40    40    40
##    probabilities: 0.333 0.333 0.333 
##   left son=2 (40 obs) right son=3 (80 obs)
##   Primary splits:
##       Petal.Length < 2.45 to the left,  improve=40.00000, (0 missing)
##       Petal.Width  < 0.8  to the left,  improve=40.00000, (0 missing)
##       Sepal.Length < 5.45 to the left,  improve=25.97403, (0 missing)
##       Sepal.Width  < 3.35 to the right, improve=17.24432, (0 missing)
##   Surrogate splits:
##       Petal.Width  < 0.8  to the left,  agree=1.000, adj=1.000, (0 split)
##       Sepal.Length < 5.45 to the left,  agree=0.908, adj=0.725, (0 split)
##       Sepal.Width  < 3.35 to the right, agree=0.850, adj=0.550, (0 split)
## 
## Node number 2: 40 observations
##   predicted class=setosa      expected loss=0  P(node) =0.3333333
##     class counts:    40     0     0
##    probabilities: 1.000 0.000 0.000 
## 
## Node number 3: 80 observations,    complexity param=0.4625
##   predicted class=versicolor  expected loss=0.5  P(node) =0.6666667
##     class counts:     0    40    40
##    probabilities: 0.000 0.500 0.500 
##   left son=6 (39 obs) right son=7 (41 obs)
##   Primary splits:
##       Petal.Length < 4.75 to the left,  improve=34.246400, (0 missing)
##       Petal.Width  < 1.65 to the left,  improve=30.798240, (0 missing)
##       Sepal.Length < 6.15 to the left,  improve= 8.120301, (0 missing)
##       Sepal.Width  < 2.45 to the left,  improve= 3.657143, (0 missing)
##   Surrogate splits:
##       Petal.Width  < 1.75 to the left,  agree=0.938, adj=0.872, (0 split)
##       Sepal.Length < 6.15 to the left,  agree=0.738, adj=0.462, (0 split)
##       Sepal.Width  < 2.95 to the left,  agree=0.638, adj=0.256, (0 split)
## 
## Node number 6: 39 observations
##   predicted class=versicolor  expected loss=0.02564103  P(node) =0.325
##     class counts:     0    38     1
##    probabilities: 0.000 0.974 0.026 
## 
## Node number 7: 41 observations
##   predicted class=virginica   expected loss=0.04878049  P(node) =0.3416667
##     class counts:     0     2    39
##    probabilities: 0.000 0.049 0.951
#5、模型优化：选择具有最小误差的决策树
opt <- which.min(model_iris$cptable[, "xerror"])
cp <- model_iris$cptable[opt, "CP"]

model_cp <- prune(model_iris, cp=cp)
print(model_cp)
## n= 120 
## 
## node), split, n, loss, yval, (yprob)
##       * denotes terminal node
## 
## 1) root 120 80 setosa (0.33333333 0.33333333 0.33333333)  
##   2) Petal.Length< 2.45 40  0 setosa (1.00000000 0.00000000 0.00000000) *
##   3) Petal.Length>=2.45 80 40 versicolor (0.00000000 0.50000000 0.50000000)  
##     6) Petal.Length< 4.75 39  1 versicolor (0.00000000 0.97435897 0.02564103) *
##     7) Petal.Length>=4.75 41  2 virginica (0.00000000 0.04878049 0.95121951) *
#6、预测
pred <- predict(model_cp, test_iris, type="class")
plot(model_cp)
text(model_cp)

#7、查看预测错误的比率
mean(test_iris[, 5]!=pred)
## [1] 0.1333333
table(pred, test_iris[, 5])
##             
## pred         setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0          6         0
##   virginica       0          4        10
library(gmodels)
CrossTable(pred, test_iris[, 5])
## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  30 
## 
##  
##              | test_iris[, 5] 
##         pred |     setosa | versicolor |  virginica |  Row Total | 
## -------------|------------|------------|------------|------------|
##       setosa |         10 |          0 |          0 |         10 | 
##              |     13.333 |      3.333 |      3.333 |            | 
##              |      1.000 |      0.000 |      0.000 |      0.333 | 
##              |      1.000 |      0.000 |      0.000 |            | 
##              |      0.333 |      0.000 |      0.000 |            | 
## -------------|------------|------------|------------|------------|
##   versicolor |          0 |          6 |          0 |          6 | 
##              |      2.000 |      8.000 |      2.000 |            | 
##              |      0.000 |      1.000 |      0.000 |      0.200 | 
##              |      0.000 |      0.600 |      0.000 |            | 
##              |      0.000 |      0.200 |      0.000 |            | 
## -------------|------------|------------|------------|------------|
##    virginica |          0 |          4 |         10 |         14 | 
##              |      4.667 |      0.095 |      6.095 |            | 
##              |      0.000 |      0.286 |      0.714 |      0.467 | 
##              |      0.000 |      0.400 |      1.000 |            | 
##              |      0.000 |      0.133 |      0.333 |            | 
## -------------|------------|------------|------------|------------|
## Column Total |         10 |         10 |         10 |         30 | 
##              |      0.333 |      0.333 |      0.333 |            | 
## -------------|------------|------------|------------|------------|
## 
##

实例二、C50包中的C5.0()函数对德国信用数据分类

#1、加载数据并查看
credit <- read.csv("F:\\R\\Rworkspace\\机器学习实践课\\德国信用数据/german.data", header=F, sep=" ", stringsAsFactors=F)
str(credit)
## 'data.frame':    1000 obs. of  21 variables:
##  $ V1 : chr  "A11" "A12" "A14" "A11" ...
##  $ V2 : int  6 48 12 42 24 36 24 36 12 30 ...
##  $ V3 : chr  "A34" "A32" "A34" "A32" ...
##  $ V4 : chr  "A43" "A43" "A46" "A42" ...
##  $ V5 : int  1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
##  $ V6 : chr  "A65" "A61" "A61" "A61" ...
##  $ V7 : chr  "A75" "A73" "A74" "A74" ...
##  $ V8 : int  4 2 2 2 3 2 3 2 2 4 ...
##  $ V9 : chr  "A93" "A92" "A93" "A93" ...
##  $ V10: chr  "A101" "A101" "A101" "A103" ...
##  $ V11: int  4 2 3 4 4 4 4 2 4 2 ...
##  $ V12: chr  "A121" "A121" "A121" "A122" ...
##  $ V13: int  67 22 49 45 53 35 53 35 61 28 ...
##  $ V14: chr  "A143" "A143" "A143" "A143" ...
##  $ V15: chr  "A152" "A152" "A152" "A153" ...
##  $ V16: int  2 1 1 1 2 1 1 1 1 2 ...
##  $ V17: chr  "A173" "A173" "A172" "A173" ...
##  $ V18: int  1 1 2 2 2 2 1 1 1 1 ...
##  $ V19: chr  "A192" "A191" "A191" "A191" ...
##  $ V20: chr  "A201" "A201" "A201" "A201" ...
##  $ V21: int  1 2 1 1 2 1 1 1 1 2 ...
summary(credit)
##       V1                  V2            V3                 V4           
##  Length:1000        Min.   : 4.0   Length:1000        Length:1000       
##  Class :character   1st Qu.:12.0   Class :character   Class :character  
##  Mode  :character   Median :18.0   Mode  :character   Mode  :character  
##                     Mean   :20.9                                        
##                     3rd Qu.:24.0                                        
##                     Max.   :72.0                                        
##        V5             V6                 V7                  V8       
##  Min.   :  250   Length:1000        Length:1000        Min.   :1.000  
##  1st Qu.: 1366   Class :character   Class :character   1st Qu.:2.000  
##  Median : 2320   Mode  :character   Mode  :character   Median :3.000  
##  Mean   : 3271                                         Mean   :2.973  
##  3rd Qu.: 3972                                         3rd Qu.:4.000  
##  Max.   :18424                                         Max.   :4.000  
##       V9                V10                 V11            V12           
##  Length:1000        Length:1000        Min.   :1.000   Length:1000       
##  Class :character   Class :character   1st Qu.:2.000   Class :character  
##  Mode  :character   Mode  :character   Median :3.000   Mode  :character  
##                                        Mean   :2.845                     
##                                        3rd Qu.:4.000                     
##                                        Max.   :4.000                     
##       V13            V14                V15                 V16       
##  Min.   :19.00   Length:1000        Length:1000        Min.   :1.000  
##  1st Qu.:27.00   Class :character   Class :character   1st Qu.:1.000  
##  Median :33.00   Mode  :character   Mode  :character   Median :1.000  
##  Mean   :35.55                                         Mean   :1.407  
##  3rd Qu.:42.00                                         3rd Qu.:2.000  
##  Max.   :75.00                                         Max.   :4.000  
##      V17                 V18            V19                V20           
##  Length:1000        Min.   :1.000   Length:1000        Length:1000       
##  Class :character   1st Qu.:1.000   Class :character   Class :character  
##  Mode  :character   Median :1.000   Mode  :character   Mode  :character  
##                     Mean   :1.155                                        
##                     3rd Qu.:1.000                                        
##                     Max.   :2.000                                        
##       V21     
##  Min.   :1.0  
##  1st Qu.:1.0  
##  Median :1.0  
##  Mean   :1.3  
##  3rd Qu.:2.0  
##  Max.   :2.0
#从上可知：共计1000条记录21个变量，有字符类型也有数字类型；每列的属性名为默认名称；

#2、数据预处理
#1)对数据的列进行命名
colnames(credit) <- c("Status.of.existing.checking.account", "Duration.in.month", 
"Credit.history", "Purpose", "Credit.amount", "Savings.account.bonds", 
"Present.employment.since", 
"Installment.rate.in.percentage.of.disposable.income",
"Personal.status.and.sex", "Other.debtors.guarantors", 
"Present.residence.since","Property", "Age.in.years", 
"Other.installment.plans", "Housing", 
"Number.of.existing.credits.at.this.bank", "Job", 
"Number.of.people.being.liable.to.provide.maintenance.for", 
"Telephone", "foreign.worker", "Good.Loan")

#2）对每列的数据进行映射：
#自定义映射
mapping <- list('All'='... < 0 DM', 
'A12'='0 <= ... < 200 DM', 
'A13'='... >=200 DM / salary assignment for at least 1 year',
'A14'='no checking accout', 
'A30'='no credits taken/all credits paid back duly',
'A31'='all credits at this bank paid back duly',
'A32'='existing credits paid back duly till now', 
'A33'='delay in paying off in the past', 
'A34'='critical account/other credits existing (not at this bank)',
'A40'='car (new)',
'A41'='car (used)',
'A42'='furniture/equipment',
'A43'='radio/television', 
'A44'='domestic appliances',
'A45'='repairs',
'A46'='education',
'A47'='(vacation - does not exist?)',
'A48'='retraining', 
'A49'='business', 
'A410'='other', 
'A61'='... < 100 DM', 
'A62'='100 <= ... < 500 DM',
'A63'='500 <= ... < 1000 DM', 
'A64'='.. >= 1000 DM', 
'A65'='unknown/ no savings account',
'A71'='unemployed', 
'A72'='... < 1 year', 
'A73'='1 <= ... < 4 years', 
'A74'='4 <= ... < 7 years', 
'A75'='.. >= 7 years', 
'A91'='male : divorced/separated', 
'A92'='female : divorced/separated/married', 
'A93'='male : single', 
'A94'='male : married/widowed', 
'A95'='female : single', 
'A101'='none', 
'A102'='co-applicant',
'A103'='auarantor', 
'A121'='real estate',
'A122'='if not A121 : building society savings agreement/life insurance',
'A123'='if not A121/A122 : car or other, not in attribute 6',
'A124'='unknown / no property', 
'A141'='bank', 
'A142'='stores', 
'A143'='none', 
'A151'='rent',
'A152'='own', 
'A153'='for free',
'A171'='unemployed/ unskilled - non-resident', 
'A172'='unskilled - resident', 
'A173'='skilled employee / official', 
'A174'='management/ self=employed/hightly qualified employee/ officer', 
'A191'='none', 
'A192'='yes, registered under the customers name', 
'A201'='yes', 
'A202'='no')

#对每列的数据进行映射
for(i in 1:ncol(credit)) {
  if(class(credit[, i])=="character"){
    credit[, i] = as.factor(as.character(mapping[credit[, i]]))
  }
}

#3）对因变量进行因子转换
credit$Good.Loan <- as.factor(ifelse(credit$Good.Loan==1, "GoodLoan", "BadLoan"))

#查看清洗后的数据：查看各列的频数
table(credit$Status.of.existing.checking.account)
## 
## ... >=200 DM / salary assignment for at least 1 year 
##                                                   63 
##                                    0 <= ... < 200 DM 
##                                                  269 
##                                   no checking accout 
##                                                  394 
##                                                 NULL 
##                                                  274
table(credit$Savings.account.bonds)
## 
##               .. >= 1000 DM                ... < 100 DM 
##                          48                         603 
##         100 <= ... < 500 DM        500 <= ... < 1000 DM 
##                         103                          63 
## unknown/ no savings account 
##                         183
summary(credit$Duration.in.month)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     4.0    12.0    18.0    20.9    24.0    72.0
#3、构建测试集和训练集数据
library(caret)
set.seed(2000)
index <- createDataPartition(credit$Good.Loan, p=0.8, list=F)
train_credit <- credit[index, ]
test_credit <- credit[-index, ]
prop.table(table(train_credit$Good.Loan))
## 
##  BadLoan GoodLoan 
##      0.3      0.7
prop.table(table(test_credit$Good.Loan))
## 
##  BadLoan GoodLoan 
##      0.3      0.7
#从上可知：测试集和训练集数据的比率相同

#4、建模
library(C50)
model_credit <- C5.0(train_credit[, -21], train_credit[, 21])

#5、模型评估
model_credit
## 
## Call:
## C5.0.default(x = train_credit[, -21], y = train_credit[, 21])
## 
## Classification Tree
## Number of samples: 800 
## Number of predictors: 20 
## 
## Tree size: 60 
## 
## Non-standard options: attempt to group attributes
summary(model_credit)
## 
## Call:
## C5.0.default(x = train_credit[, -21], y = train_credit[, 21])
## 
## 
## C5.0 [Release 2.07 GPL Edition]      Mon Feb 08 12:07:02 2016
## -------------------------------
## 
## Class specified by attribute `outcome'
## 
## Read 800 cases (21 attributes) from undefined.data
## 
## Decision tree:
## 
## Status.of.existing.checking.account in {... >=200 DM / salary assignment for at least 1 year,
## :                                       no checking accout}: GoodLoan (368/49)
## Status.of.existing.checking.account in {0 <= ... < 200 DM,NULL}:
## :...Duration.in.month <= 22: [S1]
##     Duration.in.month > 22:
##     :...Present.residence.since <= 1:
##         :...Housing = for free: GoodLoan (3)
##         :   Housing = rent: BadLoan (1)
##         :   Housing = own:
##         :   :...Status.of.existing.checking.account = NULL:
##         :       :...Credit.amount <= 1943: GoodLoan (2)
##         :       :   Credit.amount > 1943: BadLoan (6/1)
##         :       Status.of.existing.checking.account = 0 <= ... < 200 DM:
##         :       :...Telephone = none: GoodLoan (9)
##         :           Telephone = yes, registered under the customers name:
##         :           :...Credit.amount <= 3878: BadLoan (2)
##         :               Credit.amount > 3878: GoodLoan (2)
##         Present.residence.since > 1:
##         :...Savings.account.bonds = .. >= 1000 DM:
##             :...Other.debtors.guarantors = co-applicant: BadLoan (1)
##             :   Other.debtors.guarantors in {auarantor,none}: GoodLoan (4)
##             Savings.account.bonds = 500 <= ... < 1000 DM:
##             :...Other.installment.plans in {bank,none}: BadLoan (4)
##             :   Other.installment.plans = stores: GoodLoan (2)
##             Savings.account.bonds = unknown/ no savings account: [S2]
##             Savings.account.bonds = 100 <= ... < 500 DM:
##             :...Other.debtors.guarantors in {auarantor,
##             :   :                            co-applicant}: BadLoan (2)
##             :   Other.debtors.guarantors = none:
##             :   :...Duration.in.month > 54: BadLoan (3)
##             :       Duration.in.month <= 54: [S3]
##             Savings.account.bonds = ... < 100 DM:
##             :...Other.debtors.guarantors = auarantor: [S4]
##                 Other.debtors.guarantors in {co-applicant,none}:
##                 :...Duration.in.month > 45: BadLoan (23/1)
##                     Duration.in.month <= 45:
##                     :...Purpose in {car (new),domestic appliances,education,
##                         :           other,repairs,
##                         :           retraining}: BadLoan (20/3)
##                         Purpose = business: [S5]
##                         Purpose = car (used):
##                         :...Age.in.years <= 28: BadLoan (4)
##                         :   Age.in.years > 28: GoodLoan (11/1)
##                         Purpose = furniture/equipment: [S6]
##                         Purpose = radio/television: [S7]
## 
## SubTree [S1]
## 
## Credit.history = delay in paying off in the past: GoodLoan (13/1)
## Credit.history in {all credits at this bank paid back duly,
## :                  no credits taken/all credits paid back duly}: BadLoan (21/6)
## Credit.history = critical account/other credits existing (not at this bank):
## :...Other.installment.plans = stores: GoodLoan (1)
## :   Other.installment.plans = bank: [S8]
## :   Other.installment.plans = none:
## :   :...Other.debtors.guarantors = auarantor: GoodLoan (3)
## :       Other.debtors.guarantors = co-applicant: BadLoan (3/1)
## :       Other.debtors.guarantors = none:
## :       :...Purpose in {business,car (new),car (used),domestic appliances,
## :           :           furniture/equipment,other,radio/television,
## :           :           retraining}: GoodLoan (44/3)
## :           Purpose in {education,repairs}: BadLoan (3)
## Credit.history = existing credits paid back duly till now:
## :...Savings.account.bonds = .. >= 1000 DM: GoodLoan (8)
##     Savings.account.bonds in {... < 100 DM,100 <= ... < 500 DM,
##     :                         500 <= ... < 1000 DM,unknown/ no savings account}: [S9]
## 
## SubTree [S2]
## 
## Status.of.existing.checking.account = 0 <= ... < 200 DM: GoodLoan (13/1)
## Status.of.existing.checking.account = NULL: BadLoan (13/3)
## 
## SubTree [S3]
## 
## Telephone = yes, registered under the customers name: GoodLoan (6)
## Telephone = none: [S10]
## 
## SubTree [S4]
## 
## Status.of.existing.checking.account = 0 <= ... < 200 DM: BadLoan (2)
## Status.of.existing.checking.account = NULL: GoodLoan (6)
## 
## SubTree [S5]
## 
## Property in {if not A121 . building society savings agreement/life insurance,
## :            real estate,unknown / no property}: BadLoan (6)
## Property = if not A121/A122 . car or other, not in attribute 6: GoodLoan (2)
## 
## SubTree [S6]
## 
## Present.employment.since in {.. >= 7 years,... < 1 year,
## :                            4 <= ... < 7 years}: BadLoan (14)
## Present.employment.since in {1 <= ... < 4 years,unemployed}:
## :...Status.of.existing.checking.account = 0 <= ... < 200 DM: BadLoan (1)
##     Status.of.existing.checking.account = NULL: GoodLoan (3)
## 
## SubTree [S7]
## 
## Status.of.existing.checking.account = 0 <= ... < 200 DM: GoodLoan (4)
## Status.of.existing.checking.account = NULL:
## :...Present.employment.since = .. >= 7 years: GoodLoan (2)
##     Present.employment.since in {... < 1 year,4 <= ... < 7 years,
##     :                            unemployed}: BadLoan (6)
##     Present.employment.since = 1 <= ... < 4 years:
##     :...Credit.amount <= 2835: BadLoan (2)
##         Credit.amount > 2835: GoodLoan (2)
## 
## SubTree [S8]
## 
## Job in {management/ self=employed/hightly qualified employee/ officer,
## :       unemployed/ unskilled - non-resident}: BadLoan (0)
## Job = unskilled - resident: GoodLoan (3)
## Job = skilled employee / official:
## :...Purpose = business: GoodLoan (1)
##     Purpose in {car (new),car (used),domestic appliances,education,
##                 furniture/equipment,other,radio/television,repairs,
##                 retraining}: BadLoan (5)
## 
## SubTree [S9]
## 
## Property = real estate: GoodLoan (57/14)
## Property = unknown / no property:
## :...Age.in.years <= 56: BadLoan (5)
## :   Age.in.years > 56: GoodLoan (3/1)
## Property = if not A121 . building society savings agreement/life insurance:
## :...Number.of.existing.credits.at.this.bank > 1: GoodLoan (2)
## :   Number.of.existing.credits.at.this.bank <= 1: [S11]
## Property = if not A121/A122 . car or other, not in attribute 6:
## :...Credit.amount <= 1386:
##     :...Present.employment.since = .. >= 7 years: GoodLoan (2)
##     :   Present.employment.since in {... < 1 year,1 <= ... < 4 years,
##     :                                4 <= ... < 7 years,
##     :                                unemployed}: BadLoan (13)
##     Credit.amount > 1386:
##     :...Other.debtors.guarantors = auarantor: GoodLoan (0)
##         Other.debtors.guarantors = co-applicant: BadLoan (1)
##         Other.debtors.guarantors = none:
##         :...Age.in.years > 22: GoodLoan (21/3)
##             Age.in.years <= 22: [S12]
## 
## SubTree [S10]
## 
## Job in {management/ self=employed/hightly qualified employee/ officer,
## :       unskilled - resident}: BadLoan (3)
## Job in {skilled employee / official,
##         unemployed/ unskilled - non-resident}: GoodLoan (8/2)
## 
## SubTree [S11]
## 
## Status.of.existing.checking.account = 0 <= ... < 200 DM: BadLoan (9/2)
## Status.of.existing.checking.account = NULL: GoodLoan (18/7)
## 
## SubTree [S12]
## 
## Installment.rate.in.percentage.of.disposable.income <= 1: GoodLoan (2)
## Installment.rate.in.percentage.of.disposable.income > 1: BadLoan (2)
## 
## 
## Evaluation on training data (800 cases):
## 
##      Decision Tree   
##    ----------------  
##    Size      Errors  
## 
##      58   99(12.4%)   <<
## 
## 
##     (a)   (b)    <-classified as
##    ----  ----
##     158    82    (a): class BadLoan
##      17   543    (b): class GoodLoan
## 
## 
##  Attribute usage:
## 
##  100.00% Status.of.existing.checking.account
##   54.00% Duration.in.month
##   38.75% Savings.account.bonds
##   30.00% Credit.history
##   26.75% Other.debtors.guarantors
##   24.00% Present.residence.since
##   17.88% Property
##   16.25% Purpose
##    8.63% Other.installment.plans
##    7.13% Credit.amount
##    6.00% Age.in.years
##    5.63% Present.employment.since
##    3.75% Telephone
##    3.63% Number.of.existing.credits.at.this.bank
##    3.13% Housing
##    2.50% Job
##    0.50% Installment.rate.in.percentage.of.disposable.income
## 
## 
## Time: 0.0 secs
pred_credit <- predict(model_credit, train_credit)
mean(pred_credit==train_credit[, 21])
## [1] 0.87625
#6、预测
pred <- predict(model_credit, test_credit)

#7、查看预测数据
mean(pred==test_credit[, 21])
## [1] 0.745
table(pred, test_credit[, 21])
##           
## pred       BadLoan GoodLoan
##   BadLoan       26       17
##   GoodLoan      34      123
library(gmodels)
CrossTable(pred, test_credit[, 21])
## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  200 
## 
##  
##              | test_credit[, 21] 
##         pred |   BadLoan |  GoodLoan | Row Total | 
## -------------|-----------|-----------|-----------|
##      BadLoan |        26 |        17 |        43 | 
##              |    13.303 |     5.701 |           | 
##              |     0.605 |     0.395 |     0.215 | 
##              |     0.433 |     0.121 |           | 
##              |     0.130 |     0.085 |           | 
## -------------|-----------|-----------|-----------|
##     GoodLoan |        34 |       123 |       157 | 
##              |     3.644 |     1.562 |           | 
##              |     0.217 |     0.783 |     0.785 | 
##              |     0.567 |     0.879 |           | 
##              |     0.170 |     0.615 |           | 
## -------------|-----------|-----------|-----------|
## Column Total |        60 |       140 |       200 | 
##              |     0.300 |     0.700 |           | 
## -------------|-----------|-----------|-----------|
## 
## 
#从上面列联表可知：错误率为（34+17）/200=25.05%，同时有34/60=57%d bad loan我们归类为good loan。这结果显然不满足我们的要求。

#8、继续模型优化：C5.0中加入了adaptive boosting的支持
#1)使用迭代
model_boot <- C5.0(train_credit[, -21],  train_credit[, 21], trials = 10)
pred_boot <- predict(model_boot, test_credit[, -21])
table(pred_boot, test_credit[, 21])
##           
## pred_boot  BadLoan GoodLoan
##   BadLoan       28       19
##   GoodLoan      32      121
#从上可知：加入了boost后错误率没变，badloan误判为goodloan的概率为32/60=51%。然而我们知道如果把badloan 归类为goodloan的损失明显比把goodloan归类为badloan要大，因此这里加入一个cost矩阵，即为犯某类错误加上一个系数，比如如果我们按照cookbook里面的方法标记将badloan归类为goodloan的成本为5倍于goodloan归类为badloan。

#2)加入惩罚项
error_cost <- matrix(c(0, 1, 5, 0), nrow=2)
rownames(error_cost) <- c("GoodLoan", "BadLoan")
colnames(error_cost) <- c("GoodLoan", "BadLoan")
model_cost <- C5.0(train_credit[, -21], train_credit[, 21], costs=error_cost)
pred_cost <- predict(model_cost, test_credit)
table(pred_cost, test_credit[, 21])
##           
## pred_cost  BadLoan GoodLoan
##   BadLoan       51       75
##   GoodLoan       9       65

#从上可知：总错误率为84/200=42%，但是把badloan归类为goodloan的情况却大大减少了，只有9/60=15%。

#MarkDown脚本及数据集下载： http://pan.baidu.com/s/1dEx4vgX

**码上人生**
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
专栏目录
数据挖掘研究不同分类算法对不同数据集的分类效果评估
11-12
本报告将深入探讨三种常见的分类算法——支持向量机（SVM）、决策树和随机森林，并通过对比在不同数据集上的表现来评估它们的分类效果。这些算法在实际应用中有着广泛的应用，例如在西南交通大学的课程中，学生会...
Machine-Learning
03-21
2.4 决策树与随机森林 决策树通过一系列规则进行分类，随机森林是决策树的集成方法，能有效减少过拟合风险。 三、无监督学习 3.1 聚类 聚类是将数据分组，使得同一组内的数据相似度高，不同组间的相似度低。常用...
决策树算法及python实现
x_lingyun的专栏
11-08 889
本文转载自决策树算法及python实现 文中代码宜在Python2.x环境下执行 对于信息增益的计算公式宜确认 目录1 什么是决策树2 构建决策树2.1 构建决策树的基本步骤2.2 构建决策树例子2.2.1 使用ID3算法构建决策树2.2.1.1 信息增益的计算方法2.2.1.2 计算是否打球的经验熵2.2.1.3 计算outlook天气特征的信息增益2.2.1.4 计算temperature温度特征的信息增益2.2.1.5 计算humidity湿度特征的信息增益2.2.1.6 计算windy是否刮风的
机器学习方法——举例分析决策树
时生的博客
10-10 2977
在现实生活中，我们会遇到各种选择，不论是选择男女朋友，还是挑选水果，都是基于以往的经验来做判断。如果把判断背后的逻辑整理成一个结构图，你会发现它实际上是一个树状图，这就是我们今天要讲的决策树。决策树基本上就是把我们以前的经验总结出来。如果我们要出门打篮球，一般会根据“天气”、“温度”、“湿度”、“刮风”这几个条件来判断，最后得到结果：去打篮球？还是不去？
金融数据_Scikit-Learn决策树(DecisionTreeClassifier)实例
最新发布
goufeng93的博客
04-07 1039
金融数据 Scikit-Learn决策树(DecisionTreeClassifier)实例。
决策树模型及案例（Python）
热门推荐
qq_42433311的博客
04-14 8万+
目录 1 决策树模型简介 2 Gini系数（CART决策树） 3信息熵、信息增益 4决策树模型代码实现 4.1分类决策树模型(DecisionTreeClassifier) 4.2回归决策树模型(DecisionTreeRegressor) 5案例：员工离职预测模型 5.1模型搭建 5.1.1数据读取与预处理 5.1.2提取特征变量和目标变量 5.1.3划分训练集与测试集 5.1.4模型搭建及训练 5.1.5代码汇总 5.2模型预测及评估 5.2.1直...
14天数据分析与机器学习实践之Day11——使用sklearn构造决策树模型
qq_45817449的博客
07-25 465
14天数据分析与机器学习实践之Day11——使用sklearn构造决策树模型 sklearn官网 树模型参数: criterion gini or entropy splitter best or random 前者是在所有特征中找最好的切分点 后者是在部分特征中（数据量大的时候） max_features None（所有），log2，sqrt，N 特征小于50的时候一般使用所有的 max_depth 数据少或者特征少的时候可以不管这个值，如果模型样本量多，特征也多的情况下，可以尝试限制下 min_sam
参数调优：K折交叉验证与GridSearch网格搜索
m0_46388544的博客
01-27 2341
本文代码及数据集来自《Python大数据分析与机器学习商业案例实战》 一、K折交叉验证 在机器学习中，因为训练集和测试集的数据划分是随机的，所以有时会重复地使用数据，以便更好地评估模型的有效性，并选出最好的模型，该做法称为交叉验证。具体而言就是对原始样本数据进行切分，然后组合成为多组不同的训练集和测试集，用训练集训练模型，用测试集评估模型。某次的训练集可能是下次的测试集，故而称为交叉验证。 交叉验证的方法有简单交叉验证、K折交叉验证和留一交叉验证3种。其中K折交叉验证应用较为广泛，它是指将数据集随机等分.
K 近邻法（K-Nearest Neighbor, K-NN）
12-22
k近邻模型2.1 模型2.2 距离度量2.2.1 距离计算代码 Python2.3 kkk 值的选择2.4 分类决策规则3. 实现方法, kd树3.1 构造 kdkdkd 树3.2 搜索 kdkdkd 树4. 鸢尾花KNN分类4.1 KNN实现4.2 sklearn KNN k近邻法（k-...
DeepLearning的机器学习基础
12-06
2.4 多实例例学习? 2.5 分类⽹网络和回归的区别? 2.6 什什么是神经⽹网络? 2.7 理理解局部最优与全局最优 2.8 分类算法 2.8.1 常⽤用分类算法的优缺点? 2.8.2 正确率能很好的评估分类算法吗? 2.8.3 分类算法的评估⽅...
地理信息系统算法基础.rar
11-30
13．2．4二叉决策树算法与分类规则的生成 13．2．5决策树分类算法 13．2．6决策树属性的选取 13．2．7改进决策树性能的方法 13．3泛化规则算法 13．3．1概念层次 13．3．2面向属性泛化的策略与特点 13．3．3基于规则...
决策树的典型案例
kxcc_sx的专栏
10-27 5万+
   小王是一家著名高尔夫俱乐部的经理。但是他被雇员数量问题搞得心情十分不好。某些天好像所有人都來玩高尔夫，以至于所有员工都忙的团团转还是应付不过来，而有些天不知道什么原因却一个人也不来，俱乐部为雇员数量浪费了不少资金。小王的目的是通过下周天气预报寻找什么时候人们会打高尔夫，以适时调整雇员数量。因此首先他必须了解人们决定是否打球的原因。在2周时间内我们得到以下记录:天气状况有晴，云和雨；气温用华氏温度表示；相对湿度用百分比；还有有无风。当然还有顾客是不是在这些日子光顾俱乐部。最终他得到了14列5行的数据表格
【机器学习算法】决策树案例详解
莫凡的博客
12-25 4737
初识决策树 决策树是一个类似于人们决策过程的树结构，从根节点开始，每个分枝代表一个新的决策事件，会生成两个或多个分枝，每个叶子代表一个最终判定所属的类别。 例如，如下是一个决策树，代表薪水大于30W的男性会买车。 我们可以很容易的写出IF Else来实现决策树的判定。上述的决策树有两个特征区间，性别和年龄，最终的结果有两个类别，买和不买。 信息增益 信息熵表示的是不确定度。均匀分布时，...
使用Python训练好的决策树模型生成C++代码
zds的博客
04-01 4426
在实际工程中，有时候我们使用python进行模型训练，但是训练好的模型需要写在芯片中，这个时候怎么弄呢？笔者在网上找了一下，没有发现可以将训练好的模型直接转化为我们需要的语言，比如C++。因此笔者自己动手写.
数据挖掘实例
wen008215的专栏
04-07 799
引用:http://old.blog.edu.cn/user1/9065/archives/2005/145093.shtml目的：    给定某些属性，判断某贷款顾客的可信性（即”good/bad”状况）。简单思路：      该数据包含了666条贷款顾客的历史数据和21个属性。但是我们认为，这21个属性不都能够有效地帮助我们判断顾客的可行性，所以我们首先把一些比较不相关的属性去
机器学习——决策树(2020最新版)
沉浮于世的博客
08-10 3532
决策树是一种机器学习的方法。决策树的生成算法有ID3, C4.5和C5.0等。决策树是一种树形结构，其中每个内部节点表示一个属性上的判断，每个分支代表一个判断结果的输出，最后每个叶节点代表一种分类结果。
决策树详细介绍（含sklearn演示代码）
MrNeo的博客
02-06 1498
顾名思义和现实中的树类似，一般可以这样表示一般由三个部分组成根节点，如上图的1分支节点，如上图24叶子节点，如上图56378树是数据结构里的一个知识点，这里不过多赘述，更多内容请自行搜索。
建模杂谈系列63- knime模型实例2- 信用评分 Credit Score
yukai08008的博客
05-04 959
说明 使用german credit数据做信用评分卡，这是一个关于多模型选择的过程。 内容 1 读取数据 df = pd.read_csv('german-credit-scoring.csv',sep=';') 2 将分类变量转为数值 除了score之外都转为了数值 char_cols = [] for col in df.columns: if isinstance(df[col].iloc[0],str): char_cols.append(col) char_
German Credit Risk(德国信用卡违约分析)
Yaphat的博客
03-29 1万+
数据信息先看下数据格式： 总共有20个属性，1个类别特征。信息如下：Attribute 1: (qualitative) Status of existing checking account A11 : … < 0 DM A12 : 0 <= … < 200 DM A13 : … >= 200 DM / salary assignments for at least 1 year
决策树汇报专用 手动推演 代码大全
10-18
决策树是一种强大的机器学习工具，特别适用于数据可视化和解释性，常用于预测和分类任务。在进行决策树汇报时，手动推演和代码实现是关键环节，有助于加深理解和提高演示效果。本文档汇集了关于决策树算法的各种知识...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值