5.5 COMPARING DATA MINING SCHEMES

       We often need to compare two different learning schemes on the same problem to see which is the better one to use. It seems simple: Estimate the error using cross-validation (or any other suitable estimation procedure), perhaps repeated several times, and choose the scheme with the smaller estimate.This is quite sufficient in many practical applications: If one scheme has a lower estimated error than another on a particular dataset, the best we can do is to use the former scheme’s model. However, it may be that the difference is simply due to estimation error, and in some
circumstances it is important to determine whether one scheme is really better than another on a particular problem.

       假设由数据产生了两个分类模型。并且进行了10折交叉验证，得到了每个的平均错误率。如何确定哪个模型更好？直观的看，可以选择具有最低错误率的那个模型。尽管由两个模型得到的平均错误率看上去可能不同，但是差别可能不是统计显著的。如果两者之间的差别可能只是偶然的，怎么办？（本节没有读懂）