hadoop性能测试

一、hadoop自带的性能基准评测工具

（一）TestDFSIO

1、测试写性能
（1）若有必要，先删除历史数据
$hadoop jar /home/hadoop/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.2-tests.jar TestDFSIO -clean
（2）执行测试
$hadoop jar /home/hadoop/hadoop/share/hadoop/mapreduce2/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.2-tests.jar TestDFSIO -write -nrFiles 5 -fileSize 20
（3）查看结果：每一次测试生成一个结果，并以附加的形式添加到TestDFSIO_results.log中
$cat TestDFSIO_results.log
----- TestDFSIO ----- : write
           Date & time: Mon May 11 09:41:34 HKT 2015
       Number of files:
Total MBytes processed: 100.0
     Throughput mb/sec: 21.468441391155004
Average IO rate mb/sec: 25.366744995117188
 IO rate std deviation: 12.744636924030177
    Test exec time sec: 27.585

----- TestDFSIO ----- : write
           Date & time: Mon May 11 09:42:28 HKT 2015
       Number of files: 5
Total MBytes processed: 100.0
     Throughput mb/sec: 22.779043280182233
Average IO rate mb/sec: 25.440486907958984
 IO rate std deviation: 9.930490103638768
    Test exec time sec: 26.67

（4）结果说明