Hadoop，Hbase测试工具介绍

Hadoop，Hbase测试工具介绍

Hadoop自带的测试工具

hadoop-mapreduce-examples-x.x.x.jar

hadoop-mapreduce-client-jobclient-x.x.-tests.jar

pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.
说明：蒙特卡罗算法计算Pi
执行：./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar pi 10 100000
结果： Estimated value of Pi is 3.14155200000000000000

wordcount: A map/reduce program that counts the words in the input files.
说明： 统计单词数量
执行：hadoop fs –put wordsin.txt /
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /wordsin.txt /wordsout
结果：hadoop fs –cat “/wordsout/*”

teragen: Generate data for the terasort
terasort: Run the terasort
teravalidate: Checking results of terasort

说明：先执行teragen生成测试数据，然后执行terasort进行sort排序数据，完成后使用teravalidate检查terasort输出的结果。
执行：
1. teragen
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar teragen -Dmapred.map.tasks=4 10 teragendir
2. terasort
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar terasort
-Dmapred.reduce.tasks=2 teragendir terasortout
3. teravalidate
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar teravalidate terasortout teravadir
结果：可通过执行打印结果查看，可以通过输出数据结果查看

Hadoop自带测试工具-test

hadoop-mapreduce-client-jobclient-2.6.0-tests.jar
TestDFSIO：Distributed i/o benchmark
TestDFSIO [genericOptions] -read [-random | -backward | -skip [-skipSize Size] ] | -write| -append | -clean [-compression codecClassName] [-nrFiles N] [-size Size[B|KB|MB|GB|TB]] [-resFile resultFileName] [-bufferSize Bytes] [-rootDir]
说明：分布式i/o测试工具，主要测试文件系统读写
执行：1.写文件
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar TestDFSIO -write -nrFiles 10 -size 50MB
2.读文件
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar TestDFSIO -read -nrFiles 10 -size 50MB
3.清除数据（hadoop fs -ls /benchmarks 来查看数据）
./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar TestDFSIO -clean

HBase自带的测试工具
Pe,Ltt

PerformanceEvaluation
Usage: java org.apache.hadoop.hbase.PerformanceEvaluation
[-D<property=value>]*
执行：./bin/hbase pe --nomapred --rows=10 --columns=2 --table=test2017 sequentialWrite 3
解释：3个clients执行，不使用mapreduce框架，顺序写10行数据，每行2个列族，table的名字为test2017 ；最终生成30行数据。
nomapred Run multiple clients using threads (rather than use mapreduce)
rows Rows each client runs. Default: One million
columns Columns to write per row. Default: 1
table Alternate table name. Default: ‘TestTable’

LoadTestTool
-write
<avg_cols_per_key>:<avg_data_size>[:<#threads=20>]
例如：
./bin/hbase ltt -write 10:100 -num_keys 2
查看cluster_test表
-read
<verify_percent>[:<#threads=20>]
例如：
./bin/hbase ltt -read 90 -num_keys 2 -num_tables 2